public class HadoopTableOperations extends java.lang.Object implements TableOperations
This maintains metadata in a "metadata" folder under the table location.
Modifier | Constructor and Description |
---|---|
protected |
HadoopTableOperations(org.apache.hadoop.fs.Path location,
FileIO fileIO,
org.apache.hadoop.conf.Configuration conf,
LockManager lockManager) |
Modifier and Type | Method and Description |
---|---|
void |
commit(TableMetadata base,
TableMetadata metadata)
Replace the base table metadata with a new version.
|
TableMetadata |
current()
Return the currently loaded table metadata, without checking for updates.
|
protected org.apache.hadoop.fs.FileSystem |
getFileSystem(org.apache.hadoop.fs.Path path,
org.apache.hadoop.conf.Configuration hadoopConf) |
FileIO |
io()
Returns a
FileIO to read and write table data and metadata files. |
LocationProvider |
locationProvider()
Returns a
LocationProvider that supplies locations for new new data files. |
java.lang.String |
metadataFileLocation(java.lang.String fileName)
Given the name of a metadata file, obtain the full path of that file using an appropriate base
location of the implementation's choosing.
|
TableMetadata |
refresh()
Return the current table metadata after checking for updates.
|
TableOperations |
temp(TableMetadata uncommittedMetadata)
Return a temporary
TableOperations instance that uses configuration from uncommitted
metadata. |
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
encryption, newSnapshotId
protected HadoopTableOperations(org.apache.hadoop.fs.Path location, FileIO fileIO, org.apache.hadoop.conf.Configuration conf, LockManager lockManager)
public TableMetadata current()
TableOperations
current
in interface TableOperations
public TableMetadata refresh()
TableOperations
refresh
in interface TableOperations
public void commit(TableMetadata base, TableMetadata metadata)
TableOperations
This method should implement and document atomicity guarantees.
Implementations must check that the base metadata is current to avoid overwriting updates. Once the atomic commit operation succeeds, implementations must not perform any operations that may fail because failure in this method cannot be distinguished from commit failure.
Implementations must throw a CommitStateUnknownException
in cases where it cannot be
determined if the commit succeeded or failed. For example if a network partition causes the
confirmation of the commit to be lost, the implementation should throw a
CommitStateUnknownException. This is important because downstream users of this API need to
know whether they can clean up the commit or not, if the state is unknown then it is not safe
to remove any files. All other exceptions will be treated as if the commit has failed.
commit
in interface TableOperations
base
- table metadata on which changes were basedmetadata
- new table metadata with updatespublic FileIO io()
TableOperations
FileIO
to read and write table data and metadata files.io
in interface TableOperations
public LocationProvider locationProvider()
TableOperations
LocationProvider
that supplies locations for new new data files.locationProvider
in interface TableOperations
public java.lang.String metadataFileLocation(java.lang.String fileName)
TableOperations
The file may not exist yet, in which case the path should be returned as if it were to be
created by e.g. FileIO.newOutputFile(String)
.
metadataFileLocation
in interface TableOperations
public TableOperations temp(TableMetadata uncommittedMetadata)
TableOperations
TableOperations
instance that uses configuration from uncommitted
metadata.
This is called by transactions when uncommitted table metadata should be used; for example, to create a metadata file location based on metadata in the transaction that has not been committed.
Transactions will not call TableOperations.refresh()
or TableOperations.commit(TableMetadata,
TableMetadata)
.
temp
in interface TableOperations
uncommittedMetadata
- uncommitted table metadataprotected org.apache.hadoop.fs.FileSystem getFileSystem(org.apache.hadoop.fs.Path path, org.apache.hadoop.conf.Configuration hadoopConf)