public class ManifestFiles
extends java.lang.Object
Modifier and Type | Method and Description |
---|---|
static ManifestFile |
decode(byte[] manifestData)
Decode the binary data into a
ManifestFile . |
static void |
dropCache(FileIO fileIO)
Drop manifest file cache object for a FileIO if exists.
|
static byte[] |
encode(ManifestFile manifestFile)
Encode the
ManifestFile to a byte array by using avro encoder. |
static ManifestReader<DataFile> |
read(ManifestFile manifest,
FileIO io)
Returns a new
ManifestReader for a ManifestFile . |
static ManifestReader<DataFile> |
read(ManifestFile manifest,
FileIO io,
java.util.Map<java.lang.Integer,PartitionSpec> specsById)
Returns a new
ManifestReader for a ManifestFile . |
static ManifestReader<DeleteFile> |
readDeleteManifest(ManifestFile manifest,
FileIO io,
java.util.Map<java.lang.Integer,PartitionSpec> specsById)
Returns a new
ManifestReader for a ManifestFile . |
static CloseableIterable<java.lang.String> |
readPaths(ManifestFile manifest,
FileIO io)
Returns a
CloseableIterable of file paths in the ManifestFile . |
static ManifestWriter<DataFile> |
write(int formatVersion,
PartitionSpec spec,
OutputFile outputFile,
java.lang.Long snapshotId)
Create a new
ManifestWriter for the given format version. |
static ManifestWriter<DataFile> |
write(PartitionSpec spec,
OutputFile outputFile)
Create a new
ManifestWriter . |
static ManifestWriter<DeleteFile> |
writeDeleteManifest(int formatVersion,
PartitionSpec spec,
OutputFile outputFile,
java.lang.Long snapshotId)
Create a new
ManifestWriter for the given format version. |
public static void dropCache(FileIO fileIO)
public static CloseableIterable<java.lang.String> readPaths(ManifestFile manifest, FileIO io)
CloseableIterable
of file paths in the ManifestFile
.manifest
- a ManifestFileio
- a FileIOpublic static ManifestReader<DataFile> read(ManifestFile manifest, FileIO io)
ManifestReader
for a ManifestFile
.
Note: Callers should use read(ManifestFile, FileIO, Map)
to
ensure the schema used by filters is the latest table schema. This should be used only when
reading a manifest without filters.
manifest
- a ManifestFileio
- a FileIOpublic static ManifestReader<DataFile> read(ManifestFile manifest, FileIO io, java.util.Map<java.lang.Integer,PartitionSpec> specsById)
ManifestReader
for a ManifestFile
.manifest
- a ManifestFile
io
- a FileIO
specsById
- a Map from spec ID to partition specManifestReader
public static ManifestWriter<DataFile> write(PartitionSpec spec, OutputFile outputFile)
ManifestWriter
.
Manifests created by this writer have all entry snapshot IDs set to null. All entries will inherit the snapshot ID that will be assigned to the manifest on commit.
spec
- PartitionSpec
used to produce DataFile
partition tuplesoutputFile
- the destination file locationpublic static ManifestWriter<DataFile> write(int formatVersion, PartitionSpec spec, OutputFile outputFile, java.lang.Long snapshotId)
ManifestWriter
for the given format version.formatVersion
- a target format versionspec
- a PartitionSpec
outputFile
- an OutputFile
where the manifest will be writtensnapshotId
- a snapshot ID for the manifest entries, or null for an inherited IDpublic static ManifestReader<DeleteFile> readDeleteManifest(ManifestFile manifest, FileIO io, java.util.Map<java.lang.Integer,PartitionSpec> specsById)
ManifestReader
for a ManifestFile
.manifest
- a ManifestFile
io
- a FileIO
specsById
- a Map from spec ID to partition specManifestReader
public static ManifestWriter<DeleteFile> writeDeleteManifest(int formatVersion, PartitionSpec spec, OutputFile outputFile, java.lang.Long snapshotId)
ManifestWriter
for the given format version.formatVersion
- a target format versionspec
- a PartitionSpec
outputFile
- an OutputFile
where the manifest will be writtensnapshotId
- a snapshot ID for the manifest entries, or null for an inherited IDpublic static byte[] encode(ManifestFile manifestFile) throws java.io.IOException
ManifestFile
to a byte array by using avro encoder.manifestFile
- a ManifestFile
, which should always be a GenericManifestFile
.java.io.IOException
- if encounter any IO error when encoding.public static ManifestFile decode(byte[] manifestData) throws java.io.IOException
ManifestFile
.manifestData
- the binary data.ManifestFile
. To be precise, it's a GenericManifestFile
which don't
expose to public.java.io.IOException
- if encounter any IO error when decoding.