org.apache.iceberg.hadoop

Class HadoopInputFile

java.lang.Object

org.apache.iceberg.hadoop.HadoopInputFile

All Implemented Interfaces:

NativelyEncryptedFile, InputFile

public class HadoopInputFile
extends java.lang.Object
implements InputFile, NativelyEncryptedFile

InputFile implementation using the Hadoop FileSystem API.

This class is based on Parquet's HadoopInputFile.

Field Summary

Fields

Modifier and Type	Field and Description
static java.lang.String[]	NO_LOCATION_PREFERENCE

Method Summary

Modifier and Type	Method and Description
boolean	exists() Checks whether the file exists.
static HadoopInputFile	fromLocation(java.lang.CharSequence location, org.apache.hadoop.conf.Configuration conf)
static HadoopInputFile	fromLocation(java.lang.CharSequence location, org.apache.hadoop.fs.FileSystem fs)
static HadoopInputFile	fromLocation(java.lang.CharSequence location, long length, org.apache.hadoop.conf.Configuration conf)
static HadoopInputFile	fromLocation(java.lang.CharSequence location, long length, org.apache.hadoop.fs.FileSystem fs)
static HadoopInputFile	fromPath(org.apache.hadoop.fs.Path path, org.apache.hadoop.conf.Configuration conf)
static HadoopInputFile	fromPath(org.apache.hadoop.fs.Path path, org.apache.hadoop.fs.FileSystem fs)
static HadoopInputFile	fromPath(org.apache.hadoop.fs.Path path, org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.conf.Configuration conf)
static HadoopInputFile	fromPath(org.apache.hadoop.fs.Path path, long length, org.apache.hadoop.conf.Configuration conf)
static HadoopInputFile	fromPath(org.apache.hadoop.fs.Path path, long length, org.apache.hadoop.fs.FileSystem fs)
static HadoopInputFile	fromPath(org.apache.hadoop.fs.Path path, long length, org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.conf.Configuration conf)
static HadoopInputFile	fromStatus(org.apache.hadoop.fs.FileStatus stat, org.apache.hadoop.conf.Configuration conf)
static HadoopInputFile	fromStatus(org.apache.hadoop.fs.FileStatus stat, org.apache.hadoop.fs.FileSystem fs)
static HadoopInputFile	fromStatus(org.apache.hadoop.fs.FileStatus stat, org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.conf.Configuration conf)
java.lang.String[]	getBlockLocations(long start, long end)
org.apache.hadoop.conf.Configuration	getConf()
org.apache.hadoop.fs.FileSystem	getFileSystem()
long	getLength() Returns the total length of the file, in bytes
org.apache.hadoop.fs.Path	getPath()
org.apache.hadoop.fs.FileStatus	getStat()
java.lang.String	location() The fully-qualified location of the input file as a String.
NativeFileCryptoParameters	nativeCryptoParameters()
SeekableInputStream	newStream() Opens a new SeekableInputStream for the underlying data file
void	setNativeCryptoParameters(NativeFileCryptoParameters nativeCryptoParameters)
java.lang.String	toString()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

NO_LOCATION_PREFERENCE

public static final java.lang.String[] NO_LOCATION_PREFERENCE

Method Detail

fromLocation

public static HadoopInputFile fromLocation(java.lang.CharSequence location,
                                           org.apache.hadoop.conf.Configuration conf)

fromLocation

public static HadoopInputFile fromLocation(java.lang.CharSequence location,
                                           long length,
                                           org.apache.hadoop.conf.Configuration conf)

fromLocation

public static HadoopInputFile fromLocation(java.lang.CharSequence location,
                                           org.apache.hadoop.fs.FileSystem fs)

fromLocation

public static HadoopInputFile fromLocation(java.lang.CharSequence location,
                                           long length,
                                           org.apache.hadoop.fs.FileSystem fs)

fromPath

public static HadoopInputFile fromPath(org.apache.hadoop.fs.Path path,
                                       org.apache.hadoop.conf.Configuration conf)

fromPath

public static HadoopInputFile fromPath(org.apache.hadoop.fs.Path path,
                                       long length,
                                       org.apache.hadoop.conf.Configuration conf)

fromPath

public static HadoopInputFile fromPath(org.apache.hadoop.fs.Path path,
                                       org.apache.hadoop.fs.FileSystem fs)

fromPath

public static HadoopInputFile fromPath(org.apache.hadoop.fs.Path path,
                                       long length,
                                       org.apache.hadoop.fs.FileSystem fs)

fromPath

public static HadoopInputFile fromPath(org.apache.hadoop.fs.Path path,
                                       org.apache.hadoop.fs.FileSystem fs,
                                       org.apache.hadoop.conf.Configuration conf)

fromPath

public static HadoopInputFile fromPath(org.apache.hadoop.fs.Path path,
                                       long length,
                                       org.apache.hadoop.fs.FileSystem fs,
                                       org.apache.hadoop.conf.Configuration conf)

fromStatus

public static HadoopInputFile fromStatus(org.apache.hadoop.fs.FileStatus stat,
                                         org.apache.hadoop.conf.Configuration conf)

fromStatus

public static HadoopInputFile fromStatus(org.apache.hadoop.fs.FileStatus stat,
                                         org.apache.hadoop.fs.FileSystem fs)

fromStatus

public static HadoopInputFile fromStatus(org.apache.hadoop.fs.FileStatus stat,
                                         org.apache.hadoop.fs.FileSystem fs,
                                         org.apache.hadoop.conf.Configuration conf)

getLength

public long getLength()

Description copied from interface: InputFile

Returns the total length of the file, in bytes

Specified by:

getLength in interface InputFile

Returns:

the total length of the file, in bytes

newStream

public SeekableInputStream newStream()

Description copied from interface: InputFile

Opens a new SeekableInputStream for the underlying data file

Specified by:

newStream in interface InputFile

Returns:

a seekable stream for reading the file

getConf

public org.apache.hadoop.conf.Configuration getConf()

getFileSystem

public org.apache.hadoop.fs.FileSystem getFileSystem()

getStat

public org.apache.hadoop.fs.FileStatus getStat()

getPath

public org.apache.hadoop.fs.Path getPath()

getBlockLocations

public java.lang.String[] getBlockLocations(long start,
                                            long end)

location

public java.lang.String location()

Description copied from interface: InputFile

The fully-qualified location of the input file as a String.

Specified by:

location in interface InputFile

Returns:

the input file location

exists

public boolean exists()

Description copied from interface: InputFile

Checks whether the file exists.

Specified by:

exists in interface InputFile

Returns:

true if the file exists, false otherwise

nativeCryptoParameters

public NativeFileCryptoParameters nativeCryptoParameters()

Specified by:

nativeCryptoParameters in interface NativelyEncryptedFile

setNativeCryptoParameters

public void setNativeCryptoParameters(NativeFileCryptoParameters nativeCryptoParameters)

Specified by:

setNativeCryptoParameters in interface NativelyEncryptedFile

toString

public java.lang.String toString()

Overrides:

toString in class java.lang.Object