org.apache.iceberg.mr.mapreduce

Class IcebergInputFormat<T>

java.lang.Object

org.apache.hadoop.mapreduce.InputFormat<java.lang.Void,T>

org.apache.iceberg.mr.mapreduce.IcebergInputFormat<T>

Type Parameters:

T - T is the in memory data model which can either be Pig tuples, Hive rows. Default is Iceberg records

public class IcebergInputFormat<T>
extends org.apache.hadoop.mapreduce.InputFormat<java.lang.Void,T>

Generic Mrv2 InputFormat API for Iceberg.

Constructor Summary

Constructors

Constructor and Description
IcebergInputFormat()

Method Summary

Modifier and Type	Method and Description
static InputFormatConfig.ConfigBuilder	configure(org.apache.hadoop.mapreduce.Job job) Configures the Job to use the IcebergInputFormat and returns a helper to add further configuration.
org.apache.hadoop.mapreduce.RecordReader<java.lang.Void,T>	createRecordReader(org.apache.hadoop.mapreduce.InputSplit split, org.apache.hadoop.mapreduce.TaskAttemptContext context)
java.util.List<org.apache.hadoop.mapreduce.InputSplit>	getSplits(org.apache.hadoop.mapreduce.JobContext context)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

IcebergInputFormat

public IcebergInputFormat()

Method Detail

configure

public static InputFormatConfig.ConfigBuilder configure(org.apache.hadoop.mapreduce.Job job)

Configures the Job to use the IcebergInputFormat and returns a helper to add further configuration.

Parameters:

job - the Job to configure

getSplits

public java.util.List<org.apache.hadoop.mapreduce.InputSplit> getSplits(org.apache.hadoop.mapreduce.JobContext context)

Specified by:

getSplits in class org.apache.hadoop.mapreduce.InputFormat<java.lang.Void,T>

createRecordReader

public org.apache.hadoop.mapreduce.RecordReader<java.lang.Void,T> createRecordReader(org.apache.hadoop.mapreduce.InputSplit split,
                                                                                     org.apache.hadoop.mapreduce.TaskAttemptContext context)

Specified by:

createRecordReader in class org.apache.hadoop.mapreduce.InputFormat<java.lang.Void,T>