org.apache.iceberg.io

Class ClusteredDataWriter<T>

java.lang.Object

org.apache.iceberg.io.ClusteredDataWriter<T>

All Implemented Interfaces:

java.io.Closeable, java.lang.AutoCloseable, PartitioningWriter<T,DataWriteResult>

public class ClusteredDataWriter<T>
extends java.lang.Object

A data writer capable of writing to multiple specs and partitions that requires the incoming records to be properly clustered by partition spec and by partition within each spec.

Constructor Summary

Constructors

Constructor and Description
ClusteredDataWriter(FileWriterFactory<T> writerFactory, OutputFileFactory fileFactory, FileIO io, long targetFileSizeInBytes)

Method Summary

Modifier and Type	Method and Description
protected void	addResult(DataWriteResult result)
protected DataWriteResult	aggregatedResult()
void	close()
protected EncryptedOutputFile	newOutputFile(OutputFileFactory fileFactory, PartitionSpec spec, StructLike partition)
protected FileWriter<T,DataWriteResult>	newWriter(PartitionSpec spec, StructLike partition)
R	result() Returns a result that contains information about written DataFiles or DeleteFiles.
void	write(T row, PartitionSpec spec, StructLike partition) Writes a row to the provided spec/partition.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ClusteredDataWriter

public ClusteredDataWriter(FileWriterFactory<T> writerFactory,
                           OutputFileFactory fileFactory,
                           FileIO io,
                           long targetFileSizeInBytes)

Method Detail

newWriter

protected FileWriter<T,DataWriteResult> newWriter(PartitionSpec spec,
                                                  StructLike partition)

addResult

protected void addResult(DataWriteResult result)

aggregatedResult

protected DataWriteResult aggregatedResult()

write

public void write(T row,
                  PartitionSpec spec,
                  StructLike partition)

Description copied from interface: PartitioningWriter

Writes a row to the provided spec/partition.

Specified by:

write in interface PartitioningWriter<T,R>

Parameters:

row - a data or delete record

spec - a partition spec

partition - a partition or null if the spec is unpartitioned

close

public void close()
           throws java.io.IOException

Specified by:

close in interface java.io.Closeable

Specified by:

close in interface java.lang.AutoCloseable

Throws:

java.io.IOException

result

public final R result()

Description copied from interface: PartitioningWriter

Returns a result that contains information about written DataFiles or DeleteFiles. The result is valid only after the writer is closed.

Specified by:

result in interface PartitioningWriter<T,R>

Returns:

the writer result

newOutputFile

protected EncryptedOutputFile newOutputFile(OutputFileFactory fileFactory,
                                            PartitionSpec spec,
                                            StructLike partition)