org.apache.iceberg.actions

Interface RewritePositionDeleteFiles

All Superinterfaces:

Action<RewritePositionDeleteFiles,RewritePositionDeleteFiles.Result>, SnapshotUpdate<RewritePositionDeleteFiles,RewritePositionDeleteFiles.Result>

All Known Implementing Classes:

RewritePositionDeleteFilesSparkAction

@Value.Enclosing
public interface RewritePositionDeleteFiles
extends SnapshotUpdate<RewritePositionDeleteFiles,RewritePositionDeleteFiles.Result>

An action for rewriting position delete files.

Generally used for optimizing the size and layout of position delete files within a table.

Nested Class Summary

Nested Classes

Modifier and Type	Interface and Description
static interface	RewritePositionDeleteFiles.FileGroupInfo A description of a position delete file group, when it was processed, and within which partition.
static interface	RewritePositionDeleteFiles.FileGroupRewriteResult For a particular position delete file group, the number of position delete files which are newly created and the number of files which were formerly part of the table but have been rewritten.
static interface	RewritePositionDeleteFiles.Result The action result that contains a summary of the execution.

Field Summary

Fields

Modifier and Type	Field and Description
static java.lang.String	MAX_CONCURRENT_FILE_GROUP_REWRITES The max number of file groups to be simultaneously rewritten by the rewrite strategy.
static int	MAX_CONCURRENT_FILE_GROUP_REWRITES_DEFAULT
static java.lang.String	PARTIAL_PROGRESS_ENABLED Enable committing groups of files (see max-file-group-size-bytes) prior to the entire rewrite completing.
static boolean	PARTIAL_PROGRESS_ENABLED_DEFAULT
static java.lang.String	PARTIAL_PROGRESS_MAX_COMMITS The maximum amount of Iceberg commits that this rewrite is allowed to produce if partial progress is enabled.
static int	PARTIAL_PROGRESS_MAX_COMMITS_DEFAULT
static java.lang.String	REWRITE_JOB_ORDER Forces the rewrite job order based on the value.
static java.lang.String	REWRITE_JOB_ORDER_DEFAULT

Method Summary

Modifier and Type	Method and Description
RewritePositionDeleteFiles	filter(Expression expression) A filter for finding deletes to rewrite.

Methods inherited from interface org.apache.iceberg.actions.SnapshotUpdate

snapshotProperty

Methods inherited from interface org.apache.iceberg.actions.Action

execute, option, options

Field Detail

PARTIAL_PROGRESS_ENABLED

static final java.lang.String PARTIAL_PROGRESS_ENABLED

Enable committing groups of files (see max-file-group-size-bytes) prior to the entire rewrite completing. This will produce additional commits but allow for progress even if some groups fail to commit. This setting will not change the correctness of the rewrite operation as file groups can be compacted independently.

The default is false, which produces a single commit when the entire job has completed.

See Also:

Constant Field Values

PARTIAL_PROGRESS_ENABLED_DEFAULT

static final boolean PARTIAL_PROGRESS_ENABLED_DEFAULT

See Also:

Constant Field Values

PARTIAL_PROGRESS_MAX_COMMITS

static final java.lang.String PARTIAL_PROGRESS_MAX_COMMITS

The maximum amount of Iceberg commits that this rewrite is allowed to produce if partial progress is enabled. This setting has no effect if partial progress is disabled.

See Also:

Constant Field Values

PARTIAL_PROGRESS_MAX_COMMITS_DEFAULT

static final int PARTIAL_PROGRESS_MAX_COMMITS_DEFAULT

See Also:

Constant Field Values

MAX_CONCURRENT_FILE_GROUP_REWRITES

static final java.lang.String MAX_CONCURRENT_FILE_GROUP_REWRITES

The max number of file groups to be simultaneously rewritten by the rewrite strategy. The structure and contents of the group is determined by the rewrite strategy. Each file group will be rewritten independently and asynchronously.

See Also:

Constant Field Values

MAX_CONCURRENT_FILE_GROUP_REWRITES_DEFAULT

static final int MAX_CONCURRENT_FILE_GROUP_REWRITES_DEFAULT

See Also:

Constant Field Values

REWRITE_JOB_ORDER

static final java.lang.String REWRITE_JOB_ORDER

Forces the rewrite job order based on the value.

• If rewrite-job-order=bytes-asc, then rewrite the smallest job groups first.
• If rewrite-job-order=bytes-desc, then rewrite the largest job groups first.
• If rewrite-job-order=files-asc, then rewrite the job groups with the least files first.
• If rewrite-job-order=files-desc, then rewrite the job groups with the most files first.
• If rewrite-job-order=none, then rewrite job groups in the order they were planned (no specific ordering).

Defaults to none.

See Also:

Constant Field Values

REWRITE_JOB_ORDER_DEFAULT

static final java.lang.String REWRITE_JOB_ORDER_DEFAULT

Method Detail

filter

RewritePositionDeleteFiles filter(Expression expression)

A filter for finding deletes to rewrite.

The filter will be converted to a partition filter with an inclusive projection. Any file that may contain rows matching this filter will be used by the action. The matching delete files will be rewritten.

Parameters:

expression - An iceberg expression used to find deletes.

Returns:

this for method chaining