org.apache.iceberg

Interface ExpireSnapshots

All Superinterfaces:

PendingUpdate<java.util.List<Snapshot>>

public interface ExpireSnapshots
extends PendingUpdate<java.util.List<Snapshot>>

API for removing old snapshots from a table.

This API accumulates snapshot deletions and commits the new list to the table. This API does not allow deleting the current snapshot.

When committing, these changes will be applied to the latest table metadata. Commit conflicts will be resolved by applying the changes to the new latest metadata and reattempting the commit.

Manifest files that are no longer used by valid snapshots will be deleted. Data files that were deleted by snapshots that are expired will be deleted. deleteWith(Consumer) can be used to pass an alternative deletion method.

PendingUpdate.apply() returns a list of the snapshots that will be removed.

Method Summary

Modifier and Type	Method and Description
ExpireSnapshots	cleanExpiredFiles(boolean clean) Allows expiration of snapshots without any cleanup of underlying manifest or data files.
ExpireSnapshots	deleteWith(java.util.function.Consumer<java.lang.String> deleteFunc) Passes an alternative delete implementation that will be used for manifests and data files.
ExpireSnapshots	executeDeleteWith(java.util.concurrent.ExecutorService executorService) Passes an alternative executor service that will be used for manifests and data files deletion.
ExpireSnapshots	expireOlderThan(long timestampMillis) Expires all snapshots older than the given timestamp.
ExpireSnapshots	expireSnapshotId(long snapshotId) Expires a specific Snapshot identified by id.
ExpireSnapshots	planWith(java.util.concurrent.ExecutorService executorService) Passes an alternative executor service that will be used for planning.
ExpireSnapshots	retainLast(int numSnapshots) Retains the most recent ancestors of the current snapshot.

Methods inherited from interface org.apache.iceberg.PendingUpdate

apply, commit, updateEvent

Method Detail

expireSnapshotId

ExpireSnapshots expireSnapshotId(long snapshotId)

Expires a specific Snapshot identified by id.

Parameters:

snapshotId - long id of the snapshot to expire

Returns:

this for method chaining

expireOlderThan

ExpireSnapshots expireOlderThan(long timestampMillis)

Expires all snapshots older than the given timestamp.

Parameters:

timestampMillis - a long timestamp, as returned by System.currentTimeMillis()

Returns:

this for method chaining

retainLast

ExpireSnapshots retainLast(int numSnapshots)

Retains the most recent ancestors of the current snapshot.

If a snapshot would be expired because it is older than the expiration timestamp, but is one of the numSnapshots most recent ancestors of the current state, it will be retained. This will not cause snapshots explicitly identified by id from expiring.

This may keep more than numSnapshots ancestors if snapshots are added concurrently. This may keep less than numSnapshots ancestors if the current table state does not have that many.

Parameters:

numSnapshots - the number of snapshots to retain

Returns:

this for method chaining

deleteWith

ExpireSnapshots deleteWith(java.util.function.Consumer<java.lang.String> deleteFunc)

Passes an alternative delete implementation that will be used for manifests and data files.

Manifest files that are no longer used by valid snapshots will be deleted. Data files that were deleted by snapshots that are expired will be deleted.

If this method is not called, unnecessary manifests and data files will still be deleted.

Parameters:

deleteFunc - a function that will be called to delete manifests and data files

Returns:

this for method chaining

executeDeleteWith

ExpireSnapshots executeDeleteWith(java.util.concurrent.ExecutorService executorService)

Passes an alternative executor service that will be used for manifests and data files deletion.

Manifest files that are no longer used by valid snapshots will be deleted. Data files that were deleted by snapshots that are expired will be deleted.

If this method is not called, unnecessary manifests and data files will still be deleted using a single threaded executor service.

Parameters:

executorService - an executor service to parallelize tasks to delete manifests and data files

Returns:

this for method chaining

planWith

ExpireSnapshots planWith(java.util.concurrent.ExecutorService executorService)

Passes an alternative executor service that will be used for planning. If this method is not called, the default worker pool will be used.

Parameters:

executorService - an executor service to plan

Returns:

this for method chaining

cleanExpiredFiles

ExpireSnapshots cleanExpiredFiles(boolean clean)

Allows expiration of snapshots without any cleanup of underlying manifest or data files.

Allows control in removing data and manifest files which may be more efficiently removed using a distributed framework through the actions API.

Parameters:

clean - setting this to false will skip deleting expired manifests and files

Returns:

this for method chaining