@Internal public class SimpleSplitAssigner extends java.lang.Object implements SplitAssigner
Constructor and Description |
---|
SimpleSplitAssigner() |
SimpleSplitAssigner(java.util.Collection<IcebergSourceSplitState> assignerState) |
Modifier and Type | Method and Description |
---|---|
GetSplitResult |
getNext(java.lang.String hostname)
Request a new split from the assigner when enumerator trying to assign splits to awaiting
readers.
|
java.util.concurrent.CompletableFuture<java.lang.Void> |
isAvailable()
Enumerator can get a notification via CompletableFuture when the assigner has more splits
available later.
|
void |
onDiscoveredSplits(java.util.Collection<IcebergSourceSplit> splits)
Add new splits discovered by enumerator
|
void |
onUnassignedSplits(java.util.Collection<IcebergSourceSplit> splits)
Forward addSplitsBack event (for failed reader) to assigner
|
int |
pendingSplitCount()
Return the number of pending splits that haven't been assigned yet.
|
java.util.Collection<IcebergSourceSplitState> |
state()
Simple assigner only tracks unassigned splits
|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
close, onCompletedSplits, start
public SimpleSplitAssigner()
public SimpleSplitAssigner(java.util.Collection<IcebergSourceSplitState> assignerState)
public GetSplitResult getNext(@Nullable java.lang.String hostname)
SplitAssigner
If enumerator wasn't able to assign the split (e.g., reader disconnected), enumerator should
call SplitAssigner.onUnassignedSplits(java.util.Collection<org.apache.iceberg.flink.source.split.IcebergSourceSplit>)
to return the split.
getNext
in interface SplitAssigner
public void onDiscoveredSplits(java.util.Collection<IcebergSourceSplit> splits)
SplitAssigner
onDiscoveredSplits
in interface SplitAssigner
public void onUnassignedSplits(java.util.Collection<IcebergSourceSplit> splits)
SplitAssigner
onUnassignedSplits
in interface SplitAssigner
public java.util.Collection<IcebergSourceSplitState> state()
state
in interface SplitAssigner
public java.util.concurrent.CompletableFuture<java.lang.Void> isAvailable()
SplitAssigner
Assigner will return the same future if this method is called again before the previous future is completed.
The future can be completed from other thread, e.g. the coordinator thread from another thread for event time alignment.
If enumerator need to trigger action upon the future completion, it may want to run it in
the coordinator thread using SplitEnumeratorContext.runInCoordinatorThread(Runnable)
.
isAvailable
in interface SplitAssigner
public int pendingSplitCount()
SplitAssigner
The enumerator can poll this API to publish a metric on the number of pending splits.
The enumerator can also use this information to throttle split discovery for streaming read. If there are already many pending splits tracked by the assigner, it is undesirable to discover more splits and track them in the assigner. That will increase the memory footprint and enumerator checkpoint size.
Throttling works better together with ScanContext.maxPlanningSnapshotCount()
.
Otherwise, the next split discovery after throttling will just discover all non-enumerated
snapshots and splits, which defeats the purpose of throttling.
pendingSplitCount
in interface SplitAssigner