Package cz.o2.proxima.core.storage.watermark

Interface GlobalWatermarkTracker

All Superinterfaces:

java.io.Serializable, TimeProvider, WatermarkSupplier

All Known Implementing Classes:

FlinkGlobalWatermarkTracker, ZKGlobalWatermarkTracker

@Evolving
public interface GlobalWatermarkTracker
extends WatermarkSupplier

A tracker of global watermark progress among multiple (distributed) processes.

A GlobalWatermarkTracker consists of the following:

1. Name of the tracker. The name MUST uniquely identify the tracker among all possible global trackers. The name also serves as grouping identifier for calculation of the value of global watermark.
2. Predefined list of processes.

Method Summary

Modifier and Type	Method	Description
void	finished(java.lang.String name)	Remove given process from the tracker.
long	getGlobalWatermark(java.lang.String processName, long currentWatermark)	Retrieve global watermark tracked by this tracker.
java.lang.String	getName()	Retrieve name of this tracker.
default long	getWatermark()	Retrieve watermark.
void	initWatermarks(java.util.Map<java.lang.String,java.lang.Long> initialWatermarks)	Setup parallel consumers.
void	setup(java.util.Map<java.lang.String,java.lang.Object> cfg)	Configure the tracker using given configuration.
java.util.concurrent.CompletableFuture<java.lang.Void>	update(java.lang.String processName, long currentWatermark)	Update watermark of given process.

Methods inherited from interface cz.o2.proxima.core.time.WatermarkSupplier

getCurrentTime

Method Detail

getName

java.lang.String getName()

Retrieve name of this tracker.

Returns:

name of the tracker

setup

void setup(java.util.Map<java.lang.String,java.lang.Object> cfg)

Configure the tracker using given configuration.

Parameters:

cfg - the (scoped) configuration map

initWatermarks

void initWatermarks(java.util.Map<java.lang.String,java.lang.Long> initialWatermarks)

Setup parallel consumers. This SHOULD be used once when the tracker is constructed to setup initial names and watermarks of consumers. Note that consumers might be added during runtime, but their added watermark cannot move the watermark back in time.

Parameters:

initialWatermarks - map of process name to the initial watermark

update

java.util.concurrent.CompletableFuture<java.lang.Void> update(java.lang.String processName,
                                                              long currentWatermark)

Update watermark of given process. This call MAY add a new process, which was not part of initWatermarks(java.util.Map<java.lang.String, java.lang.Long>). This is asynchronous operation. Users can wait for the completion using the returned CompletableFuture.

Parameters:

processName - name of the process

currentWatermark - current processing watermark of the process

Returns:

CompletableFuture to be able to wait for result being persisted

finished

void finished(java.lang.String name)

Remove given process from the tracker. The watermark of the process (if any) will no longer hold the global watermark. This is asynchronous operation. Users can wait for the completion using the returned CompletableFuture.

Note that this is semantically equivalent to call to update(java.lang.String, long)(name, Instant.ofEpochMilli(Long.MAX_VALUE))

Parameters:

name - name of the process to remove

getGlobalWatermark

long getGlobalWatermark(@Nullable
                        java.lang.String processName,
                        long currentWatermark)

Retrieve global watermark tracked by this tracker.

Parameters:

processName - name of process querying the global watermark

currentWatermark - current watermark of process querying the watermark

Returns:

the global watermark

getWatermark

default long getWatermark()

Description copied from interface: WatermarkSupplier

Retrieve watermark.

Specified by:

getWatermark in interface WatermarkSupplier

Returns:

the current watermark.