Package cz.o2.proxima.flink.utils

Class FlinkGlobalWatermarkTracker

java.lang.Object

cz.o2.proxima.flink.utils.FlinkGlobalWatermarkTracker

All Implemented Interfaces:

TimeProvider, GlobalWatermarkTracker, WatermarkSupplier, java.io.Serializable

public class FlinkGlobalWatermarkTracker
extends java.lang.Object
implements GlobalWatermarkTracker

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor	Description
FlinkGlobalWatermarkTracker()

Method Summary

Modifier and Type	Method	Description
void	finished(java.lang.String name)	Remove given process from the tracker.
long	getGlobalWatermark(java.lang.String processName, long currentWatermark)	Retrieve global watermark tracked by this tracker.
java.lang.String	getName()	Retrieve name of this tracker.
void	initWatermarks(java.util.Map<java.lang.String,java.lang.Long> initialWatermarks)	Setup parallel consumers.
void	setup(java.util.Map<java.lang.String,java.lang.Object> cfg)	Configure the tracker using given configuration.
java.util.concurrent.CompletableFuture<java.lang.Void>	update(java.lang.String processName, long currentWatermark)	Update watermark of given process.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface cz.o2.proxima.core.storage.watermark.GlobalWatermarkTracker

getWatermark

Methods inherited from interface cz.o2.proxima.core.time.WatermarkSupplier

getCurrentTime

Constructor Detail

FlinkGlobalWatermarkTracker

public FlinkGlobalWatermarkTracker()

Method Detail

getName

public java.lang.String getName()

Description copied from interface: GlobalWatermarkTracker

Retrieve name of this tracker.

Specified by:

getName in interface GlobalWatermarkTracker

Returns:

name of the tracker

setup

public void setup(java.util.Map<java.lang.String,java.lang.Object> cfg)

Description copied from interface: GlobalWatermarkTracker

Configure the tracker using given configuration.

Specified by:

setup in interface GlobalWatermarkTracker

Parameters:

cfg - the (scoped) configuration map

initWatermarks

public void initWatermarks(java.util.Map<java.lang.String,java.lang.Long> initialWatermarks)

Description copied from interface: GlobalWatermarkTracker

Setup parallel consumers. This SHOULD be used once when the tracker is constructed to setup initial names and watermarks of consumers. Note that consumers might be added during runtime, but their added watermark cannot move the watermark back in time.

Specified by:

initWatermarks in interface GlobalWatermarkTracker

Parameters:

initialWatermarks - map of process name to the initial watermark

update

public java.util.concurrent.CompletableFuture<java.lang.Void> update(java.lang.String processName,
                                                                     long currentWatermark)

Description copied from interface: GlobalWatermarkTracker

Update watermark of given process. This call MAY add a new process, which was not part of GlobalWatermarkTracker.initWatermarks(java.util.Map<java.lang.String, java.lang.Long>). This is asynchronous operation. Users can wait for the completion using the returned CompletableFuture.

Specified by:

update in interface GlobalWatermarkTracker

Parameters:

processName - name of the process

currentWatermark - current processing watermark of the process

Returns:

CompletableFuture to be able to wait for result being persisted

finished

public void finished(java.lang.String name)

Description copied from interface: GlobalWatermarkTracker

Remove given process from the tracker. The watermark of the process (if any) will no longer hold the global watermark. This is asynchronous operation. Users can wait for the completion using the returned CompletableFuture.

Note that this is semantically equivalent to call to GlobalWatermarkTracker.update(java.lang.String, long)(name, Instant.ofEpochMilli(Long.MAX_VALUE))

Specified by:

finished in interface GlobalWatermarkTracker

Parameters:

name - name of the process to remove

getGlobalWatermark

public long getGlobalWatermark(@Nullable
                               java.lang.String processName,
                               long currentWatermark)

Description copied from interface: GlobalWatermarkTracker

Retrieve global watermark tracked by this tracker.

Specified by:

getGlobalWatermark in interface GlobalWatermarkTracker

Parameters:

processName - name of process querying the global watermark

currentWatermark - current watermark of process querying the watermark

Returns:

the global watermark