Package cz.o2.proxima.beam.core

Class PCollectionTools

  • public class PCollectionTools
extends java.lang.Object
    Various tools related to manipulation with PCollections.

    • Method Summary

      All Methods Static Methods Concrete Methods 
      Modifier and Type Method Description
      static org.apache.beam.sdk.values.PCollection<StreamElement> reduceAsSnapshot​(java.lang.String name, org.apache.beam.sdk.values.PCollection<StreamElement> input)
      Reduce given PCollection from updates to snapshot.

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    • Method Detail

      • reduceAsSnapshot

        public static org.apache.beam.sdk.values.PCollection<StreamElement> reduceAsSnapshot​(@Nullable
                                                                                     java.lang.String name,
                                                                                     org.apache.beam.sdk.values.PCollection<StreamElement> input)
        Reduce given PCollection from updates to snapshot.

        The output element's Beam timestamp is set to the maximum input element timestamp (i.e. the StreamElement.getStamp() of the winning element) via TimestampCombiner.LATEST. Without this, Combine.perKey over the global window would assign BoundedWindow.TIMESTAMP_MAX_VALUE to every output element, which would corrupt any downstream logic that stores the Beam element timestamp as a business-logic stamp.

        Parameters:
        name - name of the operation
        input - the other PCollection containing updates
        Returns:
        snapshot