Package cz.o2.proxima.direct.io.kafka

Class KafkaLogReader

    • Method Detail

      • restoresSequentialIds

        public boolean restoresSequentialIds()
        Description copied from interface: CommitLogReader
        Signals the user that this CommitLogReader is able to recover sequential IDs sent to it in StreamElements through its associated writer.
        Specified by:
        restoresSequentialIds in interface CommitLogReader
        Returns:
        true if StreamElement read from the commit log contain valid sequential IDs.

      • observe

        public ObserveHandle observe​(java.lang.String name,
                             Position position,
                             CommitLogObserver observer)
        Subscribe observer by name to the commit log. Each observer maintains its own position in the commit log, so that the observers with different names do not interfere If multiple observers share the same name, then the ingests are load-balanced between them (in an undefined manner). This is a non blocking call.
        Specified by:
        observe in interface CommitLogReader
        Parameters:
        name - identifier of the consumer
        position - the position to seek for in the commit log
        observer - the observer to subscribe to the commit log
        Returns:
        ObserveHandle to asynchronously cancel the observation

      • observePartitions

        public ObserveHandle observePartitions​(java.lang.String name,
                                       @Nullable
                                       java.util.Collection<Partition> partitions,
                                       Position position,
                                       boolean stopAtCurrent,
                                       CommitLogObserver observer)
        Description copied from interface: CommitLogReader
        Subscribe to given set of partitions. If you use this call then the reader stops being automatically load balanced and the set of partitions can only be changed by call to this method again.
        Specified by:
        observePartitions in interface CommitLogReader
        Parameters:
        name - name of the observer
        partitions - the list of partitions to subscribe to
        position - the position to seek to in the partitions
        stopAtCurrent - when true then stop the observer as soon as it reaches most recent record
        observer - the observer to subscribe to the partitions
        Returns:
        ObserveHandle to asynchronously cancel the observation

      • observeBulk

        public ObserveHandle observeBulk​(java.lang.String name,
                                 Position position,
                                 boolean stopAtCurrent,
                                 CommitLogObserver observer)
        Description copied from interface: CommitLogReader
        Subscribe to the commit log in a bulk fashion. That implies that elements are not committed one-by-one, but in a bulks, where all elements in a bulk are committed at once. This is useful for micro-batching approach of data processing.
        Specified by:
        observeBulk in interface CommitLogReader
        Parameters:
        name - name of the observer
        position - the position to seek to in the partitions
        stopAtCurrent - when true then stop the observer as soon as it reaches most recent record
        observer - the observer to subscribe
        Returns:
        ObserveHandle to asynchronously cancel the observation

      • observeBulkPartitions

        public ObserveHandle observeBulkPartitions​(java.lang.String name,
                                           java.util.Collection<Partition> partitions,
                                           Position position,
                                           boolean stopAtCurrent,
                                           CommitLogObserver observer)
        Description copied from interface: CommitLogReader
        Subscribe to given partitions in a bulk fashion.
        Specified by:
        observeBulkPartitions in interface CommitLogReader
        Parameters:
        name - name of the observer
        partitions - the partitions to subscribe to
        position - the position to seek to in the partitions
        stopAtCurrent - when true then stop the observer as soon as it reaches most recent record
        observer - the observer to subscribe to the partitions
        Returns:
        ObserveHandle to asynchronously cancel the observation

      • getPartitions

        public java.util.List<Partition> getPartitions()
        Description copied from interface: CommitLogReader
        Retrieve list of partitions of this commit log.
        Specified by:
        getPartitions in interface CommitLogReader
        Returns:
        list of partitions of this reader

      • fetchOffsets

        public java.util.Map<Partition,​Offset> fetchOffsets​(Position position,
                                                          java.util.List<Partition> partitions)
        Description copied from interface: CommitLogReader
        Fetch offsets at given position from given partitions.
        Specified by:
        fetchOffsets in interface CommitLogReader
        Parameters:
        position - position of the offsets
        partitions - partitions for which to fetch the offsets

      • hasExternalizableOffsets

        public boolean hasExternalizableOffsets()
        Description copied from interface: CommitLogReader
        Signals the user that offsets used by this reader can be externalized and reused later.
        Specified by:
        hasExternalizableOffsets in interface CommitLogReader
        Returns:
        true if Offsets of this reader are externalizable