Transformations #

While replication refers to the process of achieving eventual consistency between primary and replica attribute family of the same attribute, transformation is a way to generate new attribute(s) on each update of a given source attribute.

A typical scenario for using transformations is when we want to ensure eventual consistency between different attributes, where certain attribute is a (projected) view of the other one. For instance, imagine we want to store (a subset of) events to attribute family indexed by user. That would mean converting attribute data of entity event into something like attribute event.* of entity user. We can do exactly that using an example configuration below.

Transformations are defined in transformations section of the configuration file, an example of such transformation is defined in example/model/src/main/resources/reference.conf as follows:

transformations {
  event-to-user-history {
    entity: event
    attributes: [ "data" ]
    using: cz.o2.proxima.example.EventDataToUserHistory
  }
}

This definition declares a transformation of attribute data of entity event using a user-defined transformation implemented in cz.o2.proxima.example.EventDataToUserHistory (see here). The transformation is implementation of ElementWiseTransfomation, which receives input StreamElement for updates to the input attribute(s) and emits transformed elements through Collector. The apply method returns number of outputs that will be written through the collector, because the collection might be asynchronous.

As in the case of attribute families (see here), transformations might be invoked for all updates (upserts) to the source attribute(s), or the definition can contain optional filter implementation of StorageFilter interface.