An important but little-understood feature of Aleri’s CEP implementation is the ability to process incoming events as updates or changes to the state of a data set. Another way of looking at this is an event that reports a change to a previously reported event.

Let’s take an example: we are seeing a lot of interest these days in the capital markets around analyzing market depth. Market depth is represented in order books: the collection of open orders for a particular security. An order book has state: it is the current set of open orders on an exchange or other execution venue. Changes to the order book – new orders, changes to existing orders, order cancellations – are reported in real-time by the execution venue.

In order to analyze the depth of the market, you need to be able to analyze the current order book – i.e. all the open orders. So you have to maintain the state of the order book by applying the delta events as you receive them.

The Aleri CEP engine does this by natively supporting update and delete operations in addition to the traditional insert (or append) operation. This is fundamentally very different from a more traditional CEP implementation, where every incoming event is treated either as a completely atomic event or else as the latest event in a time series. To maintain the state of the order book, it’s not enough to keep the last value – like the last price for a given stock. You have to maintain the entire state of the order book, adding new orders, applying changes to previously reported orders

When you have the ability to maintain the state of a data set, such as an order book, then there are a number of ways you can use that information. One is to perform continuous computations against the data, updating the high level information every time the state of the data changes. Coming back to the order book example: let’s say you want to measure the depth of the market by computing the total number of shares being bid/offered within a given price range. Every update to the order book can refresh the computed values. The second way to use this information is by using the current state of the data set (in this case the order book) in the processing of other events. An example of this would be processing an incoming order to buy or sell some stock, and using the current state of the order book to determine where or how to execute the order.

Aleri’s CEP architecture allows an event source (or an adapter) to specify whether to apply the event as an insert, update, delete or upsert to the current set of retained data within a given stream. The key column of the stream is used to identify which item to update or delete.

This feature opens up CEP to a broader range of applications where events carry information about changes to the state of a system or an information set. Doing it natively, within the dataflow architecture, ensures that the low-latency characteristics of the architecture are preserved.