Here’s an interesting pattern, one that uses features from our upcoming 3.0 release of the Aleri Streaming Platform: internal pulsing to control the amount of recalculation.

The problem is simple. Suppose there are a lot of updates flowing into a stream (updates are supported natively in the Aleri Streaming Platform). A good example is a stock market feed that keeps the last tick for each symbol. Some of the downstream calculations might, however, be computationally expensive, and you might not even need to recalculate on every change. You might want, for instance, to recalculate only every second or every ten seconds. How can you collect and pulse the updates so that the expensive recalculations are done periodically instead of continuously?

[Note: the Aleri Streaming Platform has a related feature—called “pulsed subscription”—that allows an external client to coalesce changes every n seconds. But that deals with data sent to the outside world, not processed internally.]

In 3.0, the new dictionary data structure and the timer facility allow you to code internal pulsing. Let’s suppose that the stream to control is called “InStream”. We’ll write a small FlexStream that does the pulsing. First, let’s define two local variables in the FlexStream:

  int32 version := 0; 

  dictionary(typeof(InStream), int32) versionMap;

These two variables keep a current version, and a version number for each record. The SPLASH code handling events from the input stream is

  { 

    versionMap[InStream] := version; 

  }

The special Timer block within the FlexStream sends the inserts and updates:

  { 

    for (k in versionMap) { 

      if (version = versionMap[k]) output setOpcode(k, upsert); 

    } 

    version := version + 1; 

  }

(You can configure the interval between runs of the Timer block in numbers of seconds.) Notice how only those events with the current version get sent downstream, and how the version number gets incremented for the next set of updates.

That’s it. There are other patterns built from data structures, and I’ll give some more examples in future blog posts.

Marco’s CEP Blog mentioned that the upcoming 3.0 release of the Aleri Streaming Platform will include enhancements to SPLASH.  SPLASH (an acronym for “Streaming Platform LAnguage SHell”—obviously coined by programming geeks) is our embedded scripting language.  It allows you to write your own event-processing logic without being constrained by stateless relational operations.  I’ve talked about SPLASH, and its wrapper, the FlexStream, in a white paper before.  And now SPLASH will be more powerful and easier to use.

First, we’ve added variables and user-defined functions.  The syntax comes close to C syntax, while preserving some of the features of SQL expression syntax (e.g., equality test is “=” and assignment is “:=” in SPLASH.)  Here, for instance, is the rendering of a recursive factorial function:

int32 factorial(int32 x) {
if (x <= 0) {
return 1;
} else {
return factorial(x-1) * x;
}
}

Functions can be mutually recursive too.  Variables and functions can be defined globally for all streams, or locally for the computational streams: Compute, Filter, Join, Aggregate, Flex, and our new Pattern Stream.

Second, we’ve enhance the language with some familiar data structures (vectors, dictionaries), some less familiar ones (ordered record sets), and some novel ones (event caches, which hold a limited or unbounded collection of insert/update/deletes, organized into buckets by key).  Any decent scripting language provides vectors (or arrays) and dictionaries: AWK, Perl, Python, to name a few.  They’ve stood the test of time.  The other data structures have proved helpful in client use-cases.

It’s important to note that SPLASH programming doesn’t break the inherent programming model of the Aleri Streaming Platform.  The Platform is still a dataflow programming environment: streams respond to and produce events, and events are queued by streams before being processed.  SPLASH simply allows more flexibility in building streams.

Today is the first day of the SIFMA technology show, the big event of the year in the US for trading technology in the capital markets. It’s particularly significant for Aleri, since we announced version 3.0 of Aleri CEP yesterday, and will be previewing it for the public starting today at SIFMA.

We’re really excited by this new release. While we’ve always aimed to deliver a CEP platform that was unsurpassed in terms of breadth of functionality, performance and scalability, and enterprise-class features, those of you who have gotten to know current - or earlier - versions of the product know that it’s not exactly the easiest thing to learn how to use.

With 3.0 we focused our attention on reducing the learning curve for new users, making experienced users more efficient, and making it easier and faster to get up and running quickly. Part of this simply involved improving the look and feel of the Aleri Studio to make it more intuitive, but the even bigger gains were around our approach to connectivity, a new enhanced syntax for pattern matching, and some general simplifications to our language.

Connectivity is probably the biggest difference. 3.0 comes with integrated connectivity. The base product includes a range of built-in connectors for different types of data sources and destinations. Connections can easily and quickly be defined from within the Studio. A data discovery feature allows you to explore a data source and import the data schema directly into your Aleri data model. Field-installable adapters for specialized systems are discovered by the Studio and function the same as the built in connectors.

The new Pattern Matching operator uses a new syntax for expressing complex patterns of events on a single stream or across multiple streams, including things like the time window over which to watch for a pattern, the sequence and relationship of the events that make up the pattern, and even the absence of events as part of the pattern. When a pattern is detected, an output event is generated with the desired information. More info  on the new syntax will be available in the days to come on aleri.com.

And while I’m excited about all the features that make Aleri CEP easier to use, I’d be remiss if I didn’t mention that there are also a range of new features that increase the capabilities, including some significant extensions to Aleri’s SPLASH scripting language. SPLASH is an imperative language that overcomes the limitations of SQL. Standard SQL operators provide simplicity for many common operations such as filtering, joins or aggregation, but there are times when you just need a greater level of control. That’s where SPLASH comes in, with Aleri’s programmable Flex Stream that invokes methods written in SPLASH on the arrival of each new event. In 3.0 SPLASH has become more powerful, with new advanced data structures, user defined functions, some new pre-defined functions and greater control.

More to come in the coming weeks. 3.0 is currently available for previews. A public beta version is expected to be downloadable from aleri.com in early July.