The Rise of End-user Data Visualization
Most CEP platforms were designed under the assumption that they would run primarily as services. The idea was that someone would author an application — perhaps with a desktop dashboard — that would reside on a server and run more or less in the manner of a traditional database application. This is a fine approach to many problems, but I’d like to call attention to some trends I’ve noticed on the web over the last couple of years that point to another mode of use for this technology.
Yahoo released Yahoo Pipes some time ago, and it hasn’t escaped the community’s notice that it’s quite similar in concept to the boxes-and-arrows authoring mode available in Aleri, Streambase, &c. In my opinion, it sets a decent standard for a service-oriented RSS-based CEP platform that allows arbitrary users to create any number of applications on the fly, rather than depending on someone else to build the applicaion for them. The data sources available to this model continue to grow, including Google’s recent announcement that they’ve added RSS feed support to their customizable news alert service. This is one vision of continuous queries implemented in the cloud, and we should certainly watch this space.
A more recent trend, and one that I think will be important moving forward, is the rise of end-user visualization tools designed to help make sense of all the data the web has made available: the New York Times has created a “visualization lab” to allow readers to mine public data, Swivel and the Many Eyes project from IBM provide similar services, iCharts says they want to be the “YouTube of interactive charts,” a goal that seems to be shared by YouCalc, Trendrr, TracknGraph, and a host of others.
Some of these newcomers, like Widgenie and Google’s new Visualization gadget builder, produce little Javascript applets that can be anchored to one’s desktop in Mac OS X and late-model Windows.
Looking at this explosion of visualization tools converging with Internet-based streaming data sources, I see the future of at least one kind of desktop CEP, especially if the data-mining and visualization tools grow more powerful, along the lines what’s promised by Orange.
Tags: Aleri, cloud, Complex Event Processing, datamining, google, streambase, visualization
2 Responses to “The Rise of End-user Data Visualization”
Leave a Reply
November 6th, 2008 at 8:54 pm
I’m sort of curious what aspects of Orange you’re talking about. Orange is a tool to make exploratory data analysis easier. When I think about doing exploratory analysis over real-time streams - let’s just say that this punishment could be inflicted by a Greek god on an offending statistician.
Of course, I’m all in favor of real-time data in dashboards, and of allowing the end user to customize logic. But that’s not what I think of when I look at Orange…
Hans
November 7th, 2008 at 7:47 am
Hi Hans!
I believe the future of CEP authoring will include some form of exploratory analysis. This can be done by either capturing a portion of a stream up front or using historical data, then working out the application step-by-step as that data is fed through the operators in the engine’s graph. Optimally, changing the parameters of any operator would cause an immediate change in a downstream table that can be inspected via multiple viewing paradigms (spreadsheet view, line graph, whatever).
As for Orange in particular: I like that they’ve got visualizations and AI-based analysis tools in their design palette. These same sorts of components would be very handy for many CEP application authors. Some techniques are easy to move to real-time streams (sampling, for instance), others would be hard or impossible (any that require the entire history to be online, for instance), but the general approach is, I think, a useful one.