A New Breed of Systems Software Makes Inbound Processing a Reality

February 14, 2005

StreamBase Systems, Inc. launches a new class of system software at the DEMO@15! Conference today. The StreamBAse Stream Processing Engine is the first of a new class of system software that provides the ability to process, analyze, and act on streaming real-time information at blindingly fast speeds while still allowing developers to build, deploy, and modify their real-time applications in as little as a few hours. Created to address the realities of processing streaming real-time data that is high volume, moves at high velocity, is highly volatile with changes measured in fractions of seconds and has a value with a short shelf-life, StreamBase applies familiar models to a new problem in a novel and exciting way.

Dr. Mike Stonebraker, one the world's foremost RDBMS pioneers and founder of Ingres and Illustra, now at MIT, has been working for some years on what is believed to be the next exciting thing in the world of data management. The common thought process regarding data management is 'storage and retrieval.' However, there is an increasing area of data management that deals with a different kind of data--data that is valuable when it is received, but has little or no historical value. The objective with this type of data is to react to it, not store it, (although it can be stored, after it has been reacted to). Standard databases deal ineffectively with this type of data. We are moving more and more towards real-time data, without a database engine designed to handle streams of data rather than stored data. The project Dr. Stonebraker has been working on, originally called Aurora, has developed a new infrastructure, called a Stream Processing Engine, which does inbound processing and has now been released as StreamBase. The StreamBase Processing Engine responds to the challenge of how to understand and react instantaneously to the unprecedented increase in the amount of live data in today's world.

The first thing a database does is store the data. Whether it is a large relational database, or a real-time engine, the data is essentially stopped and put into some kind of store. The fundamental principle behind StreamBase is to keep the data moving--data is not stored in any container, it is not put on disk or even stored in memory. StreamBase is able to run SQL-type queries directly on the data as it slides by, a significant element of why it is as quick as it is, running as high as 142,000 messages per second.

"Once you recognize that storage is not always necessary for real-time data, and that it is better to do queries on the stream of data rather than on a disk, you will get 100 times performance improvement," states StreamBase CEO Barry Morris. In addition, StreamBase is a multi-threaded, all in one operating system process; it does not talk to another database server, nor does it talk to a middleware system.

Initial interest in this project stemmed from the possibility of sensor networks and the need for every barcode to come with an electronic tag. The advent of sensor networks and RFID technologies will accelerate a flood of event data. The possibilities going forward are endless. The types of applications that can be built with StreamBase vary as widely as the data those applications will handle.

Financial services, telecommunications, industrial process control, homeland security, and the military are key areas where these streams of real-time complex events must be processed, analyzed, and acted upon with virtually zero latency. A variety of adapters, such as TIBCO® Rendezvous, are available to interface common market data feeds or customers can build their own input/output adapters.

The StreamBase Processing Engine combines four critical elements to meet the needs of the real-time information environment.

Available from StreamBase, the product is licensed on a pre-processor basis with typical deployments starting at $60,000 per year.