Siddhi: A Second Look at Complex Event Processing Architectures

The paper titled "Siddhi: A Second Look at Complex Event Processing Architectures" has been presented in Proceedings of the Gateway Computing Environments Workshop (GCE) at Supercomputing 2011 in Seattle, WA, USA. Siddhi was a final year project of '07 batch of University of Moratuwa by a group comprising of Suho, Isuru, Subash, and me. Dr. Srinath Perera, and Ms. Vishaka Nanayakkara supervised the project.

The paper can be found at Siddhi: A Second Look at Complex Event Processing Architectures.

The abstract of the paper is as follows -

Today there are so much data being available from sources like sensors (RFIDs, Near Field Communication), web activities, transactions, social networks, etc. Making sense of this avalanche of data requires efficient and fast processing. Processing of high volume of events to derive higher-level information is a vital part of taking critical decisions, and Complex Event Processing (CEP) has become one of the most rapidly emerging fields in data processing. e-Science use-cases, business applications, financial trading applications, operational analytics applications and business activity monitoring applications are some use-cases that directly use CEP. This paper discusses different design decisions associated with CEP Engines, and proposes some approaches to improve CEP performance by using more stream processing style pipelines. Furthermore, the paper will discuss Siddhi, a CEP Engine that implements those suggestions. We present a performance study that exhibits that the resulting CEP Engine Siddhi has significantly improved performance.  Primary contributions of this paper are performing a critical analysis of the CEP Engine design and identifying suggestions for improvements, implementing those improvements through Siddhi, and demonstrating the soundness of those suggestions through empirical evidence.

Goals of this paper are to critically evaluate decisions taken at CEP design and to present Siddhi CEP Engine that incorporates several improvements uncovered at the discussions. To that end, next section will present a survey of CEP Engine design, and Section 3 will describe Siddhi. Thereafter Section 4 will present a performance comparison of Siddhi and Esper, where latter is an open source, established CEP Engine. The primary contributions of this paper are critical analysis of the CEP Engine design and identifying area for improvements, implementing those improvements, and demonstrating the soundness of those suggestions through empirical evidence. The paper describes how those improvements are achieved through a new CEP called Siddhi.


If you like to contribute to the project, join us http://siddhi.sourceforge.net/index.html