Siddhi: A Second Look at Complex Event Processing Architectures

The paper titled "Siddhi: A Second Look at Complex Event Processing Architectures" has been presented in Proceedings of the Gateway Computing Environments Workshop (GCE) at Supercomputing 2011 in Seattle, WA, USA. Siddhi was a final year project of '07 batch of University of Moratuwa by a group comprising of Suho, Isuru, Subash, and me. Dr. Srinath Perera, and Ms. Vishaka Nanayakkara supervised the project.

The paper can be found at Siddhi: A Second Look at Complex Event Processing Architectures.

The abstract of the paper is as follows -

Today there are so much data being available from sources like sensors (RFIDs, Near Field Communication), web activities, transactions, social networks, etc. Making sense of this avalanche of data requires efficient and fast processing. Processing of high volume of events to derive higher-level information is a vital part of taking critical decisions, and Complex Event Processing (CEP) has become one of the most rapidly emerging fields in data processing. e-Science use-cases, business applications, financial trading applications, operational analytics applications and business activity monitoring applications are some use-cases that directly use CEP. This paper discusses different design decisions associated with CEP Engines, and proposes some approaches to improve CEP performance by using more stream processing style pipelines. Furthermore, the paper will discuss Siddhi, a CEP Engine that implements those suggestions. We present a performance study that exhibits that the resulting CEP Engine Siddhi has significantly improved performance.  Primary contributions of this paper are performing a critical analysis of the CEP Engine design and identifying suggestions for improvements, implementing those improvements through Siddhi, and demonstrating the soundness of those suggestions through empirical evidence.

Goals of this paper are to critically evaluate decisions taken at CEP design and to present Siddhi CEP Engine that incorporates several improvements uncovered at the discussions. To that end, next section will present a survey of CEP Engine design, and Section 3 will describe Siddhi. Thereafter Section 4 will present a performance comparison of Siddhi and Esper, where latter is an open source, established CEP Engine. The primary contributions of this paper are critical analysis of the CEP Engine design and identifying area for improvements, implementing those improvements, and demonstrating the soundness of those suggestions through empirical evidence. The paper describes how those improvements are achieved through a new CEP called Siddhi.


If you like to contribute to the project, join us http://siddhi.sourceforge.net/index.html

Siddhi CEP has won a bronze award at NBQSA 2011

Siddhi Complex Event Processing Engine has won the bronze award in Tertiary category at the recently concluded National Best Quality Software Awards 2011 held in Hotel Galadari, Colombo, Sri Lanka. The winners are now published at NBQSA official website. Further, Siddhi has been nominated to APICTA (Asia Pacific ICT Awards) 2011 which will be held during November 8 – 11, 2011 in Thailand.


Siddhi was our final year project at Dept. of Computer Science & Engineering, Faculty of Engineering, University of Moratuwa. It was a four member project with other members being Suho, Isuru, and Subash. We were overwhelmed with joy on the day we heard that we have secured a place at NBQSA. They weren't specific about the exact award we got. So, we have been waiting with nervously, and full of excitement for the moment for winning the award. It's really a pleasure to know that a year old hard work has gained benefits.

Along with us, another two other projects from our department has won a merit, and a special recognition award. Further, another two projects from Dept. of Electronic & Telecommunication of University of Moratuwa also able to secure a merit and a special recognition award. In total, it's five awards.

Special thanks should go to our supervisors, Dr. Srinath Perera, and Ms. Vishaka Nanayakkara who have given us enormous support. Our project coordinator Dr. Shantha Fernando, Dr. Sanjiva Weerawarana who has provided us guidance when in need should also be remembered.

The Siddhi Team (from left-to-right Suho, me, and Subash; Isuru missed it!)

Award winners at NBQSA 2011