The DOI matches its Crossref record; 85 of the 85 references it cites were found in a registry or an index.
Processing flows of information: From data stream to complex event processing
Abstract
A large number of distributed applications requires continuous and timely processing of information as it flows from the periphery to the center of the system. Examples include intrusion detection systems which analyze network traffic in real-time to identify possible attacks; environmental monitoring applications which process raw data coming from sensor networks to identify critical situations; or applications performing online analysis of stock prices to identify trends and forecast future values. Traditional DBMSs, which need to store and index data before processing it, can hardly fulfill the requirements of timeliness coming from such domains. Accordingly, during the last decade, different research communities developed a number of tools, which we collectively call Information flow processing (IFP) systems , to support these scenarios. They differ in their system architecture, data model, rule model, and rule language. In this article, we survey these systems to help researchers, who often come from different backgrounds, in understanding how the various approaches they adopt may complement each other. In particular, we propose a general, unifying model to capture the different aspects of an IFP system and use it to provide a complete and precise classification of the systems and mechanisms proposed so far.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Time, clocks, and the ordering of events in a distributed system
Lamport, L. (1978) · Communications of the ACM · doi:10.1145/359545.359563 · Cited by 8,488
The temporal logic of programs
Pnueli, A. (1977) · 18th Annual Symposium on Foundations of Computer Science (sfcs 1977) · doi:10.1109/sfcs.1977.32 · Cited by 5,689
The many faces of publish/subscribe
Eugster, P. T., Felber, P. A., Guerraoui, R., & Kermarrec, A.-M. (2003) · ACM Computing Surveys · doi:10.1145/857076.857078 · Cited by 3,145
Rete: A fast algorithm for the many pattern/many object pattern match problem
Forgy, C. L. (1982) · Artificial Intelligence · doi:10.1016/0004-3702(82)90020-0 · Cited by 2,634
Models and issues in data stream systems
Babcock, B., Babu, S., Datar, M., Motwani, R., & Widom, J. (2002) · Proceedings of the twenty-first ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems · doi:10.1145/543613.543615 · Cited by 2,495
An overview of data warehousing and OLAP technology
Chaudhuri, S., & Dayal, U. (1997) · ACM SIGMOD Record · doi:10.1145/248603.248616 · Cited by 2,424
Aurora: a new model and architecture for data stream management
Abadi, D. J., Carney, D., �etintemel, U., Cherniack, M., Convey, C., Lee, S., Stonebraker, M., Tatbul, N., & Zdonik, S. (2003) · The VLDB Journal The International Journal on Very Large Data Bases · doi:10.1007/s00778-003-0095-z · Cited by 1,495
Design and evaluation of a wide-area event notification service
Carzaniga, A., Rosenblum, D. S., & Wolf, A. L. (2001) · ACM Transactions on Computer Systems · doi:10.1145/380749.380767 · Cited by 1,425
The Power of Events: An Introduction to Complex Event Processing in Distributed Enterprise Systems
Luckham, D. (2008) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-88808-6_2 · Cited by 1,285
The design of the borealis stream processing engine
Abadi, D. J., Ahmad, Y., Bałazińska, M., Cherniack, M., Hwang, J.-H., Lindner, W., Maskey, A. S., Rasin, A., Ryvkina, E., Tatbul, N., et al. (2005) · Cited by 1,255
The CQL continuous query language: semantic foundations and query execution
Arasu, A., Babu, S., & Widom, J. (2006) · The VLDB Journal · doi:10.1007/s00778-004-0147-z · Cited by 1,170
NiagaraCQ: a scalable continuous query system for Internet databases
Chen, J., DeWitt, D. J., Tian, F., & Wang, Y. (2000) · ACM SIGMOD Record · doi:10.1145/335191.335432 · Cited by 1,022
Issues in data stream management
Golab, L., & ÖZSU, M. T. (2003) · ACM SIGMOD Record · doi:10.1145/776985.776986 · Cited by 964
A brief account of runtime verification
Leucker, M., & Schallhart, C. (2009) · The Journal of Logic and Algebraic Programming · doi:10.1016/j.jlap.2008.08.004 · Cited by 863
High-performance complex event processing over streams
Wu, E., Diao, Y., & Rizvi, S. (2006) · Proceedings of the 2006 ACM SIGMOD international conference on Management of data · doi:10.1145/1142473.1142520 · Cited by 835
Sellis, T. K. (1988) · ACM Transactions on Database Systems · doi:10.1145/42201.42203 · Cited by 809
Eddies: continuously adaptive query processing
Avnur, R., & Hellerstein, J. M. (2000) · ACM SIGMOD Record · doi:10.1145/335191.335420 · Cited by 798
TelegraphCQ: continuous dataflow processing
Chandrasekaran, S., Cooper, O., Deshpande, A., Franklin, M. J., Hellerstein, J. M., Hong, W., Krishnamurthy, S., Madden, S. R., Reiss, F., & Shah, M. A. (2003) · Proceedings of the 2003 ACM SIGMOD international conference on Management of data · doi:10.1145/872757.872857 · Cited by 744
Continuous queries over data streams
Babu, S., & Widom, J. (2001) · ACM SIGMOD Record · doi:10.1145/603867.603884 · Cited by 721
Gigascope: a stream database for network applications
Cranor, C., Johnson, T., Spataschek, O., & Shkapenyuk, V. (2003) · Proceedings of the 2003 ACM SIGMOD international conference on Management of data · doi:10.1145/872757.872838 · Cited by 687
Query Optimization in Database Systems
Jarke, M., & Koch, J. (1984) · ACM Computing Surveys · doi:10.1145/356924.356928 · Cited by 650
Van Renesse, R., Birman, K. P., & Vogels, W. (2003) · ACM Transactions on Computer Systems · doi:10.1145/762483.762485 · Cited by 645
Active Database Systems: Triggers and Rules for Advanced Database Processing
Dayal, U., Widom, J., & Ceri, S. (1994) · Morgan Kaufmann Publishers Inc. eBooks · Cited by 644
Matching events in a content-based subscription system
Aguilera, M. K., Strom, R. E., Sturman, D. C., Astley, M., & Chandra, T. D. (1999) · Proceedings of the eighteenth annual ACM symposium on Principles of distributed computing · doi:10.1145/301308.301326 · Cited by 638
The JEDI event-based infrastructure and its application to the development of the OPSS WFMS
Cugola, G., Di Nitto, E., & Fuggetta, A. (2001) · IEEE Transactions on Software Engineering · doi:10.1109/32.950318 · Cited by 633
Self-citationAggregation and Correlation of Intrusion-Detection Alerts
Debar, H., & Wespi, A. (2001) · Lecture Notes in Computer Science · doi:10.1007/3-540-45474-8_6 · Cited by 616
Continuously adaptive continuous queries over streams
Madden, S., Shah, M., Hellerstein, J. M., & Raman, V. (2002) · Proceedings of the 2002 ACM SIGMOD international conference on Management of data · doi:10.1145/564691.564698 · Cited by 583
Load Shedding in a Data Stream Manager
Tatbul, N., Çetintemel, U., Zdonik, S., Cherniack, M., & Stonebraker, M. (2003) · Proceedings 2003 VLDB Conference · doi:10.1016/b978-012722442-8/50035-5 · Cited by 577
Paton, N. W., & Díaz, O. (1999) · ACM Computing Surveys · doi:10.1145/311531.311623 · Cited by 567
Efficient Filtering of XML Documents for Selective Dissemination of Information
Altınel, M., & Franklin, M. J. M. (2000) · Cited by 548
Deutsch, A., Fernández, M., Florescu, D., Levy, A. Y., & Suciu, D. M. (1999) · Computer Networks · doi:10.1016/s1389-1286(99)00020-1 · Cited by 544
Scalable Distributed Stream Processing
Cherniack, M., Balakrishnan, H., Bałazińska, M., Carney, D., Xing, Y., & Zdonik, S. (2003) · Cited by 496
An event-based architecture definition language
Luckham, D. C., & Vera, J. (1995) · IEEE Transactions on Software Engineering · doi:10.1109/32.464548 · Cited by 482
Hermes: a distributed event-based middleware architecture
Pietzuch, P. R., & Bacon, J. M. (2003) · Proceedings 22nd International Conference on Distributed Computing Systems Workshops · doi:10.1109/icdcsw.2002.1030837 · Cited by 482
Snoop: An expressive event specification language for active databases
Chakravarthy, S., & Mishra, D. (1994) · Data & Knowledge Engineering · doi:10.1016/0169-023x(94)90006-x · Cited by 476
Proceedings of the 26th International Conference on Data Engineering, ICDE 2010
Li, F., Moro, M. M., Ghandeharizadeh, S., Haritsa, J. R., Weikum, G., Carey, M. J., Casati, F., Chang, E. Y., Manolescu, I., Mehrotra, S., et al. (2010) · Max Planck Institute for Plasma Physics · Cited by 452
Network-Aware Operator Placement for Stream-Processing Systems
Pietzuch, P., Ledlie, J., Shneidman, J., Roussopoulos, M., Welsh, M., & Seltzer, M. (2006) · 22nd International Conference on Data Engineering (ICDE'06) · doi:10.1109/icde.2006.105 · Cited by 428
SPADE: the system s declarative stream processing engine
Gedik, B., Andrade, H., Wu, K.-L., Yu, P. S., & Doo, M. (2008) · Proceedings of the 2008 ACM SIGMOD international conference on Management of data · doi:10.1145/1376616.1376729 · Cited by 421
Efficient pattern matching over event streams
Agrawal, J., Diao, Y., Gyllstrom, D., & Immerman, N. (2008) · ACM SIGMOD International Conference on Management of Data (SIGMOD) · doi:10.1145/1376616.1376634 · Cited by 418
STREAM: The Stanford Stream Data Manager.
Arasu, A., Babcock, B., Babu, S., Datar, M., Ito, K. A., Motwani, R., Nishizawa, I., Srivastava, U., Thomas, D., Varma, R., et al. (2003) · IEEE Data(base) Engineering Bulletin · Cited by 410
Clarke, E. M., & Schlingloff, B. (2001) · Elsevier eBooks · doi:10.1016/b978-044450813-3/50026-6 · Cited by 391
Achieving scalability and expressiveness in an Internet-scale event notification service
Carzaniga, A., Rosenblum, D. S., & Wolf, A. L. (2000) · Proceedings of the nineteenth annual ACM symposium on Principles of distributed computing · doi:10.1145/343477.343622 · Cited by 385
Eddies: continuously adaptive query processing
Avnur, R., & Hellerstein, J. M. (2000) · Proceedings of the 2000 ACM SIGMOD international conference on Management of data · doi:10.1145/342009.335420 · Cited by 381
Rule-Based Runtime Verification
Howard Barringer, B., Goldberg, A., Havelund, K., & Sen, K. (2004) · Lecture notes in computer science · doi:10.1007/978-3-540-24622-0_5 · Cited by 379
Temporal management of RFID data
Wang, F., & Liu, P. (2005) · Very Large Data Bases · Cited by 340
Continual queries for Internet scale event-driven information delivery
Liu, L., Pu, C., & Tang, W. (1999) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/69.790816 · Cited by 332
Load shedding for aggregation queries over data streams
Babcock, B., Datar, M., & Motwani, R. (2004) · Proceedings. 20th International Conference on Data Engineering · doi:10.1109/icde.2004.1320010 · Cited by 311
Towards Expressive Publish/Subscribe Systems
Demers, A. J., Gehrke, J. E., Hong, M., Riedewald, M., & White, W. M. (2006) · Lecture notes in computer science · doi:10.1007/11687238_38 · Cited by 283
Adi, A., & Etzion, O. (2004) · The VLDB Journal · doi:10.1007/s00778-003-0108-y · Cited by 270
High-Availability Algorithms for Distributed Stream Processing
Hwang, J.-H., Balazinska, M., Rasin, A., Cetintemel, U., Stonebraker, M., & Zdonik, S. (2005) · 21st International Conference on Data Engineering (ICDE'05) · doi:10.1109/icde.2005.72 · Cited by 261
Ode as an Active Database: Constraints and Triggers
Gehani, N. H., & Jagadish, H. V. (1991) · Cited by 249
Cayuga: a high-performance event processing engine
Brenna, L., Demers, A. J., Gehrke, J. E., Hong, M., Ossher, J., Panda, B., Riedewald, M., Thatte, M., & White, W. M. (2007) · ACM SIGMOD International Conference on Management of Data (SIGMOD) · doi:10.1145/1247480.1247620 · Cited by 239
TESLA: a formally defined event specification language
Cugola, G., & Margara, A. (2010) · ACM International Conference on Distributed and Event-Based Systems (DEBS) · doi:10.1145/1827418.1827427 · Cited by 230
Self-citationConsistent Streaming Through Time: A Vision for Event Stream Processing
Barga, R., Goldstein, J., Ali, M., & Hong, M. (2006) · arXiv (Cornell University) · doi:10.48550/arxiv.cs/0612115 · Cited by 221
Pallickara, S., & Fox, G. (2003) · Cited by 216
GEM: a generalized event monitoring language for distributed systems
Mansouri-Samani, M., & Sloman, M. S. (1997) · Distributed Systems Engineering · doi:10.1088/0967-1846/4/2/004 · Cited by 214
Highly available, fault-tolerant, parallel dataflows
Shah, M. A., Hellerstein, J. M., & Brewer, E. (2004) · Proceedings of the 2004 ACM SIGMOD international conference on Management of data · doi:10.1145/1007568.1007662 · Cited by 204
Aurora: a data stream management system
Abadi, D., Carney, D., Çetintemel, U., Cherniack, M., Convey, C., Erwin, C., Galvez, E., Hatoun, M., Maskey, A., Rasin, A., et al. (2003) · Proceedings of the 2003 ACM SIGMOD international conference on Management of data · doi:10.1145/872757.872855 · Cited by 201
A design framework for Internet-scale event observation and notification
Rosenblum, D. S., & Wolf, A. L. (1997) · doi:10.1145/267895.267920 · Cited by 201
Distributed complex event processing with query rewriting
Schultz-Møller, N. P., Migliavacca, M., & Pietzuch, P. (2009) · Proceedings of the Third ACM International Conference on Distributed Event-Based Systems · doi:10.1145/1619258.1619264 · Cited by 192
Composite Subscriptions in Content-Based Publish/Subscribe Systems
Li, G., & Jacobsen, H. (2005) · Lecture notes in computer science · doi:10.1007/11587552_13 · Cited by 187
Time-critical on-demand data broadcast: algorithms, analysis, and performance evaluation
Xu, J., Tang, X., & Lee, W. (2006) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2006.14 · Cited by 168
Fault-tolerance in the borealis distributed stream processing system
Balazinska, M., Balakrishnan, H., Madden, S. R., & Stonebraker, M. (2008) · ACM Transactions on Database Systems · doi:10.1145/1331904.1331907 · Cited by 165
Towards a streaming SQL standard
Jain, N., Mishra, S., Srinivasan, A., Gehrke, J., Widom, J., Balakrishnan, H., Çetintemel, U., Cherniack, M., Tibbetts, R., & Zdonik, S. (2008) · Proceedings of the VLDB Endowment · doi:10.14778/1454159.1454179 · Cited by 159
Gryphon: An Information Flow Based Approach to Message BrokeringPreprint
Strom, R., Banavar, G. S., Chandra, T., Kaplan, M., Miller, K. L., Mukherjee, B., Sturman, D., & Ward, M. (1998) · arXiv (Cornell University) · doi:10.48550/arxiv.cs/9810019 · Cited by 158
SPC: a distributed, scalable platform for data mining
Amini, L. D., Andrade, H. C. M., Bhagwan, R., Eskesen, F., King, R. P., Selo, P., Park, Y., & Venkatramani, C. (2006) · doi:10.1145/1289612.1289615 · Cited by 158
Complex event processing with T-REX
Cugola, G., & Margara, A. (2012) · Journal of Systems and Software · doi:10.1016/j.jss.2012.03.056 · Cited by 146
Self-citationAdaptive Control of Extreme-scale Stream Processing Systems
Amini, L. D., Jain, N., Sehgal, A., Silber, J., & Verscheure, O. (2006) · IEEE International Conference on Distributed Computing Systems (ICDCS) · doi:10.1109/icdcs.2006.13 · Cited by 138
Query Languages and Data Models for Database Sequences and Data Streams
Law, Y.-N., Wang, H., & Zaniolo, C. (2004) · Elsevier eBooks · doi:10.1016/b978-012088469-8.50045-0 · Cited by 132
SODA : An optimizing scheduler for large-scale stream-based distributed computer systems
Wolf, J. L., Bansal, N., Profile, S., Hildrum, K. W., Rajan, D., Wolf, J., Hildrum, K. W., Parekh, S. S., Wagle, R., Wu, K., et al. (2008) · Cited by 126
A framework for event composition in distributed systems
Pietzuch, P. R., Shand, B. N., & Bacon, J. (2003) · ACM/IFIP/USENIX international conference on Middleware · doi:10.5555/1515915.1515921 · Cited by 125
Monitoring distributed systems
Mansouri-Samani, M., & Sloman, M. S. (1993) · IEEE Network · doi:10.1109/65.244791 · Cited by 106
On Supporting Kleene Closure over Event Streams
Gyllstrom, D., Agrawal, J., Diao, Y., & Immerman, N. (2008) · IEEE International Conference on Data Engineering (ICDE) · doi:10.1109/icde.2008.4497566 · Cited by 106
A data stream language and system designed for power and extensibility
Bai, Y., Thakkar, H., Wang, H., Luo, C. R., & Zaniolo, C. (2006) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/1183614.1183664 · Cited by 100
Complex event processing over uncertain data
Wasserkrug, S., Gal, A., Etzion, O., & Turchin, Y. (2008) · ACM International Conference on Distributed and Event-Based Systems (DEBS) · doi:10.1145/1385989.1386022 · Cited by 94
XNET: a reliable content-based publish/subscribe system
Chand, R., & Felber, P. A. (2004) · Proceedings/Proceedings - Symposium on Reliable Distributed Systems · doi:10.1109/reldis.2004.1353027 · Cited by 69
Model-based runtime analysis of distributed reactive systems
Bauer, A., Leucker, M., & Schallhart, C. (2006) · Australasian Software Engineering Conference (ASWEC) · doi:10.1109/aswec.2006.36 · Cited by 67
What is "next" in event processing?
White, W. M., Riedewald, M., Gehrke, J. E., & Demers, A. J. (2007) · ACM SIGACT-SIGMOD-SIGART Symposium on Principles of Database Systems (PODS) · doi:10.1145/1265530.1265567 · Cited by 55
Fiege, L., Mühl, G., & Gärtner, F. C. (2002) · The Knowledge Engineering Review · doi:10.1017/s0269888903000559 · Cited by 55
Probabilistic Event Extraction from RFID Data
Khoussainova, N., Bałazińska, M., & Suciu, D. M. (2008) · Proceedings - International Conference on Data Engineering · doi:10.1109/icde.2008.4497596 · Cited by 54
Modeling uncertainties in publish/subscribe systems
Liu, H., & Jacobsen, H. (2004) · Proceedings - International Conference on Data Engineering · doi:10.1109/icde.2004.1320023 · Cited by 52
SAGE: A Logical Agent-Based Environment Monitoring and Control System
Broda, K. B., Clark, K., Miller, R., & Russo, A. (2009) · Lecture notes in computer science · doi:10.1007/978-3-642-05408-2_14 · Cited by 49
Temporal Assertions with Parametrised Propositions
Stolz, V. (2007) · Springer eBooks · doi:10.1007/978-3-540-77395-5_15 · Cited by 23
Cited by
Machine learning with big data: challenges and approaches.
L'Heureux, A., Grolinger, K., Elyamany, H. F., & Capretz, M. A. M. (2017) · IEEE Access · doi:10.1109/access.2017.2696365 · Cited by 1,023
Sun, A. Y., & Scanlon, B. R. (2019) · Environmental Research Letters · doi:10.1088/1748-9326/ab1b7d · Cited by 551
An event-driven manufacturing information system architecture for Industry 4.0
Theorin, A., Bengtsson, K., Provost, J., Lieder, M., Johnsson, C., Lundholm, T., & Lennartson, B. (2017) · International Journal of Production Research · doi:10.1080/00207543.2016.1201604 · Cited by 348
New Horizons for a Data-Driven Economy
(2016) · Springer eBooks · doi:10.1007/978-3-319-21569-3 · Cited by 253
A survey on the evolution of stream processing systems
Fragkoulis, M., Carbone, P., Kalavri, V., & Katsifodimos, A. (2024) · The VLDB Journal · doi:10.1007/s00778-023-00819-8 · Cited by 207
Metadata
- Authors
- Cugola, G., & Margara, A.
- Venue
- ACM Computing Surveys
- Published
- 2012-06
- Volume
- 44
- Issue
- 3
- Pages
- 1-62
- ISSN
- 0360-0300 (ISSN-L)
- 1557-7341
- OpenAlex
- W1973026225
- Cited by
- 833
- References
- 85
- FWCI
- 88.63
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.