CoChat
Sign Up
Cited by 956Advanced Database Systems and QueriesCloud Computing and Resource ManagementDistributed systems and fault tolerance

The DOI matches its Crossref record.

Storm@twitter

Toshniwal, Ankit, Taneja, Siddarth, Shukla, Amit, Ramasamy, Karthik, Patel, Jignesh M., Kulkarni, Sanjeev, Jackson, Jason, Gade, Krishna, Fu, Maosong, Donham, Jake, Bhagat, Nikunj, Mittal, Sailesh, Ryaboy, Dmitriy·2014·Proceedings of the 2014 ACM SIGMOD International Conference on Management of Data
DOI 10.1145/2588555.2595641
Open this paper in CoChat — free account

Abstract

This paper describes the use of Storm at Twitter. Storm is a real-time fault-tolerant and distributed stream data processing system. Storm is currently being used to run various critical computations in Twitter at scale, and in real-time. This paper describes the architecture of Storm and its methods for distributed scale-out and fault-tolerance. This paper also describes how queries (aka. topologies) are executed in Storm, and presents some operational stories based on running Storm at Twitter. We also present results from an empirical evaluation demonstrating the resilience of Storm in dealing with machine failures. Storm is under active development at Twitter and we also present some potential directions for future work.

Overview

Integrity signals

Registry

Crossref · 10.1145/2588555.2595641

Cites

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Apache Spark, Apache Storm, and Apache Heron are considered heavyweight frameworks and fit less into the context of microservices.

    “Apache Spark (Zaharia et al., 2016), Apache Storm (Toshniwal et al., 2014), and the successor of the latter, Apache Heron (Kulkarni et al., 2015), are considered heavyweight frameworks and, thus, fit less into the context of microservices (Bellemare, 2020).”

    From Benchmarking scalability of stream processing frameworks deployed as microservices in the cloud (2024) · cites this paper as Toshniwal et al., 2014

Citations

Cites

No references could be resolved yet.

Research works printed but not linked: 13

  1. Arvind Arasu , Brian Babcock , Shivnath Babu , Mayur Datar , Keith Ito , Rajeev Motwani , Itaru Nishizawa , Utkarsh Srivastava , Dilys Thomas , Rohit Varma , Jennifer Widom: STREAM: The Stanford Stream Data Manager. IEEE Data Eng. Bull. 26 ( 1 ): 19 -- 26 ( 2003 ) Arvind Arasu, Brian Babcock, Shivnath Babu, Mayur Datar, Keith Ito, Rajeev Motwani, Itaru Nishizawa, Utkarsh Srivastava, Dilys Thomas, Rohit Varma, Jennifer Widom: STREAM: The Stanford Stream Data Manager. IEEE Data Eng. Bull. 26(1): 19--26 (2003)
  2. 10.1007/s00778-004-0133-5 · DOI
  3. Minos N. Garofalakis Johannes Gehrke: Querying and Mining Data Streams: You Only Get One Look. VLDB 2002 Minos N. Garofalakis Johannes Gehrke: Querying and Mining Data Streams: You Only Get One Look. VLDB 2002 · DOI
  4. Daniel J. Abadi , Yanif Ahmad , Magdalena Balazinska , Ugur Çetintemel , Mitch Cherniack , Jeong-Hyon Hwang , Wolfgang Lindner , Anurag Maskey , Alex Rasin , Esther Ryvkina , Nesime Tatbul , Ying Xing , Stanley B. Zdonik: The Design of the Borealis Stream Processing Engine. CIDR 2005: 277--289 Daniel J. Abadi, Yanif Ahmad, Magdalena Balazinska, Ugur Çetintemel, Mitch Cherniack, Jeong-Hyon Hwang, Wolfgang Lindner, Anurag Maskey, Alex Rasin, Esther Ryvkina, Nesime Tatbul, Ying Xing, Stanley B. Zdonik: The Design of the Borealis Stream Processing Engine. CIDR 2005: 277--289
  5. 10.14778/2536222.2536229 · DOI
  6. 10.1109/ICDE.2011.5767878 · DOI
  7. Sankar Subramanian , Srikanth Bellamkonda , Hua-Gang Li , Vince Liang , Lei Sheng , Wayne Smith , James Terry , Tsae-Feng Yu , Andrew Witkowski : Continuous Queries in Oracle. VLDB 2007 : 1173--1184 Sankar Subramanian, Srikanth Bellamkonda, Hua-Gang Li, Vince Liang, Lei Sheng, Wayne Smith, James Terry, Tsae-Feng Yu, Andrew Witkowski: Continuous Queries in Oracle. VLDB 2007: 1173--1184
  8. 10.14778/1454159.1454179 · DOI
  9. Jay Kreps , Neha Narkhede , and Jun Rao . Kafka : a distributed messaging system for log processing . SIGMOD Workshop on Networking Meets Databases , 2011 . Jay Kreps, Neha Narkhede, and Jun Rao. Kafka: a distributed messaging system for log processing. SIGMOD Workshop on Networking Meets Databases, 2011.
  10. Benjamin Hindman , Andy Konwinski , Matei Zaharia , Ali Ghodsi , Anthony D. Joseph , Randy Katz , Scott Shenker , and Ion Stoica . 2011 . Mesos: a platform for fine-grained resource sharing in the data center . In NSDI , 2011. Benjamin Hindman, Andy Konwinski, Matei Zaharia, Ali Ghodsi, Anthony D. Joseph, Randy Katz, Scott Shenker, and Ion Stoica. 2011. Mesos: a platform for fine-grained resource sharing in the data center. In NSDI, 2011.
  11. 10.1145/93597.98720 · DOI
  12. 10.1145/2463676.2465272 · DOI
  13. Vinod Kumar Vavilapalli , Arun C. Murthy , Chris Douglas , Sharad Agarwal , Mahadev Konar , Robert Evans , Thomas Graves , Jason Lowe , Hitesh Shah , Siddharth Seth , Bikas Saha , Carlo Curino , Owen O'Malley, Sanjay Radia, Benjamin Reed , Eric Baldeschwieler : Apache Hadoop YARN: yet another resource negotiator. SoCC 2013 : 5 Vinod Kumar Vavilapalli, Arun C. Murthy, Chris Douglas, Sharad Agarwal, Mahadev Konar, Robert Evans, Thomas Graves, Jason Lowe, Hitesh Shah, Siddharth Seth, Bikas Saha, Carlo Curino, Owen O'Malley, Sanjay Radia, Benjamin Reed, Eric Baldeschwieler: Apache Hadoop YARN: yet another resource negotiator. SoCC 2013: 5

Cited by

Sorted by citations

Metadata

Authors
Toshniwal, A., Taneja, S., Shukla, A., Ramasamy, K., Patel, J. M., Kulkarni, S., Jackson, J., Gade, K., Fu, M., Donham, J., Bhagat, N., Mittal, S., & Ryaboy, D.
Affiliations
  • Twitter (United States) (US)
  • University of Wisconsin–Madison (US)
ORCID
Patel, Jignesh M.
Venue
Proceedings of the 2014 ACM SIGMOD International Conference on Management of Data
Publisher
ACM
Published
2014-06-18
Pages
147-156
ISSN
0730-8078 (ISSN-L)
DOI
10.1145/2588555.2595641
OpenAlex
W2913854892
Cited by
956
References
0
Citations by year
2026: 21 · 2025: 30 · 2024: 30 · 2023: 48 · 2022: 49 · 2021: 98 · 2020: 92 · 2019: 122 · 2018: 130 · 2017: 147 · 2016: 122 · 2015: 62 · 2014: 4
FWCI
79.82
Type
conference paper
Language
en
Open access
closed
Licence
www.acm.org/publications/policies/copyright_policy#Background (vor)
Locations
ACM SIGMOD International Conference on Management of Data (SIGMOD) (publishedVersion)
Topics
  • Physical Sciences › Computer Science › Information Systems › Advanced Database Systems and Queries
  • Physical Sciences › Computer Science › Information Systems › Cloud Computing and Resource Management
  • Physical Sciences › Computer Science › Computer Networks and Communications › Distributed systems and fault tolerance
Keywords
Apache Storm, Twitter, distributed stream processing, fault tolerance, real-time data processing, horizontal scaling, system architecture

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review