The DOI matches its Crossref record; 63 of the 65 references it cites were found in a registry or an index.
Bursty and hierarchical structure in streams
Abstract
A fundamental problem in text data mining is to extract meaningful structure from document streams that arrive continuously over time. E-mail and news articles are two natural examples of such streams, each characterized by topics that appear, grow in intensity for a period of time, and then fade away. The published literature in a particular research field can be seen to exhibit similar phenomena over a much longer time scale. Underlying much of the text mining work in this area is the following intuitive premise --- that the appearance of a topic in a document stream is signaled by a "burst of activity," with certain features rising sharply in frequency as the topic emerges.The goal of the present work is to develop a formal approach for modeling such "bursts," in such a way that they can be robustly and efficiently identified, and can provide an organizational framework for analyzing the underlying content. The approach is based on modeling the stream using an infinite-state automaton, in which bursts appear naturally as state transitions; in some ways, it can be viewed as drawing an analogy with models from queueing theory for bursty network traffic. The resulting algorithms are highly efficient, and yield a nested representation of the set of bursts that imposes a hierarchical structure on the overall stream. Experiments with e-mail and research paper archives suggest that the resulting structures have a natural meaning in terms of the content that gave rise to them.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
A tutorial on hidden Markov models and selected applications in speech recognition
Rabiner, L. R. (1989) · Proceedings of the IEEE · doi:10.1109/5.18626 · Cited by 22,743
Nearest neighbor pattern classification
Cover, T., & Hart, P. (1967) · IEEE Transactions on Information Theory · doi:10.1109/tit.1967.1053964 · Cited by 16,936
Agrawal, R., & Srikant, R. (2002) · Proceedings of the Eleventh International Conference on Data Engineering · doi:10.1109/icde.1995.380415 · Cited by 5,155
The Analysis of Time Series: An Introduction
Dickey, D. A., & Chatfield, C. (1991) · Technometrics · doi:10.2307/1268794 · Cited by 3,525
Story and Discourse: Narrative Structure in Fiction and Film
Austin, T. R., & Chatman, S. (1979) · The Journal of Aesthetics and Art Criticism · doi:10.2307/430732 · Cited by 2,658
Hand, D. J., Mannila, H., & Smyth, P. (2001) · Cited by 2,581
Attention, intentions, and the structure of discourse
Grosz, B. J., & Sidner, C. L. (1986) · doi:10.5555/12457.12458 · Cited by 2,580
Agents that reduce work and information overload
Maes, P. (1994) · Communications of the ACM · doi:10.1145/176789.176792 · Cited by 2,321
The Analysis of Time Series: An Introduction with R
Chatfield, C., & Xing, H. (2019) · Chapman & Hall/CRC eBooks · doi:10.1201/9781351259446 · Cited by 1,803
Principles of mixed-initiative user interfaces
Horvitz, E. (1999) · Proceedings of the SIGCHI conference on Human factors in computing systems the CHI is the limit - CHI '99 · doi:10.1145/302979.303030 · Cited by 1,490
Research works printed but not linked: 21
- Agrawal R., 1995, Proc. Intl. Conf. on Data Engineering
- Allan J., 1998, DARPA Broadcast News Transcription and Understanding Workshop
- T. Blanton Ed. White House E-mail New Press 1995. T. Blanton Ed. White House E-mail New Press 1995.
- Chatman S., 1978, Cornell Univ. Press
- Cohen W., 1996, AAAI Spring Symp. Machine Learning and Information Access
- Genette G., 1980, Cornell Univ. Press
- G. Genette Narrative Discourse Revisited English translation (J.E. Lewin) Cornell Univ. Press 1988. G. Genette Narrative Discourse Revisited English translation (J.E. Lewin) Cornell Univ. Press 1988.
- Han J., 1998, Proc. Intl. Conf. Knowledge Discovery and Data Mining
- 10.5555/857190.857680 · doi:10.5555/857190.857680
- Kelly F. P., 1996, Press
- Keogh E., 1997, Intl. Conf. Knowledge Discovery and Data Mining
- Lavrenko V., 2000, KDD-2000 Workshop on Text Mining
- Mannila H., 1995, Intl. Conf. on Knowledge Discovery and Data Mining
- Martin R., 2001, Temporal Data Mining
- Miller N., 1998, IEEE Visualization
- Redmond M., 1998, AAAI Workshop on Case-Based Reasoning
- Rennie J., 2000, KDD Workshop on Text Mining
- Sahami M., 1998, AAAI Workshop on Learning for Text Categorization
- B. Schneier Applied Cryptography Wiley 1996. B. Schneier Applied Cryptography Wiley 1996.
- Segal R., 2000, Proc. Intl. Conf. on Machine Learning
- Wong P., 2000, IEEE Information Visualization
Cited by
Items 1,101–1,200 of 1,342
Mining Evolutionary Event Patterns of Web Texts for Text Resource
Zhao, C., & Yi, D. (2007)
Bdiri, T. (2015) · Spectrum Research Repository (Concordia University)
The Effectiveness of Time Sequence Information on a Sightseeing Spot Recommender
Shimada, K., Uehara, H., & Endo, T. (2015) · Intelligent systems reference library · doi:10.1007/978-3-662-47227-9_10
Topical analysis of text streams
He, Q. (2009) · DR-NTU (Nanyang Technological University) · doi:10.32657/10356/17764
Summarizing Results of Keyword Search on Social Photos Using Clustering-Based Burst Detection
Sakai, T., & Tamura, K. (2015) · IIAI International Congress on Advanced Applied Informatics (IIAI-AAI) · doi:10.1109/iiai-aai.2015.241
Online Cross-Lingual PLSI for Evolutionary Theme Patterns Analysis
Xin, X., Zhuang, K., Fang, Y., & Huang, H. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-37453-1_7
Learning Timeline Difference for Text Categorization
Fukumoto, F., & Suzuki, Y. (2015) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d15-1093
Change Point Detection for Information Diffusion Tree
Ohara, K., Saito, K., Kimura, M., & Motoda, H. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24282-8_14
Research Intelligence Involving Information Retrieval
杜逸寧 (2009) · 基礎研究
Identifying Event Context Using Anchor Information in Online Social Networks
Gu, H., Gartrell, M., Zhang, L., & Lv, Q. (2013) · CU Scholar (University of Colorado Boulder)
Predicting Latent Trends of Labels in the Social Media Using Infectious Capacity
Sonoda, Y., & Daisuke, I. (2015) · International Journal of Future Computer and Communication · doi:10.18178/ijfcc.2015.4.6.419
Detecting Bursty Topics of Correlated News and Twitter for Government Services
Utsuro, T., Inoue, Y., Imada, T., Yoshioka, M., & Kando, N. (2015) · Springer eBooks · doi:10.1007/978-3-319-27237-5_7
Large-scale multilingual document clustering(本文)
岸田, 和. (2013) · Institutional Repositories DataBase (IRDB)
Judging Consistency and Expertise of Blogs
Meinel, C., Broß, J., Berger, P., & Hennig, P. (2015) · Springer eBooks · doi:10.1007/978-3-662-44409-2_16
Precision-Integrated Scalable Monitoring
Jain, N. (2008) · Texas ScholarWorks (Texas Digital Library)
A Self-Adaptive Spike Detection Algorithm with Application in Performance and Capacity Management
Sahai, S., Natu, M., & Jain, M. K. (2009) · Conference on Management of Data
Application of search engine focusing on trend-related queries to market of data
Kori, S., Yamaguchi, K., Zhu, Y., Takiguchi, S., Kato, Y., Ishikawa, H., & Takama, Y. (2016) · Intelligent Decision Technologies · doi:10.3233/idt-150250
Harnessing online social media to deal with information overload.
Li, C. (2013) · DR-NTU (Nanyang Technological University) · doi:10.32657/10356/54827
Quadri, C. (2015) · Archivio Istituzionale della Ricerca (Universita Degli Studi Di Milano) · doi:10.13130/quadri-christian_phd2015-03-13
Real-time Event Analysis in Online Social Networks
Gu, H. (2013) · CU Scholar (University of Colorado Boulder)
Smoothing Temporal Difference for Text Categorization
Fukumoto, F., & Suzuki, Y. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-28940-3_16
Impact Analysis of the Person in Topic Event Mining.
Li, F., Zheng, D., & Zhao, T. (2014) · International Conference on Intelligent Information Processing
A study of feature exraction techniques for classifying topics and sentiments from news posts
Al-Dyani, W. Z. (2014) · UUM Electronic Theses and Dissertation [eTheses] (Northern University of Malaysia)
Mining Temporal Evolution of Criminal Behaviors.
Pio, G., Ceci, M., & Malerba, D. (2012) · CINECA IRIS Institutional Research Information System (University of Bari Aldo Moro)
Keywords Popularity Analysis Based on Hidden Markov Model
Xue, L., Wang, Z., Zhang, W., & Zhang, H. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-31854-7_43
Exploring Social Relationships in Text Streams
Wang, Y. (2016) · ICST Transactions on Scalable Information Systems · doi:10.4108/eai.9-8-2016.151631
Analysis of Data on the Internet IV – Search Trend Analysis over Time
Nettleton, D. (2014) · Elsevier eBooks · doi:10.1016/b978-0-12-416602-8.00017-0
Hopcroft, J. E. (2007) · Proceedings of the International Congress of Mathematicians Madrid, August 22–30, 2006 · doi:10.4171/022-1/5
Filtering News from Document Streams: Evaluation Aspects and Modeled Stream Utility
Baruah, G. M. (2016) · UWSpace (University of Waterloo)
Location-Based Temporal Burst Detection Using Outlier Factors in Geo-Tagged Tweets
Tamura, K., Sakai, T., & Kitakami, H. (2016) · IIAI International Congress on Advanced Applied Informatics (IIAI-AAI) · doi:10.1109/iiai-aai.2016.155
Tracking onscreen gender and role bias over time
Radford, W., & Gallé, M. (2016) · Forschungszentrum L3S · doi:10.34962/jws-23
Kimmey, D. (2016) · Purdue e-Pubs (Purdue University System)
Bjørkelund, E., & Burnett, T. H. (2012) · BIBSYS Brage (BIBSYS (Norway))
Active and interactive learning strategies for error-tolerant graph matching
Llosa, X. C. (2016) · Rovira & Virgili repository
The role of context in image annotation and recommendation
McParlane, P. J. (2016) · Enlighten: Theses (The University of Glasgow)
Trend Query Analysis on Heterogeneous Web Resources
Yoshida, M., Arase, Y., & Arase, Y. (2016) · Institutional Repositories DataBase (IRDB)
Efficient Emergency Event Tracking Using Features of Web Data
Wu, Q., Lv, J., Wang, H., & Ma, S. (2015) · Cybernetics and Information Technologies · doi:10.1515/cait-2015-0042
Data mining in computational proteomics and genomics
Song, Y. (2015)
Burst Detection in Social Media Streams for Tracking Interest Profiles in Real Time.
Buntain, C., & Lin, J. (2015) · Text REtrieval Conference
Multiple Uses of Frequent Episodes in Temporal Process Modeling
Patnaik, D. (2011) · VTechWorks (Virginia Tech)
DEVELOPMENT OF METHOD OF REAL-TIME KEYWORDS FOR COLLECTING TRAFFIC PHENOMENA
NAKAMURA, K., Tanaka, S., FUJIMOTO, Y., & TOYAMA, A. (2016) · Journal of Japan Society of Civil Engineers Ser F3 (Civil Engineering Informatics) · doi:10.2208/jscejcei.72.i_182
Sleep Pattern Characterization via Cluster Analysis of Audio Data
Wu, H., Ken-ichi, F., Kato, T., & Numao, M. (2015)
Bursty event discovery from online news outlets
Kochak, S. (2015)
Discovering bursts revisited: guaranteed optimization of the model parameters
Tatti, N. (2017) · Society for Industrial and Applied Mathematics eBooks · doi:10.1137/1.9781611974973.93
Analyzing the Role of Users' Interactions in Information Spread
Eftekhar, M. (2015) · TSpace (University of Toronto)
Mining and Managing User-Generated Content and Preferences
Valkanas, G. (2014) · EduLLL - Greek Educational Content Repository (EKT)
Ikegami, T., Oka, M., & Hashimoto, Y. (2015) · Transactions of the Japanese Society for Artificial Intelligence · doi:10.1527/tjsai.c-mdf03
IIRank: Co-Ranking Scientific Articles by Characterizing their Innovativeness and Importance
Li, F., Zhang, Z., Li, L., Xu, X., & He, S. (2017) · Chinese Conference on Computer Supported Cooperative Work and Social Computing (ChineseCSCW) · doi:10.1145/3127404.3127459
Mapping Intellectual Structures and Dynamics of Detecting Research Fronts Domain
Wang, J., & Li, H. (2017) · International Journal of Economics Finance and Management Sciences · doi:10.11648/j.ijefm.20170505.16
Customer State Analysis with Enthusiasm Analysis
Yanagimoto, H. (2018) · Studies in computational intelligence · doi:10.1007/978-3-319-70636-8_7
Exploiting sparsity for machine learning in big data
Zhu, R. (2017) · Illinois Digital Environment for Access to Learning and Scholarship (University of Illinois at Urbana-Champaign)
Ma, M. (2018) · CUNY Academic Works (City University of New York)
Hybrid Entity Driven News Detection on Twitter
Vikre, L., Wold, H. M., Özgöbek, Ö., & Gulla, J. A. (2017) · doi:10.17562/3760
He, W., Yu, X. M., & Sun, M. (2018) · Portland International Conference on Management of Engineering and Technology (PICMET) · doi:10.23919/picmet.2018.8481872
Hot topic extraction based on situation awareness and aging theory
Si, X.-M., & Liu, Y. (2011) · Energy Procedia · doi:10.1016/j.egypro.2011.11.478
Kifer, D. (2018) · Springer eBooks · doi:10.1007/978-1-4614-8265-9_49
Traffic Event Related Blog Post Classification by Using Traffic Related Named Entities
Unsal, A. D., Tüydeş-Yaman, H., & Karagöz, P. (2018) · IEEE International Smart Cities Conference (ISC2) · doi:10.1109/isc2.2018.8656940
Literature review on �Value Stream Mapping� using bibliometric tool
Inamdar, Z., Narwane, V. S., Raut, R. D., & Narkhede, B. E. (2019) · doi:10.26488/iej.12.3.1170
Temporal models in data mining : enrichment, summarization and recommendation
Tran, A. T. (2018) · Institutional Repository of Leibniz Universität Hannover (Leibniz Universität Hannover) · doi:10.15488/3574
Temporal Methods to Detect Content-Based Anomalies in Social Media
Skryzalin, J., Field, R. V., Fisher, A., & Bauer, T. (2019) · Lecture notes in social networks · doi:10.1007/978-3-030-11286-8_10
Profiling user activities with minimal traffic tracesPreprint
Mai, T., Ajwani, D., & Sala, A. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1504.01781
Measuring scientific buzzPreprint
Vasan, K., & West, J. D. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.03249
Global and Local Bursts Detection in Streaming Data
Khaing, P. P., & Aung, T. N. (2017) · International Conference on Applied Computing and Information Technology / International Conference on Computational Science and Intelligence (ACIT-CSI) · doi:10.1109/acit-csii-bcd.2017.19
Topic Estimation Method Using Base Selection
Wajima, K., Furukawa, T., & Satoh, T. (2019) · Joho Chishiki Gakkaishi · doi:10.2964/jsik_2019_036
Research Concerning Topic Extraction Method Using Burst Levels in Microblogs
TOYAMA, A., NAKAMURA, K., & Tanaka, S. (2020) · Journal of Japan Society for Fuzzy Theory and Intelligent Informatics · doi:10.3156/jsoft.32.1_570
Giglioli, I. A. C. (2020) · RiuNet (Universitat Politècnica de València) · doi:10.4995/thesis/10251/139075
Knowledge Maps of Tourism Big Data Research in China Based on Visualization Analysis
Jie, L. (2019) · Annual Meeting on Management Engineering (AMME) · doi:10.1145/3377672.3378049
Hexmoor, H. H. (2014) · Elsevier eBooks · doi:10.1016/b978-0-12-800891-1.00005-6
Ding, Y., Cao, Y., & Duffy, V. G. (2020) · Advances in intelligent systems and computing · doi:10.1007/978-3-030-51549-2_64
Marlowe, F. W. (2019) · University of California Press eBooks · doi:10.1525/9780520945449-006
Estimating Deflation Representing People Spreading in Stream Data
Toyoshima, T., Endo, M., KIKUCHI, T., Ohno, S., & Ishikawa, H. (2020) · International Conference on Web Intelligence, Mining and Semantics · doi:10.1145/3405962.3405992
Stochastic Sampling and Machine Learning Techniques for Social Media State Production
Bidoki, N. H. (2020) · Journal of International Crisis and Risk Communication Research
Reviewing and Predicting Human-Machine Cooperation Based on Knowledge Graph Analysis
Liu, Y. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-60114-0_12
Clustering information entities based on statistical methods
Fisichella, M. (2012) · Institutional Repository of Leibniz Universität Hannover (Leibniz Universität Hannover) · doi:10.15488/7846
INFORMATION AND SOCIAL SYSTEM INTERACTION
Haque, K. (2011)
A Study on Personalized Influencer Estimation and Bookmarking Forecast Based on Social Data Analysis
荒澤, 孔. (2021) · Institutional Repositories DataBase (IRDB) · doi:10.15118/00010389
Georgescu, M. (2015) · Institutional Repository of Leibniz Universität Hannover (Leibniz Universität Hannover) · doi:10.15488/8457
Kifer, D. (2009) · Springer eBooks · doi:10.1007/978-0-387-39940-9_49
Mapping the Alzheimer's disease clinical trial space
Schultz, T., Handly, N., & Chen, C. (2016) · iDEA - Drexel University Institutional Repository · doi:10.17918/etd-7096
Chen, C. (2011) · Springer eBooks · doi:10.1007/978-3-642-19160-2_5
Trend Extraction Method using Co-occurrence Patterns from Tweets
Fujita, K., & Noda, S. (2016) · Information Engineering Express · doi:10.52731/iee.v2.i4.133
The Retrieval of Important News Stories by Influence Propagation among Communities and Categories
Lin, Y.-F., & Kao, H. (2012) · IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology (WI-IAT) · doi:10.1109/wi-iat.2012.236
Development of a Simulation Model for Optimization of Business Process: Focus on Gap Processes
(2018) · Advances in logistics, operations, and management science book series · doi:10.4018/978-1-5225-5442-4.ch001
(2014) · Elsevier eBooks · doi:10.1016/b978-0-12-407171-1.00019-8
A Visualized Knowledge Map of Enterprise Social Capital in the Context of Incubation Networks
Ding, K., & Li, H. (2022) · International Conference on Information System and Data Mining (ICISDM) · doi:10.1145/3546157.3546174
Ho, S. Y.-C., Chien, T., Huang, C., & Tsai, K. (2022) · doi:10.2196/preprints.42695
Papadimitriou, S. (2018) · Springer eBooks · doi:10.1007/978-1-4614-8265-9_18
Temporal Methods to Detect Content-Based Anomalies in Social Media
Skryzalin, J. (2018) · OSTI OAI (U.S. Department of Energy Office of Scientific and Technical Information)
Is the Research on AI Empowered Pedagogy in China Decaying?
Wang, T., & Chen, C. (2022) · International Conference on Intelligent Education and Intelligent Research (IEIR) · doi:10.1109/ieir56323.2022.10050069
Management Perspective of “Green Strategies” Research—Scientometric Analysis
Pfeifer, S. (2023) · Springer proceedings in business and economics · doi:10.1007/978-3-031-23844-4_32
Sun, B., Hai, C., & You, Z. (2023) · International Conference on Big Data and Information Analytics (BigDIA) · doi:10.1109/bigdia60676.2023.10429253
Osiński, Z. (2023) · Zagadnienia Informacji Naukowej - Studia Informacyjne · doi:10.36702/zin.925
Hao, Z., Sun, S., Leng, R., & Xu, X. (2024) · Communications in computer and information science · doi:10.1007/978-981-97-1332-5_25
Tamai, S., Numao, M., & Fukui, K. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2404.10299
Guo, W., & Xu, N. (2023) · International Conference on Big Data, Artificial Intelligence and Risk Management (ICBAR) · doi:10.1145/3656766.3656942
Metadata
- Authors
- Kleinberg, J.
- Affiliations
- Cornell University (US)
- Corresponding author
- Kleinberg, Jon
- ORCID
- Kleinberg, Jon
- Venue
- Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining
- Publisher
- ACM
- Published
- 2002-07-23
- Pages
- 91-101
- OpenAlex
- W2026302857
- Cited by
- 1,337
- References
- 65
- Citations by year
- 2026: 101 · 2025: 68 · 2024: 63 · 2023: 77 · 2022: 43 · 2021: 44 · 2020: 45 · 2019: 41 · 2018: 58 · 2017: 61 · 2016: 59 · 2015: 92 · 2014: 73 · 2013: 81 · 2012: 70
- FWCI
- 19.27
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) (publishedVersion)
- eCommons (Cornell University) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Advanced Text Analysis Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Social Sciences › General Social Sciences › Computational and Text Analysis Methods
- Keywords
- document streams, burst detection, hierarchy, state transitions, email, topic emergence
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.