The DOI matches its Crossref record; 41 of the 41 references it cites were found in a registry or an index.
An evaluation of document clustering and topic modelling in two online social networks: Twitter and Reddit
Abstract
Abstract Methods for document clustering and topic modelling in online social networks (OSNs) offer a means of categorising, annotating and making sense of large volumes of user generated content. Many techniques have been developed over the years, ranging from text mining and clustering methods to latent topic models and neural embedding approaches. However, many of these methods deliver poor results when applied to OSN data as such text is notoriously short and noisy, and often results are not comparable across studies. In this study we evaluate several techniques for document clustering and topic modelling on three datasets from Twitter and Reddit. We benchmark four different feature representations derived from term-frequency inverse-document-frequency (tf-idf) matrices and word embedding models combined with four clustering methods, and we include a Latent Dirichlet Allocation topic model for comparison. Several different evaluation measures are used in the literature, so we provide a discussion and recommendation for the most appropriate extrinsic measures for this task. We also demonstrate the performance of the methods over data sets with different document lengths. Our results show that clustering techniques applied to neural embedding feature representations delivered the best performance over all data sets using appropriate extrinsic evaluation measures. We also demonstrate a method for interpreting the clusters with a top-words based approach using tf-idf weights combined with embedding distance measures.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
(2003) · Journal of Machine Learning Research · doi:10.5555/944919.944937 · Cited by 32,337
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568
Deep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983
Proceedings of the 25th international conference on Machine learning
Cohen, W. W., McCallum, A., & Roweis, S. T. (2008) · Cited by 5,691
Software Framework for Topic Modelling with Large Corpora
Rek, R. R. U., & Sojka, P. (2010) · doi:10.13140/2.1.2393.1847 · Cited by 3,647
A neural probabilistic language model
BengioYoshua, DucharmeRéjean, VincentPascal, & JanvinChristian (2003) · Journal of Machine Learning Research · doi:10.5555/944919.944966 · Cited by 2,644
Vinh, N. X., Epps, J., & Bailey, J. (2010) · Minerva Access (University of Melbourne) · doi:10.5555/1756006.1953024 · Cited by 1,781
Empirical study of topic modeling in Twitter
Hong, L., & Davison, B. D. (2010) · Proceedings of the First Workshop on Social Media Analytics · doi:10.1145/1964858.1964870 · Cited by 1,120
Information diffusion in online social networks: a survey
Guille, A., Hacid, H., Favre, C., & Zighed, D. A. (2013) · ACM SIGMOD Record · doi:10.1145/2503792.2503797 · Cited by 851
Social media analytics – Challenges in topic discovery, data collection, and data preparation
Stieglitz, S., Mirbabaie, M., Ross, B., & Neuberger, C. (2018) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2017.12.002 · Cited by 803
An Empirical Evaluation of doc2vec with Practical Insights into Document Embedding Generation
Lau, J. H., & Baldwin, T. (2016) · Proceedings of the 1st Workshop on Representation Learning for NLP · doi:10.18653/v1/w16-1609 · Cited by 632
A Survey of Clustering With Deep Learning: From the Perspective of Network Architecture
Min, E., Guo, X., Liu, Q., Zhang, G., Cui, J., & Long, J. (2018) · IEEE Access · doi:10.1109/access.2018.2855437 · Cited by 553
Multi-layer Representation Learning for Medical Concepts
Choi, E., Bahadori, M. T., Searles, E., Coffey, C., Thompson, M., Bost, J., Tejedor-Sojo, J., & Sun, J. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939823 · Cited by 469
A Survey of Topic Modeling in Text Mining
Alghamdi, R., & Alfalqi, K. (2015) · International Journal of Advanced Computer Science and Applications · doi:10.14569/ijacsa.2015.060121 · Cited by 377
Proceedings of the Americas Conference on Information Systems
Eseryel, Y. (2014) · Americas Conference on Information Systems · Cited by 345
Discovering Health Topics in Social Media Using Topic Models
Paul, M. J., & Dredze, M. (2014) · PLoS ONE · doi:10.1371/journal.pone.0103408 · Cited by 269
Adjusting for Chance Clustering Comparison Measures
Romano, S., Vinh, N. X., Bailey, J., & Verspoor, K. M. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1512.01286 · Cited by 192
Godin, F., Vandersmissen, B., De Neve, W., & Van de Walle, R. (2015) · Proceedings of the Workshop on Noisy User-generated Text · doi:10.18653/v1/w15-4322 · Cited by 188
The Role of Social Networks in Information DiffusionPreprint
Bakshy, E., Rosenn, I., Marlow, C. A., & Adamic, L. A. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1201.4145 · Cited by 181
Tweet2Vec: Character-Based Distributed Representations for Social Media
Dhingra, B., Zhou, Z., Fitzpatrick, D., Muehl, M., & Cohen, W. (2016) · Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) · doi:10.18653/v1/p16-2044 · Cited by 176
Using word embeddings in Twitter election classification
Yang, X., Macdonald, C., & Ounis, I. (2017) · Information Retrieval · doi:10.1007/s10791-017-9319-5 · Cited by 154
Proceedings of the First Workshop on Social Media Analytics
(2010) · doi:10.1145/1964858 · Cited by 108
Topic Modeling Reveals Distinct Interests within an Online Conspiracy Forum
Klein, C., Clutton, P., & Polito, V. (2018) · Frontiers in Psychology · doi:10.3389/fpsyg.2018.00189 · Cited by 76
Twitter Topic Modeling by Tweet Aggregation
Steinskog, A. O., Therkelsen, J. F., & Gambäck, B. (2017) · DSpace repository (University of Tartu) · Cited by 72
Detecting hot topics from Twitter: A multiview approach
Fang, Y., Zhang, H., Ye, Y., & Li, X. (2014) · Journal of Information Science · doi:10.1177/0165551514541614 · Cited by 63
Clustering memes in social media
Ferrara, E., JafariAsbagh, M., Varol, O., Qazvinian, V., & Flammini, R. (2013) · Cited by 60
Examining user perceptions of smartwatch through dynamic topic modeling
Ha, T., Beijnon, B., Kim, S., et al. (2017) · Telematics and Informatics · doi:10.1016/j.tele.2017.05.011 · Cited by 46
Proceedings of the 1st Workshop on Representation Learning for NLP
(2016) · doi:10.18653/v1/w16-16 · Cited by 44
Data Sets: Word Embeddings Learned from Tweets and General Data
Li, Q., Shah, S., Liu, X., & Nourbakhsh, A. (2017) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v11i1.14859 · Cited by 44
Latent LSTM Allocation: Joint clustering and non-linear dynamic modeling of sequence data
Zaheer, M., Ahmed, A. A., & Smola, A. J. (2017) · International Conference on Machine Learning · Cited by 33
Clustering memes in social media streamsPreprint
JafariAsbagh, M., Ferrara, E., Varol, O., & Flammini, A. (2016) · Cited by 28
Clustering memes in social media streams
JafariAsbagh, M., Ferrara, E., Varol, O., Menczer, F., & Flammini, A. (2014) · Social Network Analysis and Mining · doi:10.1007/s13278-014-0237-x · Cited by 23
An Overview of Topic Discovery in Twitter Communication through Social Media Analytics
Chinnov, A., Kerschke, P., Meske, C., Stieglitz, S., & Trautmann, H. (2015) · Journal of the Association for Information Systems · Cited by 19
A graph-based approach to ememes identification and tracking in Social Media streams
Shabunina, E., & Pasi, G. (2017) · Knowledge-Based Systems · doi:10.1016/j.knosys.2017.10.013 · Cited by 19
Zhao, J. B., Lan, M., & Tian, J. F. (2015) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/s15-2021 · Cited by 18
Proceedings of the Workshop on Noisy User-generated Text
(2015) · doi:10.18653/v1/w15-43 · Cited by 18
Cited by
Sentiment analysis on product reviews based on weighted word embeddings and deep neural networks
Onan, A. (2021) · Concurrency and Computation: Practice and Experience · doi:10.1002/cpe.5909 · Cited by 396
Shaik, T., Tao, X., Li, Y., Dann, C., McDonald, J., Redmond, P., & Galligan, L. (2022) · IEEE Access · doi:10.1109/access.2022.3177752 · Cited by 286
Grybauskas, A., Stefanini, A., & Ghobakhloo, M. (2022) · Technology in Society · doi:10.1016/j.techsoc.2022.101997 · Cited by 271
Onan, A. (2019) · IEEE Access · doi:10.1109/access.2019.2945911 · Cited by 257
Metadata
- Authors
- Curiskis, S. A., Drake, B., Osborn, T. R., & Kennedy, P. J.
- Venue
- Information Processing & Management
- Publisher
- Elsevier BV
- Published
- 2020-03
- Volume
- 57
- Issue
- 2
- Pages
- 102034
- ISSN
- 0020-0271 (ISSN-L)
- 0306-4573
- OpenAlex
- W2938824800
- Cited by
- 304
- References
- 41
- FWCI
- 16.81
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.