Topic modelling for qualitative studies
The DOI matches its Crossref record; 54 of the 54 references it cites were found in a registry or an index.
Abstract
Qualitative studies, such as sociological research, opinion analysis and media studies, can benefit greatly from automated topic mining provided by topic models such as latent Dirichlet allocation (LDA). However, examples of qualitative studies that employ topic modelling as a tool are currently few and far between. In this work, we identify two important problems along the way to using topic models in qualitative studies: lack of a good quality metric that closely matches human judgement in understanding topics and the need to indicate specific subtopics that a specific qualitative study may be most interested in mining. For the first problem, we propose a new quality metric, tf-idf coherence, that reflects human judgement more accurately than regular coherence, and conduct an experiment to verify this claim. For the second problem, we propose an interval semi-supervised approach (ISLDA) where certain predefined sets of keywords (that define the topics researchers are interested in) are restricted to specific intervals of topic assignments. Our experiments show that ISLDA is better for topic extraction than LDA in terms of tf-idf coherence, number of topics identified to predefined keywords and topic stability. We also present a case study on a Russian LiveJournal dataset aimed at ethnicity discourse analysis.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
(2003) · Journal of Machine Learning Research · doi:10.5555/944919.944937 · Cited by 32,337
Advances in Neural Information Processing Systems 19
(2007) · The MIT Press eBooks · doi:10.52202/085713 · Cited by 15,539
Advances in Neural Information Processing Systems 25
Zhang, Y., Sutton, C. A., Storkey, A., & Zoubin Ghahramani, A. (2012) · Cited by 12,653
Term-weighting approaches in automatic text retrieval
Salton, G., & Buckley, C. (1988) · Information Processing & Management · doi:10.1016/0306-4573(88)90021-0 · Cited by 9,740
Advances in Neural Information Processing Systems 14: Proceedings of the 2001 Conference
(2002) · The MIT Press eBooks · doi:10.7551/mitpress/1120.001.0001 · Cited by 8,890
Griffiths, T. L., & Steyvers, M. (2004) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0307752101 · Cited by 6,049 · cited 5×
“Over recent years, topic modelling techniques based on probabilistic latent semantic analysis [1] and latent Dirichlet allocation (LDA) [2, 3] have seen growing use for various applications, including direct applications to unsupervised analysis with many extensions to additional information and…”
BackgroundMethodInfluentialProceedings of the 19th International Joint Conference on Artificial Intelligence
Pujol, J. M., Delgado, J., Ramon Sangüesa, A., & Flache, A. (2005) · International Joint Conference on Artificial Intelligence · Cited by 5,771
Blei, D. M., & Lafferty, J. D. (2006) · Proceedings of the 23rd international conference on Machine learning - ICML '06 · doi:10.1145/1143844.1143859 · Cited by 3,173
Advances in Neural Information Processing Systems 20
Bonilla, E. V., Chai, K. M. A., & Williams, C. K. I. (2008) · neural information processing systems · Cited by 2,846
Unsupervised Learning by Probabilistic Latent Semantic Analysis
Hofmann, T. (2001) · Machine Learning · doi:10.1023/a:1007617005950 · Cited by 2,456
A Simple Generalisation of the Area Under the ROC Curve for Multiple Class Classification Problems
Hand, D. J., & Till, R. J. (2001) · Machine Learning · doi:10.1023/a:1010920819831 · Cited by 2,276 · cited 1×
“AUC can be easily computed as where n pos and n neg are the number of positive and negative examples in the dataset, and P pos r i is the sum of the ranks of all positive examples [52].”
BackgroundHierarchical Dirichlet Processes
Teh, Y. W., Jordan, M. I., Beal, M. J., & Blei, D. M. (2006) · Journal of the American Statistical Association · doi:10.1198/016214506000000302 · Cited by 2,120
ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 16
Storkey, A. (2004) · neural information processing systems · Cited by 2,085
(2009) · Cited by 1,900
Proceedings of the 20th conference on Uncertainty in artificial intelligence
Meek, C., Chickering, M., & Halpern, J. Y. (2004) · Cited by 1,574
Markov Random Field Modeling in Image Analysis
Li, S. Z. (2001) · Computer Science Workbench · doi:10.1007/978-4-431-67044-5 · Cited by 1,533
The author-topic model for authors and documents
Rosen‐Zvi, M., Griffiths, T. L., Steyvers, M., & Smyth, P. (2004) · Uncertainty in Artificial Intelligence · doi:10.5555/1036843.1036902 · Cited by 1,280
Blei, D. M., & McAuliffe, J. (2010) · arXiv (Cornell University) · doi:10.48550/arxiv.1003.0783 · Cited by 1,259
Lafferty, J., & Blei, D. M. (2005) · neural information processing systems · Cited by 1,221
Topics over time: a non-Markov continuous-time model of topical trends
Wang, X., & McCallum, A. (2006) · Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1150402.1150450 · Cited by 1,121
Hierarchical Topic Models and the Nested Chinese Restaurant Process
Griffiths, T. L., Jordan, M. I., Tenenbaum, J. B., & Blei, D. M. (2003) · neural information processing systems · Cited by 1,118
Evaluation methods for topic models
Wallach, H. M., Murray, I., Salakhutdinov, R., & Mimno, D. (2009) · Proceedings of the 26th Annual International Conference on Machine Learning · doi:10.1145/1553374.1553515 · Cited by 863
Finding the Number of Clusters in a Dataset: An Information-Theoretic Approach
Sugar, C. A., & James, G. M. (2003) · Journal of the American Statistical Association · doi:10.1198/016214503000000666 · Cited by 811
Aspect and sentiment unification model for online review analysis
Jo, Y., & Oh, A. H. (2011) · Proceedings of the fourth ACM international conference on Web search and data mining · doi:10.1145/1935826.1935932 · Cited by 767
Sharing Clusters among Related Groups: Hierarchical Dirichlet Processes
Teh, Y. W., Jordan, M. I., Beal, M. J., & Blei, D. M. (2004) · UCL Discovery (University College London) · Cited by 589
AUC: a statistically consistent and more discriminating measure than accuracy
Ling, C. X., Huang, J., & Zhang, H. (2003) · Cited by 518
On Smoothing and Inference for Topic Models
Asuncion, A., Welling, M., Smyth, P., & Teh, Y. W. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1205.2662 · Cited by 434
Incorporating domain knowledge into topic modeling via Dirichlet Forest priors
Andrzejewski, D., Zhu, X., & Craven, M. (2009) · Proceedings of the 26th Annual International Conference on Machine Learning · doi:10.1145/1553374.1553378 · Cited by 420
Understanding the Limiting Factors of Topic Modeling via Posterior Contraction Analysis
Tang, J., Meng, Z., Nguyen, X., Mei, Q., & Zhang, M. (2014) · International Conference on Machine Learning · Cited by 323
Continuous Time Dynamic Topic Models
Wang, C., Blei, D. M., & Heckerman, D. E. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1206.3298 · Cited by 311
Weakly Supervised Joint Sentiment-Topic Detection from Text
Lin, C., He, Y., Everson, R., & Ruger, S. (2012) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2011.48 · Cited by 310
Learning author-topic models from text corpora
Rosen-Zvi, M., Chemudugunta, C., Griffiths, T., Smyth, P., & Steyvers, M. (2010) · ACM Transactions on Information Systems · doi:10.1145/1658377.1658381 · Cited by 302
Incorporating Lexical Priors into Topic Models
Jagarlamudi, J., Daumé, H., & Udupa, R. (2010) · Conference of the European Chapter of the Association for Computational Linguistics · Cited by 300
Spatial Latent Dirichlet Allocation
Wang, X., & Grimson, W. E. L. (2007) · neural information processing systems · Cited by 287
Latent dirichlet allocation: stability and applications to studies of user-generated content
Koltcov, S., Koltsova, O., & Nikolenko, S. (2014) · Proceedings of the 2014 ACM conference on Web science · doi:10.1145/2615569.2615680 · Cited by 280
Self-citationDetecting offensive tweets via topical feature discovery over a large scale twitter corpus
Xiang, G., Fan, B., Wang, L., Hong, J., & Rose, C. (2012) · Proceedings of the 21st ACM international conference on Information and knowledge management · doi:10.1145/2396761.2398556 · Cited by 267
Hierarchical relational models for document networks
Chang, J., & Blei, D. M. (2010) · The Annals of Applied Statistics · doi:10.1214/09-aoas309 · Cited by 255
Multi-aspect Sentiment Analysis with Topic Models
Lu, B., Ott, M., Cardie, C., & Tsou, B. K. (2011) · 2011 IEEE 11th International Conference on Data Mining Workshops · doi:10.1109/icdmw.2011.125 · Cited by 248
Sennrich, R. (2012) · The Association for Computational Linguistics · Cited by 219
Syntactic Topic ModelsPreprint
Boyd-Graber, J. L., & Blei, D. M. (2010) · arXiv (Cornell University) · doi:10.48550/arxiv.1002.4665 · Cited by 176
Detecting topic evolution in scientific literature: how can citations help?
He, Q., Chen, B., Pei, J., Qiu, B., Mitra, P., & Giles, L. (2009) · Proceedings of the 18th ACM conference on Information and knowledge management · doi:10.1145/1645953.1646076 · Cited by 170
Topic Significance Ranking of LDA Generative Models
AlSumait, L., Barbará, D., Gentle, J. E., & Domeniconi, C. (2009) · Lecture notes in computer science · doi:10.1007/978-3-642-04180-8_22 · Cited by 165
Latent Dirichlet Allocation with topic-in-set knowledge
Andrzejewski, D., & Zhu, X. (2009) · Proceedings of the NAACL HLT 2009 Workshop on Semi-Supervised Learning for Natural Language Processing - SemiSupLearn '09 · doi:10.3115/1621829.1621835 · Cited by 159
Improving Topic Coherence with Regularized Topic Models
Newman, D. J., Bonilla, E. V., & Buntine, W. (2011) · Cited by 159
Structured Topic Models for Language
Wallach, H. (2008) · Cited by 142
TweetLDA: supervised topic classification and link prediction in Twitter
Quercia, D., Askham, H., & Crowcroft, J. (2012) · ACM Web Science Conference (WebSci) · doi:10.1145/2380718.2380750 · Cited by 118
Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence (2009)Preprint
Bilmes, J., & Ng, A. Y. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1206.3959 · Cited by 113
Bayesian Checking for Topic Models
Mimno, D., & Blei, D. M. (2011) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · Cited by 104
Mapping the public agenda with topic modeling: The case of the Russian livejournal
Koltsova, O. Y., & Koltcov, S. (2013) · Policy & Internet · doi:10.1002/1944-2866.poi331 · Cited by 89
Self-citationA Joint Topic and Perspective Model for Ideological Discourse
Lin, W., Xing, E. P., & Hauptmann, A. G. (2008) · Lecture notes in computer science · doi:10.1007/978-3-540-87481-2_2 · Cited by 66
ADM-LDA: An aspect detection model based on topic modelling using the structure of review sentences
Bagheri, A., Saraee, M. H., & de Jong, F. M. G. (2014) · Journal of Information Science · doi:10.1177/0165551514538744 · Cited by 63
The contextual focused topic model
Chen, X., Zhou, M., & Carin, L. (2012) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/2339530.2339549 · Cited by 55
(2011) · Conference on Empirical Methods in Natural Language Processing (EMNLP)
Cited by
Artificial intelligence in marketing: Topic modeling, scientometric analysis, and research agenda
Mustak, M., Salminen, J., Plé, L., & Wirtz, J. (2021) · Journal of Business Research · doi:10.1016/j.jbusres.2020.10.044 · Cited by 555
Mapping the anti-vaccination movement on Facebook
Smith, N., & Graham, T. (2019) · Information, Communication & Society · doi:10.1080/1369118x.2017.1418406 · Cited by 351
Rodríguez-Espíndola, O., Chowdhury, S., Beltagui, A., & Albores, P. (2020) · International Journal of Production Research · doi:10.1080/00207543.2020.1761565 · Cited by 308
Shaik, T., Tao, X., Li, Y., Dann, C., McDonald, J., Redmond, P., & Galligan, L. (2022) · IEEE Access · doi:10.1109/access.2022.3177752 · Cited by 286
Lampropoulos, G., Keramopoulos, E., Diamantaras, K., & Evangelidis, G. (2022) · Education Sciences · doi:10.3390/educsci12110798 · Cited by 119
Metadata
- Authors
- Nikolenko, S. I., Koltcov, S., & Koltsova, O.
- Venue
- Journal of Information Science
- Publisher
- SAGE Publications
- Published
- 2016-07-10
- Volume
- 43
- Issue
- 1
- Pages
- 88-102
- ISSN
- 0165-5515 (ISSN-L)
- 1741-6485
- OpenAlex
- W2341256577
- Cited by
- 332
- References
- 54
- FWCI
- 52.32
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.