The DOI matches its Crossref record; 26 of the 26 references it cites were found in a registry or an index.
The utility of topic modelling for discourse studies: A critical evaluation
Open accessAbstract
This article explores and critically evaluates the potential contribution to discourse studies of topic modelling, a group of machine learning methods which have been used with the aim of automatically discovering thematic information in large collections of texts. We critically evaluate the utility of the thematic grouping of texts into ‘topics’ emerging from a large collection of online patient comments about the National Health Service (NHS) in England. We take two approaches to this, one inspired by methods adopted in existing topic modelling research and the other using more established methods of discourse analysis. In the study, we compare the insights produced by each approach and consider the extent to which the automatically generated topics might be of use to discourse analysts attempting to organise and study sizeable datasets. We found that the topic modelling approach was able to group texts into ‘topics’ that were truly thematically coherent with a mixed degree of success, while the more traditional approach to discourse analysis consistently provided a more nuanced perspective on the data which was ultimately closer to the ‘reality’ of the texts it contains. This study thus highlights issues concerning the use of topic modelling and offers recommendations and caveats to researchers employing such approaches to studying discourse in the future.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“recognising the limitations of existing technologies for conducting such analysis within a broader corpus linguistic methodology (Brookes & McEnery, 2019; Gillings et al., 2023)”
From Generative AI for corpus approaches to discourse studies: A critical evaluation of ChatGPT (2024) · cites this paper as Brookes & McEnery, 2019
Citations
Cites
(2003) · Journal of Machine Learning Research · doi:10.5555/944919.944937 · Cited by 32,337
Fairclough, N. L. (2003) · Routledge eBooks · doi:10.4324/9780203697078 · Cited by 10,756
Blei, D. M. (2012) · Communications of the ACM · doi:10.1145/2133806.2133826 · Cited by 5,516
Narrative Analysis: Oral Versions of Personal Experience1
Labov, W., & Waletzky, J. (1997) · Journal of Narrative and Life History · doi:10.1075/jnlh.7.02nar · Cited by 3,328
Burling, R. (1975) · Language · doi:10.2307/412879 · Cited by 3,005
Using Corpora in Discourse Analysis
Baker, P. (2006) · Continuum eBooks · doi:10.5040/9781350933996 · Cited by 2,040
Baker, P., Gabrielatos, C., KhosraviNik, M., Krzyżanowski, M., McEnery, T., & Wodak, R. (2008) · Discourse & Society · doi:10.1177/0957926508088962 · Cited by 1,936
Self-citationComparing Twitter and Traditional Media Using Topic Models
Zhao, W. X., Jiang, J., Weng, J., He, J., Lim, E.-P., Yan, H., & Li, X. (2011) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-20161-5_34 · Cited by 1,214
van Valin, R. D., & LaPolla, R. J. (1997) · Cambridge University Press eBooks · doi:10.1017/cbo9781139166799 · Cited by 1,207
Text and Context: Explorations in the Semantics and Pragmatics of Discourse
van Dijk, T. A. (1977) · Medical Entomology and Zoology · Cited by 934
Research works printed but not linked: 5
- Blei D, 2009, Text Mining: Classification, Clustering, and Applications, 71
- Blei D, 2003, Journal of Machine Learning Research, 3, 993
- Jaworska S, 2016, Applied Linguistics, 39, 373
- Labov W, 1972, Language in the Inner City: Studies in the Black English Vernacular
- Mazarura J, 2014, Proceedings of the 2014 PRASA, RobMech and AfLaT International Joint Symposium
Cited by
Sorted by citations
Hase, V., Mahl, D., Schäfer, M. S., & Keller, T. R. (2021) · Global Environmental Change · doi:10.1016/j.gloenvcha.2021.102353 · Cited by 342
Corpus-Assisted Discourse Studies
Gillings, M., Mautner, G., & Baker, P. (2023) · doi:10.1017/9781009168144 · Cited by 146
Generative AI for corpus approaches to discourse studies: A critical evaluation of ChatGPT
Curry, N., Baker, P., & Brookes, G. (2024) · Applied Corpus Linguistics · doi:10.1016/j.acorp.2023.100082 · Cited by 102
Self-citationFrom Big Data to Rich Theory: Integrating Critical Discourse Analysis with Structural Topic Modeling
Aranda, A. M., Sele, K., Etchanchu, H., Guyt, J. Y., & Vaara, E. (2021) · European Management Review · doi:10.1111/emre.12452 · Cited by 89
Improving the quality of teaching by utilising written student feedback: A streamlined process
Hujala, M., Knutas, A., Hynninen, T., & Arminen, H. (2020) · Computers & Education · doi:10.1016/j.compedu.2020.103965 · Cited by 72
Gender Bias in the News: A Scalable Topic Modelling and Visualization Framework
Rao, P. P., & Taboada, M. (2021) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2021.664737 · Cited by 49
Gillings, M., & Hardie, A. (2022) · Digital Scholarship in the Humanities · doi:10.1093/llc/fqac075 · Cited by 49
Kang, H., & Kim, J. (2022) · Applied Sciences · doi:10.3390/app12115614 · Cited by 55
Topic modelling the news media representation of climate change
Rabitz, F., Telešienė, A., & Zolubienė, E. (2020) · Environmental Sociology · doi:10.1080/23251042.2020.1866281 · Cited by 35
Low-carbon economy and policy implications: a systematic review and bibliometric analysis
Wang, J., Zhou, Y., & Cooke, F. L. (2022) · Environmental Science and Pollution Research · doi:10.1007/s11356-022-20381-0 · Cited by 34
Palaute: An Online Text Mining Tool for Analyzing Written Student Course Feedback
Grönberg, N., Knutas, A., Hynninen, T., & Hujala, M. (2021) · IEEE Access · doi:10.1109/access.2021.3116425 · Cited by 35
Recovering the divide: A review of the big data analytics—strategy relationship
Talaoui, Y., Kohtamäki, M., Ranta, M., & Paroutis, S. (2023) · Long range planning · doi:10.1016/j.lrp.2022.102290 · Cited by 27
Eisele, O., Heidenreich, T., Litvyak, O., & Boomgaarden, H. G. (2023) · Communication Methods and Measures · doi:10.1080/19312458.2023.2230560 · Cited by 32
Feminism in women’s business networks: A freedom-centred perspective
Villesèche, F., Meliou, E., & Jha, H. K. (2022) · Human Relations · doi:10.1177/00187267221083665 · Cited by 32
Grundmann, R. (2021) · Journal of Risk Research · doi:10.1080/13669877.2021.1894471 · Cited by 31
Nagwani, N. K., & Suri, J. S. (2023) · International Journal of Information Management Data Insights · doi:10.1016/j.jjimei.2022.100153 · Cited by 37
Carter, P., Gee, M., McIlhone, H., Lally, H., & Lawson, R. (2021) · Methods in Psychology · doi:10.1016/j.metip.2021.100065 · Cited by 28
Multiple Correspondence Analysis, newspaper discourse and subregister
Clarke, I., McEnery, T., & Brookes, G. (2021) · Register Studies · doi:10.1075/rs.20024.cla · Cited by 24
Self-citationInterpersonal Function of Joe Biden’s Victory Speech (Systemic Functional Linguistics View)
Darong, H. C. (2021) · Journal of Education Research and Evaluation · doi:10.23887/jere.v5i1.31420 · Cited by 22
Topic Modeling of the Pakistani Economy in English Newspapers via Latent Dirichlet Allocation (LDA)
Ahmed, F., Nawaz, M. S., & Jadoon, A. (2022) · SAGE Open · doi:10.1177/21582440221079931 · Cited by 22
Tillery, D., & Bloomfield, E. F. (2021) · Technical Communication Quarterly · doi:10.1080/10572252.2021.2019317 · Cited by 21
International Organizations and Research Methods
Badache, F., Kimber, L. R., & Maertens, L. (2023) · University of Michigan Press eBooks · doi:10.3998/mpub.11685289 · Cited by 51
Hryniewicz, K., & Grzegorczyk, T. (2020) · PLoS ONE · doi:10.1371/journal.pone.0238714 · Cited by 21
Topic modelling in corpus-based discourse analysis: Uses and critiques
Bednarek, M. A. (2024) · Discourse Studies · doi:10.1177/14614456241293075 · Cited by 18
Allen, W., & Easton‐Calabria, E. (2022) · International Studies Quarterly · doi:10.1093/isq/sqac044 · Cited by 17
Malkawi, R., Daradkeh, M. K., Elhassan, A., & Petrov, P. (2022) · Information · doi:10.3390/info13110546 · Cited by 17
Networks in the mind – what communities reveal about the structure of the lexicon
Kovács, L., Bóta, A., Hajdú, L., & Krész, M. (2021) · Open Linguistics · doi:10.1515/opli-2021-0012 · Cited by 16
A novel, human-in-the-loop computational grounded theory framework for big social data
Alqazlan, L., Fang, Z., Castelle, M., & Procter, R. (2025) · Big Data & Society · doi:10.1177/20539517251347598 · Cited by 17
China’s digital diplomacy on Twitter: The multiple reactions to the Belt and Road Initiative
Seoane, M. F. V. (2023) · Global Media and Communication · doi:10.1177/17427665231185697 · Cited by 18
Gillings, M., Learmonth, M., & Mautner, G. (2024) · British Journal of Management · doi:10.1111/1467-8551.12816 · Cited by 14
Comparative Discourse Analysis Using Topic Models: Contrasting Perspectives on China from Reddit
Stine, Z. K., & Agarwal, N. (2020) · International Conference on Social Media and Society (SMSociety) · doi:10.1145/3400806.3400816 · Cited by 14
In authority, or peers we trust? Reviews and recommendations in social commerce
Dinulescu, C. C., & Prybutok, V. R. (2021) · Behaviour and Information Technology · doi:10.1080/0144929x.2021.1957016 · Cited by 17
Bocu, R., Băicoianu, A., & Kerestely, A. (2023) · IEEE Access · doi:10.1109/access.2023.3329732 · Cited by 12
McAllister, L., Vedula, S., Pu, W., & Boykoff, M. (2024) · Environmental Research Letters · doi:10.1088/1748-9326/ad22b7 · Cited by 14
Understanding climate-related visual storytelling on TikTok
Zeng, J., & Yan, X. (2024) · Journal of Digital Social Research · doi:10.33621/jdsr.v6i2.212 · Cited by 12
Singh, M., Saini, M., Adebayo, S. O., Singh, J., & Kaur, M. (2022) · Education and Information Technologies · doi:10.1007/s10639-022-11494-7 · Cited by 12
Framing environmental news in Bangladesh
Al-Zaman, M. S., & Khan, T. (2021) · Media Asia · doi:10.1080/01296612.2021.1997526 · Cited by 12
Hirsbrunner, S. D. (2024) · Wiley Interdisciplinary Reviews Climate Change · doi:10.1002/wcc.902 · Cited by 15
Bencekri, M., Kwak, J., Lee, D., Csiba-Herczeg, Á., & Lee, S. (2025) · International Journal of Urban Sciences · doi:10.1080/12265934.2024.2445240 · Cited by 11
Egorova, E. (2021) · Journal of Spatial Information Science · doi:10.5311/josis.2021.23.157 · Cited by 11
Oldac, Y. İ., & Olivos, F. (2025) · Review of Education · doi:10.1002/rev3.70021 · Cited by 10
Machine Learning practices and infrastructures
Berman, G. (2023) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3600211.3604689 · Cited by 10
Media Representations of Healthcare Robotics in Norway 2000-2020: A Topic Modeling Approach
Solberg, M., & Kirchhoff, R. (2023) · Social Science Computer Review · doi:10.1177/08944393231212251 · Cited by 10
Aranda, A. M., Vaara, E., Etchanchu, H., & Guyt, J. Y. (2024) · Organization Science · doi:10.1287/orsc.2022.16145 · Cited by 11
How humans and machines identify discourse topics: A methodological triangulation
Gillings, M., & Jaworska, S. (2025) · Applied Corpus Linguistics · doi:10.1016/j.acorp.2025.100121 · Cited by 9
Mostafa, M. M., Alhur, M., & Moustafa, A. M. (2025) · International Journal of Information Management Data Insights · doi:10.1016/j.jjimei.2025.100364 · Cited by 8
Operation Heron: latent topic changes in an abusive letter series
Busso, L., Petykó, M., Atkins, S., & Grant, T. (2022) · Corpora · doi:10.3366/cor.2022.0255 · Cited by 8
A tale of heroes and villains: Russia’s strategic narratives on Twitter during the COVID-19 pandemic
Moral, P. (2023) · Journal of Information Technology & Politics · doi:10.1080/19331681.2023.2182862 · Cited by 13
REGISTER CATEGORIES (FIELD, TENOR, MODE) OF THE TEXT
Darong, H. C. (2022) · LLT Journal A Journal on Language and Language Teaching · doi:10.24071/llt.v25i2.4724 · Cited by 7
Papaspyropoulos, K. G., Liakou, H., & Dimopoulos, P. (2023) · Sustainability · doi:10.3390/su15076127 · Cited by 7
Busso, L., Combei, C. R., & Tordini, O. (2020) · doi:10.30687/978-88-6969-478-3/002 · Cited by 7
Brands, networks, communities: How brand names are wired in the mind
Kovács, L., Bóta, A., Hajdú, L., & Krész, M. (2022) · PLoS ONE · doi:10.1371/journal.pone.0273192 · Cited by 7
Moral, P., & Marco, G. (2023) · Communication Research and Practice · doi:10.1080/22041451.2023.2188699 · Cited by 7
Are we listening to every word? Using multiple analytic methods to examine qualitative data
Boyd, R. L., Morrison, N. R., Horwitz, S. D., Maciag, R., Travers‐Hill, E., & Kim, Y.-S. (2024) · Cogent Mental Health · doi:10.1080/28324765.2024.2433791 · Cited by 6
Health emergencies, science contrarianism and populism: A scoping review
Schenkel, M. D. A. (2024) · Social Science & Medicine · doi:10.1016/j.socscimed.2024.116691 · Cited by 6
Pang, N., Sun, M., & Zhu, H. (2024) · Digital Scholarship in the Humanities · doi:10.1093/llc/fqae062 · Cited by 6
Hegemony, Discourse, and Political Strategy
Jacobs, T. (2022) · Discourse approaches to politics, society and culture · doi:10.1075/dapsac.97 · Cited by 6
Shayegh, J., Sumantry, D. M., Jagayat, A., & Choma, B. L. (2023) · British Journal of Social Psychology · doi:10.1111/bjso.12707 · Cited by 6
Discursive Quads: New Kinds of Lexical Co‐occurrence Data With Linguistic Concept Modelling
Mehl, S. (2022) · Transactions of the Philological Society · doi:10.1111/1467-968x.12251 · Cited by 6
Decoding paradoxical identities: The discourse construction of left-behind children in news reports
Liang, L., Wang, H., & Zhang, W. (2024) · Discourse & Communication · doi:10.1177/17504813241239895 · Cited by 5
Digital policy narratives: addressing grand challenges or exacerbating digital inequalities?
Zamani, E. D., & Vannini, S. (2025) · Information Technology for Development · doi:10.1080/02681102.2025.2525348 · Cited by 6
Silva, J. A. M. D. C., Rossi, R. G., & Lobato, F. M. F. (2019) · doi:10.5753/eniac.2019.9348 · Cited by 5
Ajšić, A. (2021) · Corpora · doi:10.3366/cor.2021.0209 · Cited by 5
Darong, H. C. (2021) · Indonesian Journal of EFL and Linguistics · doi:10.21462/ijefl.v6i1.337 · Cited by 5
Mapping the concept of health care integration: A lexicographic analysis of scientific literature
Urionagüena, A., Piquer‐Martinez, C., Benrimoj, S. I., Calvo, B., García‐Cárdenas, V., Gastelurrutia, M. Á., Martínez, F. M., & Fernández-Llimós, F. (2024) · Research in Social and Administrative Pharmacy · doi:10.1016/j.sapharm.2024.01.013 · Cited by 4
The Journal of Law and Society in context: a bibliometric analysis
Boulanger, C., Creutzfeldt, N., & Hendry, J. (2024) · Journal of Law and Society · doi:10.1111/jols.12465 · Cited by 4
Uses of Corpus Linguistics in Higher Education Research: An Adjustable Lens
Alsop, S., King, V. H., Giaimo, G. N., & Xu, X. (2020) · Emerald Group Publishing Limited eBooks · doi:10.1108/s2056-375220200000006003 · Cited by 4
Chen, F., & Wang, G. (2023) · Journal of Language and Politics · doi:10.1075/jlp.21038.che · Cited by 6
Latent Semantic Analysis based Real-world Application of Topic Modeling: A Review Study
Gupta, I., Chatterjee, I., & Gupta, N. (2022) · International Conference on Artificial Intelligence and Smart Energy (ICAIS) · doi:10.1109/icais53314.2022.9742848 · Cited by 4
Policy silences in Nordic-Arctic energy transitions: A topic-modelling study
Vikström, H., Mårald, E., & Junkka, J. (2025) · Energy Policy · doi:10.1016/j.enpol.2025.114746 · Cited by 3
Christensen, R. V. K., & Bentsen, N. S. (2023) · AMBIO · doi:10.1007/s13280-023-01964-8 · Cited by 3
Gwaka, L. (2025) · Energy Research & Social Science · doi:10.1016/j.erss.2025.104019 · Cited by 3
AI Augmented Qualitative Analysis: Is it the Right Choice for Your Study?
Hitch, D., Richards, K., Knight, R., Pépin, G., Thanekar, U., & Goldingay, S. (2025) · CABI eBooks · doi:10.1079/9781800626607.0006 · Cited by 3
Hatami, Y., Zali, N., & Barati, N. (2025) · International Journal of Urban Sciences · doi:10.1080/12265934.2025.2504674 · Cited by 3
Evans, C., & Lorenzo-Dus, N. (2025) · Applied Corpus Linguistics · doi:10.1016/j.acorp.2025.100147 · Cited by 3
Zhang, Z. (2021) · Journal of Multilingual and Multicultural Development · doi:10.1080/01434632.2021.1938087 · Cited by 3
Zhang, Z. (2021) · HERMES - Journal of Language and Communication in Business · doi:10.7146/hjlcb.vi61.127928 · Cited by 3
Rizakis, M. (2023) · French Politics · doi:10.1057/s41253-023-00205-4 · Cited by 3
The Impact of YouTube on Present and Future Firm Value: Using Unstructured Text Analysis
Na, H. J., Kim, Y. H., & Jo, H. J. (2023) · Sustainability · doi:10.3390/su15054346 · Cited by 3
Slavery and Britain in the 19th century
McEnery, T., Baker, H., & Březina, V. (2021) · Studies in corpus linguistics · doi:10.1075/scl.101.02mce · Cited by 3
Self-citationTopic modeling for short texts: comparative analysis of algorithms
Vashchenko, V. (2024) · Sociology methodology methods mathematical modeling (Sociology 4M) · doi:10.19181/4m.2023.32.1.2 · Cited by 2
An evaluation of the use of keyword analysis for keyword-assisted topic modelling
Batchelor, J., & Lee-Laminack, H. (2025) · Corpora · doi:10.3366/cor.2025.0337 · Cited by 2
It is all about clauses: Speech analysis using systemic functional linguistics theory
Darong, H. C. (2022) · Englisia Journal of language education and humanities · doi:10.22373/ej.v10i1.13029 · Cited by 2
The representations of freedom in The Sun newspaper between 2019 and 2021: A corpus-based study
Leone, L. (2023) · Topics in Linguistics · doi:10.2478/topling-2023-0003 · Cited by 2
Akar, N., & Yörük, T. (2024) · Heliyon · doi:10.1016/j.heliyon.2024.e34908 · Cited by 2
Sík, D., Rakovics, M., & Németh, R. (2024) · Journal of Technology in Behavioral Science · doi:10.1007/s41347-024-00440-8 · Cited by 3
Yao, X., McCosker, A., & Kang, Y. (2025) · Big Data & Society · doi:10.1177/20539517241311585 · Cited by 2
Pepin, J. R., & Scarborough, W. J. (2025) · American Journal of Sociology · doi:10.1086/735618 · Cited by 2
Social Group Representation in a Diachronic News Corpus
Elmerot, I. (2025) · Cambridge University Press eBooks · doi:10.1017/9781009029063 · Cited by 2
Christ, A., Röschlein, J., & Schindler, C. (2025) · European Educational Research Journal · doi:10.1177/14749041251405570 · Cited by 2
Chakriswaran, P., Vincent, P. M. D. R., & Kadry, S. (2022) · International Journal of Uncertainty Fuzziness and Knowledge-Based Systems · doi:10.1142/s0218488522400207 · Cited by 2
El debate urbano en Latinoamérica: pandemia, opinión pública y redes digitales de información
Pichihua, Y. M. P., & Sánchez, J. R. (2022) · Revista INVI · doi:10.5354/0718-8358.2022.65726 · Cited by 2
Drafting of Online Meeting Minutes Based on Video Recording Using Topic Modelling
Arianto, R., Abdullah, A., Nurhasan, U., & Wakhidah, R. (2023) · SISFORMA · doi:10.24167/sisforma.v10i1.5037 · Cited by 2
Mirocha, P. (2023) · Studia Linguistica Universitatis Iagellonicae Cracoviensis · doi:10.4467/20834624sl.23.013.18635 · Cited by 2
Building a Dataset Search for Institutions: Project Update
Mannheimer, S., Clark, J. A., Espeland, J., & Hagerman, K. (2019) · Publications · doi:10.3390/publications7020029 · Cited by 2
Rodrigues, L., Baségio, A., & Lobato, F. M. F. (2019) · doi:10.5753/eniac.2019.9336 · Cited by 2
Letters to the editor, institutional experimentation, and the public accounting professional
Everett, J., Rahaman, A. S., Neu, D., & Saxton, G. D. (2024) · Critical Perspectives on Accounting · doi:10.1016/j.cpa.2024.102725 · Cited by 3
Chun, J. W., Lee, A. R., & Chung, Y. J. (2024) · SAGE Open · doi:10.1177/21582440241286971 · Cited by 1
Ohoilulin, A., Inan, D. I., Juita, R., & Sanglise, M. (2024) · Building of Informatics Technology and Science (BITS) · doi:10.47065/bits.v6i2.5729 · Cited by 1
Roberts, S. G., Krykoniuk, K., Handford, M., Zhou, Y., Wu, J., & Chen, C. (2025) · Energy Research & Social Science · doi:10.1016/j.erss.2025.104275 · Cited by 1
Metadata
- Authors
- Brookes, G., & McEnery, T.
- Affiliations
- Lancaster University (GB)
- Corresponding author
- Brookes, Gavin
- Venue
- Discourse Studies
- Publisher
- SAGE Publications
- Published
- 2018-12-21
- Volume
- 21
- Issue
- 1
- Pages
- 3-21
- ISSN
- 1461-4456 (ISSN-L)
- 1461-7080
- OpenAlex
- W2883000515
- Cited by
- 175
- References
- 26
- Citations by year
- 2026: 22 · 2025: 32 · 2024: 25 · 2023: 28 · 2022: 32 · 2021: 20 · 2020: 10 · 2019: 5 · 2018: 1
- FWCI
- 33.30
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- Discourse Studies (publishedVersion, cc-by-nc)
- Funders
- Economic and Social Research Council (ES/R008906/1, ES/K002155/1)
- Topics
- Social Sciences › General Social Sciences › Computational and Text Analysis Methods
- Social Sciences › Linguistics and Language › Discourse Analysis in Language Studies
- Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
- Keywords
- topic modeling, discourse analysis, National Health Service, thematic coherence, large text corpora
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review