Cited by 175Computational and Text Analysis MethodsDiscourse Analysis in Language StudiesSentiment Analysis and Opinion Mining

The utility of topic modelling for discourse studies: a critical evaluation

Open access
Brookes, Gavin, McEnery, Tony·2019·Discourse Studies

Abstract

This article explores and critically evaluates the potential contribution to discourse studies of topic modelling, a group of machine learning methods which have been used with the aim of automatically discovering thematic information in large collections of texts. We critically evaluate the utility of the thematic grouping of texts into ‘topics’ emerging from a large collection of online patient comments about the National Health Service (NHS) in England. We take two approaches to this, one inspired by methods adopted in existing topic modelling research and the other using more established methods of discourse analysis. In the study, we compare the insights produced by each approach and consider the extent to which the automatically generated topics might be of use to discourse analysts attempting to organise and study sizeable datasets. We found that the topic modelling approach was able to group texts into ‘topics’ that were truly thematically coherent with a mixed degree of success, while the more traditional approach to discourse analysis consistently provided a more nuanced perspective on the data which was ultimately closer to the ‘reality’ of the texts it contains. This study thus highlights issues concerning the use of topic modelling and offers recommendations and caveats to researchers employing such approaches to studying discourse in the future.

Integrity signals

Registry

Crossref · 10.1177/1461445618814032

Cites

26 of 26 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Existing technologies have limitations for conducting automated qualitative analysis within corpus approaches to discourse studies.

    “recognising the limitations of existing technologies for conducting such analysis within a broader corpus linguistic methodology (Brookes & McEnery, 2019; Gillings et al., 2023)”

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Authors
Brookes, G., & McEnery, T.
Affiliations
Lancaster University (GB)
Corresponding author
Brookes, Gavin
Venue
Discourse Studies
Published
2018-12-21
Volume
21
Issue
1
Pages
3-21
ISSN
  • 1461-4456 (ISSN-L)
  • 1461-7080
OpenAlex
W2883000515
Cited by
175
References
26
Citations by year
2026: 22 · 2025: 32 · 2024: 25 · 2023: 28 · 2022: 32 · 2021: 20 · 2020: 10 · 2019: 5 · 2018: 1
FWCI
33.30
Type
journal article
Language
en
Open access
Locations
Discourse Studies (publishedVersion, cc-by-nc)
Funders
Economic and Social Research Council (ES/R008906/1, ES/K002155/1)
Topics
  • Social Sciences › General Social Sciences › Computational and Text Analysis Methods
  • Social Sciences › Arts and Humanities › Literature and Literary Theory › Discourse Analysis in Language Studies
  • Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
Keywords
topic modeling, discourse analysis, National Health Service, thematic coherence, large text corpora

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review