Metadiscourse Patterns in Human-Written vs. Generative AI-Authored Research Abstracts: A Comparative Corpus-Based Analysis
Open accessThe DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index; 3 of the 8 sources its claims cite were checked for support.
Abstract
Comparative corpus analysis of metadiscourse in human-written and AI-authored research abstracts.
Overview
CoChat Summary
TL;DR
This corpus-based study compares metadiscourse in human-written and ChatGPT-generated research abstracts, finding that AI abstracts use more interactive/text-organizing markers but fewer interactional stance and engagement markers. The author argues AI can mimic formal structure but lacks the nuanced authorial presence and reader engagement of human academic prose.
Key findings
- ChatGPT-generated abstracts had a higher total metadiscourse density than human abstracts, by about 15% (p<0.05), with significantly more interactive markers such as transitions and frame markers (p<0.01).
- Human abstracts used significantly more interactional markers: hedges appeared twice as often as in AI abstracts (p<0.001), and boosters and attitude markers were also more frequent in human texts.
- Engagement and self-mention were near-absent in AI abstracts: self-mentions like “we conducted” appeared in >50% of human abstracts but <10% of AI abstracts.
- Statistical tests showed significant differences for hedges/boosters (p<0.01, Cramér’s V ≈0.3) and frame markers (p<0.05, V≈0.2); no category was significantly more frequent in the human corpus.
Methods
- The study compiled a parallel corpus of human-written abstracts (N≈100, from 2018–2025 open-access journals across disciplines) and ChatGPT-generated abstracts (GPT-4, accessed April 2025) using the same titles and journal contexts; abstract lengths were comparable (mean ≈250 words).
- Metadiscourse was annotated using Hyland’s (2005) interpersonal framework, coding interactive and interactional devices; inter-annotator agreement was Cohen’s κ >0.80.
- Counts were normalized per 1,000 words and compared with chi-square tests, with effect sizes such as Cramér’s V; qualitative discourse analysis of exemplar abstracts complemented the quantitative results.
Limitations
- The paper states it used one AI model (ChatGPT/GPT-4) and a fixed prompting strategy, so different models or prompts might yield different styles.
- It states the study focused only on abstracts, a concise and formulaic genre, and used a moderate corpus of about 100 abstracts per group, limiting finer discipline-specific claims.
- The paper also notes that Hyland’s taxonomy may not capture all discourse nuance and that excerpts were not subjected to formal reading tests; evident from the text, the disciplinary comparisons are described as preliminary and limited by sample size.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
3 Valid · 1 Not valid · 5 Not checked · 42 Own results and interpretations
Every source the paper cites for this claim was checked and supports it.
In the paper
“Indeed, ChatGPT abstracts were noted by human reviewers as 'vague' or 'formulaic' , a qualitative impression that aligns with our finding of scant stance markers.”
What the sources say
- Supportedcites Gao et al. (2023) source
The supplied passage labeled Abstract reports that reviewers found abstracts they suspected were ChatGPT-generated to be “vaguer and more formulaic.” This supports the cited qualitative impression; no page number is provided in our copy. (LLM, conf=0.94)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“They favor past-tense passive or third-person phrasing.”
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Boosters (e.g. 'clearly,' 'confirm') and attitude markers (e.g. 'surprisingly,' 'critically') were also more frequent in human texts.”
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on 2 assumptions not shown here.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Engagement markers (e.g. first-person 'we,' secondperson 'readers' or rhetorical questions) were near-absent in ChatGPT abstracts: AI texts rarely used 'we' or 'our findings,' instead favoring passive constructs.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on 2 assumptions not shown here.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In contrast, human abstracts relied slightly more on evidentials (citation-like expressions) and code glosses ('that is,' 'in other words') to relate ideas.”
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on 2 assumptions not shown here.
11 more claims
Create a free account to see all 51 claims and to verify the 1 reference still open.
- Probably not a substantial claim. 26
- It may state two separate claims as one. 21
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated major: it would touch a result — a value a result rests on, reported and recomputed p-values on opposite sides of the significance level, two passages that conflict with no sign which is right, or a cited work retracted before this paper was published.
In the paper
§ Quantitative Findings · p. 4
“Hedge markers (e.g. 'suggests,' 'may,' 'possible') appeared twice as often in human abstracts as in AI abstracts (χ² test, p<0.001).”
§ Quantitative Findings · p. 5
“No category was significantly more frequent in the human corpus; even self-mentions in humans, while higher, did not reach statistical threshold given low counts.”
Citations
Cites
Research Genres: Explorations and Applications
Swales, J. M. (2004) · Cambridge University Press eBooks · doi:10.1017/cbo9781139524827 · Cited by 1,442
As can be seen: Lexical bundles and disciplinary variation
Hyland, K. (2008) · English for Specific Purposes · doi:10.1016/j.esp.2007.06.001 · Cited by 1,250
Metadiscourse in Academic Writing: A Reappraisal
Hyland, K. (2004) · Applied Linguistics · doi:10.1093/applin/25.2.156 · Cited by 1,241
Gao, C. A., Howard, F. M., Markov, N. S., Dyer, E. C., Ramesh, S., Luo, Y., & Pearson, A. T. (2023) · npj Digital Medicine · doi:10.1038/s41746-023-00819-6 · Cited by 1,099 · cited 1×
“Gao et al. (2023) found that ChatGPT can produce plausible scientific abstracts that humans often find difficult to distinguish from originals, though reviewers noted that AI abstracts tended to be “vaguer and more formulaic”.”
Humble servants of the discipline? Self-mention in research articles
Hyland, K. (2001) · English for Specific Purposes · doi:10.1016/s0889-4906(00)00012-0 · Cited by 775
Disciplinary interactions: metadiscourse in L2 postgraduate writing
Hyland, K. (2004) · Journal of Second Language Writing · doi:10.1016/j.jslw.2004.02.001 · Cited by 687
When combinations of humans and AI are useful: A systematic review and meta-analysis
Vaccaro, M., Almaatouq, A., & Malone, T. (2024) · Nature Human Behaviour · doi:10.1038/s41562-024-02024-1 · Cited by 570
Metadiscourse: What is it and where is it going?
Hyland, K. (2017) · Journal of Pragmatics · doi:10.1016/j.pragma.2017.03.007 · Cited by 455
Persuasion and context: The pragmatics of academic metadiscourse
Hyland, K. (1998) · Journal of Pragmatics · doi:10.1016/s0378-2166(98)00009-5 · Cited by 422 · cited 1×
“We compared patterns in AI vs. human texts, guided by principles from discourse pragmatics (Hyland, 1998; Camiciottoli, 2004) and stancetaking (Hyland & Jiang, 2016).”
Hu, G., & Cao, F. (2011) · Journal of Pragmatics · doi:10.1016/j.pragma.2011.04.007 · Cited by 408
Research works printed but not linked: 1
- Brezina, V. (2018). Statistics in corpus linguistics: A practical guide. Cambridge University Press.
Cited by
Three Stance Corridors: Hedging, Engagement, and Task-Flex in AI-Generated Academic WritingPreprint
Curfman, E. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7195350
Metadata
- Authors
- Sadigzade, Z.
- ORCID
- Sadigzade, Zarifa
- Venue
- Acta Globalis Humanitatis et Linguarum
- Published
- 2026-02-04
- Volume
- 3
- Issue
- 1
- Pages
- 44-54
- ISSN
- 3030-1718 (ISSN-L)
- OpenAlex
- W7127391628
- Cited by
- 1
- References
- 31
- Citations by year
- 2026: 1
- FWCI
- 36.37
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Acta Globalis Humanitatis et Linguarum (publishedVersion, cc-by-nc)
- Topics
- Social Sciences › Linguistics and Language › Discourse Analysis in Language Studies
- Social Sciences › Linguistics and Language › Linguistics and Discourse Analysis
- Social Sciences › Library and Information Sciences › Academic Writing and Publishing
- Keywords
- metadiscourse, AI-generated text, ChatGPT, scientific abstracts, stance markers, transition, academic writing, authorial voice, reader engagement
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Metadiscourse Patterns in Human-Written vs. Generative AI-Authored Research Abstracts: A Comparative Corpus-Based Analysis. https://app.cochat.ai/papers/10.69760/aghel.026001005
- The paper
- Sadigzade, Z. (2026). Metadiscourse Patterns in Human-Written vs. Generative AI-Authored Research Abstracts: A Comparative Corpus-Based Analysis. Acta Globalis Humanitatis et Linguarum. https://doi.org/10.69760/aghel.026001005
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.