An annotation scheme for discourse-level argumentation in research articles
Open accessAbstract
In order to build robust automatic abstracting systems, there is a need for better training resources than are currently available. In this paper, we introduce an annotation scheme for scientific articles which can be used to build such a resource in a consistent way. The seven categories of the scheme are based on rhetorical moves of argumentation. Our experimental results show that the scheme is stable, reproducible and intuitive to use.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Foundational efforts deconstructed the peer review process into analytical, predictive, and narrowly generative components.
“These foundational efforts deconstructed the peer review process into analytical (Teufel et al., 1999), predictive (Kang et al., 2018), and narrowly generative components (Dycke et al., 2023b).”
From Can AI Be a Good Peer Reviewer? A Survey of Peer Review Process, Evaluation, and the Future (2026) · cites this paper as Teufel et al., 1999
Section 2 describes an analytical scheme that labels the argumentative roles of sentences in research articles, so the citation supports the analytical component. The Conclusions describe its purpose as improving article summaries, not deconstructing the peer review process; that framing is unsupported by this source. The supplied passages give no page numbers. (LLM, conf=0.86)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
No references could be resolved yet.
Research works printed but not linked: 19
- Jan Alexandersson, Elisabeth Mater, and Norbert Re- ithinger. 1995. A robust and efficient three-layered dialogue component for a speech-to-speech transla- tion system. In Proceedings of the Seventh Euro- pean Meeting of the ACL, pages 188-193.
- Ronald Brandow, Karl Mitze, and Lisa F. Rau. 1995. Automatic condensation of electronic publications by sentence selection. Information Processing and Management, 31(5):675-685.
- Jean Carletta, Amy Isard, Stephen Isard, Jacque- line C. Kowtko, Gwyneth Doherty-Sneddon, and Anne H. Anderson. 1997. The reliability of a dia- logue structure coding scheme. Computational Lin- guistics, 23(1):13-31.
- Jean Carletta. 1996. Assessing agreement on classi- fication tasks: the kappa statistic. Computational Linguistics, 22(2):249-254.
- Robin Cohen. 1984. A computational theory of the function of clue words in argument understanding. In Proceedings of COLING-8~, pages 251-255.
- Anna Duszak. 1994. Academic discourse and intellec- tual styles. Journal of Pragmatics, 21:291-313.
- Joost G. Kircz. 1991. The rhetorical structure of sci- entific articles: the case for argumentational analy- sis in information retrieval. Journal of Documenta- tion, 47(4):354-372.
- Julian Kupiec, Jan O. Pedersen, and Francine Chen. 1995. A trainable document summarizer. In Pro- ceedings of the 18th ACM-SIGIR Conference, Asso- ciation for Computing Machinery, Special Interest Group Information Retrieval, pages 68-73.
- Elizabeth DuRoss Liddy. 1991. The discourse-level structure of empirical abstracts: an exploratory study. Information Processing and Management, 27(1):55-81.
- William C. Mann and Sandra A. Thompson. 1987. Rhetorical structure theory: description and con- struction of text structures. In G. Kempen, edi- tor, Natural Language Generation: New Results in Artificial Intelligence, Psychology and Linguistics, pages 85-95, Dordrecht. Nijhoff.
- Daniel Marcu. 1997. From discourse structures to text summaries. In Inderjeet Mani and Mark T. Maybury, editors, Proceedings of the workshop on Intelligent Scalable Text Summarization, in associ- ation with A CL//BA CL-97.
- Greg Myers. 1992. In this paper we report... -speech acts and scientific facts. Journal of Pragmatics, 17(4):295-313.
- G.J Rath, A. Resnick, and T. R. Savage. 1961. The formation of abstracts by the selection of sentences. American Documentation, 12(2):139-143.
- Lisa F. Rau, Paul S. Jacobs, and Uri Zernik. 1989. Information extraction and text processing using linguistic knowledge acquisition. Information Pro- cessing and Management, 25(4):419-428.
- Sidney Siegel and N.J. Jr. Castellan. 1988. Non- parametric statistics for the Behavioral Sciences. McGraw-Hill, second edition edition.
- Karen Sp~ixck Jones. 1998. Automatic summarising: factors and directions. In ACL/EACL-97 Work- shop 'Intelligent Scalable Text Summarization'.
- John Swales. 1990. Genre analysis: English in aca- demic and research settings. Cambridge University Press.
- Geoff Thompson and Yiyun Ye. 1991. Evaluation in the reporting verbs used in academic papers. Ap- plied Linguistics, 12(4):365-382.
- Sheryl R. Young and Phillip J. Hayes. 1985. Auto- matic classification and summarization of banking telexes. In Proceedings of the Second Conference on Artificial Intelligence Applications.
Cited by
Sorted by citations
Biomedical text mining for research rigor and integrity: tasks, challenges, directions
Kilicoglu, H. (2017) · Briefings in Bioinformatics · doi:10.1093/bib/bbx057 · Cited by 61 · see citing works
Can AI Be a Good Peer Reviewer? A Survey of Peer Review Process, Evaluation, and the Future
Wu, S., Jiang, O., Zhao, Y., Hu, T., Ma, Y., Zhang, K., Patwardhan, M., & Cohan, A. (2026) · Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2026.acl-long.1504 · Cited by 1 · see citing works
- Authors
- Teufel, S., Carletta, J., & Moens, M.
- Affiliations
- University of Edinburgh (GB)
- ORCID
- Teufel, Simone
- Venue
- Proceedings of the ninth conference on European chapter of the Association for Computational Linguistics -
- Published
- 1999
- Pages
- 110
- OpenAlex
- W2028473264
- Cited by
- 186
- References
- 0
- Citations by year
- 2026: 3 · 2025: 3 · 2024: 2 · 2023: 6 · 2022: 11 · 2021: 9 · 2020: 9 · 2019: 16 · 2018: 12 · 2017: 6 · 2016: 3 · 2015: 5 · 2014: 10 · 2013: 14 · 2012: 11
- FWCI
- 1.38
- Type
- conference paper
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Conference of the European Chapter of the Association for Computational Linguistics (publishedVersion)
- OLAC - Open Language Archives Community (submittedVersion)
- Americanae (AECID Library) (submittedVersion)
- Funders
- Engineering and Physical Sciences Research Council
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Advanced Text Analysis Techniques
- Social Sciences › Arts and Humanities › Literature and Literary Theory › Discourse Analysis in Language Studies
- Social Sciences › Arts and Humanities › Philosophy › Linguistics and Discourse Analysis
- Keywords
- annotation scheme, rhetorical moves, automatic abstracting, training materials, argumentation structure
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
