CoChat
Sign Up
Cited by 286Artificial Intelligence in EducationEFL/ESL Teaching and LearningEnglish Language Learning and Teaching

AI in the Foreign Language Classroom: A Pedagogical Overview of Automated Writing Assistance Tools

Open access
Alharbi, Wael·2023·Education Research International
DOI 10.1155/2023/4253331
Open this paper in CoChat — free accountOpen PDF

Abstract

Recent technological advances in artificial intelligence (AI) have paved the way for improved and in many cases the creation of entirely new and innovative, electronic writing tools. These writing support systems assist during and after the writing process making them indispensable to many writers in general and to students in particular who can get human-like sentence completion suggestions and text generation. Although the wide adoption of these tools by students has been faced with a steady growth of scientific publications in the field, the results of these studies are often contradictory and their validity may be questioned. To gain a deeper understanding of the validity of AI-powered writing assistance tools, we conducted a systematic review of the recent empirical AI-powered writing assistance studies. The purpose of this review is twofold. First, we wanted to explore the recent scholarly publications that evaluated the use of AI-powered writing assistance tools in the classroom in terms of their types, uses, limits, and potential for improving students’ writing skills. Second, the review also sought to explore the perceptions of educators and researchers about learners’ use of AI-powered writing tools and review their recommendations on how to best ingrate these tools into the contemporary and future classroom. Using the Scopus research database, a total of 104 peer-reviewed papers were identified and analyzed. The findings indicate that students are increasingly using a variety of AI-powered writing assistance tools for improving their writing. The tools they are using can be categorized into four main groups: (1) automated writing evaluation tools, (2) tools that provide automated writing corrective feedback, (3) AI-powered machine translators, and (4) GPT-3 automatic text generators. The analysis also highlighted the scholars’ recommendations regarding dealing with learners’ use of AI-powered writing assistance tools and grouped the recommendations into two groups for researchers and educators.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1155/2023/4253331

Cites

15 of 15 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Since the initial reaction against ChatGPT, an acceptance that the use of GenAI is an inescapable reality has predominated in higher education.

    “Since this initial burst of reactionism, however, an acceptance that the use of GenAI is an inescapable reality has predominated (Alharbi, 2023; Mills et al., 2023).”

    From Artificial Intelligence for Academic Purposes (AIAP): Integrating AI literacy into an EAP module (2025) · cites this paper as Alharbi, 2023

    The Introduction says language educators may need to accept AI writing tools as an inescapable fact, and §3.2 reports that several researchers advocate this approach. Neither passage establishes that acceptance has predominated across higher education since an initial reaction against ChatGPT; the paper describes views as mixed. (LLM, conf=0.86)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 98

  1. J. Ranalli and T. Yamashita, "Automated written corrective feedback: error-correction performance and timing of delivery," Language Learning & Technology, vol. 26, no. 1, pp. 1-25, 2022.
  2. E. Hellmich and K. Vinall, "FL instructor beliefs about machine translation: ecological insights to guide research and practice," International Journal of Computer-Assisted Language Learning and Teaching (IJCALLT), vol. 11, no. 4, pp. 1-18, 2021.
  3. L. Carvalho, R. Martinez-Maldonado, Y.-S. Tsai, L. Markauskaite, and M. De Laat, "How can we design for learning in an AI world?" Computers and Education: Artifi- cial Intelligence, vol. 3, Article ID 100053, 2022.
  4. S. Pellet and L. Myers, "What's wrong with "What is your name?" > "Quel est votre nom?": teaching responsible use of MT through discursive competence and metalanguage awareness," L2 Journal, vol. 14, no. 1, pp. 166-194, 2022.
  5. K. Vinall and E. A. Hellmich, "Do you speak translate? Reflections on the nature and role of translation," L2 Journal, vol. 14, no. 1, pp. 4-25, 2022.
  6. D. Adams and K.-M. Chuah, "Artificial intelligence-based tools in research writing: current trends and future poten- tials," in Artificial Intelligence in Higher Education, pp. 169- 184, CRC Press, 2022.
  7. J. Burstein, N. Elliot, and H. Molloy, "Informing automated writing evaluation using the lens of genre: two studies," CALICO Journal, vol. 33, no. 1, pp. 117-141, 2016.
  8. L. Ortega, "New CALL-SLA research interfaces for the 21st century: towards equitable multilingualism," CALICO Jour- nal, vol. 34, no. 3, pp. 283-316, 2017.
  9. B. Bridgeman and C. Ramineni, "Design and evaluation of automated writing evaluation models: relationships with writing in naturalistic settings," Assessing Writing, vol. 34, pp. 62-71, 2017.
  10. J. Ranalli, S. Link, and E. Chukharev-Hudilainen, "Auto- mated writing evaluation for formative assessment of second language writing: investigating the accuracy and usefulness of feedback as part of argument-based validation," Educational Psychology, vol. 37, no. 1, pp. 8-25, 2017.
  11. R. Godwin-Jones, "Second language writing online: an update," Language Learning & Technology, vol. 22, no. 1, pp. 1-15, 2018.
  12. A. I. Hibert, "Systematic literature review of automated writ- ing evaluation as a formative learning tool," in Transforming Learning with Meaningful Technologies, M. Scheffel, J. Broisin, V. Pammer-Schindler, A. Ioannou, and J. Schneider, Eds., pp. 199-212, Springer, 2019.
  13. M. A. Hussein, H. Hassan, and M. Nassef, "Automated lan- guage essay scoring systems: a literature review," PeerJ Com- puter Science, vol. 5, Article ID e208, 2019.
  14. M. Warschauer, S. Yim, H. Lee, and B. Zheng, "Recent con- tributions of data mining to language learning research," Annual Review of Applied Linguistics, vol. 39, pp. 93-112, 2019.
  15. A. Saricaoglu, "The impact of automated feedback on L2 learners' written causal explanations," ReCALL, vol. 31, no. 2, pp. 189-203, 2019.
  16. H. Peng, S. Jager, and W. Lowie, "A person-centred approach to L2 learners' informal mobile language learning," Computer Assisted Language Learning, vol. 35, no. 9, pp. 2148-2169, 2022.
  17. C. Palermo and J. Wilson, "Implementing automated writing evaluation in different instructional contexts: a mixed-methods study," Journal of Writing Research, vol. 12, no. 1, pp. 63-108, 2020.
  18. A. Nunes, C. Cordeiro, T. Limpo, and S. L. Castro, "Effectiveness of automated writing evaluation systems in school settings: a systematic review of studies from 2000 to 2020," Journal of Computer Assisted Learning, vol. 38, no. 2, pp. 599-620, 2022.
  19. A. Camacho, R. A. Alves, and P. Boscolo, "Writing motivation in school: a systematic review of empirical research in the early twenty-first century," Educational Psychology Review, vol. 33, pp. 213-247, 2021.
  20. R. Godwin-Jones, "Big data and language learning: opportu- nities and challenges," Language Learning & Technology, vol. 25, no. 1, pp. 4-19, 2021.
  21. Y. Huang and J. Wilson, "Using automated feedback to develop writing proficiency," Computers and Composition, vol. 62, Article ID 102675, 2021.
  22. J. M. Dembsey, "Closing the Grammarly gaps: a study of claims and feedback from an online grammar program," Writing Center Journal, vol. 36, no. 1, Article ID 5, 2017.
  23. D. C. Arroyo and Y. Yilmaz, "An open for replication study: the role of feedback timing in synchronous computer- mediated communication," Language Learning, vol. 68, no. 4, pp. 942-972, 2018.
  24. Y. Zheng and S. Yu, "Student engagement with teacher writ- ten corrective feedback in EFL writing: a case study of Chinese lower-proficiency students," Assessing Writing, vol. 37, pp. 13-24, 2018.
  25. M. Nova, "Utilizing Grammarly in evaluating academic writ- ing: a narrative research on EFL students' experience," Prem- ise: Journal of English Education and Applied Linguistics, vol. 7, no. 1, pp. 80-96, 2018.
  26. R. Conijn, M. van Zaanen, and L. van Waes, "Don't wait until it is too late: the effect of timing of automated feedback on revision in ESL writing," in Transforming Learning with Meaningful Technologies, M. Scheffel, J. Broisin, V. Pammer-Schindler, A. Ioannou, and J. Schneider, Eds., pp. 577-581, Springer, 2019.
  27. M. A. Ghufron, "Exploring an automated feedback program 'Grammarly' and teacher corrective feedback in EFL writing assessment: modern vs. traditional assessment," in Proceedings of the 3rd English Language and Literature International Con- ference, ELLiC, Semarang, Indonesia, April 2019.
  28. H.-W. Huang, Z. Li, and L. Taylor, "The effectiveness of using Grammarly to improve students' writing skills," in Proceed- ings of the 5th International Conference on Distance Educa- tion and Learning, pp. 122-127, Association for Computing Machinery, May 2020.
  29. P. John and N. Woll, "Using grammar checkers in an ESL context: an investigation of automatic corrective feedback," CALICO Journal, vol. 37, no. 2, pp. 193-196, 2020.
  30. R. O'Neill and A. M. T. Russell, "Grammarly: help or hin- drance? Academic learning advisors' perceptions of an online grammar checker," Journal of Academic Language and Learn- ing, vol. 13, no. 1, pp. A88-A107, 2020.
  31. M. Dodigovic and A. Tovmasyan, "Automated writing evalua- tion: the accuracy of Grammarly's feedback on form," Interna- tional Journal of TESOL Studies, vol. 3, no. 2, pp. 71-88, 2021.
  32. G. Zomer and A. Frankenberg-Garcia, "Beyond grammatical error correction: improving L1-influenced research writing in English using pre-trained encoder-decoder models," in Find- ings of the Association for Computational Linguistics: EMNLP 2021, pp. 2534-2540, Association for Computational Lin- guistics, 2021.
  33. E. M. O'Neill, "Measuring the impact of online translation on FL writing scores," IALLT Journal of Language Learning Technologies, vol. 46, no. 2, pp. 1-39, 2016.
  34. R. Godwin-Jones, "Data-informed language learning," Lan- guage Learning & Technology, vol. 21, no. 3, pp. 9-27, 2017.
  35. S. A. Crossley, "Technological disruption in foreign language teaching: the rise of simultaneous machine translation," Lan- guage Teaching, vol. 51, no. 4, pp. 541-552, 2018.
  36. K. Fredholm, "Effects of Google Translate on lexical diver- sity: vocabulary development among learners of Spanish as a foreign language," Revista Nebrija de Lingüística Aplicada a la Enseñanza de las Lenguas, vol. 13, no. 26, pp. 98-117, 2019.
  37. R. Godwin-Jones, "In a world of SMART technology, why learn another language?" Journal of Educational Technology & Society, vol. 22, no. 2, pp. 4-13, 2019.
  38. V. E. Raído and M. S. Torrón, "Machine translation, lan- guage learning and the knowledge economy," in Reimagining Communication: Action, M. Filimowicz and V. Tzankova, Eds., pp. 155-171, Taylor and Francis/Routedge, 2020.
  39. S.-M. Lee, "The impact of using machine translation on EFL students' writing," Computer Assisted Language Learning, vol. 33, no. 3, pp. 157-175, 2020.
  40. L. Flower and J. R. Hayes, "A cognitive process theory of writing," College Composition and Communication, vol. 32, no. 4, pp. 365-387, 1981.
  41. P. Urlaub and E. Dessein, "From disrupted classrooms to human-machine collaboration? The pocket calculator, Google Translate, and the future of language education," L2 Journal, vol. 14, no. 1, pp. 45-59, 2022.
  42. H. Zhang and O. Torres-Hostench, "Training in machine translation post-editing for foreign language students," Lan- guage Learning & Technology, vol. 26, no. 1, pp. 1-17, 2022.
  43. V. Klekovkina and L. Denié-Higney, "Machine translation: friend or foe in the language classroom?" L2 Journal, vol. 14, no. 1, pp. 105-135, 2022.
  44. J. R. Jolley and L. Maimone, "Thirty years of machine trans- lation in language teaching and learning: a review of the literature," L2 Journal, vol. 14, no. 1, pp. 26-44, 2022.
  45. J. Ryu, Y. A. Kim, S. Park, S. Eum, S. Chun, and S. Yang, "Exploring foreign language students' perceptions of the guided use of machine translation (GUMT) model for Korean writing," L2 Journal, vol. 14, no. 1, pp. 136-165, 2022.
  46. S. Ruder, "A review of the neural history of natural language processing. AYLIEN," October 2018, https://aylien.com/ blog/a-review-of-the-recent-history-of-natural-language processing#nonneuralmilestones.
  47. R. Dale, "Natural language generation: the commercial state of the art in 2020," Natural Language Engineering, vol. 26, no. 4, pp. 481-487, 2020.
  48. L. Floridi and M. Chiriatti, "GPT-3: its nature, scope, limits, and consequences," Minds and Machines, vol. 30, pp. 681- 694, 2020.
  49. L. Ferrone and F. M. Zanzotto, "Symbolic, distributed, and distributional representations for natural language proces- sing in the era of deep learning: a survey," Frontiers in Robotics and AI, vol. 6, Article ID 153, 2020.
  50. R. Dale, "GPT-3: what's it good for?" Natural Language Engineering, vol. 27, no. 1, pp. 113-118, 2021.
  51. M. Zhang and J. Li, "A commentary of GPT-3 in MIT Tech- nology Review 2021," Fundamental Research, vol. 1, no. 6, pp. 831-833, 2021.
  52. V. J. Schmalz and A. Brutti, "Automatic assessment of English CEFR levels using BERT embeddings," in Italian Conference on Computational Linguistics 2021, Bologna, Italy, http://ceur-ws.org/Vol-3033/paper14.pdf, July 2021.
  53. C. M. Anson, "AI-based text generation and the social con- struction of "Fraudulent Authorship": a revisitation," Com- position Studies, vol. 50, no. 1, pp. 37-46, 2022.
  54. C. M. Anson and I. Straume, "Amazement and trepidation: implications of AI-based natural language production for the teaching of writing," Journal of Academic Writing, vol. 12, no. 1, pp. 1-9, 2022.
  55. L. Jiang, S. Yu, and C. Wang, "Second language writing instructors' feedback practice in response to automated writ- ing evaluation: a sociocultural perspective," System, vol. 93, Article ID 102302, 2020.
  56. J. Woodworth and K. Barkaoui, "Perspectives on using auto- mated writing evaluation systems to provide written correc- tive feedback in the ESL classroom," TESL Canada Journal, vol. 37, no. 2, pp. 234-247, 2020.
  57. G. Kessler, "Current realities and future challenges for CALL teacher preparation," CALICO Journal, vol. 38, no. 3, pp. i-xx, 2021.
  58. G. Ling, N. Elliot, J. C. Burstein, D. F. McCaffrey, C. A. MacArthur, and S. Holtzman, "Writing motivation: a validation study of self-judgment and performance," Asses- sing Writing, vol. 48, Article ID 100509, 2021.
  59. Z. Li, "Teachers in automated writing evaluation (AWE) system-supported ESL writing classes: perception, implemen- tation, and influence," System, vol. 99, Article ID 102505, 2021.
  60. C. L. Knowles, "Using an ADAPT approach to integrate Google Translate into the second language classroom," L2 Journal, vol. 14, no. 1, pp. 195-236, 2022.
  61. D. T. Y. G. Sumakul, F. A. Hamied, and D. Sukyadi, "Artificial intelligence in EFL classrooms: friend or foe?" LEARN Journal: Language Education and Acquisition Research Network, vol. 15, no. 1, pp. 232-256, 2022.
  62. D. Larsen-Freeman, "Looking ahead: future directions in, and future research into, second language acquisition," Foreign Language Annals, vol. 51, no. 1, pp. 55-72, 2018.
  63. N. Hockly, "Automated writing evaluation," ELT Journal, vol. 73, no. 1, pp. 82-88, 2019.
  64. P. A. Patout and M. Cordy, "Towards context-aware automated writing evaluation systems," in Proceedings of the 1st ACM SIGSOFT International Workshop on Education through Advanced Software Engineering and Artificial Intelligence, B. Vanderose, Ed., pp. 17-20, Association for Computing Machinery, 2019.
  65. Z. V. Zhang, "Engaging with automated writing evaluation (AWE) feedback on L2 writing: student perceptions and revi- sions," Assessing Writing, vol. 43, Article ID 100439, 2020.
  66. C. van Lieshout and W. Cardoso, "Google Translate as a tool for self-directed language learning," Language Learning & Technology, vol. 26, no. 1, pp. 1-19, 2022.
  67. B. Sun and T. Fan, "The effects of an AWE-aided assessment approach on business English writing performance and writ- ing anxiety: a contextual consideration," Studies in Educa- tional Evaluation, vol. 72, Article ID 101123, 2022.
  68. J. Bitchener and D. R. Ferris, Written Corrective Feedback in Second Language Acquisition and Writing, Routledge, 2012.
  69. E. Y. Kang and Z. Han, "The efficacy of written corrective feedback in improving L2 written accuracy: a meta-analysis," The Modern Language Journal, vol. 99, no. 1, pp. 1-18, 2015.
  70. K. Hyland and F. Hyland, "Feedback on second language students' writing," Language Teaching, vol. 39, no. 2, pp. 83- 101, 2006.
  71. D. Grimes and M. Warschauer, "Utility in a fallible tool: a multi-site case study of automated writing evaluation," Jour- nal of Technology, Language, and Assessment, vol. 8, no. 6, pp. 1-43, 2010.
  72. V. Hegelheimer and J. Lee, "The role of technology in teach- ing and researching writing," in Contemporary Computer- Assisted Language Learning, M. Thomas, H. Reinders, and M. Warschauer, Eds., pp. 287-302, Bloomsbury, 2012.
  73. M. Stevenson, "A critical interpretative synthesis: the integration of automated writing evaluation into classroom writing instruc- tion," Computers and Composition, vol. 42, pp. 1-16, 2016.
  74. C. Leacock, M. Chodorow, M. Gamon, and J. Tetreault, "Automated grammatical error detection for language lear- ners," in Synthesis Lectures on Human Language Technologies, p. XV, 154, Springer, Cham, 2nd edition, 2014.
  75. K. Vinall and E. A. Hellmich, "Down the rabbit hole: machine translation, metaphor, and instructor identity and agency," Second Language Research & Practice, vol. 2, no. 1, pp. 99-118, 2021.
  76. K. Fredholm, "Online translation use in Spanish as a foreign language essay writing: effects on fluency, complexity and accu- racy," Revista Nebrija de Lingüística Aplicada a la Enseñanza de las Lenguas, vol. 18, pp. 1-18, 2015.
  77. A. Sukkhwan, "Students' attitudes and behaviors towards the use of Google Translate," Unpublished master's thesis, Prince of Songkla University, 2014.
  78. K. Fredholm, "Effects of online translation on morphosyntactic and lexical-pragmatic accuracy in essay writing in Spanish as a foreign language," in CALL Design: Principles and Practice. Proceedings of the 2014 EUROCALL Conference. Groningen, The Netherlands, S. Jager, L. Bradley, E. J. Meima, and S. Thouësny, Eds., pp. 96-101, Research-Publishing Net, 2014.
  79. E. M. O'Neill, "Training students to use online translators and dictionaries: the impact on second language writing scores," International Journal of Research Studies in Language Learning, vol. 8, no. 2, pp. 47-65, 2019.
  80. MLA Ad Hoc Committee on Foreign Languages, "Foreign languages and higher education: new structures for a changed world," Profession, vol. 2007, pp. 234-245, 2007.
  81. L. Ortega, "SLA for the 21st century: disciplinary progress, transdisciplinary relevance, and the bi/multilingual turn," Language Learning, vol. 63, no. s1, pp. 1-24, 2013.
  82. N. C. Ellis, "Cognition, corpora, and computing: triangulat- ing research in usage-based language learning," Language Learning, vol. 67, no. S1, pp. 40-65, 2017.
  83. M. Swain and S. Lapkin, "Problems in output and the cogni- tive processes they generate: a step towards second language learning," Applied Linguistics, vol. 16, no. 3, pp. 371-391, 1995.
  84. E. Cotos, "Genre-based automated writing evaluation," in Research Questions in Language Education and Applied Linguistics, pp. 645-650, Springer, Cham, 2021.
  85. M. Warschauer and P. Ware, "Automated writing evaluation: defining the classroom research agenda," Language Teaching Research, vol. 10, no. 2, pp. 157-180, 2006.
  86. I. Lee, "Revisiting teacher feedback in EFL writing from sociocultural perspectives," TESOL Quarterly, vol. 48, no. 1, pp. 201-213, 2014.
  87. W. Alharbi, "E-feedback as a scaffolding teaching strategy in the online language classroom," Journal of Educational Technology Systems, vol. 46, no. 2, pp. 239-251, 2017.
  88. W. Alharbi, "Students' perceptions and challenges in learning business English: understanding students' needs and job market requirements," International Journal of Learning, Teaching and Educational Research, vol. 21, no. 12, pp. 65- 87, 2022.
  89. A. Frankenberg-Garcia, "Combining user needs, lexico- graphic data and digital writing environments," Language Teaching, vol. 53, no. 1, pp. 29-43, 2020.
  90. J. L. Wilken, "Perceptions of L1 glossed feedback in auto- mated writing evaluation: a case study," CALICO Journal, vol. 35, no. 1, pp. 30-48, 2017.
  91. J. M. Gayed, M. K. J. Carlon, A. M. Oriola, and J. S. Cross, "Exploring an AI-based writing assistant's impact on English language learners," Computers and Education: Artificial Intelligence, vol. 3, Article ID 100055, 2022.
  92. T. N. Fitria, "Analysis on clarity and correctness of Google Translate in translating an Indonesian article into English," International Journal of Humanity Studies, vol. 4, no. 2, pp. 256-366, 2021.
  93. L. K. Wei, "The use of Google Translate in English language learning: how students view it," International Journal of Advanced Research in Education and Society, vol. 3, no. 1, pp. 47-53, 2021.
  94. J. E. Cansancio, L. I. C. Rillon, S. D. Sowaib, and C. A. G. Tuppal, "Perception of foreign language learners in the use of internet-based tools," International Journal of Research Publications, vol. 93, no. 1, pp. 208-222, 2022.
  95. W. H. Alharbi, "The affordances of augmented reality tech- nology in the English for specific purposes classroom: it's impact on vocabulary learning and students motivation in a Saudi higher education institution," Journal of Positive School Psychology, vol. 6, no. 3, pp. 6588-6602, 2022.
  96. C. Lütge, T. Merse, and P. Rauschert, Global Citizenship in Foreign Language Education: Concepts, Practices, Connections, Routledge, 2002.
  97. C. Kramsch, "General editor's preface," L2 Journal, vol. 14, no. 1, pp. 1-3, 2022.
  98. C. A. Chapelle, E. Cotos, and J. Lee, "Validity arguments for diagnostic assessment using automated writing evaluation," Language Testing, vol. 32, no. 3, pp. 385-405, 2015.

Cited by

Sorted by citations

Authors
Alharbi, W.
Affiliations
Royal Commission for Jubail and Yanbu (SA)
Corresponding author
Alharbi, Wael
ORCID
Alharbi, Wael
Venue
Education Research International
Publisher
Wiley
Published
2023-02-08
Volume
2023
Pages
1-15
ISSN
  • 2090-4002 (ISSN-L)
  • 2090-4010
DOI
10.1155/2023/4253331
OpenAlex
W4319459780
Cited by
286
References
15
Citations by year
2026: 77 · 2025: 109 · 2024: 82 · 2023: 18
FWCI
67.15
Type
journal article
Language
en
Open access
Licence
Locations
Education Research International (publishedVersion, cc-by)
Topics
  • Physical Sciences › Computer Science › Information Systems › Artificial Intelligence in Education
  • Social Sciences › Arts and Humanities › Language and Linguistics › EFL/ESL Teaching and Learning
  • Physical Sciences › Computer Science › Information Systems › English Language Learning and Teaching
Keywords
foreign language writing, automated writing evaluation, corrective feedback, machine translation, sentence completion, student perceptions of AI, teacher perceptions of AI, writing development, artificial intelligence in language teaching

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review