Grammatical Error Correction: A Survey of the State of the Art
The DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index.
Abstract
Grammatical Error Correction (GEC) is the task of automatically detecting and correcting errors in text. The task not only includes the correction of grammatical errors, such as missing prepositions and mismatched subject–verb agreement, but also orthographic and semantic errors, such as misspellings and word choice errors, respectively. The field has seen significant progress in the last decade, motivated in part by a series of five shared tasks, which drove the development of rule-based methods, statistical classifiers, statistical machine translation, and finally neural machine translation systems, which represent the current dominant state of the art. In this survey paper, we condense the field into a single article and first outline some of the linguistic challenges of the task, introduce the most popular datasets that are available to researchers (for both English and other languages), and summarize the various methods and techniques that have been developed with a particular focus on artificial error generation. We next describe the many different approaches to evaluation as well as concerns surrounding metric reliability, especially in relation to subjective human judgments, before concluding with an overview of recent progress and suggestions for future work and remaining challenges.We hope that this survey will serve as a comprehensive resource for researchers who are new to the field or who want to be kept apprised of recent developments.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“While most GEC models provide viable grammar error corrections (Bryant and Ng, 2015;Bryant et al., 2023)”
From GEE! Grammar Error Explanation with Large Language Models (2024) · cites this paper as Bryant et al., 2023
In the citing paper
“Our GEE task is built on the actively studied GEC task, which is often formulated as a neural machine translation task (Boyd, 2018;Bryant et al., 2023;Yuan and Bryant, 2021;Zhang et al., 2022).”
From GEE! Grammar Error Explanation with Large Language Models (2024) · cites this paper as Bryant et al., 2023
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
BLEU: a method for automatic evaluation of machine translation
Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073135 · Cited by 21,344
Sequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766
Neural Machine Translation by Jointly Learning to Align and Translate
Bahdanau, D., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1409.0473 · Cited by 16,213 · cited 2×
“Bahdanau, Cho, and Bengio (2015) introduced an attention mechanism to implement variable-length representations, which eased optimisation difficulty and resulted in improved performance.”
BackgroundExploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929
Lewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., Stoyanov, V., & Zettlemoyer, L. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.703 · Cited by 8,821
Lewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., Stoyanov, V., & Zettlemoyer, L. (2019) · arXiv · doi:10.48550/arxiv.1910.13461 · Cited by 8,816
Magnitude and complex based diffusion signal reconstruction
(2014) · HAL · doi:10.1007/978-3-319 · Cited by 6,731
Liu, P., Yuan, W., Fu, J., Jiang, Z., Hayashi, H., & Neubig, G. (2023) · ACM Computing Surveys · doi:10.1145/3560815 · Cited by 4,361
Liu, P., Yuan, W., Fu, J., Jiang, Z., Hayashi, H., & Neubig, G. (2021) · arXiv · doi:10.48550/arxiv.2107.13586 · Cited by 4,352 · cited 1×
“An additional challenge in generative LLM-based GEC is that prompt engineering is important (Liu et al. 2023) and output may vary depending on whether a system was asked to ‘correct’ a grammatical error or ‘fix’ a grammatical error.”
BackgroundPopular Ensemble Methods: An Empirical Study
Opitz, D., & Maclin, R. (1999) · Journal of Artificial Intelligence Research · doi:10.1613/jair.614 · Cited by 2,953 · cited 1×
“Ensembles often generate better predictions than any of the single models that are combined (Opitz and Maclin 1999).”
BackgroundBERTScore: Evaluating Text Generation with BERT
Zhang, T., Kishore, V., Wu, F., Weinberger, K. Q., & Artzi, Y. (2019) · arXiv · doi:10.48550/arxiv.1904.09675 · Cited by 2,865
PaLM: Scaling Language Modeling with Pathways
Chowdhery, A., Narang, S., Devlin, J., Bosma, M., Mishra, G., Roberts, A., Barham, P., Chung, H. W., Sutton, C., Gehrmann, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.02311 · Cited by 2,590
(2019) · doi:10.18653/v1/p19-1356 · Cited by 1,291
Finetuned Language Models Are Zero-Shot Learners
Wei, J., Bosma, M., Zhao, V. Y., Guu, K., Yu, A. W., Lester, B., Du, N., Dai, A. M., & Le, Q. V. (2021) · arXiv · doi:10.48550/arxiv.2109.01652 · Cited by 1,101 · cited 1×
“These models have furthermore been shown to be capable of generalising to new unseen tasks or languages by being fine-tuned on a wide variety of other NLP tasks (Sanh et al. 2022; Wei et al. 2022; Muennighoff et al. 2022), and so it is possible, for the first time, to build a system that is capable of carrying out mul…”
BackgroundMultitask Prompted Training Enables Zero-Shot Task Generalization
Sanh, V., Webson, A., Raffel, C., Bach, S. H., Sutawika, L., Alyafeai, Z., Chaffin, A., Stiegler, A., Scao, T. L., Raja, A., et al. (2021) · arXiv · doi:10.48550/arxiv.2110.08207 · Cited by 612 · cited 1×
“…have furthermore been shown to be capable of generalising to new unseen tasks or languages by being fine-tuned on a wide variety of other NLP tasks (Sanh et al. 2022; Wei et al. 2022; Muennighoff et al. 2022), and so it is possible, for the first time, to build a system that is capable of carrying…”
BackgroundCrosslingual Generalization through Multitask Finetuning
Muennighoff, N., Wang, T., Sutawika, L., Roberts, A., Biderman, S., Le Scao, T., Bari, M. S., Shen, S., Yong, Z. X., Schoelkopf, H., et al. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.891 · Cited by 361
Automated Essay Scoring: A Survey of the State of the Art
Ke, Z., & Ng, V. (2019) · Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/879 · Cited by 247
Towards Automated Document Revision: Grammatical Error Correction, Fluency Edits, and Beyond
Mita, M., Sakaguchi, K., Hagiwara, M., Mizumoto, T., Suzuki, J., & Inui, K. (2022) · arXiv · doi:10.48550/arxiv.2205.11484 · Cited by 11
Cited by
GEE! Grammar Error Explanation with Large Language Models
Song, Y., Krishna, K., Bhatt, R., Gimpel, K., & Iyyer, M. (2024) · Findings of the Association for Computational Linguistics: NAACL 2024 · doi:10.18653/v1/2024.findings-naacl.49 · Cited by 62
Automated Focused Feedback Generation for Scientific Writing Assistance
Chamoun, E., Schlichktrull, M., & Vlachos, A. (2024) · arXiv · doi:10.48550/arxiv.2405.20477 · Cited by 17
Metadata
- Authors
- Bryant, C., Yuan, Z., Qorib, M. R., Cao, H., Ng, H. T., & Briscoe, T.
- Affiliations
- ALTA Institute, Department of Computer Science and Technology, University of Cambridge. cjb255@cam.ac.uk
- Department of Informatics, King’s College London. zheng.yuan@kcl.ac.uk
- National University of Singapore, Department of Computer Science. mrqorib@u.nus.edu
- Department of Computer Science, National University of Singapore. caoh@u.nus.edu
- Department of Computer Science, National University of Singapore. nght@comp.nus.edu.sg
- Natural Language Processing Department, Mohamed bin Zayed University of Artificial Intelligence. ted.briscoe@mbzuai.ac.ae
- Venue
- Computational Linguistics
- Publisher
- MIT Press
- Published
- 2023-07-10
- Pages
- 1-59
- ISSN
- 0891-2017
- 1530-9312
- Cited by
- 83
- References
- 21
- Language
- en
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.