GEE! Grammar Error Explanation with Large Language Models
Open accessThe DOI matches its Crossref record; 35 of the 35 references it cites were found in a registry or an index; none of the 38 sources its claims cite is checked for support yet.
Abstract
Existing grammatical error correction tools do not provide natural language explanations of the errors that they correct in user-written text.However, such explanations are essential for helping users learn the language by gaining a deeper understanding of its grammatical rules (DeKeyser, 2003;Ellis et al., 2006).To address this gap, we propose the task of grammar error explanation, where a system needs to provide one-sentence explanations for each grammatical error in a pair of erroneous and corrected sentences.The task is not easily solved by prompting LLMs: we find that, using one-shot prompting, GPT-4 only explains 40.6% of the errors and does not even attempt to explain 39.8% of the errors.Since LLMs struggle to identify grammar errors, we develop a two-step pipeline that leverages fine-tuned and prompted large language models to perform structured atomic token edit extraction, followed by prompting GPT-4 to explain each edit.We evaluate our pipeline on German, Chinese, and English grammar error correction data.Our atomic edit extraction achieves an F1 of 0.93 on German, 0.91 on Chinese, and 0.891 on English.Human evaluation of generated explanations reveals that 93.9% of German errors, 96.4% of Chinese errors, and 92.20% of English errors are correctly detected and explained.To encourage further research, we open-source our data and code. 1
Overview
CoChat Summary
TL;DR
The paper proposes grammar error explanation (GEE), where a system generates one-sentence explanations for each grammatical error given an erroneous sentence and its corrected version. It develops a two-step LLM pipeline—atomic token edit extraction followed by GPT-4 explanation—and reports high edit-extraction F1 and human-judged explanation correctness in German, Chinese, and English.
Key findings
- One-shot GPT-4 is insufficient for GEE: it detects only 60.2% of true errors and correctly explains 67.5% of detected errors (40.6% of all errors; it does not attempt 39.8% of errors). It mentions 120 edits vs. 90 gold edits, with recall 0.602, precision 0.675, and F1 0.636. Providing gold atomic edits raises error-coverage F1 to 0.968 and 82% of true errors receive an appropriate explanation.
- Atomic edit extraction: fine-tuned GPT-3.5-turbo is best for German (recall 0.923, precision 0.939, F1 0.931) and English (F1 0.891); prompted GPT-4 is best for Chinese (average recall 0.888, precision 0.924, F1 0.906).
- Human evaluation of GEE outputs finds 93.9% of German, 96.4% of Chinese, and 92.20% of English explanations correct. German teacher agreement is 89.6%; Chinese teacher agreement is 92.9%.
- Remaining issues include GPT-4 producing overly general error types, insufficient context for some preposition errors, and unclear word-choice contrasts in Chinese.
Methods
- Task definition: input is a pair \(X_{\text{error}}\), \(X_{\text{correct}}\); output is one-sentence natural-language explanation per atomic edit, with edits typed as insert, delete, replace, or relocate.
- Pipeline: Step 1 extracts atomic edits by prompting Claude-2, GPT-3.5-turbo, and GPT-4 and by fine-tuning Llama2-7B and GPT-3.5-turbo; Step 2 appends extracted edits to the sentence pair and prompts GPT-4 with language-specific few-shot prompts to generate an explanation and error type.
- Data and evaluation: German uses Falko and Merlin (1122 GEE sentence pairs; 500 fine-tune and 50 test pairs with 1598 and 186 gold edits); Chinese uses CGED2017 (496 fine-tune, 53 test, 970 GEE pairs); English uses W&I+LOCNESS from BEA-2019 (512 fine-tune, 57 test, 93 GEE pairs). Edit extraction is evaluated with precision/recall/F1; GEE outputs are evaluated by human teachers for German and Chinese and manually by the authors for English.
Limitations
- The paper states the system only considers sentence-level inputs, so error types such as word choice and coreference may benefit from larger context.
- The paper states the Chinese data come from the HSK test, so covered topics are limited and do not include learners from all proficiency levels; error types may not be representative.
- The paper states that, due to time limitations, English language teachers were not hired to evaluate English GEE outputs; the text also makes evident that English GEE was evaluated by the authors rather than independent teachers.
- The paper notes the relocation edit type reduces model performance because models tend to label relocated tokens as deletion plus insertion.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures




Verification
Claims the paper makes
0 Valid · 2 Not valid · 25 Not checked · 131 Own results and interpretations
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“0.91 on Chinese”
- Also stated in: § 6.1 Atomic edit extraction results · plain
“The average results of GPT-4 are recall 0.888, precision 0.924, and F1 score 0.906.”
§ GEE! Grammar Error Explanation with Large Language Models > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § 6.1 Atomic edit extraction results · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“does not even attempt to explain 39.8% of the errors.”
§ GEE! Grammar Error Explanation with Large Language Models > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“GPT-4 makes mistakes even in short sentences.”
§ GEE! Grammar Error Explanation with Large Language Models > GPT-4 struggles to identify and explain er-rors.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“While being useful for GEE, the edit type relocate occasionally reduces the model performance because models tends to label a relocated token as deletion plus insertion.”
§ GEE! Grammar Error Explanation with Large Language Models > 4.1 Atomic edit extraction
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Results on Chinese: prompted GPT-4 is the most effective edit extractor.”
§ GEE! Grammar Error Explanation with Large Language Models > 6.1 Atomic edit extraction results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
86 more claims
Create a free account to see all 158 claims and to verify the 27 references still open.
- It may state two separate claims as one. 25
- The quote only partly states this claim. 7
- Probably not a substantial claim. 50
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ GEE! Grammar Error Explanation with Large Language Models > J Results of atomic edit extraction and human evaluation of English GEE
“Table: 7 Table 7 : Agreement between two German teachers on 96 sentence pairs.”
§ GEE! Grammar Error Explanation with Large Language Models > J Results of atomic edit extraction and human evaluation of English GEE
“| Sum | 84 | 100% |”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
“| Fully correct | 189 | 92.20% | | Wrong error explanation | 11 | 5.37% | | Wrong error type | 1 | 0.49% | | Hallucinated error | 4 | 1.95% | | Total explanation count | 25 | 100% |”
§ GEE! Grammar Error Explanation with Large Language Models > Additional information
Reported 25 · recomputed 205
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ GEE! Grammar Error Explanation with Large Language Models > 6.2.3 Human evaluation of English GEE
“There are 93 sentence pairs with 210 explanations.”
§ GEE! Grammar Error Explanation with Large Language Models > Additional information
“93 sentence pairs with 205 explanations are annotated.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ GEE! Grammar Error Explanation with Large Language Models > Limitations
“We acknowledge two limitations of our current work.”
§ GEE! Grammar Error Explanation with Large Language Models > Limitations
“Third, due to the time limitation, we did not hire English language teachers to evaluate the English GEE outputs.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ GEE! Grammar Error Explanation with Large Language Models > 6.2.2 Human evaluation of Chinese GEE
“Among the 356 annotated explanations, 96.37% are judged as correct by the Chinese teachers.”
§ GEE! Grammar Error Explanation with Large Language Models > 6.2.2 Human evaluation of Chinese GEE
“Two Chinese teachers evaluated Chinese GEE outputs on 356 sentence pairs with 523 explanations.”
Citations
Cites
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
BLEU: a method for automatic evaluation of machine translation
Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073135 · Cited by 21,344
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929
Lewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., Stoyanov, V., & Zettlemoyer, L. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.703 · Cited by 8,821
(2005) · Cited by 4,152
QLoRA: Efficient Finetuning of Quantized LLMs
Dettmers, T., Pagnoni, A., Holtzman, A., & Zettlemoyer, L. (2023) · arXiv · doi:10.48550/arxiv.2305.14314 · Cited by 3,832
IMPLICIT AND EXPLICIT CORRECTIVE FEEDBACK AND THE ACQUISITION OF L2 GRAMMAR
Ellis, R., Loewen, S., & Erlam, R. (2006) · Studies in Second Language Acquisition · doi:10.1017/s0272263106060141 · Cited by 1,076
Implicit and Explicit Learning
DeKeyser, R. (2003) · The Handbook of Second Language Acquisition · doi:10.1002/9780470756492.ch11 · Cited by 775
EPILOGUE: A Framework for Investigating Oral and Written Corrective Feedback
Ellis, R. (2010) · Studies in Second Language Acquisition · doi:10.1017/s0272263109990544 · Cited by 418
GECToR – Grammatical Error Correction: Tag, Not Rewrite
Omelianchuk, K., Atrasevych, V., Chernodub, A., & Skurzhanskyi, O. (2020) · Proceedings of the Fifteenth Workshop on Innovative Use of NLP for Building Educational Applications · doi:10.18653/v1/2020.bea-1.16 · Cited by 274
Cited by
Position: LLMs Can be Good Tutors in English Education
Ye, J., Wang, S., Zou, D., Yan, Y., Wang, K., Zheng, H.-T., Liu, R., Xu, Z., King, I., Yu, P. S., et al. (2025) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2025.emnlp-main.885 · Cited by 40
Alirector: Alignment-Enhanced Chinese Grammatical Error Corrector
Yang, H., & Quan, X. (2024) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2024.findings-acl.148 · Cited by 30
EXCGEC: A Benchmark for Edit-Wise Explainable Chinese Grammatical Error Correction
Ye, J., Qin, S., Li, Y., Cheng, X., Qin, L., Zheng, H.-T., Shen, Y., Xing, P., Xu, Z., Cheng, G., et al. (2025) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v39i24.34759 · Cited by 19
Wang, Z., Makarova, V., Li, Z., Kodner, J., & Rambow, O. (2025) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2025.acl-long.423 · Cited by 19
Explanation based In-Context Demonstrations Retrieval for Multilingual Grammatical Error Correction
Li, D. C. S., Luo, W., Peng, G., & Wang, H. (2025) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2025.naacl-long.251 · Cited by 14
Metadata
- Authors
- Song, Y., Krishna, K., Bhatt, R., Gimpel, K., & Iyyer, M.
- Affiliations
- University of Massachusetts Amherst (US)
- Venue
- Findings of the Association for Computational Linguistics: NAACL 2024
- Published
- 2024
- Pages
- 754-781
- OpenAlex
- W4401042780
- Cited by
- 62
- References
- 32
- Citations by year
- 2026: 25 · 2025: 28 · 2024: 9
- FWCI
- 5.03
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Findings of the Association for Computational Linguistics: NAACL (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Science Foundation (2202506, 2312949)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Keywords
- grammatical error correction, large language models, GPT-4, error detection, language learning, multilingual grammar, Chinese grammar, German grammar, English grammar
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-11-20, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-11-20 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: GEE! Grammar Error Explanation with Large Language Models. https://app.cochat.ai/papers/10.18653/v1/2024.findings-naacl.49
- The paper
- Song, Y., Krishna, K., Bhatt, R., Gimpel, K., & Iyyer, M. (2024). GEE! Grammar Error Explanation with Large Language Models. Findings of the Association for Computational Linguistics: NAACL 2024. https://doi.org/10.18653/v1/2024.findings-naacl.49
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.