CoChat
Sign Up
PreprintCited by 2

Towards Automated Document Revision: Grammatical Error Correction, Fluency Edits, and Beyond

Mita, Masato, Sakaguchi, Keisuke, Hagiwara, Masato, Mizumoto, Tomoya, Suzuki, Jun, Inui, Kentaro·2022·arXiv
DOI 10.48550/arxiv.2205.11484arXiv 2205.11484
Open this paper in CoChat — free account

Abstract

Natural language processing technology has rapidly improved automated grammatical error correction tasks, and the community begins to explore document-level revision as one of the next challenges. To go beyond sentence-level automated grammatical error correction to NLP-based document-level revision assistant, there are two major obstacles: (1) there are few public corpora with document-level revisions being annotated by professional editors, and (2) it is not feasible to elicit all possible references and evaluate the quality of revision with such references because there are infinite possibilities of revision. This paper tackles these challenges. First, we introduce a new document-revision corpus, TETRA, where professional editors revised academic papers sampled from the ACL anthology which contain few trivial grammatical errors that enable us to focus more on document- and paragraph-level edits such as coherence and consistency. Second, we explore reference-less and interpretable methods for meta-evaluation that can detect quality improvements by document revision. We show the uniqueness of TETRA compared with existing document revision corpora and demonstrate that a fine-tuned pre-trained language model can discriminate the quality of documents after revision even when the difference is subtle. This promising result will encourage the community to further explore automated document revision models and metrics in future.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2205.11484

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • More advanced processing of revisions has been approached for arXiv pre-prints and ACL publications, and in the context of peer review.

    “more advanced processing of revisions has been approached for arXiv pre-prints (Jiang et al., 2022; Du et al., 2022) and ACL publications (Mita et al., 2022), and in the context of peer review (Kuznetsov et al., 2022; D'Arcy et al., 2023).”

    From What Can Natural Language Processing Do for Peer Review? (2024) · cites this paper as Mita et al., 2022

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

No references could be resolved yet.

Research works printed but not linked: 74

  1. Linda Allal, Lucile Chanquoy, and Pierre Largy. 2004. Revision Cognitive and Instructional Processes., vol- ume 8. Springer.
  2. Larry Beason. 1993. Feedback and revision in writ- ing across the curriculum classes. Research in the Teaching of English, pages 395-422.
  3. Shane Bergsma, Matt Post, and David Yarowsky. 2012. Stylometric analysis of scientific articles. In Pro- ceedings of the 2012 Conference of the North Amer- ican Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 327-337, Montréal, Canada. Association for Com- putational Linguistics.
  4. Yevgeni Berzak, Jessica Kenney, Carolyn Spadine, Jing Xian Wang, Lucia Lam, Keiko Sophie Mori, Sebastian Garza, and Boris Katz. 2016. Universal Dependencies for learner English. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 737-746, Berlin, Germany. Association for Computational Linguistics. · DOI
  5. Eric Brill and Robert C. Moore. 2000. An improved er- ror model for noisy channel spelling correction. In Proceedings of the 38th Annual Meeting of the As- sociation for Computational Linguistics, pages 286- 293.
  6. Christopher Bryant, Mariano Felice, Øistein E. An- dersen, and Ted Briscoe. 2019. The BEA-2019 shared task on grammatical error correction. In Pro- ceedings of the Fourteenth Workshop on Innovative Use of NLP for Building Educational Applications, pages 52-75, Florence, Italy. Association for Com- putational Linguistics. · DOI
  7. Christopher Bryant, Mariano Felice, and Ted Briscoe. 2017. Automatic Annotation and Evaluation of Er- ror Types for Grammatical Error Correction. In Pro- ceedings of the 55th Annual Meeting of the Asso- ciation for Computational Linguistics (ACL 2017), pages 793-805.
  8. Sabine Buchholz and Erwin Marsi. 2006. CoNLL- X shared task on multilingual dependency parsing. In Proceedings of the Tenth Conference on Com- putational Natural Language Learning (CoNLL-X), pages 149-164, New York City. Association for Computational Linguistics.
  9. Jill Burstein, Daniel Marcu, and Kevin Knight. 2003. Finding the write stuff: Automatic identification of discourse structure in student essays. IEEE Intelli- gent Systems, 18(1):32-39.
  10. Aoife Cahill, Nitin Madnani, Joel Tetreault, and Diane Napolitano. 2013. Robust systems for preposition error correction using Wikipedia revisions. In Pro- ceedings of the 2013 Conference of the North Amer- ican Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 507-517.
  11. Martin Chodorow, Joel Tetreault, and Na-Rae Han. 2007. Detection of grammatical errors involving prepositions. In Proceedings of the Fourth ACL- SIGSEM Workshop on Prepositions, pages 25-30.
  12. Shamil Chollampatt and Hwee Tou Ng. 2018. A Mul- tilayer Convolutional Encoder-Decoder Neural Net- work for Grammatical Error Correction. In Proceed- ings of the 32nd AAAI Conference on Artificial Intel- ligence (AAAI 2018), pages 5755-5762.
  13. Leshem Choshen and Omri Abend. 2018. Reference- less measure of faithfulness for grammatical error correction. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), pages 124- 129, New Orleans, Louisiana. Association for Com- putational Linguistics. · DOI
  14. Paul F Christiano, Jan Leike, Tom Brown, Miljan Mar- tic, Shane Legg, and Dario Amodei. 2017. Deep reinforcement learning from human preferences. In Advances in Neural Information Processing Systems, volume 30.
  15. Daniel Dahlmeier and Hwee Tou Ng. 2012. Better Evaluation for Grammatical Error Correction. In Proceedings of the 2012 Conference of the North American Chapter of the Association for Computa- tional Linguistics (NAACL 2012), pages 568-572.
  16. Robert Dale, Ilya Anisimoff, and George Narroway. 2012. Hoo 2012: A report on the preposition and determiner error correction shared task. In Proceed- ings of the Seventh Workshop on Building Educa- tional Applications Using NLP, pages 54-62. Asso- ciation for Computational Linguistics.
  17. Robert Dale and Adam Kilgarriff. 2011. Helping our own: The hoo 2011 pilot shared task. In Proceed- ings of the Generation Challenges Session at the 13th European Workshop on Natural Language Gen- eration, pages 242-249. Association for Computa- tional Linguistics.
  18. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language under- standing. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Tech- nologies, Volume 1 (Long and Short Papers), pages 4171-4186. Association for Computational Linguis- tics.
  19. Lester Faigley and Stephen Witte. 1981. Analyzing revision. College composition and communication, 32(4):400-414.
  20. Mariano Felice and Ted Briscoe. 2015. Towards a stan- dard evaluation method for grammatical error detec- tion and correction. In Proceedings of the 2015 Con- ference of the North American Chapter of the Asso- ciation for Computational Linguistics: Human Lan- guage Technologies (NACL-HLT 2015), pages 578- 587.
  21. Rachele De Felice and Stephen G. Pulman. 2008. A Classifier-Based Approach to Preposition and Deter- miner Error Correction in L2 English. In COLING, pages 169-176.
  22. Simon Flachs, Ophélie Lacroix, Helen Yannakoudakis, Marek Rei, and Anders Søgaard. 2020. Grammati- cal error correction in low error density domains: A new benchmark and analyses. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 8467-8478.
  23. Linda Flower and John R Hayes. 1981. A cognitive process theory of writing. College composition and communication, 32(4):365-387.
  24. Tao Ge, Furu Wei, and Ming Zhou. 2018. Reaching Human-level Performance in Automatic Grammati- cal Error Correction: An Empirical Study. arXiv preprint arXiv:1807.01270. · arXiv
  25. Takumi Gotou, Ryo Nagata, Masato Mita, and Kazuaki Hanawa. 2020. Taking the correction difficulty into account in grammatical error correction evaluation. In Proceedings of the 28th International Conference on Computational Linguistics, pages 2085-2095.
  26. Francisco Guzmán, Shafiq Joty, Lluís Màrquez, and Preslav Nakov. 2015. Pairwise neural machine trans- lation evaluation. In Proceedings of the 53rd Annual Meeting of the Association for Computational Lin- guistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 805-814. Association for Computa- tional Linguistics.
  27. Na-Rae Han, Martin Chodorow, and Claudia Leacock. 2006. Detecting Errors in English Article Usage by Non-Native Speakers. Natural Language Engineer- ing, 12(2):115-129.
  28. Kazuaki Hanawa, Ryo Nagata, and Kentaro Inui. 2021. Exploring methods for generating feedback com- ments for writing learning. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 9719-9730, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
  29. Aminul Islam and Diana Inkpen. 2009. Real-word spelling correction using Google Web 1T 3-grams. In Proceedings of the 2009 Conference on Empiri- cal Methods in Natural Language Processing, pages 1241-1249.
  30. Md Asadul Islam and Enrico Magnani. 2021. Is this the end of the gold standard? a straightforward reference-less grammatical error correction metric. In Proceedings of the 2021 Conference on Empiri- cal Methods in Natural Language Processing, pages 3009-3015, Online and Punta Cana, Dominican Re- public. Association for Computational Linguistics.
  31. Jianshu Ji, Qinlong Wang, Kristina Toutanova, Yongen Gong, Steven Truong, and Jianfeng Gao. 2017. A Nested Attention Neural Hybrid Model for Gram- matical Error Correction. In Proceedings of the 55th Annual Meeting of the Association for Compu- tational Linguistics (ACL 2017), pages 753-762.
  32. Masahiro Kaneko, Masato Mita, Shun Kiyono, Jun Suzuki, and Kentaro Inui. 2020. Encoder-decoder models can benefit from pre-trained masked lan- guage models in grammatical error correction. In Proceedings of the 58th Annual Meeting of the Asso- ciation for Computational Linguistics (ACL 2020), pages 4248-4254.
  33. Jungo Kasai, Keisuke Sakaguchi, Ronan Le Bras, Lavinia Dunagan, Jacob Morrison, Alexander R. Fabbri, Yejin Choi, and Noah A. Smith. 2021a. Bidi- mensional leaderboards: Generate and evaluate lan- guage hand in hand. arXiv https://arxiv.org/ abs/2112.04139. · arXiv
  34. Jungo Kasai, Keisuke Sakaguchi, Lavinia Dunagan, Jacob Morrison, Ronan Le Bras, Yejin Choi, and Noah A. Smith. 2021b. Transparent human evalu- ation for image captioning. arXiv https://arxiv. org/abs/2111.08940.
  35. Omid Kashefi, Tazin Afrin, Meghan Dale, Christopher Olshefski, Amanda Godley, Diane Litman, and Re- becca Hwa. 2022. Argrewrite v. 2: an annotated ar- gumentative revisions corpus. Language Resources and Evaluation, pages 1-35.
  36. Diederik Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. In Proceed- ings of the 3rd International Conference on Learn- ing Representations (ICLR 2015).
  37. Shun Kiyono, Jun Suzuki, Masato Mita, Tomoya Mizu- moto, and Kentaro Inui. 2019. An empirical study of incorporating pseudo data into grammatical error correction. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Pro- cessing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP 2019), pages 1236-1242.
  38. Charles W. Kneupper. 1978. Teaching argument: An introduction to the toulmin model. College Compo- sition and Communication, 29(3):237-241.
  39. Aomi Koyama, Kengo Hotate, Masahiro Kaneko, and Mamoru Komachi. 2021. Comparison of grammat- ical error correction using back-translation models. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Compu- tational Linguistics: Student Research Workshop, pages 126-135.
  40. Claudia Leacock, Martin Chodorow, Michael Gamon, and Joel Tetreault. 2014. Automated grammatical error detection for language learners. Synthesis lec- tures on human language technologies, 7(1):1-170.
  41. John Lee and Stephanie Seneff. 2008. Correcting mis- use of verb forms. In Proceedings of ACL-08: HLT, pages 174-182.
  42. John Lee and Jonathan Webster. 2012. A corpus of textual revisions in second language writing. In Pro- ceedings of the 50th Annual Meeting of the Associa- tion for Computational Linguistics (Volume 2: Short Papers), pages 248-252, Jeju Island, Korea. Associ- ation for Computational Linguistics.
  43. Masato Mita, Tomoya Mizumoto, Masahiro Kaneko, Ryo Nagata, and Kentaro Inui. 2019. Cross-corpora evaluation and analysis of grammatical error correc- tion models -is single-corpus evaluation enough? In Proceedings of the 2019 Conference of the North American Chapter of the Association for Compu- tational Linguistics: Human Language Technolo- gies, Volume 1 (Long and Short Papers), pages 1309-1314, Minneapolis, Minnesota. Association for Computational Linguistics. · DOI
  44. Ryo Nagata. 2019. Toward a task of feedback comment generation for writing learning. In Proceedings of the 2019 Conference on Empirical Methods in Nat- ural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3206-3215, Hong Kong, China. Association for Computational Linguistics.
  45. Ryo Nagata, Masato Hagiwara, Kazuaki Hanawa, Masato Mita, Artem Chernodub, and Olena Na- horna. 2021. Shared task on feedback comment generation for language learners. In Proceedings of the 14th International Conference on Natural Lan- guage Generation, pages 320-324, Aberdeen, Scot- land, UK. Association for Computational Linguis- tics.
  46. Ryo Nagata, Atsuo Kawai, Koichiro Morihiro, and Naoki Isu. 2006. A Feedback-Augmented Method for Detecting Errors in the Writing of Learners of English. In COLING-ACL, pages 241-248.
  47. Ryo Nagata, Mikko Vilenius, and Edward Whittaker. 2014. Correcting preposition errors in learner En- glish using error case frames and feedback messages. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 754-764.
  48. Courtney Napoles, Maria Nȃdejde, and Joel Tetreault. 2019. Enabling robust grammatical error correction in new domains: Data sets, metrics, and analyses. Transactions of the Association for Computational Linguistics, pages 551-566.
  49. Courtney Napoles, Keisuke Sakaguchi, Matt Post, and Joel Tetreault. 2015. Ground Truth for Grammati- cal Error Correction Metrics. In Proceedings of the 53rd Annual Meeting of the Association for Compu- tational Linguistics and the 7th International Joint Conference on Natural Language Processing (ACL- IJCNLP 2015), pages 588-593.
  50. Courtney Napoles, Keisuke Sakaguchi, Matt Post, and Joel Tetreault. 2016a. GLEU Without Tuning. arXiv preprint arXiv:1605.02592. · arXiv
  51. Courtney Napoles, Keisuke Sakaguchi, and Joel Tetreault. 2016b. There's no comparison: Reference-less evaluation metrics in grammati- cal error correction. In Proceedings of the 2016 Conference on Empirical Methods in Natural Lan- guage Processing, pages 2109-2115. Association for Computational Linguistics.
  52. Courtney Napoles, Keisuke Sakaguchi, and Joel Tetreault. 2017. JFLEG: A Fluency Corpus and Benchmark for Grammatical Error Correction. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Lin- guistics (EACL 2017), pages 229-234.
  53. Hwee Tou Ng, Siew Mei Wu, Ted Briscoe, Christian Hadiwinoto, Raymond Hendy Susanto, and Christo- pher Bryant. 2014. The CoNLL-2014 Shared Task on Grammatical Error Correction. In Proceedings of the 18th Conference on Computational Natural Lan- guage Learning (CoNLL 2014): Shared Task, pages 1-14.
  54. Hwee Tou Ng, Siew Mei Wu, Yuanbin Wu, Christian Hadiwinoto, and Joel Tetreault. 2013. The CoNLL- 2013 Shared Task on Grammatical Error Correction. In Proceedings of the 17th Conference on Compu- tational Natural Language Learning (CoNLL 2013): Shared Task, pages 1-12.
  55. Myle Ott, Sergey Edunov, Alexei Baevski, Angela Fan, Sam Gross, Nathan Ng, David Grangier, and Michael Auli. 2019. fairseq: A Fast, Extensible Toolkit for Sequence Modeling. In Proceedings of the 2019 Conference of the North American Chap- ter of the Association for Computational Linguistics (NAACL 2019).
  56. Kishore Papineni, Salim Roukos, Todd Ward, and Wei- Jing Zhu. 2002. BLEU: A Method for Automatic Evaluation of Machine Translation. In Proceedings of 40th Annual Meeting of the Association for Com- putational Linguistics, pages 311-318.
  57. Sascha Rothe, Jonathan Mallinson, Eric Malmi, Sebas- tian Krause, and Aliaksei Severyn. 2021. A simple recipe for multilingual grammatical error correction. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Lan- guage Processing (Volume 2: Short Papers), pages 702-707.
  58. Alla Rozovskaya and Dan Roth. 2014. Building a state-of-the-art grammatical error correction system. Transactions of the Association for Computational Linguistics, 2:419-434.
  59. Keisuke Sakaguchi, Courtney Napoles, Matt Post, and Joel Tetreault. 2016. Reassessing the goals of gram- matical error correction: Fluency instead of gram- maticality. Transactions of the Association for Com- putational Linguistics, 4:169-182.
  60. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Neural Machine Translation of Rare Words with Subword Units. In Proceedings of the 54th An- nual Meeting of the Association for Computational Linguistics (ACL 2016), pages 1715-1725.
  61. Anthony Seow. 2002. The Writing Process and Pro- cess Writing, page 315-320. Cambridge University Press.
  62. Toshikazu Tajiri, Mamoru Komachi, and Yuji Mat- sumoto. 2012. Tense and Aspect Error Correction for ESL Learners Using Global Context. In Pro- ceedings of the 50th Annual Meeting of the Asso- ciation for Computational Linguistics (ACL 2012), pages 198-202.
  63. Joel Tetreault, Jennifer Foster, and Martin Chodorow. 2010. Using parse features for preposition selection and error detection. In Proceedings of the ACL 2010 Conference Short Papers, pages 353-358.
  64. Joel R. Tetreault and Martin Chodorow. 2008. The ups and downs of preposition error detection in ESL writing. In Proceedings of the 22nd International Conference on Computational Linguistics (Coling 2008), pages 865-872.
  65. Kristina Toutanova and Robert Moore. 2002. Pronunci- ation modeling for improved spelling correction. In Proceedings of the 40th Annual Meeting of the As- sociation for Computational Linguistics, pages 144- 151.
  66. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention Is All You Need. In Advances in Neural Information Pro- cessing Systems 31 (NIPS 2017), pages 5998-6008.
  67. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier- ric Cistac, Tim Rault, Rémi Louf, Morgan Funtow- icz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush. 2020. Transformers: State-of-the-art natural language pro- cessing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 38-45. Association for Computational Linguistics.
  68. Ziang Xie, Guillaume Genthial, Andrew Y. Ng, and Dan Jurafsky. 2018. Noising and Denoising Natu- ral Language: Diverse Backtranslation for Grammar Correction. In NAACL, pages 619-628.
  69. Helen Yannakoudakis, Ted Briscoe, and Ben Medlock. 2011. A New Dataset and Method for Automatically Grading ESOL Texts. In ACL, pages 180-189.
  70. Zheng Yuan and Ted Briscoe. 2016. Grammatical error correction using neural machine translation. In Pro- ceedings of the 2016 Conference of the North Amer- ican Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 380-386.
  71. Zheng Yuan and Christopher Bryant. 2021. Document- level grammatical error correction. In Proceedings of the 16th Workshop on Innovative Use of NLP for Building Educational Applications, pages 75-84, Online. Association for Computational Linguistics.
  72. Fan Zhang, Homa B. Hashemi, Rebecca Hwa, and Diane Litman. 2017. A corpus of annotated revi- sions for studying argumentative writing. In Pro- ceedings of the 55th Annual Meeting of the Associa- tion for Computational Linguistics (Volume 1: Long Papers), pages 1568-1578. Association for Compu- tational Linguistics.
  73. Fan Zhang, Rebecca Hwa, Diane Litman, and Homa B. Hashemi. 2016. ArgRewrite: A web-based revision assistant for argumentative writings. In Proceedings of the 2016 Conference of the North American Chap- ter of the Association for Computational Linguistics: Demonstrations, pages 37-41, San Diego, Califor- nia. Association for Computational Linguistics. · DOI
  74. Fan Zhang and Diane Litman. 2014. Sentence-level rewriting detection. In Proceedings of the Ninth Workshop on Innovative Use of NLP for Building Ed- ucational Applications, pages 149-154, Baltimore, 1 <doc id="Pxx-xxxx" editor="A" format="Conference" position="Non-student" region="Native"> 2 <abstract> 3 <text>In this paper, (...) extracted sense inventory. The</text> 4 <edit type="conciseness" crr="induction and disambiguation steps" comments="conciseness -just tightening it up a little bit.">induction step and the disambiguation step</edit> 5 <text>are based on the same principle: (...) topical dimensions</text> 6 <edit type="readability" crr=". In" comments="readability -this sentence is getting a bit long, so splitting it in two here.">; in</edit> 7 <text>a similar vein, ...</text> 8 ... 9 </abstract> 10 <introduction> 11 <text>Word sense induction (...)</text> 12 <text>\n\n Word sense disambiguation (...)</text> 13 <edit type="punctuation" crr="" comments="punctuation -comma is not appropriate.">,</edit> 14 ... 15 </ · DOI

Cited by

Sorted by citations

Authors
Mita, M., Sakaguchi, K., Hagiwara, M., Mizumoto, T., Suzuki, J., & Inui, K.
Venue
arXiv
Publisher
arXiv
Published
2022
DOI
10.48550/arxiv.2205.11484
arXiv
2205.11484
References
0
Type
Preprint
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review