Evaluating Factuality in Text Simplification
Open accessAbstract
models aim to make input texts more readable. Such methods have the potential to make complex information accessible to a wider audience, e.g., providing access to recent medical literature which might otherwise be impenetrable for a lay reader. However, such models risk introducing errors into automatically simplified texts, for instance by inserting statements unsupported by the corresponding original text, or by omitting key information. Providing more readable but inaccurate versions of texts may in many cases be worse than providing no such access at all. The problem of factual accuracy (and the lack thereof) has received heightened attention in the context of summarization models, but the factuality of automatically simplified texts has not been investigated. We introduce a taxonomy of errors that we use to analyze both references drawn from standard simplification datasets and state-of-the-art model outputs. We find that errors often appear in both that are not captured by existing evaluation metrics, motivating a need for research into ensuring the factual accuracy of automated simplification models.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Technical options to overcome limitations in factuality and mitigate bias-related harms can generally be implemented throughout the entire development process of LLMs.
“Technical options to overcome limitations in factuality and mitigate (bias-related) harms can generally be implemented throughout the entire development process of LLMs.”
From The future landscape of large language models in medicine (2023) · cites this paper as [66]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 48,293 · see citing works
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071 · see citing works
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515 · see citing works
Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks
Reimers, N., & Gurevych, I. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1410 · Cited by 12,778 · see citing works
Deep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983 · see citing works
Lewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., Stoyanov, V., & Zettlemoyer, L. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.703 · Cited by 8,821 · see citing works
Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning
Williams, R. J. (1992) · Machine Learning · doi:10.1007/bf00992696 · Cited by 7,936 · see citing works
BERTScore: Evaluating Text Generation with BERT
Zhang, T., Kishore, V., Wu, F., Weinberger, K. Q., & Artzi, Y. (2019) · arXiv · doi:10.48550/arxiv.1904.09675 · Cited by 2,865 · see citing works
Inter-Coder Agreement for Computational Linguistics
Artstein, R., & Poesio, M. (2008) · Computational Linguistics · doi:10.1162/coli.07-034-r2 · Cited by 1,665 · see citing works
The Curious Case of Neural Text Degeneration
Holtzman, A., Buys, J., Du, L., Forbes, M., & Choi, Y. (2019) · arXiv · doi:10.48550/arxiv.1904.09751 · Cited by 1,304 · see citing works
Narayan, S., Cohen, S. B., & Lapata, M. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1206 · Cited by 1,207 · see citing works
On Faithfulness and Factuality in Abstractive Summarization
Maynez, J., Narayan, S., Bohnet, B., & McDonald, R. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.173 · Cited by 1,185 · see citing works
The proof and measurement of association between two things. By C. Spearman, 1904.
Spearman, C. (1987) · PubMed · Cited by 1,023 · see citing works
The Proof and Measurement of Association between Two Things
Spearman, C. (1987) · The American Journal of Psychology · doi:10.2307/1422689 · Cited by 1,014 · see citing works
Evaluating the Factual Consistency of Abstractive Text Summarization
Kryscinski, W., McCann, B., Xiong, C., & Socher, R. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.750 · Cited by 669 · see citing works
Optimizing Statistical Machine Translation for Text Simplification
Xu, W. H., Napoles, C., Pavlick, E., Chen, Q. Z., & Callison-Burch, C. (2016) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00107 · Cited by 524 · see citing works
Estimating the Reliability, Systematic Error and Random Error of Interval Data
Krippendorff, K. (1970) · Educational and Psychological Measurement · doi:10.1177/001316447003000105 · Cited by 511 · see citing works
Klare, G. R. (1963) · Cited by 399 · see citing works
Problems in Current Text Simplification Research: New Data Can Help
Xu, W. H., Callison-Burch, C., & Napoles, C. (2015) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00139 · Cited by 369 · see citing works
(2020) · doi:10.18653/v1/2020.acl-main.450 · Cited by 356 · see citing works
Sentence Simplification with Deep Reinforcement Learning
Zhang, X., & Lapata, M. (2017) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d17-1062 · Cited by 341 · see citing works
Neural Text Summarization: A Critical Evaluation
Kryscinski, W., Keskar, N. S., McCann, B., Xiong, C., & Socher, R. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1051 · Cited by 335 · see citing works
Durmus, E., He, H., & Diab, M. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.454 · Cited by 306 · see citing works
The Effects of Simplified and Elaborated Texts on Foreign Language Reading Comprehension
Yano, Y., Long, M. H., & Ross, S. J. (1994) · Language Learning · doi:10.1111/j.1467-1770.1994.tb01100.x · Cited by 279 · see citing works
Falke, T., Ribeiro, L. F. R., Utama, P. A., Dagan, I., & Gurevych, I. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1213 · Cited by 265 · see citing works
Understanding Factuality in Abstractive Summarization with FRANK: A Benchmark for Factuality Metrics
Pagnoni, A., Balachandran, V., & Tsvetkov, Y. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.383 · Cited by 197 · see citing works
EditNTS: An Neural Programmer-Interpreter Model for Sentence Simplification through Explicit Editing
Dong, Y., Li, Z., Rezagholizadeh, M., & Cheung, J. C. K. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1331 · Cited by 141 · see citing works
Rello, L., Baeza-Yates, R. A., Dempere‐Marco, L., & Saggion, H. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-40498-6_15 · Cited by 134 · see citing works
Text simplification for children
Belder, J. D., & Moens, M. (2010) · Lirias · Cited by 126 · see citing works
Data-Driven Sentence Simplification: Survey and Benchmark
Alva-Manchego, F. E., Scarton, C., & Specia, L. (2020) · Computational Linguistics · doi:10.1162/coli_a_00370 · Cited by 117 · see citing works
Annotating and Modeling Fine-grained Factuality in Summarization
Goyal, T., & Durrett, G. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.114 · Cited by 100 · see citing works
Evaluating Factuality in Generation with Dependency-level Entailment
Goyal, T., & Durrett, G. (2020) · Findings of the Association for Computational Linguistics: EMNLP 2020 · doi:10.18653/v1/2020.findings-emnlp.322 · Cited by 93 · see citing works
Controllable Text Simplification with Explicit Paraphrasing
Maddela, M., Alva-Manchego, F. E., & Xu, W. H. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.277 · Cited by 83 · see citing works
Paragraph-level Simplification of Medical Texts
Devaraj, A., Marshall, I., Wallace, B., & Li, J. J. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.395 · Cited by 79 · see citing works
Self-citationControllable Sentence SimplificationPreprint
Martin, L., Sagot, B., de la Clergerie, É. V., & Bordes, A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.02677 · Cited by 78 · see citing works
Complexity-Weighted Loss and Diverse Reranking for Sentence Simplification
Kriz, R., Sedoc, J., Apidianaki, M., Zheng, C., Kumar, G., Miltsakaki, E., & Callison-Burch, C. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1317 · Cited by 58 · see citing works
Decontextualization: Making Sentences Stand-Alone
Choi, E., Palomaki, J., Lamm, M., Kwiatkowski, T., Das, D., & Collins, M. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00377 · Cited by 54 · see citing works
Wallace, B., Saha, S., Soboczenski, F., & Marshall, I. J. (2020) · PubMed Central · doi:10.48550/arxiv.2008.11293 · Cited by 48 · see citing works
Self-citationDynamic Multi-Level Multi-Task Learning for Sentence SimplificationPreprint
Guo, H., Pasunuru, R., & Bansal, M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.07304 · Cited by 44 · see citing works
Semi-Supervised Text Simplification with Back-Translation and Asymmetric Denoising Autoencoders
Zhao, Y., Chen, L., Chen, Z., & Yu, K. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i05.6515 · Cited by 37 · see citing works
Keep It Simple: Unsupervised Simplification of Multi-Paragraph Text
Laban, P., Schnabel, T., Bennett, P. N., & Hearst, M. A. (2021) · doi:10.18653/v1/2021.acl-long.498 · Cited by 32 · see citing works
Fact-based Content Weighting for Evaluating Abstractive Summarisation
Xu, X., Dušek, O., Li, J., Rieser, V., & Konstas, I. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.455 · Cited by 18 · see citing works
Self-citationImproving the Annotation of Sentence Specificity
Li, J. J., O'Daniel, B., Wu, Y. H., Zhao, W., & Nenkova, A. (2016) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/3r99vpkieq6w · Cited by 17 · see citing works
Self-citationOn the Helpfulness of Document Context to Sentence Simplification
Sun, R., Lin, Z., & Wan, X. (2020) · International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.121 · Cited by 10 · see citing works
Flagged first, then by citations
1 of the 45 references couldn’t be matched to a known paper.
Research works printed but not linked: 8
- Carroll John, Minnen Guido, Canning Yvonne, Devlin Siobhan, and Tait John. 1998. Practical simplification of english newspaper text to assist aphasic readers. In Proceedings of the AAAI-98 Workshop on Integrating Artificial Intelligence and Assistive Technology, pages 7– 10.
- Damay Jerwin Jan S, Lojico Gerard Jaime D, Lu Kimberly Amanda L, Tarantan D, and Ong E. 2006. SIMTEXT: Text simplification of medical literature. In Proceedings of the 3rd National Natural Language Processing Symposium-Building Language Tools and Resources, pages 34– 38.
- Falke Tobias, Ribeiro Leonardo F. R., Utama Prasetya Ajie, Dagan Ido, and Gurevych Iryna. 2019b. Ranking generated summaries by correctness: An interesting but challenging application for natural language inference. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 2214– 2220, Florence, Italy. Association for Computational Linguistics.
- Kryscinski Wojciech, McCann Bryan, Xiong Caiming, and Socher Richard. 2020b. Evaluating the factual consistency of abstractive text summarization. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 9332– 9346, Online. Association for Computational Linguistics.
- Raffel Colin, Shazeer Noam, Roberts Adam, Lee Katherine, Narang Sharan, Matena Michael, Zhou Yanqi, Li Wei, and Liu Peter J.. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 21(140): 1– 67.34305477
- Van Dijk Teun A. 2013. News as discourse. Routledge.
- Wang Alex, Cho Kyunghyun, and Lewis Mike. 2020a. Asking and answering questions to evaluate the factual consistency of summaries. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5008– 5020, Online. Association for Computational Linguistics.
- Wang Alex, Cho Kyunghyun, and Lewis Mike. 2020b. Asking and answering questions to evaluate the factual consistency of summaries. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics.
Cited by
Sorted by citations
The future landscape of large language models in medicine
Clusmann, J., Kolbinger, F. R., Muti, H. S., Carrero, Z. I., Eckardt, J.-N., Laleh, N. G., Löffler, C. M. L., Schwarzkopf, S.-C., Unger, M., Veldhuizen, G. P., et al. (2023) · Communications Medicine · doi:10.1038/s43856-023-00370-1 · Cited by 1,045 · see citing works
Repairing the Cracked Foundation: A Survey of Obstacles in Evaluation Practices for Generated Text
Gehrmann, S., Clark, E., & Sellam, T. (2023) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.13715 · Cited by 123 · see citing works
LM vs LM: Detecting Factual Errors via Cross Examination
Cohen, R., Hamri, M., Geva, M., & Globerson, A. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.778 · Cited by 60 · see citing works
Faithful AI in Medicine: A Systematic Review with Large Language Models and BeyondPreprint
Xie, Q., Schenck, E. J., Yang, H. S., Chen, Y., Peng, Y., & Wang, F. (2023) · medRxiv · doi:10.1101/2023.04.18.23288752 · Cited by 57 · see citing works
Retrieval augmentation of large language models for lay language generation
Guo, Y., Qiu, W., Gondy Leroy, A., Wang, S., & Cohen, T. (2023) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2023.104580 · Cited by 52 · see citing works
Clinical insights: A comprehensive review of language models in medicine
Neveditsin, N., Lingras, P., & Mago, V. (2025) · PLOS Digital Health · doi:10.1371/journal.pdig.0000800 · Cited by 37 · see citing works
Multilingual Simplification of Medical Texts
Joseph, S. A., Kazanas, K., Reina, K., Ramanathan, V., Xu, W. H., Wallace, B., & Li, J. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.1037 · Cited by 27 · see citing works
Self-citationBLESS: Benchmarking Large Language Models on Sentence Simplification
Kew, T., Chi, A., Vásquez-Rodríguez, L., Agrawal, S., Aumiller, D., Alva-Manchego, F. E., & Shardlow, M. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.821 · Cited by 34 · see citing works
Lu, J., Li, J., Wallace, B., He, Y., & Pergola, G. (2023) · Findings of the Association for Computational Linguistics: EACL · doi:10.18653/v1/2023.findings-eacl.80 · Cited by 18 · see citing works
Self-citationFactPICO: Factuality Evaluation for Plain Language Summarization of Medical Evidence
Joseph, S., Chen, L., Trienes, J., Göke, H., Coers, M., Xu, W., Wallace, B., & Li, J. J. (2024) · Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2024.acl-long.459 · Cited by 17 · see citing works
Self-citationTeaching the Pre-trained Model to Generate Simple Texts for Text Simplification
Sun, R., Xu, W. H., & Wan, X. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.595 · Cited by 18 · see citing works
Do Text Simplification Systems Preserve Meaning? A Human Evaluation via Reading Comprehension
Agrawal, S., & Carpuat, M. J. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00653 · Cited by 17 · see citing works
August, T., Lo, K., Smith, N. A., & Reinecke, K. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642289 · Cited by 12 · see citing works
Evaluating Synthetic Data Generation from User Generated Text
Chim, J., Ive, J., & Liakata, M. (2024) · Computational Linguistics · doi:10.1162/coli_a_00540 · Cited by 10 · see citing works
HALoGEN: Fantastic LLM Hallucinations and Where to Find Them
Ravichander, A., Ghela, S., Wadden, D., & Choi, Y. (2025) · Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2025.acl-long.71 · Cited by 13 · see citing works
Patient-friendly Clinical Notes: Towards a new Text Simplification Dataset
Trienes, J., Schlötterer, J., Schildhaus, H., & Seifert, C. (2022) · Workshop on Text Simplification, Accessibility and Readability (TSAR) · doi:10.18653/v1/2022.tsar-1.3 · Cited by 9 · see citing works
Converge to the Truth: Factual Error Correction via Iterative Constrained Editing
Chen, J., Xu, R., Zeng, W., Sun, C., Li, L., & Xiao, Y. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i11.26485 · Cited by 9 · see citing works
Editing Common Sense in Transformers
Gupta, A., Mondal, D., Sheshadri, A. K., Zhao, W., Li, X., Wiegreffe, S., & Tandon, N. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.511 · Cited by 15 · see citing works
Dancing Between Success and Failure: Edit-level Simplification Evaluation using SALSA
Heineman, D., Dou, Y., Maddela, M., & Xu, W. H. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.211 · Cited by 10 · see citing works
SimplifyMyText: An LLM-Based System for Inclusive Plain Language Text Simplification
Färber, M., Aghdam, P. M., Im, K., Tawfelis, M., & Ghoshal, H. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-88717-8_32 · Cited by 6 · see citing works
Overview of the CLEF 2025 SimpleText Track
Магдановна, Е. Л., Azarbonyad, H., Bakker, J., Vendeville, B., & Kamps, J. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-04354-2_23 · Cited by 6 · see citing works
SWiPE: A Dataset for Document-Level Simplification of Wikipedia Pages
Laban, P., Vig, J., Kryściński, W., Joty, S., Xiong, C., & Wu, C.-S. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.596 · Cited by 7 · see citing works
Models See Hallucinations: Evaluating the Factuality in Video Captioning
Liu, H., & Wan, X. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.723 · Cited by 6 · see citing works
SimpLex: a lexical text simplification architecture
Truică, C., Stan, A.-I., & Apostol, E. (2022) · Neural Computing and Applications · doi:10.1007/s00521-022-07905-y · Cited by 7 · see citing works
Representation transfer and data cleaning in multi-views for text simplification
He, W. P., Farrahi, K., Chen, B., Peng, B., & Villavicencio, A. (2023) · Pattern Recognition Letters · doi:10.1016/j.patrec.2023.11.011 · Cited by 6 · see citing works
A Survey of Factual Consistency in Summarization from 2021 to 2023
Li, Y., Hao, X., & Li, L. (2023) · International Conference on Computers and Artificial Intelligence Technology (CAIT) · doi:10.1109/cait59945.2023.10468958 · Cited by 4 · see citing works
Gao, M., Wan, X., Su, J., Wang, Z., & Huai, B. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.779 · Cited by 5 · see citing works
WeCheck: Strong Factual Consistency Checker via Weakly Supervised Learning
Wu, W., Li, W., Xiao, X., Liu, J., Li, S., & Lyu, Y. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.18 · Cited by 7 · see citing works
Improving Text Simplification with Factuality Error Detection
Ma, Y., Seneviratne, S., & Daskalaki, E. (2022) · Proceedings of the Workshop on Text Simplification, Accessibility, and Readability (TSAR-2022) · doi:10.18653/v1/2022.tsar-1.16 · Cited by 3 · see citing works
Towards Accessible Information Retrieval for Children With a Mild Intellectual Disability
Weijers, R., Ooms, S., Pelrine, K., & Hauptmann, H. (2026) · Communications in computer and information science · doi:10.1007/978-3-032-12717-4_9 · Cited by 3 · see citing works
Магдановна, Е. Л., Azarbonyad, H., Bakker, J., Vendeville, B., & Kamps, J. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-88720-8_63 · Cited by 3 · see citing works
Магдановна, Е. Л., Azarbonyad, H., Bakker, J., Shahi, G. K., Vendeville, B., & Kamps, J. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-21321-1_31 · Cited by 2 · see citing works
Entity-Based Evaluation of Political Bias in Automatic Summarization
Zhou, K., & Tan, C. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.696 · Cited by 5 · see citing works
Die kommende Entwicklung großer Sprachmodelle in der Medizin
Clusmann, J., Kolbinger, F. R., Muti, H. S., Carrero, Z. I., Eckardt, J., Laleh, N. G., Loeffler, C. M. L., Schwarzkopf, S.-C., Unger, M., Veldhuizen, G. P., et al. (2024) · Kompass Onkologie · doi:10.1159/000536600 · Cited by 2 · see citing works
Automatic Generation of a Large Multiple-Choice Question-Answer Corpus
Kauchak, D., Song, V., Mishra, P., Gondy Leroy, A., Harber, P. I., Rains, S. A., Hamre, J., & Morgenstein, N. (2024) · Lecture notes in networks and systems · doi:10.1007/978-3-031-66428-1_4 · Cited by 2 · see citing works
Rosati, D. (2023) · Workshop on Biomedical Language Processing (BioNLP) · doi:10.18653/v1/2023.bionlp-1.46 · Cited by 2 · see citing works
Enhancing inter-sentence coherence of extractive summarization with multitask learning
Jie, R., Meng, X., Lifeng, S., Jiang, X., & Liu, Q. (2025) · Journal of Intelligent Information Systems · doi:10.1007/s10844-025-00967-z · Cited by 4 · see citing works
Text and Audio Simplification: Human vs. ChatGPT
Gondy Leroy, A., Kauchak, D., Harber, P. I., Pal, A., & Shukla, A. (2024) · PubMed Central · doi:10.48550/arxiv.2405.01592
Element-Centered Multi-granularity Network for Dense Video Captioning
Dang, X., Wang, G., Wu, X., & Qin, Z. (2024) · Lecture notes in computer science · doi:10.1007/978-981-97-8792-0_31
From text to idea embeddings: a review on quintessential information extraction
Chihaia, R., Leon, F., & Trocan, M. (2025) · Journal of Information and Telecommunication · doi:10.1080/24751839.2025.2587991
Controllable and Document-Level Text Simplification
Cripwell, L. (2023) · theses.fr (ABES) · doi:10.70675/39a81752z69b3z4c9bzb9f9z4ac49474d4c5
Overview of the CLEF 2026 SimpleText Track
Магдановна, Е. Л., Azarbonyad, H., Bakker, J., Chakar, B., Shahi, G. K., & Kamps, J. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-39150-6_35
- Authors
- Devaraj, A., Sheffield, W., Wallace, B., & Li, J. J.
- Affiliations
- The University of Texas at Austin (US)
- Northeastern University (US)
- Venue
- Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
- Published
- 2022
- Volume
- 2022
- Pages
- 7331-7345
- ISSN
- 0736-587X (ISSN-L)
- PMC
- PMC9671157
- OpenAlex
- W4285265395
- Cited by
- 40
- References
- 45
- Citations by year
- 2026: 3 · 2025: 6 · 2024: 7 · 2023: 21 · 2022: 3
- FWCI
- 5.11
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Proceedings of the conference - Association for Computational Linguistics. Meeting (publishedVersion, cc-by)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Science Foundation (1901117, 2107524, 1850153)
- University of Texas at Austin
- National Institutes of Health (5r01lm012086-08)
- U.S. National Library of Medicine (R01-LM012086)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Text Readability and Simplification
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- text simplification, factual accuracy, factuality evaluation, error taxonomy, readability, information loss, summarization, evaluation metrics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
