The DOI matches its Crossref record; 52 of the 55 references it cites were found in a registry or an index.
Improving Factual Completeness and Consistency of Image-to-Text Radiology Report Generation
Open accessAbstract
Yasuhide Miura, Yuhao Zhang, Emily Tsai, Curtis Langlotz, Dan Jurafsky. Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2021.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This model also reports the best scores on the NLG metrics as well as improvements of +7.59% on factENT and +14.2% on factENTNLI over Miura et al. (2021).”
From Improving the Factual Correctness of Radiology Report Generation with Semantic Rewards (2022) · cites this paper as Miura et al., 2021
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“So far, these new methods rely on weakly-supervised approaches (rule-based) to construct NLI models for radiology reports and biomedical named entity recognition systems (Zhang et al., 2021) that are not specific to chest X-rays.”
From Improving the Factual Correctness of Radiology Report Generation with Semantic Rewards (2022) · cites this paper as Miura et al., 2021
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“rewards based solely on entities (Miura et al., 2021) or entities and relations cannot be optimized without counter-effects happening.”
From Improving the Factual Correctness of Radiology Report Generation with Semantic Rewards (2022) · cites this paper as Miura et al., 2021
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Miura et al. (2021) have shown that existing systems are inadequate in factual completeness and consistency”
From Improving the Factual Correctness of Radiology Report Generation with Semantic Rewards (2022) · cites this paper as Miura et al., 2021
The cited source says what the claim attributes to it.
In the citing paper
“On MIMIC-CXR, our baseline is up 2.3% on factENT and up 8.0% on factENTNLI compared to Miura et al. (2021).”
From Improving the Factual Correctness of Radiology Report Generation with Semantic Rewards (2022) · cites this paper as Miura et al., 2021
The cited source says what the claim attributes to it.
In the citing paper
“Despite these”
From Improving the Factual Correctness of Radiology Report Generation with Semantic Rewards (2022) · cites this paper as Miura et al., 2021
The cited source says what the claim attributes to it.
In the citing paper
“and that an image-to-text radiology report generation (RRG) system can be substantially improved by replacing the widely used NLG metrics with”
From Improving the Factual Correctness of Radiology Report Generation with Semantic Rewards (2022) · cites this paper as Miura et al., 2021
The cited source says what the claim attributes to it.
In the citing paper
“Secondly, these systems that achieve high performance on NLG metrics are not always factually complete or consistent due to both inadequate training and evaluation of these systems.”
From Improving the Factual Correctness of Radiology Report Generation with Semantic Rewards (2022) · cites this paper as Miura et al., 2021
The cited source says what the claim attributes to it.
In the citing paper
“In contrast, the generation of complete narrative radiology reports11,12,13,14,15,16,17,18,19,20,21 moves past that simplification and is consistent with how radiologists communicate diagnostic information: the narrative report allows for highly diverse and nuanced findings, including association of findings with anatomic location, and expressions of uncertainty.”
From Evaluating progress in automatic chest X-ray radiology report generation (2023) · cites this paper as [11]
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“However, their customized transformer model required 7 days of training on a single NVIDIA A100 GPU, whereas our BrainGPT model requires only 12 h of fine-tuning on two NVIDIA A100 GPUs.”
From Towards a holistic framework for multimodal LLM in 3D brain CT radiology report generation (2025) · cites this paper as [36]
Experimental — accuracy not yet measuredIn the citing paper
“Recent work by Hamamci et al. revealed that generative models are competent in 3D chest CT report generation at the state-of-the-art (SOTA) level (BLEU-1 = 46, BLEU-4 = 36.9, METEOR = 29.5, ROUGE-L = 45.9)35–37.”
From Towards a holistic framework for multimodal LLM in 3D brain CT radiology report generation (2025) · cites this paper as [36]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Jakubův, J., Chvalovský, K., Goertzel, Z., et al. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,648
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Densely Connected Convolutional Networks
Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K. Q. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.243 · Cited by 48,566
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071
BLEU: a method for automatic evaluation of machine translation
Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073135 · Cited by 21,344
WordNet: a lexical database for English
Miller, G. A. (1995) · Communications of the ACM · doi:10.1145/219717.219748 · Cited by 14,015
MIMIC-III, a freely accessible critical care database
Johnson, A. E. W., Pollard, T. J., Shen, L., Lehman, L.-W. H., Feng, M., Ghassemi, M., Moody, B., Szolovits, P., Anthony Celi, L., & Mark, R. G. (2016) · Scientific Data · doi:10.1038/sdata.2016.35 · Cited by 8,141
Show and tell: A neural image caption generator
Vinyals, O., Toshev, A., Bengio, S., & Erhan, D. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298935 · Cited by 6,318
CIDEr: Consensus-based image description evaluation
Vedantam, R., Zitnick, C. L., & Parikh, D. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299087 · Cited by 4,845
A Learning Algorithm for Continually Running Fully Recurrent Neural Networks
Williams, R. J., & Zipser, D. (1989) · Neural Computation · doi:10.1162/neco.1989.1.2.270 · Cited by 4,369
BERTScore: Evaluating Text Generation with BERT
Zhang, T., Kishore, V., Wu, F., Weinberger, K. Q., & Artzi, Y. (2019) · arXiv · doi:10.48550/arxiv.1904.09675 · Cited by 2,865
Self-Critical Sequence Training for Image Captioning
Rennie, S. J., Marcheret, E., Mroueh, Y., Ross, J., & Goel, V. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.131 · Cited by 2,231
5.7 The WORDNET lexical database of English
Ooi, V. B. Y. (2019) · Edinburgh University Press eBooks · doi:10.1515/9781474471459-050 · Cited by 2,035
MIMIC-CXR, a de-identified publicly available database of chest radiographs with free-text reports
Johnson, A. E. W., Pollard, T. J., Berkowitz, S. J., Greenbaum, N. R., Lungren, M. P., Deng, C.-Y., Mark, R. G., & Horng, S. (2019) · Scientific Data · doi:10.1038/s41597-019-0322-0 · Cited by 1,816
Stanza: A Python Natural Language Processing Toolkit for Many Human Languages
Qi, P., Zhang, Y., Zhang, Y., Bolton, J., & Manning, C. D. (2020) · Annual Meeting of the Association for Computational Linguistics: System Demonstrations (ACL Demo) · doi:10.18653/v1/2020.acl-demos.14 · Cited by 1,506
Self-citationSequence Level Training with Recurrent Neural Networks
Ranzato, M., Chopra, S., Auli, M., & Zaremba, W. (2015) · arXiv · doi:10.48550/arxiv.1511.06732 · Cited by 1,426
Meshed-Memory Transformer for Image Captioning
Cornia, M., Stefanini, M., Baraldi, L., & Cucchiara, R. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01059 · Cited by 1,259
On Faithfulness and Factuality in Abstractive Summarization
Maynez, J., Narayan, S., Bohnet, B., & McDonald, R. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.173 · Cited by 1,185
A Deep Reinforced Model for Abstractive Summarization
Paulus, R., Xiong, C., & Socher, R. (2017) · arXiv · doi:10.48550/arxiv.1705.04304 · Cited by 1,182
X-Linear Attention Networks for Image Captioning
Pan, Y., Yao, T., Li, Y., & Mei, T. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01098 · Cited by 720
Generating Radiology Reports via Memory-driven Transformer
Chen, Z., Song, Y., Chang, T.-H., & Wan, X. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.112 · Cited by 693
Evaluating the Factual Consistency of Abstractive Text Summarization
Kryscinski, W., McCann, B., Xiong, C., & Socher, R. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.750 · Cited by 669
On the Automatic Generation of Medical Imaging Reports
Jing, B., Xie, P., & Xing, E. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1240 · Cited by 657
Wang, X., Peng, Y., Lü, L., Lu, Z., & Summers, R. M. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00943 · Cited by 594
Sequence Level Training with Recurrent Neural Networks
Ranzato, M., Chopra, S., Auli, M., & Zaremba, W. (2016) · International Conference on Learning Representations · Cited by 572
Entangled Transformer for Image Captioning
Li, G., Zhu, L., Liu, P., & Yang, Y. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00902 · Cited by 411
Image Captioning: Transforming Objects into WordsPreprint
Herdade, S., Kappeler, A., Boakye, K., & Soares, J. C. V. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.05963 · Cited by 407
CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison
Irvin, J., Rajpurkar, P., Ko, M., Yu, Y., Ciurea-Ilcus, S., Chute, C., Marklund, H., Haghgoo, B., Ball, R., Shpanskaya, K., et al. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.3301590 · Cited by 392
Self-citation(2020) · doi:10.18653/v1/2020.acl-main.450 · Cited by 356
Durmus, E., He, H., & Diab, M. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.454 · Cited by 306
Hybrid Retrieval-Generation Reinforced Agent for Medical Image Report Generation
Li, C. Y., Liang, X., Hu, Z., & Xing, E. P. (2018) · arXiv · doi:10.48550/arxiv.1805.08298 · Cited by 271
Falke, T., Ribeiro, L. F. R., Utama, P. A., Dagan, I., & Gurevych, I. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1213 · Cited by 265
Smit, A., Jain, S., Rajpurkar, P., Pareek, A., Ng, A., & Lungren, M. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.117 · Cited by 262
Normalized and Geometry-Aware Self-Attention Network for Image Captioning
Guo, L., Liu, J., Zhu, X., Yao, P., Lu, S., & Lu, H. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01034 · Cited by 261
Lessons from Natural Language Inference in the Clinical Domain
Romanov, A., & Shivade, C. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1187 · Cited by 249
Dialogue Natural Language Inference
Welleck, S., Weston, J., Szlam, A., & Cho, K. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1363 · Cited by 249
Toward Best Practices in Radiology Reporting
Kahn, C. E., Langlotz, C. P., Burnside, E. S., Carrino, J. A., Channin, D. S., Hovsepian, D. M., & Rubin, D. L. (2009) · Radiology · doi:10.1148/radiol.2523081992 · Cited by 245
Self-citationShow, Describe and Conclude: On Exploiting the Structure Information of Chest X-ray Reports
Jing, B., Wang, Z., & Xing, E. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1657 · Cited by 223
Yuan, J., Liao, H., Luo, R., & Luo, J. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-32226-7_80 · Cited by 215
Clinically Accurate Chest X-Ray Report GenerationPreprint
Liu, G., Hsu, T.-M. H., McDermott, M. B. A., Boag, W., Weng, W., Peter Szolovits, B., & Ghassemi, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.02633 · Cited by 212
NegBio: a high-performance tool for negation and uncertainty detection in radiology reports
Peng, Y., Wang, X., Lu, L., Bagheri, M., Summers, R., & Lu, Z. (2017) · arXiv · doi:10.48550/arxiv.1712.05898 · Cited by 175
Optimizing the Factual Correctness of a Summary: A Study of Summarizing Radiology Reports
Zhang, Y., Merck, D., Tsai, E., Manning, C. D., & Langlotz, C. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.458 · Cited by 171
Self-citationDesign and Development of a Multimodal Biomedical Information Retrieval System
Demner-Fushman, D., Antani, S., Simpson, M., & Thoma, G. R. (2012) · Journal of Computing Science and Engineering · doi:10.5626/jcse.2012.6.2.168 · Cited by 161
Abacha, A. B., Shivade, C., & Demner‐Fushman, D. (2019) · BioNLP Workshop and Shared Task (BioNLP) · doi:10.18653/v1/w19-5039 · Cited by 128
Towards Faithful Neural Table-to-Text Generation with Content-Matching Constraints
Wang, Z., Wang, X., An, B., Yu, D., & Chen, C. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.101 · Cited by 59
Baselines for Chest X-Ray Report Generation
Boag, W., Hsu, T.-M. H., McDermott, M. B. A., Berner, G., Alesentzer, E., & Peter Szolovits, B. (2019) · Cited by 44
Improving Truthfulness of Headline Generation
Matsumaru, K., Takase, S., & Okazaki, N. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.123 · Cited by 42
Biomedical and Clinical English Model Packages in the Stanza Python NLP LibraryPreprint
Zhang, Y., Zhang, Y., Qi, P., Manning, C. D., & Langlotz, C. (2020) · arXiv.org · doi:10.48550/arxiv.2007.14640 · Cited by 15
Self-citationWTMED at MEDIQA 2019: A Hybrid Approach to Biomedical Natural Language Inference
Wu, Z., Song, Y., Huang, S., Tian, Y., & Xia, F. (2019) · BioNLP Workshop and Shared Task (BioNLP) · doi:10.18653/v1/w19-5044 · Cited by 10
Cited by
Sorted by citations
On the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676
Self-citationTransformers in medical imaging: A survey
Shamshad, F., Khan, S., Zamir, S. W., Khan, M. H., Hayat, M., Khan, F. S., & Fu, H. (2023) · Medical Image Analysis · doi:10.1016/j.media.2023.102802 · Cited by 1,523
Towards Generalist Biomedical AI
Tu, T., Azizi, S., Driess, D., Schaekermann, M., Amin, M., Chang, P.-C., Carroll, A., Lau, C., Tanno, R., Ktena, I., et al. (2024) · NEJM AI · doi:10.1056/aioa2300138 · Cited by 483
Making the Most of Text Semantics to Improve Biomedical Vision-Language Processing
Boecking, B., Usuyama, N., Bannur, S., Castro, D. C., Schwaighofer, A., Hyland, S., Wetscherek, M., Naumann, T., Nori, A., Alvarez-Valle, J., et al. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-20059-5_1 · Cited by 317
“[53] Yasuhide Miura, Yuhao Zhang, Emily Tsai, Curtis Langlotz, and Dan Jurafsky.”
BackgroundMethodResultInfluentialNeural Natural Language Processing for unstructured data in electronic health records: A review
Li, I., Pan, J. N., Goldwasser, J., Verma, N., Wong, W. P., Nuzumlalı, M. Y., Rosand, B., Li, Y., Zhang, M. M., Chang, D., et al. (2022) · Computer Science Review · doi:10.1016/j.cosrev.2022.100511 · Cited by 280
Metadata
- Authors
- Miura, Y., Zhang, Y., Tsai, E., Langlotz, C., & Jurafsky, D.
- Affiliations
- Stanford University (US)
- Venue
- Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
- Published
- 2021
- Pages
- 5288-5304
- OpenAlex
- W3166845084
- Cited by
- 149
- References
- 55
- Citations by year
- 2026: 25 · 2025: 31 · 2024: 35 · 2023: 36 · 2022: 21 · 2021: 1
- FWCI
- 7.59
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) (publishedVersion, cc-by)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Medicine › Health Informatics › Biomedical Text Mining and Ontologies
- Keywords
- image-to-text generation, radiology report generation, factual consistency, medical report summarization
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
