The DOI matches its Crossref record; 15 of the 15 references it cites were found in a registry or an index.
CLIPScore: A Reference-free Evaluation Metric for Image Captioning
Open accessAbstract
Image captioning has conventionally relied on reference-based automatic evaluations, where machine captions are compared against captions written by humans.This is in contrast to the reference-free manner in which humans assess caption quality.In this paper, we report the surprising empirical finding that CLIP (Radford et al., 2021), a cross-modal model pretrained on 400M im-age+caption pairs from the web, can be used for robust automatic evaluation of image captioning without the need for references.Experiments spanning several corpora demonstrate that our new reference-free metric, CLIPScore, achieves the highest correlation with human judgements, outperforming existing reference-based metrics like CIDEr and SPICE.Information gain experiments demonstrate that CLIPScore, with its tight focus on image-text compatibility, is complementary to existing reference-based metrics that emphasize text-text similarities.Thus, we also present a reference-augmented version, RefCLIPScore, which achieves even higher correlation.Beyond literal description tasks, several case studies reveal domains where CLIPScore performs well (clip-art images, alt-text rating), but also where it is relatively weaker in comparison to reference-based metrics, e.g., news captions that require richer contextual knowledge.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Common image generation evaluation metrics such as CLIPScore, FID, and uses of MLLMs 'as-judge' metrics often penalize authentic depictions of assistive technologies, while rewarding incorrect or even offensive renderings.
“However, common image generation evaluation metrics -such as CLIPScore [55], FID [56], and uses of multimodal large language models (MLLMs) 'as-judge' metrics [58, 71] often penalize authentic depictions of ATs, while rewarding incorrect or even offensive renderings (e.g., Braille displays shown as paper) [65].”
From Engaging Communities Meaningfully in Defining Disability Representation for AI Image Generation (2026) · cites this paper as [55]
The Introduction and “Evaluating Caption Generations with CLIP” identify CLIPScore as an evaluation metric, supporting the citation’s identification of the metric. The paper evaluates generated captions for images, not generated images, so it does not establish the cited description of CLIPScore as an image-generation evaluation metric. The claim about assistive-technology depictions is attached to [65], not this citation. (LLM, conf=0.84)
Experimental — accuracy not yet measuredCLIPScore leverages a shared multimodal embedding space to assess the semantic alignment between the generated description and the original image, without relying on the reference text.
“CLIPScore : Uses the CLIP model to measure the semantic similarity and compatibility between the original image and the generated text, without relying on the reference text [10].”
From Human vs LLM-Generated Alt Text for STEM Image Accessibility: A Comparative Study (2026) · cites this paper as [10]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2010.11929 · Cited by 47,472
BLEU: a method for automatic evaluation of machine translation
Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073135 · Cited by 21,344
Learning Transferable Visual Models From Natural Language Supervision
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. (2021) · arXiv · doi:10.48550/arxiv.2103.00020 · Cited by 15,293
Representation Learning with Contrastive Predictive Coding
Oord, A. V. D., Li, Y., & Vinyals, O. (2018) · arXiv · doi:10.48550/arxiv.1807.03748 · Cited by 6,242
BERTScore: Evaluating Text Generation with BERT
Zhang, T., Kishore, V., Wu, F., Weinberger, K. Q., & Artzi, Y. (2019) · arXiv · doi:10.48550/arxiv.1904.09675 · Cited by 2,865
Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision
Jia, C., Yang, Y., Xia, Y., Chen, Y.-T., Parekh, Z., Pham, H., Le, Q. V., Sung, Y., Li, Z., & Duerig, T. (2021) · arXiv · doi:10.48550/arxiv.2102.05918 · Cited by 1,977
Stacked Cross Attention for Image-Text Matching
Lee, K.-H., Chen, X., Hua, G., Hu, H., & He, X. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01225-0_13 · Cited by 1,402
How Multilingual is Multilingual BERT?
Pires, T., Schlinger, E., & Garrette, D. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1493 · Cited by 1,351
Object Hallucination in Image Captioning
Rohrbach, A., Hendricks, L. A., Burns, K., Darrell, T., & Saenko, K. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1437 · Cited by 420 · cited 2×
“Prior work has demonstrated that, for many literal description tasks, humans often prefer correctness in captions over specificity (Rohrbach et al., 2018, 2017).”
BackgroundWomen Also Snowboard: Overcoming Bias in Captioning Models
Hendricks, L. A., Burns, K., Saenko, K., Darrell, T., & Rohrbach, A. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01219-9_47 · Cited by 404
USR: An Unsupervised and Reference Free Evaluation Metric for Dialog Generation
Mehri, S., & Eskenazi, M. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.64 · Cited by 149
Stangl, A., Morris, M. R., & Gurari, D. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376404 · Cited by 139
Good News, Everyone! Context Driven Entity-Aware Captioning for News Images
Biten, A. F., Gomez, L., Rusiñol, M., & Karatzas, D. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01275 · Cited by 136
Flagged first, then by citations
Research works printed but not linked: 56
- Somak Aditya, Yezhou Yang, Chitta Baral, Cor- nelia Fermuller, and Yiannis Aloimonos. 2015. From images to sentences through scene description graphs using commonsense reasoning and knowl- edge. arXiv preprint arXiv:1511.03292. · arXiv
- Sandhini Agarwal, Gretchen Krueger, Jack Clark, Alec Radford, Jong Wook Kim, and Miles Brundage. 2021. Evaluating CLIP: Towards characterization of broader capabilities and downstream implications. arXiv preprint arXiv:2108.02818. · arXiv
- Peter Anderson, Basura Fernando, Mark Johnson, and Stephen Gould. 2016. Spice: Semantic proposi- tional image caption evaluation. In ECCV. Springer.
- Mikel Artetxe and Holger Schwenk. 2019. Mas- sively multilingual sentence embeddings for zero- shot cross-lingual transfer and beyond. TACL, 7:597-610.
- Satanjeev Banerjee and Alon Lavie. 2005. METEOR: an automatic metric for mt evaluation with improved correlation with human judgments. In ACL work- shop on Evaluation Measures for MT and Summa- rization.
- Alexander C. Berg, Tamara L. Berg, Hal Daumé III, Jesse Dodge, Amit Goyal, Xufeng Han, Alyssa Men- sch, Margaret Mitchell, Aneesh Sood, Karl Stratos, and Kota Yamaguchi. 2012. Understanding and pre- dicting importance in images. In CVPR.
- John Blatz, Erin Fitzgerald, George Foster, Simona Gandrabur, Cyril Goutte, Alex Kulesza, Alberto San- chis, and Nicola Ueffing. 2004. Confidence estima- tion for machine translation. In COLING.
- Yen-Chun Chen, Linjie Li, Licheng Yu, Ahmed El Kholy, Faisal Ahmed, Zhe Gan, Yu Cheng, and Jingjing Liu. 2020. Uniter: Universal image-text representation learning. In ECCV.
- Yin Cui, Guandao Yang, Andreas Veit, Xun Huang, and Serge Belongie. 2018. Learning to evaluate im- age captioning. In CVPR.
- Bo Dai and Dahua Lin. 2017. Contrastive learning for image captioning. In NeurIPS.
- Desmond Elliott and Frank Keller. 2014. Comparing automatic evaluation measures for image descrip- tion. In ACL.
- Cole Gleason, Amy Pavel, Emma McCamey, Christina Low, Patrick Carrington, Kris M Kitani, and Jef- frey P Bigham. 2020. Twitter a11y: A browser ex- tension to make twitter images accessible. In CHI.
- Ming Jiang, Qiuyuan Huang, Lei Zhang, Xin Wang, Pengchuan Zhang, Zhe Gan, Jana Diesner, and Jian- feng Gao. 2019. TIGEr: text-to-image grounding for image caption evaluation. In EMNLP.
- Karthikeyan K, Zihan Wang, Stephen Mayhew, and Dan Roth. 2020. Cross-lingual ability of multilin- gual BERT: An empirical study. In ICLR.
- Hassan Kane, Muhammed Yusuf Kocyigit, Ali Abdalla, Pelkins Ajanoh, and Mohamed Coulibali. 2020. NU- BIA: NeUral based interchangeability assessor for text generation. In 1st Workshop on Evaluating NLG Evaluation.
- Mert Kilickaya, Aykut Erdem, Nazli Ikizler-Cinbis, and Erkut Erdem. 2017. Re-evaluating automatic metrics for image captioning. In EACL.
- Hwanhee Lee, Seunghyun Yoon, Franck Dernoncourt, Trung Bui, and Kyomin Jung. 2021. UMIC: an unreferenced metric for image captioning via con- trastive learning. In ACL.
- Hwanhee Lee, Seunghyun Yoon, Franck Dernoncourt, Doo Soon Kim, Trung Bui, and Kyomin Jung. 2020. Vilbertscore: Evaluating image caption using vision- and-language bert. In First Workshop on Evaluation and Comparison of NLP Systems.
- Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. 2014. Microsoft COCO: Common objects in context. In ECCV. Springer.
- Xihui Liu, Hongsheng Li, Jing Shao, Dapeng Chen, and Xiaogang Wang. 2018. Show, tell and discrim- inate: Image captioning by self-retrieval with par- tially labeled data. In ECCV.
- Chi-kiu Lo. 2019. Yisi-a unified semantic mt quality evaluation and estimation metric for languages with different levels of available resources. In Fourth Conference on Machine Translation.
- Annie Louis and Ani Nenkova. 2013. Automatically assessing machine summary content without a gold standard. Computational Linguistics, 39(2):267- 300.
- Jiasen Lu, Dhruv Batra, Devi Parikh, and Stefan Lee. 2019. ViLBERT: Pretraining task-agnostic visi- olinguistic representations for vision-and-language tasks. In NeurIPS.
- Jiasen Lu, Vedanuj Goswami, Marcus Rohrbach, Devi Parikh, and Stefan Lee. 2020. 12-in-1: Multi-task vision and language representation learning. In CVPR.
- Grace Luo, Trevor Darrell, and Anna Rohrbach. 2021. NewsCLIPpings: automatic generation of out-of-context multimodal media. arXiv preprint arXiv:2104.05893. · arXiv
- Ruotian Luo, Brian Price, Scott Cohen, and Gregory Shakhnarovich. 2018. Discriminability objective for training descriptive captions. In CVPR.
- Haley MacLeod, Cynthia L Bennett, Meredith Ringel Morris, and Edward Cutrell. 2017. Understanding blind people's experiences with computer-generated captions of social media images. In CHI.
- Pranava Madhyastha, Josiah Wang, and Lucia Specia. 2019. VIFIDEL: Evaluating the visual fidelity of image descriptions. In ACL.
- Yashar Mehdad, Matteo Negri, and Marcello Federico. 2012. Match without a referee: evaluating mt ad- equacy without reference translations. In Seventh Workshop on Statistical Machine Translation.
- F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duch- esnay. 2011. Scikit-learn: Machine learning in Python. JMLR, 12.
- Maxime Peyrard and Iryna Gurevych. 2018. Objec- tive function learning to match human judgements for optimization-based summarization. In NAACL.
- Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9.
- Anna Rohrbach, Atousa Torabi, Marcus Rohrbach, Niket Tandon, Christopher Pal, Hugo Larochelle, Aaron Courville, and Bernt Schiele. 2017. Movie description. IJCV.
- Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Neural machine translation of rare words with subword units. In ACL.
- Ravi Shekhar, Sandro Pezzelle, Yauhen Klimovich, Au- rélie Herbelot, Moin Nabi, Enver Sangineto, and Raffaella Bernardi. 2017. FOIL it! find one mis- match between image and language caption. In ACL.
- Kurt Shuster, Samuel Humeau, Hexiang Hu, Antoine Bordes, and Jason Weston. 2019. Engaging image captioning via personality. In CVPR.
- Kihyuk Sohn. 2016. Improved deep metric learning with multi-class n-pair loss objective. In NeurIPS.
- Lucia Specia, Dhwaj Raj, and Marco Turchi. 2010. Ma- chine translation evaluation versus quality estima- tion. Machine translation, 24(1):39-50.
- Lucia Specia and Kashif Shah. 2018. Machine transla- tion quality estimation: Applications and future per- spectives. In Translation Quality Assessment, pages 201-235. Springer.
- Simeng Sun and Ani Nenkova. 2019. The feasibility of embedding based automatic evaluation for single document summarization. In EMNLP.
- Chongyang Tao, Lili Mou, Dongyan Zhao, and Rui Yan. 2018. Ruber: An unsupervised method for au- tomatic evaluation of open-domain dialog systems. In AAAI.
- Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In NeurIPS.
- Ramakrishna Vedantam, C Lawrence Zitnick, and Devi Parikh. 2015. Cider: Consensus-based image de- scription evaluation. In CVPR.
- Oriol Vinyals, Alexander Toshev, Samy Bengio, and Dumitru Erhan. 2016. Show and tell: Lessons learned from the 2015 mscoco image captioning challenge. TPAMI, 39(4):652-663.
- Sijin Wang, Ziwei Yao, Ruiping Wang, Zhongqin Wu, and Xilin Chen. 2021. FAIEr: Fidelity and adequacy ensured image caption evaluation. In CVPR.
- Shijie Wu and Mark Dredze. 2019. Beto, bentz, becas: The surprising cross-lingual effectiveness of BERT. In EMNLP.
- Elizaveta Yankovskaya, Andre Tättar, and Mark Fishel. 2019. Quality estimation and translation metrics via pre-trained word and sentence embeddings. In Fourth Conference on Machine Translation.
- Yanzhi Yi, Hangyu Deng, and Jinglu Hu. 2020. Im- proving image captioning evaluation by considering inter references variance. In ACL.
- Peter Young, Alice Lai, Micah Hodosh, and Julia Hock- enmaier. 2014. From image descriptions to visual denotations: New similarity metrics for semantic in- ference over event descriptions. TACL, 2:67-78.
- Wei Zhao, Goran Glavaš, Maxime Peyrard, Yang Gao, Robert West, and Steffen Eger. 2020. On the lim- itations of cross-lingual encoders as exposed by reference-free machine translation evaluation. In ACL.
- C Lawrence Zitnick and Devi Parikh. 2013. Bring- ing semantics into focus using visual abstraction. In CVPR. References
- Peter Anderson, Basura Fernando, Mark Johnson, and Stephen Gould. 2016. Spice: Semantic proposi- tional image caption evaluation. In ECCV. Springer.
- Dallas Card, Peter Henderson, Urvashi Khandelwal, Robin Jia, Kyle Mahowald, and Dan Jurafsky. 2020. With little power comes great responsibility. In EMNLP.
- Yin Cui, Guandao Yang, Andreas Veit, Xun Huang, and Serge Belongie. 2018. Learning to evaluate im- age captioning. In CVPR.
- Hassan Kane, Muhammed Yusuf Kocyigit, Ali Abdalla, Pelkins Ajanoh, and Mohamed Coulibali. 2020. NU- BIA: NeUral based interchangeability assessor for text generation. In 1st Workshop on Evaluating NLG Evaluation.
- Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. 2020. BERTScore: Evaluating text generation with BERT. In ICLR.
Cited by
Sorted by citations
Scaling Rectified Flow Transformers for High-Resolution Image Synthesis
Esser, P., Kulal, S., Blattmann, A., Entezari, R., Müller, J., Saini, H., Levi, Y., Lorenz, D., Sauer, A., Boesel, F., et al. (2024) · arXiv · doi:10.48550/arxiv.2403.03206 · Cited by 140
RePrompt: Automatic Prompt Editing to Refine AI-Generative Art Towards Precise Expressions
Wang, Y., Shen, S., & Lim, B. Y. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581402 · Cited by 120
Holodeck: Language Guided Generation of 3D Embodied AI Environments
Yang, Y., Sun, F.-Y., Weihs, L., Vanderbilt, E., Herrasti, A., Han, W., Wu, J., Haber, N., Krishna, R., Liu, L., et al. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.01536 · Cited by 101
“In addition to human judgments, we employ CLIP Score [17] to quantify the visual coherence between the top-down view of the scene and its corresponding scene type embedded in a prompt template “ a top-down view of [scene type] ”.”
Deepfake: definitions, performance metrics and standards, datasets, and a meta-review
Altuncu, E., Franqueira, V. N. L., & Li, S. (2024) · Frontiers in Big Data · doi:10.3389/fdata.2024.1400024 · Cited by 134
Image captioning by diffusion models: a survey
Daneshfar, F., Bartani, A., & Lotfi, P. (2024) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2024.109288 · Cited by 49
Ask in Any Modality: A Comprehensive Survey on Multimodal Retrieval-Augmented Generation
Abootorabi, M. M., Zobeiri, A., Dehghani, M., Mohammadkhani, M., Mohammadi, B., Ghahroodi, O., Baghshah, M. S., & Asgari, E. (2025) · Findings of the Association for Computational Linguistics: ACL 2025 · doi:10.18653/v1/2025.findings-acl.861 · Cited by 76
Qu, B., Liang, X., Sun, S., & Gao, W. (2024) · arXiv · doi:10.48550/arxiv.2404.13573 · Cited by 44
RankGen: Improving Text Generation with Large Ranking Models
Krishna, K., Chang, Y., Wieting, J., & Iyyer, M. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2022.emnlp-main.15 · Cited by 30
AIGCOIQA2024: Perceptual Quality Assessment of AI Generated Omnidirectional Images
Yang, L., Duan, H., Teng, L., Zhu, Y., Liu, X., Hu, M., Min, X., Zhai, G., & Callet, P. L. (2024) · arXiv · doi:10.48550/arxiv.2404.01024 · Cited by 27
WorldScore: A Unified Evaluation Benchmark for World Generation
Duan, H., Yu, H., Chen, S., Fei–Fei, L., & Wu, J. (2025) · 2025 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.02573 · Cited by 17
“Content alignment: Besides the objects (which typically occupies approximately only of the text prompt length), we also assess whether the generated scenes are aligned with the entire text N using CLIPScore [22].”
Crowdsourcing Thumbnail Captions: Data Collection and Validation
Aguirre, C., Cao, S., Mahmood, A., & Huang, C.-M. (2023) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3589346 · Cited by 6
Garg, K., Tang, X., Heo, J., Morgan, D. R., Gergle, D., Sudderth, E. B., & Piper, A. M. (2026) · Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791309 · Cited by 3
“Adding images to such measures (e.g., Vilbertscore [72], TIGEr [65], SCAN [75]) or using reference-free measures (e.g., CLIPScore [59]) can help, but degraded images can be a confound in their scores.”
Bootstrapping Language-Audio Pre-training for Music Captioning
Lanzendörfer, L. A., Pinkl, C., Perraudin, N., & Wattenhofer, R. (2025) · ICASSP 2025 - 2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp49660.2025.10887618 · Cited by 3
Human vs LLM-Generated Alt Text for STEM Image Accessibility: A Comparative Study
Cardia, M., Angileri, L., Poggianti, C., & Leporini, B. (2026) · Proceedings of the 23rd International Web for All Conference · doi:10.1145/3800424.3800457 · Cited by 2
“• CLIPScore : Uses the CLIP model to measure the semantic similarity and compatibility between the original image and the generated text, without relying on the reference text [10].”
Engaging Communities Meaningfully in Defining Disability Representation for AI Image Generation
Thieme, A., Faia Marques, R., Grayson, M., Balachandar, S., Tyler Cassidy, C., Choksi, M. Z., Longden, C., Huda, R. S., Ileve Kalovwe, N., Mallon, C., et al. (2026) · Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790768 · Cited by 1
“However, common image generation evaluation metrics — such as CLIPScore [55], FID [56], and uses of multimodal large language models (MLLMs) ‘as-judge’ metrics [58, 71] often penalize authentic depictions of ATs, while re-warding incorrect or even offensive renderings (e.g., Braille displays shown…”
SVG: 3D Stereoscopic Video Generation via Denoising Frame Matrix
Dai, P., Tan, F., Xu, Q., Futschik, D., Du, R., Fanello, S., Qi, X., & Zhang, Y. (2024) · arXiv · doi:10.48550/arxiv.2407.00367
Metadata
- Authors
- Hessel, J., Holtzman, A., Forbes, M., Le Bras, R., & Choi, Y.
- Affiliations
- University of Washington (US)
- Venue
- Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing
- Published
- 2021
- Pages
- 7514-7528
- OpenAlex
- W3153469116
- Cited by
- 1,232
- References
- 15
- Citations by year
- 2026: 353 · 2025: 400 · 2024: 313 · 2023: 126 · 2022: 34 · 2021: 6
- FWCI
- 39.16
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion, cc-by)
- Funders
- Allen Institute for Artificial Intelligence
- Defense Advanced Research Projects Agency (N66001-19-2-4031, N66001-19)
- Naval Information Warfare Center Pacific
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image Retrieval and Classification Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- image captioning, reference-free evaluation, CLIP score, CLIP, automatic evaluation metrics, human judgment correlation, cider, spices
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
