CoChat
Sign Up
PreprintCited by 2Multimodal Machine Learning ApplicationsAdvanced Neural Network ApplicationsNatural Language Processing Techniques

The DOI matches its DataCite record.

Uncertainty-Aware Evaluation for Vision-Language Models

Open access
Kostumov, Vasily, Nutfullin, Bulat, Pilipenko, Oleg, Ilyushin, Eugene·2024·arXiv
DOI 10.48550/arxiv.2402.14418arXiv 2402.14418
Open this paper in CoChat — free accountOpen PDF

Abstract

Vision-Language Models like GPT-4, LLaVA, and CogVLM have surged in popularity recently due to their impressive performance in several vision-language tasks. Current evaluation methods, however, overlook an essential component: uncertainty, which is crucial for a comprehensive assessment of VLMs. Addressing this oversight, we present a benchmark incorporating uncertainty quantification into evaluating VLMs. Our analysis spans 20+ VLMs, focusing on the multiple-choice Visual Question Answering (VQA) task. We examine models on 5 datasets that evaluate various vision-language capabilities. Using conformal prediction as an uncertainty estimation approach, we demonstrate that the models' uncertainty is not aligned with their accuracy. Specifically, we show that models with the highest accuracy may also have the highest uncertainty, which confirms the importance of measuring it for VLMs. Our empirical findings also reveal a correlation between model uncertainty and its language model part.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2402.14418

Cites

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

No references could be resolved yet.

Research works printed but not linked: 38

  1. Yue, X., Ni, Y., Zhang, K., Zheng, T., Liu, R., Zhang, G., Stevens, S., Jiang, D., Ren, W., Sun, Y., Wei, C., Yu, B., Yuan, R., Sun, R., Yin, M., Zheng, B., Yang, Z., Liu, Y., Huang, W., Sun, H., Su, Y. & Chen, W. MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI. ArXiv Preprint arXiv:2311.16502. (2023) · arXiv
  2. Ye F. et al. Benchmarking LLMs via Uncertainty Quantification //arXiv preprint arXiv:2401.12794. -2024. · arXiv
  3. Hugo Touvron, Thibaut Lavril, Gautier Izacard, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023. · arXiv
  4. Hugo Touvron, Louis Martin, Kevin Stone, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288, 2023. · arXiv
  5. Deyao Zhu, Jun Chen, Xiaoqian Shen, et al. Minigpt-4: Enhancing vision-language understanding with advanced large language models. arXiv preprint arXiv:2304.10592, 2023. · arXiv
  6. Bai, J., Bai, S., Chu, Y., Cui, Z., Dang, K., Deng, X., Fan, Y., Ge, W., Han, Y., Huang, F., Hui, B., Ji, L., Li, M., Lin, J., Lin, R., Liu, D., Liu, G., Lu, C., Lu, K., Ma, J., Men, R., Ren, X., Ren, X., Tan, C., Tan, S., Tu, J., Wang, P., Wang, S., Wang, W., Wu, S., Xu, B., Xu, J., Yang, A., Yang, H., Yang, J., Yang, S., Yao, Y., Yu, B., Yuan, H., Yuan, Z., Zhang, J., Zhang, X., Zhang, Y., Zhang, Z., Zhou, C., Zhou, J., Zhou, X. & Zhu, T. Qwen Technical Report. ArXiv Preprint arXiv:2309.16609. (2023) · arXiv
  7. Zheng, K., He, X. & Wang, X. MiniGPT-5: Interleaved Vision- and-Language Generation via Generative Vokens. ArXiv Preprint arXiv:2310.02239. (2023) · arXiv
  8. Zhang, D., Li, S., Zhang, X., Zhan, J., Wang, P., Zhou, Y. & Qiu, X. SpeechGPT: Empowering Large Language Models with In- trinsic Cross-Modal Conversational Abilities. The 2023 Conference On Empirical Methods In Natural Language Processing. (2023), https://openreview.net/forum?id=0LXEvcD3dB
  9. Chu, Y., Xu, J., Zhou, X., Yang, Q., Zhang, S., Yan, Z., Zhou, C. & Zhou, J. Qwen-audio: Advancing universal audio understanding via unified large-scale audio-language models. ArXiv Preprint arXiv:2311.07919. (2023) · arXiv
  10. Guo, C., Pleiss, G., Sun, Y., & Weinberger K. Q. (2017). On Calibration of Modern Neural Networks. In Proceedings of the 34th International Conference on Machine Learning, PMLR 70:1321-1330.
  11. Vovk V., Gammerman A., Shafer G. Algorithmic learning in a random world. -New York : Springer, 2005. -T. 29.
  12. Fontana M., Zeni G., Vantini S. Conformal prediction: a unified review of theory and new challenges //Bernoulli. -2023. -T. 29. -1. -C. 1-23.
  13. Romano Y., Sesia M., Candes E. Classification with valid and adaptive coverage //Advances in Neural Information Processing Systems. -2020.
  14. Wang, W., Lv, Q., Yu, W., Hong, W., Qi, J., Wang, Y., ... & Tang, J. (2023). Cogvlm: Visual expert for pretrained language models. arXiv preprint arXiv:2311.03079. · arXiv
  15. Agrawal, H., Desai, K., Wang, Y., Chen, X., Jain, R., Johnson, M., Batra, D., Parikh, D., Lee, S., and Anderson, P. Nocaps: Novel object captioning at scale. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 8948-8957, 2019
  16. Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollar, P., and Zitnick, C. L. Microsoft coco: ´Common objects in context. In Computer Vision-ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13, pp. 740-755. Springer, 2014.
  17. Plummer, B. A., Wang, L., Cervantes, C. M., Caicedo, J. C., Hocken- maier, J., and Lazebnik, S. Flickr30k entities: Collecting region-to-phrase correspondences for richer image-to-sentence models. In Proceedings of the IEEE international conference on computer vision, pp. 2641-2649, 2015
  18. Kembhavi, A., Salvato, M., Kolve, E., Seo, M., Hajishirzi, H., & Farhadi, A. (2016). A diagram is worth a dozen images. In Computer Vision-ECCV 2016: 14th European Conference, Amsterdam, The Nether- lands, October 11-14, 2016, Proceedings, Part IV 14 (pp. 235-251). Springer International Publishing.
  19. Goyal, Yash, et al. "Making the v in vqa matter: Elevating the role of image understanding in visual question answering." Proceedings of the IEEE conference on computer vision and pattern recognition. 2017.
  20. Hudson, Drew A., and Christopher D. Manning. "Gqa: A new dataset for real-world visual reasoning and compositional question answering." Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2019.
  21. Singh, Amanpreet, et al. "Towards vqa models that can read." Pro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2019.
  22. Yu, Weihao, et al. "Mm-vet: Evaluating large multimodal models for integrated capabilities." arXiv preprint arXiv:2308.02490 (2023). · arXiv
  23. Li, Yifan, et al. "Evaluating object hallucination in large vision-language models." arXiv preprint arXiv:2305.10355 (2023). · arXiv
  24. Lu, Pan, et al. "Mathvista: Evaluating mathematical reasoning of founda- tion models in visual contexts." arXiv preprint arXiv:2310.02255 (2023). · arXiv
  25. Fu, Chaoyou, et al. "Mme: A comprehensive evaluation benchmark for multimodal large language models." arXiv preprint arXiv:2306.13394 (2023). · arXiv
  26. Kazemzadeh, Sahar, et al. "ReferItGame: Referring to Objects in Pho- tographs of Natural Scenes." EMNLP 2014.
  27. Yu, Licheng, et al. "Modeling Context in Referring Expressions." ECCV 2016.
  28. Kumar, B., Lu, C., Gupta, G., Palepu, A., Bellamy, D., Raskar, R. and Beam, A., 2023. Conformal Prediction with Large Language Models for Multi-Choice Question Answering. arXiv preprint arXiv:2305.18404. · arXiv
  29. Kadavath, Saurav, et al. "Language models (mostly) know what they know." arXiv preprint arXiv:2207.05221 (2022). · arXiv
  30. Dey, N., Ding, J., Ferrell, J., Kapper, C., Lovig, M., Planchon, E., & Williams, J. P. (2022). Conformal Prediction for Text Infilling and Part- of-Speech Prediction. The New England Journal of Statistics in Data Science, 1(1), 69-83.
  31. Giovannotti, P., & Gammerman, A. (2021, September). Transformer- based conformal predictors for paraphrase detection. In Conformal and Probabilistic Prediction and Applications (pp. 243-265). PMLR.
  32. Maltoudoglou, L., Paisios, A., & Papadopoulos, H. (2020, August). BERT-based conformal predictor for sentiment analysis. In Conformal and Probabilistic Prediction and Applications (pp. 269-284). PMLR.
  33. Quach, V., Fisch, A., Schuster, T., Yala, A., Sohn, J. H., Jaakkola, T. S., & Barzilay, R. (2023). Conformal Language Modeling. arXiv preprint arXiv:2306.10193. · arXiv
  34. Hendrycks, D., Burns, C., Basart, S., Zou, A., Mazeika, M., Song, D., & Steinhardt, J. (2020). Measuring massive multitask language understanding. arXiv preprint arXiv:2009.03300 · arXiv
  35. Gawlikowski, J., Tassi, C. R. N., Ali, M., Lee, J., Humt, M., Feng, J., ... & Zhu, X. X. (2021). A survey of uncertainty in deep neural networks. arXiv preprint arXiv:2107.03342. · arXiv
  36. Angelopoulos, A. N., & Bates, S. (2021). A gentle introduction to con- formal prediction and distribution-free uncertainty quantification. arXiv preprint arXiv:2107.07511. · arXiv
  37. Sadinle, M., Lei, J., & Wasserman, L. (2019). Least ambiguous set- valued classifiers with bounded error levels. Journal of the American Statistical Association, 114(525), 223-234.
  38. Romano, Y., Sesia, M., & Candes, E. (2020). Classification with valid and adaptive coverage. Advances in Neural Information Processing Systems, 33, 3581-3591. LLaVA-v1.6-13B Vicuna-13b 2.21(1) 2.37(2) 2.29(1) 2.67(3) 2.67(3) 2.44(1) 95.26(1) 84.8(1) 87.01(1) 59.75(1) 59.14(2) 77.19(1) Monkey-Chat Qwen-7b 2.88(10) 3.32(10) 3.13(9) 3.51(9) 3.43(9) 3.25(9) 68.42(9) 50.91(9) 57.49(7) 42.72(8) 43.45(8) 52.6(9

Cited by

Sorted by citations

See all 2

Metadata

Authors
Kostumov, V., Nutfullin, B., Pilipenko, O., & Ilyushin, E.
ORCID
Venue
arXiv
Publisher
arXiv
Published
2024
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2402.14418
arXiv
2402.14418
OpenAlex
W4392120762
Cited by
2
References
0
Citations by year
2026: 1 · 2025: 1
Type
posted content
Language
en
Open access
Licence
Locations
  • arXiv (Cornell University) (submittedVersion, cc-by-nc-sa)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
  • Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
Keywords
vision-language models, uncertainty quantification, conformal prediction, visual question answering, benchmark, model evaluation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Uncertainty-Aware Evaluation for Vision-Language Models. https://app.cochat.ai/papers/10.48550/arxiv.2402.14418
The paper
Kostumov, V., Nutfullin, B., Pilipenko, O., & Ilyushin, E. (2024). Uncertainty-Aware Evaluation for Vision-Language Models. arXiv. https://doi.org/10.48550/arxiv.2402.14418