The DOI matches its Crossref record; 58 of the 65 references it cites were found in a registry or an index.
Hallusionbench: An Advanced Diagnostic Suite for Entangled Language Hallucination and Visual Illusion in Large Vision-Language Models
Open accessAbstract
We introduce “HALLUSIONBENCH11“Hallusion” is a portmanteau of “hallucination” and “illusion.”,” a comprehensive benchmark designed for the evaluation of image-context rea-soning. This benchmark presents significant challenges to advanced large visual-language models (LVLMs), such as GPT-4V(ision), Gemini Pro Vision, Claude 3, and LLaVA-1.5, by emphasizing nuanced understanding and interpre-tation of visual data. The benchmark comprises 346 images paired with 1129 questions, all meticulously crafted by human experts. We introduce a novel structure for these visual questions designed to establish control groups. This structure enables us to conduct a quantitative analysis of the models' response tendencies, logical consistency, and various failure modes. In our evaluation on Hallusion-bench, we benchmarked 15 different models, highlighting a 31.42% question-pair accuracy achieved by the state-of-the-art GPT-4V. Notably, all other evaluated models achieve accuracy below 16%. Moreover, our analysis not only high-lights the observed failure modes, including language hal-lucination and visual illusion but also deepens an under-standing of these pitfalls. Our comprehensive case studies within Hallusionbench shed light on the challenges of hallucination and illusion in LVLMs. Based on these in-sights, we suggest potential pathways for their future im-provement. The benchmark and codebase can be accessed at https://github.com/tianyi-labIHallusionBench.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“One such shortcoming is that models may not perceive the image in detail, often missing fine-grained details or ignoring the image entirely (Chen et al., 2024b; Guan et al., 2024; Tong et al., 2024).”
From Does Spatial Cognition Emerge in Frontier Models? (2024) · cites this paper as Guan et al., 2024
The cited source says what the claim attributes to it.
In the citing paper
“Recent studies have revealed that current LVLMs are suffering from multi-modal hallucinations, where models provide responses misaligned with the corresponding visual information [104, 187, 297].”
From A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions (2025) · cites this paper as [104]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
LLaMA: Open and Efficient Foundation Language Models
Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.13971 · Cited by 4,602
VQA: Visual Question Answering
Antol, S., Agrawal, A., Lu, J., Mitchell, M., Batra, D., Zitnick, C. L., & Parikh, D. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.279 · Cited by 4,487
Liu, H., Li, C., Wu, Q., & Lee, Y. J. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-1516 · Cited by 2,428
Proceedings of IEEE International Conference on Computer Vision
(1995) · doi:10.1109/iccv.1995 · Cited by 2,339
A Survey of Large Language Models
Zhao, W. X., Zhou, K., Li, J., Tang, T., Dong, Z., Hou, Y., Zhang, B., Min, Y., Zhang, J., Liu, P., et al. (2026) · Frontiers of Computer Science · doi:10.1007/s11704-026-60308-3 · Cited by 2,093 · cited 1×
“In recent years, Large Language Models (LLMs) [9, 10, 26, 40, 45, 46, 61] have revolutionized the field of machine learning with the ability of language understanding and content generation, offering unprecedented ca-* pabilities and potentials across a multitude of applications.”
BackgroundSelf-citationA Survey of Large Language Models
Zhao, W. X., Zhou, K., Li, J., Tang, T., Wang, X., Hou, Y., Min, Y., Zhang, B., Zhang, J., Dong, Z., et al. (2023) · arXiv · doi:10.48550/arxiv.2303.18223 · Cited by 2,090
Self-citationEvaluating Object Hallucination in Large Vision-Language Models
Li, Y., Du, Y., Zhou, K., Wang, J., Xin, Z., & Wen, J.-R. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.20 · Cited by 1,862
Improved Baselines with Visual Instruction Tuning
Liu, H., Li, C., Li, Y., & Lee, Y. J. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.02484 · Cited by 1,748
Flamingo: a Visual Language Model for Few-Shot Learning
Alayrac, J.-B., Donahue, J., Luc, P., Miech, A., Barr, I., Hasson, Y., Lenc, K., Mensch, A., Millican, K., Reynolds, M., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.14198 · Cited by 1,555
Sparks of Artificial General Intelligence: Early experiments with GPT-4
Bubeck, S., Chandrasekaran, V., Eldan, R., Gehrke, J., Horvitz, E., Kamar, E., Lee, P., Lee, Y. T., Li, Y., Lundberg, S., et al. (2023) · arXiv · doi:10.48550/arxiv.2303.12712 · Cited by 1,526
InstructBLIP: Towards General-purpose Vision-Language Models with Instruction TuningPreprint
Dai, W., Li, J., Li, D., Tiong, A. M. H., Zhao, J., Wang, W., Li, B., Fung, P., & Hoi, S. C. H. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2305.06500 · Cited by 1,372
mPLUG-Owl: Modularization Empowers Large Language Models with MultimodalityPreprint
Ye, Q.-H., Xu, H., Xu, G., Ye, J., Yan, M., Zhou, Y., Wang, J., Hu, A., Shi, P.-C., Shi, Y., et al. (2023) · arXiv.org · doi:10.48550/arxiv.2304.14178 · Cited by 1,318
Self-citationChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning
Masry, A., Long, X., Tan, J. Q., Joty, S., & Hoque, E. (2022) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2022.findings-acl.177 · Cited by 1,191
Gemini: A Family of Highly Capable Multimodal Models
Gemini Team, Anil, R., Borgeaud, S., Alayrac, J.-B., Yu, J., Soricut, R., Schalkwyk, J., Dai, A. M., Hauth, A., Millican, K., et al. (2023) · arXiv · doi:10.48550/arxiv.2312.11805 · Cited by 990
MM-Vet: Evaluating Large Multimodal Models for Integrated CapabilitiesPreprint
Yu, W., Yang, Z., Li, L., Wang, J., Lin, K., Liu, Z., Wang, X., & Wang, L. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2308.02490 · Cited by 984
Self-citationLi, J., Li, D., Savarese, S., & Hoi, S. (2023) · arXiv · doi:10.48550/arxiv.2301.12597 · Cited by 908
A survey on multimodal large language models
Yin, S., Fu, C., Zhao, S., Li, K., Sun, X., Xu, T., & Chen, E. (2024) · National Science Review · doi:10.1093/nsr/nwae403 · Cited by 893
The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)Preprint
Yang, Z., Li, L., Lin, K., Wang, J., Lin, C.-C., Liu, Z., & Wang, L. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2309.17421 · Cited by 883
SEED-Bench: Benchmarking Multimodal LLMs with Generative Comprehension
Li, B., Wang, R., Wang, G., Ge, Y., Ge, Y., & Shan, Y. (2023) · arXiv · doi:10.48550/arxiv.2307.16125 · Cited by 882
A Comprehensive Survey of Deep Learning for Image Captioning
Hossain, M. Z., Sohel, F., Shiratuddin, M. F., & Laga, H. (2019) · ACM Computing Surveys · doi:10.1145/3295748 · Cited by 879 · cited 1×
“These models have demonstrated profound capabilities in various applications and significantly enhance the performance in image reasoning tasks [5, 18, 20, 30, 31, 36, 38, 42, 47].”
BackgroundShow and Tell: Lessons Learned from the 2015 MSCOCO Image Captioning Challenge
Vinyals, O., Toshev, A., Bengio, S., & Erhan, D. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2587640 · Cited by 875
Liu, H., Li, C., Wu, Q., & Lee, Y. J. (2023) · arXiv · doi:10.48550/arxiv.2304.08485 · Cited by 682
MMBench: Is Your Multi-modal Model an All-Around Player?
Liu, Y., Duan, H., Zhang, Y., Li, B., Zhang, S., Zhao, W., Yuan, Y., Wang, J., He, C., Liu, Z., et al. (2025) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-72658-3_13 · Cited by 668
GIT: A Generative Image-to-text Transformer for Vision and LanguagePreprint
Wang, J., Yang, Z., Hu, X., Li, L., Lin, K., Gan, Z., Liu, Z., Liu, C., & Wang, L. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2205.14100 · Cited by 636
MM-REACT: Prompting ChatGPT for Multimodal Reasoning and ActionPreprint
Yang, Z., Li, L., Wang, J., Lin, K., Azarnasab, E., Ahmed, F., Liu, Z., Liu, C., Zeng, M., & Wang, L. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2303.11381 · Cited by 597
PaLM-E: An Embodied Multimodal Language Model
Driess, D., Xia, F., Sajjadi, M. S. M., Lynch, C., Chowdhery, A., Ichter, B., Wahid, A., Tompson, J., Vuong, Q., Yu, T., et al. (2023) · arXiv · doi:10.48550/arxiv.2303.03378 · Cited by 591
Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Models
Zhang, Y., Li, Y., Cui, L., Cai, D., Liu, L., Fu, T., Huang, X., Zhao, E., Zhang, Y., Chen, Y., et al. (2025) · Computational Linguistics · doi:10.1162/coli.a.16 · Cited by 534
Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering
Goyal, Y., Khot, T., Agrawal, A., Summers-Stay, D., Batra, D., & Parikh, D. (2019) · International Journal of Computer Vision · doi:10.1007/s11263-018-1116-0 · Cited by 487
MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models
Zhu, D., Chen, J., Shen, X., Li, X., & Elhoseiny, M. (2023) · arXiv · doi:10.48550/arxiv.2304.10592 · Cited by 469
mPLUG-OwI2: Revolutionizing Multi-modal Large Language Model with Modality Collaboration
Ye, Q., Xu, H., Ye, J., Yan, M., Hu, A., Liu, H., Qian, Q., Zhang, J., & Huang, F. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01239 · Cited by 431
Self-citationnocaps: novel object captioning at scale
Agrawal, H., Desai, K., Wang, Y., Chen, X., Jain, R., Johnson, M., Batra, D., Parikh, D., Lee, S., & Anderson, P. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00904 · Cited by 419
AlpaGasus: Training A Better Alpaca with Fewer DataPreprint
Chen, L., Li, S., Yan, J., Wang, H., Gunaratna, K., Yadav, V., Tang, Z., Srinivasan, V., Zhou, T., Huang, H., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2307.08701 · Cited by 331
Self-citationWoodpecker: hallucination correction for multimodal large language models
Yin, S., Fu, C., Zhao, S., Xu, T. B., Wang, H., Sui, D., Shen, Y., Li, K., Sun, X., & Chen, E. (2024) · Science China Information Sciences · doi:10.1007/s11432-024-4251-x · Cited by 328
LLaVAR: Enhanced Visual Instruction Tuning for Text-Rich Image UnderstandingPreprint
Zhang, Y., Zhang, R., Gu, J., Zhou, Y., Lipka, N., Yang, D., & Sun, T. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2306.17107 · Cited by 278
Socratic Models: Composing Zero-Shot Multimodal Reasoning with Language
Zeng, A., Attarian, M., Ichter, B., Choromanski, K., Wong, A., Welker, S., Tombari, F., Purohit, A., Ryoo, M., Sindhwani, V., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.00598 · Cited by 205
Peng, B., Li, C., He, P., Galley, M., & Gao, J. (2023) · arXiv · doi:10.48550/arxiv.2304.03277 · Cited by 183
Detecting and Preventing Hallucinations in Large Vision Language Models
Gunjal, A., Yin, J., & Bas, E. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i16.29771 · Cited by 161
MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models
Fu, C., Chen, P., Shen, Y., Qin, Y., Zhang, M., Xu, L., Yang, J., Zheng, X., Li, K., Sun, X. W., et al. (2025) · neural information processing systems · doi:10.52202/085713-4899 · Cited by 148
Visual News: Benchmark and Challenges in News Image Captioning
Liu, F., Wang, Y., Wang, T., & Ordóñez, V. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.542 · Cited by 133
Self-citationHolistic Analysis of Hallucination in GPT-4V(ision): Bias and Interference ChallengesPreprint
Cui, C., Zhou, Y., Yang, X., Wu, S., Zhang, L., Zou, J., & Yao, H. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2311.03287 · Cited by 129
Larger language models do in-context learning differently
Wei, J., Wei, J., Tay, Y., Tran, D., Webson, A., Lu, Y., Chen, X., Liu, H., Huang, D., Zhou, D., et al. (2023) · arXiv · doi:10.48550/arxiv.2303.03846 · Cited by 109
LAMM: Language-Assisted Multi-Modal Instruction-Tuning Dataset, Framework, and Benchmark
Yin, Z., Wang, J., Cao, J., Shi, Z., Liu, D., Li, M., Huang, X., Wang, Z., Sheng, L., Bai, L., et al. (2023) · neural information processing systems · doi:10.52202/075280-1158 · Cited by 108
Transform and Tell: Entity-Aware News Image Captioning
Tran, A., Mathews, A., & Xie, L. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01305 · Cited by 93
ScienceQA: a novel resource for question answering on scholarly articles
Saikh, T., Ghosal, T., Mittal, A., Ekbal, A., & Bhattacharyya, P. (2022) · International Journal on Digital Libraries · doi:10.1007/s00799-022-00329-y · Cited by 86
MultiModal-GPT: a vision and language model for dialogue with humans
Gong, T., Lyu, C., Zhang, S., Wang, Y., Zheng, M., Zhao, Q., Liu, K., Zhang, W., Luo, P., & Chen, K. (2023) · arXiv · doi:10.48550/arxiv.2305.04790 · Cited by 70
What Matters in Training a GPT4-Style Language Model with Multimodal Inputs?
Zeng, Y., Zhang, H., Zheng, J., Xia, J., Wei, G., Wei, Y., Zhang, Y., Kong, T., & Song, R. (2024) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2024.naacl-long.440 · Cited by 68
VIGC: Visual Instruction Generation and Correction
Wang, B., Wu, F., Han, X. H., Peng, J., Zhong, H., Zhang, P., Dong, X., Li, W., Li, W., Wang, J., et al. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i6.28338 · Cited by 65
StableLLaVA: Enhanced Visual Instruction Tuning with Synthesized Image-Dialogue DataPreprint
Li, Y., Zhang, C., Yu, G., Wang, Z.-B., Fu, B., Lin, G., Shen, C.-H., Chen, L., & Wei, Y.-C. (2023) · arXiv.org · doi:10.48550/arxiv.2308.10253 · Cited by 37
Self-citationCOVID-VTS: Fact Extraction and Verification on Short Video Platforms
Liu, F., Yacoob, Y., & Shrivastava, A. (2023) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.14 · Cited by 33
Self-citationPrivacyMind: Large Language Models Can Be Contextual Privacy Protection LearnersPreprint
Xiao, Y., Jin, Y., Bai, Y., Wu, Y., Yang, X., Luo, X., Yu, W.-C., Zhao, X.-J., Liu, Y.-C., Gu, Q.-Q., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2310.02469 · Cited by 25
Grounding Visual Illusions in Language: Do Vision-Language Models Perceive Illusions Like Humans?
Zhang, Y., Pan, J., Zhou, Y., Pan, R., & Chai, J. Y. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.348 · Cited by 22
SciGraphQA: A Large-Scale Synthetic Multi-Turn Question-Answering Dataset for Scientific GraphsPreprint
Li, S., & Tajbakhsh, N. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2308.03349 · Cited by 19
MiniGPT-5: Interleaved Vision-and-Language Generation via Generative Vokens
Zheng, K., He, X., & Wang, X. E. (2023) · arXiv · doi:10.48550/arxiv.2310.02239 · Cited by 15
Self-citationMathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts
Lu, P., Bansal, H., Xia, T., Liu, J., Li, C., Hajishirzi, H., Cheng, H., Chang, K.-W., Galley, M., & Gao, J. (2023) · arXiv · doi:10.48550/arxiv.2310.02255 · Cited by 12
DocumentCLIP: Linking Figures and Main Body Text in Reflowed DocumentsPreprint
Liu, F., Tan, H., & Tensmeyer, C. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2306.06306 · Cited by 5
Self-citationModule-wise Adaptive Distillation for Multimodality Foundation Models
Liang, C., Yu, J., Yang, M.-H., Brown, M., Cui, Y., Zhao, T., Gong, B., & Zhou, T. (2023) · neural information processing systems · doi:10.52202/075280-3055 · Cited by 5
Self-citationLOC-ZSON: Language-driven Object-Centric Zero-Shot Object Retrieval and NavigationPreprint
Guan, T., Yang, Y., Cheng, H. H., Lin, M., Kim, R., Madhivanan, R., Sen, A., & Manocha, D. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2405.05363 · Cited by 3
Self-citationStrohm, C. (2015) · Religion in Geschichte und Gegenwart · doi:10.1163/2405-8262_rgg4_sim_03081 · Cited by 1
Research works printed but not linked: 12
- 2023, Gpt-4v(ision) system card.
- Antol, 2015, V qa: Visual question answering, 2425
- Awadalla, 2023, Openflamingo
- Bai, 2023, Qwen-vl: A frontier large vision-language model with versatile abilities, ArXiv, abs/2308.12966 · arXiv
- Chiang, 2023, Vicuna: An open-source chatbot impressing gpt-4 with 90% * chatgpt quality
- Driess, 2023, Palm-e: An embodied multimodallanguage model
- Fu, 2023, Mme: A compre-hensive evaluation benchmark for multimodallarge language models, arXiv preprint
- Li, 2023, To-wards understanding in-context learning with contrastive demonstrations and saliency maps, arXiv preprint
- Liu, 2023, Aligning large multi-modal model with robust instruction tuning, arXiv preprint
- Yang, 2023, Embodied multi-modal agent trained by an 11m from a parallel textworld
- Yin, 2023, A survey on multimodallarge language models, arXiv preprint
- Zhai, 2023, Investigating the catastrophic for-getting in multimodallarge language models, arXiv preprint
Cited by
Sorted by citations
Huang, L., Yu, W., Ma, W., Zhong, W., Feng, Z., Wang, H., Chen, Q., Peng, W., Feng, X., Qin, B., et al. (2025) · ACM Transactions on Information Systems · doi:10.1145/3703155 · Cited by 4,688
BLINK: Multimodal Large Language Models Can See but Not Perceive
Fu, X., Hu, Y., Li, B., Feng, Y., Wang, H., Lin, X., Roth, D., Smith, N. A., Ma, W.-C., & Krishna, R. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73337-6_9 · Cited by 345
Li, Z., Wu, X., Du, H., Liu, F., Nghiem, H. T., & Shi, G. (2025) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw67362.2025.00147 · Cited by 91
Self-citationLlaVA-CoT: Let Vision Language Models Reason Step-By-Step
Xu, G., Jin, P., Wu, Z., Li, H., Song, Y., Sun, L., & Yuan, L. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.00202 · Cited by 118
Cirone, K. D., Akrout, M., Abid, L., & Oakley, A. M. (2024) · JMIR Dermatology · doi:10.2196/55508 · Cited by 43
Metadata
- Authors
- Guan, T., Liu, F., Wu, X., Xian, R., Li, Z., Liu, X., Wang, X., Chen, L., Huang, F., Yacoob, Y., Manocha, D., & Zhou, T.
- Affiliations
- University of Maryland, College Park (US)
- ORCID
- Venue
- 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2024-06-16
- Pages
- 14375-14385
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W4402726948
- Cited by
- 166
- References
- 65
- Citations by year
- 2026: 77 · 2025: 76 · 2024: 13
- FWCI
- 34.38
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- ARO (W911NF2310046,W911NF2310352)
- NSF (IIS-2147276 FAI)
- DOD (N00014-22-1-2335,FA9550-23-1-0048)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- hallucinations, optical illusions, large vision-language models, benchmark, failure modes, GPT-4V, LLaVA-1.5, control groups, visual question answering
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
