The DOI matches its Crossref record; 19 of the 19 references it cites were found in a registry or an index.
Object Hallucination in Image Captioning
Open accessAbstract
Despite continuously improving performance, contemporary image captioning models are prone to "hallucinating" objects that are not actually in a scene.One problem is that standard metrics only measure similarity to ground truth captions and may not fully capture image relevance.In this work, we propose a new image relevance metric to evaluate current models with veridical visual labels and assess their rate of object hallucination.We analyze how captioning model architectures and learning objectives contribute to object hallucination, explore when hallucination is likely due to image misclassification or language priors, and assess how well current sentence metrics capture object hallucination.We investigate these questions on the standard image captioning benchmark, MSCOCO, using a diverse set of models.Our analysis yields several interesting findings, including that models which score best on standard sentence metrics do not always have lower hallucination and that models which hallucinate more tend to make errors driven by language priors. * Denotes equal contribution.NBT: A woman talking on a cell phone while sitting on a bench.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“However, it is also apparent that deep learning based generation is prone to hallucinate unintended text, which degrades the system performance and fails to meet user expectations in many real-world scenarios.”
From Survey of Hallucination in Natural Language Generation (2023) · cites this paper as [222]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Large language model (LLM) systems, such as ChatGPT1 or Gemini2, can show impressive reasoning and question-answering capabilities but often ‘hallucinate’ false outputs and unsubstantiated answers3,4.”
From Detecting hallucinations in large language models using semantic entropy (2024) · cites this paper as [4]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“This is because there is another contributor of hallucinations - training and modeling choices of neural models [121, 215, 222, 263].”
From Survey of Hallucination in Natural Language Generation (2023) · cites this paper as [222]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 2×
“All the baseline models employ features extracted from the fourth layer of ResNet-101 (He et al., 2016), except for the GAN model which employs ResNet-152.”
MethodHochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 9×
“We also consider LRCN (Donahue et al., 2015) which considers visual information at each time step, as opposed to just initializing the LSTM hidden state with extracted features.”
BackgroundMethodInfluentialMicrosoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460
Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
Xu, K., Ba, J., Kiros, R., Cho, K., Courville, A., Salakhutdinov, R., Zemel, R., & Bengio, Y. (2015) · arXiv · doi:10.48550/arxiv.1502.03044 · Cited by 8,248 · cited 2×
“For attention based models, we consider Att2In (Rennie et al., 2017), which is similar to the original attention based model proposed by (Xu et al., 2015), except the image feature is only input into the cell gate as this was shown to lead to better performance.”
MethodResultLong-term recurrent convolutional networks for visual recognition and description
Donahue, J., Hendricks, L. A., Guadarrama, S., Rohrbach, M., Venugopalan, S., Darrell, T., & Saenko, K. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298878 · Cited by 5,228 · cited 2×
“We also consider LRCN (Donahue et al., 2015) which considers visual information at each time step, as opposed to just initializing the LSTM hidden state with extracted features.”
MethodSelf-citationVisual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations
Krishna, R., Zhu, Y., Groth, O., Johnson, J., Hata, K., Kravitz, J., Chen, S., Kalantidis, Y., Li, L.-J., Shamma, D. A., et al. (2017) · International Journal of Computer Vision · doi:10.1007/s11263-016-0981-7 · Cited by 5,107 · cited 2×
“his is perhaps because bounding box proposals come from the MSCOCO detection task and are thus “in-domain” as opposed to the TopDown-BB model which relies on proposals learned from the Visual Genome (Krishna et al., 2017) dataset. Second, frequently training models with the self-critical loss actually increases the am…”
MethodCIDEr: Consensus-based image description evaluation
Vedantam, R., Zitnick, C. L., & Parikh, D. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299087 · Cited by 4,845 · cited 6×
“Likewise, our work is related to other work which aims to build better evaluation tools ((Vedantam et al., 2015), (Anderson et al., 2016), (Cui et al., 2018)).”
BackgroundMethodInfluential(2005) · Cited by 4,152
Self-Critical Sequence Training for Image Captioning
Rennie, S. J., Marcheret, E., Mroueh, Y., Ross, J., & Goel, V. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.131 · Cited by 2,231 · cited 5×
“For non-attention based methods we consider the FC model from Rennie et al. (2017) which incorporates visual information by initializing the LSTM hidden state with high level image features.”
MethodInfluentialSPICE: Semantic Propositional Image Caption Evaluation
Anderson, P., Fernando, B., Johnson, M., & Gould, S. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46454-1_24 · Cited by 1,986 · cited 3×
“Likewise, our work is related to other work which aims to build better evaluation tools ((Vedantam et al., 2015), (Anderson et al., 2016), (Cui et al., 2018)).”
BackgroundMethod
Research works printed but not linked: 3
- Andrej Karpathy and Li Fei-Fei. 2015. Deep visual- semantic alignments for generating image descrip- tions. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3128-3137.
- Mert Kilickaya, Aykut Erdem, Nazli Ikizler-Cinbis, and Erkut Erdem. 2016. Re-evaluating automatic metrics for image captioning. In European Chapter of the Association for Computational Linguistics.
- Anna Rohrbach, Makarand Tapaswi, Atousa Torabi, Tegan Maharaj, Marcus Rohrbach, Sanja Fi- dler Christopher Pal, and Bernt Schiele. 2017a. The Joint Video and Language Understand- ing Workshop: MovieQA and The Large Scale Movie Description Challenge (LSMDC). https://sites.google.com/site/ describingmovies/lsmdc-2017.
Cited by
Sorted by citations
Items 301–400 of 427
PROTECT YOURSELF FROM AI HALLUCINATIONS: EXPLORING ORIGINSAND BEST PRACTICES
Dannhoferová, J., & Jedlička, P. (2024) · International Conference Economic Competitiveness and Sustainability · doi:10.11118/978-80-7509-990-7-0049
UHDF: Hallucination Detection Using Open Source Models Beyond Close Source Models Methods
Liu, D., Xu, B., Zhao, Z., Xu, B., & Yang, M. (2024) · Lecture notes in computer science · doi:10.1007/978-981-97-9443-0_34
Element-Centered Multi-granularity Network for Dense Video Captioning
Dang, X., Wang, G., Wu, X., & Qin, Z. (2024) · Lecture notes in computer science · doi:10.1007/978-981-97-8792-0_31
Multi-Branch Information Network for Advanced Image Captioning
Song, C. (2024) · International Conference on Artificial Intelligence, Internet of Things and Cloud Computing Technology (AIoTC) · doi:10.1109/aiotc63215.2024.10748270
“Therefore, following [10], we use the CHAIRi and CHARIs metrics on the Karpathy test split of the MSCOCO, as shown in Tab.”
MethodDeneb: A Hallucination-Robust Automatic Evaluation Metric for Image Captioning
Matsuda, K. M., Wada, Y., & Sugiura, K. (2024) · Lecture notes in computer science · doi:10.1007/978-981-96-0908-6_10
Wang, Z., Huo, H., Xu, R., Yang, S., & Fang, T. (2024) · International Conference on Machine Learning and Natural Language Processing (MLNLP) · doi:10.1109/mlnlp63328.2024.10800230
Mitigating Hallucination in Visual Language Model Segmentation with Negative Sampling
Chung, T. L., Cheng, J. S., & Pang, J. (2024) · International Symposium on Chinese Spoken Language Processing (ISCSLP) · doi:10.1109/iscslp63861.2024.10800691
Briefness-Oriented Prompting Elicits Faithfulness in LVLM
Zhang, J., & Yang, Y. (2025) · Communications in computer and information science · doi:10.1007/978-981-96-2914-5_24
Sui, Y., Huang, W., Yang, W., Zhao, C., Ren, J., & Wang, J. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10889297
“In addition, referring to the research method of Rohrbach et al. [33], we adopt the CHAIR metrics on COCO [27] to quantify automatic hallucination evaluation by calculating the ratio of all objects mentioned in the description but not present in the ground-truth label set.”
Concept-wise Feature Sampling: Mitigating Object Hallucinations for Large Vision-Language Models
Gao, J. (2025) · International Conference on Software Engineering and Computer Science (CSECS) · doi:10.1109/csecs64665.2025.11009288
“Earlier multi-modal research [12] defines the hallucination as the tendency of generative models to produce plausible content that does not actually exist within an image.”
Enhancing Factual Question Answering with Web Knowledge and Chain-of-Reevaluation
Zuo, Y., Yuan, J., & Yao, Z. (2025) · International Conference on Computer Supported Cooperative Work in Design (CSCWD) · doi:10.1109/cscwd64889.2025.11033354
Self-Improved Visual Preference Alignment by Iteratively Generating Holistic WinnerPreprint
Feng, Z., Chen, K., Zhang, J., Yang, J., & Song, M. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5352865
Probing Fine-Grained Hierarchical Concept Comprehension and Generation in Large Language Models
Sun, K., Bai, Y., Tu, S., Li, J., & Hou, L. (2025) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/taslpro.2025.3592339
Mitigating Multimodal Hallucinations Via Uncertainty-Aware Adaptive DecodingPreprint
Yang, Z., Song, X., & Shi, G. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5404834
Tran, H.-T., Truong, T.-D., & Luu, K. (2025) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw67362.2025.00527
“Object hallucination [14, 26, 36] is one of the most extensively studied challenges, where LVLMs generate inconsistent sequences of outputs given the ground-truth objects of the visual component.”
ECD: Efficient Contrastive Decoding with Probabilistic Hallucination Detection
Fieback, L., Balar, N., Spiegelberg, J., & Gottschalk, H. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-06109-6_2
Fang, X., Yuan, H. R., Li, H., Kou, J., Gu, C., Zhang, W., Duan, X., & Fang, Y. (2024) · International Conference on Universal Village (UV) · doi:10.1109/uv63228.2024.11189217 · Cited by 4
“This overreliance leads models to hallucinate based on frequently co-occurring object patterns learned during pretraining [94]–[96], rather than grounding predictions in the visual input.”
Seeing is Believing? Enhancing Vision-Language Navigation using Visual Perturbations
Zhang, X., Li, J., Xu, Y., Hu, Z., & Hong, R. (2025) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn64981.2025.11228883 · Cited by 4
Bi-directional Semantic Alignment for Large Vision Language Model Training via Echo-Former
Wang, Y., Liu, Q., Li, Z., Wang, S., Guo, S., & Wei, Z. (2025) · Lecture notes in computer science · doi:10.1007/978-981-95-3346-6_6
Mitigating Object Hallucination Through Assembled Chain-of-Thought Reasoning
Wang, X., Ma, X., Ding, S., Chao, L. S., & Wong, D. F. (2025) · Lecture notes in computer science · doi:10.1007/978-981-95-3346-6_4
Lee, S., Ko, D.-S., & Song, S. (2025) · The Journal of Korean Institute of Information Technology · doi:10.14801/jkiit.2025.23.11.279
Akilesh, S., Tulasi Raman, R., Sasithradevi, A., Kanimozhi, S., & Prakash, P. (2025) · IEEE Madhya Pradesh Section Conference (MPCON) · doi:10.1109/mpcon66082.2025.11256560
“Additionally, environmental changes or contamination over time may alter the scene, complicating the accuracy of reconstructions based on physical evidence alone [11] [14].”
Yuxia, S. (2025) · International Conference on Machine Learning, Pattern Recognition and Automation Engineering (MLPRAE) · doi:10.1109/mlprae67267.2025.11290735
“There is some work on the hallucination generation of multimodal models likewise, Rohrbach [25] and Biten [26] illustrated the phenomenon of multimodal models generating descriptions or illustrations that are inconsistent, incomplete, and nonexistent with the target image.”
O que são as “alucinações” da Inteligência Artificial
Choi, A., & Mei, K. (2025) · doi:10.64628/ade.g7nwjxcaf
Lymperaiou, M., Filandrianos, G., Dimitriou, A., Voulodimos, A., & Stamou, G. (2025) · Communications in computer and information science · doi:10.1007/978-3-032-08330-2_5
Fine-Grained Image Captioning via Dynamic Query and Deformable Cross-Attention
Xie, X., Chen, C., Cai, T., & Liao, X. (2025) · International Conference on Image Processing, Computer Vision and Machine Learning (ICICML) · doi:10.1109/icicml67980.2025.11333448
“Thus, traditional static design introduces two key bottlenecks : (1) The fixed query budget results in over-computation on simple images and insufficient coverage of complex scenes, leading to omissions or hallucinations in generated captions [2], [6].”
Manifold-Guided Geometric Enhancement for Reliable Multimodal Interaction in Consumer Electronics
Xu, X., Pan, Y., Zhang, X., Chen, X., & Tsang, K.-F. (2026) · IEEE Transactions on Consumer Electronics · doi:10.1109/tce.2026.3655466
“We focus on POPE [35] and CHAIR [36].”
CAGE: Counterfactual Attribution-Guided Decoding for Grounded Vision–Language Generation
Jiang, X., & Huang, J. (2025) · International Conference on Image Processing, Computer Vision and Machine Learning (ICICML) · doi:10.1109/icicml67980.2025.11333730
“…hallucination : the model relies on language priors rather than evidence in the pixels, producing objects that are not present or attributes that contradict the image [1], which is especially problematic in assistive or safety-critical applications where users expect visually grounded descriptions.”
InfluentialOn the Taxonomy, Tasks, and Open-Challenges for Multimodal Large Language Models
闫乐成, Li, R., Geng, J., Li, Q., Wu, M., Wang, Z., Li, W., Ji, T., Jiang, S., & Lyu, C. (2025) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/smc58881.2025.11342688
“CHAIR [95] assesses object illusions in image captioning by extracting nouns to form a ground truth set and calculating the proportion of nonexistent objects.”
A Deconfounded Model for Fault Diagnosis and Retrieval of Fault Texts
Lin, B., Chen, Z., Yang, S., Zhang, Z., & 彭红雷 (2025) · International Conference on Big-data Service and Intelligent Computation (BDSIC) · doi:10.1145/3778265.3778273
Evaluating and Mitigating Relationship Hallucinations in Large Vision-Language Models
Wu, M.-K., Li, J.-L., Ji, J.-Y., Hao, F., Sun, X.-S., & Ji, R.-R. (2026) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2026.3656175 · Cited by 5
Multi-Perspective Visual Contrastive Decoding for Reliable Assistance
Pan, B., Shi, H., & Gao, X. (2026) · ACM Transactions on Internet of Things · doi:10.1145/3785360
“However, as comprehensively documented by Rohrbach et al. [43] through empirical studies across multiple datasets, such ensemble methods often face significant practical challenges in real-world deployment.”
Cognitive Consistency Verification: Neuro-Symbolic Approach for Mitigating VLM Hallucination
Jo, H. S., & Kwon, Y. (2025) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata66926.2025.11402484
“Vision–language models (VLMs) generate fluent explanations, yet they frequently hallucinate—producing content that does not align with visual evidence [1-4].”
OPTiCAL: An Abstract Positional Reasoning Benchmark for Vision Language Models
Driggers-Ellis, C., Ayoubi, G., & Grant, C. (2025) · 2025 IEEE International Conference on Data Mining Workshops (ICDMW) · doi:10.1109/icdmw69685.2025.00171
“Object hallucinations occur when models incorrectly classify an object’s category, attributes, or its relationship with other objects and are usually studied in image captioning and VQA tasks [4], [5].”
Zero-shot video emotional description generation based on counterfactual emotional causal reasoning
Ke, L., Huang, Y., Liu, Y., Qian, Y., Tang, C., Zou, Z., & Zhang, H. (2026) · Scientia Sinica Informationis · doi:10.1360/ssi-2025-0435
Do Not Leave a Gap: Hallucination-Free Object Concealment in Vision-Language Models
Guesmi, A., & Shafique, M. H. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37041-9_26
Learning to detect visual relations
Peyre, J. (2019) · theses.fr (ABES) · doi:10.70675/e491bc42z84b1z4107zadf9z95414bcd6961
“, 2018a] in image captioning yet based on top-down and bottom-up approaches have been shown to produce fluent language before using visual information from image regions [Rohrbach et al., 2018].”
BackgroundInfluentialDeep Multimodal Learning for Vision and Language Processing
Cadène, R. (2020) · theses.fr (ABES) · doi:10.70675/674b1e7aza957z4fc5za03czde9aab142a84 · Cited by 1
Data-Based Natural Language Generation : Evaluation and Explainability
Faille, J. (2023) · theses.fr (ABES) · doi:10.70675/3e73dd43z7aa6z4405za5cez27c78b408a3c
Genest, P.-Y. (2024) · theses.fr (ABES) · doi:10.70675/88c65002z4b8cz466dz923bz13954d6729bf
AdViP: Aligning Multi-modal LLMs via Adaptive Vision-enhanced Preference Optimization
Lu, J., Li, J., Gao, Y., Wu, J., Wu, J., Wang, X., & He, X. (2026) · International Journal of Computer Vision · doi:10.1007/s11263-026-02840-0 · Cited by 8
“5 7B and 13B, our AdViP demonstrates a lower hallucination rate compared to GPT-4 on Object HalBench, AMBER Generative, and AMBER Score.”
InfluentialBaid, A., Xue, Z., & Grauman, K. L. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37038-9_4 · Cited by 4
Mitigating Language Prior-Induced Hallucinations via Bi-Level Contrastive Decoding
Xia, T., Liu, H., Yang, L., & Wang, Y. (2026) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp55912.2026.11463602
Wang, J., Li, F., & Liu, R. (2026) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp55912.2026.11460316
ORSc: Object-Aware Reinforcement with Semantic Consistency for Hallucination Mitigation in MLLMs
He, J., Shi, X., Xie, H. B., Zhang, Y., & Shang, M. (2026) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp55912.2026.11464193
Multi-Agent Brainstorming for Interpreting and Mitigating Hallucination in Multimodal-LLM
Zheng, L., Niu, Z., Wang, Z., Xu, X., Liu, H., & Gao, H. (2026) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp55912.2026.11464937
RetLLM: Training and Data-Free MLLMs for Multimodal Information Retrieval
Su, D., & Wang, D. (2026) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp55912.2026.11461975
CVSTIM: Mitigating Object Hallucination in Mllms Via Co-Occurrence Guided Visual Stimulation
Wang, L., & Kuang, P. (2026) · IEEE International Conference on Acoustics, Speech, and Signal Processing · doi:10.1109/icassp55912.2026.11464584
VCE: A Zero-Cost Hallucination Mitigation Method of LVLMs via Visual Contrastive Editing
Huang, Y., Li, Y., Tie, G., Qu, X., Zhou, P., Wang, H., Zou, Z., Sun, H., & Li, X. (2026) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp55912.2026.11464698
“We evaluate the object hallucination rate using CHAIR s and CHAIR I metrics [19].”
Kim, S., Lee, G., Shim, K., & Shim, B. (2026) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp55912.2026.11463505
Controlling Multimodal Llms Via Reward-Guided Decoding
Mañas, O., D'Oro, P., Sinha, K., Romero-Soriano, A., Drozdzal, M., & Agrawal, A. (2025) · IEEE International Conference on Computer Vision · doi:10.1109/iccv51701.2025.00137 · Cited by 2
“We evaluate our method on two standard object hallucination benchmarks, CHAIR [38] (5k) and AMBER [46] (1k), and report instance-level and sentence-level hallucination rates (the inverse of object precision), using the metrics used in respective benchmarks –”
SHIFT: Smoothing Hallucinations by Information Flow Tuning for Multimodal Large Language Models
Wang, S., Zhang, Y., Zhu, Y., liu, E., Li, J., Liu, Y., & Ji, X. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.00347
“When it comes to MLLMs, they may generate answers contradicting the input images [29, 38], indicating that they are prone to faithful hallucinations.”
Lu, W. (2026) · Journal of Computer Science and Artificial Intelligence · doi:10.54097/ge4cjn12
“Rohrbach et al.'s original work on object hallucination in image captioning demonstrated that models tend to generate descriptions for objects absent from the image, due to statistical associations among different classes of samples during training [8].”
Fazli, M., Wei, B., Sari, A., & Zhu, Z. (2026) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61042.2026.00127 · Cited by 12
“The CHAIR benchmark [28] evaluates object hallucination in image captioning on MSCOCO 2014 im-Table ages [20] by calculating two metrics: CHAIR i , the proportion of hallucinated objects relative to all mentioned objects, and CHAIR s , the ratio of sentences containing hallucination.”
GHOST: Getting to the Bottom of Hallucinations with A Multi-round Consistency Benchmark
Vibashan, V. S., Chang, N., Schmalfuß, J., Patel, V. M., Yu, Z., & Álvarez, J. M. (2026) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61042.2026.00596
“This trend has also been observed in other benchmarks, such as POPE [38] and CHAIR [49].”
Moradi, M. M., & Mudur, S. P. (2026) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61042.2026.00305
“To evaluate the impact of our filtering and feedback mechanisms on hallucination, we adopt the CHAIR metric [34], which measures the percentage of captions that contain We further quantify the effect of intra-stage filtering and inter-stage feedback on reducing irrelevant captions by measuring the…”
Bridging the Gap between Vision and Text for unsupervised text-only captioning
Wang, L., Qiu, H., Wen, H., Meng, F., Wu, Q., & Hongliang, L. (2026) · Pattern Recognition · doi:10.1016/j.patcog.2026.113937
Retrieval-Guided and Semantically Grounded Image Captioning for Open-Domain Scenes
Lin, S., Xie, X., Yang, Z., & Chen, C. (2026) · Mathematics · doi:10.3390/math14101667
Fine-Grained Attention Enhancement for Mitigating Hallucinations in LVLMs
Yang, J., Yao, H., Chen, X., & Jiang, S. (2026) · Communications in computer and information science · doi:10.1007/978-981-95-9493-1_9
Aljedaani, W., Alsulaymi, R., Alqahtani, G., & Feng, Y. (2026) · International Web for All Conference (W4A) · doi:10.1145/3800424.3800454
“The behavior aligns with evidence that models rely on language priors and typical co-occurrences [65], and with recent work documenting hallucinations and mitigation in vision-language systems [48, 51].”
Understanding Multimodal LLM Behavior in Arabic Image Captioning
Dahimi, K., Cherroun, H., Belabbaci, A., & Haouhat, A. (2026) · International Conference on Intelligent Environments (IE) · doi:10.1109/ie69249.2026.11539040
“We assess model performance using established quantitative metrics that measure caption quality [8], [9], [10], semantic alignment with visual content, and the occurrence of halluci-nated information [11], [12].”
Khalzaa, K., Karungaru, S., Terada, K., & Chimed-Ochir, T.-A. (2025) · International Symposium on Computer Science and Educational Technology (ISCSET) · doi:10.1109/iscset65760.2025.11540509
“However, these models frequently suffer from a key limitation: hallucination , where captions contain objects or actions not present in the image [4].”
Mitigating multimodal hallucinations through visual attention tracing and origin-point regeneration
Li, B. J., Yu, H., Han, Y., Ren, J., Dai, M., & Bao, S. (2026) · Scientific Reports · doi:10.1038/s41598-026-52390-1 · Cited by 1
“The CHAIR metric 20 serves as a foundational standard in image captioning, gauging object hallucination by calculating the proportion of unsupported entities against ground-truth annotations at both sentence (CHAIR S ) and instance (CHAIR I ) granularities.”
Promoting Interpretability in Image Captioning Through Attribution Segmentation Alignment
Catanpatan, W. Y., Aballe, Z., & Maderazo, C. V. (2026) · International Conference on Multi-scale Artificial Intelligence (MAI) · doi:10.1109/mai69289.2026.11543767
“Although Bootstrap-ping Language-Image Pre-training (BLIP) advances large-scale vision-language pre-training for generation [8], hallucinations persist when language priors override local visual evidence [1].”
Sree, M. L., Babu, N. J., Aaradhya, M. K., Chaithanya, A. K., & Latha, B. M. (2026) · International Conference on Signal, Systems, and Computing for Next-Gen Automation (ICSSCNA) · doi:10.1109/icsscna68616.2026.11546829
“Most of such works don’t explicitly validate the reasoning process so they may exhibit inconsistencies across modalities and hallucinations [10].”
InfluentialMitigating Multimodal Hallucination Through Effective and Perception-Aware Granularity Alignment
Zhang, Z., Lin, J., Zhang, Y., Xia, R., & Zhou, C. (2026) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/taslpro.2026.3703183
“First, we adopt the Caption Hallucination Assessment with Image Relevance (CHAIR) framework [27], which quantifies object hallucination by comparing generated objects with ground-truth annotations.”
Implicit Affective Steering: A Design Probe for Generative Curatorial Storytelling
Ma, M. C., Shi, L., Li, H., Cheng, J., & Kirk, D. S. (2026) · Designing Interactive Systems Conference · doi:10.1145/3800645.3812861
“Narratives occasionally contained ungrounded details (hallucination)[29, 37, 42, 55], inconsistent with the selected artworks.”
Mitigating Hallucinations in Vision-Language Models via Contextual Entropy Calibration
Yin, X., Peng, D., & Wei, W. (2026) · International Conference on Multimedia Retrieval · doi:10.1145/3805622.3810847 · Cited by 1
“5-VL across five benchmarks: CHAIR [33], POPE [24], MME [13], AMBER [39] and LLaVA-Bench [28].”
Audit-and-Repair Indexing: Probabilistic Caption Indexes for Robust Multimodal Retrieval
Liu, Y. (2026) · International Conference on Multimedia Retrieval · doi:10.1145/3805622.3810829
“Generated captions are not always correct: they may contain wrong attributes (e.g., color, material, count, action) or miss important details [2, 10, 35].”
Visual embedding retrieval outperforms caption-based retrieval for fish-disease image classification
Liao, Y.-H., Shih, C.-F., & Chang, C.-C. (2026) · Scientific Reports · doi:10.21203/rs.3.rs-9575788/v1
Liu, X., Fang, X., Yan, J., Fang, C., & Fang, Y. (2026) · Pattern Recognition · doi:10.1016/j.patcog.2026.114272
Dai, X. (2026) · Frontiers in Robotics and AI · doi:10.3389/frobt.2026.1781874
“These studies show that models can affirm objects that are not visually supported, especially when objects commonly co-occur with visible objects or appear frequently in instruction data (Rohrbach et al., 2018; Li et al., 2023b).”
Paying More Attention to Visual Tokens in Self-evolving Large Multimodal Models
Venkatraman, S., Thawkar, R., Thawakar, O., Anwer, R. M., Cholakkal, H., Khan, S., & Khan, F. S. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37464-6_3
See Only When Needed: Context-Aware Attention Intervention for Mitigating Hallucinations in LVLMs
Lei, Y., Lyu, W., Du, Y., Zhen, X., Snoek, C. G. M., & Shao, L. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37422-6_10 · Cited by 1
Yu, J., Zhuge, Y., Zhang, L., Huang, Z., Zhou, W., Wang, D., Lu, H., He, Y., & Chen, L. (2026) · International Journal of Computer Vision · doi:10.1007/s11263-026-02930-z
FaithViP: A Benchmark for Hallucinations With Synthetic Visual Prompts in Large Multi-Modal Models
Gao, Y., Xu, N., Tian, H., Wei, S., & Liu, A.-A. (2026) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2026.3708848
“Research on LMM hallucinations focuses on object [31], [32], attribute [33], and relation hallucinations [34].”
ADAPT: Attention Dynamics Alignment with Preference Tuning for Faithful MLLMs
Yao, Z., Fu, Z., Zheng, Z., Li, J., Tu, Y., & Mao, Z. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37602-2_29
Information-Regularized Attention for Visual-Centric Reasoning
Sun, G. X., Wang, X., Patel, Y., Liu, M. Y., Tao, Z., & Krishnan, P. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37321-2_30
LASER: A Corrective Lens for LVLMs via Visual Attention Preservation and Sink Suppression
Yuan, B., Wang, Z., Luo, Y., Wang, S., & Huang, Z. H. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37211-6_20
Secure and Transparent Exam Proctoring: Leveraging Blockchain for Enhanced Integrity and Trust
Bilgaiyan, S., Singh, M., Chouhan, M., Saha, B. A., Parida, C., & Gourisaria, M. K. (2026) · Lecture notes in networks and systems · doi:10.1007/978-981-95-7238-0_12
Visual Calibration for Text-Guided Video Summarization
Wu, Y. M., & Li, B. (2026) · International Conference on Signal Image Processing and Communication (ICSIPC) · doi:10.1109/icsipc69751.2026.11584129
“Moreover, vision-language generation remains vulnerable to omission and hallucination, meaning that the generated text may not always be fully grounded in the original visual input [30]–[32].”
HIVE: Understanding Post-hallucination Reasoning in Vision Language Models
He, F., Wang, Z., Wang, Q., Guan, Q., Liu, D., Tang, R., & Li, Q. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37271-0_34
Fu, X., Fan, C., & Yu, M. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-2852-2_38
Chen, K., Hu, Y., Zhou, Q., Zhu, Z., & Luo, W. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-3504-9_17
Zhang, Z., Liu, Y., & Yang, S. Y. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-3432-5_48
Commonsense Violations in Multimodal Large Language Models: A Knowledge-Guided Analysis
Yan, H., Cui, S., Li, F., Xie, P., & Chi, C. (2026) · Communications in computer and information science · doi:10.1007/978-981-92-3560-5_30
Focus on This, Not That! Mitigating Hallucinations in LVLMs via Discriminative Visual Enhancement
Jiang, Y., Qiang, P., & Tan, H. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-3444-8_28
VMFormer: Visual Clues-Guided Multi-Stage Transformer for Image Captioning
Ren, Y., Chen, X., Yuan, H. J., Li, J., Song, P., Chen, L., Ouyang, W., Zhang, L., & Guo, J. (2026) · IEEE Transactions on Image Processing · doi:10.1109/tip.2026.3712347
“We evaluate the e ff ectiveness of our VMFormer in mitigating hallucinations using the robust COCO split [66], following the protocol in [30], [96].”
SCST-Based Advantage Exploitation for Captioning: An Application in Traffic Scene Understanding
Zadeh, D. S., Moshiri, B., & Basir, O. A. (2026) · Conference proceedings - Canadian Conference on Electrical and Computer Engineering · doi:10.1109/ccece68150.2026.11610500
Eliminating Object Hallucination in MLLMs via Convex Potential Flow Intervention
Shi, Z., Liu, R., Yu, S., & Shirahata, K. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-31441-3_33
Mai, X. T., Tran, H. T., & Tran, T. A. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7200053
CANVAS : Captioning Art with Narrative Visual-Audio AI SystemsPreprint
Nagarajan, V. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6668618
Nath, S., Basu, A., Dasgupta, S., & Das, S. (2025) · Indian Conference on Computer Vision, Graphics and Image Processing · doi:10.1145/3774521.3774542 · Cited by 7
“These models are frequently susceptible to producing plausible sounding descriptions, therefore, being grammatically correct, but such descriptions often comes with inconsistent visual content, a phenomenon commonly referred to as hallucination [19, 23, 26].”
Beyond Co-existence: Measuring Attribute Binding Hallucinations in Audio-Language Models
Kim, M., Kwon, M., Ma, M., Pham, T. X., & Kim, J. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-31930-2_26
Seeing Red, Thinking Bad: Color Bias in Vision Language Models
Ide, K., Yamada, R., Fukuhara, Y., Kataoka, H., & Satoh, Y. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-31583-0_18
“While hallucination in VLMs has been widely studied [28,17,8,15,13], we show that reduced text contrast can increase hallucinated outputs even when the affected words are task-irrelevant.”
RadReport-GNN: A Graph-Based Multimodal Fusion Architecture for Automated Medical Report Generation
Safari, F., & Mohammadi, M. (2026) · Lecture notes in networks and systems · doi:10.1007/978-3-032-24810-7_21
Visual Dialogue System with Self-Verification Pipeline for Hallucination Mitigation
Gandikota, L. R., Gudipati, V. R. S., Aluvala, S. K., PONUGOTI, S., & Konda, S. T. R. (2026) · International Conference on Revolutionary Artificial Intelligence and Future Applications (Rev-AI) · doi:10.1109/rev-ai70456.2026.11622166
“The CHAIR metric introduced by Rohrbach et al. [10] provides complementary evaluation for image captioning, measuring the proportion of mentioned objects that do not appear in ground truth annotations.”
Tone Matters: The Impact of Linguistic Tone on Hallucination in VLMs
Hong, W., Jiang, Z., Shen, B., Guan, X., Feng, Y., Xu, M., & Li, B. (2026) · IEEE/CVF Winter Conference on Applications of Computer Vision Workshops (WACVW) · doi:10.1109/wacvw68408.2026.00147 · Cited by 3
“Hallucination and the Missing Information Gap: Early studies of hallucination in VLMs focused on object fabrication, where models describe entities that are not present in the image [3, 22].”
TruthLens: Object Hallucination Detection via Self-evaluating Truthfulness Scores in LVLMs
Wu, Y., Lai, R., Lu, X., Chen, Q., Zhou, Z., Zhuang, J., Yu, W., & Wang, R. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37490-5_30
Dang, Q.-V., Nguyen, N.-S.-A., Vo, T.-B.-D., Dinh, M., Le, D., Vu, H.-V., & Nguyen, P. T. L. (2026) · Discover Artificial Intelligence · doi:10.1007/s44163-026-01778-z
“Focus Metric Size POPE [1] Object Existence Accuracy, F1 3000 CHAIR [20] Caption Fidelity CHAIR 𝑆𝑆 , CHAIR 𝐼𝐼 500 MMHalBench [21] Multi-type Hallucination Score 96 MedHallBench [22] Medical Domain Accuracy 1200 InfoSeek [23] Knowledge VQA Accuracy 1.3M KVQA [24] Named Entities Accuracy 183K…”
Metadata
- Authors
- Rohrbach, A., Hendricks, L. A., Burns, K., Darrell, T., & Saenko, K.
- Affiliations
- Max Planck Society (DE)
- University of California, Berkeley (US)
- Boston University (US)
- Venue
- Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing
- Published
- 2018
- Pages
- 4035-4045
- OpenAlex
- W2892087592
- Cited by
- 420
- References
- 19
- Citations by year
- 2026: 103 · 2025: 80 · 2024: 70 · 2023: 42 · 2022: 32 · 2021: 45 · 2020: 32 · 2019: 15 · 2018: 1
- FWCI
- 7.01
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
- Keywords
- object hallucination, image captioning, hallucination evaluation, language priors
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
