The DOI matches its Crossref record; 40 of the 40 references it cites were found in a registry or an index.
ReferItGame: Referring to Objects in Photographs of Natural Scenes
Open accessAbstract
In this paper we introduce a new game to crowd-source natural language referring expressions.By designing a two player game, we can both collect and verify referring expressions directly within the game.To date, the game has produced a dataset containing 130,525 expressions, referring to 96,654 distinct objects, in 19,894 photographs of natural scenes.This dataset is larger and more varied than previous REG datasets and allows us to study referring expressions in real-world scenes.We provide an in depth analysis of the resulting dataset.Based on our findings, we design a new optimization based model for generating referring expressions and perform experimental evaluations on 3 test sets.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Grice, H. P. (1975) · Speech Acts · doi:10.1163/9789004368811_003 · Cited by 23,221
(2002) · Foundations of Cognitive Psychology · doi:10.7551/mitpress/3080.003.0017 · Cited by 3,953
Rosch, E., & Lloyd, B. B. (2024) · Routledge eBooks · doi:10.4324/9781032633275 · Cited by 2,829
Understanding natural language
Winograd, T. (1972) · Cognitive Psychology · doi:10.1016/0010-0285(72)90002-3 · Cited by 2,799
Labeling images with a computer game
von Ahn, L., & Dabbish, L. (2004) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/985692.985733 · Cited by 2,204
Im2Text: Describing Images Using 1 Million Captioned Photographs
Ordóñez, V., Kulkarni, G. A., & Berg, T. L. (2011) · neural information processing systems · Cited by 1,356
Self-citationRosch, E. (2024) · Cognition and Categorization · doi:10.4324/9781032633275-4 · Cited by 1,231
Building Natural-Language Generation Systems
Reiter, E. (1996) · arXiv (Cornell University) · doi:10.48550/arxiv.cmp-lg/9605002 · Cited by 1,228
Every Picture Tells a Story: Generating Sentences from Images
Farhadi, A., Hejrati, M., Sadeghi, M. A., Young, P., Rashtchian, C., Hockenmaier, J., & Forsyth, D. (2010) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-15561-1_2 · Cited by 1,204
Cited by
VQA: Visual Question Answering
Antol, S., Agrawal, A., Lu, J., Mitchell, M., Batra, D., Zitnick, C. L., & Parikh, D. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.279 · Cited by 4,487
Transformers in Vision: A Survey
Khan, S., Naseer, M., Hayat, M., Zamir, S. W., Khan, F. S., & Shah, M. (2022) · ACM Computing Surveys · doi:10.1145/3505244 · Cited by 3,595
Transformers in Vision: A Survey
Khan, S., Naseer, M., Hayat, M., Zamir, S. W., Khan, F. S., & Shah, M. (2021) · arXiv · doi:10.48550/arxiv.2101.01169 · Cited by 3,594
ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks
Lu, J., Batra, D., Parikh, D., & Lee, S. (2019) · arXiv · doi:10.48550/arxiv.1908.02265 · Cited by 2,109
UNITER: UNiversal Image-TExt Representation Learning
Chen, Y.-C., Li, L., Yu, L., El Kholy, A., Ahmed, F., Gan, Z., Cheng, Y., & Liu, J. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58577-8_7 · Cited by 2,055
CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning
Johnson, J., Hariharan, B., van der Maaten, L., Fei-Fei, L., Zitnick, C. L., & Girshick, R. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.215 · Cited by 1,781
Improved Baselines with Visual Instruction Tuning
Liu, H., Li, C., Li, Y., & Lee, Y. J. (2023) · arXiv · doi:10.48550/arxiv.2310.03744 · Cited by 1,733
Multimodal Compact Bilinear Pooling for Visual Question Answering and Visual Grounding
Fukui, A., Park, D. H., Yang, D., Rohrbach, A., Darrell, T., & Rohrbach, M. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1044 · Cited by 1,582
Improved Baselines with Visual Instruction Tuning
Liu, H., Li, C., Li, Y., & Lee, Y. J. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.02484 · Cited by 1,748
Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Models
Plummer, B. A., Wang, L., Cervantes, C. M., Caicedo, J. C., Hockenmaier, J., & Lazebnik, S. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.303 · Cited by 1,444
VisualBERT: A Simple and Performant Baseline for Vision and Language
Li, L. H., Yatskar, M., Yin, D., Hsieh, C.-J., & Chang, K.-W. (2019) · arXiv · doi:10.48550/arxiv.1908.03557 · Cited by 1,198
“Various tasks such as visual question answering (Antol et al., 2015; Goyal et al., 2017), textual grounding (Kazemzadeh et al., 2014; Plummer et al., 2015), and visual reasoning (Suhr et al., 2019; Zellers et al., 2019) have been proposed and various models (Yang et al., 2016; Anderson et al.,…”
MethodAnderson, P., Wu, Q., Teney, D., Bruce, J., Johnson, M., Sunderhauf, N., Reid, I., Gould, S., & van den Hengel, A. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00387 · Cited by 1,187
Generation and Comprehension of Unambiguous Object Descriptions
Mao, J., Huang, J., Toshev, A., Camburu, O., Yuille, A., & Murphy, K. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.9 · Cited by 1,174
(2016) · doi:10.1007/978-3-319-46475-6_5 · Cited by 1,081
Self-citationAlign before Fuse: Vision and Language Representation Learning with Momentum Distillation
Li, J., Selvaraju, R. R., Gotmare, A. D., Joty, S., Xiong, C., & Hoi, S. (2021) · arXiv · doi:10.48550/arxiv.2107.07651 · Cited by 1,078
Localizing Moments in Video with Natural Language
Hendricks, L. A., Wang, O., Shechtman, E., Sivic, J., Darrell, T., & Russell, B. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.618 · Cited by 958
VL-BERT: Pre-training of Generic Visual-Linguistic Representations
Su, W., Zhu, X., Cao, Y., Li, B., Lu, L., Wei, F., & Dai, J. (2019) · arXiv · doi:10.48550/arxiv.1908.08530 · Cited by 950
A Comprehensive Survey of Deep Learning for Image Captioning
Hossain, M. Z., Sohel, F., Shiratuddin, M. F., & Laga, H. (2019) · ACM Computing Surveys · doi:10.1145/3295748 · Cited by 879
“This method can generate a description for an specific object or region that is called referring expression [37, 46, 68, 102, 103, 136, 141].”
MethodTALL: Temporal Activity Localization via Language Query
Gao, J., Sun, C., Yang, Z., & Nevatia, R. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.563 · Cited by 842
MAttNet: Modular Attention Network for Referring Expression Comprehension
Yu, L., Lin, Z., Shen, X., Yang, J., Lu, X., Bansal, M., & Berg, T. L. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00142 · Cited by 824
Self-citationVisual7W: Grounded Question Answering in Images
Zhu, Y., Groth, O., Bernstein, M., & Fei-Fei, L. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.540 · Cited by 794
“Inspired by Geman et al. [8], we establish the link by grounding objects in the images and perform experiments in the grounded QA setting.”
MethodGatt, A., & Krahmer, E. (2018) · Journal of Artificial Intelligence Research · doi:10.1613/jair.5477 · Cited by 776
Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Concepts
Changpinyo, S., Sharma, P., Ding, N., & Soricut, R. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00356 · Cited by 612
VQA: Visual Question Answering: www.visualqa.org
Agrawal, A., Lu, J., Antol, S., Mitchell, M., Zitnick, C. L., Parikh, D., & Batra, D. (2017) · International Journal of Computer Vision · doi:10.1007/s11263-016-0966-6 · Cited by 570
Learning Two-Branch Neural Networks for Image-Text Matching Tasks
Wang, L., Li, Y., Huang, J., & Lazebnik, S. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2797921 · Cited by 535
Natural Language Object Retrieval
Hu, R., Xu, H., Rohrbach, M., Feng, J., Saenko, K., & Darrell, T. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.493 · Cited by 532
Cross-Modal Self-Attention Network for Referring Image Segmentation
Ye, L., Rochan, M., Liu, Z., & Wang, Y. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01075 · Cited by 532
Multimodal intelligence: Representation learning, information fusion, and applications
Zhang, C., Yang, Z., He, X., & Deng, L. (2020) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2020.2987728 · Cited by 511
LISA: Reasoning Segmentation via Large Language Model
Lai, X., Tian, Z., Chen, Y., Li, Y., Yuan, Y., Liu, S., & Jia, J. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.00915 · Cited by 466
Zhao, F., Zhang, C., & Geng, B. (2024) · ACM Computing Surveys · doi:10.1145/3649447 · Cited by 480
A Corpus for Reasoning about Natural Language Grounded in Photographs
Suhr, A., Zhou, S., Zhang, A., Zhang, I., Bai, H., & Artzi, Y. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1644 · Cited by 448
Segmentation from Natural Language Expressions
Hu, R., Rohrbach, M., & Darrell, T. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46448-0_7 · Cited by 438
Visual question answering: A survey of methods and datasets
Wu, Q., Teney, D., Wang, P., Shen, C., Dick, A., & van den Hengel, A. (2017) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2017.05.001 · Cited by 438
MMMU: A Massive Multi-Discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI
Yue, X., Ni, Y., Zheng, T., Zhang, K., Liu, R., Zhang, G., Stevens, S., Jiang, D., Ren, W., Sun, Y., et al. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.00913 · Cited by 429
“For instance, CogVLM [70] achieves 85 % on VQA-v2 [23], 92 % on ScienceQA-IMG [46], and 93 % on RefCOCO [28].”
Modeling Context Between Objects for Referring Expression Understanding
Nagaraja, V. K., Morariu, V. I., & Davis, L. S. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_48 · Cited by 425
Lu, J., Yang, J., Batra, D., & Parikh, D. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00754 · Cited by 411
A Fast and Accurate One-Stage Approach to Visual Grounding
Yang, Z., Gong, B., Wang, L., Huang, W., Yu, D., & Luo, J. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00478 · Cited by 410
12-in-1: Multi-Task Vision and Language Representation Learning
Lu, J., Goswami, V., Rohrbach, M., Parikh, D., & Lee, S. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01045 · Cited by 410
CRIS: CLIP-Driven Referring Image Segmentation
Wang, Z., Lu, Y., Li, Q., Tao, X., Guo, Y., Gong, M., & Liu, T. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01139 · Cited by 404
TransVG: End-to-End Visual Grounding with Transformers
Deng, J., Yang, Z., Chen, T., Zhou, W., & Li, H. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00179 · Cited by 372
GuessWhat?! Visual Object Discovery through Multi-modal Dialogue
De Vries, H., Strub, F., Chandar, S., Pietquin, O., Larochelle, H., & Courville, A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.475 · Cited by 369
In Defense of Grid Features for Visual Question Answering
Jiang, H., Misra, I., Rohrbach, M., Learned-Miller, E., & Chen, X. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01028 · Cited by 358
Learning Cooperative Visual Dialog Agents with Deep Reinforcement Learning
Das, A., Kottur, S., Moura, J. M. F., Lee, S., & Batra, D. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.321 · Cited by 356
The hateful memes challenge: Detecting hate speech in multimodal memes
Kiela, D., Firooz, H., Mohan, A., Goswami, V., Singh, A., Ringshia, P., & Testuggine, D. (2020) · arXiv · doi:10.48550/arxiv.2005.04790 · Cited by 352
“[56] for a survey), with great progress on important problems such as visual question answering [3, 26] and image caption generation and retrieval [8, 88, 43, 68, 28], with offshoot tasks focusing specifically on visual reasoning [36], referring expressions [39], visual storytelling [57, 32], visual dialogue [9, 12],…”
BackgroundTOUCHDOWN: Natural Language Navigation and Spatial Reasoning in Visual Street Environments
Chen, H., Suhr, A., Misra, D., Snavely, N., & Artzi, Y. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01282 · Cited by 343
VirTex: Learning Visual Representations from Textual Annotations
Desai, K., & Johnson, J. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01101 · Cited by 342
REVERIE: Remote Embodied Visual Referring Expression in Real Indoor Environments
Qi, Y., Wu, Q., Anderson, P., Wang, X., Wang, W. Y., Shen, C., & van den Hengel, A. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01000 · Cited by 331
ScanRefer: 3D Object Localization in RGB-D Scans Using Natural Language
Chen, D. Z., Chang, A. X., & Nießner, M. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58565-5_13 · Cited by 312
Multi-Task Collaborative Network for Joint Referring Expression Comprehension and Segmentation
Luo, G., Zhou, Y., Sun, X., Cao, L., Wu, C., Deng, C., & Ji, R. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01005 · Cited by 308
Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Models
Plummer, B. A., Wang, L., Cervantes, C. M., Caicedo, J. C., Hockenmaier, J., & Lazebnik, S. (2017) · International Journal of Computer Vision · doi:10.1007/s11263-016-0965-7 · Cited by 298
Florence-2: Advancing a Unified Representation for a Variety of Vision Tasks
Xiao, B., Wu, H., Xu, W., Dai, X., Hu, H., Lu, Y., Zeng, M., Liu, C., & Yuan, L. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.00461 · Cited by 290
Learning to Assemble Neural Module Tree Networks for Visual Grounding
Liu, D., Zhang, H., Zha, Z.-J., & Wu, F. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00477 · Cited by 287
Referring Image Segmentation via Recurrent Refinement Networks
Li, R., Li, K., Kuo, Y.-C., Shu, M., Qi, X., Shen, X., & Jia, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00602 · Cited by 275
Recurrent Multimodal Interaction for Referring Image Segmentation
Liu, C., Lin, Z., Shen, X., Yang, J., Lu, X., & Yuille, A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.143 · Cited by 272
REMIND Your Neural Network to Prevent Catastrophic Forgetting
Hayes, T. L., Kafle, K., Shrestha, R., Acharya, M., & Kanan, C. (2019) · arXiv · doi:10.48550/arxiv.1910.02509 · Cited by 268
Neighbourhood Watch: Referring Expression Comprehension via Language-Guided Graph Attention Networks
Wang, P., Wu, Q., Cao, J., Shen, C., Gao, L., & Hengel, A. V. D. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00206 · Cited by 265
TVR: A Large-Scale Dataset for Video-Subtitle Moment Retrieval
Lei, J., Yu, L., Berg, T. L., & Bansal, M. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58589-1_27 · Cited by 255
Self-citationReferIt3D: Neural Listeners for Fine-Grained 3D Object Identification in Real-World Scenes
Achlioptas, P., Abdelreheem, A., Xia, F., Elhoseiny, M., & Guibas, L. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58452-8_25 · Cited by 253
Improving Vision-and-Language Navigation with Image-Text Pairs from the Web
Majumdar, A., Shrivastava, A., Lee, S., Anderson, P., Parikh, D., & Batra, D. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58539-6_16 · Cited by 201
ScanQA: 3D Question Answering for Spatial Scene Understanding
Azuma, D., Miyanishi, T., Kurita, S., & Kawanabe, M. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01854 · Cited by 197
Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMs
Tong, S., Liu, Z., Zhai, Y., Ma, Y., LeCun, Y., & Xie, S. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.00914 · Cited by 189
Mees, O., Hermann, L., Rosete-Beas, E., & Burgard, W. B. (2022) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2022.3180108 · Cited by 189
RSVG: Exploring Data and Models for Visual Grounding on Remote Sensing Data
Zhan, Y., Xiong, Z., & Yuan, Y. (2023) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2023.3250471 · Cited by 179
“Over the years, various visual grounding datasets [18], [51], [52], [53], [54], [55], [56], [57], [58], [59], [60], [61], [62], [63], [64], [65] based on real-world and computer-generated images have been proposed to study visual grounding.”
BackgroundMethodArtEmis: Affective Language for Visual Art
Achlioptas, P., Ovsjanikov, M., Haydarov, K., Elhoseiny, M., & Guibas, L. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01140 · Cited by 161
Generative Multimodal Models are In-Context Learners
Sun, Q., Cui, Y., Zhang, X., Zhang, F., Yu, Q., Wang, Y., Rao, Y., Liu, J., Huang, T., & Wang, X. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.01365 · Cited by 147
Airbert: In-domain Pretraining for Vision-and-Language Navigation
Guhur, P.-L., Tapaswi, M., Chen, S., Laptev, I., & Schmid, C. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00166 · Cited by 139
“Fine-tuning them on downstream datasets achieves state-of-the-art performance on various VL tasks [6, 18, 45, 43].”
BackgroundReasoning about Pragmatics with Neural Listeners and Speakers
Andreas, J., & Klein, D. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1125 · Cited by 115
“Other research, focused on direct referring expression generation from a computer vision perspective, includes that of Mao et al. (2015) and Kazemzadeh et al. (2014).”
BackgroundThe Revolution of Multimodal Large Language Models: A Survey
Caffagni, D., Cocchi, F., Barsellotti, L., Moratelli, N., Sarto, S., Baraldi, L., Baraldi, L., Cornia, M., & Cucchiara, R. (2024) · Findings of the Association for Computational Linguistics ACL 2024 · doi:10.18653/v1/2024.findings-acl.807 · Cited by 98
“Differently, Kosmos-2 (Peng et al., 2023) and Ferret (You et al., 2023) exploit an open-vocabulary detector (Li et al., 2022) to ground noun chunks parsed from cap-tions and then reconstruct referring expressions.”
Emergence of Linguistic Communication from Referential Games with Symbolic and Pixel Input
Lazaridou, A., Hermann, K. M., Tuyls, K., & Clark, S. (2018) · arXiv · doi:10.48550/arxiv.1804.03984 · Cited by 86
Visual Grounding in Remote Sensing Images
Sun, Y., Feng, S., Li, X., Ye, Y., Kang, J., & Huang, X. (2022) · Proceedings of the 30th ACM International Conference on Multimedia · doi:10.1145/3503161.3548316 · Cited by 81
“RefCOCO[11], RefCOCO+[11], and RefCOCOg[19] are collected on top ofMSCOCO[13] images.”
BackgroundPACO: Parts and Attributes of Common Objects
Ramanathan, V., Kalia, A., Petrovic, V., Wen, Y., Zheng, B., Guo, B., Wang, R., Marquez, A., Kovvuri, R., Kadian, A., et al. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.00690 · Cited by 79
Qi, Y., Pan, Z., Hong, Y., Yang, M.-H., van den Hengel, A., & Wu, Q. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00168 · Cited by 80
OmniGen: Unified Image Generation
Xiao, S., Wang, Y., Zhou, J., Yuan, H., Xing, X., Yan, R., Li, C., Wang, S., Huang, T., & Liu, Z. (2025) · 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52734.2025.01241 · Cited by 75
V*: Guided Visual Search as a Core Mechanism in Multimodal LLMs
Wu, P., & Xie, S. (2023) · arXiv · doi:10.48550/arxiv.2312.14135 · Cited by 74
V*: Guided Visual Search as a Core Mechanism in Multimodal LLMs
Wu, P., & Xie, S. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.01243 · Cited by 74
“The COCO-Stuff [3], LVIS-PACO part [40], refCOCO(+/g) [18, 33], and refCLEF [18] datasets are used as both detection and segmentation data.”
MethodXGPT: Cross-modal Generative Pre-Training for Image Captioning
Xia, Q., Huang, H., Duan, N., Zhang, D., Ji, L., Sui, Z., Cui, E., Bharti, T., & Zhou, M. (2021) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-88480-2_63 · Cited by 70
A Survey of Current Datasets for Vision and Language Research
Ferraro, F., Mostafazadeh, N., Huang, T.-H., Vanderwende, L., Devlin, J., Galley, M., & Mitchell, M. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1021 · Cited by 68
“…past year, recent work has proposed meth-ods for image and video captioning (Fang et al., 2014; Donahue et al., 2014; Venugopalan et al., 2015), summarization (Kim et al., 2015), reference (Kazemzadeh et al., 2014), and question answering (Antol et al., 2015; Gao et al., 2015), to name just a few.”
BackgroundUnified Pragmatic Models for Generating and Following Instructions
Fried, D., Andreas, J., & Klein, D. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1177 · Cited by 64
“Much recent work on pragmatic models, especially models of speakers, focuses on the referring expression generation or “contrastive caption-ing” task introduced by Kazemzadeh et al. (2014).”
BackgroundEmbodied Collaborative Referring Expression Generation in Situated Human-Robot Interaction
Fang, R., Doering, M., & Chai, J. Y. (2015) · Proceedings of the Tenth Annual ACM/IEEE International Conference on Human-Robot Interaction · doi:10.1145/2696454.2696467 · Cited by 59
A survey of transformer-based multimodal pre-trained modals
Han, X., Wang, Y.-T., Feng, J.-L., Deng, C., Chen, Z.-H., Huang, Y.-A., Su, H., Hu, L., & Hu, P.-W. (2023) · Neurocomputing · doi:10.1016/j.neucom.2022.09.136 · Cited by 59
(2023) · doi:10.1109/iros55552.2023.10341379 · Cited by 48
“The RefCOCO and RefCOCO+ are based on the MSCOCO [24] and collect language by using the annotation rule [21] as well.”
BackgroundMethodInfluential(2018) · doi:10.18653/v1/n18-2123 · Cited by 48
OpenVLA: An Open-Source Vision-Language-Action Model
Kim, M. J., Pertsch, K., Karamcheti, S., Xiao, T., Balakrishna, A., Nair, S., Rafailov, R., Foster, E., Lam, G., Sanketi, P., et al. (2024) · arXiv · doi:10.48550/arxiv.2406.09246 · Cited by 62
“…Language Models Visually-conditioned language models (VLMs), which are trained on Internet-scale data to generate natural language from input image(s) and language prompts, have been adopted for myriad applications from visual question answering [28–31] to object localization [32, 33].”
BackgroundVideo Object Grounding Using Semantic Roles in Language Description
Sadhu, A., Chen, K., & Nevatia, R. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01043 · Cited by 36
CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning
Johnson, J., Hariharan, B., van der Maaten, L., Fei-Fei, L., Zitnick, C. L., & Girshick, R. (2016) · arXiv · doi:10.48550/arxiv.1612.06890 · Cited by 40
Multi-Modal Dynamic Graph Transformer for Visual Grounding
Chen, S., & Li, B. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01509 · Cited by 28
“With a focus on aligning semantically consistent phrase-region pairs from the given image and sentence, the general visual grounding problem can be extended to phrase localization [5, 9, 36] and referring expression comprehension [17, 28].”
BackgroundUniReal: Universal Image Generation and Editing via Learning Real-world Dynamics
Chen, X., Zhang, Z., Zhang, H., Zhou, Y., Kim, S. Y., Liu, Q., Li, Y., Zhang, J., Zhao, N., Wang, Y., et al. (2025) · 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52734.2025.01166 · Cited by 23
Tschannen, M., Gritsenko, A., Wang, X., Naeem, M. F., Alabdulmohsin, I., Parthasarathy, N., Evans, T., Beyer, L., Xia, Y., Mustafa, B., et al. (2025) · arXiv · doi:10.48550/arxiv.2502.14786 · Cited by 22
PaliGemma: A versatile 3B VLM for transfer
Beyer, L., Steiner, A., Pinto, A. S., Kolesnikov, A., Wang, X., Salz, D., Neumann, M., Alabdulmohsin, I., Tschannen, M., Bugliarello, E., et al. (2024) · arXiv · doi:10.48550/arxiv.2407.07726 · Cited by 17
Bai, S., Cai, Y., Chen, R., Chen, K., Chen, X., Cheng, Z., Deng, L., Ding, W., Gao, C., Ge, C., et al. (2025) · arXiv · doi:10.48550/arxiv.2511.21631 · Cited by 27
“…and Spatial Understanding: We evaluate the models on 11 benchmarks include 2D grounding, 3D grounding and spatial understanding: RefCOCO/+/g (Kazemzadeh et al., 2014; Mao et al., 2016), ODinW-13 (Li et al., 2022), CountBench (Paiss et al., 2023), ARKitScenes (Baruch et al., 2021), Hypersim…”
InfluentialPrismatic VLMs: Investigating the Design Space of Visually-Conditioned Language Models
Karamcheti, S., Nair, S., Balakrishna, A., Liang, P., Kollar, T., & Sadigh, D. (2024) · arXiv · doi:10.48550/arxiv.2402.07865 · Cited by 7
“GQA evaluates spatial reasoning, while As such, we evaluate bounding box prediction accuracy on RefCOCO, RefCOCO+, and RefCOCOg (Kazemzadeh et al., 2014; Yu et al., 2016), and on OCID-Ref (Wang et al., 2021).”
BackgroundMethodInfluentialQuality and Relevance Metrics for Selection of Multimodal Pretraining Data
Rao, R., Rao, S., Nouri, E., Dey, D., Celikyilmaz, A., & Dolan, B. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw50498.2020.00486 · Cited by 6
“For training and test we use the RefCOCO+ dataset [8].”
MethodEmergence of Hierarchical Reference Systems in Multi-agent Communication
Ohmer, X., Duda, M., & Bruni, E. (2022) · arXiv · doi:10.48550/arxiv.2203.13176 · Cited by 4
Metadata
- Authors
- Kazemzadeh, S., Ordonez, V., Matten, M., & Berg, T.
- Venue
- Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)
- Published
- 2014
- Pages
- 787-798
- OpenAlex
- W2251512949
- Cited by
- 1,048
- References
- 40
- FWCI
- 18.02
- Type
- conference paper
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.