The DOI matches its Crossref record; 40 of the 40 references it cites were found in a registry or an index.
ReferItGame: Referring to Objects in Photographs of Natural Scenes
Open accessAbstract
In this paper we introduce a new game to crowd-source natural language referring expressions.By designing a two player game, we can both collect and verify referring expressions directly within the game.To date, the game has produced a dataset containing 130,525 expressions, referring to 96,654 distinct objects, in 19,894 photographs of natural scenes.This dataset is larger and more varied than previous REG datasets and allows us to study referring expressions in real-world scenes.We provide an in depth analysis of the resulting dataset.Based on our findings, we design a new optimization based model for generating referring expressions and perform experimental evaluations on 3 test sets.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Grice, H. P. (1975) · Speech Acts · doi:10.1163/9789004368811_003 · Cited by 23,221
(2002) · Foundations of Cognitive Psychology · doi:10.7551/mitpress/3080.003.0017 · Cited by 3,953
Rosch, E., & Lloyd, B. B. (2024) · Routledge eBooks · doi:10.4324/9781032633275 · Cited by 2,829
Understanding natural language
Winograd, T. (1972) · Cognitive Psychology · doi:10.1016/0010-0285(72)90002-3 · Cited by 2,799
Labeling images with a computer game
von Ahn, L., & Dabbish, L. (2004) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/985692.985733 · Cited by 2,204
Im2Text: Describing Images Using 1 Million Captioned Photographs
Ordóñez, V., Kulkarni, G. A., & Berg, T. L. (2011) · neural information processing systems · Cited by 1,356
Self-citationRosch, E. (2024) · Cognition and Categorization · doi:10.4324/9781032633275-4 · Cited by 1,231
Building Natural-Language Generation Systems
Reiter, E. (1996) · arXiv (Cornell University) · doi:10.48550/arxiv.cmp-lg/9605002 · Cited by 1,228
Every Picture Tells a Story: Generating Sentences from Images
Farhadi, A., Hejrati, M., Sadeghi, M. A., Young, P., Rashtchian, C., Hockenmaier, J., & Forsyth, D. (2010) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-15561-1_2 · Cited by 1,204
Parsing with Compositional Vector Grammars
Socher, R., Bauer, J., Manning, C. D., & Ng, A. Y. (2013) · Cited by 828
Computational Interpretations of the Gricean Maxims in the Generation of Referring Expressions
Dale, R., & Reiter, E. (1995) · Cognitive Science · doi:10.1207/s15516709cog1902_3 · Cited by 721
Baby talk: Understanding and generating simple image descriptions
Kulkarni, G., Premraj, V., Dhar, S., Li, S., Choi, Y., Berg, A. C., & Berg, T. L. (2011) · CVPR 2011 · doi:10.1109/cvpr.2011.5995466 · Cited by 521
Self-citationPeekaboom: a game for locating objects in images
von Ahn, L., Liu, R., & Blum, M. (2006) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1124772.1124782 · Cited by 509
Guadarrama, S., Krishnamoorthy, N., Malkarnenkar, G., Venugopalan, S., Mooney, R., Darrell, T., & Saenko, K. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.337 · Cited by 500
What Does Classifying More Than 10,000 Image Categories Tell Us?
Deng, J., Berg, A. C., Li, K., & Fei-Fei, L. (2010) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-15555-0_6 · Cited by 486
Collective Generation of Natural Image Descriptions
Кузнецова, П., Ordóñez, V., Berg, A., Berg, T. L., & Choi, Y. (2012) · Cited by 402
Self-citationThe segmented and annotated IAPR TC-12 benchmark
Escalante, H. J., Hernández, C. A., Gonzalez, J. A., López-López, A., Montes, M., Morales, E. F., Enrique Sucar, L., Villaseñor, L., & Grubinger, M. (2010) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2009.03.008 · Cited by 399
Corpus-Guided Sentence Generation of Natural Images
Yang, Y., Teo, C. L., Daumé, H., & Aloimonos, Y. (2011) · Cited by 356
Computational Generation of Referring Expressions: A Survey
Krahmer, E., & van Deemter, K. (2012) · Computational Linguistics · doi:10.1162/coli_a_00088 · Cited by 325
I2T: Image Parsing to Text Description
Yao, B. Z., Yang, X., Lin, L., Lee, M. W., & Zhu, S.-C. (2010) · Proceedings of the IEEE · doi:10.1109/jproc.2010.2050411 · Cited by 310
Building Natural Language Generation Systems
Horacek, H. (2001) · Artificial Intelligence in Medicine · doi:10.1016/s0933-3657(00)00114-7 · Cited by 309
The IAPR Benchmark : A New Evaluation Resource for Visual Information Systems
Grubinger, M. (2006) · RWTH Publications (RWTH Aachen) · Cited by 285
Fine-Grained Crowdsourcing for Fine-Grained Recognition
Deng, J., Krause, J., & Fei-Fei, L. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.81 · Cited by 249
Detecting Carried Objects in Short Video Sequences
Damen, D., & Hogg, D. (2008) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-88690-7_12 · Cited by 243
Automatic Caption Generation for News Images
Feng, Y., & Lapata, M. (2012) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.118 · Cited by 136
Towards good practice in large-scale learning for image classification
Perronnin, F., Akata, Z., Harchaoui, Z., & Schmid, C. (2012) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2012.6248090 · Cited by 124
Generating Image Descriptions Using Dependency Relational Patterns
Aker, A., & Gaizauskas, R. (2010) · Cited by 108
The use of spatial relations in referring expression generation
Viethen, J., & Dale, R. (2008) · International Conference on Natural Language Generation · doi:10.3115/1708322.1708334 · Cited by 101
From Large Scale Image Categorization to Entry-Level Categories
Ordóñez, V., Deng, J., Choi, Y., Berg, A. C., & Berg, T. L. (2013) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.344 · Cited by 100
Self-citationLearning Distributions over Logical Forms for Referring Expression Generation
FitzGerald, N., Artzi, Y., & Zettlemoyer, L. (2013) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d13-1197 · Cited by 92
Toward a Computational Psycholinguistics of Reference Production
van Deemter, K., Albert Gatt, A., van Gompel, R. P. G., & Krahmer, E. (2012) · Topics in Cognitive Science · doi:10.1111/j.1756-8765.2012.01187.x · Cited by 77
Building a semantically transparent corpus for the generation of referring expressions
van Deemter, K., van der Sluis, I., & Albert Gatt, A. (2006) · International Conference on Natural Language Generation · doi:10.3115/1706269.1706296 · Cited by 75
How Many Words Is a Picture Worth? Automatic Caption Generation for News Images
Feng, Y., & Lapata, M. (2010) · Edinburgh Research Explorer (University of Edinburgh) · Cited by 73
Generating Expressions that Refer to Visible Objects
Mitchell, M., Hltcoe, J. H., van Deemter, K., & Reiter, E. (2013) · Aberdeen University Research Archive (Aberdeen University) · Cited by 61
Natural reference to objects in a visual domain
Mitchell, M., van Deemter, K., & Reiter, E. (2010) · International Conference on Natural Language Generation · Cited by 41
Speaker-Dependent Variation in Content Selection for Referring Expression Generation
Viethen, J., & Dale, R. (2010) · Cited by 39
Word sense disambiguation via human computation
Seemakurty, N., Chu, J., von Ahn, L., & Tomasic, A. (2010) · doi:10.1145/1837885.1837905 · Cited by 35
Towards Situated Dialogue: Revisiting Referring Expression Generation
Fang, R., Liu, C., She, L., & Chai, J. Y. (2013) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d13-1038 · Cited by 27
Typicality and Object Reference
Mitchell, M., Reiter, E., & van Deemter, K. (2013) · eScholarship (California Digital Library) · Cited by 23
Cited by
VQA: Visual Question Answering
Antol, S., Agrawal, A., Lu, J., Mitchell, M., Batra, D., Zitnick, C. L., & Parikh, D. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.279 · Cited by 4,487
Transformers in Vision: A Survey
Khan, S., Naseer, M., Hayat, M., Zamir, S. W., Khan, F. S., & Shah, M. (2022) · ACM Computing Surveys · doi:10.1145/3505244 · Cited by 3,595
Transformers in Vision: A Survey
Khan, S., Naseer, M., Hayat, M., Zamir, S. W., Khan, F. S., & Shah, M. (2021) · arXiv · doi:10.48550/arxiv.2101.01169 · Cited by 3,594
ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks
Lu, J., Batra, D., Parikh, D., & Lee, S. (2019) · arXiv · doi:10.48550/arxiv.1908.02265 · Cited by 2,109
UNITER: UNiversal Image-TExt Representation Learning
Chen, Y.-C., Li, L., Yu, L., El Kholy, A., Ahmed, F., Gan, Z., Cheng, Y., & Liu, J. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58577-8_7 · Cited by 2,055
Metadata
- Authors
- Kazemzadeh, S., Ordonez, V., Matten, M., & Berg, T.
- Venue
- Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)
- Published
- 2014
- Pages
- 787-798
- OpenAlex
- W2251512949
- Cited by
- 1,048
- References
- 40
- FWCI
- 18.02
- Type
- conference paper
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.