Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat
Sign Up
Cited by 1,738

The DOI matches its Crossref record; 22 of the 22 references it cites were found in a registry or an index.

The Open Images Dataset V4: Unified Image Classification, Object Detection, and Visual Relationship Detection at Scale

Open access
Kuznetsova, Alina, Rom, Hassan, Alldrin, Neil, Uijlings, Jasper, Krasin, Ivan, Pont-Tuset, Jordi, Kamali, Shahab, Popov, Stefan, Malloci, Matteo, Kolesnikov, Alexander, Duerig, Tom, Ferrari, Vittorio·2020·International Journal of Computer Vision
DOI 10.1007/s11263-020-01316-z
Open this paper in CoChat — free account

Abstract

We present Open Images V4, a dataset of 9.2M images with unified annotations for image classification, object detection and visual relationship detection. The images have a Creative Commons Attribution license that allows to share and adapt the material, and they have been collected from Flickr without a predefined list of class names or tags, leading to natural class statistics and avoiding an initial design bias. Open Images V4 offers large scale across several dimensions: 30.1M image-level labels for 19.8k concepts, 15.4M bounding boxes for 600 object classes, and 375k visual relationship annotations involving 57 classes. For object detection in particular, we provide 15x more bounding boxes than the next largest datasets (15.4M boxes on 1.9M images). The images often show complex scenes with several objects (8 annotated objects per image on average). We annotated visual relationships between them, which support visual relationship detection, an emerging task that requires structured reasoning. We provide in-depth comprehensive statistics about the dataset, we validate the quality of the annotations, we study how the performance of several modern models evolves with increasing amounts of training data, and we demonstrate two applications made possible by having unified annotations of multiple types coexisting in the same images. We hope that the scale, quality, and variety of Open Images V4 will foster further research and innovation even beyond the areas of image classification, object detection, and visual relationship detection.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1007/s11263-020-01316-z

Cites

22 of 22 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 22 references

Research works printed but not linked: 36

  1. Alexe, B., Deselaers, T., & Ferrari, V. (2010). What is an object? In CVPR. · DOI · Paper page
  2. Alexe, B., Deselaers, T., & Ferrari, V. (2012). Measuring the objectness of image windows. IEEE Transactions on PAMI, 34, 2189–2202. · DOI · Paper page
  3. Dai, B., Zhang, Y., & Lin, D. (2017). Detecting visual relationships with deep relational networks. In CVPR. · DOI · Paper page
  4. Everingham, M., Van Gool, L., Williams, C.K.I., Winn, J., & Zisserman, A. (2012). The PASCAL visual object classes challenge 2012 (VOC2012) results. http://www.pascal-network.org/challenges/VOC/voc2012/workshop/index.html.
  5. Fei-Fei, L., Fergus, R., & Perona, P. (2006). One-shot learning of object categories. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(4), 594–611. · DOI · Paper page
  6. Felzenszwalb, P., Girshick, R., & McAllester, D. (2010a). Cascade object detection with deformable part models. In CVPR. · DOI · Paper page
  7. Gao, C., Zou, Y., & Huang, J.B. (2018). iCAN: Instance-centric attention network for human-object interaction detection. In BMVC.
  8. Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014). Rich feature hierarchies for accurate object detection and semantic segmentation. In CVPR. · DOI · Paper page
  9. Gkioxari, G., Girshick, R., Dollár, P., & He, K. (2018). Detecting and recognizing human-object interactions. CVPR. · DOI · Paper page
  10. Gupta, A., Kembhavi, A., & Davis, L. (2009). Observing human-object interactions: Using spatial and functional compatibility for recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 31, 1775–1789. · DOI · Paper page
  11. Gupta, S., & Malik, J. (2015). Visual semantic role labeling. arXiv preprint arXiv:1505.04474. · arXiv
  12. Kolesnikov, A., Kuznetsova, A., Lampert, C., & Ferrari, V. (2018). Detecting visual relationships using box attention. arXiv:1807.02136. · arXiv
  13. Li, Y., Ouyang, W., Wang, X., & Tang, X. (2017). ViP-CNN: Visual phrase guided convolutional neural network. In CVPR. · DOI · Paper page
  14. Liang, K., Guo, Y., Chang, H., & Chen, X. (2018). Visual relationship detection with deep structural ranking. In AAAI. · DOI
  15. Liang, X., Lee, L., & Xing, E. P. (2017). Deep variation-structured reinforcement learning for visual relationship and attribute detection. In CVPR. · DOI · Paper page
  16. Lin, T.Y., Maire, M., Belongie, S., Bourdev, L., Girshick, R., Hays, J., Perona P, Ramanan, D., Zitnick, C.L., & Dollár, P. (2014). Microsoft COCO: Common objects in context. In ECCV. · DOI · Paper page
  17. Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.Y., & Berg, A.C. (2016). SSD: Single shot multibox detector. In ECCV. · DOI · Paper page
  18. Lu, C., Krishna, R., Bernstein, M., & Fei-Fei, L. (2016). Visual relationship detection with language priors. In European Conference on Computer Vision. · DOI · Paper page
  19. Papadopoulos, D.P., Uijlings, J.R.R., Keller, F., & Ferrari, V. (2016). We don’t need no bounding-boxes: Training object class detectors using only human verification. In CVPR. · DOI · Paper page
  20. Papadopoulos, D.P., Uijlings, J.R., Keller, F., & Ferrari, V. (2017). Extreme clicking for efficient object annotation. In ICCV. · DOI
  21. Peyre, J., Laptev, I., Schmid, C., & Sivic, J. (2017). Weakly-supervised learning of visual relations. In CVPR. · DOI · Paper page
  22. Prest, A., Schmid, C., & Ferrari, V. (2012). Weakly supervised learning of interactions between humans and objects. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34, 601–614. · DOI · Paper page
  23. Qian, N. (1999). On the momentum term in gradient descent learning algorithms. Neural Networks, 12(1), 145–151. · DOI · Paper page
  24. Redmon, J., & Farhadi, A. (2017). YOLO9000: better, faster, stronger. In CVPR. · DOI · Paper page
  25. Sandler, M., Howard, A.G., Zhu, M., Zhmoginov, A., & Chen, L. (2018). Mobilenetv2: Inverted residuals and linear bottleneck. In CVPR. · DOI · Paper page
  26. Su, H., Deng, J., & Fei-Fei, L. (2012). Crowdsourcing annotations for visual object detection. In AAAI Human Computation Workshop.
  27. Sun, C., Shrivastava, A., Singh, S., & Gupta, A. (2017). Revisiting unreasonable effectiveness of data in deep learning era. In ICCV. · DOI · Paper page
  28. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015). Going deeper with convolutions. In CVPR. · DOI · Paper page
  29. Uijlings, J., Popov, S., & Ferrari, V. (2018). Revisiting knowledge transfer for training object class detectors. In CVPR. · DOI
  30. Uijlings, J. R. R., van de Sande, K. E. A., Gevers, T., & Smeulders, A. W. M. (2013). Selective search for object recognition. International Journal of Computer Vision, 104, 154–171. · DOI · Paper page
  31. Veit, A., Alldrin, N., Chechik, G., Krasin, I., Gupta, A., & Belongie, S. (2017). Learning from noisy large-scale datasets with minimal supervision. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 839–847). http://openaccess.thecvf.com/content_cvpr_2017/papers/Veit_Learning_From_Noisy_CVPR_2017_paper.pdf.
  32. Viola, P., & Jones, M. (2001a). Rapid object detection using a boosted cascade of simple features. In CVPR.
  33. Viola, P., & Jones, M. (2001b). Robust real-time object detection. International Journal of Computer Vision, 4, 4.
  34. Yao, B., & Fei-Fei, L. (2010). Modeling mutual context of object and human pose in human-object interaction activities. In CVPR. · DOI · Paper page
  35. Zellers, R., Yatskar, M., Thomson, S., & Choi, Y. (2018). Neural motifs: Scene graph parsing with global context. In CVPR. · DOI · Paper page
  36. Zhang, H., Kyaw, Z., Yu, J., & Chang, S.F. (2017b). PPR-FCN: weakly supervised visual relation detection via parallel pairwise R-FCN. In ICCV · DOI

Cited by

See all 1,738

Metadata

Authors
Kuznetsova, A., Rom, H., Alldrin, N., Uijlings, J., Krasin, I., Pont-Tuset, J., Kamali, S., Popov, S., Malloci, M., Kolesnikov, A., Duerig, T., & Ferrari, V.
Venue
International Journal of Computer Vision
Publisher
Springer Science and Business Media LLC
Published
2020-03-13
Volume
128
Issue
7
Pages
1956-1981
ISSN
  • 2331-8422 (ISSN-L)
  • 0920-5691
  • 1573-1405
DOI
10.1007/s11263-020-01316-z
OpenAlex
W4288083516
Cited by
1,738
References
22
Type
posted content
Language
en
Open access
green
Licence

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: The Open Images Dataset V4: Unified Image Classification, Object Detection, and Visual Relationship Detection at Scale. https://app.cochat.ai/papers/10.1007/s11263-020-01316-z
The paper
Kuznetsova, A., Rom, H., Alldrin, N., Uijlings, J., Krasin, I., Pont-Tuset, J., Kamali, S., Popov, S., Malloci, M., Kolesnikov, A., Duerig, T., & Ferrari, V. (2020). The Open Images Dataset V4: Unified Image Classification, Object Detection, and Visual Relationship Detection at Scale. International Journal of Computer Vision. https://doi.org/10.1007/s11263-020-01316-z

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.