The DOI matches its Crossref record; 48 of the 48 references it cites were found in a registry or an index.
Data Distillation: Towards Omni-Supervised Learning
Open accessAbstract
We investigate omni-supervised learning, a special regime of semi-supervised learning in which the learner exploits all available labeled data plus internet-scale sources of unlabeled data. Omni-supervised learning is lower-bounded by performance on existing labeled datasets, offering the potential to surpass state-of-the-art fully supervised methods. To exploit the omni-supervised setting, we propose data distillation, a method that ensembles predictions from multiple transformations of unlabeled data, using a single model, to automatically generate new training annotations. We argue that visual recognition models have recently become accurate enough that it is now possible to apply classic ideas about self-training to challenging real-world data. Our experimental results show that in the cases of human keypoint detection and general object detection, state-of-the-art models trained with data distillation surpass the performance of using labeled data from the COCO dataset alone.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Self-citationVery Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472
Self-citationFaster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1506.01497 · Cited by 70,812
Self-citationImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380
Fully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834
Cited by
A survey on Image Data Augmentation for Deep Learning
Shorten, C., & Khoshgoftaar, T. M. (2019) · Journal of Big Data · doi:10.1186/s40537-019-0197-0 · Cited by 13,203
Knowledge Distillation: A Survey
Gou, J., Yu, B., Maybank, S. J., & Tao, D. (2021) · International Journal of Computer Vision · doi:10.1007/s11263-021-01453-z · Cited by 3,931
He, K., Gkioxari, G., Dollar, P., & Girshick, R. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2844175 · Cited by 3,457
Self-citationSelf-Training With Noisy Student Improves ImageNet Classification
Xie, Q., Luong, M.-T., Hovy, E., & Le, Q. V. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01070 · Cited by 2,393
Relational Knowledge Distillation
Park, W., Kim, D., Lu, Y., & Cho, M. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00409 · Cited by 1,681
Simple Copy-Paste is a Strong Data Augmentation Method for Instance Segmentation
Ghiasi, G., Cui, Y., Srinivas, A., Qian, R., Lin, T.-Y., Cubuk, E. D., Le, Q. V., & Zoph, B. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00294 · Cited by 1,148
Similarity-Preserving Knowledge Distillation
Tung, F., & Mori, G. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00145 · Cited by 1,079
Embracing imperfect datasets: A review of deep learning solutions for medical image segmentation
Tajbakhsh, N., Jeyaseelan, L., Li, Q., Chiang, J. N., Wu, Z., & Ding, X. (2020) · Medical Image Analysis · doi:10.1016/j.media.2020.101693 · Cited by 1,095
Wang, L., & Yoon, K.-J. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3055564 · Cited by 752
Label Propagation for Deep Semi-Supervised Learning
Iscen, A., Tolias, G., Avrithis, Y., & Chum, O. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00521 · Cited by 686
Wang, G., Li, W., Aertsen, M., Deprest, J., Ourselin, S., & Vercauteren, T. (2019) · Neurocomputing · doi:10.1016/j.neucom.2019.01.103 · Cited by 659
Papandreou, G., Zhu, T., Chen, L.-C., Gidaris, S., Tompson, J., & Murphy, K. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01264-9_17 · Cited by 621
End-to-End Semi-Supervised Object Detection with Soft Teacher
Xu, M., Zhang, Z., Hu, H., Wang, J., Wang, L., Wei, F., Bai, X., & Liu, Z. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00305 · Cited by 565
Edge Assisted Real-time Object Detection for Mobile Augmented Reality
Liu, L., Li, H., & Gruteser, M. (2019) · The 25th Annual International Conference on Mobile Computing and Networking · doi:10.1145/3300061.3300116 · Cited by 552
Learning 3D Human Dynamics From Video
Kanazawa, A., Zhang, J. Y., Felsen, P., & Malik, J. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00576 · Cited by 531
ST++: Make Self-trainingWork Better for Semi-supervised Semantic Segmentation
Yang, L., Zhuo, W., Qi, L., Shi, Y., & Gao, Y. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00423 · Cited by 518
Pham, H., Dai, Z., Xie, Q., & Le, Q. V. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01139 · Cited by 509
CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison
Irvin, J., Rajpurkar, P., Ko, M., Yu, Y., Ciurea-Ilcus, S., Chute, C., Marklund, H., Haghgoo, B., Ball, R., Shpanskaya, K., et al. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.3301590 · Cited by 392
CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison
Irvin, J., Rajpurkar, P., Ko, M., Yu, Y., Ciurea-Ilcus, S., Chute, C., Marklund, H., Haghgoo, B., Ball, R., Shpanskaya, K., et al. (2019) · arXiv · doi:10.48550/arxiv.1901.07031 · Cited by 392
Rethinking Pre-training and Self-training
Zoph, B., Ghiasi, G., Lin, T.-Y., Cui, Y., Liu, H., Cubuk, E. D., & Le, Q. V. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.06882 · Cited by 369
Annotation-efficient deep learning for automatic medical image segmentation
Wang, S., Li, C., Wang, R., Liu, Z., Wang, M., Tan, H., Wu, Y., Liu, X., Sun, H., Yang, R., et al. (2021) · Nature Communications · doi:10.1038/s41467-021-26216-9 · Cited by 312
A Simple Semi-Supervised Learning Framework for Object Detection
Sohn, K., Zhang, Z., Li, C., Zhang, H., Lee, C., & Pfister, T. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.04757 · Cited by 306
Monocular 3D Object Detection Leveraging Accurate Proposals and Shape Reconstruction
Ku, J., Pon, A. D., & Waslander, S. L. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01214 · Cited by 303
Billion-scale semi-supervised learning for image classification
Yalniz, I. Z., Jégou, H., Chen, K., Paluri, M., & Mahajan, D. (2019) · arXiv · doi:10.48550/arxiv.1905.00546 · Cited by 299
WebFace260M: A Benchmark Unveiling the Power of Million-Scale Deep Face Recognition
Zhu, Z., Huang, G., Deng, J., Ye, Y., Huang, J., Chen, X., Zhu, J., Yang, T., Lu, J., Du, D., et al. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01035 · Cited by 290
Deep learning based 2D human pose estimation: A survey
Dang, Q., Yin, J., Wang, B., & Zheng, W. (2019) · Tsinghua Science and Technology · doi:10.26599/tst.2018.9010100 · Cited by 281
Towards Accurate Generative Models of Video: A New Metric & Challenges
Unterthiner, T., van Steenkiste, S., Kurach, K., Marinier, R., Michalski, M., & Gelly, S. (2018) · arXiv · doi:10.48550/arxiv.1812.01717 · Cited by 204
Wang, T., Zhu, J.-Y., Torralba, A., & Efros, A. A. (2018) · arXiv · doi:10.48550/arxiv.1811.10959 · Cited by 194
Naive-Student: Leveraging Semi-Supervised Learning in Video Sequences for Urban Scene Segmentation
Chen, L.-C., Lopes, R. G., Cheng, B., Collins, M. D., Cubuk, E. D., Zoph, B., Adam, H., & Shlens, J. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58545-7_40 · Cited by 165
IncDet: In Defense of Elastic Weight Consolidation for Incremental Object Detection
Liu, L., Kuang, Z., Chen, Y., Xue, J.-H., Yang, W., & Zhang, W. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.3002583 · Cited by 90
Uncertainty Estimates as Data Selection Criteria to Boost Omni-Supervised Learning
Venturini, L., Papageorghiou, A. T., Noble, J. A., & Namburete, A. I. L. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-59710-8_67 · Cited by 18
Thinking Fast and Slow: Efficient Text-to-Visual Retrieval with Transformers
Miech, A., Alayrac, J.-B., Laptev, I., Sivic, J., & Zisserman, A. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00970 · Cited by 149
Transferring Dense Pose to Proximal Animal Classes
Sanakoyeu, A., Khalidov, V., McCarthy, M. S., Vedaldi, A., & Neverova, N. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00528 · Cited by 13
Metadata
- Authors
- Radosavovic, I., Dollar, P., Girshick, R., Gkioxari, G., & He, K.
- Venue
- 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition
- Publisher
- IEEE
- Published
- 2018-06
- Pages
- 4119-4128
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W2963785012
- Cited by
- 421
- References
- 48
- FWCI
- 24.34
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.