Bounding Box Regression With Uncertainty for Accurate Object Detection
Open accessAbstract
Large-scale object detection datasets (e.g., MS-COCO) try to define the ground truth bounding boxes as clear as possible. However, we observe that ambiguities are still introduced when labeling the bounding boxes. In this paper, we propose a novel bounding box regression loss for learning bounding box transformation and localization variance together. Our loss greatly improves the localization accuracies of various architectures with nearly no additional computation. The learned localization variance allows us to merge neighboring bounding boxes during non-maximum suppression (NMS), which further improves the localization performance. On MS-COCO, we boost the Average Precision (AP) of VGG-16 Faster R-CNN from 23.6% to 29.1%. More importantly, for ResNet-50-FPN Mask R-CNN, our method improves the AP and AP90 by 1.8% and 6.2% respectively, which significantly outperforms previous state-of-the-art bounding box refinement methods. Our code and models are available at github.com/yihui-he/KL-Loss.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Many works have been proposed to solve the problems of greedy selection.
“Many works have been proposed to solve the problems mentioned above [107, 112, 114, 115].”
From Object Detection in 20 Years: A Survey (2023) · cites this paper as [114]
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · see citing works
Self-citationVery Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 74,248 · see citing works
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1506.01497 · Cited by 70,812 · see citing works
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · see citing works
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 56,035 · see citing works
Distinctive Image Features from Scale-Invariant Keypoints
Lowe, D. G. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000029664.99615.94 · Cited by 55,589 · see citing works
You Only Look Once: Unified, Real-Time Object Detection
Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.91 · Cited by 53,803 · see citing works
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460 · see citing works
Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.81 · Cited by 32,684 · see citing works
(2016) · Cited by 31,590 · see citing works
Feature Pyramid Networks for Object Detection
Lin, T.-Y., Dollar, P., Girshick, R., He, K., Hariharan, B., & Belongie, S. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.106 · Cited by 30,939 · see citing works
Focal Loss for Dense Object Detection
Lin, T.-Y., Goyal, P., Girshick, R., He, K., & Dollar, P. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.324 · Cited by 30,581 · see citing works
He, K., Gkioxari, G., Dollar, P., & Girshick, R. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.322 · Cited by 29,137 · see citing works
Girshick, R. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.169 · Cited by 28,105 · see citing works
Caffe: Convolutional Architecture for Fast Feature Embedding
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., & Darrell, T. (2014) · Proceedings of the 22nd ACM international conference on Multimedia · doi:10.1145/2647868.2654889 · Cited by 11,077 · see citing works
ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices
Zhang, X., Zhou, X., Lin, M., & Sun, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00716 · Cited by 9,982 · see citing works
Self-citationFocal Loss for Dense Object Detection
Lin, T.-Y., Goyal, P., Girshick, R., He, K., & Dollar, P. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2858826 · Cited by 9,556 · see citing works
Deformable Convolutional Networks
Dai, J., Qi, H., Xiong, Y., Li, Y., Zhang, G., Hu, H., & Wei, Y. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.89 · Cited by 7,513 · see citing works
Cascade R-CNN: Delving Into High Quality Object Detection
Cai, Z., & Vasconcelos, N. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00644 · Cited by 7,370 · see citing works
Video Google: a text retrieval approach to object matching in videos
Sivic, & Zisserman (2003) · Proceedings Ninth IEEE International Conference on Computer Vision · doi:10.1109/iccv.2003.1238663 · Cited by 6,367 · see citing works
R-FCN: Object Detection via Region-based Fully Convolutional NetworksPreprint
Dai, J., Li, Y., He, K., & Sun, J. Z. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.06409 · Cited by 5,135 · see citing works
CornerNet: Detecting Objects as Paired Keypoints
Law, H., & Deng, J. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01264-9_45 · Cited by 3,852 · see citing works
Multi-task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics
Cipolla, R., Gal, Y., & Kendall, A. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00781 · Cited by 3,657 · see citing works
What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?
Kendall, A., & Gal, Y. (2017) · arXiv · doi:10.48550/arxiv.1703.04977 · Cited by 3,088 · see citing works
Channel Pruning for Accelerating Very Deep Neural Networks
He, Y., Zhang, X., & Sun, J. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.155 · Cited by 2,647 · see citing works
Self-citationAccurate, Large Minibatch SGD: Training ImageNet in 1 Hour
Goyal, P., Dollár, P., Girshick, R., Noordhuis, P., Wesolowski, L., Kyrola, A., Tulloch, A., Jia, Y., & He, K. (2017) · arXiv · doi:10.48550/arxiv.1706.02677 · Cited by 2,294 · see citing works
Soft-NMS — Improving Object Detection with One Line of Code
Bodla, N., Singh, B., Chellappa, R., & Davis, L. S. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.593 · Cited by 2,274 · cited 3× · see citing works
“Recently, soft NMS and learning NMS [1, 24] are proposed to improve NMS results.”
BackgroundThe PASCAL visual object classes challenge 2006 (VOC2006) results
Everingham, M., Zisserman, A., Williams, C. K. I., & Gool, L. V. (2006) · Cited by 2,079 · see citing works
UnitBox: An Advanced Object Detection Network
Yu, J., Jiang, Y., Wang, Z., Cao, Z., & Huang, T. (2016) · Proceedings of the 24th ACM international conference on Multimedia · doi:10.1145/2964284.2967274 · Cited by 1,731 · see citing works
(2008) · doi:10.1145/1401132.1401152 · Cited by 1,575 · see citing works
Relation Networks for Object Detection
Hu, H., Gu, J., Zhang, Z., Dai, J., & Wei, Y. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00378 · Cited by 1,524 · see citing works
AMC: AutoML for Model Compression and Acceleration on Mobile Devices
He, Y., Lin, J., Liu, Z., Wang, H., Li, L.-J., & Han, S. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01234-2_48 · Cited by 1,400 · see citing works
Self-citationcuDNN: Efficient Primitives for Deep LearningPreprint
Chetlur, S., Woolley, C., Vandermersch, P., Cohen, J., Tran, J. J., Catanzaro, B., & Shelhamer, E. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1410.0759 · Cited by 1,394 · see citing works
Feature Selective Anchor-Free Module for Single-Shot Object Detection
Zhu, C., He, Y., & Savvides, M. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00093 · Cited by 1,066 · see citing works
Self-citationAcquisition of Localization Confidence for Accurate Object Detection
Jiang, B., Luo, R., Mao, J., Xiao, T., & Jiang, Y. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01264-9_48 · Cited by 1,059 · see citing works
Object Detection via a Multi-region and Semantic Segmentation-Aware CNN Model
Gidaris, S., & Komodakis, N. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.135 · Cited by 801 · see citing works
Edge and Curve Detection for Visual Scene Analysis
Rosenfeld, A., & Thurston, M. (1971) · IEEE Transactions on Computers · doi:10.1109/t-c.1971.223290 · Cited by 792 · see citing works
Learning Non-maximum Suppression
Hosang, J., Benenson, R., & Schiele, B. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.685 · Cited by 736 · see citing works
YouTube-BoundingBoxes: A Large High-Precision Human-Annotated Data Set for Object Detection in Video
Real, E., Shlens, J., Mazzocchi, S., Pan, X., & Vanhoucke, V. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.789 · Cited by 580 · see citing works
CrowdHuman: A Benchmark for Detecting Human in a Crowd
Shao, S., Zhao, Z., Li, B., Xiao, T., Yu, G., Zhang, X., & Sun, J. (2018) · arXiv · doi:10.48550/arxiv.1805.00123 · Cited by 534 · cited 3× · see citing works
“Large scale object detection datasets like ImageNet [6], MS-COCO [35] and CrowdHuman [47] try to define the ground truth bounding boxes as clear as possible.”
BackgroundMethodSelf-citationAMC: AutoML for Model Compression and Acceleration on Mobile DevicesPreprint
He, Y., Lin, J., Liu, Z., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.03494 · Cited by 353 · see citing works
Self-citationMachine Learning, a Probabilistic Perspective
Robert, C. P. (2014) · CHANCE · doi:10.1080/09332480.2014.914768 · Cited by 342 · see citing works
Light-Head R-CNN: In Defense of Two-Stage Object Detector
Li, Z., Peng, C., Yu, G., Zhang, X., Deng, Y., & Sun, J. (2017) · arXiv · doi:10.48550/arxiv.1711.07264 · Cited by 292 · see citing works
Self-citationFeng, D., Rosenbaum, L., & Dietmayer, K. (2018) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc.2018.8569814 · Cited by 260 · see citing works
Non-maximum Suppression for Object Detection by Passing Messages Between Windows
Rothe, R., Guillaumin, M., & Van Gool, L. (2015) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-16865-4_19 · Cited by 218 · see citing works
Leveraging Heteroscedastic Aleatoric Uncertainties for Robust Real-Time LiDAR 3D Object Detection
Feng, D., Rosenbaum, L., Timm, F., & Dietmayer, K. (2019) · IEEE Intelligent Vehicles Symposium · doi:10.1109/ivs.2019.8814046 · Cited by 79 · see citing works
Softer-NMS: Rethinking Bounding Box Regression for Accurate Object Detection.Preprint
He, Y., Zhang, X., Savvides, M., & Kitani, K. (2018) · arXiv (Cornell University) · Cited by 67 · see citing works
Self-citationMotion Prediction of Traffic Actors for Autonomous Driving using Deep Convolutional Networks.Preprint
Djuric, N., Radosavljević, V., Cui, H., et al. (2018) · arXiv (Cornell University) · Cited by 56 · see citing works
Optimized Pedestrian Detection for Multiple and Occluded People
Rujikietgumjorn, S., & Collins, R. T. (2013) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2013.473 · Cited by 40 · see citing works
Learning 6-DoF Grasping and Pick-Place Using Attention FocusPreprint
Gualtieri, M., & Platt, R. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.06134 · Cited by 33 · see citing works
AddressNet: Shift-Based Primitives for Efficient Convolutional Neural Networks
He, Y., Liu, X., Zhong, H., & Ma, Y. (2019) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv.2019.00134 · Cited by 33 · see citing works
Self-citationLearning Globally Optimized Object Detector via Policy Gradient
Rao, Y., Lin, D., Lu, J., et al. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00648 · Cited by 29 · see citing works
Camera Placement Based on Vehicle Traffic for Better City Security Surveillance
Ma, X., He, Y., Luo, X., et al. (2018) · IEEE Intelligent Systems · doi:10.1109/mis.2018.223110904 · Cited by 28 · see citing works
Self-citationSingle Image Super-resolution via a Lightweight Residual Convolutional Neural NetworkPreprint
Liang, Y., Yang, Z., Zhang, K., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.08173 · Cited by 13 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Object Detection in 20 Years: A Survey
Zou, Z., Chen, K., Shi, Z., Guo, Y., & Ye, J. (2023) · Proceedings of the IEEE · doi:10.1109/jproc.2023.3238524 · Cited by 3,689 · see citing works
Deep Learning for Generic Object Detection: A Survey
Liu, L., Ouyang, W., Wang, X., Fieguth, P., Chen, J., Liu, X., & Pietikäinen, M. (2020) · International Journal of Computer Vision · doi:10.1007/s11263-019-01247-4 · Cited by 2,965 · see citing works
A review of uncertainty quantification in deep learning: Techniques, applications and challenges
Abdar, M., Pourpanah, F., Hussain, S., Rezazadegan, D., Liu, L., Ghavamzadeh, M., Fieguth, P., Cao, X., Khosravi, A., Acharya, U. R., et al. (2021) · Information Fusion · doi:10.1016/j.inffus.2021.05.008 · Cited by 2,720 · see citing works
“To date, there have been number of studies addressed uncertainty in deep learning algorithms for various applications including but not limited to image/video retrieval [399], [400], depth estimation [401], [402], object detection [403], [404], [405], semantic segmentation and scene understanding [406], [407], [408],…”
BackgroundZheng, Z., Wang, P., Ren, D., Liu, W., Ye, R., Hu, Q., & Zuo, W. (2022) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2021.3095305 · Cited by 1,552 · see citing works
Distance-IoU Loss: Faster and Better Learning for Bounding Box Regression
Zheng, Z., Wang, P., Liu, W., Li, J., Ye, R., & Ren, D. (2019) · arXiv · doi:10.48550/arxiv.1911.08287 · Cited by 5,313 · see citing works
A survey and performance evaluation of deep learning methods for small object detection
Liu, Y., Sun, P., Wergeles, N., & Shang, Y. (2021) · Expert Systems with Applications · doi:10.1016/j.eswa.2021.114602 · Cited by 665 · see citing works
A review of object detection based on deep learning
Xiao, Y., Tian, Z., Yu, J., Zhang, Y., Liu, S., Du, S., & Lan, X. (2020) · Multimedia Tools and Applications · doi:10.1007/s11042-020-08976-6 · Cited by 584 · see citing works
Bai, X., Wang, X., Liu, X., Liu, Q., Song, J., Sebe, N., & Kim, B. (2021) · Pattern Recognition · doi:10.1016/j.patcog.2021.108102 · Cited by 313 · see citing works
Deep Learning for Generic Object Detection: A Survey
Liu, L., Ouyang, W., Wang, X., Fieguth, P., Chen, J., Liu, X., & Pietikäinen, M. (2018) · arXiv · doi:10.48550/arxiv.1809.02165 · Cited by 2,965 · see citing works
End-to-End Object Detection with Fully Convolutional Network
Wang, J., Song, L., Li, Z., Sun, H., Sun, J., & Zheng, N. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01559 · Cited by 235 · see citing works
“To tackle this issue, Learnable NMS [13], Soft NMS [1] and other NMS variants [12, 24, 15], and CenterNet [5] are”
BackgroundSelf-citationLUVLi Face Alignment: Estimating Landmarks’ Location, Uncertainty, and Visibility Likelihood
Kumar, A., Marks, T. K., Mou, W., Wang, Y., Jones, M., Cherian, A., Koike-Akino, T., Liu, X., & Feng, C. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00826 · Cited by 165 · see citing works
3D Object Detection from Images for Autonomous Driving: A Survey
Ma, X., Ouyang, W., Simonelli, A., & Ricci, E. (2022) · arXiv · doi:10.48550/arxiv.2202.02980 · Cited by 147 · see citing works
Learning to Rank Proposals for Object Detection
Tan, Z., Nie, X., Qian, Q., Li, N., & Li, H. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00836 · Cited by 56 · see citing works
From Algorithm to Hardware: A Survey on Efficient and Safe Deployment of Deep Neural Networks
Geng, X., Wang, Z., Chen, C., Xu, Q., Xu, K., Jin, C., Gupta, M., Yang, X., Chen, Z., Sabry Aly, M. M., et al. (2025) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3394494 · Cited by 26 · see citing works
- Authors
- He, Y., Zhu, C., Wang, J., Savvides, M., & Zhang, X.
- Affiliations
- Carnegie Mellon University (US)
- Megvii (China) (CN)
- Venue
- 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2019-06
- Pages
- 2883-2892
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W2962677013
- Cited by
- 623
- References
- 54
- Citations by year
- 2026: 31 · 2025: 66 · 2024: 85 · 2023: 99 · 2022: 113 · 2021: 127 · 2020: 88 · 2019: 13 · 2018: 1
- FWCI
- 27.48
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Visual Attention and Saliency Detection
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image and Video Retrieval Techniques
- Keywords
- bounding box regression, uncertainty estimation, object detection, non-maximum suppression, bounding box refinement, Mask R-CNN, Faster R-CNN
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
