Enhancing Geometric Factors in Model Learning and Inference for Object Detection and Instance Segmentation
Open accessThe DOI matches its Crossref record; 57 of the 58 references it cites were found in a registry or an index, and 1 are not checked yet.
Abstract
Deep learning-based object detection and instance segmentation have achieved unprecedented progress. In this article, we propose complete-IoU (CIoU) loss and Cluster-NMS for enhancing geometric factors in both bounding-box regression and nonmaximum suppression (NMS), leading to notable gains of average precision (AP) and average recall (AR), without the sacrifice of inference efficiency. In particular, we consider three geometric factors, that is: 1) overlap area; 2) normalized central-point distance; and 3) aspect ratio, which are crucial for measuring bounding-box regression in object detection and instance segmentation. The three geometric factors are then incorporated into CIoU loss for better distinguishing difficult regression cases. The training of deep models using CIoU loss results in consistent AP and AR improvements in comparison to widely adopted$\ell _{n}$-norm loss and IoU-based loss. Furthermore, we propose Cluster-NMS, where NMS during inference is done by implicitly clustering detected boxes and usually requires fewer iterations. Cluster-NMS is very efficient due to its pure GPU implementation, and geometric factors can be incorporated to improve both AP and AR. In the experiments, CIoU loss and Cluster-NMS have been applied to state-of-the-art instance segmentation (e.g., YOLACT and BlendMask-RT), and object detection (e.g., YOLO v3, SSD, and Faster R-CNN) models. Taking YOLACT on MS COCO as an example, our method achieves performance gains as +1.7 AP and +6.2 AR100for object detection, and +1.1 AP and +3.5 AR100for instance segmentation, with 27.1 FPS on one NVIDIA GTX 1080Ti GPU. All the source code and trained models are available athttps://github.com/Zzh-tju/CIoU.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Although CIoU [44] considers aspect ratio and centroid distance when dealing with bounding box regression, its accuracy of bounding box localization still needs to be improved in some detection tasks with complex backgrounds and small targets.”
From Radiation Anomaly Detection of Sub-Band Optical Remote Sensing Images Based on Multiscale Deep Dynamic Fusion and Adaptive Optimization (2024) · cites this paper as [44]
In the citing paper
“The advantage of this type of method is that it takes full consideration of object relationships and their spatial layout [118, 119].”
From Object Detection in 20 Years: A Survey (2023) · cites this paper as [119]
Citations
Cites
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1506.01497 · Cited by 70,812
You Only Look Once: Unified, Real-Time Object Detection
Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.91 · Cited by 53,803
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460
Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.81 · Cited by 32,684
Girshick, R. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.169 · Cited by 28,105
SSD: Single Shot MultiBox Detector
Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.-Y., & Berg, A. C. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46448-0_2 · Cited by 22,128
Self-citationYOLO9000: Better, Faster, Stronger
Redmon, J., & Farhadi, A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.690 · Cited by 19,892
The pascal visual object classes (VOC) challenge
Everingham, M., Van Gool, L., Williams, C. K. I., Winn, J., & Zisserman, A. (2010) · International Journal of Computer Vision · doi:10.1007/s11263-009-0275-4 · Cited by 19,599
YOLOv3: An Incremental Improvement
Redmon, J., & Farhadi, A. (2018) · arXiv · doi:10.48550/arxiv.1804.02767 · Cited by 17,141
A Multi-Modal Distributed Real-Time IoT System for Urban Traffic Control (Invited Paper)
Khanam, Z., Achari, V. P. S., Boukhennoufa, I., Jindal, A., & Singh, A. K. (2024) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/oasics.ng-res.2024.2 · Cited by 14,040
Object Detection with Discriminatively Trained Part-Based Models
Felzenszwalb, P. F., Girshick, R. B., McAllester, D., & Ramanan, D. (2010) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2009.167 · Cited by 9,853
Focal Loss for Dense Object Detection
Lin, T.-Y., Goyal, P., Girshick, R., He, K., & Dollar, P. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2858826 · Cited by 9,556
Cascade R-CNN: Delving Into High Quality Object Detection
Cai, Z., & Vasconcelos, N. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00644 · Cited by 7,370
Robust estimation of a location parameter
Huber, P. J. (1964) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177703732 · Cited by 7,228
FCOS: Fully Convolutional One-Stage Object Detection
Tian, Z., Shen, C., Chen, H., & He, T. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00972 · Cited by 7,013
Generalized Intersection Over Union: A Metric and a Loss for Bounding Box Regression
Rezatofighi, H., Tsoi, N., Gwak, J., Sadeghian, A., Reid, I., & Savarese, S. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00075 · Cited by 6,612
Deep High-Resolution Representation Learning for Human Pose Estimation
Sun, K., Xiao, B., Liu, D., & Wang, J. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00584 · Cited by 5,984
Distance-IoU Loss: Faster and Better Learning for Bounding Box Regression
Zheng, Z., Wang, P., Liu, W., Li, J., Ye, R., & Ren, D. (2019) · arXiv · doi:10.48550/arxiv.1911.08287 · Cited by 5,313
Self-citationDistance-IoU Loss: Faster and Better Learning for Bounding Box Regression
Zheng, Z., Wang, P., Liu, W., Li, J., Ye, R., & Ren, D. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.6999 · Cited by 5,288
Self-citationVoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection
Zhou, Y., & Tuzel, O. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00472 · Cited by 4,813
He, K., Gkioxari, G., Dollar, P., & Girshick, R. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2844175 · Cited by 3,457
PointRCNN: 3D Object Proposal Generation and Detection From Point Cloud
Shi, S., Wang, X., & Li, H. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00086 · Cited by 3,093
YOLACT: Real-Time Instance Segmentation
Bolya, D., Zhou, C., Xiao, F., & Lee, Y. J. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00925 · Cited by 2,536
Soft-NMS — Improving Object Detection with One Line of Code
Bodla, N., Singh, B., Chellappa, R., & Davis, L. S. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.593 · Cited by 2,274 · cited 4×
“As for improving detection accuracy, Soft-NMS [39] penalizes the detection score of neighbors by a continuous function w.r.t. IoU, yielding softer and more robust suppression than original NMS. IoU-Net [50] introduces a new network branch to predict the localization confidence to guide NMS.…”
BackgroundMethodInfluentialLibra R-CNN: Towards Balanced Learning for Object Detection
Pang, J., Chen, K., Shi, J., Feng, H., Ouyang, W., & Lin, D. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00091 · Cited by 1,763
UnitBox: An Advanced Object Detection Network
Yu, J., Jiang, Y., Wang, Z., Cao, Z., & Huang, T. (2016) · Proceedings of the 24th ACM international conference on Multimedia · doi:10.1145/2964284.2967274 · Cited by 1,731
DSSD : Deconvolutional Single Shot Detector
Fu, C.-Y., Liu, W., Ranga, A., Tyagi, A., & Berg, A. C. (2017) · arXiv · doi:10.48550/arxiv.1701.06659 · Cited by 1,624
Self-citationRepPoints: Point Set Representation for Object Detection
Yang, Z., Liu, S., Hu, H., Wang, L., & Lin, S. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00975 · Cited by 1,307
Arbitrary-Oriented Scene Text Detection via Rotation Proposals
Ma, J., Shao, W., Ye, H., Wang, L., Wang, H., Zheng, Y., & Xue, X. (2018) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2018.2818020 · Cited by 1,260
Acquisition of Localization Confidence for Accurate Object Detection
Jiang, B., Luo, R., Mao, J., Xiao, T., & Jiang, Y. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01264-9_48 · Cited by 1,059
Object Detection via a Multi-region and Semantic Segmentation-Aware CNN Model
Gidaris, S., & Komodakis, N. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.135 · Cited by 801
PolarMask: Single Shot Instance Segmentation With Polar Representation
Xie, E., Sun, P., Song, X., Wang, W., Liu, X., Liang, D., Shen, C., & Luo, P. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01221 · Cited by 641
Bounding Box Regression With Uncertainty for Accurate Object Detection
He, Y., Zhu, C., Wang, J., Savvides, M., & Zhang, X. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00300 · Cited by 623
Advanced Deep-Learning Techniques for Salient and Category-Specific Object Detection: A Survey
Han, J., Zhang, D., Cheng, G., Liu, N., & Xu, D. (2018) · IEEE Signal Processing Magazine · doi:10.1109/msp.2017.2749125 · Cited by 621
BlendMask: Top-Down Meets Bottom-Up for Instance Segmentation
Chen, H., Sun, K., Tian, Z., Shen, C., Huang, Y., & Yan, Y. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00860 · Cited by 601
Rotation-Sensitive Regression for Oriented Scene Text Detection
Liao, M., Zhu, Z., Shi, B., Xia, G.-S., & Bai, X. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00619 · Cited by 563
Gradient Harmonized Single-Stage Detector
Li, B., Liu, Y., & Wang, X. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33018577 · Cited by 530
R2CNN: Rotational Region CNN for Orientation Robust Scene Text Detection
Jiang, Y., Zhu, X., Wang, X., Yang, S., Li, W., Wang, H., Fu, P., & Luo, Z. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.09579 · Cited by 448
Learnable Triangulation of Human Pose
Iskakov, K., Burkov, E., Lempitsky, V., & Malkov, Y. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00781 · Cited by 414
Adaptive NMS: Refining Pedestrian Detection in a Crowd
Liu, S., Huang, D., & Wang, Y. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00662 · Cited by 413
Cheng, G., Han, J., Zhou, P., & Xu, D. (2019) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2867198 · Cited by 408
He, K., Gkioxari, G., Dollár, P., & Girshick, R. (2017) · arXiv · doi:10.48550/arxiv.1703.06870 · Cited by 406
TensorMask: A Foundation for Dense Object Segmentation
Chen, X., Girshick, R., He, K., & Dollar, P. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00215 · Cited by 382
Scale-Transferrable Object Detection
Zhou, P., Ni, B., Geng, C., Hu, J., & Xu, Y. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00062 · Cited by 373
Illuminating Pedestrians via Simultaneous Detection and Segmentation
Brazil, G., Yin, X., & Liu, X. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.530 · Cited by 270
Improving Object Localization with Fitness NMS and Bounded IoU Loss
Tychsen-Smith, L., & Petersson, L. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00719 · Cited by 215
An Alternative Probabilistic Interpretation of the Huber Loss
Meyer, G. P. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00522 · Cited by 174
Face Detection With Different Scales Based on Faster R-CNN
Wu, W., Yin, Y., Wang, X., & Xu, D. (2018) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2018.2859482 · Cited by 145
Scene-Specific Pedestrian Detection for Static Video Surveillance
Wang, X., Wang, M., & Li, W. (2014) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.124 · Cited by 144
Towards Unconstrained End-to-End Text Spotting
Qin, S., Bissaco, A., Raptis, M., Fujii, Y., & Xiao, Y. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00480 · Cited by 133
Siam R-CNN: Visual Tracking by Re-Detection
Voigtlaender, P., Luiten, J., Torr, P. H. S., & Leibe, B. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00661 · Cited by 115
DIOD: Fast and Efficient Weakly Semi-Supervised Deep Complex ISAR Object Detection
Xue, B., & Tong, N. (2018) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2018.2856821 · Cited by 82
CAD: Scale Invariant Framework for Real-Time Object Detection
Zhou, H., Li, Z., Ning, C., & Tang, J. (2017) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2017.95 · Cited by 79
SSA-CNN: Semantic Self-Attention CNN for Pedestrian DetectionPreprint
Zhou, C., Wu, M., & Lam, S.-K. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.09080 · Cited by 55
Occlusion Handling via Random Subspace Classifiers for Human Detection
Marín, J., Vázquez, D., López, A. M., Amores, J., & Kuncheva, L. I. (2013) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2013.2255271 · Cited by 54
Object Detection Based on Region Decomposition and Assembly
Bae, S. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33018094 · Cited by 38
Work-efficient parallel non-maximum suppression for embedded GPU architectures
Oro, D., Fernández, C., Martorell, X., & Hernando, J. (2016) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2016.7471831 · Cited by 30
Research works printed but not linked: 1
- 10.1007/978–3-319-10602-148 · doi:10.1007/978-3-319-10602-148
Cited by
Object Detection in 20 Years: A Survey
Zou, Z., Chen, K., Shi, Z., Guo, Y., & Ye, J. (2023) · Proceedings of the IEEE · doi:10.1109/jproc.2023.3238524 · Cited by 3,689
“object relationships and their spatial layout [118], [119].”
Slim-neck by GSConv: a lightweight-design for real-time detector architectures
Li, H., Li, J., Wei, H., Liu, Z., Zhan, Z., & Ren, Q. (2024) · Journal of Real-Time Image Processing · doi:10.1007/s11554-024-01436-6 · Cited by 886
MPDIoU: A Loss for Efficient and Accurate Bounding Box RegressionPreprint
Ma, S., & Xu, Y. D. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2307.07662 · Cited by 824
SIoU Loss: More Powerful Learning for Bounding Box Regression
Gevorgyan, Z. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2205.12740 · Cited by 570
A Real-Time Detection Algorithm for Kiwifruit Defects Based on YOLOv5
Yao, J., Qi, J., Zhang, J., Shao, H., Yang, J., & Li, X. (2021) · Electronics · doi:10.3390/electronics10141711 · Cited by 337
Metadata
- Authors
- Zheng, Z., Wang, P., Ren, D., Liu, W., Ye, R., Hu, Q., & Zuo, W.
- Affiliations
- Tianjin University (CN)
- Harbin Institute of Technology (CN)
- Venue
- IEEE Transactions on Cybernetics
- Published
- 2022-08
- Volume
- 52
- Issue
- 8
- Pages
- 8574-8586
- ISSN
- 2168-2267 (ISSN-L)
- 2168-2275
- OpenAlex
- W3194790201
- Cited by
- 1,552
- References
- 52
- Citations by year
- 2026: 239 · 2025: 466 · 2024: 475 · 2023: 242 · 2022: 102
- FWCI
- 75.93
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Transactions on Cybernetics (publishedVersion)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Natural Science Foundation of China (61801326, U19A2073)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Visual Attention and Saliency Detection
- Keywords
- bounding box regression, non-maximum suppression, geometric factor, overlap area, aspect ratio, instance segmentation, object detection, YOLACT, inference efficiency
- MeSH terms
- Deep Learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.