CoChat
Sign Up
PreprintCited by 288Advanced Neural Network ApplicationsEmbedded Systems Design TechniquesParallel Computing and Optimization Techniques

Objects as Points

Open access
Zhou, Xingyi, Wang, Dequan, Krähenbühl, Philipp·2019·arXiv
DOI 10.48550/arxiv.1904.07850arXiv 1904.07850
Open this paper in CoChat — free accountOpen PDF

Abstract

Detection identifies objects as axis-aligned boxes in an image. Most successful object detectors enumerate a nearly exhaustive list of potential object locations and classify each. This is wasteful, inefficient, and requires additional post-processing. In this paper, we take a different approach. We model an object as a single point --- the center point of its bounding box. Our detector uses keypoint estimation to find center points and regresses to all other object properties, such as size, 3D location, orientation, and even pose. Our center point based approach, CenterNet, is end-to-end differentiable, simpler, faster, and more accurate than corresponding bounding box based detectors. CenterNet achieves the best speed-accuracy trade-off on the MS COCO dataset, with 28.1% AP at 142 FPS, 37.4% AP at 52 FPS, and 45.1% AP with multi-scale testing at 1.4 FPS. We use the same approach to estimate 3D bounding box in the KITTI benchmark and human pose on the COCO keypoint dataset. Our method performs competitively with sophisticated multi-stage methods and runs in real-time.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.1904.07850

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Existing methods [1,2] typically build their detection pipelines purely from 2D computations.

    “Existing methods [1, 2] typically build their detection pipelines purely from 2D computations.”

    From DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queries (2021) · cites this paper as [1]

    The Preliminary section (p. 3) describes detection from image-based, two-dimensional keypoint heatmaps using fully convolutional networks. But §4.1 (p. 4) also describes predicting depth, 3D dimensions, and orientation; the paper does not establish that detection pipelines are purely 2D, or that existing methods typically are. (LLM, conf=0.76)

    Experimental — accuracy not yet measured

  • Existing 2D-based methods predict 3D information like object pose and velocity using an object detection pipeline designed for 2D tasks (e.g., CenterNet, FCOS), without considering 3D scene structure or sensor configuration.

    “That is, they predict 3D information like object pose and velocity using an object detection pipeline designed for 2D tasks (e.g., CenterNet [1], FCOS [3]), without considering 3D scene structure or sensor configuration.”

    From DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queries (2021) · cites this paper as [1]

    The Abstract (p. 1) identifies CenterNet as an object detector that regresses 3D location, orientation, and pose from detected center points; Section 4.1 (p. 4) details its depth, dimension, and orientation outputs. The paper does not report predicting velocity or establish the broader claim that these methods disregard 3D scene structure or sensor configuration. (LLM, conf=0.85)

    Experimental — accuracy not yet measured

  • Modern detectors' performances are significantly influenced by postprocessing steps, anchor sets, and heuristics that assign target boxes to anchors.

    “Their performances are significantly influenced by postprocessing steps to collapse near-duplicate predictions, by the design of the anchor sets and by the heuristics that assign target boxes to anchors [52].”

    From End-to-End Object Detection with Transformers (2020) · cites this paper as [52]

    The Introduction (p. 1) says conventional detectors use non-maxima suppression to remove duplicate detections, and Related Work (p. 2) describes anchor designs and overlap thresholds used to assign boxes. Additional Experiments (p. 6) reports missed objects from insufficient anchor placement, supporting an effect on detection performance. The copy does not establish that all three factors significantly influence performance; its measured NMS effect is minor and concerns CenterNet. (LLM, conf=0.78)

    Experimental — accuracy not yet measured

  • Recent work demonstrates that the final performance of these systems heavily depends on the exact way these initial guesses are set.

    “Recent work [52] demonstrate that the final performance of these systems heavily depends on the exact way these initial guesses are set.”

    From End-to-End Object Detection with Transformers (2020) · cites this paper as [52]

    The Introduction (p. 1) describes anchor boxes as potential object locations, and Additional experiments (p. 6) reports that insufficient anchor placement can cause missed objects. This supports the importance of how those initial candidates are set, but the paper does not demonstrate that final system performance heavily depends on their exact configuration. (LLM, conf=0.78)

    Experimental — accuracy not yet measured

  • NMS-free detectors were developed to release from NMS and achieve a fully end-to-end object detection training network by completing one-to-one label assignment.

    “To release from NMS and achieve a fully end-to-end object detection training network, researchers developed a series of methods to complete one-to-one label assignment (a.k.a. one object with just one prediction box) [28, 40, 120].”

    From Object Detection in 20 Years: A Survey (2023) · cites this paper as [40]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

The citation lists haven’t been fetched yet

Cited by 288

Research works printed but not linked: 52

  1. [2][2] N. Bodla, B. Singh, R. Chellappa, and L. S. Davis. Soft-nmsimproving object detection with one line of code. In ICCV , 2017. -
  2. [3][3] Z. Cao, G. Hidalgo, T. Simon, S.-E. Wei, and Y. Sheikh. OpenPose: realtime multi-person 2D pose estimation using Part Affinity Fields. In arXiv preprint arXiv:1812.08008 , 2018. - · arXiv
  3. [6][6] F. Chabot, M. Chaouch, J. Rabarisoa, C. Teuliere, and T. Chateau. Deep manta: A coarse-to-fine manytask network for joint 2d and 3d vehicle analysis from monocular image. In CVPR , 2017. -
  4. [7][7] L.-C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and A. L. Yuille. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. TPAMI , 2018. -
  5. [8][8] X. Chen and A. Gupta. An implementation of faster rcnn with study for region sampling. arXiv preprint arXiv:1702.02138 , 2017. - · arXiv
  6. [9][9] X. Chen, K. Kundu, Z. Zhang, H. Ma, S. Fidler, and R. Urtasun. Monocular 3d object detection for autonomous driving. In CVPR , 2016. -
  7. [10][10] X. Chen, K. Kundu, Y. Zhu, A. G. Berneshawi, H. Ma, S. Fidler, and R. Urtasun. 3d object proposals for accurate object class detection. In NIPS , 2015. -
  8. [11][11] J. Dai, Y. Li, K. He, and J. Sun. R-fcn: Object detection via region-based fully convolutional networks. In NIPS , 2016. -
  9. [13][13] D. Eigen, C. Puhrsch, and R. Fergus. Depth map prediction from a single image using a multi-scale deep network. In NIPS , 2014. -
  10. [15][15] H.-S. Fang, S. Xie, Y.-W. Tai, and C. Lu. RMPE: Regional multi-person pose estimation. In ICCV , 2017. -
  11. [16][16] C.-Y. Fu, W. Liu, A. Ranga, A. Tyagi, and A. C. Berg. Dssd: Deconvolutional single shot detector. arXiv preprint arXiv:1701.06659 , 2017. - · arXiv
  12. [17][17] A. Geiger, P. Lenz, and R. Urtasun. Are we ready for autonomous driving? the kitti vision benchmark suite. In CVPR , 2012. -
  13. [18][18] R. Girshick. Fast r-cnn. In ICCV , 2015. -
  14. [19][19] R. Girshick, J. Donahue, T. Darrell, and J. Malik. Rich feature hierarchies for accurate object detection and semantic segmentation. In CVPR , 2014. -
  15. [20][20] P. Goyal, P. Doll´ ar, R. Girshick, P. Noordhuis, L. Wesolowski, A. Kyrola, A. Tulloch, Y. Jia, and K. He. Accurate, large minibatch sgd: Training imagenet in 1 hour. arXiv preprint arXiv:1706.02677 , 2017. - · arXiv
  16. [21][21] K. He, G. Gkioxari, P. Doll´ ar, and R. Girshick. Mask r-cnn. In ICCV , 2017. -
  17. [23][23] J. H. Hosang, R. Benenson, and B. Schiele. Learning non-maximum suppression. In CVPR , 2017. -
  18. [25][25] J. Huang, V. Rathod, C. Sun, M. Zhu, A. Korattikara, A. Fathi, I. Fischer, Z. Wojna, Y. Song, S. Guadarrama, et al. Speed/accuracy trade-offs for modern convolutional object detectors. In CVPR , 2017. -
  19. [26][26] B. Jiang, R. Luo, J. Mao, T. Xiao, and Y. Jiang. Acquisition of localization confidence for accurate object detection. In ECCV , 2018. -
  20. [27][27] Z. Kalal, K. Mikolajczyk, and J. Matas. Trackinglearning-detection. TPAMI , 2012. -
  21. [28][28] D. P. Kingma and J. Ba. Adam: A method for stochastic optimization. ICLR , 2014. -
  22. [29][29] A. Kundu, Y. Li, and J. M. Rehg. 3d-rcnn: Instance-level 3d object reconstruction via renderand-compare. In CVPR , 2018. -
  23. [30][30] H. Law and J. Deng. Cornernet: Detecting objects as paired keypoints. In ECCV , 2018. -
  24. [31][31] Y. Li, Y. Chen, N. Wang, and Z. Zhang. Scale-aware trident networks for object detection. arXiv preprint arXiv:1901.01892 , 2019. - · arXiv
  25. [32][32] Y. Li, H. Qi, J. Dai, X. Ji, and Y. Wei. Fully convolutional instance-aware semantic segmentation. In CVPR , 2017. -
  26. [34][34] T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Doll´ ar, and C. L. Zitnick. Microsoft coco: Common objects in context. In ECCV , 2014. -
  27. [35][35] S. Liu, L. Qi, H. Qin, J. Shi, and J. Jia. Path aggregation network for instance segmentation. In CVPR , 2018. -
  28. [36][36] W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C.-Y. Fu, and A. C. Berg. Ssd: Single shot multibox detector. In ECCV , 2016. -
  29. [37][37] J. Long, E. Shelhamer, and T. Darrell. Fully convolutional networks for semantic segmentation. In CVPR , 2015. -
  30. [38][38] A. Mousavian, D. Anguelov, J. Flynn, and J. Kosecka. 3d bounding box estimation using deep learning and geometry. In CVPR , 2017. -
  31. [39][39] A. Newell, Z. Huang, and J. Deng. Associative embedding: End-to-end learning for joint detection and grouping. In NIPS , 2017. -
  32. [41][41] G. Papandreou, T. Zhu, L.-C. Chen, S. Gidaris, J. Tompson, and K. Murphy. Personlab: Person pose estimation and instance segmentation with a bottomup, part-based, geometric embedding model. ECCV , 2018. -
  33. [42][42] G. Papandreou, T. Zhu, N. Kanazawa, A. Toshev, J. Tompson, C. Bregler, and K. Murphy. Towards accurate multi-person pose estimation in the wild. In CVPR , 2017. -
  34. [44][44] J. Redmon and A. Farhadi. Yolo9000: better, faster, stronger. CVPR , 2017. -
  35. [45][45] J. Redmon and A. Farhadi. Yolov3: An incremental improvement. arXiv preprint arXiv:1804.02767 , 2018. - · arXiv
  36. [46][46] S. Ren, K. He, R. Girshick, and J. Sun. Faster r-cnn: Towards real-time object detection with region proposal networks. In NIPS , 2015. -
  37. [47][47] B. Singh and L. S. Davis. An analysis of scale invariance in object detection-snip. In CVPR , 2018. -
  38. [48][48] B. Singh, M. Najibi, and L. S. Davis. SNIPER: Efficient multi-scale training. NIPS , 2018. -
  39. [49][49] X. Sun, J. Shang, S. Liang, and Y. Wei. Compositional human pose regression. In ICCV , 2017. -
  40. [50][50] X. Sun, B. Xiao, F. Wei, S. Liang, and Y. Wei. Integral human pose regression. In ECCV , 2018. -
  41. [51][51] A. Toshev and C. Szegedy. Deeppose: Human pose estimation via deep neural networks. In CVPR , 2014. -
  42. [52][52] J. R. Uijlings, K. E. Van De Sande, T. Gevers, and A. W. Smeulders. Selective search for object recognition. IJCV , 2013. -
  43. [53][53] D. Wang, C. Devin, Q.-Z. Cai, F. Yu, and T. Darrell. Deep object centric policies for autonomous driving. In ICRA , 2019. -
  44. [54][54] Y. Xiang, W. Choi, Y. Lin, and S. Savarese. Subcategory-aware convolutional neural networks for object proposals and detection. In WACV , 2017. -
  45. [55][55] B. Xiao, H. Wu, and Y. Wei. Simple baselines for human pose estimation and tracking. In ECCV , 2018. -
  46. [56][56] S. Xie, R. Girshick, P. Doll´ ar, Z. Tu, and K. He. Aggregated residual transformations for deep neural networks. In CVPR , 2017. -
  47. [57][57] J. Xu, R. Zhao, F. Zhu, H. Wang, and W. Ouyang. Attention-aware compositional network for person reidentification. In CVPR , 2018. -
  48. [58][58] F. Yu, D. Wang, E. Shelhamer, and T. Darrell. Deep layer aggregation. In CVPR , 2018. -
  49. [59][59] S. Zhang, L. Wen, X. Bian, Z. Lei, and S. Z. Li. Single-shot refinement neural network for object detection. In CVPR , 2018. -
  50. [60][60] X. Zhou, A. Karpur, L. Luo, and Q. Huang. Starmap for category-agnostic keypoint and viewpoint estimation. In ECCV , 2018. -
  51. [62][62] C. Zhu, Y. He, and M. Savvides. Feature selective anchor-free module for single-shot object detection. arXiv preprint arXiv:1903.00621 , 2019. - · arXiv
  52. [63][63] X. Zhu, H. Hu, S. Lin, and J. Dai. Deformable convnets v2: More deformable, better results. arXiv preprint arXiv:1811.11168 , 2018. Figure 6: Model diagrams. The numbers in the boxes represent the stride to the image. (a): Hourglass Network [30]. We use it as is in CornerNet [30]. (b): ResNet with transpose convolutions [55]. We add one 3 × 3 deformable convolutional layer [63] before each up-sampling layer. Specifically, we first use deformable convolution to change the channels and then use transposed convolution to upsample the feature map (such two steps are shown separately in 32 → 16 . We show these two steps together as a dashed arrow for 16 → 8 and 8 → 4 ). (c): The original DLA-34 [58] for semantic segmentation. (d): Our modified DLA-34. We add more skip connections from the bottom layers and upgrade every convolutional layer in upsampling stages to deformable convolutional layer. <!-- image --> · arXiv

Authors
Zhou, X., Wang, D., & Krähenbühl, P.
Affiliations
  • BMW (Germany) (DE)
  • Technical University of Munich (DE)
ORCID
Krähenbühl, Philipp
Venue
arXiv
Publisher
arXiv
Published
2019
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.1904.07850
arXiv
1904.07850
OpenAlex
W4394657967
Cited by
288
References
0
Citations by year
2026: 16 · 2025: 23 · 2024: 56 · 2023: 83 · 2022: 62 · 2021: 35 · 2020: 10 · 2019: 3
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
  • Physical Sciences › Computer Science › Hardware and Architecture › Embedded Systems Design Techniques
  • Physical Sciences › Computer Science › Hardware and Architecture › Parallel Computing and Optimization Techniques
Keywords
embedded inference, real-time inference, ResNet, DeepLabV3+, design space exploration

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review