CoChat
Sign Up
Cited by 1,048Advanced Neural Network Applications3D Shape Modeling and AnalysisAutonomous Vehicle Technology and Safety

Voxel R-CNN: Towards High Performance Voxel-based 3D Object Detection

Open access
Deng, Jiajun, Shi, Shaoshuai, Li, Peiwei, Zhou, Wengang, Zhang, Yanyong, Li, Houqiang·2021·Proceedings of the AAAI Conference on Artificial Intelligence
DOI 10.1609/aaai.v35i2.16207
Open this paper in CoChat — free account

Abstract

Recent advances on 3D object detection heavily rely on how the 3D data are represented, i.e., voxel-based or point-based representation. Many existing high performance 3D detectors are point-based because this structure can better retain precise point positions. Nevertheless, point-level features lead to high computation overheads due to unordered storage. In contrast, the voxel-based structure is better suited for feature extraction but often yields lower accuracy because the input data are divided into grids. In this paper, we take a slightly different viewpoint --- we find that precise positioning of raw points is not essential for high performance 3D object detection and that the coarse voxel granularity can also offer sufficient detection accuracy. Bearing this view in mind, we devise a simple but effective voxel-based framework, named Voxel R-CNN. By taking full advantage of voxel features in a two-stage approach, our method achieves comparable detection accuracy with state-of-the-art point-based models, but at a fraction of the computation cost. Voxel R-CNN consists of a 3D backbone network, a 2D bird-eye-view (BEV) Region Proposal Network, and a detect head. A voxel RoI pooling is devised to extract RoI features directly from voxel features for further refinement. Extensive experiments are conducted on the widely used KITTI Dataset and the more recent Waymo Open Dataset. Our results show that compared to existing voxel-based methods, Voxel R-CNN delivers a higher detection accuracy while maintaining a real-time frame processing rate, i.e., at a speed of 25 FPS on an NVIDIA RTX 2080 Ti GPU. The code is available at https://github.com/djiajunustc/Voxel-R-CNN.

AI snapshot

Written from the paper’s full text.

TL;DR

Voxel R-CNN is a two-stage voxel-based 3D detector that directly extracts RoI features from 3D voxel feature volumes via voxel RoI pooling, achieving accuracy comparable to point-based detectors at lower computation. On KITTI test it reaches 81.62% moderate AP for car at 25.2 FPS, and on Waymo Open Dataset it reaches 75.59% LEVEL 1 3D mAP.

Key findings

  • On the KITTI test set, Voxel R-CNN achieves 90.90/81.62/77.06 AP3D for easy/moderate/hard car at 25.2 FPS, outperforming SA-SSD by 2.15/1.83/2.90 absolute improvements and matching PV-RCNN with about 1/3 running time.
  • On the KITTI val set with 11 recall positions, Voxel R-CNN achieves 89.41/84.52/78.93 AP3D for easy/moderate/hard car, with the best moderate and hard results in the comparison.
  • On the Waymo Open Dataset validation set, Voxel R-CNN achieves 75.59% LEVEL 1 3D mAP and 88.19% LEVEL 1 BEV mAP, outperforming PV-RCNN by 5.29% mAP on LEVEL 1 3D and by +10.98% mAP in the 50m-Inf range.
  • Ablation shows adding a detect head to the BEV baseline boosts moderate AP by 4.22%; voxel query improves FPS by 1.7; the accelerated PointNet module boosts FPS from 17.4 to 21.4; the final model reaches 84.52 moderate AP at 25.2 FPS.

Methods

  • Voxel R-CNN is a two-stage voxel-based framework: a 3D backbone, a 2D BEV backbone with RPN, and a detect head with voxel RoI pooling. Voxel RoI pooling divides each proposal into a 6×6×6 grid, uses voxel query with Manhattan distance thresholds 2 and 4, and aggregates neighboring voxel features with an accelerated PointNet module.
  • Experiments use KITTI (7481 training, 7518 testing; train/val split 3712/3769) and Waymo Open Dataset (798 training sequences, ~158k samples; 202 validation sequences, ~40k samples), evaluating car/vehicle detection with AP/mAP at IoU 0.7.
  • Training uses Adam; KITTI is trained for 80 epochs with batch size 16, Waymo for 30 epochs with batch size 32, learning rate 0.01 with cosine annealing. Inference applies NMS with IoU 0.7 and keeps top-100 proposals, then NMS with IoU 0.1 after refinement.

Limitations

  • The paper does not include a dedicated limitations section. It acknowledges that voxelization often causes loss of precise position information, though it argues this is not essential for high performance 3D object detection.
  • Evident from the text: evaluation focuses on car/vehicle detection on KITTI and Waymo, with no pedestrian or cyclist results reported.
  • Evident from the text: performance is tied to specific voxel sizes and point-cloud clipping ranges, and no failure-case or robustness analysis is provided.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1609/aaai.v35i2.16207

Cites

48 of 48 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 14 cited sources checked · 0 supported · 0 flagged

  • Findingown data
    § 2.2 Analysis · p. 2

    The point-voxel interaction takes almost half of the overall running time in PV-RCNN.

    “the point-voxel interaction takes almost half of the overall running time”
    Quote in the paperSubstantial claim
  • Findingown data
    § 2.2 Analysis · p. 2

    PV-RCNN is much slower than SECOND.

    “which makes PV-RCNN much slower than SECOND”
    Quote in the paperSubstantial claim
  • Findingreasoning
    § 2.2 Analysis · p. 2

    There exists a large gap between SECOND and PV-RCNN in terms of detection performance (accuracy and efficiency).

    “There exists a large gap between SECOND and PV-RCNN in terms of the detection performance ( i.e. , accuracy and efficiency)”
    Quote in the paperSubstantial claim
  • Findingown data
    § 2.2 Analysis · p. 2

    Adding a BEV detect head on top of BEV features still trails the accuracy of PV-RCNN.

    “but has still trailed the accuracy of PV-RCNN thus far”
    Quote in the paperSubstantial claim
  • Findingown data
    § 2.2 Analysis · p. 2

    Directly adding a BEV detect head on top of BEV features leads to 0.6% AP improvement for KITTI car moderate data.

    “directly adding a BEV detect head on top of BEV features leads to 0.6% AP improvement for KITTI car moderate data”
    Quote in the paperSubstantial claim
  • May combine two claims. 15
  • Classified as probably not a substantial claim. 2
  • The quote only partly states this claim. 1

  • May combine two claims.

    Findingown data
    § Abstract · p. 1

    Voxel R-CNN delivers higher detection accuracy than existing voxel-based methods while maintaining a realtime frame processing rate of 25 FPS on an NVIDIA RTX 2080 Ti GPU.

    “Voxel R-CNN delivers a higher detection accuracy while maintaining a realtime frame processing rate, i.e., at a speed of 25 FPS on an NVIDIA RTX 2080 Ti GPU”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 4.3 Results on KITTI Dataset · p. 6

    Voxel R-CNN achieves comparable accuracy with PV-RCNN while using only about one third of the running time.

    “Particularly, V oxel RCNNachieves comparable accuracy with the strongest competitor, i.e. , PV-RCNN (Shi et al. 2020a), with only about 1 / 3 running time.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Shi et al. 2020a

      Not checked in the free run.

  • May combine two claims.

    Findingown data
    § 4.3 Results on KITTI Dataset · p. 6

    Voxel R-CNN achieves 81.62% average precision (AP) on the moderate level of class Car on the KITTI test set, with a real-time processing frame rate of 25.2 FPS.

    “By taking full advantage of voxel-based representation, Voxel R-CNN achieves 81 . 62% average precision (AP) on the moderate level of class Car, with the real time processing frame rate ( 25 . 2 FPS).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingreasoning
    § Abstract · p. 1

    Voxel R-CNN achieves comparable detection accuracy with state-of-the-art point-based models, but at a fraction of the computation cost.

    “our method achieves comparable detection accuracy with state-of-the-art point-based models, but at a fraction of the computation cost”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingcitation
    § 3.1 Voxel RoI Pooling · p. 4

    The time complexity of voxel query is O(K), whereas the time complexity of ball query is O(N).

    “Suppose there are N non-empty voxels in the 3D feature volumes and we utilize ball query to find neighboring voxels to a given query point, the time complexity is O ( N ) . Nevertheless, the time complexity of conducting voxel query is only O ( K )”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Qi et al. 2017b

      Not checked in the free run.

  • May combine two claims.

    Findingreasoning
    § 3.1 Voxel RoI Pooling · p. 4

    The grouped voxels occupy a lot of memory and lead to large computation FLOPs when applying the FC layer.

    “The grouped voxels occupy a lot of memories and lead to large computation FLOPs ( O ( M × K × ( C +3) × C ′ ) ) when applying the FC layer.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 2.2 Analysis · p. 3

    The point-voxel feature interaction is time-consuming and affects the detector's efficiency.

    “the point-voxel feature interaction is time-consuming and affects the detector's efficiency”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 4.3 Results on KITTI Dataset · p. 6

    Voxel R-CNN outperforms all existing voxel-based models by a large margin, with absolute improvements of 2.15%, 1.83%, and 2.90% for easy, moderate, and hard levels over SA-SSD.

    “Besides, Voxel R-CNN outperforms all of the existing voxel-based models by a large margin, i.e. , 2 . 15% , 1 . 83% , 2 . 90% absolute improvements for easy, moderate and hard level over the SA-SSD(He et al. 2020).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites He et al. 2020

      Not checked in the free run.

  • May combine two claims.

    Findingown data
    § 4.5 Ablation Study · p. 7

    Method (b) extends (a) with a detect head for box refinement, leading to a boost of 4.22% moderate AP.

    “Method (b) extends (a) with a detect head for box refinement, which leads to a boost of 4 . 22% moderate AP.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 4.5 Ablation Study · p. 7

    Method (a), the one-stage baseline that performs detection on BEV features, runs at 40.8 FPS but has unsatisfactory AP.

    “Method (a) is the one stage baseline that performs detection on BEV features. It runs at 40.8 FPS, but with unsatisfactory AP”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingreasoning
    § 4.5 Ablation Study · p. 7

    Method (c) replaces ball query with voxel query, which makes a 1.7 FPS improvement by taking advantage of the neighbor-aware property of voxel-based representations.

    “Method (c) replaces ball query with our voxel query, which makes 1.7 FPS improvement by taking advantage of the neighbor-aware property of voxel-based representations .”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Premisereasoning
    § 1 Introduction · p. 1

    Point-based methods are in general less efficient because it is more costly to search nearest neighbor with the point representation for point set abstraction.

    “point-based methods are in general less efficient because it is more costly to search nearest neighbor with the point representation for point set abstraction”
    Quote in the paperNot a substantial claim

  • May combine two claims. The quote only partly states this claim.

    Interpretationown data
    § Abstract · p. 1

    Coarse voxel granularity can offer sufficient detection accuracy and spatial context cues for high performance 3D object detection.

    “that the coarse voxel granularity can also offer sufficient detection accuracy”
    • Also stated in: § 1 Introduction · p. 2 · plain
      “coarser voxel granularity can also offer sufficient spatial context cues for this task”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisecitation
    § 1 Introduction · p. 1

    Voxel-based methods are more applicable for convolutional neural networks and more efficient for feature extraction due to superior memory locality.

    “The voxel-based methods (Zhou and Tuzel 2018; Yan, Mao, and Li 2018; Lang et al. 2019) divide point clouds into regular grids, which are more applicable for convolutional neural networks (CNNs) and more efficient for feature extraction due to superior memory locality”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites Zhou and Tuzel 2018

      Not checked in the free run.

    • cites Yan, Mao, and Li 2018

      Not checked in the free run.

    • cites Lang et al. 2019

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Premisenone given
    § 1 Introduction · p. 2

    Point-based methods commonly rely on abstracted point representations to restore 3D structure context and make further refinement based on point-wise features.

    “point-based methods commonly rely on abstracted point representations to restore 3D structure context and make further refinement based on the point-wise features”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning
    § 4.5 Ablation Study · p. 7

    Method (e), the proposed Voxel R-CNN, achieves state-of-the-art accuracy for 3D object detection and maintains the high efficiency of voxel-based models.

    “Voxel R-CNN achieves the state-of-the-art accuracy for 3D object detection and maintain the high efficiency of voxel-based models.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data
    § 4.3 Results on KITTI Dataset · p. 6

    Voxel R-CNN achieves state-of-the-art average precision on 3D object detection on both the KITTI val set and test set.

    “Overall, the results on both val set and test set consistently demonstrate that our proposed Voxel R-CNN achieves the state-of-the-art average precision on 3D object detection and keeps the high efficiency of voxel-based models.”
    Quote in the paperSubstantial claim
17 of 17

  • The paper relies on this without support.

    Voxelization often causes loss of precise position information.

    “Nevertheless, the downside is that voxelization often causes loss of precise position information.”
1 of 1

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Unlike ROI-level fusion of MV3D and AVOD, MVX-Net's VoxelFusion is voxel-level.

    “Unlike the ROI-level fusion of MV3D [5] and AVOD [71], the Vox-elFusion fusion strategy proposed in MVX-Net [79] is voxellevel.”

    From Multimodal fusion and vision–language models: A survey for robot vision (2026) · cites this paper as [79]

    The comparison table (4.1 Datasets, p. 5) lists MV3D and AVOD-FPN as RGB+LiDAR detectors, but does not describe their fusion level. The discussion of Voxel RoI pooling (3.1, pp. 3–4) concerns Voxel R-CNN, not MVX-Net; searches for MVX-Net and VoxelFusion found no passage addressing the claimed voxel-level fusion. (LLM, conf=0.78)

    Experimental — accuracy not yet measured

  • VoxelFusion can fuse at voxel feature encoding layer, aggregating image features into corresponding voxels and concatenating with voxel point cloud features.

    “VoxelFusion can fuse at the voxel feature encoding layer, aggregating image features into corresponding voxels, and concatenating them with voxel point cloud features.”

    From Multimodal fusion and vision–language models: A survey for robot vision (2026) · cites this paper as [79]

    The Introduction (p. 2) describes Voxel R-CNN, and §3.1 (p. 4) describes concatenating voxel features pooled from different stages and scales. Searches for VoxelFusion and image-feature concatenation found no passage about fusing image features into corresponding voxels at the voxel feature encoding layer. (LLM, conf=0.76)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Voxel R-CNN does not have the best hard-level validation AP

    Minor

    Table 5 gives Voxel R-CNN a hard-level AP of 78.93, below STD's 79.30, despite the claim that it performs best on that level.

1 of 1

Cites

Flagged first, then by citations

Research works printed but not linked: 7

  1. Chen, X.; Ma, H.; Wan, J.; Li, B.; and Xia, T. 2017. Multiview 3d object detection network for autonomous driving. In CVPR , 1907-1915.
  2. Geiger, A.; Lenz, P.; and Urtasun, R. 2012. Are we ready for autonomous driving? In CVPR , 3354-3361.
  3. Girshick, R. 2015. Fast r-cnn. In ICCV , 1440-1448.
  4. Yang, Z.; Sun, Y.; Liu, S.; and Jia, J. 2020. 3DSSD: Pointbased 3D single stage object detector. In CVPR , 1104011048.
  5. Yilun Chen, Shu Liu, X. S.; and Jia, J. 2019. Fast point rcnn. In ICCV , 9774-9783.
  6. Zhao, X.; Liu, Z.; Hu, R.; and Huang, K. 2019. 3D Object Detection Using Scale Invariant and Feature Reweighting Networks. In AAAI , 9267-9274.
  7. Zhou, Y.; Sun, P.; Zhang, Y.; Anguelov, D.; Gao, J.; Ouyang, T.; Guo, J.; Ngiam, J.; and Vasudevan, V. 2020. End-toend multi-view fusion for 3d object detection in lidar point clouds. In CoRL , 923-932.

Cited by

Sorted by citations

Related

Authors
Deng, J., Shi, S., Li, P., Zhou, W., Zhang, Y., & Li, H.
Affiliations
  • University of Science and Technology of China (CN)
  • Chinese University of Hong Kong (HK)
ORCID
Venue
Proceedings of the AAAI Conference on Artificial Intelligence
Publisher
Association for the Advancement of Artificial Intelligence (AAAI)
Published
2021-05-18
Volume
35
Issue
2
Pages
1201-1209
ISSN
  • 2159-5399 (ISSN-L)
  • 2374-3468
DOI
10.1609/aaai.v35i2.16207
OpenAlex
W3118341329
Semantic Scholar
efc669f75ad0ed2033e7c499ae2d6c8dab7489ce
Cited by
1,048
References
48
Citations by year
2026: 116 · 2025: 267 · 2024: 276 · 2023: 260 · 2022: 106 · 2021: 23
FWCI
54.28
Type
conference paper
Language
en
Open access
Locations
  • Proceedings of the AAAI Conference on Artificial Intelligence (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) (submittedVersion)
  • arXiv (Cornell University)
Funders
  • National Natural Science Foundation of China (61836011, 62021001, 61822208, 61632019, 61822208, 61632019)
  • Youth Innovation Promotion Association of the Chinese Academy of Sciences (2018497)
  • Youth Innovation Promotion Association (2018497)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
  • Physical Sciences › Engineering › Computational Mechanics › 3D Shape Modeling and Analysis
  • Physical Sciences › Engineering › Automotive Engineering › Autonomous Vehicle Technology and Safety
Keywords
3D object detection, voxel-based representation, region proposal network, two-stage detection, point clouds, real-time detection, KITTI dataset, Waymo Open Dataset

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-08.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-08.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review