The DOI matches its Crossref record; 48 of the 48 references it cites were found in a registry or an index.
Tri-Perspective View for Vision-Based 3D Semantic Occupancy Prediction
Open accessAbstract
Modern methods for vision-centric autonomous driving perception widely adopt the bird's-eye-view (BEV) representation to describe a 3D scene. Despite its better efficiency than voxel representation, it has difficulty describing the fine-grained 3D structure of a scene with a single plane. To address this, we propose a tri-perspective view (TPV) representation which accompanies BEV with two additional perpendicular planes. We model each point in the 3D space by summing its projected features on the three planes. To lift image features to the 3D TPV space, we further propose a transformer-based TPV encoder (TPVFormer) to obtain the TPV features effectively. We employ the attention mechanism to aggregate the image features corresponding to each query in each TPV plane. Experiments show that our model trained with sparse supervision effectively predicts the semantic occupancy for all voxels. We demonstrate for the first time that using only camera inputs can achieve comparable performance with LiDAR-based methods on the LiDAR segmentation task on nuScenes. Code: https://github.com/wzzheng/TPVFormer.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks
Tan, M., & Le, Q. V. (2019) · arXiv · doi:10.48550/arxiv.1905.11946 · Cited by 17,072
Deformable Convolutional Networks
Dai, J., Qi, H., Xiong, Y., Li, Y., Zhang, G., Hu, H., & Wei, Y. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.89 · Cited by 7,513 · cited 4×
“TPVFormer-Base uses the ResNet101DCN [15, 19] initialized from FCOS3D [44] checkpoint, while TPVFormer-Small adopts the ResNet-50 [19] pretrained on ImageNet [16].”
MethodInfluentialnuScenes: A Multimodal Dataset for Autonomous Driving
Caesar, H., Bankiti, V., Lang, A. H., Vora, S., Liong, V. E., Xu, Q., Krishnan, A., Pan, Y., Baldan, G., & Beijbom, O. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01164 · Cited by 7,152
NeRF: representing scenes as neural radiance fields for view synthesis
Mildenhall, B., Srinivasan, P. P., Tancik, M., Barron, J. T., Ramamoorthi, R., & Ng, R. (2022) · Communications of the ACM · doi:10.1145/3503250 · Cited by 6,852
Deformable DETR: Deformable Transformers for End-to-End Object Detection
Zhu, X., Su, W., Lu, L., Li, B., Wang, X., & Dai, J. (2020) · arXiv · doi:10.48550/arxiv.2010.04159 · Cited by 6,027 · cited 1×
“As a workaround, we employ the efficient deformable attention [15,53] to implement image cross-attention.”
MethodVoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection
Zhou, Y., & Tuzel, O. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00472 · Cited by 4,813
PointPillars: Fast Encoders for Object Detection From Point Clouds
Lang, A. H., Vora, S., Caesar, H., Zhou, L., Yang, J., & Beijbom, O. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01298 · Cited by 4,673
DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation
Park, J. J., Florence, P., Straub, J., Newcombe, R., & Lovegrove, S. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00025 · Cited by 3,422
Occupancy Networks: Learning 3D Reconstruction in Function Space
Mescheder, L., Oechsle, M., Niemeyer, M., Nowozin, S., & Geiger, A. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00459 · Cited by 2,708
SemanticKITTI: A Dataset for Semantic Scene Understanding of LiDAR Sequences
Behley, J., Garbade, M., Milioto, A., Quenzel, J., Behnke, S., Stachniss, C., & Gall, J. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00939 · Cited by 2,294
4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks
Choy, C., Gwak, J., & Savarese, S. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00319 · Cited by 2,043
Li, Z., Wang, W., Li, H., Xie, E., Sima, C., Lu, T., Yu, Q., & Dai, J. (2022) · arXiv · doi:10.48550/arxiv.2203.17270 · Cited by 1,419
Bevfusion: Multi-task multi-sensor fusion with unified bird’s-eye view representation
Liu, Z., Tang, H., Amini, A., Yang, X., Mao, H., Rus, D. L., & Han, S. (2023) · 2023 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra48891.2023.10160968 · Cited by 1,307
Lift, Splat, Shoot: Encoding Images from Arbitrary Camera Rigs by Implicitly Unprojecting to 3D
Philion, J., & Fidler, S. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58568-6_12 · Cited by 1,271
TensoRF: Tensorial Radiance Fields
Chen, A., Xu, Z., Geiger, A., Yu, J., & Su, H. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-19824-3_20 · Cited by 1,184
Efficient Geometry-aware 3D Generative Adversarial Networks
Chan, E. R., Lin, C. Z., Chan, M. A., Nagano, K., Pan, B., de Mello, S., Gallo, O., Guibas, L., Tremblay, J., Khamis, S., et al. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01565 · Cited by 1,093
Learning Continuous Image Representation with Local Implicit Image Function
Chen, Y., Liu, S., & Wang, X. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00852 · Cited by 782
BEVDepth: Acquisition of Reliable Depth for Multi-View 3D Object Detection
Li, Y., Ge, Z., Yu, G., Yang, J., Wang, Z., Shi, Y., Sun, J., & Li, Z. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i2.25233 · Cited by 769
Searching Efficient 3D Architectures with Sparse Point-Voxel Convolution
Tang, H., Liu, Z., Zhao, S., Lin, Y., Lin, J., Wang, H., & Han, S. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58604-1_41 · Cited by 733
FCOS3D: Fully Convolutional One-Stage Monocular 3D Object Detection
Wang, T., Zhu, X., Pang, J., & Lin, D. (2021) · arXiv · doi:10.48550/arxiv.2104.10956 · Cited by 683
FCOS3D: Fully Convolutional One-Stage Monocular 3D Object Detection
Wang, T., Zhu, X., Pang, J., & Lin, D. (2021) · 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW) · doi:10.1109/iccvw54120.2021.00107 · Cited by 683 · cited 4×
“TPVFormer-Base uses the ResNet101DCN [15, 19] initialized from FCOS3D [44] checkpoint, while TPVFormer-Small adopts the ResNet-50 [19] pretrained on ImageNet [16].”
MethodInfluentialKiloNeRF: Speeding up Neural Radiance Fields with Thousands of Tiny MLPs
Reiser, C., Peng, S., Liao, Y., & Geiger, A. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01407 · Cited by 668
PETR: Position Embedding Transformation for Multi-view 3D Object Detection
Liu, Y., Wang, T., Zhang, X., & Sun, J. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-19812-0_31 · Cited by 647
Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR Segmentation
Zhu, X., Zhou, H., Wang, T., Hong, F., Ma, Y., Li, W., Li, H., & Lin, D. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00981 · Cited by 623
Categorical Depth Distribution Network for Monocular 3D Object Detection
Reading, C., Harakeh, A., Chae, J., & Waslander, S. L. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00845 · Cited by 595 · cited 2×
“Recent studies in BEV-based perception focus on how to effectively transform features from the image space to the BEV space [20, 26, 27, 35, 36, 48].”
BackgroundPolarNet: An Improved Grid Representation for Online LiDAR Point Clouds Semantic Segmentation
Zhang, Y., Zhou, Z., David, P., Yue, X., Xi, Z., Gong, B., & Foroosh, H. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00962 · Cited by 590
Self-citationDeep Local Shapes: Learning Local SDF Priors for Detailed 3D Reconstruction
Chabra, R., Lenssen, J. E., Ilg, E., Schmidt, T., Straub, J., Lovegrove, S., & Newcombe, R. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58526-6_36 · Cited by 380
BEVerse: Unified Perception and Prediction in Birds-Eye-View for Vision-Centric Autonomous DrivingPreprint
Zhang, Y., Zhu, Z. D., Zheng, W., Huang, J., Huang, G., Zhou, J., & Lu, J. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2205.09743 · Cited by 329
Self-citationMonoScene: Monocular 3D Semantic Scene Completion
Cao, A.-Q., & de Charette, R. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00396 · Cited by 293
BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-View
Huang, J., Huang, G., Zhu, Z. D., Ye, Y., & Du, D. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2112.11790 · Cited by 293
Yan, X., Gao, J., Li, J., Zhang, R., Li, Z., Huang, R., & Cui, S. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i4.16419 · Cited by 282
FIERY: Future Instance Prediction in Bird’s-Eye View from Surround Monocular Cameras
Hu, A., Murez, Z., Mohan, N., Dudas, S., Hawke, J., Badrinarayanan, V., Cipolla, R., & Kendall, A. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01499 · Cited by 277
Cheng, R., Razani, R., Taghavi, E., Li, E., & Liu, B. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01236 · Cited by 264
Panoptic Nuscenes: A Large-Scale Benchmark for LiDAR Panoptic Segmentation and Tracking
Fong, W. K., Mohan, R., Hurtado, J. V., Zhou, L., Caesar, H., Beijbom, O., & Valada, A. (2022) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2022.3148457 · Cited by 201
BEVFusion: A Simple and Robust LiDAR-Camera Fusion Framework
Liang, T., Xie, H., Yu, K., Xia, Z., Lin, Z., Wang, Y., Tang, T., Wang, B., & Tang, Z. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2205.13790 · Cited by 201
LMSCNet: Lightweight Multiscale 3D Semantic CompletionPreprint
Rold\~ao, L., de Charette, R., & Verroust-Blondet, A. (2020) · arXiv (Cornell University) · Cited by 170
Anisotropic Convolutional Networks for 3D Semantic Scene Completion
Li, J., Han, K., Wang, P., Liu, Y., & Yuan, X. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00341 · Cited by 144
3D Sketch-Aware Semantic Scene Completion via Semi-Supervised Structure Prior
Chen, X., Lin, K.-Y., Qian, C., Zeng, G., & Li, H. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00425 · Cited by 140
Unifying Voxel-based Representation with Transformer for 3D Object Detection
Li, Y., Chen, Y., Qi, X., Li, Z., Sun, J., & Jia, J. (2022) · arXiv · doi:10.48550/arxiv.2206.00630 · Cited by 125
LidarMultiNet: Towards a Unified Multi-Task Network for LiDAR Perception
Ye, D., Zhou, Z., Chen, W., Xie, Y., Wang, Y., Wang, P., & Foroosh, H. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i3.25429 · Cited by 125
Multi-Scale Interaction for Real-Time LiDAR Data Segmentation on an Embedded Platform
Li, S., Chen, X., Liu, Y., Dai, D., Stachniss, C., & Gall, J. (2021) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2021.3132059 · Cited by 122
Full Surround Monodepth From Multiple Cameras
Guizilini, V., Vasiljevic, I., Ambruş, R., Shakhnarovich, G., & Gaidon, A. (2022) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2022.3150884 · Cited by 78
StretchBEV: Stretching Future Instance Prediction Spatially and Temporally
Akan, A. K., & Güney, F. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19839-7_26 · Cited by 70
DRINet++: Efficient Voxel-as-point Point Cloud SegmentationPreprint
Ye, M., Wan, R., Xu, S., Cao, T., & Chen, Q. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.08318 · Cited by 37
A Simple Baseline for Multi-Camera 3D Object Detection
Zhang, Y., Zheng, W., Zhu, Z. D., Huang, G., Lu, J., & Zhou, J. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i3.25460 · Cited by 37
Self-citationPolarStream: Streaming Object Detection and Segmentation with Polar Pillars
Chen, Q., Vora, S., & Beijbom, O. (2021) · neural information processing systems · Cited by 27
Cited by
OccWorld: Learning a 3D Occupancy World Model for Autonomous Driving
Zheng, W., Chen, W., Huang, Y., Zhang, B., Duan, Y., & Lu, J. (2023) · arXiv · doi:10.48550/arxiv.2311.16038 · Cited by 342
Self-citationBEVFormer: Learning Bird’s-Eye-View Representation From LiDAR-Camera via Spatiotemporal Transformers
Li, Z., Wang, W., Li, H., Xie, E., Sima, C., Lu, T., Yu, Q., & Dai, J. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3515454 · Cited by 285
SurroundOcc: Multi-Camera 3D Occupancy Prediction for Autonomous Driving
Wei, Y., Zhao, L., Zheng, W., Zhu, Z., Zhou, J., & Lu, J. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01986 · Cited by 260
Self-citationOpenOccupancy: A Large Scale Benchmark for Surrounding Semantic Occupancy Perception
Wang, X., Zhu, Z., Xu, W., Zhang, Y., Wei, Y., Chi, X., Ye, Y., Du, D., Lu, J., & Wang, X. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01636 · Cited by 192
Self-citationOccFusion: Multi-Sensor Fusion Framework for 3D Semantic Occupancy Prediction
Ming, Z., Stephany Berrio, J., Shan, M., & Worrall, S. (2025) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2024.3453293 · Cited by 35
“However, the currently proposed 3D semantic occupancy prediction models [1], [2], [3], [4], [5], [6], [7], [8], [9], [10], [11], [12], [13], primarily focus on vision-based approaches (Fig.”
Metadata
- Authors
- Huang, Y., Zheng, W., Zhang, Y., Zhou, J., & Lu, J.
- Venue
- 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2023-06
- Pages
- 9223-9232
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W4386083035
- Cited by
- 312
- References
- 48
- FWCI
- 35.48
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.