CoChat
Sign Up
Cited by 1,271Advanced Neural Network ApplicationsAdvanced Vision and ImagingAutonomous Vehicle Technology and Safety

Lift, Splat, Shoot: Encoding Images from Arbitrary Camera Rigs by Implicitly Unprojecting to 3D

Open access
Philion, Jonah, Fidler, Sanja·2020·Lecture Notes in Computer Science
DOI 10.1007/978-3-030-58568-6_12
Open this paper in CoChat — free accountOpen PDF

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1007/978-3-030-58568-6_12

Cites

8 of 8 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Monocular frameworks process different views separately and cannot capture information across cameras, leading to low performance and efficiency.

    “The downside of this framework is that it processes different views separately and cannot capture information across cameras, leading to low performance and efficiency [32, 47].”

    From BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformers (2022) · cites this paper as [32]

    The Introduction (p. 2) describes a simple multi-view approach that applies a single-image detector to each camera separately and says it cannot learn the best way to fuse information across cameras. The baseline discussion (Section 5.1, p. 9) reports weak performance for one independently encoded multi-view CNN. The paper does not establish that such frameworks cannot capture any cross-camera information or that they have low efficiency. (LLM, conf=0.85)

    Experimental — accuracy not yet measured

  • Depth-based BEV frameworks are sensitive to the accuracy of depth values or depth distributions.

    “A popular BEV framework that generates BEV features is based on depth information [46, 32, 34], but this paradigm is sensitive to the accuracy of depth values or the depth distributions.”

    From BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformers (2022) · cites this paper as [32]

    Section 3.1 (p. 5) says depth is required to transform image features into reference coordinates and that the predicted depth distribution determines where those features are placed. Section 5.1 (p. 9) reports that the model learns an effective depth distribution. These passages support the importance of depth information, but the paper does not directly establish how sensitive BEV performance is to the accuracy of depth values or distributions. (LLM, conf=0.82)

    Experimental — accuracy not yet measured

  • Previous map segmentation methods demonstrate BEV's effectiveness.

    “Although previous map segmentation methods demonstrate BEV's effectiveness [32, 18, 29]”

    From BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformers (2022) · cites this paper as [32]

  • With joint training, BEVFormer outperforms Lift-Splat by 11.0 NDS points on detection (52.0 vs 41.0) and 5.6 IoU points on lane segmentation (23.9 vs 18.3).

    “For example, with joint training, BEVFormer outperforms Lift-Splat ∗ [32] by 11.0 points on detation task (52.0% NDS v.s. 41.0% NDS) and IoU of 5.6 points on lane segmentation (23.9% v.s. 18.3%).”

    From BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformers (2022) · cites this paper as [32]

  • When training BEVFormer with noisy extrinsics, BEVFormer (noise) has stronger robustness, dropping only 8.9% NDS.

    “Following [32], we show that when training BEVFormer with noisy extrinsics, BEVFormer (noise) has stronger robustness (only drops 8.9% NDS).”

    From BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformers (2022) · cites this paper as [32]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 27

  1. [3]3. Chang, M.F., Ramanan, D., Hays, J., Lambert, J., Sangkloy, P., Singh, J., Bak, S., Hartnett, A., Wang, D., Carr, P., et al.: Argoverse: 3d tracking and forecasting with rich maps. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (Jun 2019)
  2. [4]4. Chen, X., Kundu, K., Zhang, Z., Ma, H., Fidler, S., Urtasun, R.: Monocular 3d object detection for autonomous driving. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 2147-2156 (06 2016)
  3. [5]5. Ghiasi, G., Lin, T., Le, Q.V.: Dropblock: A regularization method for convolutional networks. CoRR abs/1810.12890 (2018), http://arxiv.org/abs/1810.12890 · arXiv
  4. [6]6. Goodfellow, I., Bengio, Y., Courville, A.: Deep Learning. MIT Press (2016), http: //www.deeplearningbook.org
  5. [7]7. He, K., Gkioxari, G., Doll´ ar, P., Girshick, R.B.: Mask R-CNN. CoRR abs/1703.06870 (2017), http://arxiv.org/abs/1703.06870 · arXiv
  6. [10]10. Ioffe, S., Szegedy, C.: Batch normalization: Accelerating deep network training by reducing internal covariate shift. CoRR abs/1502.03167 (2015), http://arxiv. org/abs/1502.03167 · arXiv
  7. [11]11. Kayhan, O.S., Gemert, J.C.v.: On translation invariance in cnns: Convolutional layers can exploit absolute spatial location. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (June 2020)
  8. [12]12. Kehl, W., Manhardt, F., Tombari, F., Ilic, S., Navab, N.: SSD-6D: making rgbbased 3d detection and 6d pose estimation great again. CoRR abs/1711.10006 (2017) · arXiv
  9. [14]14. Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. CoRR abs/1412.6980 (2014) · arXiv
  10. [15]15. Kirillov, A., He, K., Girshick, R.B., Rother, C., Doll´ ar, P.: Panoptic segmentation. CoRR abs/1801.00868 (2018), http://arxiv.org/abs/1801.00868 · arXiv
  11. [19]19. Lecun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. In: Proceedings of the IEEE. pp. 2278-2324 (1998)
  12. [20]20. Lombardi, S., Simon, T., Saragih, J., Schwartz, G., Lehrmann, A., Sheikh, Y.: Neural volumes. ACM Transactions on Graphics 38 (4), 114 (Jul 2019). https://doi.org/10.1145/3306346.3323020, http://dx.doi.org/10.1145/ 3306346.3323020 · DOI
  13. [21]21. Mani, K., Daga, S., Garg, S., Shankar, N.S., Jatavallabhula, K.M., Krishna, K.M.: Monolayout: Amodal scene layout from a single image. ArXiv abs/2002.08394 (2020) · arXiv
  14. [22]22. Nair, V., Hinton, G.E.: Rectified linear units improve restricted boltzmann machines. In: ICML (2010)
  15. [23]23. Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., Desmaison, A., K¨ opf, A., Yang, E., DeVito, Z., Raison, M., Tejani, A., Chilamkurthy, S., Steiner, B., Fang, L., Bai, J., Chintala, S.: Pytorch: An imperative style, high-performance deep learning library. In: NeurIPS (2019)
  16. [24]24. Philion, J.: Fastdraw: Addressing the long tail of lane detection by adapting a sequential prediction network. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (June 2019)
  17. [25]25. Philion, J., Kar, A., Fidler, S.: Learning to evaluate perception models using planner-centric metrics. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (June 2020)
  18. [26]26. Poirson, P., Ammirato, P., Fu, C., Liu, W., Kosecka, J., Berg, A.C.: Fast single shot detection and pose estimation. CoRR abs/1609.05590 (2016) · arXiv
  19. [27]27. Qin, Z., Wang, J., Lu, Y.: Monogrnet: A geometric reasoning network for monocular 3d object localization. Proceedings of the AAAI Conference on Artificial Intelligence 33 , 8851-8858 (07 2019). https://doi.org/10.1609/aaai.v33i01.33018851 · DOI · Paper page
  20. [28]28. Roddick, T., Cipolla, R.: Predicting semantic map representations from images using pyramid occupancy networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (June 2020)
  21. [29]29. Roddick, T., Kendall, A., Cipolla, R.: Orthographic feature transform for monocular 3d object detection. CoRR abs/1811.08188 (2018) · arXiv
  22. [31]31. Simonelli, A., Bul` o, S.R., Porzi, L., L´ opez-Antequera, M., Kontschieder, P.: Disentangling monocular 3d object detection. CoRR abs/1905.12365 (2019) · arXiv
  23. [33]33. Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: A simple way to prevent neural networks from overfitting. Journal of Machine Learning Research 15 , 1929-1958 (2014)
  24. [34]34. Su, H., Jampani, V., Sun, D., Maji, S., Kalogerakis, E., Yang, M., Kautz, J.: Splatnet: Sparse lattice networks for point cloud processing. CoRR abs/1802.08275 (2018), http://arxiv.org/abs/1802.08275 · arXiv
  25. [36]36. Takikawa, T., Acuna, D., Jampani, V., Fidler, S.: Gated-scnn: Gated shape cnns for semantic segmentation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) (October 2019)
  26. [40]40. You, Y., Wang, Y., Chao, W., Garg, D., Pleiss, G., Hariharan, B., Campbell, M., Weinberger, K.Q.: Pseudo-lidar++: Accurate depth for 3d object detection in autonomous driving. CoRR abs/1906.06310 (2019) · arXiv
  27. [41]41. Zeng, W., Luo, W., Suo, S., Sadat, A., Yang, B., Casas, S., Urtasun, R.: End-to-end interpretable neural motion planner. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) pp. 8652-8661 (2019)

Cited by

Sorted by citations

Authors
Philion, J., & Fidler, S.
Affiliations
University of Toronto (CA)
Corresponding author
Philion, Jonah
ORCID
Fidler, Sanja
Venue
Lecture Notes in Computer Science
Publisher
Springer International Publishing
Published
2020
Pages
194-210
ISSN
  • 0302-9743 (ISSN-L)
  • 1611-3349
DOI
10.1007/978-3-030-58568-6_12
OpenAlex
W3048770793
Cited by
1,271
References
8
Citations by year
2026: 249 · 2025: 389 · 2024: 381 · 2023: 182 · 2022: 46 · 2021: 23 · 2020: 1
FWCI
19.43
Type
conference paper
Language
en
Open access
Licence
Locations
  • Lecture notes in computer science (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Vision and Imaging
  • Physical Sciences › Engineering › Automotive Engineering › Autonomous Vehicle Technology and Safety
Keywords
bird's-eye-view representation, multi-camera fusion, rasterization, autonomous driving perception, object segmentation, map segmentation, cost map, calibration robustness

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review