CoChat
Sign Up
PreprintCited by 1,419Advanced Neural Network ApplicationsAutonomous Vehicle Technology and SafetyAdvanced Vision and Imaging

BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformers

Open access
Li, Zhiqi, Wang, Wenhai, Li, Hongyang, Xie, Enze, Sima, Chonghao, Lu, Tong, Yu, Qiao, Dai, Jifeng·2022·arXiv
DOI 10.48550/arxiv.2203.17270arXiv 2203.17270
Open this paper in CoChat — free accountOpen PDF

Abstract

3D visual perception tasks, including 3D detection and map segmentation based on multi-camera images, are essential for autonomous driving systems. In this work, we present a new framework termed BEVFormer, which learns unified BEV representations with spatiotemporal transformers to support multiple autonomous driving perception tasks. In a nutshell, BEVFormer exploits both spatial and temporal information by interacting with spatial and temporal space through predefined grid-shaped BEV queries. To aggregate spatial information, we design spatial cross-attention that each BEV query extracts the spatial features from the regions of interest across camera views. For temporal information, we propose temporal self-attention to recurrently fuse the history BEV information. Our approach achieves the new state-of-the-art 56.9% in terms of NDS metric on the nuScenes test set.

AI snapshot

Written from the paper’s full text.

TL;DR

BEVFormer is a spatiotemporal transformer encoder that generates unified bird’s-eye-view (BEV) features from multi-camera images for 3D detection and map segmentation. It achieves 56.9% NDS on the nuScenes test set, 9.0 points above DETR3D, and shows temporal information improves velocity estimation and recall for occluded objects.

Key findings

  • On nuScenes test, BEVFormer reaches 56.9% NDS and 48.1% mAP, outperforming DETR3D by 9.0 NDS points (47.9% NDS); on nuScenes val, it reaches 51.7% NDS vs. DETR3D’s 42.5% NDS.
  • Temporal self-attention greatly improves velocity estimation: BEVFormer’s mAVE is 0.378 m/s on nuScenes test, approaching LiDAR-based methods such as PointPainting.
  • With joint detection and segmentation training, BEVFormer achieves 52.0% NDS vs. Lift-Splat*’s 41.0% NDS, and 23.9% lane IoU vs. 18.3% for Lift-Splat*.
  • On the lowest-visibility subset (0–40% visible objects), BEVFormer’s average recall outperforms BEVFormer-S and DETR3D by more than 6.0%.

Methods

  • BEVFormer uses 6 transformer encoder layers with grid-shaped learnable BEV queries, spatial cross-attention based on deformable attention, and temporal self-attention that recurrently fuses history BEV features aligned by ego-motion.
  • Experiments use nuScenes and Waymo; backbones are ResNet101-DCN initialized from FCOS3D and VoVNet-99 initialized from DD3D, with detection and map segmentation heads based on Deformable DETR and Panoptic SegFormer.
  • Training uses 24 epochs, learning rate 2×10⁻⁴, AdamW, and randomly samples 4 frames from the past 2 seconds; inference is online and chronological, saving the previous timestamp’s BEV features.

Limitations

  • The paper states that camera-based methods still have a particular gap with LiDAR-based methods in effect and efficiency, and accurate 3D location inference from 2D information remains a long-standing challenge.
  • Evident from the text: BEVFormer relies on camera intrinsics/extrinsics and degrades under extrinsics noise; at noise level 4, default BEVFormer drops 14.3% NDS, while training with noisy extrinsics reduces the drop to 8.9%.
  • Evident from the text: inference is bottlenecked by the backbone (391 ms of 540 ms total for the default configuration), and joint training shows negative transfer for road and lane segmentation compared with individual training.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

DataCite · 10.48550/arxiv.2203.17270

Cites

61 of 61 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

26 of 39 cited sources checked · 11 supported · 2 flagged · accuracy not yet measured

  • Interpretationreasoning
    § 3.4 Temporal Self-Attention · p. 6

    BEVFormer's temporal self-attention can more effectively model long temporal dependency than simply stacking BEV features.

    “Compared to simply stacking BEV in [18, 38, 6], our temporal self-attention can more effectively model long temporal dependency.”
    Quote in the paperSubstantial claim2 of 3 sources checked
    • cites [18] [18] Hu, A., Murez, Z., Mohan, N., Dudas, S., Hawke, J., Badrinarayanan, V., Cipolla, R., Kendall, A.: Fiery: Future instance prediction in bird's-eye view from surround monocular cameras. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 15273-15282 (2021) - source
    • cites [38] [38] Saha, A., Maldonado, O.M., Russell, C., Bowden, R.: Translating images into maps. arXiv preprint arXiv:2110.00966 (2021) - source

      The citation is attached to the description of prior work as ‘simply stacking BEV.’ In ‘C. Model architecture’ (page unavailable), the cited paper instead says it applies axial attention across spatial and temporal axes to BEV features from multiple timesteps. Its temporal method is therefore not simply stacking features; the paper does not assess BEVFormer’s comparative claim. (LLM, conf=0.86)

      Checked by a reasoning model — may be wrong.

      Experimental — accuracy not yet measured
    • cites [6] [6] Can, Y.B., Liniger, A., Unal, O., Paudel, D., Van Gool, L.: Understanding bird's-eye view semantic hd-maps using an onboard monocular camera. arXiv preprint arXiv:2012.03040 (2020) - source

      The citation is attached to the characterization of this paper as ‘simply stacking BEV,’ not to the citing authors’ claim about their own self-attention. The source’s ‘Aggregation’ section (page unavailable) instead describes warping features into BEV, combining frames with symmetric max or mean aggregation, and then concatenating the resulting maps. It does not simply stack BEV features across frames. (LLM, conf=0.85)

      Checked by a reasoning model — may be wrong.

      Experimental — accuracy not yet measured
    • Rests on an assumption: Simply stacking BEV features of cross timestamps brings extra computational cost and interference information, which might not be ideal.
    • Rests on an assumption: Stacking BEV features constraints the available temporal information within fixed time duration and brings extra computational cost.
  • Findingcitation
    § B Robustness on Camera Extrinsics · p. 17

    When training BEVFormer with noisy extrinsics, BEVFormer (noise) has stronger robustness, dropping only 8.9% NDS.

    “Following [32], we show that when training BEVFormer with noisy extrinsics, BEVFormer (noise) has stronger robustness (only drops 8.9% NDS).”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [32] [32] Philion, J., Fidler, S.: Lift, splat, shoot: Encoding images from arbitrary camera rigs by implicitly unprojecting to 3d. In: European Conference on Computer Vision. pp. 194-210. Springer (2020) - source
    • Rests on an assumption: During the deployment phase of autonomous driving systems, camera extrinsics may be biased due to calibration errors, camera offsets, and similar reasons.
  • Findingown data
    § Abstract · p. 1

    BEVFormer outperforms previous best method DETR3D by 9.0 points NDS on the nuScenes test set.

    “which is 9.0 points higher than previous best arts”
    • Also stated in: § 1 Introduction · p. 3 · plain
      “outperforming previous best detection method DETR3D [47] by 9.0 points (56.9% vs. 47.9%)”
    • Also stated in: § 4.3 3D Object Detection Results · p. 7 · plain
      “9.0 points higher than DETR3D (47.9% NDS)”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [47] [47] Wang, Y., Guizilini, V.C., Zhang, T., Wang, Y., Zhao, H., Solomon, J.: Detr3d: 3d object detection from multi-view images via 3d-to-2d queries. In: Conference on Robot Learning. pp. 180-191. PMLR (2022) - source

      Section 4.1 (p. 6) reports DETR3D’s nuScenes test-set NDS as 0.479, and Section 4.2 (p. 6) says it outperformed existing methods as of October 13, 2021. The paper does not report BEVFormer’s score or a 9.0-point NDS difference, so it supports the DETR3D benchmark but not the claimed margin. (LLM, conf=0.84)

      Experimental — accuracy not yet measured
  • Findingown data
    § Abstract · p. 1

    BEVFormer's performance on the nuScenes test set is on par with LiDAR-based baselines.

    “and on par with the performance of LiDAR-based baselines”
    • Also stated in: § 4.3 3D Object Detection Results · p. 7 · plain
      “Our method can even achieve comparable performance to some LiDAR-based baselines such as SSN (56.9% NDS) [55] and PointPainting (58.1% NDS) [43].”
    Quote in the paperSubstantial claim1 of 2 sources checked
    • cites [55] [55] Zhu, X., Ma, Y., Wang, T., Xu, Y., Shi, J., Lin, D.: Ssn: Shape signature networks for multi-class object detection from point clouds. In: European Conference on Computer Vision. pp. 581-597. Springer (2020) - source
    • cites [43] [43] Vora, S., Lang, A.H., Helou, B., Beijbom, O.: Pointpainting: Sequential fusion for 3d object detection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 4604-4612 (2020) - source

      Table 4 (nuScenes test results; page unavailable in our copy) reports performance for LiDAR-only baselines, including PointPillars+ and MEGVII. The paper does not report BEVFormer's performance or compare it with those baselines, so it cannot establish that BEVFormer is on par with them. (LLM, conf=0.86)

      Experimental — accuracy not yet measured
  • Premisecitation
    § 2.2 Camera-based 3D Perception · p. 3

    Previous works consider temporal information by stacking BEV features from several timestamps.

    “In addition to the spatial information, previous works [18, 38, 6] also consider the temporal information by stacking BEV features from several timestamps.”
    Quote in the paperSubstantial claim2 of 3 sources checked
    • cites [18] [18] Hu, A., Murez, Z., Mohan, N., Dudas, S., Hawke, J., Badrinarayanan, V., Cipolla, R., Kendall, A.: Fiery: Future instance prediction in bird's-eye view from surround monocular cameras. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 15273-15282 (2021) - source
    • cites [38] [38] Saha, A., Maldonado, O.M., Russell, C., Bowden, R.: Translating images into maps. arXiv preprint arXiv:2110.00966 (2021) - source
    • cites [6] [6] Can, Y.B., Liniger, A., Unal, O., Paudel, D., Van Gool, L.: Understanding bird's-eye view semantic hd-maps using an onboard monocular camera. arXiv preprint arXiv:2012.03040 (2020) - source

      The Temporal Aggregation section describes transforming features from multiple frames into BEV grids. But the Aggregation subsection says those grids are combined across frames using max or mean aggregation; the feature maps subsequently concatenated are not simply BEV features stacked from several timestamps. (LLM, conf=0.85)

      Experimental — accuracy not yet measured
  • May combine two claims. 21
  • Classified as probably not a substantial claim. 8
  • The quote only partly states this claim. 1

  • May combine two claims.

    Premisecitation
    § 1 Introduction · p. 2

    Monocular frameworks process different views separately and cannot capture information across cameras, leading to low performance and efficiency.

    “The downside of this framework is that it processes different views separately and cannot capture information across cameras, leading to low performance and efficiency [32, 47].”
    Quote in the paperSubstantial claim2 of 2 sources checked
    • cites [32] [32] Philion, J., Fidler, S.: Lift, splat, shoot: Encoding images from arbitrary camera rigs by implicitly unprojecting to 3d. In: European Conference on Computer Vision. pp. 194-210. Springer (2020) - source

      The Introduction (p. 2) describes a simple multi-view approach that applies a single-image detector to each camera separately and says it cannot learn the best way to fuse information across cameras. The baseline discussion (Section 5.1, p. 9) reports weak performance for one independently encoded multi-view CNN. The paper does not establish that such frameworks cannot capture any cross-camera information or that they have low efficiency. (LLM, conf=0.85)

      Experimental — accuracy not yet measured
    • cites [47] [47] Wang, Y., Guizilini, V.C., Zhang, T., Wang, Y., Zhao, H., Solomon, J.: Detr3d: 3d object detection from multi-view images via 3d-to-2d queries. In: Conference on Robot Learning. pp. 180-191. PMLR (2022) - source

      Related Work (p. 3) says monocular methods process each camera frame independently and merge outputs afterward, supporting the point that they do not fuse information across cameras during detection. The comparison in overlap regions (p. 6) finds better detection performance with fused inference, but the paper does not establish that separate-view processing generally causes low performance or efficiency. (LLM, conf=0.85)

      Experimental — accuracy not yet measured

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 1 Introduction · p. 2

    Previous map segmentation methods demonstrate BEV's effectiveness.

    “Although previous map segmentation methods demonstrate BEV's effectiveness [32, 18, 29]”
    Quote in the paperNot a substantial claim2 of 3 sources checked
    • cites [32] [32] Philion, J., Fidler, S.: Lift, splat, shoot: Encoding images from arbitrary camera rigs by implicitly unprojecting to 3d. In: European Conference on Computer Vision. pp. 194-210. Springer (2020) - source
    • cites [18] [18] Hu, A., Murez, Z., Mohan, N., Dudas, S., Hawke, J., Badrinarayanan, V., Cipolla, R., Kendall, A.: Fiery: Future instance prediction in bird's-eye view from surround monocular cameras. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 15273-15282 (2021) - source
    • cites [29] [29] Ng, M.H., Radia, K., Chen, J., Wang, D., Gog, I., Gonzalez, J.E.: Bev-seg: Bird's eye view semantic segmentation using geometry and semantic point cloud. arXiv preprint arXiv:2006.11436 (2020) - source

  • May combine two claims.

    Findingown data
    § 4.4 Multi-tasks Perception Results · p. 8

    With joint training, BEVFormer outperforms Lift-Splat by 11.0 NDS points on detection (52.0 vs 41.0) and 5.6 IoU points on lane segmentation (23.9 vs 18.3).

    “For example, with joint training, BEVFormer outperforms Lift-Splat ∗ [32] by 11.0 points on detation task (52.0% NDS v.s. 41.0% NDS) and IoU of 5.6 points on lane segmentation (23.9% v.s. 18.3%).”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [32] [32] Philion, J., Fidler, S.: Lift, splat, shoot: Encoding images from arbitrary camera rigs by implicitly unprojecting to 3d. In: European Conference on Computer Vision. pp. 194-210. Springer (2020) - source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 2.2 Camera-based 3D Perception · p. 3

    Lift-Splat generates BEV features based on depth distribution.

    “Lift-Splat [32] generates the BEV features based on the depth distribution.”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [32] [32] Philion, J., Fidler, S.: Lift, splat, shoot: Encoding images from arbitrary camera rigs by implicitly unprojecting to 3d. In: European Conference on Computer Vision. pp. 194-210. Springer (2020) - source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 4.4 Multi-tasks Perception Results · p. 8

    The jointly trained model's worse performance on road and lane segmentation is a common phenomenon called negative transfer in multi-task learning.

    “However, the jointly trained model does not perform as well as individually trained models for road and lane segmentation, which is a common phenomenon called negative transfer [11, 13] in multi-task learning.”
    Quote in the paperNot a substantial claim1 of 2 sources checked
    • cites [11] [11] Crawshaw, M.: Multi-task learning with deep neural networks: A survey. arXiv preprint arXiv:2009.09796 (2020) - source
    • cites [13] [13] Fifty, C., Amid, E., Zhao, Z., Yu, T., Anil, R., Finn, C.: Efficiently identifying task groupings for multi-task learning. Advances in Neural Information Processing Systems 34 (2021) - source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 2.1 Transformer-based 2D perception · p. 3

    DETR has long training time.

    “However, the main drawback of DETR is the long training time.”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [7] [7] Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., Zagoruyko, S.: End-to-end object detection with transformers. In: European conference on computer vision. pp. 213-229. Springer (2020) - source

      The Introduction (p. 2) says DETR requires an extra-long training schedule, and the Experiments section (p. 8) reports three days for a 300-epoch baseline run. This supports long training time, but the paper does not identify it as DETR’s main drawback. (LLM, conf=0.82)

      Experimental — accuracy not yet measured

  • May combine two claims.

    Premisecitation
    § 1 Introduction · p. 2

    Detection performance of BEV-based methods is subject to compounding errors, and inaccurate BEV features can seriously hurt final performance.

    “The detection performance of BEV-based methods is thus subject to compounding errors [47], and inaccurate BEV features can seriously hurt the final performance.”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [47] [47] Wang, Y., Guizilini, V.C., Zhang, T., Wang, Y., Zhao, H., Solomon, J.: Detr3d: 3d object detection from multi-view images via 3d-to-2d queries. In: Conference on Robot Learning. pp. 180-191. PMLR (2022) - source

      The Introduction (p. 1) says that pipelines reconstructing pseudo-LiDAR or range data from images are subject to compounding errors because inaccurate depth estimates harm 3D detection. Section 4.4 (p. 7) makes the same point about pseudo-LiDAR detectors. The paper does not extend that finding to BEV-based methods generally or identify inaccurate BEV features as the cause. (LLM, conf=0.86)

      Experimental — accuracy not yet measured

  • May combine two claims.

    Premisecitation
    § 2.2 Camera-based 3D Perception · p. 3

    PYVA's cross-view transformer converts front-view monocular image into BEV, but is not suitable for fusing multi-camera features due to computational cost of global attention.

    “PYVA [51] proposes a cross-view transformer that converts the front-view monocular image into the BEV, but this paradigm is not suitable for fusing multi-camera features due to the computational cost of global attention mechinism [42].”
    Quote in the paperSubstantial claim1 of 2 sources checked
    • cites [51] [51] Yang, W., Li, Q., Liu, W., Yu, Y., Ma, Y., He, S., Pan, J.: Projecting your view attentively: Monocular road scene layout estimation via cross-view transformation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 15536-15545 (2021) - source
    • cites [42] [42] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) - source

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation
    § 1 Introduction · p. 2

    BEV is a commonly used representation of the surrounding scene since it clearly presents the location and scale of objects and is suitable for various autonomous driving tasks.

    “The bird's-eye-view (BEV) is a commonly used representation of the surrounding scene since it clearly presents the location and scale of objects and is suitable for various autonomous driving tasks, such as perception and planning [29].”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [29] [29] Ng, M.H., Radia, K., Chen, J., Wang, D., Gog, I., Gonzalez, J.E.: Bev-seg: Bird's eye view semantic segmentation using geometry and semantic point cloud. arXiv preprint arXiv:2006.11436 (2020) - source

      The Abstract (p. 1) calls BEV widely adopted and says it captures surrounding objects and their spatial locations. The Introduction (p. 1) describes its use in perception and its suitability for planning and prediction. Our copy does not clearly establish the separate claim that BEV presents the scale of objects. (LLM, conf=0.83)

      Experimental — accuracy not yet measured

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation
    § 2.1 Transformer-based 2D perception · p. 3

    Deformable attention interacts with local regions of interest, samples K points near each reference point, resulting in high efficiency and shortening training time.

    “Different from vanilla global attention in DETR, the deformable attention interacts with local regions of interest, which only samples K points near each reference point and calculates attention results, resulting in high efficiency and significantly shortening the training time.”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [56] [56] Zhu, X., Su, W., Lu, L., Li, B., Wang, X., Dai, J.: Deformable detr: Deformable transformers for end-to-end object detection. In: International Conference on Learning Representations (2020) - source

  • May combine two claims.

    Premisecitation
    § 1 Introduction · p. 2

    Temporal information plays a crucial role in inferring the motion state of objects and identifying occluded objects, and many works have demonstrated the effectiveness of using video data.

    “For the human visual perception system, temporal information plays a crucial role in inferring the motion state of objects and identifying occluded objects, and many works in vision fields have demonstrated the effectiveness of using video data [2, 27, 26, 33, 19].”
    Quote in the paperSubstantial claim1 of 5 sources checked
    • cites [2] [2] Brazil, G., Pons-Moll, G., Liu, X., Schiele, B.: Kinematic 3d object detection in monocular video. In: European Conference on Computer Vision. pp. 135-152. Springer (2020) - source
    • cites [27] [27] Ma, X., Ouyang, W., Simonelli, A., Ricci, E.: 3d object detection from images for autonomous driving: A survey. arXiv preprint arXiv:2202.02980 (2022) - source
    • cites [26] [26] Luo, W., Yang, B., Urtasun, R.: Fast and furious: Real time end-to-end 3d detection, tracking and motion forecasting with a single convolutional net. In: Proceedings of the IEEE conference on Computer Vision and Pattern Recognition. pp. 3569-3577 (2018) - source
    • cites [33] [33] Qi, C.R., Zhou, Y., Najibi, M., Sun, P., V o, K., Deng, B., Anguelov, D.: Offboard 3d object detection from point cloud sequences. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 6134-6144 (2021) - source

      Section 4.4 and the motion-state estimator section show that object tracks make motion-state classification easier than using a few frames. The qualitative analysis reports recognition of some objects despite occlusion, but also missed severely occluded objects. The paper studies temporal LiDAR point clouds, not human visual perception or the effectiveness of video data generally; page numbers are unavailable in our copy. (LLM, conf=0.85)

      Experimental — accuracy not yet measured
    • cites [19] [19] Kang, K., Ouyang, W., Li, H., Wang, X.: Object detection from video tubelets with convolutional neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 817-825 (2016) - source

  • May combine two claims.

    Findingown data
    § 4.3 3D Object Detection Results · p. 7

    BEVFormer achieves 51.7% NDS on the nuScenes val set, outperforming DETR3D by 9.2 points.

    “Our method outperforms previous best method DETR3D [47] over 9.2 points on val set (51.7% NDS vs. 42.5% NDS)”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [47] [47] Wang, Y., Guizilini, V.C., Zhang, T., Wang, Y., Zhao, H., Solomon, J.: Detr3d: 3d object detection from multi-view images via 3d-to-2d queries. In: Conference on Robot Learning. pp. 180-191. PMLR (2022) - source

  • May combine two claims.

    Findingown data
    § 4.4 Multi-tasks Perception Results · p. 8

    Model training with multi-task heads performs even better on detection tasks and vehicles segmentation.

    “In this paper, we show that the BEV features generated by our BEV encoder can be well adapted to different tasks, and the model training with multi-task heads performs even better on detection tasks and vehicles segmentation.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 4.5 Ablation Study · p. 11

    Configuration D, with single-scale view features, smaller BEV size, and only 1 encoder layer, consumes only 7ms during inference, losing 3.9 points compared to the default configuration.

    “Configuration D, with single-scale view features, smaller BEV size, and only 1 encoder layer, consumes only 7ms during inference, although it loses 3.9 points compared to the default configuration.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data
    § B Robustness on Camera Extrinsics · p. 17

    With noise level 4, the NDS of BEVFormer-S drops 15.2%, while the NDS of BEVFormer-S (point) drops 17.3%.

    “For example, with the noise level being 4, the NDS of BEVFormer-S drops 15.2% (calculated by 1 -0 . 380 0 . 448 ), while the NDS of BEVFormer-S (point) drops 17.3%.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: During the deployment phase of autonomous driving systems, camera extrinsics may be biased due to calibration errors, camera offsets, and similar reasons.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data
    § D Visualization · p. 18

    The detection results and segmentation results are highly consistent.

    “We also show both object detection and map segmentation results in Fig. 8, where we see that the detection results and segmentation results are highly consistent.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data
    § 4.4 Multi-tasks Perception Results · p. 8

    Under joint training, BEVFormer achieves higher performance than other BEV encoders on all tasks except road segmentation, where it is comparable with BEVFormer-S.

    “While comparing different BEV encoders under same settings, BEVFormer achieves higher performances of all tasks except for road segmentation results is comparable with BEVFormer-S.”
    Quote in the paperSubstantial claim

  • May combine two claims. The quote only partly states this claim.

    Findingown data
    § 4.3 3D Object Detection Results · p. 8

    BEVFormer outperforms DETR3D on Waymo by 3.2% NDS and 5.2% AP on nuScenes metrics.

    “On nuScenes metrics, BEVFormer outperforms DETR3D with a margin of 3.2% NDS and 5.2% AP.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § C Ablation Studies · p. 18

    Using both BEV queries and history BEV features to predict offsets and weights during temporal self-attention contains more clues about past BEV features and benefits location prediction (51.3% NDS vs. 51.7% NDS).

    “Compared to only using the BEV query to predict offsets and weights during the temporal self-attention module (see #3), using both BEV queries and history BEV features (see #4) contain more clues about the past BEV features and benefits location prediction (51.3% NDS vs. 51.7% NDS).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data
    § 4.5 Ablation Study · p. 11

    Configuration C using one encoder layer in BEVFormer achieves 50.1% NDS and reduces latency from 130ms to 25ms.

    “We can observe that configuration C using one encoder layer in BEVFormer achieves 50.1 % NDS and reduces the latency of BEVFormer from the original 130ms to 25ms.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data
    § C Ablation Studies · p. 17

    The NDS on nuScenes val set keeps rising with the growth of the frame number and begins to level off when the frame number is at least 4.

    “We see that the NDS on nuScenes val set keeps rising with the growth of the frame number and begins to level off the frame number ≥ 4 .”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning
    § 3.4 Temporal Self-Attention · p. 6

    BEVFormer extracts temporal information from previous BEV features rather than multiple stacking BEV features, requiring less computational cost and suffering less disturbing information.

    “BEVFormer extracts temporal information from the previous BEV features rather than multiple stacking BEV features, thus requiring less computational cost and suffering less disturbing information.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: Simply stacking BEV features of cross timestamps brings extra computational cost and interference information, which might not be ideal.
    • Rests on an assumption: Stacking BEV features constraints the available temporal information within fixed time duration and brings extra computational cost.

  • May combine two claims.

    Interpretationreasoning
    § 4.4 Multi-tasks Perception Results · p. 8

    Compared with training tasks individually, multi-task learning saves computational cost and reduces inference time by sharing more modules, including the backbone and the BEV encoder.

    “Compared with training tasks individually, multi-task learning saves computational cost and reduces the inference time by sharing more modules, including the backbone and the BEV encoder.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data
    § 4.5 Ablation Study · p. 10

    BEVFormer obtains higher recall on heavily occluded objects because temporal information contains past object clues.

    “We obtain higher recall on heavily occluded objects since the temporal information contains past objects clues, as showed in Fig. 3.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data
    § 4.5 Ablation Study · p. 11

    BEVFormer architecture can adapt to various model scales and be flexible to trade off performance and efficiency.

    “Overall, our architecture can adapt to various model scales and be flexible to trade off performance and efficiency.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning
    § A.4 Spatial Cross-Attention · p. 16

    One BEV query usually interacts with only one or two views rather than all views, making it possible to use global attention in spatial cross-attention.

    “This strategy makes that one BEV query usually interacts with only one or two views rather than all views, making it possible to use global attention in the spatial cross-attention.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning
    § 4.5 Ablation Study · p. 11

    Due to multi-view image inputs, the bottleneck limiting efficiency lies in the backbone.

    “However, due to the multi-view image inputs, the bottleneck that limits the efficiency lies in the backbone, and efficient backbones for autonomous driving deserve in-depth study.”
    Quote in the paperNot a substantial claim
27 of 27

  • The paper relies on this without support.

    Generating BEV from 2D planes is ill-posed.

    “but generating BEV from the 2D planes is ill-posed.”

  • The paper relies on this without support.

    Cameras own desirable advantages to detect long-range distance objects and identify vision-based road elements compared to LiDAR-based counterparts.

    “Apart from the low cost for deployment, cameras own the desirable advantages to detect long-range distance objects and identify vision-based road elements ( e.g. , traffic lights, stoplines), compared to LiDAR-based counterparts.”

  • The paper relies on this without support.

    BEV is a desirable bridge to connect temporal and spatial space.

    “Another motivation for using BEV features to perform perception tasks is that BEV is a desirable bridge to connect temporal and spatial space.”

  • The paper relies on this without support.

    Existing state-of-the-art multi-camera 3D detection methods rarely exploit temporal information.

    “However, the existing state-of-the-art multi-camera 3D detection methods rarely exploit temporal information.”

  • The paper relies on this without support.

    Simply stacking BEV features of cross timestamps brings extra computational cost and interference information, which might not be ideal.

    “and thus simply stacking BEV features of cross timestamps brings extra computational cost and interference information, which might not be ideal.”

  • The paper relies on this without support.

    It is challenging to construct the precise association of the same objects between the BEV features of different times.

    “It is challenging to construct the precise association of the same objects between the BEV features of different times.”

  • The paper relies on this without support.

    The 3D object detection task requires strong BEV features to support accurate 3D bounding box prediction.

    “The underlying reason is that the 3D object detection task requires strong BEV features to support accurate 3D bounding box prediction”

  • The paper relies on this without support.

    Autonomous driving is time-critical and objects in the scene change rapidly.

    “The significant challenges are that autonomous driving is time-critical and objects in the scene change rapidly”

  • The paper relies on this without support.

    Stacking BEV features constraints the available temporal information within fixed time duration and brings extra computational cost.

    “Stacking BEV features constraints the available temporal information within fixed time duration and brings extra computational cost.”

  • The paper relies on this without support.

    During the deployment phase of autonomous driving systems, camera extrinsics may be biased due to calibration errors, camera offsets, and similar reasons.

    “During the deployment phase of autonomous driving systems, extrinsics may be biased due to various reasons such as calibration errors, camera offsets, etc.”

  • The paper relies on this without support.

    Accurate inference of 3D location from 2D information remains a long-standing challenge for camera-based methods.

    “Accurate inference of 3D location from 2D information remains a long-stand challenge for camera-based methods.”

  • The paper relies on this without support.

    Camera-based methods still have a particular gap with LiDAR-based methods in effect and efficiency.

    “At present, the camera-based methods still have a particular gap with the LiDAR-based methods in effect and efficiency.”
12 of 12

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Waymo NDS footnote contradicts the reported ATE values

    Minor

    The footnote says ATE is set to 1 when computing the Waymo NDS scores, but the table reports ATE values other than 1. The listed NDS scores are consistent with using those reported ATE values and setting the unreported AVE and AAE values to 1 instead.

1 of 1

Cites

Flagged first, then by citations

Research works printed but not linked: 1

  1. [25][25] Loshchilov, I., Hutter, F.: Decoupled weight decay regularization. In: ICLR (2019) -

Cited by

Sorted by citations

Related

Authors
Li, Z., Wang, W., Li, H., Xie, E., Sima, C., Lu, T., Yu, Q., & Dai, J.
Affiliations
  • Shanghai Artificial Intelligence Laboratory (CN)
  • University of Hong Kong (HK)
Corresponding author
Dai, Jifeng
ORCID
Venue
arXiv
Publisher
arXiv
Published
2022
Pages
1-18
ISSN
0302-9743 (ISSN-L)
DOI
10.48550/arxiv.2203.17270
arXiv
2203.17270
OpenAlex
W4312894406
Cited by
1,419
References
55
Citations by year
2026: 225 · 2025: 468 · 2024: 502 · 2023: 212 · 2022: 12
FWCI
108.26
Type
conference paper
Language
en
Open access
Licence
creativecommons.org/licenses/by/4.0/legalcode
Locations
  • Lecture notes in computer science (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • The HKU Scholars Hub (University of Hong Kong) (submittedVersion)
  • arXiv (Cornell University) (cc-by)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
  • Physical Sciences › Engineering › Automotive Engineering › Autonomous Vehicle Technology and Safety
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Vision and Imaging
Keywords
bird's-eye-view representation, multi-camera images, spatio-temporal transformer, autonomous driving perception, 3D object detection, map segmentation, cross-attention, temporal self-attention, nuScenes

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-10.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-10.

Cited sources are checked by gpt-6-sol.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review