FIERY: Future Instance Prediction in Bird’s-Eye View from Surround Monocular Cameras
Abstract
Driving requires interacting with road agents and predicting their future behaviour in order to navigate safely. We present FIERY: a probabilistic future prediction model in bird’s-eye view from monocular cameras. Our model predicts future instance segmentation and motion of dynamic agents that can be transformed into non-parametric future trajectories. Our approach combines the perception, sensor fusion and prediction components of a traditional autonomous driving stack by estimating bird’s-eye-view prediction directly from surround RGB monocular camera inputs. FIERY learns to model the inherent stochastic nature of the future solely from camera driving data in an end-to-end manner, without relying on HD maps, and predicts multimodal future trajectories. We show that our model outperforms previous prediction baselines on the NuScenes and Lyft datasets. The code and trained models are available at https://github.com/wayveai/fiery.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
BEVFormer's temporal self-attention can more effectively model long temporal dependency than simply stacking BEV features.
“Compared to simply stacking BEV in [18, 38, 6], our temporal self-attention can more effectively model long temporal dependency.”
From BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformers (2022) · cites this paper as [18]
Previous map segmentation methods demonstrate BEV's effectiveness.
“Although previous map segmentation methods demonstrate BEV's effectiveness [32, 18, 29]”
From BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformers (2022) · cites this paper as [18]
Previous works consider temporal information by stacking BEV features from several timestamps.
“In addition to the spatial information, previous works [18, 38, 6] also consider the temporal information by stacking BEV features from several timestamps.”
From BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformers (2022) · cites this paper as [18]
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · see citing works
EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks
Tan, M., & Le, Q. V. (2019) · arXiv · doi:10.48550/arxiv.1905.11946 · Cited by 17,072 · see citing works
The Hungarian method for the assignment problem
Kuhn, H. W. (1955) · Naval Research Logistics Quarterly · doi:10.1002/nav.3800020109 · Cited by 13,120 · see citing works
nuScenes: A Multimodal Dataset for Autonomous Driving
Caesar, H., Bankiti, V., Lang, A. H., Vora, S., Liong, V. E., Xu, Q., Krishnan, A., Pan, Y., Baldan, G., & Beijbom, O. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01164 · Cited by 7,152 · see citing works
Wang, Y., Chao, W.-L., Garg, D., Hariharan, B., Campbell, M., & Weinberger, K. Q. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00864 · Cited by 1,154 · see citing works
MultiXNet: Multiclass Multistage Multimodal Motion Prediction
Djuric, N., Cui, H., Su, Z., Wu, S., Wang, H., Chou, F.-C., Martin, L. S., Feng, S., Hu, R., Xu, Y., et al. (2021) · 2021 IEEE Intelligent Vehicles Symposium (IV) · doi:10.1109/iv48863.2021.9575718 · Cited by 53 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
CRN: Camera Radar Net for Accurate, Robust, Efficient 3D Perception
Kim, Y., Shin, J., Kim, S., Lee, I.-J., Choi, J. W., & Kum, D. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01615 · Cited by 155 · see citing works
Li, Z., Wang, W., Li, H., Xie, E., Sima, C., Lu, T., Yu, Q., & Dai, J. (2022) · arXiv · doi:10.48550/arxiv.2203.17270 · Cited by 1,419 · see citing works
“Compared to simply stacking BEV in [18, 38, 6], our temporal self-attention can more effectively model long temporal dependency.”
BackgroundPan, C., He, Y., Peng, J., Zhang, Q., Sui, W., & Zhang, Z. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.00925 · Cited by 33 · see citing works
GAIA-1: A Generative World Model for Autonomous Driving
Hu, A., Russell, L., Yeo, H., Murez, Z., Fedoseev, G., Kendall, A., Shotton, J., & Corrado, G. (2023) · arXiv · doi:10.48550/arxiv.2309.17080 · Cited by 28 · see citing works
“To achieve this, the development of a robust model of the world is imperative [1] and huge efforts have been made in the past to build such predictive world models for autonomous driving [2, 3, 4, 5, 6].”
BackgroundSelf-citation
- Authors
- Hu, A., Murez, Z., Mohan, N., Dudas, S., Hawke, J., Badrinarayanan, V., Cipolla, R., & Kendall, A.
- Affiliations
- University of Cambridge (GB)
- Wayve,UK
- Venue
- 2021 IEEE/CVF International Conference on Computer Vision (ICCV)
- Publisher
- IEEE
- Published
- 2021-10
- Pages
- 15253-15262
- OpenAlex
- W4214530037
- Cited by
- 277
- References
- 6
- Citations by year
- 2026: 39 · 2025: 69 · 2024: 93 · 2023: 60 · 2022: 14 · 2021: 2
- FWCI
- 41.83
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE/CVF International Conference on Computer Vision (ICCV) (publishedVersion)
- Topics
- Physical Sciences › Engineering › Automotive Engineering › Autonomous Vehicle Technology and Safety
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Vision and Imaging
- Keywords
- bird's-eye view, autonomous driving, probabilistic forecasting, sensor fusion, end-to-end learning, multimodal trajectory prediction, road agents
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
