The DOI matches its Crossref record; 37 of the 37 references it cites were found in a registry or an index.
Multimodal Motion Prediction with Stacked Transformers
Open accessAbstract
Predicting multiple plausible future trajectories of the nearby vehicles is crucial for the safety of autonomous driving. Recent motion prediction approaches attempt to achieve such multimodal motion prediction by implicitly regularizing the feature or explicitly generating multiple candidate proposals. However, it remains challenging since the latent features may concentrate on the most frequent mode of the data while the proposal-based methods depend largely on the prior knowledge to generate and select the proposals. In this work, we propose a novel transformer framework for multimodal motion prediction, termed as mmTransformer. A novel network architecture based on stacked transformers is designed to model the multimodality at feature level with a set of fixed independent proposals. A region-based training strategy is then developed to induce the multimodality of the generated proposals. Experiments on Argoverse dataset show that the proposed model achieves the state-of-the-art performance on motion prediction, substantially improving the diversity and the accuracy of the predicted trajectories. Demo video and code are available at https://decisionforce.github.io/mmTransformer.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Exploiting Generative AI to Scale up Intelligent Tutoring Systems
Jakubův, J., Chvalovský, K., Goertzel, Z., Kaliszyk, C., Olšák, M., Piotrowski, B., Schulz, S., Suda, M., & Urban, J. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,658
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
End-to-End Object Detection with Transformers
Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., & Zagoruyko, S. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58452-8_13 · Cited by 18,285
End-to-End Object Detection with Transformers
Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., & Zagoruyko, S. (2020) · arXiv · doi:10.48550/arxiv.2005.12872 · Cited by 18,253
Decoupled Weight Decay Regularization
Loshchilov, I., & Hutter, F. (2017) · arXiv · doi:10.48550/arxiv.1711.05101 · Cited by 12,522
Social LSTM: Human Trajectory Prediction in Crowded Spaces
Alahi, A., Goel, K., Ramanathan, V., Robicquet, A., Fei-Fei, L., & Savarese, S. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.110 · Cited by 3,483
Multi-task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics
Cipolla, R., Gal, Y., & Kendall, A. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00781 · Cited by 3,151
Constrained K-means Clustering with Background Knowledge
Wagstaff, K. L., Cardie, C., Rogers, S., & Schrödl, S. (2001) · Cited by 2,469
Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networks
Gupta, A., Johnson, J., Fei-Fei, L., Savarese, S., & Alahi, A. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00240 · Cited by 2,466
Soft-NMS — Improving Object Detection with One Line of Code
Bodla, N., Singh, B., Chellappa, R., & Davis, L. S. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.593 · Cited by 2,274 · cited 1×
“Besides, the accruacy drop caused by post-processing is a common issue in machine learning community [3], especially when selecting final predictions from a large candidate set.”
BackgroundArgoverse: 3D Tracking and Forecasting With Rich Maps
Chang, M.-F., Lambert, J., Sangkloy, P., Singh, J., Bak, S., Hartnett, A., Wang, D., Carr, P., Lucey, S., Ramanan, D., et al. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00895 · Cited by 1,474
DESIRE: Distant Future Prediction in Dynamic Scenes with Interacting Agents
Lee, N., Choi, W., Vernaza, P., Choy, C. B., Torr, P. H. S., & Chandraker, M. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.233 · Cited by 1,067
SoPhie: An Attentive GAN for Predicting Paths Compliant to Social and Physical Constraints
Sadeghian, A., Kosaraju, V., Sadeghian, A., Hirose, N., Rezatofighi, H., & Savarese, S. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00144 · Cited by 1,065
Convolutional Social Pooling for Vehicle Trajectory Prediction
Deo, N., & Trivedi, M. M. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2018.00196 · Cited by 1,050
VectorNet: Encoding HD Maps and Agent Dynamics From Vectorized Representation
Gao, J., Sun, C., Zhao, H., Shen, Y., Anguelov, D., Li, C., & Schmid, C. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01154 · Cited by 982
MultiPath: Multiple Probabilistic Anchor Trajectory Hypotheses for Behavior PredictionPreprint
Chai, Y., Sapp, B., Bansal, M., & Anguelov, D. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.05449 · Cited by 868
Learning Lane Graph Representations for Motion Forecasting
Liang, M., Yang, B., Hu, R., Chen, Y., Liao, R., Feng, S., & Urtasun, R. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58536-5_32 · Cited by 700
An LSTM Network for Highway Trajectory PredictionPreprint
Altch\'e, F., & Fortelle, A. D. L. (2018) · arXiv (Cornell University) · Cited by 663
Transformer Networks for Trajectory Forecasting
Giuliari, F., Hasan, I., Cristani, M., & Galasso, F. (2021) · 2020 25th International Conference on Pattern Recognition (ICPR) · doi:10.1109/icpr48806.2021.9412190 · Cited by 572
CoverNet: Multimodal Behavior Prediction Using Trajectory Sets
Phan-Minh, T., Grigore, E. C., Boulton, F. A., Beijbom, O., & Wolff, E. M. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01408 · Cited by 438
Multiple Futures PredictionPreprint
Tang, Y., & Salakhutdinov, R. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.00997 · Cited by 306
Rules of the Road: Predicting Driving Behavior With a Convolutional Model of Semantic Interactions
Hong, J., Sapp, B., & Philbin, J. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00865 · Cited by 277
RWTH ASR Systems for LibriSpeech: Hybrid vs Attention
Lüscher, C., Beck, E., Irie, K., Kitza, M., Michel, W., Zeyer, A., Schlüter, R., & Ney, H. (2019) · Interspeech 2019 · doi:10.21437/interspeech.2019-1780 · Cited by 241
TPNet: Trajectory Proposal Network for Motion Prediction
Fang, L., Jiang, Q., Shi, J., & Zhou, B. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00683 · Cited by 213
Self-citationTNT: Target-driveN Trajectory Prediction
Zhao, H., Gao, J., Lan, T., Sun, C., Sapp, B., Varadarajan, B., Shen, Y., Shen, Y., Chai, Y., Schmid, C., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.08294 · Cited by 204
Multi-Head Attention for Multi-Modal Joint Vehicle Motion Forecasting
Mercat, J., Gilles, T., El Zoghby, N., Sandou, G., Beauvois, D., & Gil, G. P. (2020) · 2020 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra40945.2020.9197340 · Cited by 201
Future Person Localization in First-Person Videos
Yagi, T., Mangalam, K., Yonetani, R., & Sato, Y. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00792 · Cited by 175
Auxiliary Tasks in Multi-task Learning
Liebel, L., & Körner, M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.06334 · Cited by 169
End-to-end Contextual Perception and Prediction with Interaction Transformer
Li, L. L., Yang, B., Liang, M., Zeng, W., Ren, M., Segal, S., & Urtasun, R. (2020) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros45743.2020.9341392 · Cited by 164
What-If Motion Prediction for Autonomous DrivingPreprint
Khandelwal, S., Qi, W., Singh, J., Hartnett, A. T., & Ramanan, D. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.10587 · Cited by 149
Neural Turtle Graphics for Modeling City Road Layouts
Chu, H., Li, D., Acuna, D., Kar, A., Shugrina, M., Wei, X., Liu, M., Torralba, A. B., & Fidler, S. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00462 · Cited by 108
Lane-Attention: Predicting Vehicles’ Moving Trajectories by Learning Their Attention Over Lanes
Pan, J., Sun, H., Xu, K., Jiang, Y., Xiao, X., Hu, J., & Miao, J. (2020) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros45743.2020.9341233 · Cited by 55
Motion Prediction of Traffic Actors for Autonomous Driving using Deep Convolutional Networks.Preprint
Djuric, N., Radosavljević, V., Cui, H., Nguyen, T., Chou, F., Lin, T.-H., & Schneider, J. (2018) · arXiv (Cornell University) · Cited by 49
A Novel Learning Framework for Sampling-Based Motion Planning in Autonomous Driving
Zhang, Y., Zhang, J., Zhang, J., Wang, J., Lu, K., & Hong, L. J. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i01.5473 · Cited by 30
Self-citation
Cited by
Multimodal learning with transformers: A survey
Xu, P., Zhu, X., & Clifton, D. A. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3275156 · Cited by 1,072
Planning-oriented Autonomous Driving
Hu, Y., Yang, J., Chen, L., Li, K., Sima, C., Zhu, X., Chai, S., Du, S., Lin, T., Wang, W., et al. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01712 · Cited by 931
A Survey on Trajectory-Prediction Methods for Autonomous Driving
Huang, Y., Du, J., Yang, Z., Zhou, Z., Zhang, L., & Chen, H. (2022) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2022.3167103 · Cited by 677
HiVT: Hierarchical Vector Transformer for Multi-Agent Motion Prediction
Zhou, Z., Ye, L., Wang, J., Wu, K., & Lu, K. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00862 · Cited by 460
VAD: Vectorized Scene Representation for Efficient Autonomous Driving
Jiang, B., Chen, S., Xu, Q., Liao, B., Chen, J., Zhou, H., Zhang, Q., Liu, W., Huang, C., & Wang, X. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00766 · Cited by 332
Metadata
- Authors
- Liu, Y., Zhang, J., Fang, L., Jiang, Q., & Zhou, B.
- Venue
- 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2021-06
- Pages
- 7573-7582
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W3169575318
- Cited by
- 474
- References
- 37
- FWCI
- 79.64
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.