The DOI matches its Crossref record; 64 of the 64 references it cites were found in a registry or an index.
Deep Learning-Based Vehicle Behavior Prediction for Autonomous Driving Applications: A Review
Open accessAbstract
Behaviour prediction function of an autonomous vehicle predicts the future states of the nearby vehicles based on the current and past observations of the surrounding environment. This helps enhance their awareness of the imminent hazards. However, conventional behavior prediction solutions are applicable in simple driving scenarios that require short prediction horizons. Most recently, deep learning-based approaches have become popular due to their promising performance in more complex environments compared to the conventional approaches. Motivated by this increased popularity, we provide a comprehensive review of the state-of-the-art of deep learning-based approaches for vehicle behavior prediction in this article. We firstly give an overview of the generic problem of vehicle behavior prediction and discuss its challenges, followed by classification and review of the most recent deep learning-based solutions based on three criteria: input representation, output type, and prediction method. The article also discusses the performance of several well-known solutions, identifies the research gaps in the literature and outlines potential new research directions.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404
MobileNetV2: Inverted Residuals and Linear Bottlenecks
Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., & Chen, L.-C. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00474 · Cited by 28,755
Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545
Semi-supervised classification with graph convolutional networks
Kipf, T. N., & Welling, M. (2016) · arXiv · doi:10.48550/arxiv.1609.02907 · Cited by 24,853
Veličković, P., Cucurull, G., Casanova, A., Romero, A., Liò, P., & Bengio, Y. (2017) · arXiv · doi:10.48550/arxiv.1710.10903 · Cited by 12,107
The graph neural network model
Scarselli, F., Gori, M., Tsoi, A. C., Hagenbuchner, M., & Monfardini, G. (2009) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2008.2005605 · Cited by 9,582
Veličković, P., Casanova, A., Liò, P., Cucurull, G., Romero, A., & Bengio, Y. (2017) · arXiv (Cornell University) · doi:10.17863/cam.48429 · Cited by 8,657
Social force model for pedestrian dynamics
Helbing, D., & Molnár, P. (1995) · Physical Review E · doi:10.1103/physreve.51.4282 · Cited by 6,845
WaveNet: a generative model for raw audio
Oord, A. V. D., Dieleman, S., Zen, H., Simonyan, K., Vinyals, O., Graves, A., Kalchbrenner, N., Senior, A., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1609.03499 · Cited by 4,911
Learning Deconvolution Network for Semantic Segmentation
Noh, H., Hong, S., & Han, B. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.178 · Cited by 4,422
Social LSTM: Human Trajectory Prediction in Crowded Spaces
Alahi, A., Goel, K., Ramanathan, V., Robicquet, A., Fei-Fei, L., & Savarese, S. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.110 · Cited by 3,483
Graves, A. (2012) · Studies in Computational Intelligence · doi:10.1007/978-3-642-24797-2_4 · Cited by 2,727
Learning Hierarchical Features for Scene Labeling
Farabet, C., Couprie, C., Najman, L., & LeCun, Y. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.231 · Cited by 2,620
Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networks
Gupta, A., Johnson, J., Fei-Fei, L., Savarese, S., & Alahi, A. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00240 · Cited by 2,466
An Overview of Multi-Task Learning in Deep Neural Networks
Ruder, S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.05098 · Cited by 2,265
A new model for learning in graph domains
Gori, M., Monfardini, G., & Scarselli, F. (2006) · Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005. · doi:10.1109/ijcnn.2005.1555942 · Cited by 2,075
Kiros, R., Zhu, Y., Salakhutdinov, R., Zemel, R. S., Torralba, A., Urtasun, R., & Fidler, S. (2015) · arXiv · doi:10.48550/arxiv.1506.06726 · Cited by 1,293 · cited 1×
“f the TV as a regression task. Despite having less parameters and complexity, single layer LSTMs are reported to achieve competitive results compared to the multilayer counterpart in some tasks [56], [57]. To predict an intention-based trajectory, Ding and Shen [30] use an LSTM encoder to predict the intention of the…”
MethodA survey on motion prediction and risk assessment for intelligent vehicles
Lefèvre, S., Vasquez, D., & Laugier, C. (2014) · ROBOMECH Journal · doi:10.1186/s40648-014-0001-z · Cited by 1,112
DESIRE: Distant Future Prediction in Dynamic Scenes with Interacting Agents
Lee, N., Choi, W., Vernaza, P., Choy, C. B., Torr, P. H. S., & Chandraker, M. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.233 · Cited by 1,067
Convolutional Social Pooling for Vehicle Trajectory Prediction
Deo, N., & Trivedi, M. M. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2018.00196 · Cited by 1,050
On Human Motion Prediction Using Recurrent Neural Networks
Martinez, J., Black, M. J., & Romero, J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.497 · Cited by 984
A Survey on 3D Object Detection Methods for Autonomous Driving Applications
Arnold, E., Al-Jarrah, O. Y., Dianati, M., Fallah, S., Oxtoby, D., & Mouzakitis, A. (2019) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2019.2892405 · Cited by 713
Self-citationMultimodal Trajectory Predictions for Autonomous Driving using Deep Convolutional Networks
Cui, H., Radosavljevic, V., Chou, F.-C., Lin, T.-H., Nguyen, T., Huang, T.-K., Schneider, J., & Djuric, N. (2019) · 2019 International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2019.8793868 · Cited by 676
Social Attention: Modeling Attention in Human Crowds
Vemula, A., Muelling, K., & Oh, J. (2018) · 2018 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2018.8460504 · Cited by 675
An LSTM Network for Highway Trajectory PredictionPreprint
Altch\'e, F., & Fortelle, A. D. L. (2018) · arXiv (Cornell University) · Cited by 663
Human motion trajectory prediction: a survey
Rudenko, A., Palmieri, L., Herman, M., Kitani, K. M., Gavrila, D. M., & Arras, K. O. (2020) · The International Journal of Robotics Research · doi:10.1177/0278364920917446 · Cited by 658
Luo, W., Yang, B., & Urtasun, R. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00376 · Cited by 644
Multi-Modal Trajectory Prediction of Surrounding Vehicles with Maneuver based LSTMs
Deo, N., & Trivedi, M. M. (2018) · IEEE Intelligent Vehicles Symposium · doi:10.1109/ivs.2018.8500493 · Cited by 549
Sequence-to-Sequence Prediction of Vehicle Trajectory via LSTM Encoder-Decoder Architecture
Park, S. H., Kim, B., Kang, C. M., Chung, C. C., & Choi, J. W. (2018) · 2018 IEEE Intelligent Vehicles Symposium (IV) · doi:10.1109/ivs.2018.8500658 · Cited by 504
Multi-Agent Tensor Fusion for Contextual Trajectory Prediction
Zhao, T., Xu, Y., Monfort, M., Choi, W., Baker, C., Zhao, Y., Wang, Y., & Wu, Y. N. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01240 · Cited by 477
TrafficPredict: Trajectory Prediction for Heterogeneous Traffic-Agents
Ma, Y., Zhu, X., Zhang, S., Yang, R., Wang, W., & Manocha, D. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33016120 · Cited by 446
Fernando, T., Denman, S., Sridharan, S., & Fookes, C. (2018) · Neural Networks · doi:10.1016/j.neunet.2018.09.002 · Cited by 371
Convolutional Sequence to Sequence Model for Human Dynamics
Li, C., Zhang, Z., Lee, W. S., & Lee, G. H. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00548 · Cited by 364
Modeling Vehicle Interactions via Modified LSTM Models for Trajectory Prediction
Dai, S., Li, L., & Li, Z. (2019) · IEEE Access · doi:10.1109/access.2019.2907000 · Cited by 301
Driver Behavior Classification at Intersections and Validation on Large Naturalistic Data Set
Aoude, G. S., Desaraju, V. R., Stephens, L. H., & How, J. P. (2012) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2011.2179537 · Cited by 245
Uncertainty-aware Short-term Motion Prediction of Traffic Actors for Autonomous Driving
Djuric, N., Radosavljevic, V., Cui, H., Nguyen, T., Chou, F.-C., Lin, T.-H., Singh, N., & Schneider, J. (2020) · 2020 IEEE Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv45572.2020.9093332 · Cited by 225
Naturalistic Driver Intention and Path Prediction Using Recurrent Neural Networks
Zyner, A., Worrall, S., & Nebot, E. (2020) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2019.2913166 · Cited by 219
IntentNet: Learning to Predict Intention from Raw Sensor Data
Casas, S., Luo, W., & Urtasun, R. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.07907 · Cited by 218
Neural Machine Translation and Sequence-to-sequence Models: A TutorialPreprint
Neubig, G. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.01619 · Cited by 213
r2p2: A ReparameteRized Pushforward Policy for Diverse, Precise Generative Path Forecasting
Rhinehart, N., Kitani, K. M., & Vernaza, P. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01261-8_47 · Cited by 211
Video-Based Human Behavior Understanding: A Survey
Borges, P. V. K., Conci, N., & Cavallaro, A. (2013) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2013.2270402 · Cited by 208
Graph Neural Networks for Modelling Traffic Participant Interaction
Diehl, F., Brunner, T., Le, M. T., & Knoll, A. (2019) · IEEE Intelligent Vehicles Symposium · doi:10.1109/ivs.2019.8814066 · Cited by 202
Gindele, T., Brechtel, S., & Dillmann, R. (2010) · 13th International IEEE Conference on Intelligent Transportation Systems · doi:10.1109/itsc.2010.5625262 · Cited by 200
Generalizable intention prediction of human drivers at intersections
Phillips, D. J., Wheeler, T. A., & Kochenderfer, M. J. (2017) · 2017 IEEE Intelligent Vehicles Symposium (IV) · doi:10.1109/ivs.2017.7995948 · Cited by 191
A Recurrent Neural Network Solution for Predicting Driver Intention at Unsignalized Intersections
Zyner, A., Worrall, S., & Nebot, E. (2018) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2018.2805314 · Cited by 191
GRIP++: Enhanced Graph-based Interaction-aware Trajectory Prediction for Autonomous DrivingPreprint
Li, X., Ying, X., & Chuah, C. (2019) · arXiv (Cornell University) · doi:10.13140/rg.2.2.16886.63044 · Cited by 190
Intention-aware Long Horizon Trajectory Prediction of Surrounding Vehicles using Dual LSTM Networks
Xin, L., Wang, P., Chan, C.-Y., Chen, J., Li, S. E., & Cheng, B. (2018) · 2018 21st International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc.2018.8569595 · Cited by 189
Probabilistic Prediction of Vehicle Semantic Intention and Motion
Hu, Y., Zhan, W., & Tomizuka, M. (2018) · IEEE Intelligent Vehicles Symposium · doi:10.1109/ivs.2018.8500419 · Cited by 188
Shirazi, M. S., & Morris, B. T. (2017) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2016.2568920 · Cited by 178
Hoermann, S., Bach, M., & Dietmayer, K. (2018) · 2018 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2018.8460874 · Cited by 166
Long short term memory for driver intent prediction
Zyner, A., Worrall, S., Ward, J., & Nebot, E. (2017) · IEEE Intelligent Vehicles Symposium · doi:10.1109/ivs.2017.7995919 · Cited by 164
A random finite set approach for dynamic occupancy grid maps with real-time application
Nuss, D., Reuter, S., Thom, M., Yuan, T., Krehl, G., Maile, M., Gern, A., & Dietmayer, K. (2018) · The International Journal of Robotics Research · doi:10.1177/0278364918775523 · Cited by 160
On the approximation capability of recurrent neural networks
Hammer, B. (2000) · Neurocomputing · doi:10.1016/s0925-2312(99)00174-5 · Cited by 132
Convolution neural network-based lane change intention prediction of surrounding vehicles for ACC
Lee, D., Kwon, Y. P., McMains, S., & Hedrick, J. K. (2017) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc.2017.8317874 · Cited by 131
Long-Term Occupancy Grid Prediction Using Recurrent Neural Networks
Schreiber, M., Hoermann, S., & Dietmayer, K. (2019) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2019.8793582 · Cited by 91
Ding, W., & Shen, S. (2019) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2019.8793568 · Cited by 75
Motion Prediction of Traffic Actors for Autonomous Driving using Deep Convolutional Networks.Preprint
Djuric, N., Radosavljević, V., Cui, H., Nguyen, T., Chou, F., Lin, T.-H., & Schneider, J. (2018) · arXiv (Cornell University) · Cited by 49
Zhan, W., Fortelle, A. D. L., Chen, Y., Chan, C., & Tomizuka, M. (2018) · IEEE Intelligent Vehicles Symposium · doi:10.1109/ivs.2018.8500697 · Cited by 46
Survey on Vision-Based Path Prediction
Hirakawa, T., Yamashita, T., Tamaki, T., & Fujiyoshi, H. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-91131-1_4 · Cited by 33
How good is my prediction? Finding a similarity measure for trajectory prediction evaluation
Quehl, J., Hu, H., Taş, Ö. Ş., Rehder, E., & Lauer, M. (2017) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc.2017.8317825 · Cited by 23
Anomaly Detection in Road Traffic Using Visual Surveillance: A SurveyPreprint
Kumaran, S. K., Dogra, D. P., & Roy, P. P. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.08292 · Cited by 17
Chen, T., Wang, Z., Li, L., Li, D., Li, Z., Chang, X., Bian, P., Liang, G., Wang, Q., & Xie, T. (2025) · Schloss Dagstuhl – Leibniz-Zentrum für Informatik · doi:10.4230/lipics.ecoop.2025.6
Cited by
Sensor and Sensor Fusion Technology in Autonomous Vehicles: A Review
Yeong, D. J., Velasco-Hernandez, G., Barry, J., & Walsh, J. (2021) · Sensors · doi:10.3390/s21062140 · Cited by 1,032
A Survey on Trajectory-Prediction Methods for Autonomous Driving
Huang, Y., Du, J., Yang, Z., Zhou, Z., Zhang, L., & Chen, H. (2022) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2022.3167103 · Cited by 677
Milestones in Autonomous Driving and Intelligent Vehicles: Survey of Surveys
Chen, L., Li, Y., Huang, C., Li, B., Xing, Y., Tian, D., Li, L., Hu, Z., Na, X., Li, Z., et al. (2023) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2022.3223131 · Cited by 511
Deep Learning Methods for Vessel Trajectory Prediction Based on Recurrent Neural Networks
Capobianco, S., Millefiori, L. M., Forti, N., Braca, P., & Willett, P. (2021) · IEEE Transactions on Aerospace and Electronic Systems · doi:10.1109/taes.2021.3096873 · Cited by 307
Multi-Agent Trajectory Prediction With Heterogeneous Edge-Enhanced Graph Attention Network
Mo, X., Huang, Z., Xing, Y., & Lv, C. (2022) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2022.3146300 · Cited by 302
Metadata
- Authors
- Mozaffari, S., Al-Jarrah, O. Y., Dianati, M., Jennings, P., & Mouzakitis, A.
- Venue
- IEEE Transactions on Intelligent Transportation Systems
- Published
- 2022-01
- Volume
- 23
- Issue
- 1
- Pages
- 33-47
- ISSN
- 1524-9050 (ISSN-L)
- 1558-0016
- OpenAlex
- W2997958396
- Cited by
- 671
- References
- 64
- FWCI
- 31.02
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.