Intent Prediction of Pedestrians via Motion Trajectories Using Stacked Recurrent Neural Networks
Open accessThe DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.
Abstract
The problem of intent understanding between highly and fully automated vehicles and vulnerable road users (VRUs) such as pedestrians in urban traffic environment has got some momentum over the past few years. Previous work has been tackling the problem using two common approaches, namely dynamical motion modeling and motion planning. In this paper, a novel radical end-to-end data-driven approach is proposed for long-term intent prediction of VRUs in urban traffic environment based solely on their motion trajectories. In the proposed approach, we utilized the widely adopted architecture of recurrent neural networks with Long-Short Term Memory (LSTM) modules to form a deep stacked LSTM network. Three common approaches used in the literature were compared against our proposed approach over two different real-world datasets involving pedestrians collected from vehicle-based stereo cameras. The results over the testing datasets showed that the proposed approach achieved higher accuracies over most of the scenarios of the testing datasets with a small mean lateral position error of 0.48 m. Moreover, the proposed approach showed also a significant generalization capability over totally unobserved testing scenes during the training phase with only 0.58 m in mean lateral position error.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“| RNN | Process sequential data via recurrent structures | Simple temporal analysis, low latency | Basic motion recognition (e.g., arm |”
From Recent advances in intelligent wearable systems: from multiscale biomechanical features towards human motion intent prediction (2026) · cites this paper as [147]
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 11×
“They have also introduced two data-driven approaches for the pedestrian intent recognition task based on CNNs and RNNs respectively.”
MethodInfluentialAdam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · cited 1×
“We utilized the Adam optimizer [32], to minimize the”
MethodLeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472 · cited 1×
“have witnessed a huge success in a number of different spatial and sequential tasks [12], [17]–[19].”
BackgroundFaster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1506.01497 · Cited by 70,812 · cited 1×
“have witnessed a huge success in a number of different spatial and sequential tasks [12], [17]–[19].”
BackgroundRich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.81 · Cited by 32,684
Sequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766 · cited 2×
“Recurrent Neural Networks (RNN) have been commonly used in many sequence-based prediction tasks such as handwriting imitation [24], driver behavior modeling [25], human gait”
BackgroundMethodUnscented Filtering and Nonlinear Estimation
Julier, S. J., & Uhlmann, J. K. (2004) · Proceedings of the IEEE · doi:10.1109/jproc.2003.823141 · Cited by 6,588 · cited 7×
“In [6], a dynamical motion model based on Extended Kalman filter (EKF) [14] with different motion models was proposed for short time prediction (i.e. less than 2 seconds) of pedestrians’ motion trajectories.”
MethodInfluentialOn the difficulty of training recurrent neural networks
Pascanu, R., Mikolov, T., & Bengio, Y. (2012) · arXiv · doi:10.48550/arxiv.1211.5063 · Cited by 4,310 · cited 1×
“Since traditional RNNs have difficult times in giving an accurate prediction when it comes to memorizing past long sequences [28].”
BackgroundSocial LSTM: Human Trajectory Prediction in Crowded Spaces
Alahi, A., Goel, K., Ramanathan, V., Robicquet, A., Fei-Fei, L., & Savarese, S. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.110 · Cited by 3,483 · cited 4×
“Similar to [20] and [21], we will also follow the data-driven approach based on RNNs for the intent prediction task of pedestrians in urban traffic environments.”
BackgroundMethodInfluentialYou'll never walk alone: Modeling social behavior for multi-target tracking
Pellegrini, S., Ess, A., Schindler, K., & van Gool, L. (2009) · 2009 IEEE 12th International Conference on Computer Vision · doi:10.1109/iccv.2009.5459260 · Cited by 1,683 · cited 1×
“The advantages of this formulation is that a long-term prediction about the pedestrians’ intentions, which is essential for fully and highly automated vehicles, could be effectively inferred [22], [23].”
BackgroundRecurrent Network Models for Human Dynamics
Fragkiadaki, K., Levine, S., Felsen, P., & Malik, J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.494 · Cited by 860 · cited 1×
“analysis [26], human-human interactions [20].”
Kitani, K. M., Ziebart, B. D., Bagnell, J. A., & Hebert, M. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33765-9_15 · Cited by 701
Planning-based prediction for pedestrians
Ziebart, B. D., Ratliff, N., Gallagher, G., Mertz, C., Peterson, K., Bagnell, J. A., Hebert, M., Dey, A. K., & Srinivasa, S. (2009) · 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems · doi:10.1109/iros.2009.5354147 · Cited by 456
Will the Pedestrian Cross? A Study on Pedestrian Path Prediction
Keller, C. G., & Gavrila, D. M. (2014) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2013.2280766 · Cited by 338 · cited 1×
“[7] proposed pedestrian intent prediction and action classification for short-term intervals.”
BackgroundImproving Neural Networks with Dropout
Srivastava, N. (2013) · Cited by 327
Rothenbucher, D., Li, J., Sirkin, D., Mok, B., & Ju, W. (2016) · 2016 25th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN) · doi:10.1109/roman.2016.7745210 · Cited by 303 · cited 1×
“That being said, these vehicles are still however faced by a number of challenges when it comes to understanding the behaviors and the intentions of other road sharing users especially the vulnerable ones such as pedestrians [1]–[3].”
BackgroundContext-Based Pedestrian Path Prediction
Kooij, J. F. P., Schneider, N., Flohr, F., & Gavrila, D. M. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10599-4_40 · Cited by 286 · cited 7×
“Lateral position error with prediction horizon of 70 steps ahead (almost 4 secs) over all the two scenarios of the context testing dataset [8].”
BackgroundMethodInfluentialPedestrian Path Prediction with Recursive Bayesian Filters: A Comparative Study
Schneider, N., & Gavrila, D. M. (2013) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-40602-7_18 · Cited by 249
Pedestrians at the kerb – Recognising the action intentions of humans
Schmidt, S., & Färber, B. (2009) · Transportation Research Part F: Traffic Psychology and Behaviour · doi:10.1016/j.trf.2009.02.003 · Cited by 229 · cited 1×
“For example, the motion trajectories did by VRUs in urban traffic environments when intending to cross the street [4]–[6].”
BackgroundIntent-aware long-term prediction of pedestrian motion
Karasev, V. I., Ayvacı, A., Heisele, B., & Soatto, S. (2016) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2016.7487409 · Cited by 193 · cited 3×
“[11] presented another planningbased model to predict motion trajectories of pedestrians’ intentions in the urban traffic environment.”
BackgroundDriving behavior classification based on sensor data fusion using LSTM recurrent neural networks
Saleh, K., Hossny, M., & Nahavandi, S. (2017) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc.2017.8317835 · Cited by 176 · cited 1×
“Recurrent Neural Networks (RNN) have been commonly used in many sequence-based prediction tasks such as handwriting imitation [24], driver behavior modeling [25], human gait”
MethodSelf-citationWiering, M., & Schmidhuber, J. (1997) · Adaptive Behavior · doi:10.1177/105971239700600202 · Cited by 163 · cited 1×
“This feedback connection loop can efficiently capture the temporal dependency in their input time series sequences by maintaining an internal state, called “hidden unit” [27].”
BackgroundGoal-Directed Pedestrian Prediction
Rehder, E., & Kloeden, H. (2015) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2015.28 · Cited by 136 · cited 1×
“In [16], Rehder et al.”
A data-driven approach for pedestrian intention estimation
Völz, B., Behrendt, K., Mielenz, H., Gilitschenski, I., Siegwart, R., & Nieto, J. I. (2016) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc.2016.7795975 · Cited by 108 · cited 4×
“Similar to [20] and [21], we will also follow the data-driven approach based on RNNs for the intent prediction task of pedestrians in urban traffic environments.”
BackgroundMethodResultInfluentialWill the Pedestrian Cross? Probabilistic Path Prediction Based on Learned Motion Features
Keller, C. G., Hermes, C., & Gavrila, D. M. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-23123-0_39 · Cited by 70
Intent prediction of vulnerable road users from motion trajectories using stacked LSTM network
Saleh, K., Hossny, M., & Nahavandi, S. (2017) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc.2017.8317941 · Cited by 69 · cited 1×
“The main contribution of this work as an extension to our early results in [13] is threefold: A novel approach for inferring long-term intentions of pedestrians in urban traffic environments from their motion”
BackgroundSelf-citationUncertain reasoning and forecasting
Dagum, P., Galper, A. R., Horvitz, E., & Seiver, A. J. (1995) · International Journal of Forecasting · doi:10.1016/0169-2070(94)02009-e · Cited by 68 · cited 3×
“They utilized a Dynamic Bayesian Network (DBN) [15] for trajectory prediction of pedestrians’ intention to cross the street while walking on the curb.”
MethodInfluentialTowards trusted autonomous vehicles from vulnerable road users perspective
Saleh, K., Hossny, M., & Nahavandi, S. (2017) · IEEE International Systems Conference (SysCon) · doi:10.1109/syscon.2017.7934782 · Cited by 68 · cited 1×
“The advantages of this formulation is that a long-term prediction about the pedestrians’ intentions, which is essential for fully and highly automated vehicles, could be effectively inferred [22], [23].”
BackgroundSelf-citationA Philosophy for Developing Trust in Self-driving Cars
Wagner, M., & Koopman, P. J. (2015) · Lecture notes in mobility · doi:10.1007/978-3-319-19078-5_14 · Cited by 59
Saleh, K., Hossny, M., & Nahavandi, S. (2017) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/smc.2017.8123150 · Cited by 23 · cited 1×
“have witnessed a huge success in a number of different spatial and sequential tasks [12], [17]–[19].”
BackgroundSelf-citation
Research works printed but not linked: 1
- lee, 2007, Loss functions in time series forecasting, International Encyclopedia of the Social Sciences
Cited by
Industry 5.0—A Human-Centric Solution
Nahavandi, S. (2019) · Sustainability · doi:10.3390/su11164371 · Cited by 1,695
“Computer vision [28], combined with deep learning [29], reinforcement learning, and GPU-based computation [30], has shown great promise in replicating primitive vision and sensory capabilities.”
BackgroundSelf-citationHuman motion trajectory prediction: a survey
Rudenko, A., Palmieri, L., Herman, M., Kitani, K. M., Gavrila, D. M., & Arras, K. O. (2020) · The International Journal of Robotics Research · doi:10.1177/0278364920917446 · Cited by 658
Mourtzis, D., Angelopoulos, J., & Panopoulos, N. (2022) · Journal of Machine Engineering · doi:10.36897/jme/147160 · Cited by 174
“Combining computer vision [81] with DL [82], reinforcement learning [83], and GPU-based computation [84] has shown great promise in replicating primitive vision and sensory abilities.”
MethodBehavioral Intention Prediction in Driving Scenes: A Survey
Fang, J., Wang, F., Xue, J., & Chua, T. (2024) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2024.3374342 · Cited by 149
“For example, Saleh et al. [134], [211] predict the long-term intention of pedestrians by a stacked LSTM over the trajectory points.”
BackgroundGhorai, P., Eskandarian, A., Kim, Y., & Mehr, G. (2022) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2022.3160932 · Cited by 128
Metadata
- Authors
- Saleh, K., Hossny, M., & Nahavandi, S.
- Affiliations
- Deakin University (AU)
- Venue
- IEEE Transactions on Intelligent Vehicles
- Published
- 2018-12
- Volume
- 3
- Issue
- 4
- Pages
- 414-424
- ISSN
- 2379-8858 (ISSN-L)
- 2379-8904
- OpenAlex
- W2895763020
- Cited by
- 103
- References
- 32
- Citations by year
- 2026: 4 · 2025: 16 · 2024: 13 · 2023: 22 · 2022: 11 · 2021: 17 · 2020: 12 · 2019: 7 · 2018: 1
- FWCI
- 4.92
- Type
- journal article
- Language
- en
- Open access
- green
- figshare.com
- Locations
- IEEE Transactions on Intelligent Vehicles (publishedVersion)
- Deakin Research Online (Deakin University) (submittedVersion)
- Figshare (submittedVersion, cc-by)
- Funders
- Institute for Intelligent Systems Research and Innovation (IISRI)
- Topics
- Physical Sciences › Engineering › Automotive Engineering › Autonomous Vehicle Technology and Safety
- Physical Sciences › Engineering › Safety, Risk, Reliability and Quality › Traffic and Road Safety
- Social Sciences › Safety Research › Evacuation and Crowd Dynamics
- Keywords
- vulnerable road users, movement trajectories, stacked LSTM, long short-term memory, end-to-end learning, autonomous vehicles, urban traffic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.