The DOI matches its Crossref record; 7 of the 7 references it cites were found in a registry or an index.
Social LSTM: Human Trajectory Prediction in Crowded Spaces
Open accessAbstract
Pedestrians follow different trajectories to avoid obstacles and accommodate fellow pedestrians. Any autonomous vehicle navigating such a scene should be able to foresee the future positions of pedestrians and accordingly adjust its path to avoid collisions. This problem of trajectory prediction can be viewed as a sequence generation task, where we are interested in predicting the future trajectory of people based on their past positions. Following the recent success of Recurrent Neural Network (RNN) models for sequence prediction tasks, we propose an LSTM model which can learn general human movement and predict their future trajectories. This is in contrast to traditional approaches which use hand-crafted functions such as Social forces. We demonstrate the performance of our method on several public datasets. Our model outperforms state-of-the-art methods on some of these datasets. We also analyze the trajectories predicted by our model to demonstrate the motion behaviour learned by our model.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
Chung, J., Gulcehre, C., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1412.3555 · Cited by 11,008
Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
Xu, K., Ba, J., Kiros, R., Cho, K., Courville, A., Salakhutdinov, R., Zemel, R., & Bengio, Y. (2015) · arXiv · doi:10.48550/arxiv.1502.03044 · Cited by 8,248
Show and tell: A neural image caption generator
Vinyals, O., Toshev, A., Bengio, S., & Erhan, D. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298935 · Cited by 6,318
Conditional Random Fields as Recurrent Neural Networks
Zheng, S., Jayasumana, S., Romera-Paredes, B., Vineet, V., Su, Z., Du, D., Huang, C., & Torr, P. H. S. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.179 · Cited by 2,386
Unsupervised Learning of Video Representations using LSTMs
Srivastava, N., Mansimov, E., & Salakhutdinov, R. (2015) · arXiv · doi:10.48550/arxiv.1502.04681 · Cited by 1,861
You'll never walk alone: Modeling social behavior for multi-target tracking
Pellegrini, S., Ess, A., Schindler, K., & van Gool, L. (2009) · 2009 IEEE 12th International Conference on Computer Vision · doi:10.1109/iccv.2009.5459260 · Cited by 1,683 · cited 7×
“Finally, we demonstrate that our Social-LSTM is capable of predicting trajectories of pedestrians much more accurately than state-of-the-art methods on two publicly available datasets: ETH [49], and UCY [39].”
BackgroundMethodResultInfluential
Cited by
Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networks
Gupta, A., Johnson, J., Fei-Fei, L., Savarese, S., & Alahi, A. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00240 · Cited by 2,466
Self-citationArgoverse: 3D Tracking and Forecasting With Rich Maps
Chang, M.-F., Lambert, J., Sangkloy, P., Singh, J., Bak, S., Hartnett, A., Wang, D., Carr, P., Lucey, S., Ramanan, D., et al. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00895 · Cited by 1,474
Spatio-Temporal LSTM with Trust Gates for 3D Human Action Recognition
Liu, J., Shahroudy, A., Xu, D., & Wang, G. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46487-9_50 · Cited by 1,351
Dense-Captioning Events in Videos
Krishna, R., Hata, K., Ren, F., Fei-Fei, L., & Niebles, J. C. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.83 · Cited by 1,251
Self-citationTrajectron++: Dynamically-Feasible Trajectory Forecasting with Heterogeneous Data
Salzmann, T., Ivanovic, B., Chakravarty, P., & Pavone, M. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58523-5_40 · Cited by 1,076
DESIRE: Distant Future Prediction in Dynamic Scenes with Interacting Agents
Lee, N., Choi, W., Vernaza, P., Choy, C. B., Torr, P. H. S., & Chandraker, M. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.233 · Cited by 1,067
SoPhie: An Attentive GAN for Predicting Paths Compliant to Social and Physical Constraints
Sadeghian, A., Kosaraju, V., Sadeghian, A., Hirose, N., Rezatofighi, H., & Savarese, S. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00144 · Cited by 1,065
Self-citationConvolutional Social Pooling for Vehicle Trajectory Prediction
Deo, N., & Trivedi, M. M. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2018.00196 · Cited by 1,050
TrackFormer: Multi-Object Tracking with Transformers
Meinhardt, T., Kirillov, A., Leal-Taixe, L., & Feichtenhofer, C. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00864 · Cited by 1,081
“Motion can be modelled for trajectory prediction [1, 25, 42] using a constant velocity assumption (CVA) [2, 9] or the social force model [25, 34, 43, 58].”
MethodMohamed, A., Qian, K., Elhoseiny, M., & Claudel, C. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01443 · Cited by 989
VectorNet: Encoding HD Maps and Agent Dynamics From Vectorized Representation
Gao, J., Sun, C., Zhao, H., Shen, Y., Anguelov, D., Li, C., & Schmid, C. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01154 · Cited by 982
Tracking Without Bells and Whistles
Bergmann, P., Meinhardt, T., & Leal-Taixe, L. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00103 · Cited by 1,110
Human Action Recognition and Prediction: A Survey
Kong, Y., & Fu, Y. (2022) · International Journal of Computer Vision · doi:10.1007/s11263-022-01594-9 · Cited by 780
From Recognition to Cognition: Visual Commonsense Reasoning
Zellers, R., Bisk, Y., Farhadi, A., & Choi, Y. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00688 · Cited by 769
A Survey on Trajectory-Prediction Methods for Autonomous Driving
Huang, Y., Du, J., Yang, Z., Zhou, Z., Zhang, L., & Chen, H. (2022) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2022.3167103 · Cited by 677
Multimodal Trajectory Predictions for Autonomous Driving using Deep Convolutional Networks
Cui, H., Radosavljevic, V., Chou, F.-C., Lin, T.-H., Nguyen, T., Huang, T.-K., Schneider, J., & Djuric, N. (2019) · 2019 International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2019.8793868 · Cited by 676
Social Attention: Modeling Attention in Human Crowds
Vemula, A., Muelling, K., & Oh, J. (2018) · 2018 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2018.8460504 · Cited by 675
STGAT: Modeling Spatial-Temporal Interactions for Human Trajectory Prediction
Huang, Y., Bi, H., Li, Z., Mao, T., & Wang, Z. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00637 · Cited by 658
Human motion trajectory prediction: a survey
Rudenko, A., Palmieri, L., Herman, M., Kitani, K. M., Gavrila, D. M., & Arras, K. O. (2020) · The International Journal of Robotics Research · doi:10.1177/0278364920917446 · Cited by 658
Luo, W., Yang, B., & Urtasun, R. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00376 · Cited by 644
AgentFormer: Agent-Aware Transformers for Socio-Temporal Multi-Agent Forecasting
Yuan, Y., Weng, X., Ou, Y., & Kitani, K. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00967 · Cited by 631
Deep Learning with Long Short-Term Memory for Time Series Prediction
Hua, Y., Zhao, Z., Li, R., Chen, X., Liu, Z., & Zhang, H. (2019) · IEEE Communications Magazine · doi:10.1109/mcom.2019.1800155 · Cited by 629
Chen, C., Liu, Y., Kreiss, S., & Alahi, A. (2019) · 2019 International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2019.8794134 · Cited by 615
Self-citationMotion Planning Among Dynamic, Decision-Making Agents with Deep Reinforcement Learning
Everett, M., Chen, Y. F., & How, J. P. (2018) · 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2018.8593871 · Cited by 614
Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Dataset
Ettinger, S., Cheng, S., Caine, B., Liu, C., Zhao, H., Pradhan, S., Chai, Y., Sapp, B., Qi, C., Zhou, Y., et al. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00957 · Cited by 584
Transformer Networks for Trajectory Forecasting
Giuliari, F., Hasan, I., Cristani, M., & Galasso, F. (2021) · 2020 25th International Conference on Pattern Recognition (ICPR) · doi:10.1109/icpr48806.2021.9412190 · Cited by 572
Spatio-Temporal Graph Transformer Networks for Pedestrian Trajectory Prediction
Yu, C., Ma, X., Ren, J., Zhao, H., & Yi, S. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58610-2_30 · Cited by 565
Tracking the Untrackable: Learning to Track Multiple Cues with Long-Term Dependencies
Sadeghian, A., Alahi, A., & Savarese, S. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.41 · Cited by 553
Self-citationSkeleton-Based Action Recognition Using Spatio-Temporal LSTM Network with Trust Gates
Liu, J., Shahroudy, A., Xu, D., Kot, A. C., & Wang, G. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2771306 · Cited by 530
It Is Not the Journey But the Destination: Endpoint Conditioned Trajectory Prediction
Mangalam, K., Girase, H., Agarwal, S., Lee, K.-H., Adeli, E., Malik, J., & Gaidon, A. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58536-5_45 · Cited by 512
Multi-Agent Tensor Fusion for Contextual Trajectory Prediction
Zhao, T., Xu, Y., Monfort, M., Choi, W., Baker, C., Zhao, Y., Wang, Y., & Wu, Y. N. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01240 · Cited by 477
Multimodal Motion Prediction with Stacked Transformers
Liu, Y., Zhang, J., Fang, L., Jiang, Q., & Zhou, B. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00749 · Cited by 474
CoverNet: Multimodal Behavior Prediction Using Trajectory Sets
Phan-Minh, T., Grigore, E. C., Boulton, F. A., Beijbom, O., & Wolff, E. M. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01408 · Cited by 438
SS-LSTM: A Hierarchical LSTM Model for Pedestrian Trajectory Prediction
Xue, H., Huynh, D. Q., & Reynolds, M. (2018) · 2018 IEEE Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv.2018.00135 · Cited by 433
Attention Based Vehicle Trajectory Prediction
Messaoud, K., Yahiaoui, I., Verroust-Blondet, A., & Nashashibi, F. (2021) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2020.2991952 · Cited by 425
Deep learning for unmanned aerial vehicle-based object detection and tracking: a survey
Wu, X., Li, W., Hong, D., Tao, R., & Du, Q. (2022) · IEEE Geoscience and Remote Sensing Magazine · doi:10.1109/mgrs.2021.3115137 · Cited by 421
Deep Learning for Intelligent Transportation Systems: A Survey of Emerging Trends
Veres, M., & Moussa, M. (2020) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2019.2929020 · Cited by 394
Cui, Z., Ke, R., Pu, Z., & Wang, Y. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1801.02143 · Cited by 377
Fernando, T., Denman, S., Sridharan, S., & Fookes, C. (2018) · Neural Networks · doi:10.1016/j.neunet.2018.09.002 · Cited by 371
Panoptic Studio: A Massively Multiview System for Social Interaction Capture
Joo, H., Simon, T., Li, X., Liu, H., Tan, L., Gui, L., Banerjee, S., Godisart, T., Nabbe, B., Matthews, I., et al. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2782743 · Cited by 344
Learning Regularity in Skeleton Trajectories for Anomaly Detection in Videos
Morais, R., Le, V., Tran, T., Saha, B., Mansour, M., & Venkatesh, S. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01227 · Cited by 341
Fan, T., Long, P., Liu, W., & Pan, J. (2020) · The International Journal of Robotics Research · doi:10.1177/0278364920916531 · Cited by 332
Social Ways: Learning Multi-Modal Distributions of Pedestrian Trajectories With GANs
Amirian, J., Hayet, J.-B., & Pettre, J. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2019.00359 · Cited by 323
MultiPath++: Efficient Information Fusion and Trajectory Aggregation for Behavior Prediction
Varadarajan, B., Hefny, A., Srivastava, A., Refaat, K. S., Nayakanti, N., Cornman, A., Chen, K., Douillard, B., Lam, C. P., Anguelov, D., et al. (2022) · 2022 International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra46639.2022.9812107 · Cited by 313
Stochastic Trajectory Prediction via Motion Indeterminacy Diffusion
Gu, T., Chen, G., Li, J., Lin, C., Rao, Y., Zhou, J., & Lu, J. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01660 · Cited by 313
Query-Centric Trajectory Prediction
Zhou, Z., Wang, J., Li, Y., & Huang, Y. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01713 · Cited by 333
One Thousand and One Hours: Self-driving Motion Prediction Dataset
Houston, J., Zuidhof, G., Bergamini, L., Ye, Y., Chen, L., Jain, A., Omari, S., Iglovikov, V., & Ondruska, P. (2020) · arXiv · doi:10.48550/arxiv.2006.14480 · Cited by 292
From Goals, Waypoints & Paths To Long Term Human Trajectory Forecasting
Mangalam, K., An, Y., Girase, H., & Malik, J. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01495 · Cited by 281
FIERY: Future Instance Prediction in Bird’s-Eye View from Surround Monocular Cameras
Hu, A., Murez, Z., Mohan, N., Dudas, S., Hawke, J., Badrinarayanan, V., Cipolla, R., & Kendall, A. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01499 · Cited by 277
Human Trajectory Forecasting in Crowds: A Deep Learning Perspective
Kothari, P., Kreiss, S., & Alahi, A. (2022) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2021.3069362 · Cited by 267
Self-citationLearning to Track with Object Permanence
Tokmakov, P., Li, J., Burgard, W., & Gaidon, A. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01068 · Cited by 226
Intent Prediction of Pedestrians via Motion Trajectories Using Stacked Recurrent Neural Networks
Saleh, K., Hossny, M., & Nahavandi, S. (2018) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2018.2873901 · Cited by 103
Trace and Pace: Controllable Pedestrian Animation via Guided Trajectory Diffusion
Rempe, D., Luo, Z., Peng, X. B., Yuan, Y., Kitani, K., Kreis, K., Fidler, S., & Litany, O. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01322 · Cited by 97
Scene Transformer: A unified architecture for predicting multiple agent trajectories
Ngiam, J., Caine, B., Vasudevan, V., Zhang, Z., Chiang, H.-T. L., Ling, J., Roelofs, R., Bewley, A., Liu, C., Venugopal, A., et al. (2021) · arXiv · doi:10.48550/arxiv.2106.08417 · Cited by 84
Deep Fisher discriminant learning for mobile hand gesture recognition
Li, C., Xie, C., Zhang, B., Chen, C., & Han, J. (2018) · Pattern Recognition · doi:10.1016/j.patcog.2017.12.023 · Cited by 72
MultiXNet: Multiclass Multistage Multimodal Motion Prediction
Djuric, N., Cui, H., Su, Z., Wu, S., Wang, H., Chou, F.-C., Martin, L. S., Feng, S., Hu, R., Xu, Y., et al. (2021) · 2021 IEEE Intelligent Vehicles Symposium (IV) · doi:10.1109/iv48863.2021.9575718 · Cited by 53
Shifts: A Dataset of Real Distributional Shift Across Multiple Large-Scale Tasks
Malinin, A., Band, N., Ganshin, A., Chesnokov, G., Gal, Y., Gales, M. J. F., Noskov, A., Ploskonosov, A., Prokhorenkova, L., Provilkov, I., et al. (2021) · arXiv · doi:10.48550/arxiv.2107.07455 · Cited by 48
ATPFL: Automatic Trajectory Prediction Model Design under Federated Learning Framework
Wang, C., Chen, X., Wang, J., & Wang, H. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00645 · Cited by 45
Spatially-Aware Graph Neural Networks for Relational Behavior Forecasting from Sensor Data
Casas, S., Gulino, C., Liao, R., & Urtasun, R. (2019) · arXiv · doi:10.48550/arxiv.1910.08233 · Cited by 44
AGENT: A Benchmark for Core Psychological Reasoning
Shu, T., Bhandwaldar, A., Gan, C., Smith, K. A., Liu, S., Gutfreund, D., Spelke, E., Tenenbaum, J. B., & Ullman, T. D. (2021) · arXiv · doi:10.48550/arxiv.2102.12321 · Cited by 34
Directed Acyclic Graph Neural Network for Human Motion Prediction
Li, Q., Chalvatzaki, G., Peters, J., & Wang, Y. (2021) · 2021 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra48506.2021.9561540 · Cited by 29
Conformal Decision Theory: Safe Autonomous Decisions from Imperfect Predictions
Lekeufack, J., Angelopoulos, A. N., Bajcsy, A., Jordan, M. I., & Malik, J. (2024) · 2024 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra57147.2024.10610041 · Cited by 18
CityTracker: Citywide Individual and Crowd Trajectory Analysis Using Hidden Markov Model
Fang, S.-H., Lin, L., Yang, Y.-T., Yu, X., & Xu, Z. (2019) · IEEE Sensors Journal · doi:10.1109/jsen.2019.2916693 · Cited by 13
Crowd Density Forecasting by Modeling Patch-Based Dynamics
Minoura, H., Yonetani, R., Nishimura, M., & Ushiku, Y. (2021) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2020.3043169 · Cited by 16
Metadata
- Authors
- Alahi, A., Goel, K., Ramanathan, V., Robicquet, A., Fei-Fei, L., & Savarese, S.
- Venue
- 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2016-06
- Pages
- 961-971
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W2424778531
- Cited by
- 3,483
- References
- 7
- FWCI
- 222.89
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.