The DOI matches its Crossref record; 35 of the 35 references it cites were found in a registry or an index.
PRECOG: PREdiction Conditioned on Goals in Visual Multi-Agent Settings
Open accessAbstract
For autonomous vehicles (AVs) to behave appropriately on roads populated by human-driven vehicles, they must be able to reason about the uncertain intentions and decisions of other drivers from rich perceptual information. Towards these capabilities, we present a probabilistic forecasting model of future interactions between a variable number of agents. We perform both standard forecasting and the novel task of conditional forecasting, which reasons about how all agents will likely respond to the goal of a controlled agent (here, the AV). We train models on real and simulated data to forecast vehicle trajectories given past positions and LIDAR. Our evaluation shows that our model is substantially more accurate in multi-agent driving scenarios compared to existing state-of-the-art. Beyond its general ability to perform conditional forecasting queries, we show that our model's predictions of all agents improve when conditioned on knowledge of the AV's goal, further illustrating its capability to model agent interactions.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Auto-Encoding Variational Bayes
Kingma, D. P., & Welling, M. (2013) · arXiv · doi:10.48550/arxiv.1312.6114 · Cited by 20,067
On Estimation of a Probability Density Function and Mode
Parzen, E. (1962) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177704472 · Cited by 10,585
nuScenes: A Multimodal Dataset for Autonomous Driving
Caesar, H., Bankiti, V., Lang, A. H., Vora, S., Liong, V. E., Xu, Q., Krishnan, A., Pan, Y., Baldan, G., & Beijbom, O. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01164 · Cited by 7,152
Remarks on Some Nonparametric Estimates of a Density Function
Rosenblatt, M. (1956) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177728190 · Cited by 4,547
CARLA: An open urban driving simulator
Dosovitskiy, A., Ros, G., Codevilla, F., Lopez, A., & Koltun, V. (2017) · arXiv · doi:10.48550/arxiv.1711.03938 · Cited by 3,718
Social LSTM: Human Trajectory Prediction in Crowded Spaces
Alahi, A., Goel, K., Ramanathan, V., Robicquet, A., Fei-Fei, L., & Savarese, S. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.110 · Cited by 3,483
Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networks
Gupta, A., Johnson, J., Fei-Fei, L., Savarese, S., & Alahi, A. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00240 · Cited by 2,466
Density estimation using Real NVP
Dinh, L., Sohl-Dickstein, J., & Bengio, S. (2016) · arXiv · Cited by 2,067
Multi-Agent Reinforcement Learning: Independent vs. Cooperative Agents
Tan, M. (1993) · Machine Learning Proceedings 1993 · doi:10.1016/b978-1-55860-307-3.50049-6 · Cited by 1,810
You'll never walk alone: Modeling social behavior for multi-target tracking
Pellegrini, S., Ess, A., Schindler, K., & van Gool, L. (2009) · 2009 IEEE 12th International Conference on Computer Vision · doi:10.1109/iccv.2009.5459260 · Cited by 1,683 · cited 1×
“In contrast to the commonly-used average displacement error (ADE) and final displacement error (FDE) metrics that computes the mean Euclidean error from a batch of samples to a single ground-truth sample [1, 6, 10, 14, 28], minMSD has the desirable property of not penalizing plausible samples that correspond to decisi…”
Bayesian Reasoning and Machine Learning
Barber, D. (2012) · Cambridge University Press eBooks · doi:10.1017/cbo9780511804779 · Cited by 1,631
ALVINN, an autonomous land vehicle in a neural network
Pomerleau, D. (2018) · doi:10.1184/r1/6603146 · Cited by 1,338
The dynamics of reinforcement learning in cooperative multiagent systems
Claus, C., & Boutilier, C. E. (1998) · Cited by 1,089
DESIRE: Distant Future Prediction in Dynamic Scenes with Interacting Agents
Lee, N., Choi, W., Vernaza, P., Choy, C. B., Torr, P. H. S., & Chandraker, M. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.233 · Cited by 1,067
FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Models
Grathwohl, W., Chen, R. T. Q., Bettencourt, J., Sutskever, I., & Duvenaud, D. (2018) · arXiv · doi:10.48550/arxiv.1810.01367 · Cited by 796
Sequence-to-Sequence Prediction of Vehicle Trajectory via LSTM Encoder-Decoder Architecture
Park, S. H., Kim, B., Kang, C. M., Chung, C. C., & Choi, J. W. (2018) · 2018 IEEE Intelligent Vehicles Symposium (IV) · doi:10.1109/ivs.2018.8500658 · Cited by 504
Multi-Agent Tensor Fusion for Contextual Trajectory Prediction
Zhao, T., Xu, Y., Monfort, M., Choi, W., Baker, C., Zhao, Y., Wang, Y., & Wu, Y. N. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01240 · Cited by 477
An algorithmic perspective on imitation learning
Osa, T., Pajarinen, J., Neumann, G., Bagnell, J. A., Abbeel, P., & Peters, J. (2018) · Foundations and Trends in Robotics · doi:10.1561/2300000053 · Cited by 405 · cited 4×
“In this sense, forecasting can be considered non-interactive Imitation Learning without execution.”
BackgroundInfluentialFernando, T., Denman, S., Sridharan, S., & Fookes, C. (2018) · Neural Networks · doi:10.1016/j.neunet.2018.09.002 · Cited by 371
An Algorithmic Perspective on Imitation Learning
Osa, T., Pajarinen, J., Neumann, G., Bagnell, J. A., Abbeel, P., & Peters, J. (2018) · now publishers, Inc. eBooks · doi:10.1561/9781680834116 · Cited by 368
Concrete Problems for Autonomous Vehicle Safety: Advantages of Bayesian Deep Learning
McAllister, R., Gal, Y., Kendall, A., van der Wilk, M., Shah, A., Cipolla, R., & Weller, A. (2017) · Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2017/661 · Cited by 229
Self-citationCoordinated Multi-Agent Imitation LearningPreprint
Le, H. M., Yue, Y., Carr, P., & Lucey, P. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.03121 · Cited by 220
Hierarchical Game-Theoretic Planning for Autonomous Vehicles
Fisac, J. F., Bronstein, E., Stefansson, E., Sadigh, D., Sastry, S. S., & Dragan, A. D. (2019) · 2019 International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2019.8794007 · Cited by 216
r2p2: A ReparameteRized Pushforward Policy for Diverse, Precise Generative Path Forecasting
Rhinehart, N., Kitani, K. M., & Vernaza, P. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01261-8_47 · Cited by 211
Self-citationContext-Aware Trajectory Prediction
Bartoli, F., Lisanti, G., Ballan, L., & Del Bimbo, A. (2018) · 2018 24th International Conference on Pattern Recognition (ICPR) · doi:10.1109/icpr.2018.8545447 · Cited by 180
Forecasting Interactive Dynamics of Pedestrians with Fictitious Play
Ma, W.-C., Huang, D.-A., Lee, N., & Kitani, K. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.493 · Cited by 178
Self-citationConcrete Problems for Autonomous Vehicle Safety: Advantages of Bayesian Deep Learning
McAllister, R., Gal, Y., Kendall, A., van der Wilk, M., Shah, A., Cipolla, R., & Weller, A. (2017) · Cambridge University Engineering Department Publications Database · doi:10.17863/cam.12760 · Cited by 161
Self-citationMultimodal Probabilistic Model-Based Planning for Human-Robot Interaction
Schmerling, E., Leung, K., Vollprecht, W., & Pavone, M. (2018) · 2018 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2018.8460766 · Cited by 160
Deep Imitative Models for Flexible Inference, Planning, and ControlPreprint
Rhinehart, N., McAllister, R., & Levine, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.06544 · Cited by 155
Self-citationDecentralized MDPs with sparse interactions
Melo, F. S., & Veloso, M. (2011) · Artificial Intelligence · doi:10.1016/j.artint.2011.05.001 · Cited by 112
Generating Multi-Agent Trajectories using Programmatic Weak SupervisionPreprint
Zhan, E., Zheng, S., Yue, Y., Sha, L., & Lucey, P. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.07612 · Cited by 74
Generative Modeling of Multimodal Multi-Human Behavior
Ivanovic, B., Schmerling, E., Leung, K., & Pavone, M. (2018) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2018.8594393 · Cited by 73
Felsen, P., Lucey, P., & Ganguly, S. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01252-6_45 · Cited by 71
Generative Hybrid Representations for Activity Forecasting With No-Regret Learning
Guan, J., Yuan, Y., Kitani, K., & Rhinehart, N. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00025 · Cited by 34
Self-citationGenerative Multi-Agent Behavioral CloningPreprint
Zhan, E., Zheng, S., Yue, Y., Sha, L., & Lucey, P. (2018) · arXiv (Cornell University) · Cited by 22
Cited by
Trajectron++: Dynamically-Feasible Trajectory Forecasting with Heterogeneous Data
Salzmann, T., Ivanovic, B., Chakravarty, P., & Pavone, M. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58523-5_40 · Cited by 1,076
VectorNet: Encoding HD Maps and Agent Dynamics From Vectorized Representation
Gao, J., Sun, C., Zhao, H., Shen, Y., Anguelov, D., Li, C., & Schmid, C. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01154 · Cited by 982
Human motion trajectory prediction: a survey
Rudenko, A., Palmieri, L., Herman, M., Kitani, K. M., Gavrila, D. M., & Arras, K. O. (2020) · The International Journal of Robotics Research · doi:10.1177/0278364920917446 · Cited by 658
Self-citationAgentFormer: Agent-Aware Transformers for Socio-Temporal Multi-Agent Forecasting
Yuan, Y., Weng, X., Ou, Y., & Kitani, K. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00967 · Cited by 631
Self-citationLarge Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Dataset
Ettinger, S., Cheng, S., Caine, B., Liu, C., Zhao, H., Pradhan, S., Chai, Y., Sapp, B., Qi, C., Zhou, Y., et al. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00957 · Cited by 584
Metadata
- Authors
- Rhinehart, N., Mcallister, R., Kitani, K., & Levine, S.
- Venue
- 2019 IEEE/CVF International Conference on Computer Vision (ICCV)
- Publisher
- IEEE
- Published
- 2019-10
- Pages
- 2821-2830
- OpenAlex
- W2943516367
- Cited by
- 345
- References
- 35
- FWCI
- 50.75
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.