The DOI matches its Crossref record; 35 of the 35 references it cites were found in a registry or an index.
Rules of the Road: Predicting Driving Behavior With a Convolutional Model of Semantic Interactions
Open accessAbstract
We focus on the problem of predicting future states of entities in complex, real-world driving scenarios. Previous research has approached this problem via low-level signals to predict short time horizons, and has not addressed how to leverage key assets relied upon heavily by industry self-driving systems: (1) large 3D perception efforts which provide highly accurate 3D states of agents with rich attributes, and (2) detailed and accurate semantic maps of the environment (lanes, traffic lights, crosswalks, etc). We present a unified representation which encodes such high-level semantic information in a spatial grid, allowing the use of deep convolutional models to fuse complex scene context. This enables learning entity-entity and entity-environment interactions with simple, feed-forward computations in each timestep within an overall temporal model of an agent's behavior. We propose different ways of modelling the future as a {\em distribution} over future states using standard supervised learning. We introduce a novel dataset providing industry-grade rich perception and semantic inputs, and empirically show we can effectively learn fundamentals of driving behavior.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
(2003) · Journal of Machine Learning Research · doi:10.5555/944919.944937 · Cited by 32,337
Are we ready for autonomous driving? The KITTI vision benchmark suite
Geiger, A., Lenz, P., & Urtasun, R. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6248074 · Cited by 14,897
Blei, D. M., Ng, A. Y., & Jordan, M. I. (2002) · Advances in Neural Information Processing Systems 14 · doi:10.7551/mitpress/1120.003.0082 · Cited by 12,223
PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation
Charles, R. Q., Su, H., Kaichun, M., & Guibas, L. J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.16 · Cited by 9,237
Thrun, S. (2002) · Communications of the ACM · doi:10.1145/504729.504754 · Cited by 7,994
VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection
Zhou, Y., & Tuzel, O. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00472 · Cited by 4,813
An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling
Bai, S., Kolter, J. Z., & Koltun, V. (2018) · arXiv · doi:10.48550/arxiv.1803.01271 · Cited by 4,655
Categorical Reparameterization with Gumbel-Softmax
Jang, E., Gu, S., & Poole, B. (2016) · arXiv · doi:10.48550/arxiv.1611.01144 · Cited by 3,616
Social LSTM: Human Trajectory Prediction in Crowded Spaces
Alahi, A., Goel, K., Ramanathan, V., Robicquet, A., Fei-Fei, L., & Savarese, S. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.110 · Cited by 3,483
Multi-view 3D Object Detection Network for Autonomous Driving
Chen, X., Ma, H., Wan, J., Li, B., & Xia, T. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.691 · Cited by 3,450
End to End Learning for Self-Driving Cars
Bojarski, M., Del Testa, D., Dworakowski, D., Firner, B., Flepp, B., Goyal, P., Jackel, L. D., Monfort, M., Muller, U., Zhang, J., et al. (2016) · arXiv · doi:10.48550/arxiv.1604.07316 · Cited by 3,198
What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?
Kendall, A., & Gal, Y. (2017) · arXiv · doi:10.48550/arxiv.1703.04977 · Cited by 3,088
PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation
Qi, C. R., Su, H., Mo, K., & Guibas, L. J. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1612.00593 · Cited by 2,817
Pictorial Structures for Object Recognition
Felzenszwalb, P. F., & Huttenlocher, D. P. (2005) · International Journal of Computer Vision · doi:10.1023/b:visi.0000042934.15159.49 · Cited by 2,180
DeepDriving: Learning Affordance for Direct Perception in Autonomous Driving
Chen, C., Seff, A., Kornhauser, A., & Xiao, J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.312 · Cited by 1,876
ALVINN, an autonomous land vehicle in a neural network
Pomerleau, D. (2018) · doi:10.1184/r1/6603146 · Cited by 1,338
3D Bounding Box Estimation Using Deep Learning and Geometry
Mousavian, A., Anguelov, D., Flynn, J., & Kosecka, J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.597 · Cited by 1,129
DESIRE: Distant Future Prediction in Dynamic Scenes with Interacting Agents
Lee, N., Choi, W., Vernaza, P., Choy, C. B., Torr, P. H. S., & Chandraker, M. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.233 · Cited by 1,067
An Intriguing Failing of Convolutional Neural Networks and the CoordConv Solution
Liu, R., Lehman, J., Molino, P., Such, F. P., Frank, E., Sergeev, A., & Yosinski, J. (2018) · arXiv · doi:10.48550/arxiv.1807.03247 · Cited by 784
Human Action Recognition and Prediction: A Survey
Kong, Y., & Fu, Y. (2022) · International Journal of Computer Vision · doi:10.1007/s11263-022-01594-9 · Cited by 780
ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worst
Bansal, M., Krizhevsky, A., & Ogale, A. (2019) · Robotics: Science and Systems XV · doi:10.15607/rss.2019.xv.031 · Cited by 731
Kitani, K. M., Ziebart, B. D., Bagnell, J. A., & Hebert, M. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33765-9_15 · Cited by 701
Luo, W., Yang, B., & Urtasun, R. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00376 · Cited by 644
Deep Predictive Coding Networks for Video Prediction and Unsupervised Learning
Lotter, W. E., Kreiman, G., & Cox, D. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.08104 · Cited by 368
Probabilistic trajectory prediction with Gaussian mixture models
Wiest, J., Hoffken, M., Kresel, U., & Dietmayer, K. (2012) · 2012 IEEE Intelligent Vehicles Symposium · doi:10.1109/ivs.2012.6232277 · Cited by 337
Context-Based Pedestrian Path Prediction
Kooij, J. F. P., Schneider, N., Flohr, F., & Gavrila, D. M. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10599-4_40 · Cited by 286
Bagautdinov, T., Alahi, A., Fleuret, F., Fua, P., & Savarese, S. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.365 · Cited by 258
Long-Term On-board Prediction of People in Traffic Scenes Under Uncertainty
Bhattacharyya, A., Fritz, M., & Schiele, B. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00441 · Cited by 243
IntentNet: Learning to Predict Intention from Raw Sensor Data
Casas, S., Luo, W., & Urtasun, R. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.07907 · Cited by 218
r2p2: A ReparameteRized Pushforward Policy for Diverse, Precise Generative Path Forecasting
Rhinehart, N., Kitani, K. M., & Vernaza, P. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01261-8_47 · Cited by 211
First-Person Activity Forecasting with Online Inverse Reinforcement Learning
Rhinehart, N., & Kitani, K. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.399 · Cited by 122
CarFusion: Combining Point Tracking and Part Detection for Dynamic 3D Reconstruction of Vehicles
Reddy, N. D., Vo, M., & Narasimhan, S. G. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00204 · Cited by 79
Generative Modeling of Multimodal Multi-Human Behavior
Ivanovic, B., Schmerling, E., Leung, K., & Pavone, M. (2018) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2018.8594393 · Cited by 73
3D Vehicle Trajectory Reconstruction in Monocular Video Data Using Environment Structure Constraints
Bullinger, S., Bodensteiner, C., Arens, M., & Stiefelhagen, R. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01249-6_3 · Cited by 13
Cited by
VectorNet: Encoding HD Maps and Agent Dynamics From Vectorized Representation
Gao, J., Sun, C., Zhao, H., Shen, Y., Anguelov, D., Li, C., & Schmid, C. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01154 · Cited by 982
Learning Lane Graph Representations for Motion Forecasting
Liang, M., Yang, B., Hu, R., Chen, Y., Liao, R., Feng, S., & Urtasun, R. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58536-5_32 · Cited by 700
A Survey on Trajectory-Prediction Methods for Autonomous Driving
Huang, Y., Du, J., Yang, Z., Zhou, Z., Zhang, L., & Chen, H. (2022) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2022.3167103 · Cited by 677
Human motion trajectory prediction: a survey
Rudenko, A., Palmieri, L., Herman, M., Kitani, K. M., Gavrila, D. M., & Arras, K. O. (2020) · The International Journal of Robotics Research · doi:10.1177/0278364920917446 · Cited by 658
Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Dataset
Ettinger, S., Cheng, S., Caine, B., Liu, C., Zhao, H., Pradhan, S., Chai, Y., Sapp, B., Qi, C., Zhou, Y., et al. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00957 · Cited by 584
Self-citation
Metadata
- Authors
- Hong, J., Sapp, B., & Philbin, J.
- Venue
- 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2019-06
- Pages
- 8446-8454
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W2963759562
- Cited by
- 277
- References
- 35
- FWCI
- 32.65
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.