Recurrent Neural Networks for driver activity anticipation via sensory-fusion architecture
Open accessThe DOI matches its Crossref record; 48 of the 51 references it cites were found in a registry or an index; 12 of the 48 sources its claims cite were checked for support.
Abstract
Anticipating the future actions of a human is a widely studied problem in robotics that requires spatio-temporal reasoning. In this work we propose a deep learning approach for anticipation in sensory-rich robotics applications. We introduce a sensory-fusion architecture which jointly learns to anticipate and fuse information from multiple sensory streams. Our architecture consists of Recurrent Neural Networks (RNNs) that use Long Short-Term Memory (LSTM) units to capture long temporal dependencies. We train our architecture in a sequence-to-sequence prediction manner, and it explicitly learns to predict the future given only a partial temporal context. We further introduce a novel loss layer for anticipation which prevents over-fitting and encourages early anticipation. We use our architecture to anticipate driving maneuvers several seconds before they happen on a natural driving data set of 1180 miles. The context for maneuver anticipation comes from multiple sensors installed on the vehicle. Our approach shows significant improvement over the state-of-the-art in maneuver anticipation by increasing the precision from 77.4% to 90.5% and recall from 71.2% to 87.4%.
Overview
CoChat Summary
TL;DR
The paper proposes a sensory-fusion RNN-LSTM architecture for anticipating driver maneuvers from multiple vehicle sensors, trained sequence-to-sequence with an exponential loss layer. On 1180 miles of natural driving data, it improves maneuver anticipation over prior state-of-the-art, reaching 90.5% precision and 87.4% recall when using 3D head-pose features.
Key findings
- The Fusion-RNN with exponential loss (F-RNN-EL) improved all-maneuver anticipation over AIO-HMM on the same features, increasing precision from 77.4% to 84.5% and recall from 71.2% to 77.1%, with time-to-maneuver 3.58 s.
- Adding CLNF-tracked 68 facial landmarks and 3D head-pose features further increased F-RNN-EL to 90.5% precision and 87.4% recall, with time-to-maneuver 3.16 s.
- Fusing high-level representations from separate sensory RNNs outperformed simple concatenation: in the all-maneuvers setting, F-RNN-EL had about 6% higher precision and recall than S-RNN.
- The exponential loss improved over uniform loss by about 2% precision and recall in the all-maneuvers setting, while time-to-maneuver remained comparable.
Methods
- Architecture: each sensory stream is processed by an independent RNN with LSTM units; high-level representations are combined through a fully connected fusion layer and softmax output. Training is sequence-to-sequence, mapping partial observation sequences to the future event, with an exponential loss \(e^{-(T-t)}\log(y_t^k)\) that penalizes later mistakes more.
- Data: 1180 miles of natural freeway and city driving from 10 drivers, containing 700 annotated events (274 lane changes, 131 turns, 295 straight-driving instances), augmented to 2250 events. Context comes from a driver-facing camera, road-facing camera, GPS, and vehicle dynamics; improved vision uses CLNF for 68 facial landmarks and 3D head pose.
- Evaluation: 5-fold cross-validation with precision, recall, and time-to-maneuver, choosing a prediction threshold that maximizes F1. Baselines include Chance, Random-Forest, SVM, IOHMM, AIO-HMM, S-RNN, F-RNN-UL, and F-RNN-EL.
Limitations
- The paper does not include an explicit limitations section.
- Evident from the text: evaluation is on a single natural driving dataset of 1180 miles from 10 drivers with 5-fold cross-validation; no external or cross-dataset validation is reported.
- Evident from the text: precision and recall exclude the “driving straight” maneuver because anticipation algorithms default to it when not confident, and data augmentation only adds redundancy rather than new information.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures

![Fig. 2 Variations in the data set. Images from the data set [19] for a left lane change. ( Left ) Views from the road facing camera. ( Right ) Driving style of the drivers vary for the same maneuver.](https://figures.cochat.ai/paper-figures/f183cae4b17b4bccb552955c36acb9d6/recurrent-neural-networks-for-driver-activity-anticipation-via-sensory-fusion-figure-2.png)


![Fig. 5 Maneuver anticipation pipeline. Temporal context in maneuver anticipation comes from cameras facing the driver and the road, the GPS, and the vehicle's dynamics. ( a , b and c ) We improve upon the vision pipeline from Jain et al. [19] by tracking 68 landmark points on the driver's face and including the 3D head-pose features. ( d ) Using the Fusion-RNN we combine the sensory streams of features from inside the vehicle (driver facing camera), with the features from outside the vehicle (road camera, GPS, vehicle dynamics). ( e ) The model outputs the predicted probabilities of future maneuvers.](https://figures.cochat.ai/paper-figures/9617ccca82c44db4a16ed8745a46abd8/recurrent-neural-networks-for-driver-activity-anticipation-via-sensory-fusion-figure-5.png)
![Fig. 6 Improved features for maneuver anticipation. We track facial landmark points using the CLNF tracker [3] which results in more consistent 2D trajectories as compared to the KLT tracker [32] used by Jain et al. [19]. Furthermore, the CLNF also gives an estimate of the driver's 3D head pose.](https://figures.cochat.ai/paper-figures/c2c2ce8af37648d3918180e38c0abe7b/recurrent-neural-networks-for-driver-activity-anticipation-via-sensory-fusion-figure-6.png)


Verification
Claims the paper makes
4 Valid · 9 Not valid · 18 Not checked · 47 Own results and interpretations
The cited sources that could be checked couldn’t confirm or dispute this claim.
In the paper
“Anticipating human activities has been shown to improve human-robot collaboration [40, 21, 25, 10].”
What the sources say
- Not checked — no full text of the sourcecites [40] [40] Z. Wang, K. M¨ ulling, M. Deisenroth, H. Amor, D. Vogt, B. Sch¨ olkopf, and J. Peters. Probabilistic movement modeling for intention inference in human-robot interaction. IJRR , 2013. - source
- Not matchedcites [21] [21] H. Koppula and A. Saxena. Anticipating human activities using object affordances for reactive robotic response. In RSS , 2013. -
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not checked — no full text of the sourcecites [25] [25] J. Mainprice and D. Berenson. Human-robot collaborative manipulation planning using early prediction of human motion. In IROS , 2013. - source
- Could not verifycites [10] [10] A. Dragan and S. Srinivasa. Formalizing assistive teleoperation. In RSS , 2012. - sourceExperimental — accuracy not yet measured
The Related Work section mentions human activity detection but does not say that anticipating activities improves human-robot collaboration. Searches for activity anticipation and collaborative outcomes returned passages about object manipulation planning instead, so our copy does not address the attributed claim. (LLM, conf=0.76)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The cited sources that could be checked couldn’t confirm or dispute this claim.
In the paper
“In another application, anticipating driving maneuvers several seconds in advance [19, 27, 38, 36] enables assistive cars to alert drivers before they make a dangerous maneuver.”
What the sources say
- Not checked — no full text of the sourcecites [19] [19] A. Jain, H. S. Koppula, B. Raghavan, S. Soh, and A. Saxena. Car that knows before you do: Anticipating maneuvers via learning temporal driving models. In ICCV (To appear) , 2015. - source
- Could not verifycites [27] [27] B. Morris, A. Doshi, and M. Trivedi. Lane change intent prediction for driver assistance: On-road design and evaluation. In IEEE IVS , 2011. - sourceExperimental — accuracy not yet measured
The passages from §2.2 (pp. 3–4) discuss recurrent-network dynamics, not driving. Searches for maneuver anticipation, driver alerts, and vehicle prediction found no passage in our copy addressing the claim. (LLM, conf=0.76)
- Not checked — no full text of the sourcecites [38] [38] R. Vasudevan, V. Shia, Y. Gao, R. Cervera-Navarro, R. Bajcsy, and F. Borrelli. Safe semi-autonomous control with enhanced driver modeling. In ACC , 2012. - source
- Not checked — no full text of the sourcecites [36] [36] A. Tawari, S. Sivaraman, M. Trivedi, T. Shannon, and M. Tippelhofer. Looking-in and looking-out vision for urban intelligent assistance: Estimation of driver attentive state and dynamic surround for safe merging and braking. In IEEE IVS , 2014. - source
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Furthermore, the time-to-maneuver remains comparable for F-RNN with and without exponential loss.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As shown in Table I, exponentially growing the loss improves performance.”
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The F1-score does not undergo large variations with changes to the prediction threshold.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
32 more claims
Create a free account to see all 78 claims and to verify the 1 reference still open.
- It may state two separate claims as one. 23
- Probably not a substantial claim. 25
- The quote only partly states this claim. 1
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ VII. CONCLUSION · p. 8
“It improved the precision from 78% to 84.5% and recall from 71.1% to 77.1%.”
§ I. INTRODUCTION · p. 1
“On this data set, our deep learning approach improves the state-ofthe-art in maneuver anticipation by increasing the precision from 77.4% to 84.5% and recall from 71.2% to 77.1% .”
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 4×
“A common remedy to vanishing gradients is to replace tanh non-linearities by Long ShortTerm Memory cells [16].”
BackgroundMethodInfluentialSequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766 · cited 1×
“previous works on RNNs and LSTM from the language [35], speech [15], and vision [8] communities.”
Are we ready for autonomous driving? The KITTI vision benchmark suite
Geiger, A., Lenz, P., & Urtasun, R. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6248074 · Cited by 14,897 · cited 1×
“cs applications is challenging because the contextual information can come from multiple sensors with different data modalities. Examples include autonomous vehicles that reason from multiple sensors [2] or robots that jointly reason over perception and language instructions [26]. In such applications the way informat…”
BackgroundShi, J., & Tomasi (1994) · Proceedings of IEEE Conference on Computer Vision and Pattern Recognition CVPR-94 · doi:10.1109/cvpr.1994.323794 · Cited by 7,724 · cited 2×
“[19], the driver facing camera detects discriminative points on the driver’s face and tracks the detected points across frames using the KLT tracker [32].”
MethodLong-term recurrent convolutional networks for visual recognition and description
Donahue, J., Hendricks, L. A., Guadarrama, S., Rohrbach, M., Venugopalan, S., Darrell, T., & Saenko, K. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298878 · Cited by 5,228 · cited 2×
“We consider RNNs in multimodal setting, which is related to the recent use of RNNs in image-captioning [8].”
MethodOn the difficulty of training recurrent neural networks
Pascanu, R., Mikolov, T., & Bengio, Y. (2012) · arXiv · doi:10.48550/arxiv.1211.5063 · Cited by 4,310 · cited 3×
“RNNs with this choice of f suffer from a well-studied problem of vanishing gradients [29], and hence are poor at capturing long temporal dependencies which are essential for anticipation.”
BackgroundMethodAction Recognition with Improved Trajectories
Wang, H., & Schmid, C. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.441 · Cited by 3,407
(2011) · Cited by 2,752
Hierarchical recurrent neural network for skeleton based action recognition
Du, Y., Wang, W., & Wang, L. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298714 · Cited by 1,895 · cited 14×
“The high level representations from both RNNs {(hx1 , ...,hxT ), (hz1, ...,hzT ) are then concatenated at each time step t and passed through a fully connected (fusion) layer which fuses the two representations (11), as shown in Figure 4.”
MethodInfluential
Research works printed but not linked: 2
- jain, 2015, Neural
- 0, Bosch urban
Cited by
Items 201–272 of 272
Review of Video Predictive Understanding: Early Action Recognition and Future Action PredictionPreprint
Zhao, H., & Wildes, R. P. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.05140 · Cited by 13
A Bi-hemisphere Capsule Network Model for Cross-Subject EEG Emotion Recognition
Luan, X., Zhang, G., & Yang, K. (2023) · Communications in computer and information science · doi:10.1007/978-981-99-1645-0_27 · Cited by 5
Cheng, K., Eric, L. K., & Liu, Q. (2020) · IEEE Access · doi:10.1109/access.2020.3008848 · Cited by 4
Zhang, Z., Chen, S., Yao, H., Ong, G. P., & Ma, Y. (2024) · Traffic Injury Prevention · doi:10.1080/15389588.2024.2378131 · Cited by 5
Multi-Sensor Fusion Based Robot Self-Activity Recognition
Luo, D., Ma, Y., Zhang, X., & Wu, X. H. (2018) · IEEE-RAS International Conference on Humanoid Robots (Humanoids) · doi:10.1109/humanoids.2018.8624918 · Cited by 3
Vehicle Action Prediction Using Artificial Intelligence
Meng, K., Shi, C., & Yu, M. (2018) · International Conference on Machine Learning and Applications · doi:10.1109/icmla.2018.00200 · Cited by 4
Action Prediction Using Extremely Low-Resolution Thermopile Sensor Array For Elderly Monitoring
Morawski, I., Lie, W., & Chiang, J. (2021) · Proceedings - International Conference on Image Processing · doi:10.1109/icip42928.2021.9506024 · Cited by 4
What makes an on-road object important?
Ohn-Bar, E., & Trivedi, M. M. (2016) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr.2016.7900158 · Cited by 5
Towards a Multimodal and Context-Aware Framework for Human Navigational Intent Inference
Zhang, Z. (2020) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/3382507.3421156 · Cited by 4
Scene-segmentation based control strategy of energy storage system for urban railway
Zhu, F., Yang, Z., Lin, F., & Xia, H. (2017) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc.2017.8317678 · Cited by 3
PSINES: Activity and Availability Prediction for Adaptive Ambient Intelligence
Cumin, J., Lefebvre, G., Ramparany, F., & Crowley, J. L. (2020) · ACM Transactions on Autonomous and Adaptive Systems · doi:10.1145/3424344 · Cited by 4
Spatio-Temporal Adaptation With Dilated Neighbourhood Attention For Accident Anticipation
Patera, P., Chen, Y.-T., & Fang, W.-H. (2024) · 2024 IEEE International Conference on Image Processing (ICIP) · doi:10.1109/icip51287.2024.10647316 · Cited by 3
Human Motion Prediction Based on Space-Time-Separable Graph Convolutional Network
Li, R., He, D., Yang, S., Yan, A., Zeng, X. A., & Li, D. (2023) · IEEE Access · doi:10.1109/access.2023.3325291 · Cited by 3
Driver Classification Using K-Means Clustering of Within-Car Accelerometer Data
SERTTAŞ, T. N., Gerek, Ö. N., & Hocaoğlu, F. O. (2019) · Signal Processing and Communications Applications Conference (SIU) · doi:10.1109/siu.2019.8806602 · Cited by 3
Predicting the Future: A Jointly Learnt Model for Action AnticipationPreprint
Gammulle, H., Denman, S., Sridharan, S., & Fookes, C. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.07148 · Cited by 8
Real-Time Workload Classification during Driving using HyperNetworks
Wang, R., Amadori, P. V., & Demiris, Y. (2018) · arXiv · doi:10.48550/arxiv.1810.03145 · Cited by 3
Higher Order Recurrent Space-Time Transformer for Video Action PredictionPreprint
Tai, T., Fiameni, G., Lee, C., & Lanz, O. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.08665 · Cited by 13
Vehicle Behavior Recognition and Prediction Method for Intelligent Driving in Highway Scene
Sun, C., Shangguan, W., & Chai, L. (2020) · Chinese Automation Congress · doi:10.1109/cac51589.2020.9327730 · Cited by 3
Domain Adaptation with One-step Transformation
Peng, X., Li, Y., Murphey, Y. L., Wei, X., & Jianhua, L. (2018) · IEEE Symposium Series on Computational Intelligence (SSCI) · doi:10.1109/ssci.2018.8628835 · Cited by 4
Providing Context Awareness in the Smart Car Environment: State of the Art
Soultana, A., Benabbou, F., & Sael, N. (2020) · Lecture notes in intelligent transportation and infrastructure · doi:10.1007/978-3-030-37629-1_59 · Cited by 3
Agent-Centric Risk Assessment: Accident Anticipation and Risky Region LocalizationPreprint
Zeng, K.-H., Chou, S.-H., Chan, F.-H., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.06560 · Cited by 3
HDD4DBP: A Large-Scale Multi-Modal Benchmark on Driving Behavior Prediction
Dong, Q., Zhang, Z., Chang, Y., Chen, W., & Wang, L. (2024) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/smc54092.2024.10831786 · Cited by 3
Integrated security of cyber-physical vehicle networked systems in the age of 5G
Xue, L., Peng, J., Guan, X., & Ma, X. (2019) · Scientia Sinica Informationis · doi:10.1360/ssi-2019-0176 · Cited by 3
E-DNet: An End-to-End Dual-Branch Network for Driver Steering Intention Detection
Fu, Y., Xue, H., Fu, J., & Xu, Z. (2024) · Electronics · doi:10.3390/electronics13132477 · Cited by 2
Nishio, Y., & Shen, T. (2018) · IFAC-PapersOnLine · doi:10.1016/j.ifacol.2018.10.148 · Cited by 2
Uncertainty-Aware Mixture of Experts for Video Action Anticipation
Qi, Z., Wang, S., Zhang, W., & Huang, Q. (2025) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2025.3577027 · Cited by 2
Inferring and Learning Multi-Robot Policies by Observing an ExpertPreprint
Pierpaoli, P., Ravichandar, H. C., Waytowich, N. R., Li, A., Asher, D. E., & Egerstedt, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.07887 · Cited by 3
Leelavathy, S., Muthu, N., Dhaya, R., Muthuselvan, S., Kanthavel, R., & Rajakumari, K. (2023) · International Conference on Smart Electronics and Communication (ICOSEC) · doi:10.1109/icosec58147.2023.10276057 · Cited by 3
Joint streaming model for backchannel prediction and automatic speech recognition
Choi, Y., Bang, J., & Kim, S. H. (2024) · ETRI Journal · doi:10.4218/etrij.2023-0358 · Cited by 2
Prediction of driver actions with long short-term memory recurrent neural networks
Torstensson, M. (2018) · Chalmers Publication Library (Chalmers University of Technology) · Cited by 1
Leveraging Bioclimatic Context for Supervised and Self-supervised Land Cover Classification
Leonhardt, J., Drees, L., Gall, J., & Roscher, R. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-54605-1_15 · Cited by 2
A Review of Cockpit-driving Integration for Human-centric Autonomous DrivingPreprint
Zhu, Z., Zhao, S., Wang, C., & Du, A. (2023) · doi:10.36227/techrxiv.24428014 · Cited by 1
Early Anticipation of Driving Maneuvers
Wasi, A., Gangisetty, S., Rai, S. N., & Jawahar, C. V. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72897-6_9 · Cited by 6
Text Descriptions of Actions and Objects Improve Action Anticipation
Beedu, A., Haresamudram, H., & Essa, I. A. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10888324 · Cited by 2
Wang, H., Zhou, D., Zhang, Y., Ning, N., & Zhou, Y. F. (2021) · International Conference on Electronic Information Technology and Computer Engineering (EITCE) · doi:10.1145/3501409.3501598 · Cited by 1
Sensor Fusion: Gated Recurrent Fusion to Learn Driving Behavior from Temporal Multimodal DataPreprint
Narayanan, A., Siravuru, A., & Dariush, B. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.00628 · Cited by 4
SIBI Syllable Recognition System With LSTM
Sulaksono, J., Dwi Girinatari, I. A., Sudarma, M., et al. (2023) · 2023 International Conference on Smart-Green Technology in Electrical and Information Systems (ICSGTEIS) · doi:10.1109/icsgteis60500.2023.10424084 · Cited by 1
Driver Pose Estimation by Hybrid Convolutional Network Architecture
Li, P., Lu, M., & Zhang, X. (2018) · Chinese Automation Congress · doi:10.1109/cac.2018.8623392 · Cited by 1
Multimodal adaptive fusion for enhanced long-term action anticipation
Jing, Y., Xiao, Y., Fang, H., Mao, K., Zhao, J., & Xiao, X. (2025) · Machine Vision and Applications · doi:10.1007/s00138-025-01774-w · Cited by 1
Automotive Cockpit-Driving Integration for Human-Centric Autonomous Driving: A Survey
Zhu, Z., Zhao, S., Cai, M., Wang, C., & Du, A. (2025) · IEEE Transactions on Human-Machine Systems · doi:10.1109/thms.2025.3616947 · Cited by 2
Pedestrian Intention Anticipation with Uncertainty Based Decision for Autonomous Driving
Correia, J. R., Moreno, P., & Avelino, J. (2022) · IEEE International Conference on Robotic Computing (IRC) · doi:10.1109/irc55401.2022.00038 · Cited by 1
Automated Vehicle Research by Recognizing Maneuvering Modes using LSTM Model
Kim, E., & Oh, A. H. (2017) · The Journal of The Korea Institute of Intelligent Transport Systems · doi:10.12815/kits.2017.16.4.153 · Cited by 2
Action Anticipation By Predicting Future Dynamic Images
Rodriguez, C., Fernando, B., & Li, H. (2018) · arXiv · doi:10.48550/arxiv.1808.00141 · Cited by 1
Driver's Hand Trajectory-Guided Takeover Intention Inference in Conditionally Automated Driving
Zhu, J., Ma, Y., Yang, H., Lv, C., Zhang, Y., & Hao, S. (2025) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2025.3568449 · Cited by 1
Anticipating Daily Intention using On-Wrist Motion Triggered SensingPreprint
Wu, T.-Y., Chien, T.-A., Chan, C.-S., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1710.07477 · Cited by 1
Trajectory-Guided Driving Behavior Prediction for Autonomous Driving
Zhang, Z., Chang, Y., Dong, Q., & Wang, L. (2024) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2024.3486108 · Cited by 1
Toward Egocentric Compositional Action Anticipation with Adaptive Semantic Debiasing
Zhang, T., Min, W., Liu, T., Jiang, S., & Rui, Y. (2024) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3633333 · Cited by 2
A Review of Cockpit-driving Integration for Human-centric Autonomous DrivingPreprint
Zhu, Z., Zhao, S., Wang, C., & Du, A. (2023) · doi:10.36227/techrxiv.24428014.v1 · Cited by 2
Driver Cognitive Workload Detection via Eye-tracking and Physiological Modalities
Zhao, X. (2018) · TSpace (University of Toronto) · Cited by 1
Self-Supervised Learning for 3D Action Prediction Based on Past Completeness and Future Trend
Wang\S, Y., Zheng\S, T., Hua, S., Zhao, C., & Wu, E. (2025) · Computing and Informatics · doi:10.31577/cai_2025_2_300
Distilling What and Why: Enhancing Driver Intention Prediction with MLLMs
Artham, S., Dasgupta, A., Gangisetty, S., & Jawahar, C. V. (2026) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61042.2026.00703 · Cited by 1
From Driver Talk To Future Action: Vehicle Maneuver Prediction by Learning from Driving Exam Dialogs
Roitberg, A., Reib, S., & Stiefelhagen, R. (2021) · IEEE Intelligent Vehicles Symposium · doi:10.1109/iv48863.2021.9575655
Multi-Modal Temporal Convolutional Network for Anticipating Actions in Egocentric VideosPreprint
Zatsarynna, O., Farha, Y. A., & Gall, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.09504
Chang, Y., Cheng, Y., Murray, J. F., Khalid, M., & Manzoor, U. (2025) · Journal of Field Robotics · doi:10.1002/rob.70057
Vellenga, K. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5804595
Ankita, D., Khilar, R., & Kumar, M. N. (2022) · International Conference on Mathematics, Actuarial Science, Computer Science and Statistics (MACS) · doi:10.1109/macs56771.2022.10022711
RetractedCNN-BiLSTM-Based Vehicle Intention Recognition From the Ego-Vehicle Perspective
Liu, X., Guo, B., Jin, L., et al. (2025) · 2025 9th CAA International Conference on Vehicular Control and Intelligence (CVCI) · doi:10.1109/cvci66304.2025.11348448
Wang, S., & Murphey, Y. L. (2021) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn52387.2021.9534110 · Cited by 2
Long, X., Wang, S., & Chen, Y. (2025) · Sensors · doi:10.3390/s25185644
Rangesh, A., Deo, N., Greer, R., Gunaratne, P., & Trivedi, M. M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.12932 · Cited by 19
Driver Intention Anticipation Based on In-Cabin and Driving Scene Monitoring Using Deep LearningPreprint
Bonyani, M., Rahmanian, M., & Jahangard, S. (2021) · arXiv (Cornell University)
Predicting Driver Intention Using Deep Neural NetworkPreprint
Bonyani, M., Rahmanian, M., & Jahangard, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.14790 · Cited by 2
Action Anticipation with RBF Kernelized Feature Mapping RNNPreprint
Shi, Y., Fernando, B., & Hartley, R. I. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.07806 · Cited by 6
Knowledge Distillation For Human Action AnticipationPreprint
Tran, V., Wang, Y., Zhang, Z., & Nguyen, M. H. (2019) · arXiv (Cornell University) · doi:10.17023/2q8c-x832 · Cited by 6
Towards Human-in-the-Loop Cyber-Physical Systems
Rill, R. A. (2021) · Eötvös Loránd Tudományegyetem · doi:10.15476/elte.2021.113
ICPR 2024 Competition on Rider Intention Prediction
Gangisetty, S., Wasi, A., Rai, S. N., Jawahar, C. V., Raj, S., Prajapati, M., Choudhary, A., Chandra, A., Chandan, D., Chand, S., et al. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-80139-6_3 · Cited by 2
Deng, Y., He, S., & Wang, H. (2024) · IEEE Smart World Congress (SWC) · doi:10.1109/swc62898.2024.00154
Learning From Natural Human Interactions For Assistive Robots
Jain, A. (2016) · eCommons (Cornell University) · doi:10.7298/x4mw2f2v
Self-citationAnticipating Maneuvers with Dilated Convolutions
Rekabdar, B. (2018) · ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1145/3173386.3176972
An Intelligent Seat Heating System for Commuting Scenarios Integrating Driver Intentions
Wang, W., Tian, C., Zhou, B., Jiang, D., Zhang, J. R., & Hu, H. (2025) · CAA International Conference on Vehicular Control and Intelligence (CVCI) · doi:10.1109/cvci66304.2025.11348246
Metadata
- Authors
- Jain, A., Singh, A., Koppula, H. S., Soh, S., & Saxena, A.
- Affiliations
- Cornell University (US)
- Stanford University (US)
- Venue
- 2016 IEEE International Conference on Robotics and Automation (ICRA)
- Publisher
- IEEE
- Published
- 2016-05
- Pages
- 3118-3125
- ISSN
- 1050-4729 (ISSN-L)
- OpenAlex
- W2174887554
- Cited by
- 271
- References
- 51
- Citations by year
- 2026: 2 · 2025: 16 · 2024: 24 · 2023: 34 · 2022: 19 · 2021: 42 · 2020: 33 · 2019: 41 · 2018: 31 · 2017: 21 · 2016: 8
- FWCI
- 43.54
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- Proceedings - IEEE International Conference on Robotics and Automation/Proceedings (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Science Foundation (1426452)
- Nvidia (K40 GPU)
- Office of Naval Research (N00014)
- Topics
- Physical Sciences › Engineering › Automotive Engineering › Autonomous Vehicle Technology and Safety
- Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
- Social Sciences › Transportation › Traffic Prediction and Management Techniques
- Keywords
- sensory fusion, recurrent neural networks, long short-term memory, sequence-to-sequence prediction, driving data, spatio-temporal reasoning, automotive sensors
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.