Human-Factors-in-Driving-Loop: Driver Identification and Verification via a Deep Learning Approach using Psychological Behavioral Data
Open accessThe DOI matches its Crossref record; 51 of the 51 references it cites were found in a registry or an index.
Abstract
Driver identification has been popular in the field of driving behavior analysis, which has a broad range of applications in anti-thief, driving style recognition, insurance strategy, and fleet management. However, most studies to date have only researched driver identification without a robust verification stage. This paper addresses driver identification and verification through a deep learning (DL) approach using psychological behavioral data, i.e., vehicle control operation data and eye movement data collected from a driving simulator and an eye tracker, respectively. We design an architecture that analyzes the segmentation windows of three-second data to capture unique driving characteristics and then differentiate drivers on that basis. The proposed model includes a fully convolutional network (FCN) and a squeeze-and-excitation (SE) block. Experimental results were obtained from 24 human participants driving in 12 different scenarios. The proposed driver identification system achieves an accuracy of 99.60% out of 15 drivers. To tackle driver verification, we combine the proposed architecture and a Siamese neural network, and then map all behavioral data into two embedding layers for similarity computation. The identification system achieves significant performance with average precision of 96.91%, recall of 95.80%, F1 score of 96.29%, and accuracy of 96.39%, respectively. Importantly, we scale out the verification system to imposter detection and achieve an average verification accuracy of 90.91%. These results imply the invariable characteristics from human factors rather than other traditional resources, which provides a superior solution for driving behavior authentication systems.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“A hierarchical approach to DBN, which is created by the hierarchical layer-by-layer RBN structure, is more effective in extracting characteristics from the dataset [110-112].”
From Vehicle Detection and Classification via YOLOv8 and Deep Belief Network over Aerial Image Sequences (2023) · cites this paper as [111]
Citations
Cites
Squeeze-and-Excitation Networks
Hu, J., Shen, L., & Sun, G. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00745 · Cited by 35,667
(2008) · Journal of Machine Learning Research · Cited by 33,051
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818 · cited 1×
“A convolutional layer followed by a batch normalization layer [47] and a ReLU activation layer is used as the basic block in FCN.”
MethodA Review of Recurrent Neural Networks: LSTM Cells and Network Architectures
Yu, Y., Si, X., Hu, C., & Zhang, J. (2019) · Neural Computation · doi:10.1162/neco_a_01199 · Cited by 5,591 · cited 1×
“More approaches have recently been proposed, such as convolutional neural network (CNN) [22], recurrent neural network (RNN) [23], auto encoder (AE) [24], etc .”
MethodAn introduction to hidden Markov models
Rabiner, L., & Juang, B. (1986) · IEEE ASSP Magazine · doi:10.1109/massp.1986.1165342 · Cited by 4,775 · cited 1×
“The hidden Markov model [20] was utilized to model these individual characteristics based on the operation data and achieved an accuracy of 85%.”
MethodConvolutional networks for images, speech, and time series
LeCun, Y., & Bengio, Y. (1998) · HAL (Le Centre pour la Communication Scientifique Directe) · doi:10.5555/303568.303704 · Cited by 4,231
Distance Metric Learning for Large Margin Nearest Neighbor Classification
Weinberger, K. Q., & Saul, L. K. (2009) · Journal of Machine Learning Research · doi:10.5555/1577069.1577078 · Cited by 4,044
The Handbook of brain theory and neural networks
(1996) · Choice Reviews Online · doi:10.5860/choice.33-4484 · Cited by 3,870
CARLA: An open urban driving simulator
Dosovitskiy, A., Ros, G., Codevilla, F., Lopez, A., & Koltun, V. (2017) · arXiv · doi:10.48550/arxiv.1711.03938 · Cited by 3,718 · cited 2×
“The vehicle models in Carla have high fidelity and provide realistic signals of acceleration, gear, and braking behavior [41].”
MethodTime series classification from scratch with deep neural networks: A strong baseline
Wang, Z., Yan, W., & Oates, T. (2017) · 2017 International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2017.7966039 · Cited by 2,299 · cited 4×
“based on FCN [45], which is improved by adding an SE block.”
BackgroundMethodInfluentialSignature Verification Using A "Siamese" Time Delay Neural Network
BROMLEY, J., BENTZ, J. W., BOTTOU, L., GUYON, I., LECUN, Y., MOORE, C., SÄCKINGER, E., & SHAH, R. (1993) · International Journal of Pattern Recognition and Artificial Intelligence · doi:10.1142/s0218001493000339 · Cited by 2,077 · cited 3×
“To overcome this challenge, Siamese Neural Networks [33] have been utilized to verify drivers from limited training samples in some driver verification studies [34], [35], [37].”
MethodMillimeter-Wave Technology for Automotive Radar Sensors in the 77 GHz Frequency Band
Hasch, J., Topak, E., Schnabel, R., Zwick, T., Weigel, R., & Waldschmidt, C. (2012) · IEEE Transactions on Microwave Theory and Techniques · doi:10.1109/tmtt.2011.2178427 · Cited by 1,372 · cited 1×
“Carla supports the recording of multi-channel signals from different sensors, such as collision detectors [38], Inertial Measurement Unit (IMU) sensors [39], radar sensors [40], etc .”
BackgroundCollision detection for interactive graphics applications
Hubbard, P. M. (1995) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/2945.466717 · Cited by 414 · cited 1×
“Carla supports the recording of multi-channel signals from different sensors, such as collision detectors [38], Inertial Measurement Unit (IMU) sensors [39], radar sensors [40], etc .”
BackgroundReviews on Various Inertial Measurement Unit (IMU) Sensor Applications
Ahmad, N., Ghazilla, R. A. R., Khairi, N. M., & Kasi, V. (2013) · International Journal of Signal Processing Systems · doi:10.12720/ijsps.1.2.256-262 · Cited by 406 · cited 1×
“Carla supports the recording of multi-channel signals from different sensors, such as collision detectors [38], Inertial Measurement Unit (IMU) sensors [39], radar sensors [40], etc .”
BackgroundRecent Advances in Autoencoder-Based Representation Learning
Tschannen, M., Bachem, O., & Lučić, M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.05069 · Cited by 329 · cited 1×
“More approaches have recently been proposed, such as convolutional neural network (CNN) [22], recurrent neural network (RNN) [23], auto encoder (AE) [24], etc .”
Method(2000) · Methods of Shape-Preserving Spline Approximation · doi:10.1142/9789812813381_0003 · Cited by 307 · cited 1×
“Finally, since the control operation data and eye movement data have different sampling rates, we adopt cubic spline interpolation [44] to resample vehicle control operation data with a sampling rate of 60 Hz.”
MethodAutomobile Driver Fingerprinting
Enev, M., Takakuwa, A., Koscher, K., & Kohno, T. (2016) · Proceedings on Privacy Enhancing Technologies · doi:10.1515/popets-2015-0029 · Cited by 228 · cited 3×
“This window length and overlap rate is adopted from [5] and [37].”
BackgroundMethodInsights Into LSTM Fully Convolutional Networks for Time Series Classification
Karim, F., Majumdar, S., & Darabi, H. (2019) · IEEE Access · doi:10.1109/access.2019.2916828 · Cited by 208 · cited 1×
“This model has achieved stateof-the-art performance on the tasks of classifying time series signals [48].”
BackgroundDriving Behavior Analysis Guidelines for Intelligent Transportation Systems
Nasr Azadani, M., & Boukerche, A. (2022) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2021.3076140 · Cited by 173 · cited 2×
“Hence driving simulators are currently a popular approach to obtaining driving data as they provide a wide range of driving scenarios and conveniently present data in a pre-controlled environment [6].”
BackgroundA validation study of driving errors using a driving simulator
Meuleners, L. B., & Fraser, M. L. (2015) · Transportation Research Part F Traffic Psychology and Behaviour · doi:10.1016/j.trf.2014.11.009 · Cited by 159 · cited 1×
“Besides, there is no clear difference between on-road and simulator driving performance in many tasks [18].”
BackgroundKnow your master: Driver profiling-based anti-theft method
Kwak, B. I., Woo, J., & Kim, H. K. (2016) · 2016 14th Annual Conference on Privacy, Security and Trust (PST) · doi:10.1109/pst.2016.7906929 · Cited by 159 · cited 3×
“Subsequently, it is now widely applied in the fields of anti-thief [1], driving style recognition [2], insurance strategy [3], and fleet management [4].”
BackgroundA Deep Learning Framework for Driving Behavior Identification on In-Vehicle CAN-BUS Sensor Data
Zhang, J., Wu, Z., Li, F., Xie, C., Ren, T., Chen, J., & Liu, L. (2019) · Sensors · doi:10.3390/s19061356 · Cited by 150 · cited 2×
“[28] adopted attention based DeepConvGRU and DeepConvLSTM by fusing CNN and RNN with an attention mechanism to identify 10 drivers.”
BackgroundMethodDriver identification using automobile sensor data from a single turn
Hallac, D., Sharang, A., Stahlmann, R., Lamprecht, A., Huber, M., Roehder, M., Sosič, R., & Leskovec, J. (2016) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc.2016.7795670 · Cited by 150
Automobile Driver Fingerprinting: A New Machine Learning Based Authentication Scheme
Xun, Y., Liu, J., Kato, N., Fang, Y., & Zhang, Y. (2020) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2019.2946626 · Cited by 141 · cited 9×
“Some studies [9], [10] use a prediction”
MethodInfluentialThe Improvement of Road Driving Safety Guided by Visual Inattentional Blindness
Xu, J., Park, S. H., Zhang, X., & Hu, J. (2022) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2020.3044927 · Cited by 131
Self-citationDriving Performance Under Violations of Traffic Rules: Novice vs. Experienced Drivers
Xu, J., Guo, K., & Sun, P. Z. H. (2022) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2022.3200592 · Cited by 116 · cited 1×
“their personal bias even conducting the same driving tasks, such as novice driver against experienced driver [31].”
BackgroundSelf-citationCharacterizing Driving Styles with Deep Learning
Dong, W., Li, J., Yao, R., Li, C., Yuan, T., & Wang, L. (2016) · arXiv · doi:10.48550/arxiv.1607.03611 · Cited by 100 · cited 1×
“As current smartphones are typically equipped with GPS [11] and gyroscope [12], more recent studies [13], [14] have utilized this low-cost equipment to acquire driving data.”
MethodImproving Driver Identification for the Next-Generation of In-Vehicle Software Systems
Mekki, A. E., Bouhoute, A., & Berrada, I. (2019) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2019.2924906 · Cited by 99 · cited 9×
“Another FCN-based deep learning model, LSTM-FCN [29], also achieves a competitive accuracy of 96.”
BackgroundMethodInfluentialDriver Profiling Using Long Short Term Memory (LSTM) and Convolutional Neural Network (CNN) Methods
Cura, A., Küçük, H., Ergen, E., & Oksuzoglu, I. B. (2020) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2020.2995722 · Cited by 91 · cited 1×
“Subsequently, it is now widely applied in the fields of anti-thief [1], driving style recognition [2], insurance strategy [3], and fleet management [4].”
BackgroundWho is behind the wheel? Driver identification and fingerprinting
Ezzini, S., Berrada, I., & Ghogho, M. (2018) · Journal Of Big Data · doi:10.1186/s40537-018-0118-7 · Cited by 85 · cited 1×
“In recent years, the driver verification task has attracted researchers’ attention [1], [32].”
BackgroundXu, J., Zhang, X., Park, S. H., & Guo, K. (2022) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2022.3149994 · Cited by 84
Self-citationAutoencoder Regularized Network For Driving Style Representation Learning
Dong, W., Yuan, T., Yang, K., Li, C., & Zhang, S. (2017) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2017/222 · Cited by 82 · cited 1×
“Dong et al. [26] proposed a novel Autoencoder Regularized deep neural Network (ARNet) that combines unsupervised and supervised learning to predict drivers’ driving styles from GPS data.”
BackgroundDeng, Q., Wang, J., Hillebrand, K., Benjamin, C. R., & Söffker, D. (2019) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2019.2937287 · Cited by 79 · cited 1×
“According to recent studies [42] incorporating eye-tracking measurement in driving behavior analysis, we obtained eye”
ResultA “pay-how-you-drive” car insurance approach through cluster analysis
Carfora, M. F., Martinelli, F., Mercaldo, F., Nardone, V., Orlando, A., Santone, A., & Vaglini, G. (2018) · Soft Computing · doi:10.1007/s00500-018-3274-y · Cited by 73 · cited 1×
“Subsequently, it is now widely applied in the fields of anti-thief [1], driving style recognition [2], insurance strategy [3], and fleet management [4].”
BackgroundInvestigations on Driver Unique Identification from Smartphone’s GPS Data Alone
Chowdhury, A., Chakravarty, T., Ghose, A., Banerjee, T., & Balamuralidhar, P. (2018) · Journal of Advanced Transportation · doi:10.1155/2018/9702730 · Cited by 64 · cited 1×
“[15] presented an identification methodology by using smartphone GPS data alone and achieved an average accuracy of 82.”
MethodDriver Identification Based on Wavelet Transform Using Driving Patterns
Kwak, B. I., Han, M. L., & Kim, H. K. (2020) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2020.2999911 · Cited by 62 · cited 2×
“Most studies [10], [29] train their models with a large dataset,”
BackgroundA Study of Individual Characteristics of Driving Behavior Based on Hidden Markov Model
Zhang, X., Zhao, X., & Rong, J. (2012) · DOAJ (DOAJ: Directory of Open Access Journals) · Cited by 60
Driver Identification Based on Vehicle Telematics Data using LSTM-Recurrent Neural Network
Girma, A., Yan, X., & Homaifar, A. (2019) · IEEE International Conference on Tools with Artificial Intelligence (ICTAI) · doi:10.1109/ictai.2019.00127 · Cited by 58 · cited 1×
“[27] proposed a Long-Short Term Memory (LSTM) model to predict driver identification.”
BackgroundChen, J., Wu, Z., & Zhang, J. (2018) · Applied Soft Computing · doi:10.1016/j.asoc.2018.09.030 · Cited by 47 · cited 1×
“[30] constructed a deep auto-encoder to automatically extract the hidden features from 51 original driving behavior features collected through the OBD-II.”
BackgroundDriver Identification and Verification From Smartphone Accelerometers Using Deep Neural Networks
Hernández, S., Pozo, R. F., & Gómez, L. A. H. (2020) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2020.3008210 · Cited by 43 · cited 1×
“To overcome this challenge, Siamese Neural Networks [33] have been utilized to verify drivers from limited training samples in some driver verification studies [34], [35], [37].”
MethodEvaluation of gyroscope-embedded mobile phones
Barthold, C., Subbu, K. P., & Dantu, R. (2011) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/icsmc.2011.6083905 · Cited by 41 · cited 1×
“As current smartphones are typically equipped with GPS [11] and gyroscope [12], more recent studies [13], [14] have utilized this low-cost equipment to acquire driving data.”
MethodAzadani, M. N., & Boukerche, A. (2022) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2022.3151264 · Cited by 39 · cited 4×
“This window length and overlap rate is adopted from [5] and [37].”
BackgroundMethodInfluentialA Temporally Irreversible Visual Attention Model Inspired by Motion Sensitive Neurons
Xu, J., Park, S. H., & Zhang, X. (2019) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2019.2934144 · Cited by 36
Self-citationCombining Unsupervised Anomaly Detection and Neural Networks for Driver Identification
Tanprasert, T., Saiprasert, C., & Thajchayapong, S. (2017) · Journal of Advanced Transportation · doi:10.1155/2017/6057830 · Cited by 34 · cited 1×
“[25] combined unsupervised anomaly detection and feed-”
Zero-to-Stable Driver Identification: A Non-Intrusive and Scalable Driver Identification Scheme
Rahim, M. A., Zhu, L., Li, X., Liu, J., Zhang, Z., Qin, Z., Khan, S., & Gai, K. (2019) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2019.2954529 · Cited by 33 · cited 1×
“[16] used three signals (orientation, velocity, and acceleration) from smartphone GPS data to identify 25 drivers and performed experiments on both the collected data and the real-world open datasets.”
MethodFleet management: A vehicle and driver assignment model
Monnerat, F., Dias, J., & Alves, M. J. (2019) · European Journal of Operational Research · doi:10.1016/j.ejor.2019.03.021 · Cited by 31 · cited 1×
“Subsequently, it is now widely applied in the fields of anti-thief [1], driving style recognition [2], insurance strategy [3], and fleet management [4].”
BackgroundSmartwatch-Based Open-Set Driver Identification by Using GMM-Based Behavior Modeling Approach
Putri, R. R. M., Yang, C., Chang, C.-C., & Liang, D. (2020) · IEEE Sensors Journal · doi:10.1109/jsen.2020.3030810 · Cited by 21
Driver authentication by quantifying driving style using GPS only
Banerjee, T., Chowdhury, A., Chakravarty, T., & Ghose, A. (2020) · IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops) · doi:10.1109/percomworkshops48775.2020.9156080 · Cited by 14 · cited 1×
“As current smartphones are typically equipped with GPS [11] and gyroscope [12], more recent studies [13], [14] have utilized this low-cost equipment to acquire driving data.”
MethodDriver Information Embedding with Siamese LSTM networks
Dang, H. Q., & Fürnkranz, J. (2019) · IEEE Intelligent Vehicles Symposium · doi:10.1109/ivs.2019.8813795 · Cited by 13 · cited 1×
“To overcome this challenge, Siamese Neural Networks [33] have been utilized to verify drivers from limited training samples in some driver verification studies [34], [35], [37].”
MethodDriver classification in vehicle following behavior by using dynamic potential field method
Woo, H., Ji, Y., Tamura, Y., Kuroda, Y., Sugano, T., Yamamoto, Y., Yamashita, A., & Asama, H. (2017) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc.2017.8317774 · Cited by 10
Driver Identification for Different Road Shapes Using Vehicle IoT Sensing Data
Lee, J.-H., Kim, M., Park, S., Choi, J. K., & Hwang, Y. (2021) · International Conference on Consumer Electronics · doi:10.1109/icce50685.2021.9427668 · Cited by 6
Research works printed but not linked: 1
- Amin, Mobile century using GPS mobile phones as traffic sensors: A field experiment, 16
Cited by
Amiri, Z., Heidari, A., Navimipour, N. J., Unal, M., & Mousavi, A. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-023-16382-x · Cited by 189
Robust human locomotion and localization activity recognition over multisensory
Khan, D., Alonazi, M., Abdelhaq, M., Mudawi, N. A., Algarni, A. M., Jalal, A., & Liu, H. (2024) · Frontiers in Physiology · doi:10.3389/fphys.2024.1344887 · Cited by 220
Zhou, X., Gui, W., Heidari, A. A., Cai, Z., Liang, G., & Chen, H. (2023) · Applied Soft Computing · doi:10.1016/j.asoc.2023.110513 · Cited by 80
Vehicle Detection and Classification via YOLOv8 and Deep Belief Network over Aerial Image Sequences
Al Mudawi, N., Qureshi, A. M., Abdelhaq, M., Alshahrani, A., Alazeb, A., Alonazi, M., & Algarni, A. (2023) · Sustainability · doi:10.3390/su151914597 · Cited by 68
A Wearable Inertial Sensor Approach for Locomotion and Localization Recognition on Physical Activity
Khan, D., Mudawi, N. A., Abdelhaq, M., Alazeb, A., Alotaibi, S. S., Algarni, A. M., & Jalal, A. (2024) · Sensors · doi:10.3390/s24030735 · Cited by 67
“We applied a methodical approach to compute the linear prediction cepstral coeffi-cients (LPCC) [42 , 43] for various locomotion activities.”
BackgroundMethod
Metadata
- Authors
- Xu, J., Pan, S., Sun, P. Z. H., Hyeong Park, S., & Guo, K.
- Affiliations
- Wenzhou University (CN)
- Shanghai Jiao Tong University (CN)
- Division of Software, Hallym University, Chuncheon, South Korea
- University of Lincoln (GB)
- Venue
- IEEE Transactions on Intelligent Transportation Systems
- Published
- 2023-03
- Volume
- 24
- Issue
- 3
- Pages
- 3383-3394
- ISSN
- 1524-9050 (ISSN-L)
- 1558-0016
- OpenAlex
- W4312242979
- Cited by
- 120
- References
- 51
- Citations by year
- 2026: 11 · 2025: 18 · 2024: 32 · 2023: 59
- FWCI
- 26.99
- Type
- journal article
- Language
- en
- Open access
- green
- figshare.com
- Licence
- Locations
- IEEE Transactions on Intelligent Transportation Systems (publishedVersion)
- Lincoln Repository (University of Lincoln) (acceptedVersion)
- Figshare (submittedVersion, cc-by)
- Funders
- EU’s Seventh Framework Programme (EU FP7) International Research Staff Exchange Scheme (IRSES) Project–Towards zero road Accidents—Nature Inspired Hazard Perception (318907)
- Zhejiang Provincial Natural Science Foundation (LY23F030001)
- Topics
- Physical Sciences › Computer Science › Information Systems › User Authentication and Security Systems
- Physical Sciences › Computer Science › Signal Processing › Biometric Identification and Security
- Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
- Keywords
- driver identification, driver verification, driver behavior analysis, eye movement data, driving simulator, eye tracking, driving style recognition, theft prevention, Siamese network, squeeze-and-excitation blocks, fully convolutional network, human factors
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.