CoChat
Sign Up
Cited by 54Privacy-Preserving Technologies in DataSleep and Work-Related FatigueDomain Adaptation and Few-Shot Learning

Privacy-Preserving Federated Transfer Learning for Driver Drowsiness Detection

Open access
Zhang, Linlin, Saito, Hideo, Yang, Liang, Wu, Jiajie·2022·IEEE Access
DOI 10.1109/access.2022.3192454
Open this paper in CoChat — free accountOpen PDF

Abstract

The drowsiness affects the driver’s sensory, cognitive, and psychomotor abilities, which are necessary for safe driving. Drowsiness detection is a critical technique to avoid traffic accidents. The federated learning (FL) can solve the problem of insufficient driver facial data by utilizing different industrial entities’ data. However, in the FL system, the privacy information of the drivers might be leaked. In addition, reducing the communication costs and maintaining the model performance is also a challenge in industrial scenarios. In this work, we propose a federated transfer learning method with privacy-preserving protocol for driver drowsiness detection, named PFTL-DDD. We use fine-tuning transfer learning on the initial model of the drowsiness detection FL system. Furthermore, a CKKS-based privacy-preserving protocol is applied to preserve the drivers’ privacy data by encrypting the exchanged parameters. The experimental results show that the PFTL-DDD method is superior in terms of accuracy and efficiency as compared to the conventional federated learning on the NTHU-DDD dataset and YAWDD dataset. The theoretical analysis demonstrates that the proposed transfer learning method can reduce the communication cost of the system, and the CKKS-based security protocol can protect the personal privacy.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/access.2022.3192454

Cites

3 of 3 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 44

  1. V. Saini and R. Saini, "Driver drowsiness detection system and techniques: a review," International Journal of Computer Science and Information Technologies, vol. 5, no. 3, pp. 4245-4249, 2014.
  2. M. Ramzan, H. U. Khan, S. M. Awan, A. Ismail, M. Ilyas, and A. Mahmood, "A survey on state-of-the-art drowsiness detection techniques," IEEE Access, vol. 7, pp. 61904-61919, 2019.
  3. S. Kaplan, M. A. Guvensan, A. G. Yavuz, and Y. Karalurt, "Driver behavior analysis for safe driving: A survey," IEEE Transactions on Intelligent Transportation Systems, vol. 16, no. 6, pp. 3017-3032, 2015.
  4. A. L. A. Ramos, J. C. Erandio, D. H. T. Mangilaya, N. D. Carmen, E. M. Enteria, and L. J. Enriquez, "Driver drowsiness detection based on eye movement and yawning using facial landmark analysis.," International Journal of Simulation-Systems, Science & Technology, vol. 20, 2019.
  5. R. Ghoddoosian, M. Galib, and V. Athitsos, "A realistic dataset and baseline temporal model for early drowsiness detection," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 0-0, 2019.
  6. D. J. Fremont, E. Kim, Y. V. Pant, S. A. Seshia, A. Acharya, X. Bruso, P. Wells, S. Lemke, Q. Lu, and S. Mehta, "Formal scenario-based testing of autonomous vehicles: From simulation to the real world," in 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1-8, IEEE, 2020.
  7. C.-H. Weng, Y.-H. Lai, and S.-H. Lai, "Driver drowsiness detection via a hierarchical temporal deep belief network," in Asian Conference on Computer Vision, pp. 117-133, Springer, 2016.
  8. B. K. Savas and Y. Becerikli, "Real time driver fatigue detection system based on multi-task conNN," Ieee Access, vol. 8, pp. 12491- 12498, 2020.
  9. A. Zafar, C. Prehofer, and C.-H. Cheng, "Federated learning for driver status monitoring," in 2021 IEEE International Intelligent Transportation Systems Conference (ITSC), pp. 1463-1469, IEEE, 2021.
  10. Z. Zhang, L. Zhang, Q. Li, K. Wang, N. He, and T. Gao, "Privacy- enhanced momentum federated learning via differential privacy and chaotic system in industrial cyber-physical systems," ISA transactions, 2021.
  11. Z. Zhang, N. He, Q. Li, K. Wang, H. Gao, and T. Gao, "Detectpmfl: Privacy-preserving momentum federated learning considering unreliable industrial agents," IEEE Transactions on Industrial Informatics, 2022.
  12. E. Novikova, D. Fomichov, I. Kholod, and E. Filippov, "Analysis of privacy-enhancing technologies in open-source federated learning frameworks for driver activity recognition," Sensors, vol. 22, no. 8, p. 2983, 2022.
  13. J. Cabrero-Holgueras and S. Pastrana, "Sok: Privacy-preserving computation techniques for deep learning," Proceedings on Privacy Enhancing Technologies, vol. 2021, no. 4, pp. 139-162, 2021.
  14. Y. Rahulamathavan, "Privacy-preserving similarity calculation of speaker features using fully homomorphic encryption," arXiv preprint arXiv:2202.07994, 2022. · arXiv
  15. E. M. Shiriaev, A. S. Nazarov, N. N. Kycherov, and N. A. Sotikova, "Efficient implementation of the ckks scheme using a quadratic residue number system," in 2021 IEEE Conference of Russian Young Researchers in Electrical and Electronic Engineering (ElConRus), pp. 665-669, IEEE, 2021.
  16. C. Zhang, Y. Xie, H. Bai, B. Yu, W. Li, and Y. Gao, "A survey on federated learning," Knowledge-Based Systems, vol. 216, p. 106775, 2021.
  17. H. Zhu, H. Zhang, and Y. Jin, "From federated learning to federated neural architecture search: a survey," Complex & Intelligent Systems, vol. 7, no. 2, pp. 639-657, 2021.
  18. L. U. Khan, W. Saad, Z. Han, E. Hossain, and C. S. Hong, "Federated learning for internet of things: Recent advances, taxonomy, and open challenges," IEEE Communications Surveys & Tutorials, 2021.
  19. F. Zhuang, Z. Qi, K. Duan, D. Xi, Y. Zhu, H. Zhu, H. Xiong, and Q. He, "A comprehensive survey on transfer learning," Proceedings of the IEEE, vol. 109, no. 1, pp. 43-76, 2020.
  20. J. Yosinski, J. Clune, Y. Bengio, and H. Lipson, "How transferable are features in deep neural networks?," Advances in neural information processing systems, vol. 27, 2014.
  21. G. Vrbancic and V. Podgorelec, "Transfer learning with adaptive fine- tuning," IEEE Access, vol. 8, pp. 196197-196211, 2020.
  22. B. Q. Huynh, H. Li, and M. L. Giger, "Digital mammographic tumor classification using transfer learning from deep convolutional neural networks," Journal of Medical Imaging, vol. 3, no. 3, p. 034501, 2016.
  23. C. Wang, D. Chen, J. Chen, X. Lai, and T. He, "Deep regression adaptation networks with model-based transfer learning for dynamic load identification in the frequency domain," Engineering Applications of Artificial Intelligence, vol. 102, p. 104244, 2021.
  24. L. Zhu, S. Ö. Ark, Y. Yang, and T. Pfister, "Learning to transfer learn: Reinforcement learning-based selection for adaptive transfer learning," in European Conference on Computer Vision, pp. 342-358, Springer, 2020.
  25. Y. Cui, Y. Song, C. Sun, A. Howard, and S. Belongie, "Large scale fine-grained categorization and domain-specific transfer learning," in Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4109-4118, 2018.
  26. Hao M, Li H, Xu G, Liu S, Yang H (2019) Towards efficient and privacy-preserving federated deep learning. In: ICC 2019-2019 IEEE International Conference on Communications (ICC), IEEE, 2019, pp 1-6
  27. Zhou C, Fu A, Yu S, Yang W, Wang H, Zhang Y (2020) Privacy- preserving federated learning in fog computing. IEEE Internet Things J 7:10782-10793
  28. M. Sumathi and S. Sangeetha, "A group-key-based sensitive attribute protection in cloud storage using modified random fibonacci cryptography," Complex & Intelligent Systems, vol. 7, no. 4, pp. 1733-1747, 2021.
  29. W. Ou, J. Zeng, Z. Guo, W. Yan, D. Liu, and S. Fuentes, "A homomorphic-encryption-based vertical federated learning scheme for rick management," Computer Science and Information Systems, vol. 17, no. 3, pp. 819-834, 2020.
  30. H. Chen, W. Dai, M. Kim, and Y. Song, "Efficient multi-key homomorphic encryption with packed ciphertexts with application to oblivious neural network inference," in Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security, pp. 395-412, 2019.
  31. J. Bai, W. Yu, Z. Xiao, V. Havyarimana, A. C. Regan, H. Jiang, and L. Jiao, "Two-stream spatial-temporal graph convolutional networks for driver drowsiness detection," IEEE Transactions on Cybernetics, 2021.
  32. Q. Zhuang, Z. Kehua, J. Wang, and Q. Chen, "Driver fatigue detection method based on eye states with pupil and iris segmentation," Ieee Access, vol. 8, pp. 173440-173449, 2020.
  33. J. Lyu, Z. Yuan, and D. Chen, "Long-term multi-granularity deep framework for driver drowsiness detection," arXiv preprint arXiv:1801.02325, 2018. · arXiv
  34. T. H. Vu, A. Dang, and J.-C. Wang, "A deep neural network for real- time driver drowsiness detection," IEICE TRANSACTIONS on Information and Systems, vol. 102, no. 12, pp. 2637-2641, 2019.
  35. S. Park, F. Pan, S. Kang, and C. D. Yoo, "Driver drowsiness detection system based on feature representation learning using various deep networks," in Asian Conference on Computer Vision, pp. 154-164, Springer, 2016.
  36. S. Abtahi, M. Omidyeganeh, S. Shirmohammadi, and B. Hariri, "Yawdd: A yawning detection dataset," in Proceedings of the 5th ACM multimedia systems conference, pp. 24-28, 2014.
  37. R. M. Salman, M. Rashid, R. Roy, M. M. Ahsan, and Z. Siddique, "Driver drowsiness detection using ensemble convolutional neural networks on yawdd," arXiv preprint arXiv:2112.10298, 2021. · arXiv
  38. S. Junaedi and H. Akbar, "Driver drowsiness detection based on face feature and perclos," in Journal of Physics: Conference Series, vol. 1090, p. 012037, IOP Publishing, 2018.
  39. S. Saurav, S. Mathur, I. Sang, S. S. Prasad, and S. Singh, "Yawn detection for driver's drowsiness prediction using bi-directional lstm with cnn features," in International Conference on Intelligent Human Computer Interaction, pp. 189-200, Springer, 2019.
  40. N. N. Pandey and N. B. Muppalaneni, "[]," in 2021 International Conference on Artificial Intelligence and Smart Systems (ICAIS), pp. 1182-1187, IEEE, 2021.
  41. M. S. Laxshmi, L. Chandana, et al., "An enhanced driver drowsiness detection system using transfer learning," in 2021 5th International Conference on Electronics, Communication and Aerospace Technology (ICECA), pp. 1671-1678, IEEE, 2021.
  42. C. Zhao, Z. Gao, Q. Wang, K. Xiao, Z. Mo, and M. J. Deen, "Fedsup: A communication-efficient federated learning fatigue driving behaviors supervision framework," arXiv preprint arXiv:2104.12086, 2021. · arXiv
  43. Wang S, Tuor T, Salonidis T, et al. Adaptive Federated Learning in Resource Constrained Edge Computing Systems[J]. IEEE Journal on Selected Areas in Communications, 2019:1-1.
  44. Liu W, Chen L, Chen Y, et al. Accelerating federated learning via momentum gradient descent[J]. IEEE Transactions on Parallel and Distributed Systems, 2020, 31(8): 1754-1766.

Cited by

Sorted by citations

Authors
Zhang, L., Saito, H., Yang, L., & Wu, J.
Affiliations
  • Keio University (JP)
  • Graduate School of Science and Technology, Keio University, Tokyo, Japan
  • China Auto Information Technology Co., Ltd., Tianjin, China
  • China Automotive Technology and Research Center (CN)
ORCID
Venue
IEEE Access
Publisher
Institute of Electrical and Electronics Engineers (IEEE)
Published
2022
Volume
10
Pages
80565-80574
ISSN
2169-3536 (ISSN-L)
DOI
10.1109/access.2022.3192454
OpenAlex
W4286361358
Cited by
54
References
3
Citations by year
2026: 6 · 2025: 17 · 2024: 16 · 2023: 15
FWCI
3.95
Type
journal article
Language
en
Open access
Licence
Locations
  • IEEE Access (publishedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Sleep and Work-Related Fatigue
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
Keywords
driver drowsiness detection, federated transfer learning, privacy preservation, CKKS homomorphic encryption, communication cost reduction, fine-tuning, road safety

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review