CoChat
Sign Up
PreprintCited by 305Gait Recognition and AnalysisUser Authentication and Security SystemsBiometric Identification and Security

Deep Learning-Based Gait Recognition Using Smartphones in the Wild

Open access
Zou, Qin, Wang, Yanling, Wang, Qian, Zhao, Yi, Li, Qingquan·2018·arXiv
DOI 10.48550/arxiv.1811.00338arXiv 1811.00338
Open this paper in CoChat — free accountOpen PDF

Abstract

Compared to other biometrics, gait is difficult to conceal and has the advantage of being unobtrusive. Inertial sensors, such as accelerometers and gyroscopes, are often used to capture gait dynamics. These inertial sensors are commonly integrated into smartphones and are widely used by the average person, which makes gait data convenient and inexpensive to collect. In this paper, we study gait recognition using smartphones in the wild. In contrast to traditional methods, which often require a person to walk along a specified road and/or at a normal walking speed, the proposed method collects inertial gait data under unconstrained conditions without knowing when, where, and how the user walks. To obtain good person identification and authentication performance, deep-learning techniques are presented to learn and model the gait biometrics based on walking data. Specifically, a hybrid deep neural network is proposed for robust gait feature representation, where features in the space and time domains are successively abstracted by a convolutional neural network and a recurrent neural network. In the experiments, two datasets collected by smartphones for a total of 118 subjects are used for evaluations. The experiments show that the proposed method achieves higher than 93.5\% and 93.7\% accuracies in person identification and authentication, respectively.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.1811.00338

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Tran et al. proposed an IMU-based gait recognition approach using LSTMs, achieving state-of-the-art performance for verification and identification on whuGAIT and OU-ISIR.

    “Recently, Tran et al. [103] proposed an Inertial Measurement Units (IMUs)-based gait recognition approach. The authors employed LSTMs to exploit the temporal information on video sequences, thus extracting hidden patterns inside such sequences. Experiments conducted over whuGAIT [141] and OU-ISIR [59] provided state-of-the-art performance for both verification and identification tasks.”

    From Gait Recognition Based on Deep Learning: A Survey (2023) · cites this paper as [141]

    The whuGAIT Datasets and User-Identification Performance sections report inertial-sensor gait data and LSTM experiments on whuGAIT and OU-ISIR; LSTM performs best among the compared methods for OU-ISIR identification. But this is Zou et al.’s paper, not Tran et al.’s, and its LSTM processes inertial time series rather than video sequences. The User-Authentication Performance section reports that CNN+LSTM, not LSTM alone, performs best on OU-ISIR, so the claim of state-of-the-art performance for both tasks is overstated. (LLM, conf=0.90)

    Experimental — accuracy not yet measured

  • The whuGAIT Datasets were released in 2018 by the Wuhan University and made available along with the source code and pre-trained models to replicate the paper results.

    “The whuGAIT Datasets [140] were released in 2018 by the Wuhan University and made available along with the source code and pre-trained models to replicate the paper results 9 .”

    From Gait Recognition Based on Deep Learning: A Survey (2023) · cites this paper as [140]

    The whuGAIT Datasets section describes the datasets, and the Conclusion says the authors shared the datasets, code, and trained model. Our copy does not establish that Wuhan University released them specifically in 2018 or that the shared materials replicate the paper’s results. (LLM, conf=0.85)

    Experimental — accuracy not yet measured

  • Other models combining accelerometer and gyroscope data for gait recognition reached recognition rates above 91%.

    “Other models combined specific information, such as data obtained from accelerometer and gyroscope, for gait recognition through deep learning-based approaches, reaching recognition rates above 91% [141].”

    From Gait Recognition Based on Deep Learning: A Survey (2023) · cites this paper as [141]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

The citation lists haven’t been fetched yet

References 98 · Cited by 305

Research works printed but not linked: 84

  1. S. Kim, S. Yu, K. Kim, Y. Ban, and S. Lee, "Face liveness detection using variable focusing," in ICB, 2013, pp. 1-6.
  2. M. Nixon, T. Tan, and R. Chellapppa, "Human identification based on gait," Springer Science + Business Media Inc., 2006, ch. 1.
  3. J. Zhang, J. Pu, C. Chen, and R. Fleischer, "Low-resolution gait recognition," IEEE Trans. on SMC, Part B: Cybernetics, vol. 40, no. 4, pp. 986-996, 2010.
  4. M. Ding and G. Fan, "Multilayer joint gait-pose manifolds for human gait motion modeling," IEEE Transactions on Cybernetics, vol. 45, no. 11, pp. 2413-2424, 2015.
  5. S. Minaee, A. Abdolrashidi, H. Su, M. Bennamoun, and D. Zhang, "Biometric recognition using deep learning: A survey," arXiv preprint arXiv:1912.00271, 2019. · arXiv
  6. Y. Makihara, R. Sagawa, Y. Mukaigawa, T. Echigo, and Y. Yagi, "Gait recognition using a view transformation model in the frequency domain," in ECCV, 2006, pp. 151-163.
  7. G. Lin, A. Milan, C. Shen, and I. Reid, "Refinenet: Multi-path refinement networks for high-resolution semantic segmentation," in CVPR, 2017, pp. 1925-1934.
  8. K. He, G. Gkioxari, P. Dollár, and R. Girshick, "Mask r-cnn," in ICCV, 2017, pp. 2961-2969.
  9. Z. Zhang, L. Tran, X. Yin, Y. Atoum, X. Liu, J. Wan, and N. Wang, "Gait recognition via disentangled representation learning," in CVPR, 2019, pp. 4710-4719.
  10. R. Liao, C. Cao, E. B. Garcia, S. Yu, and Y. Huang, "Pose-based temporal-spatial network (ptsn) for gait recognition with carrying and clothing variations," in CCBR, 2017, pp. 474-483.
  11. W. An, R. Liao, S. Yu, Y. Huang, and P. C. Yuen, "Improving gait recognition with 3d pose estimation," in CCBR, 2018, pp. 137-147.
  12. Y. Zhang, Y. Huang, L. Wang, and S. Yu, "A comprehensive study on gait biometrics using a joint cnn-based method," Pattern Recognition, vol. 93, pp. 228-236, 2019.
  13. Q. Zou, L. Ni, Q. Wang, Q. Li, and S. Wang, "Robust gait recognition by integrating inertial and rgbd sensors," IEEE Transactions on Cybernetics, vol. 48, no. 4, pp. 1136-1150, 2018.
  14. D. Gafurov and E. Snekkenes, "Gait recognition using wearable motion recording sensors," EURASIP Journal on Advances in Signal Processing, vol. 2009, p. 7, 2009.
  15. F. Xu, C. Bhagavatula, A. Jaech, U. Prasad, and M. Savvides, "Gait- ID on the move: Pace independent human identification using cell phone accelerometer dynamics," in BTAS, 2012, pp. 8-15.
  16. N. Trung, Y. Makihara, H. Nagahara, Y. Mukaigawa, and Y. Yagi, "Performance evaluation of gait recognition using the largest inertial sensor-based gait database," in IAPR ICB, 2012, pp. 360-366.
  17. Y. Zhang, G. Pan, K. Jia, M. Lu, Y. Wang, and Z. Wu, "Accelerometer- based gait recognition by sparse representation of signature points with clusters," IEEE Transactions on Cybernetics, vol. 45, no. 9, pp. 1864-1875, 2015.
  18. S. Sprager and M. B. Juric, "An efficient hos-based gait authentication of accelerometer data," IEEE T-IFS, vol. 10, no. 7, pp. 1486-1498, 2015.
  19. C. Shen, Y. Chen, X. Guan, and R. A. Maxion, "Pattern-growth based mining mouse-interaction behavior for an active user authentication system," IEEE Transactions on Dependable and Secure Computing, vol. 17, no. 2, pp. 335-349, 2020.
  20. C. Shen, Y. Li, Y. Chen, X. Guan, and R. A. Maxion, "Performance analysis of multi-motion sensor behavior for active smartphone au- thentication," IEEE T-IFS, vol. 13, no. 1, pp. 48-62, 2018.
  21. J. Kwapisz, G. Weiss, and S. Moore, "Cell phone-based biometric identification," in BTAS, 2010, pp. 1-7.
  22. B. Sun, Y. Wang, and J. Banda, "Gait characteristic analysis and identification based on the iphones accelerometer and gyrometer," Sensors, vol. 14, no. 9, pp. 17 037-17 054, 2014.
  23. P. Fernandez-Lopez, J. Sanchez-Casanova, P. Tirado-Martín, and J. Liu-Jimenez, "Optimizing resources on smartphone gait recogni- tion," in ICB, 2017, pp. 31-36.
  24. A. Graves, A.-r. Mohamed, and G. Hinton, "Speech recognition with deep recurrent neural networks," in ICASSP, 2013, pp. 6645-6649.
  25. M. Zhou, Q. Wang, J. Yang, Q. Li, P. Jiang, Y. Chen, and Z. Wang, "Stealing your android patterns via acoustic signals," IEEE T-MC, 2019.
  26. J. Donahue, L. Anne Hendricks, S. Guadarrama, M. Rohrbach, S. Venugopalan, K. Saenko, and T. Darrell, "Long-term recurrent convolutional networks for visual recognition and description," in CVPR, 2015, pp. 2625-2634.
  27. Q. Zou, Z. Zhang, Q. Li, X. Qi, Q. Wang, and S. Wang, "Deepcrack: Learning hierarchical convolutional features for crack detection," IEEE Transactions on Image Processing, vol. 28, no. 3, pp. 1498- 1512, 2019.
  28. L. Chen, W. Zhan, W. Tian, Y. He, and Q. Zou, "Deep integration: A multi-label architecture for road scene recognition," IEEE Trans- actions on Image Processing, vol. 28, no. 10, pp. 4883-4898, 2019.
  29. Z. Shao, L. Wang, Z. Wang, W. Du, and W. Wu, "Saliency-aware convolution neural network for ship detection in surveillance video," IEEE T-SCVT, 2019.
  30. Z. Zhang, Q. Zou, Y. Lin, L. Chen, and S. Wang, "Improved deep hashing with soft pairwise similarity for multi-label image retrieval," IEEE Transactions on Multimedia, vol. 22, no. 2, pp. 540-553, 2020.
  31. Z. Shao, Y. Pan, C. Diao, and J. Cai, "Cloud detection in remote sensing images based on multiscale features-convolutional neural network," IEEE Transactions on Geoscience and Remote Sensing, vol. 57, no. 6, pp. 4062-4076, 2019.
  32. Q. Zou, H. Jiang, Q. Dai, Y. Yue, L. Chen, and Q. Wang, "Robust lane detection from continuous driving scenes using deep neural networks," IEEE Transactions on Vehicular Technology, vol. 69, no. 1, pp. 41-54, 2020.
  33. M. D. Addlesee, A. Jones, F. Livesey, and F. Samaria, "The orl active floor [sensor system]," IEEE Personal Communications, vol. 4, no. 5, pp. 35-41, 1997.
  34. B. Huang, M. Chen, P. Huang, and Y. Xu, "Gait modeling for human identification," in ICRA, 2007, pp. 4833-4838.
  35. T. Plötz, N. Y. Hammerla, and P. Olivier, "Feature learning for activity recognition in ubiquitous computing," in IJCAI, 2011.
  36. J.-Y. Yang, J.-S. Wang, and Y.-P. Chen, "Using acceleration measure- ments for activity recognition: An effective learning algorithm for constructing neural classifiers," Pattern Recognition Letters, vol. 29, no. 16, pp. 2213-2220, 2008.
  37. J. A. Ward, P. Lukowicz, G. Troster, and T. E. Starner, "Activity recognition of assembly tasks using body-worn microphones and accelerometers," IEEE T-PAMI, vol. 28, no. 10, pp. 1553-1567, 2006.
  38. B. Shrestha, D. Ma, Y. Zhu, H. Li, and N. Saxena, "Tap-wave-rub: Lightweight human interaction approach to curb emerging smartphone malware," IEEE T-IFS, vol. 10, no. 11, pp. 2270-2283, 2015.
  39. R. E. Mayagoitia, A. V. Nene, and P. H. Veltink, "Accelerometer and rate gyroscope measurement of kinematics: an inexpensive alternative to optical motion analysis systems," Journal of biomechanics, vol. 35, no. 4, pp. 537-542, 2002.
  40. H. Zhao, Z. Wang, S. Qiu, Y. Shen, L. Zhang, K. Tang, and G. Fortino, "Heading drift reduction for foot-mounted inertial navigation system via multi-sensor fusion and dual-gait analysis," IEEE Sensors Journal, 2018.
  41. J. Frank, S. Mannor, and D. Precup, "Activity and gait recognition with time-delay embeddings." in AAAI, 2010.
  42. J. Mäntyjärvi, M. Lindholm, E. Vildjiounaite, S.-M. Mäkelä, and H. Ailisto, "Identifying users of portable devices from gait pattern with accelerometers." in ICASSP, 2005, pp. 973-976.
  43. D. Gafurov, K. Helkala, and T. Søndrol, "Biometric gait authentication using accelerometer sensor," Journal of Computers, vol. 1, no. 7, pp. 51-59, 2006.
  44. D. Gafurov, E. Snekkenes, and P. Bours, "Spoof attacks on gait authentication system," IEEE T-IFS, vol. 2, no. 3, pp. 491-502, 2007.
  45. --, "Gait authentication and identification using wearable ac- celerometer sensor," in AIATW, 2007, pp. 220-225.
  46. R. Liu, J. Zhou, M. Liu, and X. Hou, "A wearable acceleration sensor system for gait recognition," in ICIEA, 2007, pp. 2654-2659.
  47. G. Trivino, A. Alvarez-Alvarez, and G. Bailador, "Application of the computational theory of perceptions to human gait pattern recogni- tion," Pattern Recognition, vol. 43, no. 7, pp. 2572-2581, 2010.
  48. S. Sprager and M. B. Juric, "Inertial sensor-based gait recognition: a review," Sensors, vol. 15, no. 9, pp. 22 089-22 127, 2015.
  49. T. T. Ngo, Y. Makihara, H. Nagahara, Y. Mukaigawa, and Y. Yagi, "Orientation-compensative signal registration for owner authentication using an accelerometer," IEICE Transactions on Information and Systems, vol. 97, no. 3, pp. 541-553, 2014.
  50. T. Ngo, Y. Makihara, H. Nagahara, Y. Mukaigawa, and Y. Yagi, "The largest inertial sensor-based gait database and performance evaluation of gait-based personal authentication," Pattern Recognition, vol. 47, no. 1, pp. 228-237, 2014.
  51. Y. Zhong and Y. Deng, "Sensor orientation invariant mobile gait biometrics," in IJCB, 2014, pp. 1-8.
  52. L. Wu, J. Yang, M. Zhou, Y. Chen, and Q. Wang, "LVID: a multimodal biometrics authentication system on smartphones," IEEE T-IFS, 2019.
  53. A. H. Johnston and G. M. Weiss, "Smartwatch-based biometric gait recognition," in BTAS, 2015, pp. 1-6.
  54. S. Yao, S. Hu, Y. Zhao, A. Zhang, and T. Abdelzaher, "Deepsense: A unified deep learning framework for time-series mobile sensing data processing," in WWW, 2017, pp. 351-360.
  55. H. Abujrida, E. Agu, and K. Pahlavan, "Smartphone-based gait assessment to infer parkinson's disease severity using crowdsourced data," in HI-POCT, 2017, pp. 208-211.
  56. J. Juen, Q. Cheng, V. Prieto-Centurion, J. A. Krishnan, and B. Schatz, "Health monitors for chronic disease by gait analysis with mobile phones," Telemedicine and e-Health, vol. 20, no. 11, pp. 1035-1041, 2014.
  57. Y. Ren, Y. Chen, M. C. Chuah, and J. Yang, "User verification leveraging gait recognition for smartphone enabled mobile healthcare systems," IEEE T-MC, vol. 14, no. 9, pp. 1961-1974, 2015.
  58. M. Muaaz and R. Mayrhofer, "Smartphone-based gait recognition: From authentication to imitation," IEEE T-MC, vol. 16, no. 11, pp. 3209-3221, 2017.
  59. A. Ferreira, G. Santos, A. Rocha, and S. Goldenstein, "User-centric coordinates for applications leveraging 3-axis accelerometer data," IEEE Sensors Journal, vol. 17, no. 16, pp. 5231-5243, 2017.
  60. J. Lu, G. Wang, and P. Moulin, "Human identity and gender recog- nition from gait sequences with arbitrary walking directions," IEEE T-IFS, vol. 9, no. 1, pp. 51-61, 2014.
  61. P. Chattopadhyay, S. Sural, and J. Mukherjee, "Frontal gait recognition from incomplete sequences using rgb-d camera," IEEE T-IFS, vol. 9, no. 11, pp. 1843-1856, 2014.
  62. K. Ren, Q. Wang, C. Wang, Z. Qin, and X. Lin, "The security of autonomous driving: Threats, defences, and future directions," Proceedings of the IEEE, pp. 1-14, 2019.
  63. L. Zhao, Q. Wang, Q. Zou, Y. Zhang, and Y. Chen, "Privacy- preserving collaborative deep learning with unreliable participants," IEEE T-IFS, vol. 15, no. 1, pp. 1486-1500, 2020.
  64. S. Ji, W. Xu, M. Yang, and K. Yu, "3d convolutional neural networks for human action recognition," IEEE T-PAMI, vol. 35, no. 1, pp. 221- 231, 2013.
  65. J. Man and B. Bhanu, "Individual recognition using gait energy image," IEEE T-PAMI, vol. 28, no. 2, pp. 316-322, 2006.
  66. L. Lee and W. E. L. Grimson, "Gait analysis for recognition and classification," in AFGR, 2002, pp. 155-162.
  67. C. Nakajima, M. Pontil, B. Heisele, and T. Poggio, "Full-body person recognition system," Pattern Recognition, vol. 36, no. 9, pp. 1997- 2006, 2003.
  68. Z. Wu, Y. Huang, L. Wang, X. Wang, and T. Tan, "A comprehensive study on cross-view gait based human identification with deep cnns," IEEE T-PAMI, vol. 39, no. 2, pp. 209-226, 2017.
  69. M. Zeng, L. T. Nguyen, B. Yu, O. J. Mengshoel, J. Zhu, P. Wu, and J. Zhang, "Convolutional neural networks for human activity recognition using mobile sensors," in MCAS, 2014, pp. 197-205.
  70. E. P. Ijjina and C. K. Mohan, "One-shot periodic activity recognition using convolutional neural networks," in ICMLA, 2014, pp. 388-391.
  71. M. Alotaibi and A. Mahmood, "Improved gait recognition based on specialized deep convolutional neural network," CVIU, vol. 164, pp. 103-110, 2017.
  72. N. Takemura, Y. Makihara, D. Muramatsu, T. Echigo, and Y. Yagi, "On input/output architectures for convolutional neural network-based cross-view gait recognition," IEEE Transactions on Circuits and Systems for Video Technology, 2017.
  73. W. Yuan and L. Zhang, "Gait classification and identity authentication using cnn," in Asian Simulation Conference, 2018, pp. 119-128.
  74. Y. Zhao and S. Zhou, "Wearable device-based gait recognition using angle embedded gait dynamic images and a convolutional neural network," Sensors, vol. 17, no. 3, p. 478, 2017.
  75. M. Gadaleta and M. Rossi, "Idnet: Smartphone-based gait recognition with convolutional neural networks," Pattern Recognition, vol. 74, pp. 25-37, 2018.
  76. C. Li, X. Min, S. Sun, W. Lin, and Z. Tang, "Deepgait: a learning deep convolutional representation for view-invariant gait recognition using joint bayesian," Applied Sciences, vol. 7, no. 3, p. 210, 2017.
  77. F. M. Castro, M. J. Marín-Jiménez, N. Guil, and N. P. de la Blanca, "Automatic learning of gait signatures for people identification," in ICANN, 2017, pp. 257-270.
  78. S. Hochreiter and J. Schmidhuber, "Long short-term memory," Neural computation, vol. 9, no. 8, pp. 1735-1780, 1997.
  79. L. Wu, C. Shen, and A. Hengel, "Convolutional lstm networks for video-based person re-identification," arXiv:1606.01609, 2016. · arXiv
  80. N. Y. Hammerla, S. Halloran, and T. Ploetz, "Deep, convolutional, and recurrent models for human activity recognition using wearables," in IJCAI, 2016.
  81. S. Yu, H. Chen, E. B. G. Reyes, and P. Norman, "Gaitgan: invariant gait feature extraction using generative adversarial networks," in CVPRW, 2017, pp. 30-37.
  82. C. Zhang, W. Liu, H. Ma, and H. Fu, "Siamese neural network based gait recognition for human identification," in ICASSP, 2016.
  83. L. v. d. Maaten and G. Hinton, "Visualizing data using t-sne," Journal of machine learning research, vol. 9, pp. 2579-2605, 2008.
  84. H. Iwama, M. Okumura, Y. Makihara, and Y. Yagi, "The ou-isir gait database comprising the large population dataset and performance evaluation of gait recognition," IEEE T-IFS, vol. 7, no. 5, pp. 1511- 1521, 2012.

Authors
Zou, Q., Wang, Y., Wang, Q., Zhao, Y., & Li, Q.
Affiliations
  • Wuhan University (CN)
  • Shenzhen University (CN)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2018
Volume
15
Pages
3197-3212
ISSN
1556-6013 (ISSN-L)
DOI
10.48550/arxiv.1811.00338
arXiv
1811.00338
OpenAlex
W3014370958
Cited by
305
References
98
Citations by year
2026: 20 · 2025: 38 · 2024: 58 · 2023: 52 · 2022: 62 · 2021: 44 · 2020: 27 · 2019: 4
FWCI
14.53
Type
journal article
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • IEEE Transactions on Information Forensics and Security (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
  • National Natural Science Foundation of China (91546106, U1934215, 61872277, U1636219, 61822207)
  • Natural Science Foundation of Hubei Province (2018CFB482)
  • State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing (16S01)
Topics
  • Physical Sciences › Engineering › Biomedical Engineering › Gait Recognition and Analysis
  • Physical Sciences › Computer Science › Information Systems › User Authentication and Security Systems
  • Physical Sciences › Computer Science › Signal Processing › Biometric Identification and Security
Keywords
gait recognition, smartphone inertial sensors, personal identification, user authentication, hybrid deep neural network, convolutional neural networks, recurrent neural networks, gait biometrics

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review