CoChat
Sign Up
Cited by 78Gait Recognition and AnalysisBiometric Identification and SecurityUser Authentication and Security Systems

Analysis of Spatio-Temporal Representations for Robust Footstep Recognition with Deep Residual Neural Networks

Open access
Costilla-Reyes, Omar, Vera-Rodriguez, Ruben, Scully, Patricia, Ozanyan, Krikor B.·2019·IEEE Transactions on Pattern Analysis and Machine Intelligence
DOI 10.1109/tpami.2018.2799847
Open this paper in CoChat — free accountOpen PDF

Abstract

Human footsteps can provide a unique behavioural pattern for robust biometric systems. We propose spatio-temporal footstep representations from floor-only sensor data in advanced computational models for automatic biometric verification. Our models deliver an artificial intelligence capable of effectively differentiating the fine-grained variability of footsteps between legitimate users (clients) and impostor users of the biometric system. The methodology is validated in the largest to date footstep database, containing nearly 20,000 footstep signals from more than 120 users. The database is organized by considering a large cohort of impostors and a small set of clients to verify the reliability of biometric systems. We provide experimental results in 3 critical data-driven security scenarios, according to the amount of footstep data made available for model training: at airports security checkpoints (smallest training set), workspace environments (medium training set) and home environments (largest training set). We report state-of-the-art footstep recognition rates with an optimal equal false acceptance and false rejection rate (equal error rate) of 0.7 percent an improvement ratio of 371 percent compared to previous state-of-the-art. We perform a feature analysis of deep residual neural networks showing effective clustering of client's footstep data and to provide insights of the feature learning process.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/tpami.2018.2799847

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Gait identification can also be performed by footprint, which considers the pressure and size of the area, but the obvious disadvantage is the cost of deploying the necessary equipment.

    “Gait identification can also be performed by distinct characteristics, e.g., the footprint [11], which considers the pressure and size of the area. The obvious disadvantage of such models is the cost of deploying the equipment necessary to acquire such features.”

    From Gait Recognition Based on Deep Learning: A Survey (2023) · cites this paper as [11]

    Section 2.1 (p. 3) describes biometric footstep recognition using spatial information from accumulated pressure images, and Section 3 (p. 4) describes the pressure-sensing floor mats used to acquire footsteps. These passages support identification from pressure and spatial footprint features, but the paper does not state that equipment deployment cost is a disadvantage. (LLM, conf=0.86)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

The citation lists haven’t been fetched yet

References 38 · Cited by 78

Research works printed but not linked: 33

  1. [2][2] M. W. Whittle, 'Clinical gait analysis: A review,' Human Movement Science , vol. 15, no. 3, pp. 369-387, 1996. -
  2. [4][4] D. Gafurov, 'A survey of biometric gait recognition: Approaches, security and challenges,' in Annual Norwegian computer science conference , 2007, pp. 19-21. -
  3. [5][5] Z. Wu, Y. Huang, L. Wang, X. Wang, and T. Tan, 'A comprehensive study on cross-view gait based human identification with deep cnns,' IEEE transactions on pattern analysis and machine intelligence , vol. 39, no. 2, pp. 209-226, 2017. -
  4. [6][6] J. E. Mason, I. Traor´ e, and I. Woungang, Machine Learning Techniques for Gait Biometric Recognition . Springer, 2016. -
  5. [7][7] P. C. Cattin, 'Biometric Authentication System Using Human Gait,' Ph.D. dissertation, Diss., ETH Zurich, Nr. 14603, 2002. -
  6. [8][8] L. Middleton, A. Buss, A. Bazin, and M. Nixon, 'A Floor Sensor System For Gait Recognition,' in Proc. of Fourth IEEE Workshop on Automatic Identification Advanced Technologies , 2005, pp. 171-176. -
  7. [9][9] J. P . Stevenson, S. L. Firebaugh, and H. Charles, 'Biometric identification from a floor based pvdf sensor array using hidden markov models,' in Proc. SAS , vol. 7, 2007. -
  8. [10][10] A. Mostayed, S. Kim, M. M. G. Mazumder, and S. J. Park, 'Foot step based person identification using histogram similarity and wavelet decomposition,' in Proc. of the 2nd International Conference on Information Security and Assurance . IEEE, 2008, pp. 307-311. -
  9. [11][11] R. Vera-Rodriguez, J. S. D. Mason, J. Fierrez, and J. Ortega-Garcia, 'Comparative analysis and fusion of spatiotemporal information for footstep recognition,' IEEE Transactions on Pattern Analysis and Machine Intelligence , vol. 35, no. 4, pp. 823-834, 2013. -
  10. [12][12] S. P. Moustakidis, J. B. Theocharis, and G. Giakas, 'Subject recognition based on ground reaction force measurements of gait signals,' IEEE Transactions on Systems, Man, and Cybernetics , vol. 38, no. 6, pp. 1476-1485, 2008. -
  11. [13][13] O. Costilla Reyes, P. Scully, and K. B. Ozanyan, 'Temporal Pattern Recognition in Gait Activities Recorded with a Footprint Imaging Sensor System,' IEEE Sensors Journal , vol. 16, no. c, dec 2016.
  12. [14][14] R. Vera-Rodriguez, N. Evans, and J. S. D. Mason, 'Footstep recog- nition,' Encyclopedia of Biometrics , pp. 693-700, 2015. -
  13. [15][15] Y. LeCun, Y. Bengio, and G. Hinton, 'Deep learning,' Nature , vol. 521, no. 7553, pp. 436-444, 2015. -
  14. [17][17] K. Simonyan and A. Zisserman, 'Two-stream convolutional networks for action recognition in videos,' in Advances in Neural Information Processing Systems , 2014, pp. 568-576. -
  15. [18][18] C. Feichtenhofer, A. Pinz, and R. Wildes, 'Spatiotemporal residual networks for video action recognition,' in Advances in Neural Information Processing Systems , 2016, pp. 3468-3476. -
  16. [19][19] J. Wu and J. M. Rehg, Ensemble Machine Learning . Springer, 2012. -
  17. [20][20] J. Yun, 'User identification using gait patterns on UbiFloorII,' Sensors , vol. 11, no. 3, pp. 2611-2639, 2011. -
  18. [21][21] G. Qian, J. Zhang, and A. Kidane, 'People identification using floor pressure sensing and analysis,' IEEE Sensors Journal , vol. 10, no. 9, pp. 1447-1460, 2010. -
  19. [22][22] O. Costilla-Reyes, R. Vera-Rodriguez, P. Scully, and K. B. Ozanyan, 'Spatial footstep recognition by convolutional neural networks for biometric applications,' in Proc. of 2016 IEEE Sensors conference . IEEE, 2016. -
  20. [23][23] S. Ji, W. Xu, M. Yang, and K. Yu, '3D Convolutional neural networks for human action recognition,' IEEE Transactions on Pattern Analysis and Machine Intelligence , vol. 35, no. 1, pp. 221231, 2013. -
  21. [24][24] A. Antoniou, Digital signal processing . McGraw-Hill Toronto, Canada, 2006. -
  22. [25][25] J. Bergstra, D. Yamins, and D. D. Cox, 'Hyperopt: A python library for optimizing the hyperparameters of machine learning algorithms,' in Proc. of the 12th Python in Science Conference , 2013, pp. 13-20. -
  23. [26][26] J. Bergstra, R. Bardenet, Y. Bengio, and B. K´ egl, 'Algorithms for Hyper-Parameter Optimization,' in Advances in Neural Information Processing Systems , 2011, pp. 2546-2554. -
  24. [27][27] S. Ioffe and C. Szegedy, 'Batch normalization: Accelerating deep network training by reducing internal covariate shift,' in Proc. of the 32nd International Conference on Machine Learning , 2015, pp. 448456. -
  25. [28][28] X. Glorot, A. Bordes, and Y. Bengio, 'Deep sparse rectifier neural networks,' in in Porc. of the 14th International Conference on Artificial Intelligence and Statistics , vol. 15, 2011, pp. 315-323. -
  26. [29][29] K. He, X. Zhang, S. Ren, and J. Sun, 'Identity mappings in deep residual networks,' in in Proc. of European Conference on Computer Vision . Springer, 2016, pp. 630-645. -
  27. [30][30] K. Simonyan and A. Zisserman, 'Very deep convolutional networks for large-scale image recognition,' arXiv preprint arXiv:1409.1556 , 2014. - · arXiv
  28. [33][33] A. K. Jain, K. Nandakumar, and A. a. Ross, 'Score Normalization in Multimodal Biometric Systems Multimodal Biometric Systems,' Pattern recognition , vol. 38, no. 12, pp. 4-5, 2005. -
  29. [35][35] M. Abadi, A. Agarwal et al. , 'Tensorflow: Large-scale machine learning on heterogeneous distributed systems,' arXiv preprint arXiv:1603.04467 , 2016. - · arXiv
  30. [36][36] F. Pedregosa, G. Varoquaux et al. , 'Scikit-learn: Machine learning in python,' Journal of Machine Learning Research , vol. 12, pp. 28252830, 2011. -
  31. [37][37] A. Anjos, L. El-Shafey et al. , 'Bob: a free signal processing and machine learning toolbox for researchers,' in Proc. of the 20th ACM international conference on Multimedia . ACM, 2012, pp. 1449-1452. -
  32. [38][38] A. Veit, M. Wilber, and S. Belongie, 'Residual networks are exponential ensembles of relatively shallow networks,' arXiv preprint arXiv:1605.06431 , vol. 1, 2016. - · arXiv
  33. [39][39] M. A. Weber, S. Zoubaa et al. , 'Diagnostic performance of spectroscopic and perfusion MRI for distinction of brain tumors,' Neurology , vol. 66, no. 12, pp. 1899-1906, 2006. <!-- image --> Omar Costilla-Reyes Received the M.Sc. degree in Electrical Engineering from the University of North Texas, Texas, U.S.A. in 2014. During his master studies, he was a research assistant in projects with funding from the National Science Foundation (NSF) and National Aeronautics and Space Administration (NASA). His M.Sc. dissertation was on dynamic indoor positioning systems using wireless sensor networks. He is currently a PhD candidate in Electrical and Electronics Engineering at the University of Manchester, U.K. He has published papers and has research interest in applications of machine learning for gait analysis in security and healthcare. He received the Best Student Paper Award in Optical Sensing applications at the 2015 IEEE Sensors Conference. He has won scholarships and awards for academic achievement including an academic scholarship for his master's and doctorate studies from the Mexican Science council (CONACyT). <!-- image --> Ruben Vera-Rodriguez Received the M.Sc. degree in telecommunications engineering from Universidad de Sevilla, Spain, in 2006, and the PhD degree in electrical and electronic engineering from Swansea University, U.K., in 2010. Since 2010, he has been with the Biometric Recognition Group, Universidad Autonoma de Madrid, Spain, first as the recipient of a Juan de la Cierva Post-Doctoral Fellowship from the Spanish Ministry of Innovation and Sciences, and as an Assistant Professor since 2013. His research interests include signal and image processing, pattern recognition, and biometrics, with emphasis on signature, face and gait verification and forensic applications of biometrics. Dr Vera-Rodriguez is actively involved in several National and European projects focused on biometrics. He was the recipient of the best paper award at the 4th International Summer School on Biometrics, Alghero, Italy, in 2007. <!-- image --> Patricia Scully Received the PhD degree in Engineering from the University of Liverpool, Liverpool, U.K., in 1992, and reached Reader position with Liverpool John Moores University, Liverpool, in 2000 before joining the University of Manchester as a Senior Lecturer/Associate Professor in Sensor Instrumentation with the University of Manchester in 2002. She is experienced in leading industrial and research council/government funded research projects at national and international levels, and has research interests in sensors and monitoring for industrial processes, including optical fibre technology and photonic materials for sensors and devices, ranging from functional chemically sensitive optical coatings, to laser inscribed photonic and conducting structures in transparent materials that affect the properties of light. <!-- image --> Krikor B. Ozanyan received the M.Sc. degree in engineering physics (semiconductors) and the PhD degree in solid-state physics in 1980 and 1989, respectively. He has more than 300 publications in the areas of devices, materials and systems for sensing and imaging. He is currently Director of Research at the School of EEE, at the University of Manchester, U.K. He is a Fellow of the Institute of Engineering and Technology, U.K, and the Institute of Physics, U.K. He was a Distinguished Lecturer of the IEEE Sensors Council in 2009 and 2010, and Guest Editor for the 10th Anniversary Issue of IEEE SENSORS JOURNAL in 2010, as well as the Special Issues on Sensors for Industrial Process Tomography in 2005 and THz Sensing: Materials, Devices and Systems in 2012. He is currently in his second term as Editor-in-Chief of the IEEE SENSORS JOURNAL and is General Co-Chair of the IEEE SENSORS 2017 Conference.

Authors
Costilla-Reyes, O., Vera-Rodriguez, R., Scully, P., & Ozanyan, K. B.
Affiliations
University of Manchester (GB)
ORCID
Venue
IEEE Transactions on Pattern Analysis and Machine Intelligence
Publisher
Institute of Electrical and Electronics Engineers (IEEE)
Published
2019-02-01
Volume
41
Issue
2
Pages
285-296
ISSN
  • 0162-8828 (ISSN-L)
  • 2160-9292
  • 1939-3539
DOI
10.1109/tpami.2018.2799847
OpenAlex
W2786185471
Cited by
78
References
38
Citations by year
2026: 2 · 2025: 7 · 2024: 5 · 2023: 23 · 2022: 17 · 2021: 10 · 2020: 6 · 2019: 7 · 2018: 1
FWCI
2.40
Type
journal article
Language
en
Open access
Licence
Locations
  • IEEE Transactions on Pattern Analysis and Machine Intelligence (publishedVersion)
  • PubMed (publishedVersion)
  • Biblos-e Archivo (Universidad Autónoma de Madrid) (submittedVersion, public-domain)
  • LA Referencia (Red Federada de Repositorios Institucionales de Publicaciones Científicas) (submittedVersion, other-oa)
  • Hispana (submittedVersion)
  • Research Explorer (The University of Manchester) (submittedVersion, cc-by)
Funders
  • Consejo Nacional de Ciencia y Tecnologia (467373)
  • MINECO FEDER (Cognimetrics TEC2015-70627-R)
  • EPSRC (IAA grant)
Topics
  • Physical Sciences › Engineering › Biomedical Engineering › Gait Recognition and Analysis
  • Physical Sciences › Computer Science › Signal Processing › Biometric Identification and Security
  • Physical Sciences › Computer Science › Information Systems › User Authentication and Security Systems
Keywords
gait biometrics, spatio-temporal representation, deep residual neural network, biometric authentication, impostor rejection, feature analysis, representation learning
MeSH terms
Deep Learning, Foot, Humans, Pattern Recognition, Automated, Pressure, Video Recording, Databases, Factual, Biometric Identification

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review