Explaining Machine Learning Models for Clinical Gait Analysis
Open accessThe DOI matches its Crossref record; 75 of the 75 references it cites were found in a registry or an index.
Abstract
Machine Learning (ML) is increasingly used to support decision-making in the healthcare sector. While ML approaches provide promising results with regard to their classification performance, most share a central limitation, their black-box character. This article investigates the usefulness of Explainable Artificial Intelligence (XAI) methods to increase transparency in automated clinical gait classification based on time series. For this purpose, predictions of state-of-the-art classification methods are explained with a XAI method called Layer-wise Relevance Propagation (LRP). Our main contribution is an approach that explains class-specific characteristics learned by ML models that are trained for gait classification. We investigate several gait classification tasks and employ different classification methods, i.e., Convolutional Neural Network, Support Vector Machine, and Multi-layer Perceptron. We propose to evaluate the obtained explanations with two complementary approaches: a statistical analysis of the underlying data using Statistical Parametric Mapping and a qualitative evaluation by two clinical experts. A gait dataset comprising ground reaction force measurements from 132 patients with different lower-body gait disorders and 62 healthy controls is utilized. Our experiments show that explanations obtained by LRP exhibit promising statistical properties concerning inter-class discriminativity and are also in line with clinically relevant biomechanical gait characteristics.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Building on earlier efforts that compared different model outputs [70] or validated the intuitiveness of visual explanations (like those from LRP) [48,49], recent studies have begun to experimentally test the impact of different explanation strategies on user performance and reliance.”
From Explainable AI for Clinical Decision Support Systems: Literature Review, Key Gaps, and Research Synthesis (2025) · cites this paper as [49]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
(2008) · Journal of Machine Learning Research · Cited by 33,051
A Simple Sequentially Rejective Multiple Test Procedure
(1979) · Scandinavian Journal of Statistics · doi:10.2307/4615733 · Cited by 21,526
Dermatologist-level classification of skin cancer with deep neural networks
Esteva, A., Kuprel, B., Novoa, R. A., Ko, J., Swetter, S. M., Blau, H. M., & Thrun, S. (2017) · Nature · doi:10.1038/nature21056 · Cited by 14,362
High-performance medicine: the convergence of human and artificial intelligence
Topol, E. J. (2019) · Nature Medicine · doi:10.1038/s41591-018-0300-7 · Cited by 10,450 · cited 1×
“Therefore, it is not surprising that ML approaches are currently becoming popular in the healthcare sector [73].”
BackgroundPeeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)
Adadi, A., & Berrada, M. (2018) · IEEE Access · doi:10.1109/access.2018.2870052 · Cited by 6,294 · cited 2×
“While ML approaches yield promising results regarding classification performance, most share a central limitation, which is their black-box character [1].”
BackgroundBiomechanics and Motor Control of Human Movement
Winter, D. A. (2009) · Wiley eBooks · doi:10.1002/9780470549148 · Cited by 6,066 · cited 3×
“The employed dataset contains bilateral three-dimensional ground reaction force (GRF) recordings of patients and healthy controls walking unassisted at self-selected walking speed on an approximately 10 m walkway with two centrally-embedded force plates (Kistler, Type 9281B12, Winterthur, CH).”
MethodDeep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps
Simonyan, K., Vedaldi, A., & Zisserman, A. (2013) · arXiv · doi:10.48550/arxiv.1312.6034 · Cited by 5,370 · cited 5×
“[51] proposed a method for generating a representative visualization for a specific class that was learned by a CNN.”
MethodInfluentialMeta-Analytic Procedures for Social Research
Rosenthal, R. (1991) · doi:10.4135/9781412984997 · Cited by 4,635 · cited 2×
“Additionally, we computed the effect size by transforming the resulting t-values to Pearson’s correlation coefficient r using the definition by Rosenthal [56].”
MethodOn Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation
Bach, S., Binder, A., Montavon, G., Klauschen, F., Müller, K.-R., & Samek, W. (2015) · PLOS ONE · doi:10.1371/journal.pone.0130140 · Cited by 4,524 · cited 4×
“For example, the LRPε rule [7] is defined as:”
BackgroundMethodInfluentialSelf-citationInternational evaluation of an AI system for breast cancer screening
McKinney, S. M., Sieniek, M., Godbole, V., Godwin, J., Antropova, N., Ashrafian, H., Back, T., Chesus, M., Corrado, G. S., Darzi, A., et al. (2020) · Nature · doi:10.1038/s41586-019-1799-6 · Cited by 3,287 · cited 2×
“Impressively, there are certain medical tasks, such as the detection of skin or breast cancer, that ML approaches have already been able to solve more efficiently and effectively than humans [16, 21, 42].”
Background
Research works printed but not linked: 10
- 10.5555/3327546.3327621 · doi:10.5555/3327546.3327621
- David Balduzzi, Marcus Frean, Lennox Leary, J. P. Lewis, Kurt Wan-Duo Ma, and Brian McWilliams. 2017. The shattered gradients problem: If resnets are the answer, then what is the question? In Proceedings of the International Conference on Machine Learning (ICML). PMLR, 342–350.
- 10.5555/3203489 · doi:10.5555/3203489
- European Union. 2016. Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (General Data Protection Regulation). Offic. J. Eur. Union L 119 (2016), 1–88. Retrieved from https://eur-lex.europa.eu/eli/reg/2016/679/oj.
- Sebastian Lapuschkin, Alexander Binder, Grégoire Montavon, Klaus-Robert Müller, and Wojciech Samek. 2016. Analyzing classifiers: Fisher vectors and deep neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, (CVPR). IEEE Computer Society, 2912–2920.
- Scott M. Lundberg and Su-In Lee. 2017. A unified approach to interpreting model predictions. In Proceedings of the International Conference on Advances in Neural Information Processing Systems (NIPS). Curran Associates, Inc., 4765–4774. Retrieved from http://papers.nips.cc/paper/7062-a-unified-approach-to-interpreting-model-predictions.pdf.
- Marina Meila and Jianbo Shi. 2001. A random walks view of spectral segmentation. In Proceedings of the International Workshop on Artificial Intelligence and Statistics (AISTATS).
- Andrew Y. Ng, Michael I. Jordan, and Yair Weiss. 2002. On spectral clustering: Analysis and an algorithm. In Proceedings of the International Conference on Advances in Neural Information Processing Systems. 849–856.
- Anh Nguyen, Alexey Dosovitskiy, Jason Yosinski, Thomas Brox, and Jeff Clune. 2016. Synthesizing the preferred inputs for neurons in neural networks via deep generator networks. In Proceedings of the International Conference on Advances in Neural Information Processing Systems. Curran Associates, Inc., 3387–3395. Retrieved from http://papers.nips.cc/paper/6519-synthesizing-the-preferred-inputs-for-neurons-in-neural-networks-via-deep-generator-networks.pdf.
- Erik Strumbelj and Igor Kononenko. 2010. An efficient explanation of individual classifications using game theory. The Journal of Machine Learning Research 11 (2010) 1–18. JMLR. org.
Cited by
A., S., & R., S. (2023) · Decision Analytics Journal · doi:10.1016/j.dajour.2023.100230 · Cited by 432
Ali, S., Akhlaq, F., Imran, A. S., Kastrati, Z., Daudpota, S. M., & Moosa, M. (2023) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2023.107555 · Cited by 338
Explainable AI for clinical and remote health applications: A survey on tabular and time series data
Di Martino, F., & Delmastro, F. (2023) · Artificial Intelligence Review · doi:10.1007/s10462-022-10304-3 · Cited by 216
“Sun et al. (2021) proposed AttenSurv, an attention-based RNN for Heart Failure (HF) survival prediction of seriously ill patients from longitudinal and heterogeneous EHR.”
MethodExplainable artificial intelligence in Alzheimer’s disease classification: a systematic review
Viswan, V., Shaffi, N., Mahmud, M., Subramanian, K., & Hajamohideen, F. (2024) · Cognitive Computation · doi:10.1007/s12559-023-10192-x · Cited by 195
“One of the significant drawbacks of XAI-based AD diagnosis is the absence of ground truth data [176].”
A Large Comparison of Normalization Methods on Time Series
Lima, F. T., & Souza, V. M. A. (2023) · Big Data Research · doi:10.1016/j.bdr.2023.100407 · Cited by 202
Metadata
- Authors
- Slijepcevic, D., Horst, F., Lapuschkin, S., Horsak, B., Raberger, A.-M., Kranzl, A., Samek, W., Breiteneder, C., Schöllhorn, W. I., & Zeppelzauer, M.
- Affiliations
- Institute of Creative Media Technologies, Department of Media & DigitalTechnologies, St. Pölten University of Applied Sciences, St. Pölten, Austria
- Johannes Gutenberg University Mainz (DE)
- Fraunhofer Institute for Telecommunications, Heinrich Hertz Institute (DE)
- University of Applied Sciences St Pölten (AT)
- Orthopaedic Hospital Speising (AT)
- TU Wien (AT)
- ORCID
- Venue
- ACM Transactions on Computing for Healthcare
- Published
- 2021-12-20
- Volume
- 3
- Issue
- 2
- Pages
- 1-27
- ISSN
- 2637-8051 (ISSN-L)
- 2691-1957
- OpenAlex
- W4200347914
- Cited by
- 82
- References
- 72
- Citations by year
- 2026: 13 · 2025: 21 · 2024: 24 · 2023: 13 · 2022: 8 · 2021: 3
- FWCI
- 4.20
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- ACM Transactions on Computing for Healthcare (publishedVersion, cc-by-nc-sa)
- arXiv (Cornell University) (submittedVersion)
- Fraunhofer-Publica (Fraunhofer-Gesellschaft) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Funders
- Austrian Research Promotion Agency
- Austrian Federal Ministry for Digital and Economic Affairs (ReMoCapLab – #866855 and BigDataAnalytics – #866880)
- Lower Austrian Research and Education Company
- Provincial Government of Lower Austria (IntelliGait3D – #FTI17-014)
- German Ministry for Education and Research as BIFOLD (#01IS18025A and #01IS18037A)
- TraMeExCo (#01IS18056A)
- European Union’s Horizon 2020 (#965221)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Engineering › Biomedical Engineering › Gait Recognition and Analysis
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Keywords
- explainable AI, layer-wise relevance propagation, gait classification, clinical gait analysis, black-box models, ground reaction force, statistical parametric mapping, convolutional neural networks, support vector machine, multilayer perceptron, time series classification
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Explaining Machine Learning Models for Clinical Gait Analysis. https://app.cochat.ai/papers/10.1145/3474121
- The paper
- Slijepcevic, D., Horst, F., Lapuschkin, S., Horsak, B., Raberger, A.-M., Kranzl, A., Samek, W., Breiteneder, C., Schöllhorn, W. I., & Zeppelzauer, M. (2022). Explaining Machine Learning Models for Clinical Gait Analysis. ACM Transactions on Computing for Healthcare. https://doi.org/10.1145/3474121
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.