CoChat

Interpretable video-based tracking and quantification of parkinsonism clinical motor states

Open access

Published version of a preprint

The DOI matches its Crossref record; 70 of the 70 references it cites were found in a registry or an index.

Abstract

Quantification of motor symptom progression in Parkinson’s disease (PD) patients is crucial for assessing disease progression and for optimizing therapeutic interventions, such as dopaminergic medications and deep brain stimulation. Cumulative and heuristic clinical experience has identified various clinical signs associated with PD severity, but these are neither objectively quantifiable nor robustly validated. Video-based objective symptom quantification enabled by machine learning (ML) introduces a potential solution. However, video-based diagnostic tools often have implementation challenges due to expensive and inaccessible technology, and typical “black-box” ML implementations are not tailored to be clinically interpretable. Here, we address these needs by releasing a comprehensive kinematic dataset and developing an interpretable video-based framework that predicts high versus low PD motor symptom severity according to MDS-UPDRS Part III metrics. This data driven approach validated and robustly quantified canonical movement features and identified new clinical insights, not previously appreciated as related to clinical severity, including pinkie finger movements and lower limb and axial features of gait. Our framework is enabled by retrospective, single-view, seconds-long videos recorded on consumer-grade devices such as smartphones, tablets, and digital cameras, thereby eliminating the requirement for specialized equipment. Following interpretable ML principles, our framework enforces robustness and interpretability by integrating (1) automatic, data-driven kinematic metric evaluation guided by pre-defined digital features of movement, (2) combination of bi-domain (body and hand) kinematic features, and (3) sparsity-inducing and stability-driven ML analysis with simple-to-interpret models. These elements ensure that the proposed framework quantifies clinically meaningful motor features useful for both ML predictions and clinical analysis.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1038/s41531-024-00742-x

Cites

70 of 70 references verified0 citation errors · 0 not checkable

Statistics

Figures

Fig. 1 Schematic overview of automatic feature extraction from video recordings for classification of Parkinson’s disease motor symptom severity. a From recordings of participants performing prescribed motor assessment tasks, we extracted movement (kinematic) time series at key landmarks using the pose estimation library MediaPipe. Relative movement measurements were computed based on the extracted signals, from which various temporal and spectral metrics were computed as features. Pairwise combinations of samples from the same patient under the same medication state were performed on the body and hand feature sets to form the bi-modality combination feature set. b To obtain objective measurements of classification performances, we performed a leave-one-subject-out cross-validation (CV), where samples from one patient are held out as the training set for each CV iteration, on each dataset. The CV process is repeated 16 times to account for variabilities in models trained. During CV, a small subset of features with high predictive power with regard to the assigned group labels was selected via least absolute shrinkage and selection operator (LASSO) feature selection based on the training set. The fitted feature selection is then applied to both the training and validation sets. The selected features of the training and validation sets were used to train and validate, respectively, the various machine learning (ML) models. c LASSO feature selection consistently identified small sets of salient features during each CV iteration. The average number of features selected at each iteration is reported as mean ± standard deviation.
Fig. 1 Schematic overview of automatic feature extraction from video recordings for classification of Parkinson’s disease motor symptom severity. a From recordings of participants performing prescribed motor assessment tasks, we extracted movement (kinematic) time series at key landmarks using the pose estimation library MediaPipe. Relative movement measurements were computed based on the extracted signals, from which various temporal and spectral metrics were computed as features. Pairwise combinations of samples from the same patient under the same medication state were performed on the body and hand feature sets to form the bi-modality combination feature set. b To obtain objective measurements of classification performances, we performed a leave-one-subject-out cross-validation (CV), where samples from one patient are held out as the training set for each CV iteration, on each dataset. The CV process is repeated 16 times to account for variabilities in models trained. During CV, a small subset of features with high predictive power with regard to the assigned group labels was selected via least absolute shrinkage and selection operator (LASSO) feature selection based on the training set. The fitted feature selection is then applied to both the training and validation sets. The selected features of the training and validation sets were used to train and validate, respectively, the various machine learning (ML) models. c LASSO feature selection consistently identified small sets of salient features during each CV iteration. The average number of features selected at each iteration is reported as mean ± standard deviation.CC BY · PubMed Central
Fig. 2 Classification performances of seven selected ML classification models. SVM support vector machine, LR logistic regression, LDA linear discriminant analysis, RF random forest, AB adaptive-boosted trees, KNN K-nearest neighbors, GNB Gaussian naive Bayes ROC AUC area under receiver operating characteristics curve. a Here we show the average ROC curves of the trained ML models during 16-times repeated leave-on-subject-out cross-validation (CV). The corresponding AUC scores are reported in the right margins as mean ± standard deviation. Overall, integrating body and hand features led to sustained or mildly improved performances in most models. b Here we show histograms of the classification accuracies of all models aggregated over all CV iterations. The accuracy scores are reported as mean ± standard deviation at the base of the histogram bars. Similar to the findings when reading the ROC plots, models trained with combined features had comparable accuracies with those trained only on body features, which were more accurate than those trained only on hand features.
Fig. 2 Classification performances of seven selected ML classification models. SVM support vector machine, LR logistic regression, LDA linear discriminant analysis, RF random forest, AB adaptive-boosted trees, KNN K-nearest neighbors, GNB Gaussian naive Bayes ROC AUC area under receiver operating characteristics curve. a Here we show the average ROC curves of the trained ML models during 16-times repeated leave-on-subject-out cross-validation (CV). The corresponding AUC scores are reported in the right margins as mean ± standard deviation. Overall, integrating body and hand features led to sustained or mildly improved performances in most models. b Here we show histograms of the classification accuracies of all models aggregated over all CV iterations. The accuracy scores are reported as mean ± standard deviation at the base of the histogram bars. Similar to the findings when reading the ROC plots, models trained with combined features had comparable accuracies with those trained only on body features, which were more accurate than those trained only on hand features.CC BY · PubMed Central
Fig. 3 Group differences in stable features and their contribution to ML model predictions. STD standard deviation, MAD median absolute deviation. The stable features were chosen according to the criteria that they must be selected in at least 50% of all models trained during cross-validation (CV). a Point plot showing the statistically significant differences (p ≪ 0.01) in group means for the selected features between groups with low and high levels of motor impairment. The feature values are z-score normalized. b The swarm plots on the left show individual SHAP values of feature values encountered during CV training of a logistic regression (LR) classifier. The histograms on the right show the mean absolute SHAP values of each feature. A higher SHAP value indicates a stronger bias toward predicting the “less severe” class label. A higher mean absolute SHAP value for each feature corresponds to its level of feature attribution in relation to the predicted label. c Visualizations of landmark relationships from which kinematic metrics were derived. The body and hand skeletons also show important landmarks and other landmark relationships were based on in our extraction pipeline.
Fig. 3 Group differences in stable features and their contribution to ML model predictions. STD standard deviation, MAD median absolute deviation. The stable features were chosen according to the criteria that they must be selected in at least 50% of all models trained during cross-validation (CV). a Point plot showing the statistically significant differences (p ≪ 0.01) in group means for the selected features between groups with low and high levels of motor impairment. The feature values are z-score normalized. b The swarm plots on the left show individual SHAP values of feature values encountered during CV training of a logistic regression (LR) classifier. The histograms on the right show the mean absolute SHAP values of each feature. A higher SHAP value indicates a stronger bias toward predicting the “less severe” class label. A higher mean absolute SHAP value for each feature corresponds to its level of feature attribution in relation to the predicted label. c Visualizations of landmark relationships from which kinematic metrics were derived. The body and hand skeletons also show important landmarks and other landmark relationships were based on in our extraction pipeline.CC BY · PubMed Central
Fig. 4 Scatter plots comparing 2D projections of datasets with and without stable feature selection. The projections were computed using MDS (multidimensional scaling). a Poor separability of class clusters in 2D projection space when all features in the datasets were used. b Improved separation (especially when both body and hand features were included) of class clusters when only stable features were used.
Fig. 4 Scatter plots comparing 2D projections of datasets with and without stable feature selection. The projections were computed using MDS (multidimensional scaling). a Poor separability of class clusters in 2D projection space when all features in the datasets were used. b Improved separation (especially when both body and hand features were included) of class clusters when only stable features were used.CC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 70 references

Research works printed but not linked: 1

  1. Hand, D. J. & Yu, K. Idiot’s Bayes? Not so stupid after all? Int Stat. Rev. 69, 385–398 (2001).

Cited by

See all 25

Metadata

Authors
Deng, D., Ostrem, J. L., Nguyen, V., Cummins, D. D., Sun, J., Pathak, A., Little, S., & Abbasi-Asl, R.
Affiliations
  • University of California, San Francisco (US)
  • Google (United States) (US)
Corresponding author
  • Little, Simon
  • Abbasi-Asl, Reza
Venue
npj Parkinson's Disease
Published
2024-06-25
Volume
10
Issue
1
Pages
122-122
ISSN
2373-8057 (ISSN-L)
OpenAlex
W4399999456
Cited by
25
References
70
Citations by year
2026: 11 · 2025: 13 · 2024: 1
FWCI
6.69
Type
journal article
Language
en
Open access
Licence
Locations
  • npj Parkinson s Disease (publishedVersion, cc-by)
  • medRxiv (acceptedVersion, cc-by-nc-nd)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • eScholarship (California Digital Library) (submittedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
Funders
UCSF Innovation Ventures
Topics
  • Health Sciences › Medicine › Clinical Neurology › Parkinson's Disease Mechanisms and Treatments
  • Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
  • Health Sciences › Health Professions › Physical Therapy, Sports Therapy and Rehabilitation › Balance, Gait, and Falls Prevention
Keywords
Parkinson's disease, parkinsonism, motor impairment severity, model interpretability, kinematic parameters, MDS-UPDRS III, gait, axial symptoms, smartphone-based assessment, disease progression

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Interpretable video-based tracking and quantification of parkinsonism clinical motor states. https://app.cochat.ai/papers/10.1038/s41531-024-00742-x
The paper
Deng, D., Ostrem, J. L., Nguyen, V., Cummins, D. D., Sun, J., Pathak, A., Little, S., & Abbasi-Asl, R. (2024). Interpretable video-based tracking and quantification of parkinsonism clinical motor states. npj Parkinson's Disease. https://doi.org/10.1038/s41531-024-00742-x

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.