CoChat
Sign Up
Cited by 31Osteoarthritis Treatment and MechanismsArtificial Intelligence in Healthcare and EducationRadiomics and Machine Learning in Medical Imaging

Multimodal Machine Learning-based Knee Osteoarthritis Progression Prediction from Plain Radiographs and Clinical Data

Open access
Tiulpin, Aleksei, Klein, Stefan, Bierma-Zeinstra, Sita M. A., Thevenot, Jérôme, Rahtu, Esa, Meurs, Joyce van, Oei, Edwin H. G., Saarakkala, Simo·2019·Scientific Reports
DOI 10.1038/s41598-019-56527-3PMC PMC6934728
Open this paper in CoChat — free accountOpen PDF

Abstract

Knee osteoarthritis (OA) is the most common musculoskeletal disease without a cure, and current treatment options are limited to symptomatic relief. Prediction of OA progression is a very challenging and timely issue, and it could, if resolved, accelerate the disease modifying drug development and ultimately help to prevent millions of total joint replacement surgeries performed annually. Here, we present a multi-modal machine learning-based OA progression prediction model that utilises raw radiographic data, clinical examination results and previous medical history of the patient. We validated this approach on an independent test set of 3,918 knee images from 2,129 subjects. Our method yielded area under the ROC curve (AUC) of 0.79 (0.78–0.81) and Average Precision (AP) of 0.68 (0.66–0.70). In contrast, a reference approach, based on logistic regression, yielded AUC of 0.75 (0.74–0.77) and AP of 0.62 (0.60–0.64). The proposed method could significantly improve the subject selection process for OA drug-development trials and help the development of personalised therapeutic plans.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1038/s41598-019-56527-3

Cites

2 of 2 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • A machine learning study for early-stage knee osteoarthritis used XGBoost for clinical features, convolutional neural networks for radiographic features, and demographic data to regularize intercorrelations, and performed better than single-modal strategies.

    “A machine learning study for early-stage knee osteoarthritis used XGBoost to extract clinical features, convolutional neural networks for radiographic features, and demographic data to regularize intercorrelations among many features using L1-norm-based optimization [116]. This method performed better than single-modal strategies, highlighting the need for integrated evaluation [117].”

    From Biomarker-Guided Imaging and AI-Augmented Diagnosis of Degenerative Joint Disease (2025) · cites this paper as [116]

    Split into 2 atoms — 1 confirmed, 1 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 41

  1. [1]Arden N Nevitt MC Osteoarthritis: epidemiology Best practice & research Clinical rheumatology 2006 20 3 25 10.1016/j.berh.2005.09.007 16483904 · DOI
  2. [2]Ferket BS Impact of total knee replacement practice: cost effectiveness analysis of data from the osteoarthritis initiative bmj 2017 356 j1131 10.1136/bmj.j1131 28351833 PMC6284324 · DOI
  3. [3]Bedson J Jordan K Croft P The prevalence and history of knee osteoarthritis in general practice: a case–control study Family practice 2005 22 103 108 10.1093/fampra/cmh700 15640302 · DOI
  4. [4]Jamshidi, A., Pelletier, J.-P. & Martel-Pelletier, J. Machine-learning-based patient-specific prediction models for knee osteoarthritis. Nature Reviews Rheumatology 1 (2018). 30523334
  5. [5]van Oudenaarde K General practitioners referring adults to mr imaging for knee pain: a randomized controlled trial to assess cost-effectiveness Radiology 2018 288 170 176 10.1148/radiol.2018171383 29664339 · DOI
  6. [6]Kellgren J Lawrence J Radiological assessment of osteo-arthrosis Annals of the rheumatic diseases 1957 16 494 10.1136/ard.16.4.494 13498604 PMC1006995 · DOI
  7. [7]Tiulpin A Thevenot J Rahtu E Lehenkari P Saarakkala S Automatic knee osteoarthritis diagnosis from plain radiographs: A deep learning-based approach Scientific reports 2018 8 1727 10.1038/s41598-018-20132-7 29379060 PMC5789045 · DOI
  8. [8]Norman, B., Pedoia, V., Noworolski, A., Link, T. M. & Majumdar, S. Applying densely connected convolutional neural networks for staging osteoarthritis severity from plain radiographs. Journal of digital imaging 1–7 (2018). PMC6499841 30306418
  9. [9]Antony, J., McGuinness, K., O’Connor, N. E. & Moran, K. Quantifying radiographic knee osteoarthritis severity using deep convolutional neural networks. In 2016 23rd International Conference on Pattern Recognition (ICPR), 1195–1200 (IEEE, 2016).
  10. [10]Kerkhof HJ Prediction model for knee osteoarthritis incidence, including clinical, genetic and biochemical risk factors Annals of the rheumatic diseases 2014 73 2116 2121 10.1136/annrheumdis-2013-203620 23962456 · DOI
  11. [11]Janvier T Subchondral tibial bone texture analysis predicts knee osteoarthritis progression: data from the osteoarthritis initiative: tibial bone texture & knee oa progression Osteoarthritis and cartilage 2017 25 259 266 10.1016/j.joca.2016.10.005 27742531 · DOI
  12. [12]Janvier T Jennane R Toumi H Lespessailles E Subchondral tibial bone texture predicts the incidence of radiographic knee osteoarthritis: data from the osteoarthritis initiative Osteoarthritis and cartilage 2017 25 2047 2054 10.1016/j.joca.2017.09.004 28935435 · DOI
  13. [13]Kraus VB Trabecular morphometry by fractal signature analysis is a novel marker of osteoarthritis progression Arthritis & Rheumatism: Official Journal of the American College of Rheumatology 2009 60 3711 3722 10.1002/art.25012 PMC3711179 19950282 · DOI
  14. [14]Yu D Development and validation of prediction models to estimate risk of primary total hip and knee replacements using data from the uk: two prospective open cohorts using the uk clinical practice research datalink Annals of the rheumatic diseases 2019 78 91 99 10.1136/annrheumdis-2018-213894 30337425 PMC6317440 · DOI
  15. [15]Hosnijeh FS Development of a prediction model for future risk of radiographic hip osteoarthritis Osteoarthritis and cartilage 2018 26 540 546 10.1016/j.joca.2018.01.015 29382605 · DOI
  16. [16]Emrani PS Joint space narrowing and kellgren–lawrence progression in knee osteoarthritis: an analytic literature synthesis Osteoarthritis and Cartilage 2008 16 873 882 10.1016/j.joca.2007.12.004 18280757 PMC2701468 · DOI
  17. [17]LaValley MP McAlindon TE Chaisson CE Levy D Felson DT The validity of different definitions of radiographic worsening for longitudinal studies of knee osteoarthritis Journal of clinical epidemiology 2001 54 30 39 10.1016/S0895-4356(00)00273-0 11165466 · DOI
  18. [18]Schmidhuber Jürgen Deep learning in neural networks: An overview Neural Networks 2015 61 85 117 10.1016/j.neunet.2014.09.003 25462637 · DOI · Paper page
  19. [19]LeCun Y Bengio Y Hinton G Deep learning Nature 2015 521 436 10.1038/nature14539 26017442 · DOI · Paper page
  20. [20]Friedman Jerome H. machine The Annals of Statistics 2001 29 5 1189 1232 10.1214/aos/1013203451 · DOI · Paper page
  21. [21]Friedman J Hastie T Tibshirani R The elements of statistical learning 2001 New York Springer series in statistics
  22. [22]Bellamy N Buchanan WW Goldsmith CH Campbell J Stitt LW Validation study of womac: a health status instrument for measuring clinically important patient relevant outcomes to antirheumatic drug therapy in patients with osteoarthritis of the hip or knee The Journal of rheumatology 1988 15 1833 1840 3068365
  23. [23]Hu, J., Shen, L. & Sun, G. Squeeze-and-excitation networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, 7132–7141 (2018).
  24. [24]Selvaraju, R. R. et al. Grad-cam: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE International Conference on Computer Vision, 618–626 (2017).
  25. [25]Wolpert DH Stacked generalization Neural networks 1992 5 241 259 10.1016/S0893-6080(05)80023-1 · DOI · Paper page
  26. [26]Olah C The building blocks of interpretability Distill 2018 3 e10 10.23915/distill.00010 · DOI · Paper page
  27. [27]Bach S On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation PLoS One 2015 10 e0130140 10.1371/journal.pone.0130140 26161953 PMC4498753 · DOI · Paper page
  28. [28]Kinds MB Quantitative radiographic features of early knee osteoarthritis: development over 5 years and relationship with symptoms in the check cohort The Journal of rheumatology 2013 40 58 65 10.3899/jrheum.120320 23118113 · DOI
  29. [29]Neumann G Location specific radiographic joint space width for osteoarthritis progression Osteoarthritis and cartilage 2009 17 761 765 10.1016/j.joca.2008.11.001 19073368 PMC3138121 · DOI
  30. [30]Kothari M Fixed-flexion radiography of the knee provides reproducible joint space width measurements in osteoarthritis European radiology 2004 14 1568 1573 10.1007/s00330-004-2312-6 15150666 · DOI
  31. [31]Lindner C Bromiley PA Ionita MC Cootes TF Robust and accurate shape model matching using random forest regression-voting IEEE transactions on pattern analysis and machine intelligence 2015 37 1862 1874 10.1109/TPAMI.2014.2382106 26353132 · DOI
  32. [32]Pedregosa F Scikit-learn: Machine learning in python Journal of machine learning research 2011 12 2825 2830
  33. [33]Seabold, S. & Perktold, J. Statsmodels: Econometric and statistical modeling with python. In Proceedings of the 9th Python in Science Conference, vol. 57, 61 (Scipy, 2010).
  34. [34]Ke, G. et al. Lightgbm: A highly efficient gradient boosting decision tree. In Advances in Neural Information Processing Systems, 3146–3154 (2017).
  35. [35]Paszke, A. et al. Automatic differentiation in pytorch. In NIPS-W (2017).
  36. [36]Bergstra, J., Yamins, D. & Cox, D. D. Hyperopt: A python library for optimizing the hyperparameters of machine learning algorithms. In Proceedings of the 12th Python in science conference, 13–20 (Citeseer, 2013).
  37. [37]Deng, J. et al. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, 248–255 (Ieee, 2009).
  38. [39]Srivastava N Hinton G Krizhevsky A Sutskever I Salakhutdinov R Dropout: a simple way to prevent neural networks from overfitting The Journal of Machine Learning Research 2014 15 1929 1958
  39. [41]Esteva A Dermatologist-level classification of skin cancer with deep neural networks Nature 2017 542 115 10.1038/nature21056 28117445 PMC8382232 · DOI · Paper page
  40. [43]Cortes, C. & Mohri, M. Auc optimization vs. error rate minimization. In Advances in neural information processing systems, 313–320 (2004).
  41. [44]DeLong ER DeLong DM Clarke-Pearson DL Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach Biometrics 1988 44 837 845 10.2307/2531595 3203132 · DOI · Paper page

Cited by

Sorted by citations

Authors
Tiulpin, A., Klein, S., Bierma-Zeinstra, S. M. A., Thevenot, J., Rahtu, E., Meurs, J. V., Oei, E. H. G., & Saarakkala, S.
Affiliations
  • Oulu University Hospital (FI)
  • Erasmus MC (NL)
  • University of Oulu (FI)
  • Tampere University of Technology (FI)
Corresponding author
Tiulpin, Aleksei
ORCID
Venue
Scientific Reports
Publisher
Springer Science and Business Media LLC
Published
2019-12-27
Volume
9
Issue
1
Pages
20038-20038
ISSN
2045-2322 (ISSN-L)
DOI
10.1038/s41598-019-56527-3
PMC
PMC6934728
OpenAlex
W2942238494
Cited by
31
References
2
Citations by year
2026: 11 · 2025: 2 · 2024: 5 · 2023: 1 · 2022: 1 · 2021: 6 · 2020: 4 · 2019: 1
FWCI
2.28
Type
journal article
Language
en
Open access
Licence
Locations
  • Scientific Reports (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • EUR Research Repository (Erasmus University Rotterdam) (publishedVersion, other-oa)
  • arXiv (Cornell University) (submittedVersion)
  • EUR Research Repository (Erasmus University Rotterdam) (submittedVersion, other-oa)
  • RePub (Erasmus University Rotterdam) (submittedVersion, other-oa)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
  • Tampere University Institutional Repository (Tampere University) (submittedVersion, cc-by)
  • arXiv (Cornell University) (cc-by)
Funders
  • KAUTE-Säätiö (201700751)
  • Sigrid Juséliuksen Säätiö
Topics
  • Health Sciences › Medicine › Rheumatology › Osteoarthritis Treatment and Mechanisms
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Radiomics and Machine Learning in Medical Imaging
Keywords
knee osteoarthritis, disease progression prediction, plain radiographs, multimodal learning, clinical data integration
MeSH terms
Machine Learning, Aged, Female, Humans, Male, Middle Aged, Disease Progression, Osteoarthritis, Knee

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review