CoChat

AutoPrognosis 2.0: Democratizing diagnostic and prognostic modeling in healthcare with automated machine learning

Open access
Imrie, Fergus, Cebere, Bogdan, McKinney, Eoin F., van der Schaar, Mihaela · 2023 · PLOS Digital Health
Cited by 45

The DOI matches its Crossref record; 24 of the 24 references it cites were found in a registry or an index.

Abstract

Diagnostic and prognostic models are increasingly important in medicine and inform many clinical decisions. Recently, machine learning approaches have shown improvement over conventional modeling techniques by better capturing complex interactions between patient covariates in a data-driven manner. However, the use of machine learning introduces technical and practical challenges that have thus far restricted widespread adoption of such techniques in clinical settings. To address these challenges and empower healthcare professionals, we present an open-source machine learning framework, AutoPrognosis 2.0, to facilitate the development of diagnostic and prognostic models. AutoPrognosis leverages state-of-the-art advances in automated machine learning to develop optimized machine learning pipelines, incorporates model explainability tools, and enables deployment of clinical demonstrators,without requiring significant technical expertise. To demonstrate AutoPrognosis 2.0, we provide an illustrative application where we construct a prognostic risk score for diabetes using the UK Biobank, a prospective study of 502,467 individuals. The models produced by our automated framework achieve greater discrimination for diabetes than expert clinical risk scores. We have implemented our risk score as a web-based decision support tool, which can be publicly accessed by patients and clinicians. By open-sourcing our framework as a tool for the community, we aim to provide clinicians and other medical practitioners with an accessible resource to develop new risk scores, personalized diagnostics, and prognostics using machine learning techniques. Software:https://github.com/vanderschaarlab/AutoPrognosis

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1371/journal.pdig.0000276

Cites

24 of 24 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 24 references

Research works printed but not linked: 48

  1. EJ Topol, 2019, High-performance medicine: The convergence of human and artificial intelligence, Nat Med, 25, 44, 10.1038/s41591-018-0300-7 · DOI · Paper page
  2. L Petersson, 2022, Challenges to implementing artificial intelligence in healthcare: A qualitative interview study with healthcare leaders in Sweden, BMC Health Serv Res, 22, 850, 10.1186/s12913-022-08215-8 · DOI · Paper page
  3. Alaa A, van der Schaar M. AutoPrognosis: Automated clinical prognostic modeling via Bayesian optimization with structured kernel learning. In: Proceedings of the 35th International Conference on Machine Learning. 2018;80:139–148.
  4. J Bergstra, 2011, Algorithms for hyper-parameter optimization, Advances in Neural Information Processing Systems, 24
  5. M Feurer, 2015, Efficient and robust automated machine learning, Advances in Neural Information Processing Systems, 28, 2755
  6. AM Alaa, 2019, Cardiovascular disease risk prediction using automated machine learning: A prospective study of 423,604 UK Biobank participants, PLoS One, 14, 1
  7. AM Alaa, 2021, Machine learning to guide the use of adjuvant therapies for breast cancer, Nat Mach Intell, 3, 716, 10.1038/s42256-021-00353-8 · DOI · Paper page
  8. H Rahbar, 2020, The value of patient and tumor factors in predicting preoperative breast MRI outcomes, Radiol Imaging Cancer, 2, e190099, 10.1148/rycan.2020190099 · DOI
  9. Z Qian, 2021, CPAS: The UK’s national machine learning-based hospital capacity planning system for COVID-19, Machine Learning, 110, 15, 10.1007/s10994-020-05921-4 · DOI
  10. AA Shah, 2021, Development of a novel, potentially universal machine learning algorithm for prediction of complications after total hip arthroplasty, J Arthroplasty, 36, 1655, 10.1016/j.arth.2020.12.040 · DOI
  11. SK Devana, 2021, A novel, potentially universal machine learning algorithm to predict complications in total knee arthroplasty, Arthroplast Today, 10, 135, 10.1016/j.artd.2021.06.020 · DOI
  12. AA Shah, 2022, Machine learning-driven identification of novel patient factors for prediction of major complications after posterior cervical spinal fusion, Eur Spine J, 31, 1952, 10.1007/s00586-021-06961-7 · DOI
  13. AA Shah, 2022, A risk calculator for the prediction of C5 nerve root palsy after instrumented cervical fusion, World Neurosurg, 166, e703, 10.1016/j.wneu.2022.07.082 · DOI
  14. T Callender, 2023, Assessing eligibility for lung cancer screening: Parsimonious multi-country ensemble machine learning models for lung cancer prediction, medRxiv, 2023
  15. S van Buuren, 2011, mice: Multivariate imputation by chained equations in R, J Stat Softw, 45, 1
  16. Jarrett D, Cebere BC, Liu T, Curth A, van der Schaar M. HyperImpute: Generalized iterative imputation with automatic model selection. In: Proceedings of the 39th International Conference on Machine Learning. 2022;162:9916–9937.
  17. J Liu, 2013, On the stationary distribution of iterative imputations, Biometrika, 101, 155, 10.1093/biomet/ast044 · DOI
  18. Wang Z, Li C, Jegelka S, Kohli P. Batched high-dimensional Bayesian optimization via structural kernel learning. In: Proceedings of the 34th International Conference on Machine Learning. 2017; p. 3656–3664.
  19. L Li, 2018, Hyperband: A novel bandit-based approach to hyperparameter optimization, J Mach Learn Res, 18, 1
  20. SF Crone, 2006, The impact of preprocessing on data mining: An evaluation of classifier sensitivity in direct marketing, Eur J Oper Res, 173, 781, 10.1016/j.ejor.2005.07.023 · DOI · Paper page
  21. MW Kattan, 2016, American Joint Committee on Cancer acceptance criteria for inclusion of risk models for individualized prognosis in the practice of precision medicine, CA Cancer J Clin, 66, 370, 10.3322/caac.21339 · DOI · Paper page
  22. SM Lundberg, 2017, A unified approach to interpreting model predictions, Advances in Neural Information Processing Systems, 30
  23. J Crabbe, 2021, Explaining latent representations with a corpus of examples, Advances in Neural Information Processing Systems, 34, 12154
  24. J Crabbe, 2020, Learning outside the Black-Box: The pursuit of interpretable models, Advances in Neural Information Processing Systems, 33, 17838
  25. G Luo, 2017, Automating construction of machine learning models with clinical big data: Proposal rationale and methods, JMIR Res Protoc, 6, e175, 10.2196/resprot.7757 · DOI
  26. Elshawi R, Maher M, Sakr S. Automated machine learning: State-of-the-art and open challenges. arXiv preprint arXiv:190602287. 2019;.
  27. D Sculley, 2015, Hidden technical debt in machine learning systems, Advances in Neural Information Processing Systems, 28
  28. B Nestor, 2018, Rethinking clinical prediction: Why machine learning must consider year of care and feature aggregation, Machine Learning for Health (ML4H) Workshop at NeurIPS
  29. DR Cox, 1972, Regression models and life-tables, J R Stat Soc Series B Stat Methodol, 34, 187, 10.1111/j.2517-6161.1972.tb00899.x · DOI · Paper page
  30. O Akbilgic, 2019, The promise of machine learning: When will it be delivered?, J Card Fail, 25, 484, 10.1016/j.cardfail.2019.04.006 · DOI
  31. MA Schulz, 2020, Different scaling of linear models and deep learning in UKBiobank brain images versus machine-learning datasets, Nat Commun, 11, 4238, 10.1038/s41467-020-18037-z · DOI · Paper page
  32. E Steyerberg, 2008, Clinical prediction models: A practical approach to development, validation, and updating
  33. I Guyon, 2003, An introduction to variable and feature selection, J Mach Learn Res, 3, 1157
  34. AJ DeGrave, 2021, AI for radiographic COVID-19 detection selects shortcuts over signal, Nat Mach Intell, 3, 610, 10.1038/s42256-021-00338-7 · DOI · Paper page
  35. CH Yoon, 2022, Machine learning in medicine: Should the pursuit of enhanced interpretability be abandoned?, J Med Ethics, 48, 581, 10.1136/medethics-2020-107102 · DOI · Paper page
  36. N Laiteerapong, 2015, The pace of change in medical practice and health policy: Collision or coexistence?, J Gen Intern Med, 30, 848, 10.1007/s11606-015-3182-0 · DOI
  37. DM Gupta, 2017, The physician’s experience of changing clinical practice: a struggle to unlearn, Implement Sci, 12, 28, 10.1186/s13012-017-0555-2 · DOI · Paper page
  38. RJ LeVeque, 2012, Reproducible research for scientific computing: Tools and strategies for changing the culture, Comput Sci Eng, 14, 13, 10.1109/MCSE.2012.38 · DOI · Paper page
  39. M Miłkowski, 2018, Replicability or reproducibility? On the replication crisis in computational neuroscience and sharing only relevant detail, J Comput Neurosci, 45, 163, 10.1007/s10827-018-0702-z · DOI · Paper page
  40. C Sudlow, 2015, UK Biobank: An open access resource for identifying the causes of a wide range of complex diseases of middle and old age, PLoS Med, 12, 1, 10.1371/journal.pmed.1001779 · DOI · Paper page
  41. L Adamska, 2015, Challenges of linking to routine healthcare records in UK Biobank, Trials, 16, O68, 10.1186/1745-6215-16-S2-O68 · DOI
  42. H Bang, 2009, Development and validation of a patient self-assessment score for diabetes risk, Ann Intern Med, 151, 775, 10.7326/0003-4819-151-11-200912010-00005 · DOI · Paper page
  43. J Lindström̈, 2003, The Diabetes Risk Score: A practical tool to predict type 2 diabetes risk, Diabetes Care, 26, 725, 10.2337/diacare.26.3.725 · DOI · Paper page
  44. J Hippisley-Cox, 2017, Development and validation of QDiabetes-2018 risk prediction algorithm to estimate future risk of type 2 diabetes: cohort study, BMJ, 359
  45. AJ Vickers, 2008, Decision analysis for the evaluation of diagnostic tests, prediction models, and molecular markers, Am Stat, 62, 314, 10.1198/000313008X370302 · DOI · Paper page
  46. J Cohen, 2013, Statistical power analysis for the behavioral sciences, 10.4324/9780203771587 · DOI · Paper page
  47. J Nano, 2017, Gamma-glutamyltransferase levels, prediabetes and type 2 diabetes: A Mendelian randomization study, Int J Epidemiol, 46, 1400, 10.1093/ije/dyx006 · DOI
  48. International Diabetes Federation, 2013, IDF Diabetes Atlas

Cited by

Metadata

Authors
Imrie, F., Cebere, B., McKinney, E. F., & van der Schaar, M.
Venue
PLOS Digital Health
Published
2023-06-22
Volume
2
Issue
6
Pages
e0000276
ISSN
2767-3170 (ISSN-L)
OpenAlex
W4381611976
Cited by
45
References
24
FWCI
3.95
Type
journal article
Language
en
Open access
gold

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.