Machine Learning and Prediction in Medicine — Beyond the Peak of Inflated Expectations
Open accessAbstract
Big data, we have all heard, promise to transform health care. But in the “hype cycle” of emerging technologies, machine learning now rides atop the “peak of inflated expectations,” and we need to better appreciate the technology’s capabilities and limitations.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Deployment strategies themselves constitute a major and underexamined source of risk in real-world clinical AI use.
“However, accumulating evidence indicates that deployment strategies themselves constitute a major and underexamined source of risk in real world clinical AI use [1, 4].”
From Adaptive FHIR Native AI Governance for Clinical Decision Support: A Modular, Auditable Deployment Framework for Real World Clinical AI (2026) · cites this paper as [1]
In the main text (page unavailable), Chen and Asch call the last mile of clinical implementation critical and explain that accurate predictions may not improve care unless they inform a timely intervention. This supports concern about real-world use, but the paper does not identify deployment strategies themselves as a major, underexamined source of risk or describe accumulating evidence for that claim. (LLM, conf=0.84)
Experimental — accuracy not yet measuredMany deployed clinical AI systems fail to deliver sustained benefit in routine clinical care despite strong retrospective performance.
“Despite strong retrospective performance, many deployed systems fail to deliver sustained benefit in routine clinical care [1, 2].”
From Adaptive FHIR Native AI Governance for Clinical Decision Support: A Modular, Auditable Deployment Framework for Real World Clinical AI (2026) · cites this paper as [1]
In the main text (page not provided), the paper says that even a perfectly calibrated prediction model may not improve clinical care and that historical performance is not enough to establish future usefulness. It does not report that many deployed systems have failed to deliver sustained benefit in routine care. (LLM, conf=0.84)
Experimental — accuracy not yet measuredFailures of clinical AI systems frequently stem from static and opaque deployment strategies rather than deficiencies in predictive models themselves.
“Clinical AI failures often arise from static and opaque deployment strategies rather than deficiencies in predictive models themselves.”
From Adaptive FHIR Native AI Governance for Clinical Decision Support: A Modular, Auditable Deployment Framework for Real World Clinical AI (2026) · cites this paper as [1]
In the main text (page unavailable), the paper says that even a perfectly calibrated model may not improve care and that the 'last mile' of clinical implementation is critical. It also warns that historical clinical data rapidly lose relevance. These passages support concerns about implementation and static approaches, but they do not identify opaque deployment strategies or establish that such strategies are a frequent cause of clinical AI failures rather than model deficiencies. (LLM, conf=0.86)
Experimental — accuracy not yet measuredThe first requirement for the successful development of multimodal data-enabled applications is the collection, curation and harmonization of well-phenotyped and large annotated datasets, as no amount of technical sophistication can derive information not present in the data.
“The first requirement for the successful development of multimodal data-enabled applications is the collection, curation and harmonization of well-phenotyped and large annotated datasets, as no amount of technical sophistication can derive information not present in the data 80 .”
From Multimodal biomedical AI (2022) · cites this paper as [80]
Split into 2 atoms — 1 confirmed, 1 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Predicting the future - big data, machine learning, and clinical medicine
Obermeyer, Z., & Emanuel, E. J. (2016) · New England Journal of Medicine · doi:10.1056/nejmp1606181 · Cited by 3,652 · see citing works
The Parable of Google Flu: Traps in Big Data Analysis
Lazer, D., Kennedy, R., King, G., & Vespignani, A. (2014) · Science · doi:10.1126/science.1248506 · Cited by 2,523 · see citing works
Escobar, G., Turk, B. J., Ragins, A. I., Ha, J., Hoberman, B., Levine, S., Ballesca, M. A., Liu, V. X., & Kipnis, P. (2016) · Journal of Hospital Medicine · doi:10.1002/jhm.2652 · Cited by 102 · see citing works
Chen, J. H., Alagappan, M., Goldstein, M. K., Asch, S. M., & Altman, R. B. (2017) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2017.03.006 · Cited by 101 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Christodoulou, E., Ma, J., Collins, G. S., Steyerberg, E. W., Verbakel, J. Y., & Van Calster, B. (2019) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2019.02.004 · Cited by 2,119 · see citing works
Artificial intelligence in surgery: promises and perils
Hashimoto, D. A., Rosman, G., Rus, D., & Meireles, O. R. (2018) · Annals of Surgery · doi:10.1097/sla.0000000000002693 · Cited by 1,468 · see citing works
Acosta, J. N., Falcone, G. J., Rajpurkar, P., & Topol, E. J. (2022) · Nature Medicine · doi:10.1038/s41591-022-01981-2 · Cited by 1,365 · see citing works
Komorowski, M., Celi, L. A., Badawi, O., Gordon, A. C., & Faisal, A. A. (2018) · Nature Medicine · doi:10.1038/s41591-018-0213-5 · Cited by 1,290 · see citing works
Park, S. H., & Han, K. (2018) · Radiology · doi:10.1148/radiol.2017171920 · Cited by 894 · see citing works
External validation of prognostic models: what, why, how, when and where?
Ramspek, C. L., Jager, K. J., Dekker, F. W., Zoccali, C., & van Diepen, M. (2021) · Clinical Kidney Journal · doi:10.1093/ckj/sfaa188 · Cited by 887 · see citing works
Current Applications and Future Impact of Machine Learning in Radiology
Choy, G., Khalilzadeh, O., Michalski, M., Do, S., Samir, A. E., Pianykh, O. S., Geis, J. R., Pandharipande, P. V., Brink, J. A., & Dreyer, K. J. (2018) · Radiology · doi:10.1148/radiol.2018171820 · Cited by 800 · see citing works
Reporting of artificial intelligence prediction models
Collins, G. S., & Moons, K. G. M. (2019) · The Lancet · doi:10.1016/s0140-6736(19)30037-6 · Cited by 701 · see citing works
Machine learning in cardiovascular medicine: are we there yet?
Shameer, K., Johnson, K. W., Glicksberg, B. S., Dudley, J. T., & Sengupta, P. P. (2018) · Heart · doi:10.1136/heartjnl-2017-311198 · Cited by 555 · see citing works
Deep learning interpretation of echocardiograms
Ghorbani, A., Ouyang, D., Abid, A., He, B., Chen, J. H., Harrington, R. A., Liang, D. H., Ashley, E. A., & Zou, J. Y. (2020) · npj Digital Medicine · doi:10.1038/s41746-019-0216-8 · Cited by 518 · see citing works
Self-citationDeveloping clinical prediction models: a step-by-step guide
Efthimiou, O., Seo, M., Chalkou, K., Debray, T., Egger, M., & Salanti, G. (2024) · BMJ · doi:10.1136/bmj-2023-078276 · Cited by 544 · see citing works
Simon, G. E., Johnson, E., Lawrence, J. M., Rossom, R. C., Ahmedani, B., Lynch, F. L., Beck, A., Waitzfelder, B., Ziebell, R., Penfold, R. B., et al. (2018) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2018.17101167 · Cited by 400 · see citing works
Laranjo, L., Ding, D., Heleno, B., Kocaballi, B., Quiroz, J. C., Tong, H. L., Chahwan, B., Neves, A. L., Gabarron, E., Dao, K. P., et al. (2021) · British Journal of Sports Medicine · doi:10.1136/bjsports-2020-102892 · Cited by 400 · see citing works
SHIFTing artificial intelligence to be responsible in healthcare: a systematic review
Siala, H., & Wang, Y. (2022) · Social Science & Medicine · doi:10.1016/j.socscimed.2022.114782 · Cited by 358 · see citing works
Machine learning for real-time prediction of complications in critical care: a retrospective study
Meyer, A., Zverinski, D., Pfahringer, B., Kempfert, J., Kuehne, T., Sündermann, S. H., Stamm, C., Hofmann, T., Falk, V., & Eickhoff, C. (2018) · The Lancet Respiratory Medicine · doi:10.1016/s2213-2600(18)30300-x · Cited by 350 · see citing works
Machine learning and medical education
Kolachalama, V. B., & Garg, P. S. (2018) · npj Digital Medicine · doi:10.1038/s41746-018-0061-1 · Cited by 320 · see citing works
Bertsimas, D., Dunn, J., Velmahos, G. C., & Kaafarani, H. M. A. (2018) · Annals of Surgery · doi:10.1097/sla.0000000000002956 · Cited by 321 · see citing works
Implications of large language models such as <scp>ChatGPT</scp> for dental medicine
Eggmann, F., Weiger, R., Zitzmann, N. U., & Blatz, M. B. (2023) · Journal of Esthetic and Restorative Dentistry · doi:10.1111/jerd.13046 · Cited by 312 · see citing works
Alami, H., Rivard, L., Lehoux, P., Hoffman, S. J., Cadeddu, S. B. M., Savoldelli, M., Samri, M. A., Ag Ahmed, M. A., Fleet, R., & Fortin, J.-P. (2020) · Globalization and Health · doi:10.1186/s12992-020-00584-1 · Cited by 283 · see citing works
Reimagining Medical Education in the Age of AI
(2019) · AMA Journal of Ethics · doi:10.1001/amajethics.2019.146 · Cited by 263 · see citing works
From promise to practice: towards the realisation of AI-informed mental health care
Koutsouleris, N., Hauser, T. U., Skvortsova, V., & De Choudhury, M. (2022) · The Lancet Digital Health · doi:10.1016/s2589-7500(22)00153-4 · Cited by 251 · see citing works
Exercise therapy and cardiovascular toxicity in cancer
Scott, J. M., Nilsen, T. S., Gupta, D., & Jones, L. W. (2018) · Circulation · doi:10.1161/circulationaha.117.024671 · Cited by 239 · see citing works
Homecare robotic systems for healthcare 4.0: visions and enabling technologies
Yang, G., Pang, Z., Jamal Deen, M., Dong, M., Zhang, Y.-T., Lovell, N., & Rahmani, A. M. (2020) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2020.2990529 · Cited by 213 · see citing works
Henry, K. E., Kornfield, R., Sridharan, A., Linton, R. C., Groh, C., Wang, T., Wu, A., Mutlu, B., & Saria, S. (2022) · npj Digital Medicine · doi:10.1038/s41746-022-00597-7 · Cited by 207 · see citing works
Beyond Sparsity: Tree Regularization of Deep Models for Interpretability
Wu, M., Hughes, M., Parbhoo, S., Zazzi, M., Roth, V., & Doshi-Velez, F. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11501 · Cited by 213 · see citing works
Xu, H., & Shuttleworth, K. M. J. (2024) · Intelligent Medicine · doi:10.1016/j.imed.2023.08.001 · Cited by 196 · see citing works
Bridging the implementation gap of machine learning in healthcare
Seneviratne, M. G., Shah, N. H., & Chu, L. (2020) · BMJ Innovations · doi:10.1136/bmjinnov-2019-000359 · Cited by 173 · see citing works
Putting the data before the algorithm in big data addressing personalized healthcare
Cahan, E. M., Hernandez-Boussard, T., Thadaney-Israni, S., & Rubin, D. L. (2019) · npj Digital Medicine · doi:10.1038/s41746-019-0157-2 · Cited by 149 · see citing works
Dhiman, P., Ma, J., Andaur Navarro, C. L., Speich, B., Bullock, G., Damen, J. A. A., Hooft, L., Kirtley, S., Riley, R. D., Van Calster, B., et al. (2022) · BMC Medical Research Methodology · doi:10.1186/s12874-022-01577-x · Cited by 140 · see citing works
Merath, K., Hyer, J. M., Mehta, R., Farooq, A., Bagante, F., Sahara, K., Tsilimigras, D. I., Beal, E., Paredes, A. Z., Wu, L., et al. (2020) · Journal of Gastrointestinal Surgery · doi:10.1007/s11605-019-04338-2 · Cited by 131 · see citing works
A contemporary review of machine learning in otolaryngology–head and neck surgery
Crowson, M. G., Ranisau, J., Eskander, A., Babier, A., Xu, B., Kahmke, R. R., Chen, J. M., & Chan, T. C. Y. (2020) · The Laryngoscope · doi:10.1002/lary.27850 · Cited by 127 · see citing works
Self-citationA framework for the oversight and local deployment of safe and high-quality prediction models
Bedoya, A. D., Economou-Zavlanos, N. J., Goldstein, B. A., Young, A., Jelovsek, J. E., O’Brien, C., Parrish, A. B., Elengold, S., Lytle, K., Balu, S., et al. (2022) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocac078 · Cited by 119 · see citing works
Behzadi-khormouji, H., Rostami, H., Salehi, S., Derakhshande-Rishehri, T., Masoumi, M., Salemi, S., Keshavarz, A., Gholamrezanezhad, A., Assadi, M., & Batouli, A. (2020) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2019.105162 · Cited by 112 · see citing works
Rundo, L., Pirrone, R., Vitabile, S., Sala, E., & Gambino, O. (2020) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2020.103479 · Cited by 113 · see citing works
Preventive Treatments for Psychosis: Umbrella Review (Just the Evidence)
Fusar-Poli, P., Davies, C., Solmi, M., Brondino, N., De Micheli, A., Kotlicka-Antczak, M., Shin, J. I., & Radua, J. (2019) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2019.00764 · Cited by 108 · see citing works
From images to actions: Opportunities for artificial intelligence in radiology
Kahn, C. E. (2017) · Radiology · doi:10.1148/radiol.2017171734 · Cited by 88 · see citing works
Barbieri, S., Kemp, J., Perez-Concha, O., Kotwal, S., Gallagher, M., Ritchie, A., & Jorm, L. (2020) · Scientific Reports · doi:10.1038/s41598-020-58053-z · Cited by 86 · see citing works
Advancing Alzheimer’s research: a review of big data promises
Zhang, R., Simon, G., & Yu, F. (2017) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2017.07.002 · Cited by 77 · see citing works
Foy, B. H., Sundt, T. M., Carlson, J. C. T., Aguirre, A. D., & Higgins, J. M. (2022) · Nature Communications · doi:10.1038/s41467-022-32222-2 · Cited by 84 · see citing works
London, A. J. (2022) · Cell Reports Medicine · doi:10.1016/j.xcrm.2022.100622 · Cited by 76 · see citing works
Ebert, D. D., Harrer, M., Apolinário-Hagen, J., & Baumeister, H. (2019) · Advances in Experimental Medicine and Biology · doi:10.1007/978-981-32-9721-0_29 · Cited by 76 · see citing works
Cabitza, F., & Zeitoun, J.-D. (2019) · Annals of Translational Medicine · doi:10.21037/atm.2019.04.07 · Cited by 72 · see citing works
Rethinking clinical trials for medical AI with dynamic deployments of adaptive systems
Rosenthal, J. T., Beecy, A., & Sabuncu, M. R. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01674-3 · Cited by 59 · see citing works
Lin, M.-Y., Li, C.-C., Lin, P.-H., Wang, J.-L., Chan, M.-C., Wu, C.-L., & Chao, W.-C. (2021) · Frontiers in Medicine · doi:10.3389/fmed.2021.663739 · Cited by 45 · see citing works
Dhiman, P., Ma, J., Andaur Navarro, C. L., Speich, B., Bullock, G., Damen, J. A. A., Hooft, L., Kirtley, S., Riley, R. D., Van Calster, B., et al. (2023) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2023.03.012 · Cited by 41 · see citing works
Artificial intelligence for dementia prevention
Newby, D., Orgeta, V., Marshall, C. R., Lourida, I., Albertyn, C. P., Tamburin, S., Raymont, V., Veldsman, M., Koychev, I., Bauermeister, S., et al. (2023) · Alzheimer's & Dementia · doi:10.1002/alz.13463 · Cited by 42 · see citing works
Sampurno, J., Vallaeys, V., Ardianto, R., & Hanert, E. (2022) · Nonlinear Processes in Geophysics · doi:10.5194/npg-29-301-2022 · Cited by 42 · see citing works
Prevalence and Predictability of Low-Yield Inpatient Laboratory Diagnostic Tests
Xu, S., Hom, J., Balasubramanian, S., Schroeder, L. F., Najafi, N., Roy, S., & Chen, J. H. (2019) · JAMA Network Open · doi:10.1001/jamanetworkopen.2019.10967 · Cited by 40 · see citing works
Self-citationTowards evidence-based practice 2.0: leveraging artificial intelligence in healthcare
Nilsen, P., Sundemo, D., Heintz, F., Neher, M., Nygren, J., Svedberg, P., & Petersson, L. (2024) · Frontiers in Health Services · doi:10.3389/frhs.2024.1368030 · Cited by 39 · see citing works
Regional Tree Regularization for Interpretability in Deep Neural Networks
Wu, M., Parbhoo, S., Hughes, M., Kindle, R., Celi, L., Zazzi, M., Roth, V., & Doshi-Velez, F. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i04.6112 · Cited by 38 · see citing works
Sáez, C., Ferri, P., & García-Gómez, J. M. (2024) · Journal of Medical Internet Research · doi:10.2196/50295 · Cited by 36 · see citing works
Dataset artefacts in anti-spoofing systems: A case study on the ASVspoof 2017 benchmark
Chettri, B., Benetos, E., & Sturm, B. L. T. (2020) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2020.3036777 · Cited by 35 · see citing works
Fraile Navarro, D., Kocaballi, A. B., Dras, M., & Berkovsky, S. (2023) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3569893 · Cited by 28 · see citing works
Cho, J., Kim, Y. J., Sunwoo, L., Lee, G. P., Nguyen, T. Q., Cho, S. J., Baik, S. H., Bae, Y. J., Choi, B. S., Jung, C., et al. (2021) · Frontiers in Oncology · doi:10.3389/fonc.2021.739639 · Cited by 30 · see citing works
Wang, Y., Gao, Z., Zhang, Y., Lu, Z., & Sun, F. (2025) · Internal and Emergency Medicine · doi:10.1007/s11739-024-03732-2 · Cited by 29 · see citing works
Toward a Patient-Centered, Data-Driven Cardiology
Ribeiro, A. L., & Oliveira, G. M. M. D. (2019) · Arquivos Brasileiros de Cardiologia · doi:10.5935/abc.20190069 · Cited by 18 · see citing works
Prevent Medical Errors through Artificial Intelligence: A Review
Ahmad, S., & Wasim, S. (2023) · Saudi Journal of Medical and Pharmaceutical Sciences · doi:10.36348/sjmps.2023.v09i07.007 · Cited by 14 · see citing works
Patil, S. S., Jha, G. K., Mohsin, N., Patel, P., Ralte, L., & Shukla, A. K. (2026) · Cureus · doi:10.7759/cureus.109905
- Authors
- Chen, J. H., & Asch, S. M.
- Affiliations
- VA Palo Alto Health Care System (US)
- Venue
- New England Journal of Medicine
- Publisher
- Massachusetts Medical Society
- Published
- 2017-06-29
- Volume
- 376
- Issue
- 26
- Pages
- 2507-2509
- ISSN
- 0028-4793 (ISSN-L)
- 1533-4406
- PMC
- PMC5953825
- OpenAlex
- W2727650337
- Cited by
- 1,171
- References
- 4
- Citations by year
- 2026: 123 · 2025: 177 · 2024: 122 · 2023: 115 · 2022: 130 · 2021: 146 · 2020: 129 · 2019: 107 · 2018: 105 · 2017: 15 · 2012: 1
- FWCI
- 58.05
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- New England Journal of Medicine (publishedVersion)
- PubMed (publishedVersion)
- Europe PMC (PubMed Central) (acceptedVersion)
- PubMed Central (acceptedVersion)
- Funders
- National Institute of Environmental Health Sciences (K01ES026837)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Keywords
- machine learning, medical prediction, big data, health care, hype cycle, technology limitations, clinical decision support
- MeSH terms
- Machine Learning, Algorithms, Probability, Prognosis, Risk Assessment
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
