Path From Predictive Analytics to Improved Patient Outcomes: A Framework to Guide Use, Implementation, and Evaluation of Accurate Surgical Predictive Models
Open accessThe DOI matches its Crossref record; 1 of the 1 references it cites were found in a registry or an index.
Abstract
An entire industry is booming on the promise that electronic health predictive analytics (e-HPA) can improve surgical outcomes by, for example, predicting whether a procedure is likely to benefit a patient compared with alternative treatments, or if a patient will experience short or long-term complications.1 The hope is that surgeons can use model predictions to improve the continuum of surgical care including patient selection, informed consent, shared decision making, preoperative risk modification, and perioperative management with the ultimate goal of producing better outcomes. Although so much effort is expended developing and selling predictive models, almost no attention is paid to rigorously testing if patients treated in health care settings that implement e-HPA have better outcomes than those treated in absence of e-HPA. Therefore, practicing surgeons need to be skeptical regarding the inherent benefits of e-HPA, as should model developers, implementers, and patients. Before buying or using an e-HPA system, all stakeholders should insist on research that rigorously evaluates not only the accuracy of the predictive models, but their effects on health outcomes when used in particular ways in real clinical settings. Without evidence of effectiveness and a careful examination of unintended consequences, implementation of e-HPA systems may produce more risks and/or costs than benefits. Before describing the many ways even accurate and technically integrated e-HPA can fail to produce benefits to patients, we should review some characteristics of good predictive models, without which there is no hope. Good predictive models need to include important and modifiable outcomes, be adequately accurate (eg, sensitive, specific) given the clinical context, only include inputs that will be available at the time of decision or possible intervention, be cross-validated in data not used to develop the model, and be implemented in a usable and accessible technology platform. Moreover, model developers should be able and willing to transparently share the details of the model development methodology, model performance metrics, and model coefficients to facilitate replication, comparisons with alternative models, and refinements. The Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD) standards and the recent Consensus Statement on E-HPA elaborate these points.2,3 Unfortunately, a disappointing number of publically available predictive models meet these criteria.4 However, developing good predictive models that meet these criteria is just the beginning of successful implementation and perhaps easier than the downstream challenges to which we now turn. In rare cases when a particular implementation of e-HPA is evaluated and shown to produce real improvements in treatment quality and outcomes,5 it is generally under-appreciated how many intervening steps happened to translate model predictions into improved health. In prominent cases where the anticipated benefits of e-HPA failed to materialize,6 people are left wondering what went wrong along the pathway between accurate predictions and patient outcomes. Figure 1 represents a framework to better understand the conditions and events that must occur to enable good predictive models to create real value and benefit in the surgical context. The framework has a least 2 purposes. First, the framework is designed to sensitize all stakeholders, especially practicing surgeons, to the many implementation challenges that follow model development and technical integration. Second, the framework provides guidance to researchers and administrators who seek to rigorously test the clinical impacts of e-HPA—not only to estimate overall impacts, but to identify processes that might have broken down if no effects are observed.FIGURE 1: Framework to Guide Use, Implementation, and Evaluation of Accurate Surgical Predictive Models.Condition 1 is that model outputs need to be accessed by someone who has the potential to act in beneficial ways. Prediction models are not helpful if nobody knows about them. Placing a risk calculator on a SharePoint Site or even as a pop-up in the electronic medical record does not ensure that potential users know of the model's existence or will interact with it. Putting systems in place to track interactions with the model (eg, who, where, for which patients) is critical to understanding downstream clinical impacts or lack thereof. Condition 2 is that models must produce information not already known to users. Thus, models can be accurate, but unhelpful unless they provide new insights. The best models accurately predict events that were not expected, accurately predict the absence of expected events, or help clinical staff refine the probability of events in patients with less certain risk, perhaps for informed consent or shared decision making. Condition 3 is that model outputs need to be received by some
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Collins, G. S., Reitsma, J. B., Altman, D. G., & Moons, K. G. M. (2015) · Annals of Internal Medicine · doi:10.7326/l15-5093-2 · Cited by 493
Cited by
Lee, T. C., Shah, N. U., Haack, A., & Baxter, S. L. (2020) · Informatics · doi:10.3390/informatics7030025 · Cited by 133
Watson, J., Hutyra, C. A., Clancy, S. M., Chandiramani, A., Bedoya, A., Ilangovan, K., Nderitu, N., & Poon, E. G. (2020) · JAMIA Open · doi:10.1093/jamiaopen/ooz046 · Cited by 95
Barriers to Implementing an Artificial Intelligence Model for Unplanned Readmissions
Baxter, S. L., Bass, J. S., & Sitapati, A. M. (2020) · ACI Open · doi:10.1055/s-0040-1716748 · Cited by 30
Metadata
- Authors
- Harris, A. H.
- Venue
- Annals of Surgery
- Published
- 2017-03
- Volume
- 265
- Issue
- 3
- Pages
- 461-463
- ISSN
- 0003-4932 (ISSN-L)
- OpenAlex
- W2539405816
- Cited by
- 37
- References
- 1
- FWCI
- 1.40
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Path From Predictive Analytics to Improved Patient Outcomes: A Framework to Guide Use, Implementation, and Evaluation of Accurate Surgical Predictive Models. https://app.cochat.ai/papers/10.1097/sla.0000000000002023
- The paper
- Harris, A. H. (2017). Path From Predictive Analytics to Improved Patient Outcomes: A Framework to Guide Use, Implementation, and Evaluation of Accurate Surgical Predictive Models. Annals of Surgery. https://doi.org/10.1097/sla.0000000000002023
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.