CoChat

Path From Predictive Analytics to Improved Patient Outcomes: A Framework to Guide Use, Implementation, and Evaluation of Accurate Surgical Predictive Models

Open access
Harris, Alex HS. · 2017 · Annals of Surgery
Cited by 37

The DOI matches its Crossref record; 1 of the 1 references it cites were found in a registry or an index.

Abstract

An entire industry is booming on the promise that electronic health predictive analytics (e-HPA) can improve surgical outcomes by, for example, predicting whether a procedure is likely to benefit a patient compared with alternative treatments, or if a patient will experience short or long-term complications.1 The hope is that surgeons can use model predictions to improve the continuum of surgical care including patient selection, informed consent, shared decision making, preoperative risk modification, and perioperative management with the ultimate goal of producing better outcomes. Although so much effort is expended developing and selling predictive models, almost no attention is paid to rigorously testing if patients treated in health care settings that implement e-HPA have better outcomes than those treated in absence of e-HPA. Therefore, practicing surgeons need to be skeptical regarding the inherent benefits of e-HPA, as should model developers, implementers, and patients. Before buying or using an e-HPA system, all stakeholders should insist on research that rigorously evaluates not only the accuracy of the predictive models, but their effects on health outcomes when used in particular ways in real clinical settings. Without evidence of effectiveness and a careful examination of unintended consequences, implementation of e-HPA systems may produce more risks and/or costs than benefits. Before describing the many ways even accurate and technically integrated e-HPA can fail to produce benefits to patients, we should review some characteristics of good predictive models, without which there is no hope. Good predictive models need to include important and modifiable outcomes, be adequately accurate (eg, sensitive, specific) given the clinical context, only include inputs that will be available at the time of decision or possible intervention, be cross-validated in data not used to develop the model, and be implemented in a usable and accessible technology platform. Moreover, model developers should be able and willing to transparently share the details of the model development methodology, model performance metrics, and model coefficients to facilitate replication, comparisons with alternative models, and refinements. The Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD) standards and the recent Consensus Statement on E-HPA elaborate these points.2,3 Unfortunately, a disappointing number of publically available predictive models meet these criteria.4 However, developing good predictive models that meet these criteria is just the beginning of successful implementation and perhaps easier than the downstream challenges to which we now turn. In rare cases when a particular implementation of e-HPA is evaluated and shown to produce real improvements in treatment quality and outcomes,5 it is generally under-appreciated how many intervening steps happened to translate model predictions into improved health. In prominent cases where the anticipated benefits of e-HPA failed to materialize,6 people are left wondering what went wrong along the pathway between accurate predictions and patient outcomes. Figure 1 represents a framework to better understand the conditions and events that must occur to enable good predictive models to create real value and benefit in the surgical context. The framework has a least 2 purposes. First, the framework is designed to sensitize all stakeholders, especially practicing surgeons, to the many implementation challenges that follow model development and technical integration. Second, the framework provides guidance to researchers and administrators who seek to rigorously test the clinical impacts of e-HPA—not only to estimate overall impacts, but to identify processes that might have broken down if no effects are observed.FIGURE 1: Framework to Guide Use, Implementation, and Evaluation of Accurate Surgical Predictive Models.Condition 1 is that model outputs need to be accessed by someone who has the potential to act in beneficial ways. Prediction models are not helpful if nobody knows about them. Placing a risk calculator on a SharePoint Site or even as a pop-up in the electronic medical record does not ensure that potential users know of the model's existence or will interact with it. Putting systems in place to track interactions with the model (eg, who, where, for which patients) is critical to understanding downstream clinical impacts or lack thereof. Condition 2 is that models must produce information not already known to users. Thus, models can be accurate, but unhelpful unless they provide new insights. The best models accurately predict events that were not expected, accurately predict the absence of expected events, or help clinical staff refine the probability of events in patients with less certain risk, perhaps for informed consent or shared decision making. Condition 3 is that model outputs need to be received by some

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1097/sla.0000000000002023

Cites

1 of 1 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 1 references

Cited by

See all 37

Metadata

Authors
Harris, A. H.
Venue
Annals of Surgery
Published
2017-03
Volume
265
Issue
3
Pages
461-463
ISSN
0003-4932 (ISSN-L)
OpenAlex
W2539405816
Cited by
37
References
1
FWCI
1.40
Type
journal article
Language
en
Open access
green

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Path From Predictive Analytics to Improved Patient Outcomes: A Framework to Guide Use, Implementation, and Evaluation of Accurate Surgical Predictive Models. https://app.cochat.ai/papers/10.1097/sla.0000000000002023
The paper
Harris, A. H. (2017). Path From Predictive Analytics to Improved Patient Outcomes: A Framework to Guide Use, Implementation, and Evaluation of Accurate Surgical Predictive Models. Annals of Surgery. https://doi.org/10.1097/sla.0000000000002023

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.