“How did you get to this number?” Stakeholder needs for implementing predictive analytics: a pre-implementation qualitative study
Open accessThe DOI matches its Crossref record; 8 of the 8 references it cites were found in a registry or an index.
Abstract
Objective: Predictive analytics are potentially powerful tools, but to improve healthcare delivery, they must be carefully integrated into healthcare organizations. Our objective was to identify facilitators, challenges, and recommendations for implementing a novel predictive algorithm which aims to prospectively identify patients with high preventable utilization to proactively involve them in preventative interventions. Materials and Methods: In preparation for implementing the predictive algorithm in 3 organizations, we interviewed 3 stakeholder groups: health systems operations (eg, chief medical officers, department chairs), informatics personnel, and potential end users (eg, physicians, nurses, social workers). We applied thematic analysis to derive key themes and categorize them into the dimensions of Sittig and Singh’s original sociotechnical model for studying health information technology in complex adaptive healthcare systems. Recruiting and analysis were conducted iteratively until thematic saturation was achieved. Results: Forty-nine interviews were conducted in 3 healthcare organizations. Technical components of the implementation (hardware and software) raised fewer concerns than alignment with sociotechnical factors. Stakeholders wanted decision support based on the algorithm to be clear and actionable and incorporated into current workflows. However, how to make this disease-independent classification tool actionable was perceived as a challenge, and appropriate patient interventions informed by the algorithm appeared likely to require substantial external and institutional resources. Stakeholders also described the criticality of trust, credibility, and interpretability of the predictive algorithm. Conclusions: Although predictive analytics can classify patients with high accuracy, they cannot advance healthcare processes and outcomes without careful implementation that takes into account the sociotechnical system. Key stakeholders have strong perceptions about facilitators and challenges to shape successful implementation.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Scalable and accurate deep learning with electronic health records
Rajkomar, A., Oren, E., Chen, K., Dai, A. M., Hajaj, N., Hardt, M., Liu, P. J., Liu, X., Marcus, J., Sun, M., et al. (2018) · npj Digital Medicine · doi:10.1038/s41746-018-0029-1 · Cited by 2,648
Self-citationBig Data In Health Care: Using Analytics To Identify And Manage High-Risk And High-Cost Patients
Bates, D. W., Saria, S., Ohno-Machado, L., Shah, A., & Escobar, G. (2014) · Health Affairs · doi:10.1377/hlthaff.2014.0041 · Cited by 1,235
Sittig, D. F., & Singh, H. (2010) · Quality and Safety in Health Care · doi:10.1136/qshc.2010.042085 · Cited by 897 · cited 2×
“23 We describe each key takeaway in detail below.”
BackgroundMethodA targeted real-time early warning score (TREWScore) for septic shock.
Henry, K. E., Hager, D. N., Pronovost, P. J., & Saria, S. (2015) · Science Translational Medicine · doi:10.1126/scitranslmed.aab3719 · Cited by 598
Improving Outcomes with Clinical Decision Support: An Implementer's Guide, Second Edition
Osheroff, J. (2012) · doi:10.4324/9781498757461 · Cited by 517
Levin, S., Toerper, M., Hamrock, E., Hinson, J. S., Barnes, S., Gardner, H., Dugas, A., Linton, B., Kirsch, T., & Kelen, G. (2018) · Annals of Emergency Medicine · doi:10.1016/j.annemergmed.2017.08.005 · Cited by 424
Making Machine Learning Models Clinically Useful
Shah, N. H., Milstein, A., & Bagley, P. (2019) · JAMA · doi:10.1001/jama.2019.10306 · Cited by 294 · cited 1×
“Recent predictive algorithms have demonstrated a superior ability to classify patients, but progress has been slow in leveraging classification power to improve outcomes.(17) A tool to identify HNHC patients might pose challenges for healthcare organizations because the optimal action for these patients may not be obv…”
Participatory design of probability-based decision support tools for in-hospital nurses
Jeffery, A. D., Novak, L. L., Kennedy, B., Dietrich, M. S., & Mion, L. C. (2017) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocx060 · Cited by 46
Cited by
Hogg, H. D. J., Al-Zubaidy, M., Technology Enhanced Macular Services Study Reference Group, Talks, J., Denniston, A. K., Kelly, C. J., Malawana, J., Papoutsi, C., Teare, M. D., Keane, P. A., et al. (2023) · Journal of Medical Internet Research · doi:10.2196/39742 · Cited by 163
Lee, T. C., Shah, N. U., Haack, A., & Baxter, S. L. (2020) · Informatics · doi:10.3390/informatics7030025 · Cited by 133
Trust in AI: why we should be designing for APPROPRIATE reliance
Benda, N. C., Novak, L. L., Reale, C., & Ancker, J. S. (2021) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocab238 · Cited by 76
Self-citationMatthiesen, S., Diederichsen, S. Z., Hansen, M. K. H., Villumsen, C., Lassen, M. C. H., Jacobsen, P. K., Risum, N., Winkel, B. G., Philbert, B. T., Svendsen, J. H., et al. (2021) · JMIR Human Factors · doi:10.2196/26964 · Cited by 45
van der Meijden, S. L., de Hond, A. A. H., Thoral, P. J., Steyerberg, E. W., Kant, I. M. J., Cinà, G., & Arbous, M. S. (2023) · JMIR Human Factors · doi:10.2196/39114 · Cited by 39
Metadata
- Authors
- Benda, N. C., Das, L. T., Abramson, E. L., Blackburn, K., Thoman, A., Kaushal, R., Zhang, Y., & Ancker, J. S.
- Venue
- Journal of the American Medical Informatics Association
- Publisher
- Oxford University Press (OUP)
- Published
- 2020-03-06
- Volume
- 27
- Issue
- 5
- Pages
- 709-716
- ISSN
- 1067-5027 (ISSN-L)
- 1527-974X
- OpenAlex
- W3011882394
- Cited by
- 45
- References
- 8
- FWCI
- 1.61
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: “How did you get to this number?” Stakeholder needs for implementing predictive analytics: a pre-implementation qualitative study. https://app.cochat.ai/papers/10.1093/jamia/ocaa021
- The paper
- Benda, N. C., Das, L. T., Abramson, E. L., Blackburn, K., Thoman, A., Kaushal, R., Zhang, Y., & Ancker, J. S. (2020). “How did you get to this number?” Stakeholder needs for implementing predictive analytics: a pre-implementation qualitative study. Journal of the American Medical Informatics Association. https://doi.org/10.1093/jamia/ocaa021
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.