Targeting repetitive laboratory testing with electronic health records-embedded predictive decision support: A pre-implementation study
Open accessAbstract
Introduction Unnecessary laboratory testing contributes to patient morbidity and healthcare waste. Despite prior attempts at curbing such overutilization, there remains opportunity for improvement using novel data-driven approaches. This study presents the development and early evaluation of a clinical decision support tool that uses a predictive model to help providers reduce low-yield, repetitive laboratory testing in hospitalized patients.Methods We developed an EHR-embedded SMART on FHIR application that utilizes a laboratory test result prediction model based on historical laboratory data. A combination of semi-structured physician interviews, usability testing, and quantitative analysis on retrospective laboratory data were used to inform the tool's development and evaluate its acceptability and potential clinical impact.Key results Physicians identified culture and lack of awareness of repeat orders as key drivers for overuse of inpatient blood testing. Users expressed an openness to a lab prediction model and 13/15 physicians believed the tool would alter their ordering practices. The application received a median System Usability Scale score of 75, corresponding to the 75th percentile of software tools. On average, physicians desired a prediction certainty of 85% before discontinuing a routine recurring laboratory order and a higher certainty of 90% before being alerted. Simulation on historical lab data indicates that filtering based on accepted thresholds could have reduced ∼22% of repeat chemistry panels.Conclusions The use of a predictive algorithm as a means to calculate the utility of a diagnostic test is a promising paradigm for curbing laboratory test overutilization. An EHR-embedded clinical decision support tool employing such a model is a novel and acceptable intervention with the potential to reduce low-yield, repetitive laboratory testing.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Clinician stakeholders indicated that predicting 'clinical stability' of a test was more meaningful than predicting whether a laboratory result would be 'normal'.
“While earlier iterations of model development aimed to predict whether or not a laboratory result would be “normal,” clinician stakeholders indicated that predicting “clinical stability” of a test was more meaningful.14”
From SmartAlert - Implementing Machine Learning-Driven Clinical Decision Support for Inpatient Laboratory Utilization Reduction (2026) · cites this paper as [14]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Palinkas, L. A., Horwitz, S. M., Green, C. A., Wisdom, J. P., Duan, N., & Hoagwood, K. (2015) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-013-0528-y · Cited by 12,038 · see citing works
An Empirical Evaluation of the System Usability Scale
Bangor, A., Kortum, P. T., & Miller, J. T. (2008) · International Journal of Human–Computer Interaction · doi:10.1080/10447310802205776 · Cited by 5,181 · see citing works
The Significance of Saturation
Morse, J. M. (1995) · Qualitative Health Research · doi:10.1177/104973239500500201 · Cited by 2,663 · see citing works
Qualitative methods in implementation research: An introduction
Hamilton, A. B., & Finley, E. P. (2019) · Psychiatry Research · doi:10.1016/j.psychres.2019.112516 · Cited by 1,414 · see citing works
(2013) · Journal of Usability Studies archive · Cited by 1,357 · see citing works
Waste in the US Health Care System
Shrank, W. H., Rogstad, T. L., & Parekh, N. (2019) · JAMA · doi:10.1001/jama.2019.13978 · Cited by 1,071 · see citing works
‘Choosing Wisely’: a growing international campaign
Levinson, W. S., Kallewaard, M., Bhatia, R. S., Wolfson, D. B., Shortt, S., & Kerr, E. A. (2014) · BMJ Quality & Safety · doi:10.1136/bmjqs-2014-003821 · Cited by 654 · see citing works
The Landscape of Inappropriate Laboratory Testing: A 15-Year Meta-Analysis
Zhi, M., Ding, E. L., Theisen-Toupal, J., Whelan, J., & Arnaout, R. (2013) · PLoS ONE · doi:10.1371/journal.pone.0078962 · Cited by 535 · see citing works
Melnick, E. R., Dyrbye, L. N., Sinsky, C. A., Trockel, M., West, C. P., Nedelec, L., Tutty, M. A., & Shanafelt, T. (2020) · Mayo Clinic Proceedings · doi:10.1016/j.mayocp.2019.09.024 · Cited by 413 · see citing works
Usability ratings for everyday products measured with the System Usability Scale
Kortum, P. T., & Bangor, A. (2013) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2012.681221 · Cited by 323 · see citing works
Thavendiranathan, P., Bagai, A., Ebidia, A., Detsky, A. S., & Choudhry, N. K. (2005) · Journal of General Internal Medicine · doi:10.1111/j.1525-1497.2005.0094.x · Cited by 260 · see citing works
Miyakis, S., Karamanof, G., Liontos, M., & Mountokalakis, T. D. (2006) · Postgraduate Medical Journal · doi:10.1136/pgmj.2006.049551 · Cited by 259 · see citing works
Cascade Effects of Medical Technology
Deyo, R. A. (2002) · Annual Review of Public Health · doi:10.1146/annurev.publhealth.23.092101.134534 · Cited by 214 · see citing works
Three myths about risk thresholds for prediction models
on behalf of the Topic Group ‘Evaluating diagnostic tests and prediction models’ of the STRATOS initiative, Wynants, L., van Smeden, M., McLernon, D. J., Timmerman, D., Steyerberg, E. W., & Van Calster, B. (2019) · BMC Medicine · doi:10.1186/s12916-019-1425-3 · Cited by 213 · see citing works
Training Physicians to Provide High-Value, Cost-Conscious Care
Stammen, L. A., Stalmeijer, R. E., Paternotte, E., Pool, A. O., Driessen, E. W., Scheele, F., & Stassen, L. P. S. (2015) · JAMA · doi:10.1001/jama.2015.16353 · Cited by 207 · see citing works
Impact of Providing Fee Data on Laboratory Test Ordering
Feldman, L. S., Shihab, H. M., Thiemann, D. R., Yeh, H., Ardolino, M., Mandell, S. F., & Brotman, D. J. (2013) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2013.232 · Cited by 203 · see citing works
The Impact of Choosing Wisely Interventions on Low‐Value Medical Services: A Systematic Review
Cliff, B. Q., Avanceña, A. L. V., Hirth, R. A., & Lee, S.-Y. D. (2021) · Milbank Quarterly · doi:10.1111/1468-0009.12531 · Cited by 188 · see citing works
Evidence-Based Guidelines to Eliminate Repetitive Laboratory Testing
Eaton, K. P., Levy, K., Soong, C., Pahwa, A. K., Petrilli, C., Ziemba, J. B., Cho, H. J., Alban, R., Blanck, J. F., & Parsons, A. S. (2017) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2017.5152 · Cited by 178 · see citing works
Using Machine Learning to Predict Laboratory Test Results
Luo, Y., Szolovits, P., Dighe, A. S., & Baron, J. M. (2016) · American Journal of Clinical Pathology · doi:10.1093/ajcp/aqw064 · Cited by 162 · see citing works
Population-based study of repeat laboratory testing.
van Walraven, C., & Raymond, M. (2003) · Clinical Chemistry · doi:10.1373/clinchem.2003.021220 · Cited by 137 · see citing works
Applications of machine learning in routine laboratory medicine: Current state and future directions
Rabbani, N., Kim, G. Y. E., Suarez, C. J., & Chen, J. H. (2022) · Clinical Biochemistry · doi:10.1016/j.clinbiochem.2022.02.011 · Cited by 118 · see citing works
Self-citationUse of a Checklist and Clinical Decision Support Tool Reduces Laboratory Use and Improves Cost
Algaze, C. A., Wood, M., Pageler, N. M., Sharek, P. J., Longhurst, C., & Shin, A. Y. (2015) · PEDIATRICS · doi:10.1542/peds.2014-3019 · Cited by 109 · see citing works
Krasowski, M. D., Chudzik, D., Dolezal, A., Steussy, B. W., Gailey, M. P., Koch, B., Kilborn, S. B., Darbro, B. W., Rysgaard, C. D., & Klesney‐Tait, J. (2015) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-015-0137-7 · Cited by 101 · see citing works
Residents' self‐report on why they order perceived unnecessary inpatient laboratory tests
Sedrak, M. S., Patel, M. S., Ziemba, J. B., Murray, D., Kim, E. J., Dine, C. J., & Myers, J. S. (2016) · Journal of Hospital Medicine · doi:10.1002/jhm.2645 · Cited by 100 · see citing works
Utilization management in the clinical laboratory: An introduction and overview of the literature
Huck, A. E., & Lewandrowski, K. B. (2013) · Clinica Chimica Acta · doi:10.1016/j.cca.2013.09.021 · Cited by 85 · see citing works
Olakotan, O. O., & Yusof, M. M. (2020) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2020.103453 · Cited by 75 · see citing works
Morgen, E. K., & Naugler, C. T. (2015) · American Journal of Clinical Pathology · doi:10.1309/ajcpyxdaus2f8xjy · Cited by 69 · see citing works
Pageler, N. M., Franzon, D. E., Longhurst, C., Wood, M., Shin, A. Y., Adams, E. S., Widen, E., & Cornfield, D. N. (2013) · Pediatric Critical Care Medicine · doi:10.1097/pcc.0b013e318272010c · Cited by 56 · see citing works
Felcher, A. H., Gold, R. B., Mosen, D. M., & Stoneburner, A. B. (2017) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocw182 · Cited by 55 · see citing works
Reasons for ordering laboratory tests and relationship with frequency of abnormal results
Houben, P., Winkens, R. A. G., van der Weijden, T., Vossen, R. C. R. M., Naus, A., & Grol, R. P. T. M. (2010) · Scandinavian Journal of Primary Health Care · doi:10.3109/02813430903281758 · Cited by 50 · see citing works
Reducing Unnecessary Imaging and Pathology Tests: A Systematic Review
Hiscock, H., Neely, R. J., Warren, H., Soon, J., & Georgiou, A. (2018) · PEDIATRICS · doi:10.1542/peds.2017-2862 · Cited by 50 · see citing works
Impact of price display on provider ordering: A systematic review
Silvestri, M. T., Bongiovanni, T., Glover, J. G., & Gross, C. P. (2015) · Journal of Hospital Medicine · doi:10.1002/jhm.2500 · Cited by 45 · see citing works
Prevalence and Predictability of Low-Yield Inpatient Laboratory Diagnostic Tests
Xu, S., Hom, J., Balasubramanian, S., Schroeder, L. F., Najafi, N., Roy, S., & Chen, J. H. (2019) · JAMA Network Open · doi:10.1001/jamanetworkopen.2019.10967 · Cited by 40 · see citing works
Self-citationFrom Bad to Worse: Anemia on Admission and Hospital-Acquired Anemia
Koch, C. G., Li, L., Sun, Z., Hixson, E. D., Tang, A. S., Phillips, S. C., Blackstone, E. H., & Henderson, J. M. (2017) · Journal of Patient Safety · doi:10.1097/pts.0000000000000142 · Cited by 39 · see citing works
Appropriate utilization and cost control of the hospital laboratory: panel testing and repeat orders
Werner, M. (1995) · Clinica Chimica Acta · doi:10.1016/0009-8981(94)05964-t · Cited by 31 · see citing works
A novel approach to the determination of clinical decision thresholds
Ebell, M. H., Locatelli, I., & Senn, N. (2015) · BMJ Evidence-Based Medicine · doi:10.1136/ebmed-2014-110140 · Cited by 31 · see citing works
Lidbury, B. A., Richardson, A., & Badrick, T. C. (2015) · Diagnosis · doi:10.1515/dx-2014-0063 · Cited by 27 · see citing works
Reducing Electrolyte Testing in Hospitalized Children by Using Quality Improvement Methods
Tchou, M. J., Girdwood, S. T., Wormser, B., Poole, M., Davis-Rodriguez, S., Caldwell, J. T., Shannon, L., Hagedorn, P. A., Biondi, E., Simmons, J. M., et al. (2018) · PEDIATRICS · doi:10.1542/peds.2017-3187 · Cited by 24 · see citing works
Cascade Effects in Critical Care Medicine: A Call for Practice Changes
Bruce, C. R., Fetter, J. E., & Blumenthal-Barby, J. S. S. (2013) · American Journal of Respiratory and Critical Care Medicine · doi:10.1164/rccm.201309-1606ed · Cited by 22 · see citing works
Reducing Blood Testing in Pediatric Patients After Heart Surgery
Delgado‐Corcoran, C., Bodily, S. A., Frank, D. U., Witte, M. K., Castillo, R., & Bratton, S. L. (2014) · Pediatric Critical Care Medicine · doi:10.1097/pcc.0000000000000194 · Cited by 21 · see citing works
A new method for determining physician decision thresholds using empiric, uncertain recommendations
Boland, M. V., & Lehmann, H. P. (2010) · BMC Medical Informatics and Decision Making · doi:10.1186/1472-6947-10-20 · Cited by 19 · see citing works
An Initiative to Decrease Laboratory Testing in a NICU
Klunk, C. J., Barrett, R. E., Peterec, S. M., Blythe, E., Brockett, R., Kenney, M., Natusch, A., Thursland, C., Gallagher, P. G., Pando, R., et al. (2021) · PEDIATRICS · doi:10.1542/peds.2020-000570 · Cited by 18 · see citing works
Learning decision thresholds for risk stratification models from aggregate clinician behavior
Patel, B., Steinberg, E., Pfohl, S. R., & Shah, N. H. (2021) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocab159 · Cited by 18 · see citing works
Jun, T. W., Kwang, H., Mou, E., Berube, C., Bentley, J., Shieh, L. Y., & Hom, J. (2018) · Journal of General Internal Medicine · doi:10.1007/s11606-018-4663-8 · Cited by 16 · see citing works
Self-citationA Machine Learning Approach to Predicting the Stability of Inpatient Lab Test Results.
Aikens, R. C., Balasubramanian, S., & Chen, J. H. (2019) · PubMed Central · Cited by 14 · see citing works
Self-citationKandalam, V., Lau, C. K. Y., Guo, M., Ma, I. W. Y., & Naugler, C. T. (2019) · Clinical Biochemistry · doi:10.1016/j.clinbiochem.2019.12.011 · Cited by 13 · see citing works
Flagged first, then by citations
Research works printed but not linked: 2
- [28]Rabbani N The Labogram: A data analysis framework for characterizing and reducing redundant laboratory testing in pediatric hospital medicine. American Medical Informatics Association Clinical Informatics Conference; 2022 May 26; Houston, Texas.
- [33]Brooke J SUS: A “quick and dirty” usability scale. In: Usability Evaluation In Industry. Taylor & Francis; 1996. p. 189– 94.
Cited by
Sorted by citations
Corbin, C. K., Maclay, R., Acharya, A., Mony, S., Punnathanam, S., Thapa, R., Kotecha, N., Shah, N. H., & Chen, J. H. (2023) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocad114 · Cited by 59 · see citing works
Self-citationKim, J. Y., Ma, S. P., Chen, M. L., Galatzer‐Levy, I. R., Torous, J. B., van Roessel, P., Sharp, C., Pfeffer, M. A., Rodríguez, C. I., Linos, E., et al. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01969-5 · Cited by 17 · see citing works
Self-citationFHIR in Focus: Enabling Biomedical Data Harmonization for Intelligent Healthcare Systems
Hornback, A. M., Marteau, B. L., Tan, S. M., Kim, K., Patil, O. R., Traynelis, J., Zhu, Y., Giuste, F., & Wang, M. D. (2025) · IEEE Reviews in Biomedical Engineering · doi:10.1109/rbme.2025.3632213 · Cited by 18 · see citing works
Gahane, S., & Verma, P. (2024) · AIP conference proceedings · doi:10.1063/5.0240191 · Cited by 15 · see citing works
Basile, L. J., Carbonara, N., Panniello, U., & Pellegrino, R. (2025) · Management Review Quarterly · doi:10.1007/s11301-024-00482-5 · Cited by 8 · see citing works
Chen, H.-H., Zeng, Y., & Cai, D. (2025) · Frontiers Artif. Intell. · doi:10.3389/frai.2025.1580445 · Cited by 8 · see citing works
Machine learning-based error detection in the clinical laboratory: a critical review
Lin, Y., Mensah, I. K., Doering, M. M., Shean, R. C., & Spies, N. C. (2025) · Critical Reviews in Clinical Laboratory Sciences · doi:10.1080/10408363.2025.2512468 · Cited by 6 · see citing works
Popoff, B., Cabon, S., Cuggia, M., Bouzillé, G., & Clavier, T. (2025) · JMIR Medical Informatics · doi:10.2196/63709 · Cited by 6 · see citing works
“It was developed from instruments already used in the literature to cover all the main axes of reflection relating to the development of a machine learning-based CDSS [20,25-29].”
A review on intelligent information systems for handling electronic child patient health records
Gahane, S., & Verma, P. (2025) · AIP conference proceedings · doi:10.1063/5.0241637 · Cited by 5 · see citing works
S1-Leitlinie Nachhaltigkeit in der Intensiv- und Notfallmedizin
Kochanek, M., Berek, M., Gibb, S., Hermes, C., Hilgarth, H., Janssens, U., Kessel, J. M., Kitz, V. M., Kreutziger, J., Krone, M., et al. (2025) · Medizinische Klinik - Intensivmedizin und Notfallmedizin · doi:10.1007/s00063-025-01261-0 · Cited by 4 · see citing works
Kim, G. Y. E., Schwede, M., Corbin, C. K., Fouladvand, S., Brar, R. S., Iberri, D. J., Shomali, W. E., Oak, J. S., Gratzinger, D. A., Stehr, H., et al. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01816-7 · Cited by 3 · see citing works
Self-citationMa, S. P., Rohatgi, N., & Chen, J. H. (2024) · Journal of Hospital Medicine · doi:10.1002/jhm.13404 · Cited by 3 · see citing works
Self-citationDiagnostic stewardship monitoring systems: institutional and system-level approaches
Park, K. (2026) · Critical Reviews in Clinical Laboratory Sciences · doi:10.1080/10408363.2026.2666326 · Cited by 2 · see citing works
Mathura, P. V., Grasdal, M. C., Marini, S., Podder, M., & Kassam, N. (2025) · Journal of General Internal Medicine · doi:10.1007/s11606-025-09513-7 · Cited by 1 · see citing works
Zha, L. (2026) · Critical Reviews in Clinical Laboratory Sciences · doi:10.1080/10408363.2026.2712373 · Cited by 1 · see citing works
Dziorny, A. C., Jones, C., Salant, J. A., Kubis, S. E., Zand, M. S., Wolfe, H. A., & Srinivasan, V. (2023) · Pediatric Critical Care Medicine · doi:10.1097/pcc.0000000000003287 · Cited by 1 · see citing works
Probabilistic Prediction of Laboratory Test Information Yield.
Jiang, Y., Lee, A. H., Ni, X., Corbin, C. K., Irvin, J., Ng, A. Y., & Chen, J. H. (2023) · PubMed Central · Cited by 1 · see citing works
Self-citationLiang, A. S., Amrollahi, F., Jiang, Y., Corbin, C. K., Kim, G. Y. E., Mui, D., Crowell, T., Acharya, A., Mony, S., Punnathanam, S., et al. (2026) · NEJM AI · doi:10.1056/aics2501086
Self-citationDeshwal, P. R., & Tiwari, P. (2025) · International Journal of Risk & Safety in Medicine · doi:10.1177/09246479251337933
Sustainability and waste reduction in clinical laboratories
Ejilemele, A. A., & Singh, I. R. (2026) · Critical Reviews in Clinical Laboratory Sciences · doi:10.1080/10408363.2026.2687509
- Authors
- Rabbani, N., Ma, S. P., Li, R. C., Winget, M., Weber, S., Boosi, S., Pham, T. D., Svec, D., Shieh, L., & Chen, J. H.
- Affiliations
- Stanford Medicine (US)
- Corresponding author
- Rabbani, Naveed
- ORCID
- Venue
- Clinical Biochemistry
- Publisher
- Elsevier BV
- Published
- 2023-03
- Volume
- 113
- Pages
- 70-77
- ISSN
- 0009-9120 (ISSN-L)
- PMC
- PMC9936847
- OpenAlex
- W4313627365
- Cited by
- 20
- References
- 46
- Citations by year
- 2026: 4 · 2025: 11 · 2024: 2 · 2023: 3
- FWCI
- 4.48
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- Locations
- Clinical Biochemistry (acceptedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- Funders
- Canada Excellence Research Chairs, Government of Canada
- National Institutes of Health
- National Institute on Aging (P30AG059307, P30AG066515)
- National Institute of Allergy and Infectious Diseases (R01AI178121)
- U.S. National Library of Medicine (R56LM013365)
- Topics
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Health Sciences › Health Professions › General Health Professions › Healthcare cost, quality, practices
- Health Sciences › Medicine › Physiology › Clinical Laboratory Practices and Quality Control
- Keywords
- clinical decision support, predictive modeling, electronic health records, hospitalized patients, physician acceptability, usability, SMART on FHIR, diagnostic test utility
- MeSH terms
- Computer Simulation, Humans, Physicians, Retrospective Studies, Software, Decision Support Systems, Clinical, Electronic Health Records
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
