Predicting self-intercepted medication ordering errors using machine learning
Open accessAbstract
Current approaches to understanding medication ordering errors rely on relatively small manually captured error samples. These approaches are resource-intensive, do not scale for computerized provider order entry (CPOE) systems, and are likely to miss important risk factors associated with medication ordering errors. Previously, we described a dataset of CPOE-based medication voiding accompanied by univariable and multivariable regression analyses. However, these traditional techniques require expert guidance and may perform poorly compared to newer approaches. In this paper, we update that analysis using machine learning (ML) models to predict erroneous medication orders and identify its contributing factors. We retrieved patient demographics (race/ethnicity, sex, age), clinician characteristics, type of medication order (inpatient, prescription, home medication by history), and order content. We compared logistic regression, random forest, boosted decision trees, and artificial neural network models. Model performance was evaluated using area under the receiver operating characteristic curve (AUROC) and the area under the precision-recall curve (AUPRC). The dataset included 5,804,192 medication orders, of which 28,695 (0.5%) were voided. ML correctly classified voids at reasonable accuracy; with a positive predictive value of 10%, ~20% of errors were included. Gradient boosted decision trees achieved the highest AUROC (0.7968) and AUPRC (0.0647) among all models. Logistic regression had the poorest performance. Models identified predictive factors with high face validity (e.g., student orders), and a decision tree revealed interacting contexts with high rates of errors not identified by previous regression models. Prediction models using order-entry information offers promise for error surveillance, patient safety improvements, and targeted clinical review. The improved performance of models with complex interactions points to the importance of contextual medication ordering information for understanding contributors to medication errors.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
King et al. decision trees using gradient boost achieved the highest AUROC (0.7968) and AUPRC (0.0647) among all models for predicting medication ordering errors.
“Decision trees using gradient boost achieved the highest AUROC (0.7968) and AUPRC (0.0647) among all models, showing promise for error surveillance, patient safety improvement, and targeted clinical review.”
From Artificial intelligence in healthcare: transforming patient safety with intelligent systems—A systematic review (2025) · cites this paper as [54]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · see citing works
SMOTE synthetic minority over-sampling technique
Chawla, N. V., Bowyer, K. W., Hall, L. O., & Kegelmeyer, W. P. (2002) · Journal of Artificial Intelligence Research · doi:10.1613/jair.953 · Cited by 32,865 · see citing works
A Unified Approach to Interpreting Model Predictions
Lundberg, S., & Lee, S.-I. (2017) · arXiv · doi:10.48550/arxiv.1705.07874 · Cited by 23,707 · see citing works
Distilling the Knowledge in a Neural Network
Hinton, G., Vinyals, O., & Dean, J. (2015) · arXiv · doi:10.48550/arxiv.1503.02531 · Cited by 14,204 · see citing works
From local explanations to global understanding with explainable AI for trees
Lundberg, S. M., Erion, G., Chen, H., DeGrave, A., Prutkin, J. M., Nair, B., Katz, R., Himmelfarb, J., Bansal, N., & Lee, S.-I. (2020) · Nature Machine Intelligence · doi:10.1038/s42256-019-0138-9 · Cited by 10,551 · see citing works
The relationship between Precision-Recall and ROC curves
Davis, J., & Goadrich, M. (2006) · Proceedings of the 23rd international conference on Machine learning - ICML '06 · doi:10.1145/1143844.1143874 · Cited by 6,559 · see citing works
Striving for Simplicity: The All Convolutional Net
Springenberg, J. T., Dosovitskiy, A., Brox, T., & Riedmiller, M. (2014) · arXiv · doi:10.48550/arxiv.1412.6806 · Cited by 3,447 · see citing works
Bates, D. W. (1998) · JAMA · doi:10.1001/jama.280.15.1311 · Cited by 2,107 · see citing works
Explainable machine-learning predictions for the prevention of hypoxaemia during surgery
Lundberg, S. M., Nair, B., Vavilala, M. S., Horibe, M., Eisses, M. J., Adams, T., Liston, D. E., Low, D. K.-W., Newman, S.-F., Kim, J., et al. (2018) · Nature Biomedical Engineering · doi:10.1038/s41551-018-0304-0 · Cited by 1,967 · see citing works
Consistent Individualized Feature Attribution for Tree Ensembles
Lundberg, S. M., Erion, G. G., & Lee, S.-I. (2018) · arXiv · doi:10.48550/arxiv.1802.03888 · Cited by 1,430 · see citing works
The Impact of Computerized Physician Order Entry on Medication Error Prevention
Bates, D. W., Teich, J. M., Lee, J., Seger, D., Kuperman, G. J., Ma'Luf, N., Boyle, D., & Leape, L. (1999) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.1999.00660313 · Cited by 1,260 · see citing works
Ammenwerth, E., Schnell‐Inderst, P., Machan, C., & Siebert, U. (2008) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m2667 · Cited by 669 · see citing works
Using information technology to reduce rates of medication errors in hospitals
Bates, D. W. (2000) · BMJ · doi:10.1136/bmj.320.7237.788 · Cited by 487 · see citing works
Computer Physician Order Entry: Benefits, Costs, and Issues
Kuperman, G. J., & Gibson, R. F. (2003) · Annals of Internal Medicine · doi:10.7326/0003-4819-139-1-200307010-00010 · Cited by 438 · see citing works
Computerized Physician Order Entry and Medication Errors in a Pediatric Critical Care Unit
Potts, A. L., Barr, F. E., Gregory, D. F., Wright, L., & Patel, N. R. (2004) · Pediatrics · doi:10.1542/peds.113.1.59 · Cited by 410 · see citing works
Nuckols, T. K., Smith-Spangler, C., Morton, S. C., Asch, S. M., Patel, V. M., Anderson, L. J., Deichsel, E. L., & Shekelle, P. G. (2014) · Systematic Reviews · doi:10.1186/2046-4053-3-56 · Cited by 340 · see citing works
CBNet: A Novel Composite Backbone Network Architecture for Object Detection
Liu, Y., Wang, Y., Wang, S., Liang, T., Zhao, Q., Tang, Z., & Ling, H. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.6834 · Cited by 280 · see citing works
Reckmann, M., Westbrook, J. I., Koh, Y., Lo, C., & Day, R. O. (2009) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m3050 · Cited by 266 · see citing works
van Rosse, F., Maat, B., Rademaker, C. M. A., van Vught, A. J., Egberts, T. C. G., & Bollen, C. W. (2009) · PEDIATRICS · doi:10.1542/peds.2008-1494 · Cited by 265 · see citing works
Systematic Review of Medication Errors in Pediatric Patients
Ghaleb, M. A., Barber, N., Franklin, B. D., Yeung, V. W., Khaki, Z. F., & Wong, I. C. K. (2006) · Annals of Pharmacotherapy · doi:10.1345/aph.1g717 · Cited by 256 · see citing works
Prgomet, M., Li, L., Niazkhani, Z., Georgiou, A., & Westbrook, J. I. (2016) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocw145 · Cited by 243 · see citing works
Barriers to reporting medication errors and near misses among nurses: A systematic review
Vrbnjak, D., Denieffe, S., O’Gorman, C., & Pajnkihar, M. (2016) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2016.08.019 · Cited by 242 · see citing works
Westbrook, J. I., Reckmann, M., Li, L., Runciman, W. B., Burke, R. A., Lo, C., Baysari, M. T., Braithwaite, J., & Day, R. O. (2012) · PLoS Medicine · doi:10.1371/journal.pmed.1001164 · Cited by 216 · see citing works
Impact of Computerized Physician Order Entry on Clinical Practice in a Newborn Intensive Care Unit
Cordero, L., Kuehn, L., Kumar, R. R., & Mekhjian, H. S. (2004) · Journal of Perinatology · doi:10.1038/sj.jp.7211000 · Cited by 204 · see citing works
Schiff, G. D., Amato, M. G., Eguale, T., Boehne, J. J., Wright, A., Koppel, R., Rashidee, A. H., Elson, R. B., Whitney, D. L., Thach, T.-T., et al. (2015) · BMJ Quality & Safety · doi:10.1136/bmjqs-2014-003555 · Cited by 138 · see citing works
van Doormaal, J. E., van den Bemt, P. M. L. A., Zaal, R. J., Egberts, T. C. G., Lenderink, B. W., Kosterink, J. G. W., Haaijer‐Ruskamp, F. M., & Mol, P. G. M. (2009) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m3099 · Cited by 131 · see citing works
Qin, G., & Hotilovac, L. (2007) · Statistical Methods in Medical Research · doi:10.1177/0962280207087173 · Cited by 126 · see citing works
Factors contributing to an increase in duplicate medication order errors after CPOE implementation
Wetterneck, T. B., Walker, J. M., Blosky, M. A., Cartmill, R. S., Hoonakker, P., Johnson, M., Norfolk, E. R., & Carayon, P. (2011) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2011-000255 · Cited by 108 · see citing works
Korb‐Savoldelli, V., Boussadi, A., Durieux, P., & Sabatier, B. (2017) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2017.12.022 · Cited by 64 · see citing works
Electronic prescribing reduces prescribing error in public hospitals
Shawahna, R., Rahman, N.-U., Ahmad, M., Debray, M., Yliperttula, M. L., & Declèves, X. (2011) · Journal of Clinical Nursing · doi:10.1111/j.1365-2702.2011.03714.x · Cited by 57 · see citing works
Abraham, J., Kannampallil, T. G., Jarman, A., Sharma, S., Rash, C., Schiff, G., & Galanter, W. (2018) · BMJ Quality & Safety · doi:10.1136/bmjqs-2017-006606 · Cited by 22 · see citing works
Self-citationLearning from errors: analysis of medication order voiding in CPOE systems
Kannampallil, T. G., Abraham, J., Solotskaya, A., Philip, S. G., Lambert, B. L., Schiff, G. D., Wright, A., & Galanter, W. L. (2017) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocw187 · Cited by 18 · see citing works
Self-citationcgam: An R Package for the Constrained Generalized Additive Model
Liao, X., & Meyer, M. C. (2019) · Journal of Statistical Software · doi:10.18637/jss.v089.i05 · Cited by 17 · see citing works
Risk factors associated with medication ordering errors
Abraham, J., Galanter, W., Touchette, D. R., Xia, Y., Holzer, K. J., Leung, V., & Kannampallil, T. (2020) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaa264 · Cited by 16 · see citing works
Self-citationHickman, T.-T. T., Quist, A., Salazar, A., Amato, M. G., Wright, A. T., Volk, L. A., Bates, D. W., & Schiff, G. D. (2017) · BMJ Quality & Safety · doi:10.1136/bmjqs-2017-006597 · Cited by 11 · see citing works
Flagged first, then by citations
1 of the 36 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
De Micco, F., Di Palma, G., Ferorelli, D., De Benedictis, A., Tomassini, L., Tambone, V., Cingolani, M., & Scendoni, R. (2025) · Frontiers in Medicine · doi:10.3389/fmed.2024.1522554 · Cited by 58 · see citing works
Risk Management and Patient Safety in the Artificial Intelligence Era: A Systematic Review
Ferrara, M., Bertozzi, G., Fazio, N. D., Aquila, I., Fazio, A. D., Maiese, A., Volonnino, G., Frati, P., & Russa, R. L. (2024) · Healthcare · doi:10.3390/healthcare12050549 · Cited by 62 · see citing works
Deimazar, G., & Sheikhtaheri, A. (2023) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2023.105246 · Cited by 45 · see citing works
Johns, E., Godet, J., Alkanj, A., Beck, M., Mas, L. D., Gourieux, B., Sauleau, E., & Michel, B. (2023) · European Journal of Hospital Pharmacy · doi:10.1136/ejhpharm-2023-003857 · Cited by 22 · see citing works
Yi, M., Cao, Y., Yujin, Z., Cao, Y., Zheng, X., Wang, J., Chen, W., Wei, L., & Zhang, K. (2022) · Frontiers in Public Health · doi:10.3389/fpubh.2022.993946 · Cited by 11 · see citing works
Detecting Unusual Intravenous Infusion Alerting Patterns with Machine Learning Algorithms
Obuseh, M., Yu, D., & DeLaurentis, P.-C. C. (2022) · Biomedical Instrumentation & Technology · doi:10.2345/0899-8205-56.2.58 · Cited by 7 · see citing works
Analytical approaches for medication management for patient safety: a scoping review
Yao, X., Rao, A. A., & Padman, R. (2025) · npj Health Systems · doi:10.1038/s44401-025-00052-1 · Cited by 3 · see citing works
Machine learning for medication error detection: a scoping review protocol
Hêche, F., Yazdani, A., Ferdowsi, S., Kabak, R., Mu, G., & Teodoro, D. (2026) · BMJ Open · doi:10.1136/bmjopen-2025-109192 · Cited by 1 · see citing works
Cohen, T., Patel, V. L., & Shortliffe, E. H. (2022) · Cognitive informatics in biomedicine and healthcare · doi:10.1007/978-3-031-09108-7_20
Embracing the electronic era: the role of digital prescribing solutions in paediatrics
Hassan, H. (2024) · Archives of Disease in Childhood · doi:10.1136/archdischild-2024-326898
Analytical approaches for medication reconciliation-related topics: a scoping reviewPreprint
Yao, X., Rao, A. A., & Padman, R. (2025) · medRxiv · doi:10.1101/2025.03.16.25324060
Sodikin, M. I., & Utami, E. (2025) · sudo Jurnal Teknik Informatika · doi:10.56211/sudo.v4i2.819
Machine learning for medication error detection: a scoping review protocolPreprint
Hêche, F., Yazdani, A., Ferdowsi, S., Kabak, R., Mu, G., & Teodoro, D. (2025) · medRxiv · doi:10.1101/2025.08.06.25333103
MIMIC-RxBench: Benchmarking Large Language Models for Prescription Error Classification
Zu, S., Li, Y., Zhan, Y., & Tao, D. (2026) · Lecture notes in computer science · doi:10.1007/978-981-95-5722-6_11
Machine learning for medication error detection: a scoping reviewPreprint
Hêche, F., Ferdowsi, S., Yazdani, A., Sansaloni-Pastor, S., & Teodoro, D. (2026) · Research Square · doi:10.21203/rs.3.rs-8919709/v1
Smart Medication Management System with Face Recognition and Automated Pills Dispenser
Agnes, A., Evangelin Emeema, S., & Nirosha, R. (2026) · IEEE International Conference on Quantum Photonics, Artificial Intelligence & Networking (QPAIN) · doi:10.1109/qpain69676.2026.11545694
Ali, A. B., Gharbi, S., & Jebali, N. (2025) · Archives of Pharmacy Practice · doi:10.51847/dk74gof42o
Wiśniewski, K., Nowak, M., & Adamczyk, T. (2025) · Archives of Pharmacy Practice · doi:10.51847/komxgftaby
Khalid, O., & Nadeem, S. (2024) · Journal of Health Informatics and Digital Systems · doi:10.68159/y683892568
Al-Farsi, M., Al-Harthy, S., & Al-Rawahi, N. (2025) · Journal of Health Informatics and Digital Systems · doi:10.68159/w988431769
Rahman, S., Zaki, A., Huda, N., Faisal, A., & Wei, L. (2024) · Journal of Health Informatics and Digital Systems · doi:10.68159/l065530971
Gobran, Y. (2026) · Research in Social and Administrative Pharmacy · doi:10.1016/j.sapharm.2026.09.009
- Authors
- King, C. R., Abraham, J., Fritz, B. A., Cui, Z., Galanter, W., Chen, Y., & Kannampallil, T.
- Affiliations
- Washington University in St. Louis (US)
- University of Illinois Chicago (US)
- Corresponding author
- Kannampallil, Thomas
- ORCID
- Venue
- PLOS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2021-07-14
- Volume
- 16
- Issue
- 7
- Pages
- e0254358
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC8279397
- OpenAlex
- W3177830211
- Cited by
- 22
- References
- 36
- Citations by year
- 2026: 5 · 2025: 8 · 2024: 4 · 2023: 2 · 2022: 3
- FWCI
- 2.75
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS ONE (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Digital Commons@Becker (Washington University School of Medicine) (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Agency for Healthcare Research and Quality (R21HS025443)
- Topics
- Health Sciences › Health Professions › Emergency Medical Services › Patient Safety and Medication Errors
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Keywords
- computerized physician order entry, error prediction, gradient boosting decision tree, artificial neural networks, random forest, logistic regression, patient safety
- MeSH terms
- Machine Learning, Adolescent, Adult, Aged, Aged, 80 and over, Child, Child, Preschool, Female, Humans, Infant, Male, Medication Errors, Middle Aged, ROC Curve, Logistic Models, Medical Order Entry Systems, Young Adult, Patient Safety
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
