Cited by 228Gastrointestinal Bleeding Diagnosis and TreatmentHelicobacter pylori-related gastroenterology studiesArtificial Intelligence in Healthcare

The DOI matches its Crossref record; 50 of the 54 references it cites were found in a registry or an index.

Validation of a Machine Learning Model That Outperforms Clinical Risk Scoring Systems for Upper Gastrointestinal Bleeding

Open access

Shung, Dennis L., Au, Benjamin, Taylor, Richard Andrew, Tay, J. Kenneth, Laursen, Stig B., Stanley, Adrian J., Dalton, Harry R., Ngu, Jeffrey, Schultz, Michael, Laine, Loren·2020·Gastroenterology

DOI 10.1053/j.gastro.2019.09.009PMC PMC7004228

Open this paper in CoChat — free account Open PDF

Abstract

Background & Aims: Scoring systems are suboptimal for determining risk in patients with gastrointestinal bleeding (UGIB); these might be improved by a machine learning model. We used machine learning to develop a model to calculate risk of hospital-based intervention or death in patients with UGIB and compared its performance with other scoring systems. Methods: We analyzed data collected from consecutive unselected patients with UGIB from medical centers in 4 countries (United States, Scotland, England, Denmark; n=1958) from March 2014 through March 2015. We used the data to derive and internally validate a gradient-boosting machine learning model to identify patients who met a composite endpoint of hospital-based intervention (transfusion or hemostatic intervention) or death within 30 days. We compared the performance of the machine learning prediction model with validated pre-endoscopic clinical risk scoring systems (the Glasgow-Blatchford score [GBS], admission-Rockall score, and AIMS65). We externally validated the machine learning model using data from 2 Asia-Pacific sites (Singapore and New Zealand; n=399). Performance was measured by area under receiver operating characteristic curve (AUC) analysis. Results: The machine learning model identified patients who met the composite endpoint with an AUC of 0.91 in the in the internal validation set; the clinical scoring systems identified patients who met the composite endpoint with AUC values of 0.88 for GBS (P=.001), 0.73 for Rockall score (P<.001), and 0.78 for AIMS65 score (P<.001). In the external validation cohort, the machine learning model identified patients who met the composite endpoint with an AUC of 0.90, the GBS with an AUC of 0.87 (P=.004), the Rockall score with an AUC of 0.66 (P<.001), and the AIMS65 with an AUC of 0.64 (P<.001). At cutoff scores at which the machine learning model and GBS identified patients who met the composite endpoint with 100% sensitivity, the specificity values were 26% with the machine learning model vs 12% with GBS (P<.001). Conclusions: We developed a machine learning model that identifies patients with UGIB who met a composite endpoint of hospital-based intervention or death within 30 days with a greater AUC and higher levels of specificity, at 100% sensitivity, than validated clinical risk scoring systems. This model could increase identification of low-risk patients who can be safely discharged from the emergency department for outpatient management.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Integrity signals

Registry

Crossref · 10.1053/j.gastro.2019.09.009

Cites

50 of 50 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Random Forests
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538
Regression shrinkage and selection via the lasso
Tibshirani, R. (1996) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1996.tb02080.x · Cited by 51,482 · cited 1×
“Currently the models deployed on electronic health records include support vector machines, regression models, and decision trees (classification and regression trees (CART), random forest, and gradient boosting decision trees).(21-28)”
Method
Comparing the Areas under Two or More Correlated Receiver Operating Characteristic Curves: A Nonparametric Approach
DeLong, E. R., DeLong, D. M., & Clarke-Pearson, D. L. (1988) · Biometrics · doi:10.2307/2531595 · Cited by 22,649 · cited 1×
“For external validation, the AUC was calculated and then compared using a 2-tailed nonparametric method.(35) McNemar’s matched pairs test was used to compare specificities.”
Method
Multilayer feedforward networks are universal approximators
Hornik, K., Stinchcombe, M., & White, H. (1989) · Neural Networks · doi:10.1016/0893-6080(89)90020-8 · Cited by 21,728 · cited 1×
“21-28 Currently, the one use of clinical risk assessment tools for patients with UGIB generally agreed upon by guidelines and experts is to identify very low-risk patients who may be safely discharged from emergency departments with outpatient management 15, 18, 29 .”
Regularization and Variable Selection Via the Elastic Net
Zou, H., & Hastie, T. (2005) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2005.00503.x · Cited by 20,957
Regularization Paths for Generalized Linear Models via Coordinate Descent
Friedman, J., Hastie, T., & Tibshirani, R. (2010) · Journal of Statistical Software · doi:10.18637/jss.v033.i01 · Cited by 19,927
ggplot2: Elegant Graphics for Data Analysis
Ginestet, C. E. (2011) · Journal of the Royal Statistical Society Series A (Statistics in Society) · doi:10.1111/j.1467-985x.2010.00676_9.x · Cited by 14,511
ggplot2: Elegant Graphics for Data Analysis
Wickham, H. (2009) · doi:10.1007/978-0-387-98141-3 · Cited by 14,447
Ridge regression: Biased estimation for nonorthogonal problems
Hoerl, A. E., & Kennard, R. W. (1970) · Technometrics · doi:10.1080/00401706.1970.10488634 · Cited by 9,648
Least Squares Support Vector Machine Classifiers
Suykens, J., & Vandewalle, J. (1999) · Neural Processing Letters · doi:10.1023/a:1018628609742 · Cited by 9,481

Show all 54 references

Research works printed but not linked: 2

Rotondano, 2011, Artificial neural networks accurately predict mortality in patients with nonvariceal upper GI bleeding, Gastrointest Endosc, 73, 218, 10.1016/j.gie.2010.10.006 · DOI
Shung, 2019, Machine Learning to Predict Outcomes in Patients with Acute Gastrointestinal Bleeding: A Systematic Review, Dig Dis Sci, 64, 2078, 10.1007/s10620-019-05645-z · DOI

Sorted by citations

Items 201–228 of 228

Additional file 8 of Single-cell RNA sequencing integrated with bulk RNA sequencing analysis identifies a tumor immune microenvironment-related lncRNA signature in lung adenocarcinoma
Ren, Y., Wu, R., Li, C., Liu, L., Li, L., Weng, S., Xu, H., Xing, Z., Zhang, Y., Wang, L., et al. (2024) · Figshare · doi:10.6084/m9.figshare.26696742
Additional file 3 of Machine learning-based prediction model for responses of bDMARDs in patients with rheumatoid arthritis and ankylosing spondylitis
Lee, S., Kang, S., Eun, Y., Won, H.-H., Kim, H., Lee, J., Koh, E.-M., & Cha, H.-S. (2021) · Figshare · doi:10.6084/m9.figshare.16779640
Additional file 13 of Single-cell RNA sequencing integrated with bulk RNA sequencing analysis identifies a tumor immune microenvironment-related lncRNA signature in lung adenocarcinoma
Ren, Y., Wu, R., Li, C., Liu, L., Li, L., Weng, S., Xu, H., Xing, Z., Zhang, Y., Wang, L., et al. (2024) · Figshare · doi:10.6084/m9.figshare.26696757
Additional file 7 of Single-cell RNA sequencing integrated with bulk RNA sequencing analysis identifies a tumor immune microenvironment-related lncRNA signature in lung adenocarcinoma
Ren, Y., Wu, R., Li, C., Liu, L., Li, L., Weng, S., Xu, H., Xing, Z., Zhang, Y., Wang, L., et al. (2024) · Figshare · doi:10.6084/m9.figshare.26696739
Additional file 11 of Single-cell RNA sequencing integrated with bulk RNA sequencing analysis identifies a tumor immune microenvironment-related lncRNA signature in lung adenocarcinoma
Ren, Y., Wu, R., Li, C., Liu, L., Li, L., Weng, S., Xu, H., Xing, Z., Zhang, Y., Wang, L., et al. (2024) · Figshare · doi:10.6084/m9.figshare.26696751
Additional file 2 of Machine learning-based prediction model for responses of bDMARDs in patients with rheumatoid arthritis and ankylosing spondylitis
Lee, S., Kang, S., Eun, Y., Won, H.-H., Kim, H., Lee, J., Koh, E.-M., & Cha, H.-S. (2021) · Figshare · doi:10.6084/m9.figshare.16779637
Additional file 1 of Single-cell RNA sequencing integrated with bulk RNA sequencing analysis identifies a tumor immune microenvironment-related lncRNA signature in lung adenocarcinoma
Ren, Y., Wu, R., Li, C., Liu, L., Li, L., Weng, S., Xu, H., Xing, Z., Zhang, Y., Wang, L., et al. (2024) · Figshare · doi:10.6084/m9.figshare.25464936
Additional file 5 of Single-cell RNA sequencing integrated with bulk RNA sequencing analysis identifies a tumor immune microenvironment-related lncRNA signature in lung adenocarcinoma
Ren, Y., Wu, R., Li, C., Liu, L., Li, L., Weng, S., Xu, H., Xing, Z., Zhang, Y., Wang, L., et al. (2024) · Figshare · doi:10.6084/m9.figshare.26696733
Additional file 10 of Single-cell RNA sequencing integrated with bulk RNA sequencing analysis identifies a tumor immune microenvironment-related lncRNA signature in lung adenocarcinoma
Ren, Y., Wu, R., Li, C., Liu, L., Li, L., Weng, S., Xu, H., Xing, Z., Zhang, Y., Wang, L., et al. (2024) · Figshare · doi:10.6084/m9.figshare.26696748
Additional file 14 of Single-cell RNA sequencing integrated with bulk RNA sequencing analysis identifies a tumor immune microenvironment-related lncRNA signature in lung adenocarcinoma
Ren, Y., Wu, R., Li, C., Liu, L., Li, L., Weng, S., Xu, H., Xing, Z., Zhang, Y., Wang, L., et al. (2024) · Figshare · doi:10.6084/m9.figshare.26696760
Additional file 3 of Single-cell RNA sequencing integrated with bulk RNA sequencing analysis identifies a tumor immune microenvironment-related lncRNA signature in lung adenocarcinoma
Ren, Y., Wu, R., Li, C., Liu, L., Li, L., Weng, S., Xu, H., Xing, Z., Zhang, Y., Wang, L., et al. (2024) · Figshare · doi:10.6084/m9.figshare.26696727
Medically-Constrained Gradient Boosting for Predicting Blood Transfusion Need and Dose in Upper Gastrointestinal Bleeding (Preprint)Preprint
Li, X., He, Y., Hou, M., Yu, Z., Miao, S., Huang, Z., & Yang, M. (2025) · doi:10.2196/preprints.83889
Artificial Intelligence in Bleeding Control and Other Advanced Gastrointestinal Procedures
Gibiino, G., Secco, M., Tuccillo, L., Coluccio, C., Fabbri, C., & Binda, C. (2026) · Hot topics in acute care surgery and trauma · doi:10.1007/978-3-032-09357-8_21
Artificial intelligence in upper gastrointestinal bleeding: Can machine learning predict endotherapy requirements?
Kumar, S. R., Panigrahi, M. K., & Sasmal, P. K. (2026) · Artificial Intelligence in Gastrointestinal Endoscopy · doi:10.37126/aige.v7.i1.114426
Prognostic scores for risk stratification in patients with acute upper gastrointestinal bleeding
Yordanov, A. (2026) · Scripta Scientifica Medica · doi:10.14748/zbcvxb42
“Evidence from recent meta-analyses confirms that no single prognostic score achieves optimal predictive accuracy across all clinically relevant end-points in acute UGIB [21].”
Development and internal validation of a machine learning-based model to predict 1-year all-cause mortality in patients with gastrointestinal bleeding with type 2 diabetes
Wang, X., Wang, Y., & Liu, H. (2026) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-026-03447-8
A Comparison of the ABC and AIMS65 Scores in Predicting Outcomes in Patients with Acute Upper Gastrointestinal Bleeding: A Retrospective Multicenter Study
Sohail, A., Vankayala, U. S. A., George, B., Aslam, T., Javor, J., & Deeb, L. S. (2026) · Digestive Diseases and Sciences · doi:10.1007/s10620-026-09978-4
“Shung et al. developed a machine learning–based AI score using gra-dient-boosted decision trees, which demonstrated superior discrimination compared with traditional clinical scores in a multicenter validation cohort [21].”
Clinical practice patterns in the pre-endoscopic management of individuals presenting with acute upper gastrointestinal bleeding: results of an international survey
Obeidat, M., Floria, D., Teutsch, B., Abonyi-Tóth, Z., Laursen, S. B., Camus, M., Verdonk, F., Vincze, Á., Bognár, Z., Drug, V. L., et al. (2026) · Therapeutic Advances in Gastroenterology · doi:10.1177/17562848261455285
“22,23 The integration of such models, automatically calculated from electronic health records, could facilitate broader adoption of risk stratification in clinical practice.”
Self-citation
AI in Pulmonary Oncology-Driven Innovations in Early Lung Cancer Detection and Risk Stratification
Kumar, R., & Aves, M. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-7759-9.ch007
Prediction of Blood Transfusion Need and Dose in Patients With Upper Gastrointestinal Bleeding: Retrospective Multicenter Prediction Model Study
Li, X., He, Y., Hou, M., Yu, Z. H., Miao, S., Huang, Z., & Yang, M. (2026) · JMIR Medical Informatics · doi:10.2196/83889
“Machine learning has improved discrimination in many clinical prediction tasks and sometimes outperforms traditional scores [15,16].”
A novel prediction score for rebleeding in suspected small-bowel bleeding post-capsule endoscopy examination: a multicenter, retrospective cohort study
Tang, X., Li, M. M., Cheng, C., Wang, L., Cao, J., Lu, F., Huang, S. H. M., Chen, L., & Shi, R. (2026) · European journal of medical research · doi:10.1186/s40001-026-04787-x
“…to improve prediction accuracy [32]; integrating objective indicators such as the number, size, and shape of lesions automatically identified by AI to achieve refined risk assessment [33]; comparing or fusing with existing scoring systems to construct a more efficient composite model [34].”
Development and Validation of a Machine Learning Model for Predicting 30-Day Mortality and Early Endoscopic Hemostatic Intervention in Acute Upper Gastrointestinal Bleeding
Özden, Y., Tekiş, D., & Kiremitçi, S. (2026) · Journal of Clinical Gastroenterology · doi:10.1097/mcg.0000000000002415
Bayesian Assessment of Pre-Endoscopic Risk Scores in Upper Gastrointestinal Bleeding
Jiménez‐Rosales, R., Muñoz, A. L., García, R. F., Cortes, P. M., Martínez‐Cara, J. G., Redondo‐Cerezo, E., & Valverde‐López, F. (2026) · Revista Española de Enfermedades Digestivas · doi:10.17235/reed.2026.12001/2026
Comprehensively analyzing the impact of comprehensive complication index on the long-term prognostic for patients with hepatocellular carcinoma after hepatectomy
Bao, Z.-Y., Xiao, Y.-T., Lu, W., Wang, K., Yuan, J., Yu, Y., Wang, K.-D., Yu, Z.-C., Fang, Z.-K., Shi, Z.-J., et al. (2026) · Current Problems in Surgery · doi:10.1016/j.cpsurg.2026.102126
HOLD-B: A Practical Score for Early Risk Stratification in Upper Gastrointestinal Bleeding
Güler, E. E., Bora, E. S., Kanter, E., Özçelik, N., ÖZ, E. E., & Tekindal, M. A. (2026) · Medicina · doi:10.3390/medicina62081476
“Machine learning models and artificial neural networks have outperformed traditional scoring systems, such as the Glasgow-Blatchford, Rockall, and AIMS65 scores, in multiple studies, with reported AUC values ranging from 0.84 to 0.93 across various cohorts [12,27].”
Influential
Interpretable machine learning for early in-hospital mortality prediction in intensive care unit patients with acute upper gastrointestinal bleeding: Development and multicenter external validation
Shi, J., li, Y., Li, W., Zhu, S., Chen, X., Chen, S., Jin, G., & Liu, W. (2026) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2026.106737
Early mortality prediction in upper gastrointestinal bleeding using explainable machine learning: comparison with established clinical risk scores
Sıddıkoğlu, D., Daş, M., Çetin, E. Ü., Kurtkulağı, Ö., ÇETİN, A. U., Kayan, H., Güler, S., Mutlay, F., & Beyazıt, Y. (2026) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-026-03883-6
Microbiome Preterm Birth DREAM Challenge
Sirota, M. (2023) · ImmPort · doi:10.21430/m316mgbo3o

Authors: Shung, D. L., Au, B., Taylor, R. A., Tay, J. K., Laursen, S. B., Stanley, A. J., Dalton, H. R., Ngu, J., Schultz, M., & Laine, L.
Affiliations: Yale University (US)
Stanford University (US)
Odense University Hospital (DK)
Glasgow Royal Infirmary (GB)
Royal Cornwall Hospital (GB)
Christchurch Hospital (NZ)
Dunedin Public Hospital (NZ)
Corresponding author: Shung, Dennis L.
Laine, Loren
ORCID: Shung, Dennis L.
Taylor, Richard Andrew
Tay, J. Kenneth
Laursen, Stig B.
Stanley, Adrian J.
Dalton, Harry R.
Schultz, Michael
Laine, Loren
Venue: Gastroenterology
Publisher: Elsevier BV
Published: 2020-01
Volume: 158
Issue: 1
Pages: 160-167
ISSN: 0016-5085 (ISSN-L)
DOI: 10.1053/j.gastro.2019.09.009
PMC: PMC7004228
OpenAlex: W2976369463
Semantic Scholar: 49e263c97948d8177c777a1106e0ab0795effa58
Cited by: 227
References: 54
Citations by year: 2026: 22 · 2025: 30 · 2024: 49 · 2023: 29 · 2022: 31 · 2021: 37 · 2020: 25
FWCI: 21.87
Type: journal article
Language: en
Open access: bronze
doi.org
Licence: www.elsevier.com/tdm/userlicense/1.0/ (tdm)
www.elsevier.com/legal/tdmrep-license (tdm)
Locations: Gastroenterology (publishedVersion)
PubMed (publishedVersion)
University of Southern Denmark Research Portal (University of Southern Denmark) (publishedVersion, cc-by-nc-nd)
Europe PMC (PubMed Central) (acceptedVersion)
PubMed Central (acceptedVersion)
Funders: National Institutes of Health (T32 DK007017)
Topics: Health Sciences › Medicine › Gastroenterology › Gastrointestinal Bleeding Diagnosis and Treatment
Health Sciences › Medicine › Gastroenterology › Helicobacter pylori-related gastroenterology studies
Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
Keywords: upper gastrointestinal bleeding, machine learning prediction models, Glasgow-Blatchford score, Rockall score, AIMS65, gradient boosting, hospital-based intervention, 30-day mortality, external validation, risk stratification, endoscopic hemostasis, blood transfusion, internal validation, sensitivity and specificity
MeSH terms: Machine Learning, Adult, Aged, Aged, 80 and over, Blood Transfusion, Emergency Service, Hospital, Female, Gastrointestinal Hemorrhage, Hemostatic Techniques, Humans, Male, Middle Aged, Models, Biological, Prognosis, ROC Curve, Risk Assessment

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Validation of a Machine Learning Model That Outperforms Clinical Risk Scoring Systems for Upper Gastrointestinal Bleeding

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Self-check

Citations

Cites

Cited by

Related

Metadata

Record

About these checks