Cited by 228Gastrointestinal Bleeding Diagnosis and TreatmentHelicobacter pylori-related gastroenterology studiesArtificial Intelligence in Healthcare

The DOI matches its Crossref record; 50 of the 54 references it cites were found in a registry or an index.

Validation of a Machine Learning Model That Outperforms Clinical Risk Scoring Systems for Upper Gastrointestinal Bleeding

Open access

Shung, Dennis L., Au, Benjamin, Taylor, Richard Andrew, Tay, J. Kenneth, Laursen, Stig B., Stanley, Adrian J., Dalton, Harry R., Ngu, Jeffrey, Schultz, Michael, Laine, Loren·2020·Gastroenterology

DOI 10.1053/j.gastro.2019.09.009PMC PMC7004228

Open this paper in CoChat — free account Open PDF

Abstract

Background & Aims: Scoring systems are suboptimal for determining risk in patients with gastrointestinal bleeding (UGIB); these might be improved by a machine learning model. We used machine learning to develop a model to calculate risk of hospital-based intervention or death in patients with UGIB and compared its performance with other scoring systems. Methods: We analyzed data collected from consecutive unselected patients with UGIB from medical centers in 4 countries (United States, Scotland, England, Denmark; n=1958) from March 2014 through March 2015. We used the data to derive and internally validate a gradient-boosting machine learning model to identify patients who met a composite endpoint of hospital-based intervention (transfusion or hemostatic intervention) or death within 30 days. We compared the performance of the machine learning prediction model with validated pre-endoscopic clinical risk scoring systems (the Glasgow-Blatchford score [GBS], admission-Rockall score, and AIMS65). We externally validated the machine learning model using data from 2 Asia-Pacific sites (Singapore and New Zealand; n=399). Performance was measured by area under receiver operating characteristic curve (AUC) analysis. Results: The machine learning model identified patients who met the composite endpoint with an AUC of 0.91 in the in the internal validation set; the clinical scoring systems identified patients who met the composite endpoint with AUC values of 0.88 for GBS (P=.001), 0.73 for Rockall score (P<.001), and 0.78 for AIMS65 score (P<.001). In the external validation cohort, the machine learning model identified patients who met the composite endpoint with an AUC of 0.90, the GBS with an AUC of 0.87 (P=.004), the Rockall score with an AUC of 0.66 (P<.001), and the AIMS65 with an AUC of 0.64 (P<.001). At cutoff scores at which the machine learning model and GBS identified patients who met the composite endpoint with 100% sensitivity, the specificity values were 26% with the machine learning model vs 12% with GBS (P<.001). Conclusions: We developed a machine learning model that identifies patients with UGIB who met a composite endpoint of hospital-based intervention or death within 30 days with a greater AUC and higher levels of specificity, at 100% sensitivity, than validated clinical risk scoring systems. This model could increase identification of low-risk patients who can be safely discharged from the emergency department for outpatient management.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Integrity signals

Registry

Crossref · 10.1053/j.gastro.2019.09.009

Cites

50 of 50 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Random Forests
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538
Regression shrinkage and selection via the lasso
Tibshirani, R. (1996) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1996.tb02080.x · Cited by 51,482 · cited 1×
“Currently the models deployed on electronic health records include support vector machines, regression models, and decision trees (classification and regression trees (CART), random forest, and gradient boosting decision trees).(21-28)”
Method
Comparing the Areas under Two or More Correlated Receiver Operating Characteristic Curves: A Nonparametric Approach
DeLong, E. R., DeLong, D. M., & Clarke-Pearson, D. L. (1988) · Biometrics · doi:10.2307/2531595 · Cited by 22,649 · cited 1×
“For external validation, the AUC was calculated and then compared using a 2-tailed nonparametric method.(35) McNemar’s matched pairs test was used to compare specificities.”
Method
Multilayer feedforward networks are universal approximators
Hornik, K., Stinchcombe, M., & White, H. (1989) · Neural Networks · doi:10.1016/0893-6080(89)90020-8 · Cited by 21,728 · cited 1×
“21-28 Currently, the one use of clinical risk assessment tools for patients with UGIB generally agreed upon by guidelines and experts is to identify very low-risk patients who may be safely discharged from emergency departments with outpatient management 15, 18, 29 .”
Regularization and Variable Selection Via the Elastic Net
Zou, H., & Hastie, T. (2005) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2005.00503.x · Cited by 20,957
Regularization Paths for Generalized Linear Models via Coordinate Descent
Friedman, J., Hastie, T., & Tibshirani, R. (2010) · Journal of Statistical Software · doi:10.18637/jss.v033.i01 · Cited by 19,927
ggplot2: Elegant Graphics for Data Analysis
Ginestet, C. E. (2011) · Journal of the Royal Statistical Society Series A (Statistics in Society) · doi:10.1111/j.1467-985x.2010.00676_9.x · Cited by 14,511
ggplot2: Elegant Graphics for Data Analysis
Wickham, H. (2009) · doi:10.1007/978-0-387-98141-3 · Cited by 14,447
Ridge regression: Biased estimation for nonorthogonal problems
Hoerl, A. E., & Kennard, R. W. (1970) · Technometrics · doi:10.1080/00401706.1970.10488634 · Cited by 9,648
Least Squares Support Vector Machine Classifiers
Suykens, J., & Vandewalle, J. (1999) · Neural Processing Letters · doi:10.1023/a:1018628609742 · Cited by 9,481

Show all 54 references

Research works printed but not linked: 2

Rotondano, 2011, Artificial neural networks accurately predict mortality in patients with nonvariceal upper GI bleeding, Gastrointest Endosc, 73, 218, 10.1016/j.gie.2010.10.006 · DOI
Shung, 2019, Machine Learning to Predict Outcomes in Patients with Acute Gastrointestinal Bleeding: A Systematic Review, Dig Dis Sci, 64, 2078, 10.1007/s10620-019-05645-z · DOI

Sorted by citations

ACG Clinical Guideline: Upper Gastrointestinal and Ulcer Bleeding
Laine, L., Barkun, A. N., Saltzman, J. R., Martel, M., & Leontiadis, G. I. (2021) · The American Journal of Gastroenterology · doi:10.14309/ajg.0000000000001245 · Cited by 661
“(8) can be set to provide sensitivities of 99% (comparable with GBS 5 0–1) or 100% (comparable with GBS 5 0) with specificities that are higher than GBS.”
BackgroundSelf-citation
ABC score: a new risk score that accurately predicts mortality in acute upper and lower gastrointestinal bleeding: an international multicentre study
Laursen, S. B., Oakland, K., Laine, L., Bieber, V., Marmo, R., Redondo‐Cerezo, E., Dalton, H. R., Ngu, J., Schultz, M., Soncini, M., et al. (2020) · Gut · doi:10.1136/gutjnl-2019-320002 · Cited by 185
“35 Although several risk scores have been developed for assessment of patients with gastrointestinal bleeding, no risk score has been widely accepted in clinical practice.”
Self-citation
Artificial intelligence in gastroenterology: A state-of-the-art review
Kröner, P. T., Engels, M. M. L., Glicksberg, B. S., Johnson, K. W., Mzaik, O., van Hooft, J. E., Wallace, M. B., El‐Serag, H. B., & Krittanawong, C. (2021) · World Journal of Gastroenterology · doi:10.3748/wjg.v27.i40.6794 · Cited by 175
Clinical impact and quality of randomized controlled trials involving interventions evaluating artificial intelligence prediction tools: a systematic review
Zhou, Q., Chen, Z.-H., Cao, Y.-H., & Peng, S. (2021) · npj Digital Medicine · doi:10.1038/s41746-021-00524-2 · Cited by 126
Randomized Controlled Trials of Artificial Intelligence in Clinical Practice: Systematic Review
Lam, T. Y. T., Cheung, M. F. K., Munro, Y. L., Lim, K. M., Shung, D., & Sung, J. J. Y. (2022) · Journal of Medical Internet Research · doi:10.2196/37188 · Cited by 120
“AI-assisted tools have been successfully applied in various clinical settings to assist diagnosis [5], improve therapy [6], and predict risk of mortality [7].”
Self-citation
Preliminary Evidence of the Use of Generative AI in Health Care Clinical Services: Systematic Narrative Review
Yim, D., Khuntia, J., Parameswaran, V., & Meyers, A. D. (2024) · JMIR Medical Informatics · doi:10.2196/52073 · Cited by 140
“[86-91].”
Background
Explainable Machine Learning Model for Predicting GI Bleed Mortality in the Intensive Care Unit
Deshmukh, F., & Merchant, S. S. (2020) · The American Journal of Gastroenterology · doi:10.14309/ajg.0000000000000632 · Cited by 95
Update on the management of upper gastrointestinal bleeding
Orpen‐Palmer, J., & Stanley, A. J. (2022) · BMJ Medicine · doi:10.1136/bmjmed-2022-000202 · Cited by 88
Self-citation
A comparative study of explainable ensemble learning and logistic regression for predicting in-hospital mortality in the emergency department
Rahmatinejad, Z., Dehghani, T., Hoseini, B., Rahmatinejad, F., Lotfata, A., Reihani, H., & Eslami, S. (2024) · Scientific Reports · doi:10.1038/s41598-024-54038-4 · Cited by 65
“Moreover, recent developments in ML models have yielded promising outcomes in predicting clinical scenarios, including mortality within EDs 29–36 .”
Background
An update on the management of non-variceal upper gastrointestinal bleeding
Alali, A. A., & Barkun, A. N. (2022) · Gastroenterology report · doi:10.1093/gastro/goad011 · Cited by 63
“It usually presents as ongoing hematemesis, recurrent melena, hemodynamic instability, or drop of hemoglobin of >2 g/dL [136].”
Background
Treatment of upper gastrointestinal bleeding in 2020: New techniques and outcomes
Lau, L. H. S., & Sung, J. J. �. (2020) · Digestive Endoscopy · doi:10.1111/den.13674 · Cited by 63
Machine Learning Versus Usual Care for Diagnostic and Prognostic Prediction in the Emergency Department: A Systematic Review
Kareemi, H., Vaillancourt, C., Rosenberg, H., Fournier, K., & Yadav, K. (2020) · Academic Emergency Medicine · doi:10.1111/acem.14190 · Cited by 65
Machine learning–based personalized prediction of gastric cancer incidence using the endoscopic and histologic findings at the initial endoscopy
Arai, J., Aoki, T., Sato, M., Niikura, R., Suzuki, N., Ishibashi, R., Tsuji, Y., Yamada, A., Hirata, Y., Ushiku, T., et al. (2022) · Gastrointestinal Endoscopy · doi:10.1016/j.gie.2021.12.033 · Cited by 59
Machine Learning for Prediction and Risk Stratification of Lupus Nephritis Renal Flare
Chen, Y., Huang, S., Chen, T., Liang, D., Yang, H. J., Zeng, C., Li, X., Xie, G., & Liu, Z. (2021) · American Journal of Nephrology · doi:10.1159/000513566 · Cited by 59
Applications of machine learning models in the prediction of gastric cancer risk in patients after Helicobacter pylori eradication
Leung, W. K., Cheung, K. S., Li, B., Law, S. Y.-K., & Lui, T. K.-L. (2021) · Alimentary Pharmacology & Therapeutics · doi:10.1111/apt.16272 · Cited by 51
“14,15 The machine learning model has been recently applied in the prediction of risk of hospital-based intervention or death in patients with upper gastrointestinal bleeding, 16 and was found to be superior to validated clinical risk scoring systems.”
Method
Evaluation of machine learning solutions in medicine
Antoniou, T., & Mamdani, M. (2021) · Canadian Medical Association Journal · doi:10.1503/cmaj.210036 · Cited by 58
“22,23 As with approaches involving predictions by clinicians, comparisons with validated risk-scoring tools should be undertaken using data that were not part of the machine-learned model’s development process.”
Machine learning-based prediction model for responses of bDMARDs in patients with rheumatoid arthritis and ankylosing spondylitis
Lee, S., Kang, S., Eun, Y., Won, H., Kim, H., Lee, J., Koh, E., & Cha, H. (2021) · Arthritis Research & Therapy · doi:10.1186/s13075-021-02635-3 · Cited by 49
Microbiome preterm birth DREAM challenge: Crowdsourcing machine learning approaches to advance preterm birth research
Golob, J. L., Oskotsky, T., Tang, A., Roldan, A., Chung, V., Ha, C. W. Y., Wong, R. J., Flynn, K. J., Chai, R., Dubin, C., et al. (2023) · Cell Reports Medicine · doi:10.1016/j.xcrm.2023.101350 · Cited by 53
“14–16 By applying ML methods to large amounts of heterogeneous data, patterns in data can be discerned that would be otherwise difﬁcult for humans to distinguish.”
Development and validation of a gradient boosting machine to predict prognosis after liver resection for intrahepatic cholangiocarcinoma
Ji, G., Jiao, C., Xu, Z., Li, X., Wang, X., & Wang, X. (2022) · BMC Cancer · doi:10.1186/s12885-022-09352-3 · Cited by 47
Use of Machine Learning to Develop a Risk-Stratification Tool for Emergency Department Patients With Acute Heart Failure
Sax, D. R., Mark, D. G., Huang, J., Sofrygin, O., Rana, J. S., Collins, S. P., Storrow, A. B., Liu, D., & Reed, M. (2020) · Annals of Emergency Medicine · doi:10.1016/j.annemergmed.2020.09.436 · Cited by 48
Machine Learning Clustering for Blood Pressure Variability Applied to Systolic Blood Pressure Intervention Trial (SPRINT) and the Hong Kong Community Cohort
Tsoi, K. K. F., Chan, N. B., Yiu, K. K. L., Poon, S. K.-S., Lin, B. Y., & Ho, K. (2020) · Hypertension · doi:10.1161/hypertensionaha.119.14213 · Cited by 42
Machine learning for individualized prediction of hepatocellular carcinoma development after the eradication of hepatitis C virus with antivirals
Minami, T., Sato, M., Toyoda, H., Yasuda, S., Yamada, T., Nakatsuka, T., Enooku, K., Nakagawa, H., Fujinaga, H., Izumiya, M., et al. (2023) · Journal of Hepatology · doi:10.1016/j.jhep.2023.05.042 · Cited by 43
A Review of Risk Scores within Upper Gastrointestinal Bleeding
Orpen‐Palmer, J., & Stanley, A. J. (2023) · Journal of Clinical Medicine · doi:10.3390/jcm12113678 · Cited by 43
“identification of low-risk patients, with a specificity of 26% at 100% sensitivity compared to a specificity of 12% for GBS 0 (sensitivity 100%) [48].”
BackgroundSelf-citation
Understanding and interpreting artificial intelligence, machine learning and deep learning in Emergency Medicine
Ramlakhan, S. L., Saatchi, R., Sabir, L., Singh, Y., Hughes, R., Shobayo, O., & Ventour, D. (2022) · Emergency Medicine Journal · doi:10.1136/emermed-2021-212068 · Cited by 40
Comparing the performance of the ABC, AIMS65, GBS, and pRS scores in predicting 90-day mortality or rebleeding among emergency department patients with acute upper gastrointestinal bleeding: A prospective multicenter study
Liu, S., Zhang, X., Walline, J. H., Yu, X., & Zhu, H. (2021) · Journal of Translational Internal Medicine · doi:10.2478/jtim-2021-0026 · Cited by 33
Clinical utility of pre-endoscopy risk scores in upper gastrointestinal bleeding
Tham, J., & Stanley, A. J. (2019) · Expert Review of Gastroenterology & Hepatology · doi:10.1080/17474124.2019.1698292 · Cited by 33
Self-citation
Machine learning in gastrointestinal surgery
Sakamoto, T., Goto, T., Fujiogi, M., & Lefor, A. K. (2021) · Surgery Today · doi:10.1007/s00595-021-02380-9 · Cited by 33
Artificial intelligence in health care: preparing for the fifth Industrial Revolution
Sung, J. J. �., Stewart, C. L., & Freedman, B. (2020) · The Medical Journal of Australia · doi:10.5694/mja2.50755 · Cited by 36
Towards personalised management for non-variceal upper gastrointestinal bleeding
Sung, J. J. �., Laine, L., Kuipers, E. J., & Barkun, A. N. (2021) · Gut · doi:10.1136/gutjnl-2020-323846 · Cited by 31
Self-citation
Spleen volume-based non-invasive tool for predicting hepatic decompensation in people with compensated cirrhosis (CHESS1701)
Yu, Q., Xu, C., Li, Q., Ding, Z., Lv, Y., Liu, C., Huang, Y., Zhou, J., Huang, S., Xia, C., et al. (2022) · JHEP Reports · doi:10.1016/j.jhepr.2022.100575 · Cited by 31
“3) Child–Pugh score, n (%) A, 5 273 (50) 109 (52) 22 (49) 142 (48) A, 6 134 (24) 60 (29) 10 (22) 64 (22) B, 7 78 (14) 21 (10) 9 (20) 48 (16) B, 8 29 (5) 9 (4) 3 (7) 17 (6) B, 9 36 (7) 9 (4) 1 (2) 26 (9) MELD 6.”
An imaging-based machine learning model outperforms clinical risk scores for prognosis of cirrhotic variceal bleeding
Gao, Y., Yu, Q., Li, X., Xia, C., Zhou, J., Xia, T., Zhao, B. Y., Qiu, Y., Zha, J., Wang, Y., et al. (2023) · European Radiology · doi:10.1007/s00330-023-09938-w · Cited by 30
“And these scoring systems for AVB patients have always focused on a single clinical endpoint, never on a composite endpoint of the need for hospital-based intervention or death [15].”
Background
The biomedical applications of artificial intelligence: an overview of decades of research
Naskar, S., Sharma, S., Kuotsu, K., Halder, S., Pal, G., Saha, S., Mondal, S., Biswas, U. K., Jana, M., & Bhattacharjee, S. (2025) · Journal of drug targeting · doi:10.1080/1061186x.2024.2448711 · Cited by 35
Horibe GI bleeding prediction score: a simple score for triage decision-making in patients with suspected upper GI bleeding
Horibe, M., Iwasaki, E., Bazerbachi, F., Kaneko, T., Matsuzaki, J., Minami, K., Masaoka, T., Hosoe, N., Ogura, Y., Namiki, S., et al. (2020) · Gastrointestinal Endoscopy · doi:10.1016/j.gie.2020.03.3846 · Cited by 29
Neural network predicts need for red blood cell transfusion for patients with acute gastrointestinal bleeding admitted to the intensive care unit
Shung, D. L., Huang, J., Castro, E., Tay, J. K., Simonov, M., Laine, L., Batra, R., & Krishnaswamy, S. (2021) · Scientific Reports · doi:10.1038/s41598-021-88226-3 · Cited by 29
Self-citation
Identifying predictors of varices grading in patients with cirrhosis using ensemble learning
Bayani, A., Hosseini, A., Asadi, F., Hatami, B., Kavousi, K., Aria, M., & Zali, M. R. (2022) · Clinical Chemistry and Laboratory Medicine (CCLM) · doi:10.1515/cclm-2022-0508 · Cited by 29
Validation of an Electronic Health Record–Based Machine Learning Model Compared With Clinical Risk Scores for Gastrointestinal Bleeding
Shung, D. L., Chan, C., You, K., Nakamura-Sakai, S., Saarinen, T., Zheng, N. S., Simonov, M., Li, D. K., Tsay, C. J., Kawamura, Y. I., et al. (2024) · Gastroenterology · doi:10.1053/j.gastro.2024.06.030 · Cited by 29
Self-citation
Data-driven decision-making for precision diagnosis of digestive diseases
Jiang, S., Wang, T., & Zhang, K.-H. (2023) · BioMedical Engineering OnLine · doi:10.1186/s12938-023-01148-1 · Cited by 29
Review article: Upper gastrointestinal bleeding – review of current evidence and implications for management
Shung, D. L., & Laine, L. (2024) · Alimentary Pharmacology & Therapeutics · doi:10.1111/apt.17949 · Cited by 26
Self-citation
Prediction of the need for emergency endoscopic treatment for upper gastrointestinal bleeding and new score model: a retrospective study
Sasaki, Y., Abe, T., Kawamura, N., Keitoku, T., Shibata, I., Ohno, S., Ono, K., & Makishima, M. (2022) · BMC Gastroenterology · doi:10.1186/s12876-022-02413-8 · Cited by 26
“It showed higher area under ROC curve, sensitivity, and specificity than the conventional GBS and AIMS65 score [30].”
Method
Interpretable machine learning model to predict surgical difficulty in laparoscopic resection for rectal cancer
Yu, M., Yuan, Z., Li, R., Shi, B., Wan, D., & Dong, X. (2024) · Frontiers in Oncology · doi:10.3389/fonc.2024.1337219 · Cited by 25
Management of acute variceal bleeding: updated APASL guidelines
Lesmana, C. R. A., Shukla, A. K., Kumar, A., Shalimar, S., Qi, X., Gani, R. A., Zhuang, Z., Dökmeci, A. K., Lo, G., Maruyama, H., et al. (2025) · Hepatology International · doi:10.1007/s12072-025-10894-4 · Cited by 24
“Study from Levi et al. showed that the machine learning algorithm can offer reliable guidance (AUC > 0.80) for predicting the need for transfusion in the next 24 h. [60]”
Influential
Pre-Endoscopic Scores Predicting Low-Risk Patients with Upper Gastrointestinal Bleeding: A Systematic Review and Meta-Analysis
Boustany, A., Alali, A. A., Almadi, M. A., Martel, M., & Barkun, A. N. (2023) · Journal of Clinical Medicine · doi:10.3390/jcm12165194 · Cited by 24
Application of Machine Learning for Predicting Anastomotic Leakage in Patients with Gastric Adenocarcinoma Who Received Total or Proximal Gastrectomy
Shao, S., Liu, L., Zhao, Y., Mu, L., Lu, Q., & Qin, J. (2021) · Journal of Personalized Medicine · doi:10.3390/jpm11080748 · Cited by 25
“Machine learning, a subset of artificial intelligence, outperforms other technologies in developing predictive models [10,11].”
Background
Machine learning to predict major bleeding during anticoagulation for venous thromboembolism: possibilities and limitations
Mora, D., Mateo, J., Nieto, J. A., Bikdeli, B., Yamashita, Y., Barco, S., Jiménez, D., Rodríguez, P. D., Rosa, V., Yoo, H. H. B., et al. (2023) · British Journal of Haematology · doi:10.1111/bjh.18737 · Cited by 23
Prediction of Adverse Events in Stable Non-Variceal Gastrointestinal Bleeding Using Machine Learning
Seo, D., Yi, H., Park, B., Kim, Y., Jung, D. H., Woo, I., Sohn, C. H., Ko, B. S., Kim, N., & Kim, W. Y. (2020) · Journal of Clinical Medicine · doi:10.3390/jcm9082603 · Cited by 22
History, current status, and future directions of artificial intelligence
Kubassova, O., Shaikh, F., Melus, C., & Mähler, M. (2021) · Elsevier eBooks · doi:10.1016/b978-0-12-820239-5.00002-4 · Cited by 25
Current Status and Future Perspective of Artificial Intelligence in the Management of Peptic Ulcer Bleeding: A Review of Recent Literature
Yen, H., Wu, P., Chen, M.-F., Lin, W., Tsai, C.-L., & Lin, K. (2021) · Journal of Clinical Medicine · doi:10.3390/jcm10163527 · Cited by 22
“[18] were the first to conduct a large prospective international study for building an ML model for patients with PUB by testing and comparing the performance of the ML model and the conventional scoring system in 2020.”
Method
Multimodal deep learning for COVID-19 prognosis prediction in the emergency department: a bi-centric study
Dipaola, F., Gatti, M., Levra, A. G., Menè, R., Shiffer, D., Faccincani, R., Raouf, Z., Secchi, A., Rovere, P. Q., Voza, A., et al. (2023) · Scientific Reports · doi:10.1038/s41598-023-37512-3 · Cited by 22
Single-cell RNA sequencing integrated with bulk RNA sequencing analysis identifies a tumor immune microenvironment-related lncRNA signature in lung adenocarcinoma
Ren, Y., Wu, R., Li, C., Liu, L., Li, L., Weng, S., Xu, H., Xing, Z., Zhang, Y., Wang, L., et al. (2024) · BMC Biology · doi:10.1186/s12915-024-01866-5 · Cited by 21
“With advances in next-generation sequencing (NGS) technology and the vigorous development of machine learning algorithms, an increasing number of researchers prefer to apply machine learning algorithms to develop survival prediction signatures [46, 47].”
Background
Early identification of patients with acute gastrointestinal bleeding using natural language processing and decision rules
Shung, D. L., Tsay, C. J., Laine, L., Chang, D., Li, F., Thomas, P. G., Partridge, C. M., Simonov, M., Hsiao, A., Tay, J. K., et al. (2020) · Journal of Gastroenterology and Hepatology · doi:10.1111/jgh.15313 · Cited by 20
“21,22 This algorithm would then prompt an alert that would be delivered to the provider to identify very low risk patients who could be discharged.”
BackgroundSelf-citation
Predicting in-hospital mortality in adult non-traumatic emergency department patients: a retrospective comparison of the Modified Early Warning Score (MEWS) and machine learning approach
Wu, K.-H., Cheng, F., Tai, H.-L., Wang, J.-C., Huang, Y.-T., Su, C.-M., & Chang, Y.-N. (2021) · PeerJ · doi:10.7717/peerj.11988 · Cited by 21
“Previous studies have shown that ML techniques are also better than conventional scoring systems in other fields, such as in patients with upper gastrointestinal bleeding (Shung et al., 2020) or chest pain (Liu et al.”
Result
A New Dawn for the Use of Artificial Intelligence in Gastroenterology, Hepatology and Pancreatology
Oka, A., Ishimura, N., & Ishihara, S. (2021) · Diagnostics · doi:10.3390/diagnostics11091719 · Cited by 22
“Then, the authors published an excellent risk scoring system using machine learning models with a greater AUC, higher levels of specificity, and a 100% sensitivity compared to the clinical risk scores [77].”
Background
Systematic review and meta-analysis of risk scores in prediction for the clinical outcomes in patients with acute variceal bleeding
Yang, L., Sun, R., Wei, N., & Chen, H. (2021) · Annals of Medicine · doi:10.1080/07853890.2021.1990394 · Cited by 21
“Shung Dennis L developed and validated a machine learning odel for UGIB, showing 100% sensitivity with a specificity of 26% [51].”
Explainable machine learning model for predicting acute pancreatitis mortality in the intensive care unit
Jiang, M., Wu, X., Lin, X., & Li, C.-L. (2025) · BMC Gastroenterology · doi:10.1186/s12876-025-03723-3 · Cited by 21
“%) for death prediction in the external validation cohort, and to compare the specificity of different models [15, 16].”
Comparative Evaluation of the ABC Score to Other Risk Stratification Scales in Managing High-risk Patients Presenting With Acute Upper Gastrointestinal Bleeding
Kherad, O., Restellini, S., Almadi, M. A., Martel, M., & Barkun, A. N. (2022) · Journal of Clinical Gastroenterology · doi:10.1097/mcg.0000000000001720 · Cited by 20
Challenges involved in the application of artificial intelligence in gastroenterology: The race is on!
Christou, C. D., & Tsoulfas, G. (2023) · World Journal of Gastroenterology · doi:10.3748/wjg.v29.i48.6168 · Cited by 22
“…such studies include a study developing an ML tool to predict overall survival in patients with esophageal cancer[41], a model predicting in-hospital death for patients with gastrointestinal bleeding[42], and an ML model predicting the chance of recurrence in operated gastric cancer patients [43].”
Prediction model of emergency mortality risk in patients with acute upper gastrointestinal bleeding: a retrospective study
Chen, L., Zheng, H., & Wang, S. (2021) · PeerJ · doi:10.7717/peerj.11656 · Cited by 19
“AIMS65, GBS, mGBS, and PERS were calculated for every patient based on the first available data.”
BackgroundInfluential
Application of Machine Learning Based on Structured Medical Data in Gastroenterology
Kim, H., Gong, E. J., & Bang, C. S. (2023) · Biomimetics · doi:10.3390/biomimetics8070512 · Cited by 20
“This study encompassed 26 clinical variables in 89,568 individuals and yielded an internal test AUC of 0.97, surpassing the AUC of the conventional logistic regression model (AUC of 0.90) [35].”
Influential
New machine-learning models outperform conventional risk assessment tools in Gastrointestinal bleeding
Boros, E., Pintér, J., Molontay, R., Prószéky, K. G., Vörhendi, N., Simon, O. A., Teutsch, B., Pálinkás, D., Frim, L., Tari, E., et al. (2025) · Scientific Reports · doi:10.1038/s41598-025-90986-1 · Cited by 18
“Shung et al. 25 developed multiple ML models outperforming GBS, AIMS65, and pre-endoscopic Rockall scores in assessing a composite endpoint (mortality and interventions).”
The Impact of Artificial Intelligence in the Endoscopic Assessment of Premalignant and Malignant Esophageal Lesions: Present and Future
Lazăr, D. C., Avram, M. F., Faur, A. C., Goldiș, A., Romosan, I., Tăban, S. M., & Cornianu, M. (2020) · Medicina · doi:10.3390/medicina56070364 · Cited by 18
Understanding the Financial Transaction Security through Blockchain and Machine Learning for Fraud Detection in Data Privacy and Security
Masud, S. B., Rana, M. M., Sohag, H. J., Shikder, F., Faraji, M. R., & Hasan, M. M. (2024) · Pakistan Journal of Life and Social Sciences (PJLSS) · doi:10.57239/pjlss-2024-22.2.001296 · Cited by 17
Deep Learning in Upper Gastrointestinal Disorders: Status and Future Perspectives
Bang, C. S. (2020) · Korean Journal of Gastroenterology · doi:10.4166/kjg.2020.75.3.120 · Cited by 18
The automation of doctors and machines: A classification for AI in medicine (ADAM framework)
Kazzazi, F. (2021) · Future Healthcare Journal · doi:10.7861/fhj.2020-0189 · Cited by 17
“26 Decisions to operate are archetypical examples of medical deliberations that require hypothetical-deductive reasoning, individual judgment and heuristics.”
Background
Endoscopist’s Judgment Is as Useful as Risk Scores for Predicting Outcome in Peptic Ulcer Bleeding: A Multicenter Study
Brullet, E., García‐Iglesias, P., Calvet, X., Papo, M., Planella, M., Pardo, A., Junquera, F., Montoliu, S., Ballester, R., Martínez‐Bauer, E., et al. (2020) · Journal of Clinical Medicine · doi:10.3390/jcm9020408 · Cited by 17
“Also, in a very recent study, a machine‐learning model identified patients with upper gastrointestinal bleeding who met a composite endpoint of hospital‐based intervention or death within 30 days with a great area under the curve (AUC) and high levels of specificity, at 100% sensitivity [30].”
Background
Development of a Dynamic Diagnosis Grading System for Infertility Using Machine Learning
Liao, S., Pan, W., Dai, W.-Q., Jin, L., Huang, G., Wang, R., Hu, C., Pan, W., & Tu, H. (2020) · JAMA Network Open · doi:10.1001/jamanetworkopen.2020.23654 · Cited by 17
Utility of a deep learning model and a clinical model for predicting bleeding after endoscopic submucosal dissection in patients with early gastric cancer
Na, J. E., Lee, Y. C., Kim, T. J., Lee, H., Won, H., Min, Y. W., Min, B., Lee, J. H., Rhee, P., & Kim, J. J. (2022) · World Journal of Gastroenterology · doi:10.3748/wjg.v28.i24.2721 · Cited by 16
Prediction of Important Factors for Bleeding in Liver Cirrhosis Disease Using Ensemble Data Mining Approach
Aleksić, A., Nedeljković, S., Jovanović, M., Ranđelović, M., Vuković, M., Stojanović, V. S., Radovanović, R., Ranđelović, M., & Ranđelović, D. M. (2020) · Mathematics · doi:10.3390/math8111887 · Cited by 16
Big Data in Gastroenterology Research
Alizadeh, M. A., Moura, N. S., Schledwitz, A., Patil, S. A., Ravel, J., & Raufman, J. (2023) · International Journal of Molecular Sciences · doi:10.3390/ijms24032458 · Cited by 17
Predicting mortality in patients with nonvariceal upper gastrointestinal bleeding using machine-learning
Ungureanu, B. S., Gheonea, D. I., Florescu, D. N., Iordache, S., Cazacu, S. M., Iovănescu, V. F., Rogoveanu, I., & Turcu-Ştiolică, A. (2023) · Frontiers in Medicine · doi:10.3389/fmed.2023.1134835 · Cited by 15
Machine learning based CAGIB score predicts in-hospital mortality of cirrhotic patients with acute gastrointestinal bleeding
Bai, Z., Lin, S., Sun, M., Yuan, S., Marcondes, M. B., Ma, D., Zhu, Q., Li, Y., He, Y., Philips, C. A., et al. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01883-w · Cited by 16
“Shung et al. developed a ML-based model and showed that its AUC was higher than those of Glasgow-Blatchford, Rockall, and AIMS65 scores 20 .”
Machine learning algorithms to predict intraoperative hemorrhage in surgical patients: a modeling study of real-world data in Shanghai, China
Shi, Y., Zhang, G., Ma, C., Xu, J., Xu, K., Zhang, W., Wu, J., & Xu, L. (2023) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-023-02253-w · Cited by 14
“The value of XGB in post-partum hemorrhage prediction reached 0.93 [55], and the AUC was 0.91 in upper gastrointestinal bleeding [56], which is similar to our study.”
ResultInfluential
Artificial intelligence-assisted system for the assessment of Forrest classification of peptic ulcer bleeding: a multicenter diagnostic study
He, X.-J., Wang, X.-L., Su, T., Yao, L.-J., Zheng, J., Wen, X., Xu, Q., Huang, Q., Chen, L., Chen, C.-X., et al. (2024) · Endoscopy · doi:10.1055/a-2252-4874 · Cited by 14
“An AI-based diagnostic system has been applied to assess the Glasgow – Blatchford and AIMS65 scores of patients with upper GI bleeding using clinical data; it has been shown to be helpful in the early prediction of high risk patients during the pre-endoscopy period [16,17].”
Method
A machine learning model for predicting the lymph node metastasis of early gastric cancer not meeting the endoscopic curability criteria
Kato, M., Hayashi, Y., Uema, R., Kanesaka, T., Yamaguchi, S., Maekawa, A., Yamada, T., Yamamoto, M., Kitamura, S., Inoue, T., et al. (2024) · Gastric Cancer · doi:10.1007/s10120-024-01511-8 · Cited by 14
Application of artificial intelligence in gastroenterology: Potential role in clinical practice
Kuo, C., & Chiu, H. (2021) · Journal of Gastroenterology and Hepatology · doi:10.1111/jgh.15403 · Cited by 12
“24 For example, while treating in ﬂ ammatory bowel disease patients, doctors are very frequently facing the selection between step-up or top-down strategies, selection of different biologic agents, cost consideration, and patient preferences based on different local clinical scenarios or…”
Strengths and limitations of risk stratification tools for patients with upper gastrointestinal bleeding: a narrative review
Alali, A. A., Boustany, A., Martel, M., & Barkun, A. N. (2023) · Expert Review of Gastroenterology & Hepatology · doi:10.1080/17474124.2023.2242252 · Cited by 13
“[51] concluded that the ML model performed better than currently available clinical risk assessment tools (GBS, pRS, AIMS65) in identifying very-low-risk patients defined by a composite outcome (need for blood transfusions, hemostatic”
MethodInfluential
Advancing care for acute gastrointestinal bleeding using artificial intelligence
Shung, D. L. (2021) · Journal of Gastroenterology and Hepatology · doi:10.1111/jgh.15372 · Cited by 14
Self-citation
Assessment of Prognostic Scores for Emergency Department Patients With Upper Gastrointestinal Bleeding
Thiebaud, P., Wassermann, E., de Caluwe, M., Prebin, C., Noel, F., Dechartres, A., Raynal, P.-A., Leblanc, J., & Yordanov, Y. (2024) · Annals of Emergency Medicine · doi:10.1016/j.annemergmed.2024.06.024 · Cited by 12
Early identification of older individuals at risk of mobility decline with machine learning
do Nascimento, C. F., Batista, A. F. D. M., Duarte, Y. A. O., & Filho, A. D. P. C. (2022) · Archives of Gerontology and Geriatrics · doi:10.1016/j.archger.2022.104625 · Cited by 12
Predicting mortality for cirrhotic patients with acute oesophageal variceal haemorrhage using liver‐specific scores
Terres, A. Z., Balbinot, R. S., Muscope, A. L. F., Eberhardt, L. Z., Balensiefer, J. I. L., Cini, B. T., Rost, G. L., Longen, M. L., Schena, B., Balbinot, R. Â., et al. (2021) · GastroHep · doi:10.1002/ygh2.460 · Cited by 11
“Platelets (103/mm3) 102 (63) 90 (55) 115 (72)”
Artificial intelligence for gastroenterology: Singapore artificial intelligence for Gastroenterology Working Group Position Statement
Sung, J. J. �., Savulescu, J., Ngiam, K. Y., An, B., Ang, T. L., Yeoh, K. G., Cham, T., Tsao, S. K. K., & Chua, T. (2023) · Journal of Gastroenterology and Hepatology · doi:10.1111/jgh.16241 · Cited by 13
Scoring systems for risk stratification in upper and lower gastrointestinal bleeding
Radaelli, F., Rocchetto, S., Piagnani, A., Savino, A., Paolo, D. D., Scardino, G., Paggi, S., & Rondonotti, E. (2023) · Best Practice & Research Clinical Gastroenterology · doi:10.1016/j.bpg.2023.101871 · Cited by 13
A new definition for feature selection stability analysis
Lazebnik, T., & Rosenfeld, A. (2024) · Annals of Mathematics and Artificial Intelligence · doi:10.1007/s10472-024-09936-8 · Cited by 14
“These models are often developed on a relatively small amount of records [19] and therefore must demonstrate their generality to other real-world settings – which often does not occur in the model testing phase [20–22].”
Comparison and validation of several scoring systems for non-variceal upper gastrointestinal bleeding: a retrospective study
In, K. R., Oh, Y. E., Moon, H. S., Jung, S., Kang, S. H., Sung, J. K., & Jeong, H. Y. (2024) · Scientific Reports · doi:10.1038/s41598-024-79643-1 · Cited by 11
Digital Health Landscape in Gastroenterology and Hepatology
Mathews, S. C., & Sakulsaengprapha, V. (2020) · Clinical Gastroenterology and Hepatology · doi:10.1016/j.cgh.2020.11.001 · Cited by 11
Comparison of assessment tools in acute upper gastrointestinal bleeding: which one for which decision
Marmo, R., Soncini, M., Bucci, C., & Zullo, A. (2021) · Scandinavian Journal of Gastroenterology · doi:10.1080/00365521.2021.1976268 · Cited by 12
Development of a Success Prediction Model for Crowdfunding Based on Machine Learning Reflecting ESG Information
Park, J., Na, H. J., & Kim, H. (2024) · IEEE Access · doi:10.1109/access.2024.3519219 · Cited by 10
Machine Learning Model Development and Validation for Predicting Outcome in Stage 4 Solid Cancer Patients with Septic Shock Visiting the Emergency Department: A Multi-Center, Prospective Cohort Study
Ko, B. S., Jeon, S., Son, D., Choi, S., Shin, T. G., Jo, Y. H., Ryoo, S. M., Kim, Y., Park, Y. S., Kwon, W. Y., et al. (2022) · Journal of Clinical Medicine · doi:10.3390/jcm11237231 · Cited by 10
“To develop the ML model, a total of five classifiers were considered [10,22].”
Method
A Novel Prediction Tool for Endoscopic Intervention in Patients with Acute Upper Gastro-Intestinal Bleeding
Veisman, I., Oppenheim, A., Maman, R., Kofman, N., Edri, I., Dar, L., Klang, E., Sina, S., Gabriely, D., Levy, I., et al. (2022) · Journal of Clinical Medicine · doi:10.3390/jcm11195893 · Cited by 10
“Introduction Acute upper gastro-intestinal bleeding (UGIB) is a common and urgent medical condition, usually requiring hospital admission with reported annual incidences in the range of 48 to 172 per 100,000 [1].”
Background
Machine Learning Improves the Prediction Rate of Non-Curative Resection of Endoscopic Submucosal Dissection in Patients with Early Gastric Cancer
Yun, H., Huh, C. W., Jung, D. H., Lee, G., Son, N., Kim, J., Youn, Y. H., Park, J. C., Shin, S. K., Lee, S. K., et al. (2022) · Cancers · doi:10.3390/cancers14153742 · Cited by 10
“To overcome these limitations, ML is emerging as an alternative method to improve prediction performance [25].”
Method
Machine Learning Prognostic Models for Gastrointestinal Bleeding Using Electronic Health Record Data
Shung, D. L., & Laine, L. (2020) · The American Journal of Gastroenterology · doi:10.14309/ajg.0000000000000720 · Cited by 10
“These tools include clinical risk scores, such as the Glasgow-Blatchford Score, and, more recently, machine-learning models.(1,2) Currently, such tools are only recommended by guidelines to identify very-low-risk patients who may be discharged from the emergency department with outpatient management.”
MethodSelf-citation
Adoption of New Technologies
Brown, J. R. G., & Berzin, T. M. (2021) · Gastrointestinal Endoscopy Clinics of North America · doi:10.1016/j.giec.2021.05.010 · Cited by 10
Neutrophil-lymphocyte ratio predict outcome of upper gastrointestinal bleeding in emergency
Chen, X., Li, X., Zhao, G., & Xu, W. (2024) · Frontiers in Medicine · doi:10.3389/fmed.2024.1366715 · Cited by 9
Machine learning in the assessment and management of acute gastrointestinal bleeding
Nigam, G. B., Murphy, M. F., Travis, S. P. L., & Stanley, A. J. (2024) · BMJ Medicine · doi:10.1136/bmjmed-2023-000699 · Cited by 9
Self-citation
A machine learning model for early diagnosis of type 1 Gaucher disease using real-life data
Tenenbaum, A., Revel‐Vilk, S., Gazit, S., Roimi, M., Gill, A., Gilboa, D., Paltiel, O., Manor, O., Shalev, V., & Chodick, G. (2024) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2024.111517 · Cited by 8
Prognosis of major bleeding based on residual variables and machine learning for critical patients with upper gastrointestinal bleeding: A multicenter study
Deng, F., Cao, Y., Wang, H. Z., & Zhao, S. (2024) · Journal of Critical Care · doi:10.1016/j.jcrc.2024.154923 · Cited by 8
Machine learning for risk stratification in the emergency department (MARS-ED): a randomized controlled trial
van Dam, P. M. E. L., van Doorn, W. P. T. M., Sevenich, L., Lambriks, L., Cals, J., Bekers, O., Stassen, P. M., & Meex, S. J. R. (2025) · Nature Communications · doi:10.1038/s41467-025-66947-7 · Cited by 8
Endoscopic Diagnosis, Grading, and Treatment of Bleeding Peptic Ulcer Disease
Vakil, N. B. (2023) · Gastrointestinal Endoscopy Clinics of North America · doi:10.1016/j.giec.2023.09.003 · Cited by 8
Development and validation of machine learning models to predict the need for haemostatic therapy in acute upper gastrointestinal bleeding
Nazarian, S., Lo, F. P.-W., Qiu, J., Patel, N., Lo, B. P. L., & Ayaru, L. (2024) · Therapeutic Advances in Gastrointestinal Endoscopy · doi:10.1177/26317745241246899 · Cited by 8
“16 used ML (Gradient Boost) to develop a model to predict a composite outcome of blood transfusion, hospital-based intervention and death which identified very low-risk patients rather than the need for haemostatic intervention alone.”
Method
Risk Assessment of Liver Metastasis in Pancreatic Cancer Patients Using Multiple Models Based on Machine Learning: A Large Population-Based Study
Li, Q., Bai, L., Xing, J., Liu, X., Liu, D., & Hu, X. (2022) · Disease Markers · doi:10.1155/2022/1586074 · Cited by 8
A convenient machine learning model to predict full stomach and evaluate the safety and comfort improvements of preoperative oral carbohydrate in patients undergoing elective painless gastrointestinal endoscopy
Jin, Y., Ma, M., Yan, Y., Guo, Y., Feng, Y., Chen, C., Zhong, Y., Huang, K., Xia, H., Libo, Y., et al. (2023) · Annals of Medicine · doi:10.1080/07853890.2023.2292778 · Cited by 7
“Ml models have been increasingly mentioned in personalized medicine, including the diagnosis and risk prediction of gastrointestinal diseases [18,19]. its convenience and accuracy may help to overcome the limitations of POcUs. therefore, we build an Ml model to discriminate patients with empty-or…”

Authors: Shung, D. L., Au, B., Taylor, R. A., Tay, J. K., Laursen, S. B., Stanley, A. J., Dalton, H. R., Ngu, J., Schultz, M., & Laine, L.
Affiliations: Yale University (US)
Stanford University (US)
Odense University Hospital (DK)
Glasgow Royal Infirmary (GB)
Royal Cornwall Hospital (GB)
Christchurch Hospital (NZ)
Dunedin Public Hospital (NZ)
Corresponding author: Shung, Dennis L.
Laine, Loren
ORCID: Shung, Dennis L.
Taylor, Richard Andrew
Tay, J. Kenneth
Laursen, Stig B.
Stanley, Adrian J.
Dalton, Harry R.
Schultz, Michael
Laine, Loren
Venue: Gastroenterology
Publisher: Elsevier BV
Published: 2020-01
Volume: 158
Issue: 1
Pages: 160-167
ISSN: 0016-5085 (ISSN-L)
DOI: 10.1053/j.gastro.2019.09.009
PMC: PMC7004228
OpenAlex: W2976369463
Semantic Scholar: 49e263c97948d8177c777a1106e0ab0795effa58
Cited by: 227
References: 54
Citations by year: 2026: 22 · 2025: 30 · 2024: 49 · 2023: 29 · 2022: 31 · 2021: 37 · 2020: 25
FWCI: 21.87
Type: journal article
Language: en
Open access: bronze
doi.org
Licence: www.elsevier.com/tdm/userlicense/1.0/ (tdm)
www.elsevier.com/legal/tdmrep-license (tdm)
Locations: Gastroenterology (publishedVersion)
PubMed (publishedVersion)
University of Southern Denmark Research Portal (University of Southern Denmark) (publishedVersion, cc-by-nc-nd)
Europe PMC (PubMed Central) (acceptedVersion)
PubMed Central (acceptedVersion)
Funders: National Institutes of Health (T32 DK007017)
Topics: Health Sciences › Medicine › Gastroenterology › Gastrointestinal Bleeding Diagnosis and Treatment
Health Sciences › Medicine › Gastroenterology › Helicobacter pylori-related gastroenterology studies
Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
Keywords: upper gastrointestinal bleeding, machine learning prediction models, Glasgow-Blatchford score, Rockall score, AIMS65, gradient boosting, hospital-based intervention, 30-day mortality, external validation, risk stratification, endoscopic hemostasis, blood transfusion, internal validation, sensitivity and specificity
MeSH terms: Machine Learning, Adult, Aged, Aged, 80 and over, Blood Transfusion, Emergency Service, Hospital, Female, Gastrointestinal Hemorrhage, Hemostatic Techniques, Humans, Male, Middle Aged, Models, Biological, Prognosis, ROC Curve, Risk Assessment

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Validation of a Machine Learning Model That Outperforms Clinical Risk Scoring Systems for Upper Gastrointestinal Bleeding

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Self-check

Citations

Cites

Cited by

Related

Metadata

Record

About these checks