Performance of intensive care unit severity scoring systems across different Ethnicities in the USA: a retrospective observational study
Open accessPublished version of a preprint
Abstract
BACKGROUND: Despite wide use of severity scoring systems for case-mix determination and benchmarking in the intensive care unit (ICU), the possibility of scoring bias across ethnicities has not been examined. Guidelines on the use of illness severity scores to inform triage decisions for allocation of scarce resources, such as mechanical ventilation, during the current COVID-19 pandemic warrant examination for possible bias in these models. We investigated the performance of the severity scoring systems Acute Physiology and Chronic Health Evaluation IVa (APACHE IVa), Oxford Acute Severity of Illness Score (OASIS), and Sequential Organ Failure Assessment (SOFA) across four ethnicities in two large ICU databases to identify possible ethnicity-based bias. METHODS: Data from the electronic ICU Collaborative Research Database (eICU-CRD) and the Medical Information Mart for Intensive Care III (MIMIC-III) database, built from patient episodes in the USA from 2014-15 and 2001-12, respectively, were analysed for score performance in Asian, Black, Hispanic, and White people after appropriate exclusions. Hospital mortality was the outcome of interest. Discrimination and calibration were determined for all three scoring systems in all four groups, using area under receiver operating characteristic (AUROC) curve for different ethnicities to assess discrimination, and standardised mortality ratio (SMR) or proxy measures to assess calibration. FINDINGS: We analysed 166 751 participants (122 919 eICU-CRD and 43 832 MIMIC-III). Although measurements of discrimination were significantly different among the groups (AUROC ranging from 0·86 to 0·89 [p=0·016] with APACHE IVa and from 0·75 to 0·77 [p=0·85] with OASIS), they did not display any discernible systematic patterns of bias. However, measurements of calibration indicated persistent, and in some cases statistically significant, patterns of difference between Hispanic people (SMR 0·73 with APACHE IVa and 0·64 with OASIS) and Black people (0·67 and 0·68) versus Asian people (0·77 and 0·95) and White people (0·76 and 0·81). Although calibrations were imperfect for all groups, the scores consistently showed a pattern of overpredicting mortality for Black people and Hispanic people. Similar results were seen using SOFA scores across the two databases. INTERPRETATION: The systematic differences in calibration across ethnicities suggest that illness severity scores reflect statistical bias in their predictions of mortality. FUNDING: There was no specific funding for this study.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Bias in clinical parameters such as glomerular filtration rate, pulse oximetry (with undiagnosed hidden hypoxemia for historically underserved patients), and differences in ICU severity scoring shape radiology data sets for machine learning through a cascade effect as they affect ordering patterns for medical images.
“Bias in clinical parameters such as glomerular filtration rate, pulse oximetry (with undiagnosed hidden hypoxemia for historically underserved patients), 35 differences in ICU severity scoring 36 shape radiology data sets for machine learning through a cascade effect as they affect ordering patterns for medical images.”
From AI pitfalls and what not to do: mitigating bias in AI (2023) · cites this paper as [36]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Vincent, J. L., Moreno, R., Takala, J., Willatts, S., De Mendonça, A., Bruining, H., Reinhart, C. K., Suter, P. M., & Thijs, L. G. (1996) · Intensive Care Medicine · doi:10.1007/bf01709751 · Cited by 11,674 · see citing works
MIMIC-III, a freely accessible critical care database
Johnson, A. E. W., Pollard, T. J., Shen, L., Lehman, L.-W. H., Feng, M., Ghassemi, M., Moody, B., Szolovits, P., Anthony Celi, L., & Mark, R. G. (2016) · Scientific Data · doi:10.1038/sdata.2016.35 · Cited by 8,141 · see citing works
Dissecting racial bias in an algorithm used to manage the health of populations
Obermeyer, Z., Powers, B., Vogeli, C., & Mullainathan, S. (2019) · Science · doi:10.1126/science.aax2342 · Cited by 7,137 · see citing works
Pollard, T. J., Johnson, A. E. W., Raffa, J. D., Celi, L. A., Mark, R. G., & Badawi, O. (2018) · Scientific Data · doi:10.1038/sdata.2018.178 · Cited by 1,993 · see citing works
Health equity in England: the Marmot review 10 years on
Marmot, M. (2020) · BMJ · doi:10.1136/bmj.m693 · Cited by 1,834 · see citing works
Zimmerman, J. E., Kramer, A. A., McNair, D. S., & Malila, F. M. (2006) · Critical Care Medicine · doi:10.1097/01.ccm.0000215112.84523.f0 · Cited by 1,764 · see citing works
Hidden in Plain Sight — Reconsidering the Use of Race Correction in Clinical Algorithms
Vyas, D. A., Eisenstein, L. G., & Jones, D. S. (2020) · New England Journal of Medicine · doi:10.1056/nejmms2004740 · Cited by 1,731 · see citing works
Socioeconomic Disparities in Health in the United States: What the Patterns Tell Us
Braveman, P. A., Cubbin, C., Egerter, S., Williams, D. R., & Pamuk, E. (2010) · American Journal of Public Health · doi:10.2105/ajph.2009.166082 · Cited by 1,548 · see citing works
Race, Socioeconomic Status and Health: Complexities, Ongoing Challenges and Research Opportunities
Williams, D. R., Mohammed, S., Leavell, J. P., & Collins, C. A. (2010) · Annals of the New York Academy of Sciences · doi:10.1111/j.1749-6632.2009.05339.x · Cited by 1,407 · see citing works
Zhang, M., Zhou, Q., Luo, Y., Nguyen, T., Rosenblatt, M. I., & Guaiquil, V. H. (2018) · PLoS ONE · doi:10.1371/journal.pone.0191962 · Cited by 990 · see citing works
Clinical review: Scoring systems in the critically ill
Vincent, J.-L., & Moreno, R. (2010) · Critical Care · doi:10.1186/cc8204 · Cited by 634 · see citing works
Thorsen-Meyer, H.-C., Nielsen, A. B., Nielsen, A. P., Kaas-Hansen, B. S., Toft, P., Schierbeck, J., Strøm, T., Chmura, P. J., Heimann, M., Dybdahl, L., et al. (2020) · The Lancet Digital Health · doi:10.1016/s2589-7500(20)30018-2 · Cited by 336 · see citing works
Johnson, A. E. W., Kramer, A. A., & Clifford, G. D. (2013) · Critical Care Medicine · doi:10.1097/ccm.0b013e31828a24fe · Cited by 292 · see citing works
An international comparative study of blood pressure in populations of European vs. African descent
Cooper, R., Wolf-Maier, K., Luke, A. H., Adeyemo, A., Banegas, J. R., Forrester, T., Giampaoli, S., Joffres, M., Kastarinen, M. J., Primatesta, P., et al. (2005) · BMC Medicine · doi:10.1186/1741-7015-3-2 · Cited by 189 · see citing works
AAPA statement on biological aspects of race
(1996) · American Journal of Physical Anthropology · doi:10.1002/ajpa.1331010408 · Cited by 175 · see citing works
Orlović, M., Smith, K., & Mossialos, E. A. (2018) · SSM - Population Health · doi:10.1016/j.ssmph.2018.100331 · Cited by 143 · see citing works
Marafino, B. J., Park, M., Davies, J. M., Thombley, R. L., Luft, H. S., Sing, D. C., Kazi, D. S., DeJong, C., Boscardin, W. J., Dean, M. L., et al. (2018) · JAMA Network Open · doi:10.1001/jamanetworkopen.2018.5097 · Cited by 120 · see citing works
Optimal intensive care outcome prediction over time using machine learning
Meiring, C., Dixit, A., Harris, S. K., MacCallum, N. S., Brealey, D. A., Watkinson, P., Jones, A., Ashworth, S., Beale, R. J., Brett, S. J., et al. (2018) · PLoS ONE · doi:10.1371/journal.pone.0206862 · Cited by 94 · see citing works
Calvert, J. S., Mao, Q., Hoffman, J. L., Jay, M., Desautels, T., Mohamadlou, H., Chettipally, U. K., & Das, R. (2016) · Annals of Medicine and Surgery · doi:10.1016/j.amsu.2016.09.002 · Cited by 77 · see citing works
African-American COVID-19 Mortality
Ferdinand, K. C., & Nasser, S. A. (2020) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2020.04.040 · Cited by 77 · see citing works
Real-time mortality prediction in the Intensive Care Unit.
Johnson, A. E. W., & Mark, R. G. (2017) · PubMed · Cited by 73 · see citing works
Temporal Trends in Critical Care Outcomes in U.S. Minority-Serving Hospitals
Danziger, J., de la Hoz, M. Á. A., Li, W., Komorowski, M., Deliberato, R. O., Rush, B. N. M., Mukamal, K. J., Celi, L. A., & Badawi, O. (2020) · American Journal of Respiratory and Critical Care Medicine · doi:10.1164/rccm.201903-0623oc · Cited by 69 · see citing works
Diagnosing bias in data-driven algorithms for healthcare
Wiens, J., Price, W. N., & Sjoding, M. W. (2020) · Nature Medicine · doi:10.1038/s41591-019-0726-6 · Cited by 59 · see citing works
Poncet, A., Perneger, T., Merlani, P., Capuzzo, M., & Combescure, C. (2017) · Critical Care · doi:10.1186/s13054-017-1673-6 · Cited by 52 · see citing works
Rush, B. N. M., Danziger, J., Walley, K. R., Kumar, A., & Celi, L. A. (2020) · Critical Care Medicine · doi:10.1097/ccm.0000000000004375 · Cited by 44 · see citing works
(2018) · doi:10.15620/cdc:82212 · Cited by 40 · see citing works
Health equity and distributive justice considerations in critical care resource allocation
Galiatsatos, P., Kachalia, A. B., Belcher, H. M. E., Hughes, M. T., Kahn, J. P., Rushton, C. H., Suárez, J. I., Biddison, L. D., & Golden, S. H. (2020) · The Lancet Respiratory Medicine · doi:10.1016/s2213-2600(20)30277-0 · Cited by 34 · see citing works
McLennan, S., Lee, M. M., Fiske, A., & Celi, L. A. (2020) · The American Journal of Bioethics · doi:10.1080/15265161.2020.1819470 · Cited by 21 · see citing works
Severity of Illness Scores May Misclassify Critically Ill Obese Patients*
Deliberato, R. O., Ko, S. Q., Komorowski, M., de la Hoz, M. Á. A., Frushicheva, M. P., Raffa, J. D., Johnson, A. E. W., Celi, L. A., & Stone, D. J. (2017) · Critical Care Medicine · doi:10.1097/ccm.0000000000002868 · Cited by 19 · see citing works
Migrant and minority family members in the intensive care unit. A review of the literature
Quindemil, K., Nagl‐Cupal, M., Anderson, K. H., & Mayer, H. (2013) · HeilberufeScience · doi:10.1007/s16024-013-0171-2 · Cited by 14 · see citing works
Mortality Prediction of ICU patients using Machine Leaning
Sharma, A. K., Shukla, A., Tiwari, R., & Mishra, A. (2017) · International Conference on Compute and Data Analysis (ICCDA) · doi:10.1145/3093241.3093267 · Cited by 11 · see citing works
(2020) · Nature Machine Intelligence · doi:10.1038/s42256-020-0210-5 · Cited by 6 · see citing works
Rehman, K. A., Unai, S., & Reed, G. W. (2020) · Journal of the American College of Cardiology · doi:10.1016/s0735-1097(20)33373-8 · Cited by 5 · see citing works
Real-time mortality prediction in the Intensive Care Unit
Johnson, A. E. W., & Mark, R. G. (2018) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 2 · see citing works
Flagged first, then by citations
Research works printed but not linked: 1
- [17]Revisions to the Standards for the Classification of Federal Data on Race and Ethnicity. Off Manag Budget; Fed Regist 2010; 75: 56928– 35.
Cited by
Sorted by citations
AI recognition of patient race in medical imaging: a modelling study
Gichoya, J. W., Banerjee, I., Bhimireddy, A. R., Burns, J. L., Celi, L. A., Chen, L.-C., Correa, R., Dullerud, N., Ghassemi, M., Huang, S.-C., et al. (2022) · The Lancet Digital Health · doi:10.1016/s2589-7500(22)00063-2 · Cited by 611 · see citing works
AI pitfalls and what not to do: mitigating bias in AI
Gichoya, J. W., Thomas, K., Celi, L. A., Safdar, N., Banerjee, I., Banja, J. D., Seyyed-Kalantari, L., Trivedi, H., & Purkayastha, S. (2023) · The British Journal of Radiology · doi:10.1259/bjr.20230023 · Cited by 277 · see citing works
“…such as glomerular filtration rate, pulse oximetry (with undiagnosed hidden hypoxemia for historically underserved patients), 35 differences in ICU severity scoring 36 shape radiology data sets for machine learning through a cascade effect as they affect ordering patterns for medical images.”
The Impact of Health Care Algorithms on Racial and Ethnic Disparities
Siddique, S. M., Tipton, K. N., Leas, B. F., Jepson, C., Aysola, J., Cohen, J. B., Flores, E. J., Harhay, M. O., Schmidt, H., Weissman, G. E., et al. (2024) · Annals of Internal Medicine · doi:10.7326/m23-2960 · Cited by 88 · see citing works
Kruser, J. M., Ashana, D. C., Courtright, K. R., Kross, E. K., Neville, T. H., Rubin, E., Schenker, Y., Sullivan, D. R., Thornton, J. D., Viglianti, E. M., et al. (2023) · Annals of the American Thoracic Society · doi:10.1513/annalsats.202310-925st · Cited by 69 · see citing works
Liu, X., Hu, P., Yeung, W. L. K., Zhang, Z., Ho, V., Liu, C., DuMontier, C., Thoral, P. J., Mao, Z., Cao, D., et al. (2023) · The Lancet Digital Health · doi:10.1016/s2589-7500(23)00128-0 · Cited by 61 · see citing works
Charoensappakit, A., Sae‐khow, K., Rattanaliam, P., Vutthikraivit, N., Pecheenbuvan, M., Udomkarnjananun, S., & Leelahavanichkul, A. (2023) · Scientific Reports · doi:10.1038/s41598-023-46663-2 · Cited by 55 · see citing works
Harnisch, L.-O., & Moerer, O. (2021) · Membranes · doi:10.3390/membranes11080584 · Cited by 43 · see citing works
de Hond, A., van Buchem, M. M., & Hernandez‐Boussard, T. (2022) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocac156 · Cited by 40 · see citing works
Barboi, C., Tzavelis, A., & Muhammad, L. N. (2022) · JMIR Medical Informatics · doi:10.2196/35293 · Cited by 38 · see citing works
Critical Bias in Critical Care Devices
Charpignon, M.-L., Byers, J., Cabral, S., Celi, L. A., Fernandes, C., Gallifant, J., Lough, M. E., Mlombwa, D., Moukheiber, L., Ong, B. A., et al. (2023) · Critical Care Clinics · doi:10.1016/j.ccc.2023.02.005 · Cited by 35 · see citing works
Ahlberg, C. D., Wallam, S., Tirba, L. A., Itumba, S. N., Gorman, L., & Galiatsatos, P. (2023) · Journal of Critical Care · doi:10.1016/j.jcrc.2023.154324 · Cited by 30 · see citing works
de Hond, A., van Buchem, M. M., Fanconi, C., Roy, M., Blayney, D. W., Kant, I. M. J., Steyerberg, E. W., & Hernandez‐Boussard, T. (2024) · JMIR Medical Informatics · doi:10.2196/51925 · Cited by 30 · see citing works
Clinical data mining: challenges, opportunities, and recommendations for translational applications
Qiao, H., Chen, Y., Qian, C., & Guo, Y. (2024) · Journal of Translational Medicine · doi:10.1186/s12967-024-05005-0 · Cited by 27 · see citing works
Minimizing bias when using artificial intelligence in critical care medicine
Ranard, B. L., Park, S., Jia, Y., Zhang, Y., Alwan, F. S., Celi, L. A., & Lusczek, E. R. (2024) · Journal of Critical Care · doi:10.1016/j.jcrc.2024.154796 · Cited by 27 · see citing works
Efficient adversarial debiasing with concept activation vector — Medical image case-studies
Correa, R., Pahwa, K., Patel, B. N., Vachon, C. M., Gichoya, J. W., & Banerjee, I. (2023) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2023.104548 · Cited by 23 · see citing works
The Glycemic Ratio Is Strongly and Independently Associated With Mortality in the Critically Ill
Roberts, G., Krinsley, J. S., Preiser, J., Quinn, S. J., Rule, P. R., Brownlee, M., Schwartz, M. W., Umpierrez, G. E., & Hirsch, I. B. (2022) · Journal of Diabetes Science and Technology · doi:10.1177/19322968221124114 · Cited by 22 · see citing works
Jin, G., Hu, W., Zeng, L., Ma, B., & Zhou, M. (2023) · Frontiers in Neurology · doi:10.3389/fneur.2023.1148185 · Cited by 22 · see citing works
Zheng, Z. Q., Luo, J., Zhu, Y., Du, L., Lan, L., Zhou, X., Yang, X., & Huang, S. (2025) · Journal of Medical Internet Research · doi:10.2196/69293 · Cited by 21 · see citing works
Jin, G., Hu, W., Zeng, L., Diao, M., Chen, H., Chen, J., Gu, N., Qiu, K., Lv, H., Pan, L., et al. (2023) · Scientific Reports · doi:10.1038/s41598-023-39781-4 · Cited by 18 · see citing works
Jin, P., Bian, Y., Cui, Q., Liang, X., Sun, Y., & Zheng, Q. (2024) · Scientific Reports · doi:10.1038/s41598-024-73788-9 · Cited by 17 · see citing works
Association Between Patient Race and Ethnicity and Use of Invasive Ventilation in the United States
Abdelmalek, F. M., Angriman, F., Moore, J. E., Liu, K., Burry, L. D., Seyyed-Kalantari, L., Mehta, S., Gichoya, J. W., Celi, L. A., TOMLINSON, G. A., et al. (2023) · Annals of the American Thoracic Society · doi:10.1513/annalsats.202305-485oc · Cited by 18 · see citing works
Gershengorn, H. B., Patel, S., Shukla, B., Warde, P. R., Soorus, S. M., Holt, G. E., Kett, D. H., Parekh, D. J., & Ferreira, T. (2021) · Annals of the American Thoracic Society · doi:10.1513/annalsats.202106-680oc · Cited by 17 · see citing works
Circulating extracellular vesicles are associated with the clinical outcomes of sepsis
Li, P., Wu, Y., Goodwin, A. J., IV, Wolf, B. J., Halushka, P. V., Wang, H., Zingarelli, B., & Fan, H. (2023) · Frontiers in Immunology · doi:10.3389/fimmu.2023.1150564 · Cited by 15 · see citing works
Huang, W., Xie, H.-J., Fan, H.-T., Yan, M., & Hong, Y. (2021) · Medicine · doi:10.1097/md.0000000000027380 · Cited by 14 · see citing works
Zarei, J., Jamshidnezhad, A., Shoushtari, M. H., Hadianfard, A. M., Cheraghi, M., & Sheikhtaheri, A. (2022) · Journal of Healthcare Engineering · doi:10.1155/2022/1644910 · Cited by 15 · see citing works
Kataria, S., & Ravindran, V. K. (2021) · Rheumatology · doi:10.1093/rheumatology/keab674 · Cited by 12 · see citing works
Fang, L., Zhou, M., Mao, F., Diao, M., Hu, W., & Jin, G. (2024) · PLoS ONE · doi:10.1371/journal.pone.0302227 · Cited by 11 · see citing works
Keim‐Malpass, J., Moorman, L. P., Monfredi, O., Clark, M., & Bourque, J. M. (2022) · Learning Health Systems · doi:10.1002/lrh2.10323 · Cited by 10 · see citing works
Tan, D. J., Chen, J., Zhou, Y., Ong, J. S. Q., Sin, R. J. X., Bui, T. V., Mehta, A. A., Feng, M., & See, K. C. (2024) · European journal of medical research · doi:10.1186/s40001-023-01616-3 · Cited by 10 · see citing works
Prediction of incident atrial fibrillation in hypertrophic cardiomyopathy
Losi, M. A., Monda, E., Lombardi, R., Lioncino, M., Canciello, G., Rubino, M., Todde, G., Caiazza, M., Borrelli, F., Fusco, A., et al. (2023) · International Journal of Cardiology · doi:10.1016/j.ijcard.2023.131575 · Cited by 9 · see citing works
Li, Q., Tong, Y., Wang, H., Ren, J., Liu, S., Liu, T., Qu, K., Liu, C., & Zhang, J. (2021) · International Journal of General Medicine · doi:10.2147/ijgm.s345050 · Cited by 8 · see citing works
Overt and Occult Hypoxemia in Patients Hospitalized With COVID-19
Gadrey, S. M., Mohanty, P., Haughey, S. P., Jacobsen, B. A., Dubester, K. J., Webb, K. M., Kowalski, R. L., Dreicer, J. J., Andris, R. T., Clark, M., et al. (2023) · Critical Care Explorations · doi:10.1097/cce.0000000000000825 · Cited by 8 · see citing works
Operationalising fairness in medical algorithms
Parbhoo, S., Gichoya, J. W., Celi, L. A., & de la Hoz, M. Á. A. (2022) · BMJ Health & Care Informatics · doi:10.1136/bmjhci-2022-100617 · Cited by 8 · see citing works
Griffee, M., Thomson, D. A., Fanning, J. P., Rosenberger, D., Barnett, A. G., White, N. M., Suen, J. Y., Fraser, J. F., Bassi, G. L., Cho, S., et al. (2023) · International Journal for Equity in Health · doi:10.1186/s12939-023-02051-w · Cited by 7 · see citing works
SOFA score performs worse than age for predicting mortality in patients with COVID-19
Sherak, R. A. G., Sajjadi, S., Khimani, N., Tolchin, B. D., Jubanyik, K. J., Taylor, R. A., Schulz, W. L., Mortazavi, B. J., & Haimovich, A. D. (2024) · PLoS ONE · doi:10.1371/journal.pone.0301013 · Cited by 7 · see citing works
Modise, L. M., Avanaki, M. A., Ameen, S., Celi, L. A., Chen, V. X. Y., Cordes, A., Elmore, M., Fiske, A., Gallifant, J., Hayes, M., et al. (2025) · PLOS Digital Health · doi:10.1371/journal.pdig.0000495 · Cited by 6 · see citing works
Liu, X., Yeung, W. L. K., Chen, Z., Hao, S., Yang, Z., Sun, X., Liu, C., Mao, Z., Yan, M., Yan, W., et al. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01875-w · Cited by 6 · see citing works
Vital signs as a source of racial biasPreprint
Velichkovska, B., Gjoreski, H., Denkovski, D., Kalendar, M., Mamandipoor, B., Celi, L. A., & Osmani, V. (2022) · medRxiv · doi:10.1101/2022.02.03.22270291 · Cited by 6 · see citing works
Ethical Considerations of Artificial Intelligence Applications in Healthcare
Gichoya, J. W., MELTZER, C. C., Newsome, J. M., Correa, R., Trivedi, H., Banerjee, I., Davis, M. A., & Celi, L. A. (2022) · Contemporary medical imaging · doi:10.1007/978-3-030-92087-6_52 · Cited by 4 · see citing works
She, Y., Zhou, L., & Li, Y. (2023) · Seizure · doi:10.1016/j.seizure.2023.11.017 · Cited by 4 · see citing works
Ethnicity-based bias in clinical severity scores
Gumbsch, T., & Borgwardt, K. M. (2021) · The Lancet Digital Health · doi:10.1016/s2589-7500(21)00044-3 · Cited by 4 · see citing works
Wang, T., Lee, S., Wong, L.-T., & Chao, W. (2024) · International Journal of Clinical Practice · doi:10.1155/ijcp/5202122 · Cited by 3 · see citing works
Dai, X., Wei, H., Zou, D., Yang, Y., Zhang, C., Chen, J., & Hu, C. (2025) · BMC Anesthesiology · doi:10.1186/s12871-025-02906-5 · Cited by 3 · see citing works
ICU scoring systems: current perspectives and future directions
Salluh, J. I., Moralez, G. M., Tracy, A., & Deliberato, R. O. (2025) · Current Opinion in Critical Care · doi:10.1097/mcc.0000000000001305 · Cited by 4 · see citing works
Yao, X., Xue, H., & Luo, J. (2025) · International Journal of COPD · doi:10.2147/copd.s521192 · Cited by 3 · see citing works
Hauschildt, K. E., Thornton, J. D., Brown, C. E., Hope, A. A., Jain, S., Valley, T. S., Aslakson, R. A., Carlton, E. F., Iwashyna, T. J., Mikkelsen, M. E., et al. (2025) · American Journal of Respiratory and Critical Care Medicine · doi:10.1164/rccm.202509-2229st · Cited by 4 · see citing works
Zhou, W., & He, Y. (2023) · Acta Neurochirurgica · doi:10.1007/s00701-023-05811-3 · Cited by 3 · see citing works
Liu, X., Shen, M., Lie, M., Zhang, Z., Liu, C., Li, D., Mark, R. G., Zhang, Z., & Celi, L. A. (2024) · Critical Care Explorations · doi:10.1097/cce.0000000000001033 · Cited by 3 · see citing works
Kamel, T., & Boulain, T. (2026) · Critical Care · doi:10.1186/s13054-026-05948-4 · Cited by 2 · see citing works
Karwa, M. L., Naqvi, A. A., Betchen, M., & Puri, A. K. (2024) · Critical Care Clinics · doi:10.1016/j.ccc.2024.03.001 · Cited by 2 · see citing works
Decision-Making Under Resource Constraints
Beil, M., van Heerden, P. V., & Sviri, S. (2022) · Lessons from the ICU · doi:10.1007/978-3-030-94133-8_16 · Cited by 2 · see citing works
Investigating Presence of Ethnoracial Bias in Clinical Data using Machine LearningPreprint
Velichkovska, B., Gjoreski, H., Denkovski, D., Kalendar, M., Celi, L. A., & Osmani, V. (2021) · medRxiv · doi:10.1101/2021.09.01.21262949 · Cited by 2 · see citing works
Wang, J., Ding, P.-F., Peng, Z., Hang, C., & Li, W. (2025) · Frontiers in Neurology · doi:10.3389/fneur.2025.1552462 · Cited by 1 · see citing works
Velichkovska, B., Gjoreski, H., Denkovski, D., Kalendar, M., Dankwa‐Mullan, I., Gichoya, J. W., Martinez‐Martin, N., Celi, L. A., & Osmani, V. (2025) · BMJ Health & Care Informatics · doi:10.1136/bmjhci-2024-101098 · Cited by 5 · see citing works
ReAdmit : Predicting Early Unplanned ICU Readmission Using Radiology Notes and Structured Data
Hu, H., Ma, L., Ge, H., Li, J., Feng, T., & Wu, X. (2025) · Nursing in Critical Care · doi:10.1111/nicc.70200 · Cited by 1 · see citing works
Explainable AI for mortality prediction: a comparative study using the MIMIC-III dataset
Shafiabady, N., Akume, D., Haghighat, M., Din, F. U., Sattarshetty, K., Karim, A., Zhou, J. J., & Alsharaydeh, E. (2026) · BMJ Health & Care Informatics · doi:10.1136/bmjhci-2024-101406 · Cited by 1 · see citing works
Lan, L., Huang, S., Chen, Y., Lu, S., & Luo, J. (2026) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-026-03448-7 · Cited by 1 · see citing works
Liu, P., Li, X., & Zhao, Y. (2026) · Journal of Racial and Ethnic Health Disparities · doi:10.1007/s40615-026-03041-8 · Cited by 1 · see citing works
Gichoya, J. W., & Celi, L. A. (2021) · American Journal of Roentgenology · doi:10.2214/ajr.21.26020 · Cited by 1 · see citing works
Liu, X., Shen, M., Lie, M., Zhang, Z., Li, D., Liu, C., Mark, R. G., Zhang, Z., & Celi, L. A. (2022) · medRxiv · doi:10.1101/2022.08.01.22277736 · Cited by 1 · see citing works
AI learns racial information from the values of vital signsPreprint
Velichkovska, B., Gjoreski, H., Denkovski, D., Kalendar, M., Dankwa‐Mullan, I., Gichoya, J. W., Martinez‐Martin, N., Celi, L. A., & Osmani, V. (2023) · medRxiv · doi:10.1101/2023.12.11.23299819 · Cited by 1 · see citing works
Zeng, C., Xiang, W., Allen, A. O., Shokouhi, S., Chetty, S. C., Barnes, G., Iqbal, Z., Tsou, P., Singh, N. P., Calvert, J. S., et al. (2022) · Research Square · doi:10.21203/rs.3.rs-1743328/v1
Zeng, C., Xiang, W., Allen, A. O., Shokouhi, S., Chetty, S. C., Barnes, G., Iqbal, Z., Tsou, P., Calvert, J. S., Hoffman, J. L., et al. (2022) · Research Square · doi:10.21203/rs.3.rs-1735655/v1
Association between patient race/ethnicity and invasive ventilation in hypoxemic respiratory failurePreprint
Abdelmalek, F. M., Angriman, F., Moore, J. E., Liu, K., Burry, L. D., Seyyed-Kalantari, L., Mehta, S., Gichoya, J. W., Celi, L. A., TOMLINSON, G. A., et al. (2022) · medRxiv · doi:10.1101/2022.10.31.22281761
Mikell, I. M., Hoggard, C. L. S., & Schmidt, H. (2023) · The AMA Journal of Ethic · doi:10.1001/amajethics.2023.179
PRIYADHARSHINI, K., Karpagam, P., & KalaBarathi, S. (2023) · Cardiometry · doi:10.18137/cardiometry.2023.26.742746
Sun, A., Yang, W., Sun, D., Lan, Y., Chen, Y., Han, Z., Liu, W., & Lin, Q. (2024) · doi:10.2196/preprints.59166
Machine Learning and Medical Data: Predicting ICU Mortality and Re-admission Risks
Bayram, U., & Roy, R. (2024) · Journal of Advanced Research in Natural and Applied Sciences · doi:10.28979/jarnas.1533962
Lin, F., Xu, D., Tang, J., Huang, T., Zhang, J., Li, L., & Yang, Z. (2025) · Respiratory Medicine · doi:10.1016/j.rmed.2025.108273
Ethical Issues in Preparing for and Responding to Infectious Outbreaks and Bioterror Attacks
Wynia, M. K., Abbott, J., & Little, C. (2023) · Elsevier eBooks · doi:10.1016/b978-0-323-54841-0.00014-7
Severity scoring and outcome prediction
Barlow, C. J., & Pilcher, D. V. (2025) · Elsevier eBooks · doi:10.1016/b978-0-443-11611-7.00010-4
Tan, D. J., Chen, J., Zhou, Y., Ong, J. S. Q., Sin, R. J. X., Bui, T. V., Mehta, A. A., Feng, M., & See, K. C. (2024) · Figshare · doi:10.6084/m9.figshare.26663800
Tan, D. J., Chen, J., Zhou, Y., Ong, J. S. Q., Sin, R. J. X., Bui, T. V., Mehta, A. A., Feng, M., & See, K. C. (2024) · Figshare · doi:10.6084/m9.figshare.26663812
Tan, D. J., Chen, J., Zhou, Y., Ong, J. S. Q., Sin, R. J. X., Bui, T. V., Mehta, A. A., Feng, M., & See, K. C. (2024) · Figshare · doi:10.6084/m9.figshare.26663806
Tan, D. J., Chen, J., Zhou, Y., Ong, J. S. Q., Sin, R. J. X., Bui, T. V., Mehta, A. A., Feng, M., & See, K. C. (2024) · Figshare · doi:10.6084/m9.figshare.26663818
Griffee, M., Thomson, D. A., Fanning, J. P., Rosenberger, D., Barnett, A. G., White, N. M., Suen, J. Y., Fraser, J. F., Bassi, G. L., & Cho, S.-M. (2024) · Figshare · doi:10.6084/m9.figshare.26652930
Tan, D. J., Chen, J., Zhou, Y., Ong, J. S. Q., Sin, R. J. X., Bui, T. V., Mehta, A. A., Feng, M., & See, K. C. (2024) · Figshare · doi:10.6084/m9.figshare.26663830
Tan, D. J., Chen, J., Zhou, Y., Ong, J. S. Q., Sin, R. J. X., Bui, T. V., Mehta, A. A., Feng, M., & See, K. C. (2024) · Figshare · doi:10.6084/m9.figshare.26663803
Tan, D. J., Chen, J., Zhou, Y., Ong, J. S. Q., Sin, R. J. X., Bui, T. V., Mehta, A. A., Feng, M., & See, K. C. (2024) · Figshare · doi:10.6084/m9.figshare.26663824
Griffee, M., Thomson, D. A., Fanning, J. P., Rosenberger, D., Barnett, A. G., White, N. M., Suen, J. Y., Fraser, J. F., Bassi, G. L., & Cho, S.-M. (2024) · Figshare · doi:10.6084/m9.figshare.26652927
The Impact of Black Race on Pneumonia Severity Scoring Systems
Bradley, J. A., WALLACE, M., Chandler, T. R., Furmanek, S. P., Ramirez, J., & Cavallazzi, R. (2026) · CHEST Pulmonary · doi:10.1016/j.chpulm.2026.100234
Wadia, H. E., Karim, M., Buh, A. W., Kabli, A. O., Biyani, N., Shorr, R., Lee, I., Fremont, D., Wald, R., English, S. W., et al. (2026) · Kidney International Reports · doi:10.1016/j.ekir.2026.106560
Zhang, J., Dai, J., & Lyu, Q. (2026) · Research Square · doi:10.21203/rs.3.rs-10529143/v1
Yang, S., Zong, J., Yang, Q., Zhou, W., & Ji, K. (2026) · Diabetes Research and Clinical Practice · doi:10.1016/j.diabres.2026.113579
Anakapalli, M., Mangu, H., Nanaiah, P. B., Arulmurugan, B., Nair, H. R., & Maheswarappa, A. (2026) · International Journal of Research in Medical Sciences · doi:10.18203/2320-6012.ijrms20263491
- Authors
- Sarkar, R., Martin, C., Mattie, H., Gichoya, J. W., Stone, D. J., & Celi, L. A.
- Affiliations
- King's College London (GB)
- University College London (GB)
- Harvard University (US)
- Emory University (US)
- University of Virginia Medical Center (US)
- Beth Israel Deaconess Medical Center (US)
- Corresponding author
- Celi, Leo Anthony
- ORCID
- Venue
- The Lancet Digital Health
- Publisher
- Elsevier BV
- Published
- 2021-04
- Volume
- 3
- Issue
- 4
- Pages
- e241-e249
- ISSN
- 2589-7500 (ISSN-L)
- PMC
- PMC7836131
- OpenAlex
- W3137590022
- Cited by
- 85
- References
- 34
- Citations by year
- 2026: 9 · 2025: 12 · 2024: 22 · 2023: 20 · 2022: 14 · 2021: 8
- FWCI
- 11.53
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- The Lancet Digital Health (publishedVersion, cc-by-nc-nd)
- medRxiv (acceptedVersion, cc-by-nd)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion, cc-by-nc-nd)
- PubMed Central (submittedVersion, cc-by-nd)
- DSpace@MIT (Massachusetts Institute of Technology) (submittedVersion, cc-by-nc-nd)
- Funders
- National Institute of Biomedical Imaging and Bioengineering (R01 EB017205)
- National Institutes of Health
- Topics
- Health Sciences › Medicine › Epidemiology › Sepsis Diagnosis and Treatment
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Health Sciences › Medicine › Emergency Medicine › Emergency and Acute Care Studies
- Keywords
- severity scoring systems, SOFA score, ethnic prejudice, racial disparities, in-hospital mortality prediction, discrimination and calibration, Black patients, Hispanic patients, eICU-CRD, MIMIC-III
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
