Algor-ethics: charting the ethical path for AI in critical care
The DOI matches its Crossref record; 14 of the 14 references it cites were found in a registry or an index.
Abstract
The integration of Clinical Decision Support Systems (CDSS) based on artificial intelligence (AI) in healthcare is groundbreaking evolution with enormous potential, but its development and ethical implementation, presents unique challenges, particularly in critical care, where physicians often deal with life-threating conditions requiring rapid actions and patients unable to participate in the decisional process. Moreover, development of AI-based CDSS is complex and should address different sources of bias, including data acquisition, health disparities, domain shifts during clinical use, and cognitive biases in decision-making. In this scenario algor-ethics is mandatory and emphasizes the integration of ‘Human-in-the-Loop’ and ‘Algorithmic Stewardship’ principles, and the benefits of advanced data engineering. The establishment of Clinical AI Departments (CAID) is necessary to lead AI innovation in healthcare, ensuring ethical integrity and human-centered development in this rapidly evolving field.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“This mismatch can produce substantial data drift and a corresponding drop in accuracy when applied to patients from a different institution altogether, posing a risk to patient safety wherever models are deployed without rigorous local validation.56,60”
From Translational Potential and Explainability of Artificial Intelligence-Based Clinical Decision Support for Adults in Intensive Care: A Scoping Review (2026) · cites this paper as [60]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Benjamens, S., Dhunnoo, P., & Meskó, B. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-00324-0 · Cited by 1,215
A Universal Model of Diagnostic Reasoning
Croskerry, P. (2009) · Academic Medicine · doi:10.1097/acm.0b013e3181ace703 · Cited by 1,016 · cited 1×
“This understanding is crucial in healthcare, where the decisional process is usually complex and clinical judgment should follow a structured pathway and should not merely follow the intuitive approach [34].”
BackgroundBias in artificial intelligence algorithms and recommendations for mitigation
Nazer, L. H., Zatarah, R., Waldrip, S., Ke, J. X. C., Moukheiber, M., Khanna, A. K., Hicklen, R. S., Moukheiber, L., Moukheiber, D., Ma, H., et al. (2023) · PLOS Digital Health · doi:10.1371/journal.pdig.0000278 · Cited by 621 · cited 2×
“…other sources of bias have been reported in the phases of algorithm development such as the managing of missing data or outliers, features selection, or modeling of underrepresented groups [23, 28], data sharing restrictions and data privacy represent a fundamental issue that should be considered.”
BackgroundDo as AI say: susceptibility in deployment of clinical decision-aids
Gaube, S., Suresh, H., Raue, M., Merritt, A., Berkowitz, S. J., Lermer, E., Coughlin, J. F., Guttag, J. V., Colak, E., & Ghassemi, M. (2021) · npj Digital Medicine · doi:10.1038/s41746-021-00385-9 · Cited by 543 · cited 1×
“Recog - nizing, evaluating, and addressing these cognitive dynamics is crucial when considering the use of IA-based CDSS. Gaube et al. highlight a critical concern where physicians might unconsciously adopt biases present in AI systems [36 ].”
BackgroundShifting machine learning for healthcare from development to deployment and from models to data
Zhang, A., Xing, L., Zou, J., & Wu, J. C. (2022) · Nature Biomedical Engineering · doi:10.1038/s41551-022-00898-y · Cited by 420
Ethical limitations of algorithmic fairness solutions in health care machine learning
McCradden, M. D., Joshi, S., Mazwi, M., & Anderson, J. A. (2020) · The Lancet Digital Health · doi:10.1016/s2589-7500(20)30065-0 · Cited by 291
van de Sande, D., van Genderen, M. E., Huiskens, J., Gommers, D., & van Bommel, J. (2021) · Intensive Care Medicine · doi:10.1007/s00134-021-06446-7 · Cited by 279 · cited 1×
“However, in the field of critical care, despite the increasing number of developed AI-based algorithms, the vast majority remains within the testing and prototyping phase, lacking external validation and certification [ 4].”
BackgroundOn Human Dignity as a Foundation for the Right to Privacy
Floridi, L. (2016) · Philosophy & Technology · doi:10.1007/s13347-016-0220-8 · Cited by 190 · cited 1×
“The dilemma is highlighted by Floridi, who has questioned the priority to data protection over patient protection [29].”
BackgroundAddressing algorithmic bias and the perpetuation of health inequities: An AI bias aware framework
Agarwal, R., Bjarnadottir, M., Rhue, L., Dugas, M., Crowley, K., Clark, J., & Gao, G. (2023) · Health Policy and Technology · doi:10.1016/j.hlpt.2022.100702 · Cited by 170 · cited 2×
“…other sources of bias have been reported in the phases of algorithm development such as the managing of missing data or outliers, features selection, or modeling of underrepresented groups [23, 28], data sharing restrictions and data privacy represent a fundamental issue that should be considered.”
BackgroundHumans inherit artificial intelligence biases
Vicente, L., & Matute, H. (2023) · Scientific Reports · doi:10.1038/s41598-023-42384-8 · Cited by 158
Human Factors Challenges for the Safe Use of Artificial Intelligence in Patient Care
Sujan, M., Furniss, D., Grundy, K., Grundy, H., Nelson, D., Elliott, M., White, S., Habli, I., & Reynolds, N. (2019) · BMJ Health & Care Informatics · doi:10.1136/bmjhci-2019-100081 · Cited by 144 · cited 1×
“It necessitates not only technical precision but also an awareness of human cognitive biases and the development of a digital culture among all medical stakeholders to critically engage with AI-based CDSS [38].”
BackgroundPooch, E. H. P., Ballester, P., & Barros, R. C. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-62469-9_7 · Cited by 134 · cited 1×
“…algorithms, it poses the problem of the “domain shift” referring to the phenomenon wherein the statistical properties or distribution of the data that an algorithm is applied to during real-world usage differ from the properties or distribution of the data that the algorithm was trained on [30].”
BackgroundThe case for algorithmic stewardship for artificial intelligence and machine learning technologies
Eaneff, S., Obermeyer, Z., & Butte, A. J. (2020) · JAMA · doi:10.1001/jama.2020.9371 · Cited by 101
Bias in Artificial Intelligence
Nelson, G. S. (2019) · North Carolina Medical Journal · doi:10.18043/ncm.80.4.220 · Cited by 100 · cited 1×
“In recent years, there have been also advancements in machine learning that specifically address fairness such as adversarial de-biasing, where a model is trained to make predictions that are statistically independent of certain sensitive attributes, such as gender or ethnicity [33].”
Background
Research works printed but not linked: 32
- Moazemi S, Vahdati S, Li J, Kalkhoff S, Castano LJV, Dewitz B, et al. Artificial intelligence for clinical decision support for monitoring patients in cardiovascular ICUs: a systematic review. Front Med. 2023;10:1109411. · DOI · Paper page
- Muehlematter UJ, Bluethgen C, Vokinger KN. FDA-cleared artificial intelligence and machine learning-based medical devices and their 510(k) predicate networks. Lancet Digit Health. 2023;5:e618–26. · DOI · Paper page
- van de Sande D, van Bommel J, Fung Fen Chung E, Gommers D, van Genderen ME. Algorithmic fairness audits in intensive care medicine: artificial intelligence for all? Crit Care. 2022;26:315. · DOI
- Grote T, Keeling G. Enabling Fairness in Healthcare through Machine Learning. Ethics Inf Technol. 2022;24:39. · DOI
- Mosqueira-Rey E, Hernández-Pereira E, Alonso-Ríos D, Bobes-Bascarán J, Fernández-Leal Á. Human-in-the-loop machine learning: a state of the art. Artif Intell Rev. 2023;56:3005–54. · DOI · Paper page
- Friedman TL. Thank you for being late: an Optimist’s guide to thriving in the age of accelerations. London, UK: Allen Lane; 2016.
- Nakanishi R, Okubo R, Sobue Y, Kaneko U, Sato H, Fujimoto S, Nozaki Y, Kajiya T, Miyoshi T, Ichikawa K, Abe M, Kitagawa T, Ikenaga H, Osawa K, Saji M, Iguchi N, Nakazawa G, Takahashi K, Ijich T, Mikamo H, Kurata A, Moroi M, Iijima R, Malkasian S, Crabtree T, Chamie D, Alexandra LJ, Min JK, Earls JP, Matsuo H. Rationale and design of the INVICTUS Registry: (Multicenter Registry of Invasive and Non-Invasive imaging modalities to compare Coronary Computed Tomography Angiography, Intravascular Ultrasound and Optical Coherence Tomography for the determination of Severity, Volume and Type of coronary atherosclerosiS). J Cardiovasc Comput Tomogr. 2023 Sep 5:S1934-5925(23)00427-6. · DOI
- Gerard SE, Herrmann J, Xin Y, Martin KT, Rezoagli E, Ippolito D, Bellani G, Cereda M, Guo J, Hoffman EA, Kaczka DW, Reinhardt JM. CT image segmentation for inflamed and fibrotic lungs using a multi-resolution convolutional neural network. Sci Rep. 2021;11(1):1455. · DOI
- Connell M, Xin Y, Gerard SE, Herrmann J, Shah PK, Martin KT, Rezoagli E, Ippolito D, Rajaei J, Baron R, Delvecchio P, Humayun S, Rizi RR, Bellani G, Cereda M. Unsupervised segmentation and quantification of COVID-19 lesions on computed tomography scans using CycleGAN. Methods. 2022;205:200–9. · DOI
- Maddali MV, Churpek M, Pham T, Rezoagli E, Zhuo H, Zhao W, He J, Delucchi KL, Wang C, Wickersham N, McNeil JB, Jauregui A, Ke S, Vessel K, Gomez A, Hendrickson CM, Kangelaris KN, Sarma A, Leligdowicz A, Liu KD, Matthay MA, Ware LB, Laffey JG, Bellani G, Calfee CS, Sinha P. LUNG SAFE investigators and the ESICM Trials Group. Validation and utility of ARDS subphenotypes identified by machine-learning models using clinical data: an observational, multicohort, retrospective analysis. Lancet Respir Med. 2022;10(4):367–77. · DOI · Paper page
- Stephens AF, Šeman M, Diehl A, Pilcher D, Barbaro RP, Brodie D, Pellegrino V, Kaye DM, Gregory SD, Hodgson C. Extracorporeal Life Support Organization Member centres. ECMO PAL: using deep neural networks for survival prediction in venoarterial extracorporeal membrane oxygenation. Intensive Care Med. 2023;49(9):1090–9. · DOI
- Chen YY, Liu CF, Shen YT, Kuo YT, Ko CC, Chen TY, Wu TC, Shih YJ. Development of real-time individualized risk prediction models for contrast associated acute kidney injury and 30-day dialysis after contrast enhanced computed tomography. Eur J Radiol. 2023;167:111034. · DOI
- Avidan A, Sprung CL, Schefold JC, Ricou B, Hartog CS, Nates JL, et al. Variations in end-of-life practices in intensive care units worldwide (Ethicus-2): a prospective observational study. Lancet Respiratory Med. 2021;9:1101–10. · DOI · Paper page
- Denney MJ, Long DM, Armistead MG, Anderson JL, Conway BN. Validating the extract, transform, load process used to populate a large clinical research database. Int J Med Inf. 2016;94:271–4. · DOI
- Quiroz JC, Chard T, Sa Z, Ritchie A, Jorm L, Gallego B. Extract, transform, load framework for the conversion of health databases to OMOP. Deserno TM, editor. PLoS ONE. 2022;17:e0266911. · DOI
- Henke E, Peng Y, Reinecke I, Zoch M, Sedlmayr M, Bathelt F. An extract-transform-load process design for the Incremental Loading of German Real-World Data based on FHIR and OMOP CDM: Algorithm Development and Validation. JMIR Med Inf. 2023;11:e47310.
- Unberath P, Prokosch HU, Gründner J, Erpenbeck M, Maier C, Christoph J. EHR-Independent Predictive decision Support Architecture based on OMOP. Appl Clin Inf. 2020;11:399–404. · DOI
- Fleuren LM, Dam TA, Tonutti M, De Bruin DP, Lalisang RCA, Gommers D, et al. The Dutch Data Warehouse, a multicenter and full-admission electronic health records database for critically ill COVID-19 patients. Crit Care. 2021;25:304. · DOI
- McNicholas BA, Madotto F, Pham T, Rezoagli E, Masterson CH, Horie S, Bellani G, Brochard L, Laffey JG, LUNG SAFE Investigators and the ESICM Trials Group. Demographics, management and outcome of females and males with acute respiratory distress syndrome in the LUNG SAFE prospective cohort study. Eur Respir J. 2019;54(4):1900609. · DOI
- Rezoagli E, McNicholas BA, Madotto F, Pham T, Bellani G, Laffey JG, LUNG SAFE Investigators, the ESICM Trials Group. Presence of comorbidities alters management and worsens outcome of patients with acute respiratory distress syndrome: insights from the LUNG SAFE study. Ann Intensive Care. 2022;12(1):42. · DOI
- Majid Z, Welch C, Davies J, Jackson T. Global frailty: the role of ethnicity, migration and socioeconomic factors. Maturitas. 2020;139:33–41. · DOI
- Bellini V, Montomoli J, Bignami E. Poor quality data, privacy, lack of certifications: the lethal triad of new technologies in intensive care. Intensive Care Med 202147:1052–3. · DOI
- Farahani A, Voghoei S, Rasheed K, Arabnia HR. A Brief Review of Domain Adaptation. In: Stahlbock R, Weiss GM, Abou-Nasr M, Yang C-Y, Arabnia HR, Deligiannidis L, editors. Advances in Data Science and Information Engineering [Internet]. Cham: Springer International Publishing; 2021 [cited 2023 Jun 16]. pp. 877–94. https://link.springer.com/https://doi.org/10.1007/978-3-030-71704-9_65. · DOI · Paper page
- Kahneman D. Thinking, fast and slow. London: Penguin Books; 2012.
- Adler-Milstein J, Chen JH, Dhaliwal G. Next-generation Artificial Intelligence for diagnosis: from Predicting Diagnostic labels to Wayfinding. JAMA. 2021;326:2467. · DOI
- Montomoli J, Rezoagli E, Bellini V, Finazzi S, Bignami EG. A generalized wayfinding paradigm for improving AKI understanding and classification: insights from the Dutch registries. Minerva Anestesiol. 2023. · DOI
- Celi LA, Cellini J, Charpignon M-L, Dee EC, Dernoncourt F, Eber R et al. Sources of bias in artificial intelligence that perpetuate healthcare disparities—A global review. Fraser HS, editor. PLOS Digit Health. 2022;1:e0000022. · DOI · Paper page
- Montomoli J, Hilty MP, Ince C. Artificial intelligence in intensive care: moving towards clinical decision support systems. Minerva Anestesiol. 2022. · DOI
- Blanchard MD, Kleitman S, Aidman E. Are two naïve and distributed heads better than one? Factors influencing the performance of teams in a challenging real-time task. Front Psychol. 2023;14:1042710. · DOI
- Koriat A. When two heads are better than one and when they can be worse: the amplification hypothesis. J Exp Psychol Gen. 2015;144:934–50. · DOI
- Aporta C, Higgs E. Satellite Culture: Global Positioning Systems, Inuit Wayfinding, and the need for a New Account of Technology. Curr Anthropol. 2005;46:729–53. · DOI · Paper page
- Cosgriff CV, Stone DJ, Weissman G, Pirracchio R, Celi LA. The clinical artificial intelligence department: a prerequisite for success. BMJ Health Care Inf. 2020;27:e100183. · DOI · Paper page
Cited by
Artificial intelligence in extended perioperative medicine
Bignami, E., Darhour, L. J., & Bellini, V. (2024) · Trends in Anaesthesia and Critical Care · doi:10.1016/j.tacc.2024.101376 · Cited by 4
Self-citationPranata, Y., Mirwanti, R., Bahtiar, Y. A., & Purwanti, H. (2026) · Journal of Multidisciplinary Healthcare · doi:10.2147/jmdh.s621469 · Cited by 1
“56,60 This pattern is clearly visible within the present corpus: 22 of the 29 included studies (75.9”
InfluentialArtificial Intelligence to Facilitate SEP-1 Measure Compliance and Fluid Management in Sepsis
Nguyen, H. B., Krishtopaytis, E., Lopez, E., Farnoudi, N., Van, T., Kharalampova, V., & Coz Yataco, A. (2026) · Journal of Clinical Medicine · doi:10.3390/jcm15093477
Metadata
- Authors
- Montomoli, J., Bitondo, M. M., Cascella, M., Rezoagli, E., Romeo, L., Bellini, V., Semeraro, F., Gamberini, E., Frontoni, E., Agnoletti, V., Altini, M., Benanti, P., & Bignami, E. G.
- Venue
- Journal of Clinical Monitoring and Computing
- Published
- 2024-04-04
- Volume
- 38
- Issue
- 4
- Pages
- 931-939
- ISSN
- 1387-1307
- 1573-2614
- Cited by
- 36
- References
- 14
- Language
- en
- Licence
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.