Diagnostic performance of machine learning models using cell population data for the detection of sepsis: a comparative study
The DOI matches its Crossref record; 49 of the 50 references it cites were found in a registry or an index.
Abstract
Objectives: To compare the artificial intelligence algorithms as powerful machine learning methods for evaluating patients with suspected sepsis using data from routinely available blood tests performed on arrival at the hospital. Results were compared with those obtained from the classical logistic regression method. Methods: The study group consisted of consecutive patients with fever and suspected infection admitted to the Emergency Department. The complete blood counts (CBC) were acquired using the Mindray BC-6800 Plus analyser (Mindray Diagnostics, Shenzhen, China). Cell Population Data (CPD) were also recorded. The ML and artificial intelligence (AI) models were developed; their performance was evaluated using several indicators, such as the area under the receiver operating curve (AUC), calibration plots and decision curve analysis (DCA). Results: Overall, all the tested approaches obtained an AUC>0.90. The logistic regression (LR) performed well compared to the ML/AI models. The naïve Bayes and the K-nearest neighbour (KNN) methods did not show good calibration properties. The multi-layer perceptron (MLP) model was the best in terms of discrimination, calibration and clinical usefulness. Conclusions: The best performance in the early detection of sepsis was achieved using the ML and AI models. However, external validation studies are needed to strengthen model derivation and procedure updating.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The reviewed studies demonstrated the efficacy of ML algorithms in several diagnostic applications.”
From Machine learning-based clinical decision support using laboratory data (2024) · cites this paper as [41]
Citations
Cites
1 retracted
Singh, Y. V., Singh, P. K., Khan, S., & Singh, R. S. (2022) · Journal of Healthcare Engineering · doi:10.1155/2022/9263391 · Cited by 51
RetractedScikit-learn: Machine Learning in Python
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Müller, A., Nothman, J., Louppe, G., et al. (2012) · arXiv · doi:10.48550/arxiv.1201.0490 · Cited by 71,016
XGBoost: A Scalable Tree Boosting System
Chen, T., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939785 · Cited by 54,569
The third international consensus definitions for sepsis and septic shock (Sepsis-3).
Singer, M., Deutschman, C. S., Seymour, C. W., Shankar-Hari, M., Annane, D., Bauer, M., Bellomo, R., Bernard, G. R., Chiche, J.-D., Coopersmith, C. M., et al. (2016) · JAMA · doi:10.1001/jama.2016.0287 · Cited by 28,250
The Elements of Statistical Learning
Hastie, T., Tibshirani, R., & Friedman, J. (2009) · Springer Series in Statistics · doi:10.1007/978-0-387-84858-7 · Cited by 25,970
From local explanations to global understanding with explainable AI for trees
Lundberg, S. M., Erion, G., Chen, H., DeGrave, A., Prutkin, J. M., Nair, B., Katz, R., Himmelfarb, J., Bansal, N., & Lee, S.-I. (2020) · Nature Machine Intelligence · doi:10.1038/s42256-019-0138-9 · Cited by 10,551
Kuhn, M., & Johnson, K. (2013) · doi:10.1007/978-1-4614-6849-3 · Cited by 6,788
Severe Sepsis and Septic Shock
Angus, D. C., & van der Poll, T. (2013) · New England Journal of Medicine · doi:10.1056/nejmra1208623 · Cited by 3,961
Recognizing Sepsis as a Global Health Priority — A WHO Resolution
Reinhart, K., Daniels, R., Kissoon, N., Machado, F. R., Schachter, R. D., & Finfer, S. (2017) · New England Journal of Medicine · doi:10.1056/nejmp1707170 · Cited by 1,356
Ruopp, M. D., Perkins, N. J., Whitcomb, B. W., & Schisterman, E. F. (2008) · Biometrical Journal · doi:10.1002/bimj.200710415 · Cited by 1,320
Unintended consequences of machine learning in medicine
Cabitza, F., Rasoini, R., & Gensini, G. F. (2017) · JAMA · doi:10.1001/jama.2017.7797 · Cited by 1,066
Faix, J. D. (2013) · Critical Reviews in Clinical Laboratory Sciences · doi:10.3109/10408363.2013.764490 · Cited by 663
Rajula, H. S. R., Verlato, G., Manchia, M., Antonucci, N., & Fanos, V. (2020) · Medicina · doi:10.3390/medicina56090455 · Cited by 606
The epidemiology of the systemic inflammatory response
Brun-Buisson, C. (2000) · Intensive Care Medicine · doi:10.1007/s001340051121 · Cited by 534
Sepsis: The evolution in definition, pathophysiology, and management
Gyawali, B., Ramakrishna, K., & Dhamoon, A. S. (2019) · SAGE Open Medicine · doi:10.1177/2050312119835043 · Cited by 512
A tutorial on calibration measurements and calibration models for clinical prediction models
Huang, Y., Li, W., Macheret, F., Gabriel, R. A., & Ohno-Machado, L. (2020) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocz228 · Cited by 486
Use of Brier score to assess binary predictions
Rufibach, K. (2010) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2009.11.009 · Cited by 454
On the Relationship between Classical Grid Search and Probabilistic Roadmaps
LaValle, S. M., Branicky, M. S., & Lindemann, S. R. (2004) · The International Journal of Robotics Research · doi:10.1177/0278364904045481 · Cited by 436
The Discrete Brier and Ranked Probability Skill Scores
Weigel, A. P., Liniger, M. A., & Appenzeller, C. (2007) · Monthly Weather Review · doi:10.1175/mwr3280.1 · Cited by 254
Webb, G. I. (2011) · Encyclopedia of Machine Learning · doi:10.1007/978-0-387-30164-8_576 · Cited by 217
Hu, C., Li, L., Huang, W., Wu, T., Xu, Q., Liu, J., & Hu, B. (2022) · Infectious Diseases and Therapy · doi:10.1007/s40121-022-00628-6 · Cited by 160
A Machine Learning Model for Accurate Prediction of Sepsis in ICU Patients
Wang, D., Li, J., Sun, Y., Ding, X., Zhang, X., Liu, S., Han, B., Wang, H., Duan, X., & Sun, T. (2021) · Frontiers in Public Health · doi:10.3389/fpubh.2021.754348 · Cited by 145
Martins, E. C., Silveira, L. D. F., Viégas, K., Beck, A. D., Fioravanti, G. A., Cremonese, R. V., & Lora, P. S. (2019) · Revista Brasileira de Terapia Intensiva · doi:10.5935/0103-507x.20190010 · Cited by 138
Neutrophil to Lymphocyte Ratio (NLR)—A Useful Tool for the Prognosis of Sepsis in the ICU
Drăgoescu, A. N., Pădureanu, V., Stănculescu, A. D., Chiuțu, L. C., Tomescu, P. I., Geormăneanu, C., Pădureanu, R., Iovănescu, V. F., Ungureanu, B. S., Pănuş, A., et al. (2021) · Biomedicines · doi:10.3390/biomedicines10010075 · Cited by 138
An intelligent warning model for early prediction of cardiac arrest in sepsis patients
Layeghian Javan, S., Sepehri, M. M., Layeghian Javan, M., & Khatibi, T. (2019) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2019.06.010 · Cited by 86
Sepsis biomarkers: past, present and future
Lippi, G. (2019) · Clinical Chemistry and Laboratory Medicine (CCLM) · doi:10.1515/cclm-2018-1347 · Cited by 76
Role of leucocytes cell population data in the early detection of sepsis
Urrechaga, E., Bóveda, O., & Aguirre, U. (2017) · Journal of Clinical Pathology · doi:10.1136/jclinpath-2017-204524 · Cited by 73
Self-citationCalibrating random forests for probability estimation
Dankowski, T., & Ziegler, A. (2016) · Statistics in Medicine · doi:10.1002/sim.6959 · Cited by 65
Kijpaisalratana, N., Sanglertsinlapachai, D., Techaratsami, S., Musikatavorn, K., & Saoraya, J. (2022) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2022.104689 · Cited by 65
Zhang, Z., & Hong, Y. (2017) · Oncotarget · doi:10.18632/oncotarget.17870 · Cited by 57
Improvement in detecting sepsis using leukocyte cell population data (CPD)
Urrechaga, E., Bóveda, O., & Aguirre, U. (2018) · Clinical Chemistry and Laboratory Medicine (CCLM) · doi:10.1515/cclm-2018-0979 · Cited by 55
Self-citationMartens, R. J. H., van Adrichem, A. J., Mattheij, N. J. A., Brouwer, C. G., van Twist, D. J. L., Broerse, J. J. C. R., Magro‐Checa, C., van Dongen, C. M. P., Mostard, R. L. M., Ramiro, S., et al. (2020) · Clinical Chemistry and Laboratory Medicine (CCLM) · doi:10.1515/cclm-2020-1529 · Cited by 54
Rawson, T. M., Hernandez, B., Moore, L. S. P., Blandy, O., Herrero, P., Gilchrist, M., Gordon, A., Toumazou, C., Sriskandan, S., Georgiou, P., et al. (2019) · Journal of Antimicrobial Chemotherapy · doi:10.1093/jac/dky514 · Cited by 52
Zeng, Z., Yao, S., Zheng, J., & Gong, X. (2021) · BioData Mining · doi:10.1186/s13040-021-00276-5 · Cited by 46
Wang, X., Zhu, T., Xia, M., Liu, Y., Wang, Y., Wang, X., Zhuang, L., Zhong, D., Zhu, J. Y., He, H., et al. (2022) · Frontiers in Cardiovascular Medicine · doi:10.3389/fcvm.2022.764629 · Cited by 40
Zhang, W., Zhang, Z., Pan, S., Li, J., Yang, Y., Qi, H., Xie, J., & Qu, J. (2021) · Annals of Translational Medicine · doi:10.21037/atm-21-5639 · Cited by 39
Buoro, S., Carobene, A., Seghezzi, M., Manenti, B., Pacioni, A., Ceriotti, F., Ottomano, C., & Lippi, G. (2017) · Clinica Chimica Acta · doi:10.1016/j.cca.2017.07.009 · Cited by 38
Tang, H., Jing, J., Bo, D., & Xu, D. (2012) · Archives of Pathology & Laboratory Medicine · doi:10.5858/arpa.2011-0679-oa · Cited by 25
Buoro, S., Manenti, B., Seghezzi, M., Dominoni, P., Barbui, T., Ghirardi, A., Carobbio, A., Marchesi, G., Riva, I., Nasi, A., et al. (2017) · Journal of Clinical Pathology · doi:10.1136/jclinpath-2017-204643 · Cited by 25
Seghezzi, M., Buoro, S., Previtali, G., Moioli, V., Manenti, B., Simon‐Lopez, R., Ottomano, C., & Lippi, G. (2018) · Journal of Medical Biochemistry · doi:10.2478/jomb-2018-0005 · Cited by 22
Lu, J., Bu, P., Xia, X., Lu, N., Yao, L., & Jiang, H. (2021) · Environmental Science and Pollution Research · doi:10.1007/s11356-021-12658-7 · Cited by 19
Urrechaga, E., Aguirre, U., España, P. P., & de Guadiana, L. G. (2020) · Clinical Chemistry and Laboratory Medicine (CCLM) · doi:10.1515/cclm-2020-1416 · Cited by 16
Self-citationHarte, J. V., & Mykytiv, V. (2021) · Clinical Chemistry and Laboratory Medicine (CCLM) · doi:10.1515/cclm-2021-0096 · Cited by 15
Sepsis as a model for improving diagnosis
Graber, M. L., Patel, M., & Claypool, S. (2018) · Diagnosis · doi:10.1515/dx-2017-0036 · Cited by 14
Can biomarkers help us to better diagnose and manage sepsis?
Schüetz, P., & Plebani, M. (2015) · Diagnosis · doi:10.1515/dx-2014-0073 · Cited by 12
Using Machine Learning Algorithms to predict sepsis and its stages in ICU patientsPreprint
Ghias, N., Haq, S. U., Arshad, H., Sultan, H., Bashir, F., Ghaznavi, S. A., Shabbir, M., Badshah, Y., & Rafiq, M. (2022) · medRxiv · doi:10.1101/2022.03.15.22271655 · Cited by 9
Urrechaga, E. (2021) · Book Publisher International (a part of SCIENCEDOMAIN International) · doi:10.9734/bpi/hmmr/v9/8560d · Cited by 1
Self-citationAnalysis of leukocyte cell population data (CPD) as biomarkers in the diagnosis of sepsis
Rigo, I. M. P., Alarcón Rodríguez, R., Bellido, B., & Avivar Oyonarte, C. (2022) · Hematology & Transfusion International Journal · doi:10.15406/htij.2022.10.00278 · Cited by 1
Research works printed but not linked: 2
- Kohavi, R. A study of cross-validation and bootstrap for accuracy estimation and model selection. In: International joint conference on artificial intelligence IJCA; 1995, vol. 2:9 p.
- Hoffmann, J. New hematology analyzer parameters and their clinical relevance. EFLM Newslett 2018;1:3.
Cited by
Machine learning-based clinical decision support using laboratory data
Çubukçu, H. C., Topcu, D. İ., & Yenice, S. (2024) · Clinical Chemistry and Laboratory Medicine (CCLM) · doi:10.1515/cclm-2023-1037 · Cited by 66
“However, similar to many healthcare ML applications, the adoption of ML in routine clinical laboratory practice is often hindered by substantial concerns regarding its inherent behavior, often referred to as the “ black-box ” problem [41, 61 – 63 , 111].”
BackgroundMethodArtificial Intelligence in Sepsis Management: An Overview for Clinicians
Bignami, E. G., Berdini, M., Panizzi, M., Domenichetti, T., Bezzi, F., Allai, S., Damiano, T., & Bellini, V. (2025) · Journal of Clinical Medicine · doi:10.3390/jcm14010286 · Cited by 53
“In this scenario, another study from Arrigue and Urrechaga [19] tested different MLMs to diagnose sepsis in patients admitted to the ED with suspected infection, using data from blood tests.”
Cheah, B. C. J., Vicente, C. R., & Chan, K. R. (2025) · Viruses · doi:10.3390/v17070882 · Cited by 64
Machine learning algorithms in sepsis
Agnello, L., Vidali, M., Padoan, A., Lucis, R., Mancini, A., Guerranti, R., Plebani, M., Ciaccio, M., & Carobene, A. (2023) · Clinica Chimica Acta · doi:10.1016/j.cca.2023.117738 · Cited by 40
Lin, T.-H., Chung, H.-Y., Jian, M.-J., Chang, C.-K., Lin, H.-H., Yen, C.-T., Tang, S.-H., Pan, P.-C., Perng, C.-L., Chang, F.-Y., et al. (2025) · Journal of Medical Internet Research · doi:10.2196/56155 · Cited by 29
“Enhanced diagnostic accuracy can be achieved using new parameters that provide information on the morphological and functional characteristics of leukocytes, reflecting their activation and functional activity in response to infections [17].”
Metadata
- Authors
- Aguirre, U., & Urrechaga, E.
- Affiliations
- Osasun Sistemen Ikerketa Institutua (ES)
- BioCruces Health research Institute (ES)
- Corresponding author
- Aguirre, Urko
- Venue
- Clinical Chemistry and Laboratory Medicine (CCLM)
- Publisher
- Walter de Gruyter GmbH
- Published
- 2022-11-11
- Volume
- 61
- Issue
- 2
- Pages
- 356-365
- ISSN
- 1434-6621 (ISSN-L)
- 1437-4331
- OpenAlex
- W4308632660
- Cited by
- 17
- References
- 50
- Citations by year
- 2026: 1 · 2025: 7 · 2024: 5 · 2023: 4
- FWCI
- 2.89
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Clinical Chemistry and Laboratory Medicine (CCLM) (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Health Sciences › Medicine › Critical Care and Intensive Care Medicine › Sepsis Diagnosis and Treatment
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Keywords
- sepsis detection, cell population data, complete blood count, machine learning, artificial intelligence, multilayer perceptron, logistic regression, naive Bayes, k-nearest neighbors, emergency department, early diagnosis, model calibration, decision curve analysis
- MeSH terms
- Machine Learning, Artificial Intelligence, Bayes Theorem, Humans, Logistic Models, Sepsis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.