The MIMIC Code Repository: enabling reproducibility in critical care research
Open accessThe DOI matches its Crossref record; 39 of the 39 references it cites were found in a registry or an index.
Abstract
Objective: Lack of reproducibility in medical studies is a barrier to the generation of a robust knowledge base to support clinical decision-making. In this paper we outline the Medical Information Mart for Intensive Care (MIMIC) Code Repository, a centralized code base for generating reproducible studies on an openly available critical care dataset. Materials and Methods: Code is provided to load the data into a relational structure, create extractions of the data, and reproduce entire analysis plans including research studies. Results: Concepts extracted include severity of illness scores, comorbid status, administrative definitions of sepsis, physiologic criteria for sepsis, organ failure scores, treatment administration, and more. Executable documents are used for tutorials and reproduce published studies end-to-end, providing a template for future researchers to replicate. The repository’s issue tracker enables community discussion about the data and concepts, allowing users to collaboratively improve the resource. Discussion: The centralized repository provides a platform for users of the data to interact directly with the data generators, facilitating greater understanding of the data. It also provides a location for the community to collaborate on necessary concepts for research progress and share them with a larger audience. Consistent application of the same code for underlying concepts is a key step in ensuring that research studies on the MIMIC database are comparable and reproducible. Conclusion: By providing open source code alongside the freely accessible MIMIC-III database, we enable end-to-end reproducible analysis of electronic health records.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures






Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“The MIMIC Code Repository was originally created for MIMIC-III to enable researchers to share and reuse code24.”
From MIMIC-IV, a freely accessible electronic health record dataset (2023) · cites this paper as [24]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The third international consensus definitions for sepsis and septic shock (Sepsis-3).
Singer, M., Deutschman, C. S., Seymour, C. W., Shankar-Hari, M., Annane, D., Bauer, M., Bellomo, R., Bernard, G. R., Chiche, J.-D., Coopersmith, C. M., et al. (2016) · JAMA · doi:10.1001/jama.2016.0287 · Cited by 28,250 · cited 1×
“21 The precise onset of sepsis is not typically documented in the EHR, and is, in fact, a difficult item to capture clinically.”
BackgroundVincent, J. L., Moreno, R., Takala, J., Willatts, S., De Mendonça, A., Bruining, H., Reinhart, C. K., Suter, P. M., & Thijs, L. G. (1996) · Intensive Care Medicine · doi:10.1007/bf01709751 · Cited by 11,674 · cited 1×
“Some scores assess multiple organ systems: the SOFA score 12 and LODS 13 both assess 6 organ systems for failure.”
BackgroundCoding algorithms for defining comorbidities in ICD-9-CM and ICD-10 administrative data
Quan, H., Sundararajan, V., Halfon, P., Fong, A., Burnand, B., Luthi, J.-C., Saunders, L. D., Beck, C. A., Feasby, T. E., & Ghali, W. A. (2005) · Medical Care · doi:10.1097/01.mlr.0000182534.19832.83 · Cited by 10,669 · cited 1×
“27 Finally, Quan et al. 28 proposed an enhanced ICD-9 coding methodology based on examining inconsistencies among previous definitions.”
MethodComorbidity measures for use with administrative data.
Elixhauser, A., Steiner, C., Harris, D. R., & Coffey, R. M. (1998) · Medical Care · doi:10.1097/00005650-199801000-00004 · Cited by 9,921 · cited 4×
“Van Walraven et al. 29 later aggregated comorbidities codified by Elixhauser et al. 26 into a single point score for in-hospital mortality prediction, which is also available in the repository.”
BackgroundMethodInfluentialAngus, D. C., Linde-Zwirble, W. T., Lidicker, J., Clermont, G., Carcillo, J., & Pinsky, M. R. (2001) · Critical Care Medicine · doi:10.1097/00003246-200107000-00002 · Cited by 8,503 · cited 3×
“The criteria as proposed by Angus et al. 23 were validated in a later study by Iwashyna et al. 25 These 3 criteria – explicit coding, those proposed by Angus et al., 23 and those proposed by Martin et al. 24 – are available in the repository.”
MethodMIMIC-III, a freely accessible critical care database
Johnson, A. E. W., Pollard, T. J., Shen, L., Lehman, L.-W. H., Feng, M., Ghassemi, M., Moody, B., Szolovits, P., Anthony Celi, L., & Mark, R. G. (2016) · Scientific Data · doi:10.1038/sdata.2016.35 · Cited by 8,141 · cited 1×
“3,4 The database comprises detailed clinical information regarding > 60000 stays in ICUs at the Beth Israel Deaconess Medical Center in Boston, Massachusetts, collected as part of routine clinical care.”
BackgroundSelf-citationLe Gall, J.-R. (1993) · JAMA: The Journal of the American Medical Association · doi:10.1001/jama.270.24.2957 · Cited by 7,773
Acute Kidney Injury Network: report of an initiative to improve outcomes in acute kidney injury
Mehta, R. L., Kellum, J. A., Shah, S. V., Molitoris, B. A., Ronco, C., Warnock, D. G., Levin, A., & the Acute Kidney Injury Network (2007) · Critical Care · doi:10.1186/cc5713 · Cited by 7,048 · cited 1×
“Examples include the Model for End-stage Liver Disease, 14 the Risk/Injury/Failure/Loss/End-stage renal disease criteria, 15 the Acute Kidney Injury Network classification, 16 and the Kidney Disease: Improving Global Outcomes criteria.”
BackgroundThe epidemiology of sepsis in the United States from 1979 through 2000
Martin, G. S., Mannino, D. M., Eaton, S., & Moss, M. (2003) · New England Journal of Medicine · doi:10.1056/nejmoa022139 · Cited by 5,841 · cited 4×
“The criteria as proposed by Angus et al. 23 were validated in a later study by Iwashyna et al. 25 These 3 criteria – explicit coding, those proposed by Angus et al., 23 and those proposed by Martin et al. 24 – are available in the repository.”
BackgroundInfluential1,500 scientists lift the lid on reproducibility
Baker, M. (2016) · Nature · doi:10.1038/533452a · Cited by 4,209 · cited 1×
“Concerns about the reproducibility of results in science are becoming increasingly prominent in both scientific and mainstream literature.(1) Some commentators have gone so far as to call the current state a crisis, citing causes such as pressure to publish positive results, the cost of replicating studies such as dou…”
Background
Research works printed but not linked: 2
- Torio CM, Andrews RM National inpatient hospital costs: The most expensive conditions by payer, 2011. hCUP statistical brief #160. Agency for Healthcare Research and Quality, Rockville, MD [Internet]. 2013 http://www.hcup-us.ahrq.gov/reports/statbriefs/sb160.pdf. Accessed May 25, 2017. 24199255
- Torio CM, Moore BJ National inpatient hospital costs: The most expensive conditions by payer, 2013. hCUP statistical brief #204. Agency for Healthcare Research and Quality, Rockville, MD [Internet]. 2016 http://www.hcup-us.ahrq.gov/reports/statbriefs/sb204-Most-Expensive-Hospital-Conditions.pdf. Accessed May 25, 2017. 27359025
Cited by
Items 101–200 of 442
Lin, S., Ge, S., He, W., & Zeng, M. (2020) · QJM · doi:10.1093/qjmed/hcaa192 · Cited by 24
Yang, W., Zou, H., Wang, M., Zhang, Q., Li, S., & Liang, H. (2023) · Heliyon · doi:10.1016/j.heliyon.2023.e13200 · Cited by 25
“Relevant measures, including fluid balance and severity assessment, were also constructed based on official MIMIC database documentation (https://github.com/MIT-LCP/mimic-code/tree/main/mimic-iii) [18].”
MethodLessons Learned: Surveying the Practicality of Differential Privacy in the Industry
Garrido, G. M., Liu, X., Matthes, F., & Song, D. X. (2023) · Proceedings on Privacy Enhancing Technologies · doi:10.56553/popets-2023-0045 · Cited by 55
Adekkanattu, P. M., Jiang, G., Luo, Y., Kingsbury, P., Xu, Z., Rasmussen, L. V., Pacheco, J. A., Kiefer, R. C., Stone, D. J., Brandt, P., et al. (2019) · PubMed · doi:10.48550/arxiv.1905.01961 · Cited by 21
Self-citationVitalDB: fostering collaboration in anaesthesia research
Vistisen, S. T., Pollard, T., Enevoldsen, J., & Scheeren, T. (2021) · British Journal of Anaesthesia · doi:10.1016/j.bja.2021.03.011 · Cited by 26
Self-citationMao, Y., Zhu, S., & Geng, Y. (2022) · Hematology · doi:10.1080/16078454.2022.2095948 · Cited by 20
“Data regarding the length of hospital stay were extracted from the table named ‘admissions’ of the MIMIC-IV database [16].”
MethodShickel, B., Silva, B., Ozrazgat‐Baslanti, T., Ren, Y., Khezeli, K., Guan, Z., Tighe, P. J., Bihorac, A., & Rashidi, P. (2022) · Frontiers in Digital Health · doi:10.3389/fdgth.2022.1029191 · Cited by 33
Association between hyperglycemia and adverse clinical outcomes of sepsis patients with diabetes
Lin, S., Lai, D., & He, W. (2023) · Frontiers in Endocrinology · doi:10.3389/fendo.2022.1046736 · Cited by 22
“The code for assisting in the investigation of MIMIC-III is openly available on the website (16).”
BackgroundPanch, T., Pollard, T., Mattie, H., Lindemer, E., Keane, P. A., & Celi, L. A. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-0295-6 · Cited by 20
Self-citationZhang, M., Rao, Z.-G., Ma, T., Tang, M., Xu, T., He, X.-X., Li, Z.-P., Liu, Y., Xu, Q.-J., Yang, K., et al. (2022) · Frontiers in Medicine · doi:10.3389/fmed.2022.952611 · Cited by 19
Association Between Patient Race and Ethnicity and Use of Invasive Ventilation in the United States
Abdelmalek, F. M., Angriman, F., Moore, J. E., Liu, K., Burry, L. D., Seyyed-Kalantari, L., Mehta, S., Gichoya, J. W., Celi, L. A., TOMLINSON, G. A., et al. (2023) · Annals of the American Thoracic Society · doi:10.1513/annalsats.202305-485oc · Cited by 19
Self-citationDoctor2Vec: Dynamic Doctor Representation Learning for Clinical Trial Recruitment
Biswal, S., Xiao, C., Glass, L. M., Milkovits, E., & Sun, J. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i01.5394 · Cited by 25
Wu, Y., Huang, S., & Chang, X. (2021) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-021-01690-9 · Cited by 22
“The computation of these scores for each patient is accomplished with the help of severity scores SQL scripts from the mimic-code repository [51].”
MethodLin, S., Ge, S., He, W., & Zeng, M. (2021) · Annals of Translational Medicine · doi:10.21037/atm-20-4360 · Cited by 16
“The code that facilitates documentation of the website and MIMIC-III is accessible to the public, and input from users is encouraged (13).”
BackgroundGuo, Q., Xiao, Z., Lin, M.-H., Yuan, G., Qiu, Q., Yang, Y., Zhao, H., Zhang, Y., Zhou, S., & Wang, J. (2021) · Annals of Translational Medicine · doi:10.21037/atm-20-7897 · Cited by 16
“Structure Query Language was appl ied for data extraction (15).”
MethodNistal‐Nuño, B. (2021) · Journal of Clinical Monitoring and Computing · doi:10.1007/s10877-021-00703-2 · Cited by 19
“These established systems’ scores were generated using open source code accompanying the MIMIC-III database [25].”
MethodTowards early sepsis detection from measurements at the general ward through deep learning
Oei, S. P., van Sloun, R. J. G., van der Ven, M., Korsten, H. H. M., & Mischi, M. (2021) · Intelligence-Based Medicine · doi:10.1016/j.ibmed.2021.100042 · Cited by 18
Examining the weekend effect across ICU performance metrics
Faust, L. J., Feldman, K., & Chawla, N. V. (2019) · Critical Care · doi:10.1186/s13054-019-2479-5 · Cited by 16
AI enabled RPM for mental health facility
Shaik, T., Tao, X., Higgins, N., Xie, H., Gururajan, R., & Zhou, X. (2022) · doi:10.1145/3556551.3561191 · Cited by 15
“The proposed framework can be evaluatedwith publicly available benchmark datasets like MIMIC-III [15] and MHEALTH (Mobile HEALTH) dataset [2, 19].”
MethodHwang, S. Y., Kim, I., Jeong, D., Park, J.-E., Lee, G. T., Yoo, J., Choi, K., Shin, T. G., & Kim, K. (2023) · Journal of Clinical Medicine · doi:10.3390/jcm12196402 · Cited by 19
“The data codes for the study analyses were from the MIMIC Code Repository [29].”
MethodQian, X., Sheng, Y., Jiang, Y., & Xu, Y. (2024) · Frontiers in Neurology · doi:10.3389/fneur.2024.1371827 · Cited by 17
Lin, S., He, W., & Zeng, M. (2020) · Journal of Inflammation Research · doi:10.2147/jir.s287510 · Cited by 16
Medical Dead-ends and Learning to Identify High-risk States and TreatmentsPreprint
Fatemi, M., Killian, T. W., Subramanian, J., & Ghassemi, M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.04186 · Cited by 50
“[48] Alistair Ew Johnson, David J Stone, Leo A Celi, and Tom J Pollard.”
BackgroundMethodHuang, W., Xie, H.-J., Fan, H.-T., Yan, M., & Hong, Y. (2021) · Medicine · doi:10.1097/md.0000000000027380 · Cited by 16
“The data extraction processes were performed mainly using structured query language codes from the Medical Information Mart for Intensive Care Code Repository.([11])”
MethodA Multidatabase ExTRaction PipEline (METRE) for facile cross validation in critical care research
Liao, W., & Voldman, J. (2023) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2023.104356 · Cited by 24
“We also made use of existing derived tables from both repositories [22,23] to develop our SQL queries.”
MethodNguyen, T.-N., Yang, H.-J., Kho, B., Kang, S., & Kim, S.-H. (2024) · IEEE Access · doi:10.1109/access.2024.3447759 · Cited by 18
“We followed the tutorial in [85] to streamline data gathering and eliminate unnecessary data groups due to the enormous size of this dataset, which exceeded the research resource capabilities.”
MethodA novel model to label delirium in an intensive care unit from clinician actions
Coombes, C. E., Coombes, K. R., & Fareed, N. (2021) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-021-01461-6 · Cited by 17
Liu, X., DuMontier, C., Hu, P., Liu, C., Yeung, W., Mao, Z., Ho, V., Thoral, P. J., Kuo, P.-C., Hu, J., et al. (2023) · The Journals of Gerontology: Series A · doi:10.1093/gerona/glac107 · Cited by 14
Self-citationThe “Ecosystem as a Service (EaaS)” approach to advance clinical artificial intelligence (cAI)
Ishii-Rousseau, J. E., Seino, S., Ebner, D. K., Vareth, M., Po, M. J., & Celi, L. A. (2022) · PLOS Digital Health · doi:10.1371/journal.pdig.0000011 · Cited by 15
“The LCP launched the Medical Information Mart for Intensive Care (MIMIC) database in the early 2000s, an open-source and free dataset of more than 60,000 de-identified intensive care unit admission records, which has since become the golden standard for public clinical databases [18,19].”
BackgroundSelf-citationRao, A. R., Clarke, D. J. B., & Vargas, M. (2018) · Journal of Healthcare Informatics Research · doi:10.1007/s41666-018-0014-0 · Cited by 14
“Johnson [9] indicates that in addition to making data available, it is also important to run code consistently, thereby enabling end-to-end reproducibility of analysis.”
BackgroundMethodYao, S., Chen, X., Liu, J., Chen, X., & Zhou, Y. (2022) · Frontiers in Neurology · doi:10.3389/fneur.2022.1017849 · Cited by 14
Hu, W., Yuan, L., Wang, X., Zang, B., Zhang, Y., Yan, X., Zhao, W., & Chao, Y. (2022) · International Journal of Clinical Practice · doi:10.1155/2022/7644535 · Cited by 13
“In critical care medicine, the Medical Information Mart for Intensive Care IV (MIMIC-IV) database is a comprehensive single-center database that contains demographic and clinical data on all patients admitted to the ICU between 2008 and 2019 [10].”
Mohammed, S., Matos, J., Doutreligne, M., Celi, L. A., & Struja, T. (2023) · The Yale Journal of Biology and Medicine · doi:10.59249/wdji8829 · Cited by 13
Self-citationGuideline-informed reinforcement learning for mechanical ventilation in critical care
den Hengst, F., Otten, M., Elbers, P., van Harmelen, F., François-Lavet, V., & Hoogendoorn, M. (2023) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2023.102742 · Cited by 21
“We evaluate our approach in a case study on mechanical ventilation (MV) optimization using the MIMIC-III database [6].”
Grolleau, F., Petit, F., Gaudry, S., Diard, É., Quenot, J., Dreyfuss, D., Tran, V.-T., & Porcher, R. (2024) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocae004 · Cited by 17
Wang, X., Li, X., Sun, J., Wang, M., Lang, W., & Xu, X. (2024) · Frontiers in Neurology · doi:10.3389/fneur.2024.1349710 · Cited by 14
Enabling Machine Learning in Critical Care.
Pollard, T., & Celi, L. A. (2017) · PubMed · Cited by 13
Self-citationAl-Shwaheen, T. I., Moghbel, M., Hau, Y. W., & Ooi, C. Y. (2021) · Artificial Intelligence Review · doi:10.1007/s10462-021-09982-2 · Cited by 16
The effect of dobutamine in sepsis: a propensity score matched analysis
Zhu, Y., Yin, H., Zhang, R., Ye, X., & Wei, J. (2021) · BMC Infectious Diseases · doi:10.1186/s12879-021-06852-8 · Cited by 15
“Other source codes for our analyses, which were provided by the authors of the MIMICIII database, can be found at GitHub [16, 18].”
MethodDai, Y., Qin, S., Pan, H., Chen, T., & Bian, D. (2020) · International Journal of COPD · doi:10.2147/copd.s275573 · Cited by 12
Liu, C., Yuan, Q., Mao, Z., Hu, P., Wu, R., Liu, X., Hong, Q., Chi, K., Geng, X., & Sun, X. (2021) · The American Journal of Emergency Medicine · doi:10.1016/j.ajem.2021.03.006 · Cited by 13
Wu, M., Shi, L., Zhang, H., Liu, H., Liu, Y., & Zhang, W. (2022) · Postgraduate Medicine · doi:10.1080/00325481.2022.2027189 · Cited by 12
“IV) database is a comprehensive single-center database for critical care medicine that includes the demographic and clinical data on all patients admitted to the ICU from 2008 to 2019 [15].”
BackgroundHuang, S., Teng, Y., Du, J., Zhou, X., Duan, F., & Feng, C. (2022) · Australian Critical Care · doi:10.1016/j.aucc.2022.06.001 · Cited by 13
Solís-García, J., Vega-Márquez, B., Nepomuceno, J. A., Riquelme, J. C., & Nepomuceno-Chamorro, I. A. (2023) · Applied Intelligence · doi:10.1007/s10489-023-05124-z · Cited by 19
Early prediction of mortality upon intensive care unit admission
Yeh, Y., Kuo, Y., Kuo, K.-C., Cheng, Y., Liu, D.-S., Lai, F., Kuo, L.-C., Lee, T.-J., Chan, W.-S., Chiu, C., et al. (2024) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-024-02807-6 · Cited by 20
The effect of dobutamine vs milrinone in sepsis: A big data, real‐world study
Zhu, Y., Yin, H., Zhang, R., Ye, X., & Wei, J. (2021) · International Journal of Clinical Practice · doi:10.1111/ijcp.14689 · Cited by 14
Predicting risk for trauma patients using static and dynamic information from the MIMIC III database
Tsiklidis, E. J., Sinno, T. R., & Diamond, S. L. (2022) · PLoS ONE · doi:10.1371/journal.pone.0262523 · Cited by 19
“Since modern intensive care units (ICU) monitor patients continuously, the data-rich environment can be used to predict mortality, time-dependent risk, and provide opportunities for data science and machine learning [2,3].”
BackgroundMulti-modal Contrastive Learning for Healthcare Data Analytics
Li, R., & Gao, J. (2022) · IEEE International Conference on Healthcare Informatics (ICHI) · doi:10.1109/ichi54592.2022.00029 · Cited by 22
“MIMIC-III and MIMIC-IV [27] are publicly available EHR datasets, which consist the admission records of ICU patients.”
MethodOpenSep: a generalizable open source pipeline for SOFA score calculation and Sepsis-3 classification
Hofford, M. R., Yu, S., Johnson, A. E. W., Lai, A. M., Payne, P., & Michelson, A. P. (2022) · JAMIA Open · doi:10.1093/jamiaopen/ooac105 · Cited by 14
“We describe the degree of agreement between Sepsis-3 classification, time of onset, and calculated SOFA scores between the previously developed and well validated MIMICpipeline and the OpenSep pipeline we developed, when used on the MIMIC-IV data.(8) Finally, in cases where the 2 pipelines disagree the reasons for the…”
MethodSelf-citationData collection and research with MargheritaTre
Finazzi, S., Mandelli, G., Garbero, E., Mondini, M., Trussardi, G., Giardino, M., Tavola, M., & Bertolini, G. (2018) · Physiological Measurement · doi:10.1088/1361-6579/aad10f · Cited by 12
“In particular, the creation of a code repository (Johnson et al 2017b) is an important effort to ensure reproducibility of the analysis from an algorithmic stand-point.”
BackgroundCan Neo4j Replace PostgreSQL in Healthcare?
(2020) · PubMed Central · Cited by 12
Guo, Q., Li, H., Ouyang, H., Sun, R., Wang, J., Wu, M., Pan, Y., Wang, J., & Zhang, Y. (2021) · Frontiers in Cardiovascular Medicine · doi:10.3389/fcvm.2021.577742 · Cited by 12
“Study Cohort The Structure Query Language was used for data extraction (9).”
MethodMIMIC-ExtractPreprint
Wang, S., McDermott, M. B. A., Chauhan, G., Hughes, M. C., Naumann, T., & Ghassemi, M. (2020) · arXiv (Cornell University) · Cited by 11
Yuan, Z., Wang, H., Gao, Y., Qu, S.-N., Huang, C., Wang, H., Zhang, H., & Xing, X.-Z. (2021) · Annals of Palliative Medicine · doi:10.21037/apm-21-2352 · Cited by 11
Stulberg, E. L., Harris, B., Zheutlin, A. R., Delic, A., Sheibani, N., Anadani, M., Yaghi, S. R., Petersen, N., & de Havenon, A. (2023) · Neurology · doi:10.1212/wnl.0000000000207599 · Cited by 16
“shown to have strong prognostic value in MIMIC-III.(32,33) Higher scores represent worse prognosis and illness severity.”
BackgroundEL‐Manzalawy, Y., Abbas, M. M., Hoaglund, I., Cerna, A. U., Morland, T., Haggerty, C. M., Hall, E. S., & Fornwalt, B. K. (2021) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-021-01517-7 · Cited by 26
Huang, J., Zhang, J., Wang, F.-X., Liang, J.-Z., Chen, Q., & Lin, Z. (2021) · Scientific Reports · doi:10.1038/s41598-021-93907-0 · Cited by 11
Xie, K., Zheng, C., Wang, G.-M., Diao, Y., Luo, C., Wang, E., Hu, L., Ren, Z., Jing, L., Ren, B., et al. (2022) · BMC Surgery · doi:10.1186/s12893-022-01625-9 · Cited by 11
“Materials and methods Data source This retrospective cohort study was based on the Multiparameter Intelligent Monitoring in Intensive Care III (MIMIC III) database [16] (https:// mimic.”
MethodWang, W., & Jin, X. (2023) · Intelligent Pharmacy · doi:10.1016/j.ipha.2023.04.010 · Cited by 14
Biomedical informatics applications for precision management of neurodegenerative diseases
Miller, J. B., Shan, G., Lombardo, J. M., & Jimenez‐Maggiora, G. A. (2018) · Alzheimer s & Dementia Translational Research & Clinical Interventions · doi:10.1016/j.trci.2018.03.007 · Cited by 13
“Providing open access to data will also help address issues of transparency in research, reproducibility of results, and independent replication, especially when paired with simultaneous sharing of software and code [61–63].”
Liu, N., Ren, J., Yu, L., & Xie, J. (2020) · Journal of Emergency and Critical Care Medicine · doi:10.21037/jeccm.2020.01.01 · Cited by 12
“org/) with the sepsis3 code for MIMIC-III recomposed (13).”
MethodICD-9 Tagging of Clinical Notes Using Topical Word Embedding
Samonte, M. J. C., Gerardo, B. D., Fajardo, A. C., & Medina, R. P. (2018) · International Conference on Internet and E-Business (ICIEB) · doi:10.1145/3230348.3230357 · Cited by 13
“Bi-LSTM framework in Recurrent Neural Network [33].”
BackgroundInfluentialKashyap, A., Burris, H. H., Callison-Burch, C., & Boland, M. R. (2020) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2020.104101 · Cited by 12
Huang, S., Peng, Y., Wang, L., Wang, J., Zhou, X., Cui, X., Chen, L., Yang, B., & Feng, C. (2022) · Frontiers in Pharmacology · doi:10.3389/fphar.2022.974271 · Cited by 10
“4) that included 76540 critically ill admissions between 2008 and 2019 among 6 ICUs from a single center hospital (Beth Israel Deaconess Medical Center, BIDMC; Boston, Massachusetts) (Johnson et al., 2016; Johnson et al., 2018; Johnson et al., 2020).”
BackgroundLi, M., & Du, S. (2022) · Frontiers in Public Health · doi:10.3389/fpubh.2022.912151 · Cited by 13
Tang, S., Wu, C., Su, Y., & Li, Y. (2025) · BMC Cardiovascular Disorders · doi:10.1186/s12872-025-04546-6 · Cited by 15
“Clinical variables were extracted using SQL in conjunction with PostgreSQL (version 13.0) [11] and Navicat (version 16.0), ensuring uniformity and standardization of data processing.”
InfluentialShi, J., Hubbard, A., Fong, N., & Pirracchio, R. (2025) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-025-03058-9 · Cited by 12
“We extracted data on 11 vital signs and 35 laboratory tests, selected from the top 80% of the most commonly performed tests [21] (Appendix A, Table S1), as well as baseline characteristics and severity scores for each ICU stay [22].”
Sampling methods and feature selection for mortality prediction with neural networks
Steinmeyer, C., & Wiese, L. (2020) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2020.103580 · Cited by 12
“vant measurements, and survival data [6, 7].”
BackgroundChen, D., Wang, R., Jiang, Y., Xing, Z., Sheng, Q., Liu, X., Wang, R., Xie, H. C., & Zhao, L. (2023) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-023-02274-5 · Cited by 10
Blank, S. P., & Blank, R. M. (2023) · Critical Care Medicine · doi:10.1097/ccm.0000000000006069 · Cited by 10
“MIMIC-IV derived tables based on the centralized MIMIC code repository (19) were used to extract the sepsis-3 criteria, SOFA and Acute Physiology and Chronic Health Evaluation (APACHE) III scores, Charlson Comorbidity indexes, and ventilator status.”
MethodMining Typical Treatment Duration Patterns for Rational Drug Use from Electronic Medical Records
Chen, J., Guo, C., Sun, L., & Lu, M. (2019) · Journal of Systems Science and Systems Engineering · doi:10.1007/s11518-019-5427-7 · Cited by 10
Dong, N., Gao, N., Hu, W., Mu, Y., & Pang, L. (2022) · Frontiers in Medicine · doi:10.3389/fmed.2022.714384 · Cited by 10
“To improve the reproducibility of our study, we used the MIMIC Code Repository (15), which provides an open-source code alongside the freely accessible MIMIC-III database, to identify study cohorts and outcomes.”
MethodPredicting length of stay ranges by using novel deep neural networks
Zou, H., Yang, W., Wang, M., Qiao, Z., Liang, H., Wu, H. K., & Tang, L. (2023) · Heliyon · doi:10.1016/j.heliyon.2023.e13573 · Cited by 13
“Relevant measures, including fluid balance and severity assessment, were also constructed based on official MIMIC database documentation (https://github.com/MIT-LCP/mimic-code/tree/main/mimic-iii) [32].”
MethodRepresentation Balancing MDPs for Off-Policy Policy EvaluationPreprint
Liu, Y., Gottesman, O., Raghu, A., Komorowski, M., Faisal, A. A., Doshi‐Velez, F., & Brunskill, E. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.09044 · Cited by 15
“We also demonstrate our approach 72 can yield improved results on a challenging sepsis treatment management dataset extracted from the 73 MIMIC-III database [12].”
Wu, L., Li, Y., Zhang, X., Chen, X., Li, D., Nie, S., Li, X., & Bellou, A. (2023) · Nephrology Dialysis Transplantation · doi:10.1093/ndt/gfad065 · Cited by 11
Luo, Y., Peng, Y., Tang, Y., Huang, P., Zhang, Q., Wang, C., Zhang, W., Zhou, J., Liang, L., Zhang, Y., et al. (2024) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-024-02462-x · Cited by 9
“This retrospective analysis utilized the MIMIC-IV data-base (https://mimic-iv.mit.edu/), an improved version of MIMIC-III [13], containing comprehensive hospitalization data for patients admitted to the Advanced Medical Center in Boston, USA, from 2008 to 2019.”
MethodNatural Language-Assisted Multi-modal Medication Recommendation
Tan, J., Rong, Y., Zhao, K., Bian, T., Xu, T., Huang, J., Cheng, H., & Meng, H. M. L. (2024) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3627673.3679529 · Cited by 24
In with the old, in with the new: machine learning for time to event biomedical research
Danciu, I., Agasthya, G. A., Tate, J. P., Chandrashekar, M., Goethert, I., Ovchinnikova, O. S., McMahon, B. H., & Justice, A. C. (2022) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocac106 · Cited by 9
Clinical concept recognition: Evaluation of existing systems on EHRs
Lossio-Ventura, J. A., Sun, R., Boussard, S., & Hernandez‐Boussard, T. (2023) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2022.1051724 · Cited by 12
Hedjoudje, A., Cheurfa, C., Talby, M. E., Lévy, P., Prat, F., & Piton, G. (2023) · Digestive and Liver Disease · doi:10.1016/j.dld.2023.01.158 · Cited by 9
Emdad, F. B., Tian, S., Nandy, E., Hanna, K., & He, Z. (2023) · PubMed Central · Cited by 22
Chen, Q., Huang, K., Li, L., Lin, X., Ding, C., Zhang, J., & Chen, Q. (2018) · Therapeutics and Clinical Risk Management · doi:10.2147/tcrm.s170647 · Cited by 9
“Variables We applied Structured Query Language to extract data from the database mainly by using codes from the MIMIC Code Repository.16,24 Age, sex, ICU mortality and hospital mortal- ity, length of ICU stay and length of hospital stay, 28-day mortality and 90-day mortality, admission serum uric acid (admission was d…”
MethodHuang, W., Xie, R., Hong, Y., & Chen, Q. (2020) · International Journal of COPD · doi:10.2147/copd.s244020 · Cited by 9
“com/MIT-LCP/mimiccode).(14) COPD was identified using International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) codes (490, 4910, 4911, 4912, 49,120, 49,121, 49,122, 4918, 4919, 4920, 4928, 494, 4940, 4941, 496).”
Impact of obesity on outcomes in patients with acute respiratory syndrome
Liu, Q.-Y., Chen, Y., He, Y., & Zhu, R. (2021) · Journal of International Medical Research · doi:10.1177/03000605211024860 · Cited by 10
“The Simplified Acute Physiology Score II (SAPS II), Sequential Organ Failure Assessment (SOFA),(22) Glasgow Coma Scale (GCS), and Acute Physiology and Chronic Health Evaluation (APACHE) III(23) scores were included.(24) Body mass index (BMI) was calculated using height and weight recorded in the first 24 hours after p…”
MethodPopoff, B., Besnier, E., Dureuil, B., Veber, B., & Clavier, T. (2021) · European Journal of Emergency Medicine · doi:10.1097/mej.0000000000000854 · Cited by 8
Chen, D., Zhou, K., Tian, R., Wang, R., & Zhou, Z. (2024) · BMJ Open · doi:10.1136/bmjopen-2024-084562 · Cited by 9
Zhang, X., Kong, W., Shi, R., Sun, L., Xu, M., & Gong, L. (2025) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2025.110670 · Cited by 11
ChatICD: Prompt Learning for Few-shot ICD Coding through ChatGPT
Liu, J., Yang, S., Peng, T., Hu, X., & Zhu, Q. (2023) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm58861.2023.10385482 · Cited by 9
“The MIMIC-III dataset [3] contains health data from approximately 58,000 individuals, including diagnostic codes, discharge summaries, laboratory measurements, and body measurements.”
BackgroundMethodInfluentialJiang, J., Zhang, J., Liu, Y., Xu, D., & Peng, Z. (2021) · Shock · doi:10.1097/shk.0000000000001789 · Cited by 8
“Diagnosis of AKI was made when SCr or UO calculated by either ABW or IBW met the aforementioned thresholds using established code (11).”
MethodCyclOps: Cyclical development towards operationalizing ML models for healthPreprint
Krishnan, A., Subasri, V., McKeen, K., Kore, A., Ogidi, F., Alinoori, M., Lalani, N., Dhalla, A., Verma, A. A., Razak, F. A., et al. (2022) · medRxiv · doi:10.1101/2022.12.02.22283021 · Cited by 12
“Most notably, the mimic-code repository, a toolkit that processes the Medical Information Mart for Intensive Care (MIMIC) into an accessible relational database [7], has enabled researchers to easily set up the database and extract data towards their research use cases.”
BackgroundWang, H., Sun, H., & Sun, J. (2024) · Heliyon · doi:10.1016/j.heliyon.2024.e34702 · Cited by 7
“Data extraction included survival information from the ’ patients ’ table and hospital stay durations from the ’ admissions ’ table of the MIMIC-IV database [21].”
MethodXu, J., Wang, X., Chen, W., Tian, M., & You, C. (2024) · Frontiers in Neurology · doi:10.3389/fneur.2024.1464216 · Cited by 7
Lei, C., Ding, Z., Wang, Q., Tao, S., Zhou, Q., Yin, P., Luo, Y., Yang, F., Chen, X., Cai, Y., et al. (2025) · Clinical and Experimental Medicine · doi:10.1007/s10238-025-01744-6 · Cited by 7
“Data were extracted using Structured Query Language (SQL) programmatically in PostgreSQL (version 14.0), and the SQL script code used to extract patient information was obtained from the GitHub website (https:// github. com/ MIT-LCP/ mimic-code/ tree/ main/ mimic-iv) [6].”
InfluentialLi, T., Zhou, D., Zhao, D., Lin, Q., Wang, D., & Wang, C. (2022) · BMC Anesthesiology · doi:10.1186/s12871-022-01708-3 · Cited by 9
“We used the MIMIC Code Repository to define many concepts in MIMIC-III database [17].”
MethodLiao, Q., Pu, Y., Jin, X., Zhuang, Z., Xu, X., Ren, X., Liu, G., & Ding, Q. (2023) · BMC Pulmonary Medicine · doi:10.1186/s12890-023-02384-w · Cited by 7
“The database has been approved by the Massachusetts Institute of Technology and BIDMC Institutional Review Board [21].”
MethodSex Differences in Short- and Long-Term Survival Among Critically Ill Patients with Sepsis
Lin, S., He, W., Hu, Z., Bai, L., & Zeng, M. (2021) · International Journal of General Medicine · doi:10.2147/ijgm.s294229 · Cited by 8
Zhang, B., Han, Y., Chen, X., Cao, K. X., You, Y., Chen, M., Zhu, Z., & Yu, W. (2022) · Journal of Trauma Nursing · doi:10.1097/jtn.0000000000000672 · Cited by 8
Wang, Y.-R., Yang, L., Lee, C., Chang, S., Chen, P., & Jhou, H. (2023) · Journal of Clinical Medicine · doi:10.3390/jcm12020580 · Cited by 7
“0), with approval from the Institutional Review Board (IRB) of Beth Israel Deaconess Medical Center (BIDMC) and the Massachusetts Institute of Technology [14,15].”
MethodInfluentialAn HBase-Based Optimization Model for Distributed Medical Data Storage and Retrieval
Zhu, C., Liu, Z., Zou, B., Xiao, Y., Zeng, M., Wang, H., & Fan, Z. (2023) · Electronics · doi:10.3390/electronics12040987 · Cited by 7
Bouvarel, B., Carrat, F., & Lapidus, N. (2023) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-023-02264-7 · Cited by 8
“These models were built from ICU hospitalizations lasting more than 48 h, using longitudinal health care data with missing values from electronic health records available in the freely accessible Medical Information Mart for Intensive Care (MIMIC-III) critical care database [14, 15].”
Method
Metadata
- Authors
- Johnson, A. E. W., Stone, D. J., Celi, L. A., & Pollard, T. J.
- Affiliations
- Massachusetts Institute of Technology (US)
- University of Virginia (US)
- Beth Israel Deaconess Medical Center (US)
- Corresponding author
- Johnson, Alistair E W
- Venue
- Journal of the American Medical Informatics Association
- Publisher
- Oxford University Press (OUP)
- Published
- 2017-09-27
- Volume
- 25
- Issue
- 1
- Pages
- 32-39
- ISSN
- 1067-5027 (ISSN-L)
- 1527-974X
- PMC
- PMC6381763
- OpenAlex
- W2757504960
- Cited by
- 442
- References
- 36
- Citations by year
- 2026: 44 · 2025: 67 · 2024: 56 · 2023: 72 · 2022: 57 · 2021: 62 · 2020: 39 · 2019: 24 · 2018: 17 · 2017: 3
- FWCI
- 17.77
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Journal of the American Medical Informatics Association (publishedVersion, cc-by-nc)
- Europe PMC (PubMed Central) (publishedVersion, cc-by-nc)
- PubMed Central (publishedVersion, cc-by-nc)
- PubMed (publishedVersion)
- Funders
- National Institutes of Health (NIH-R01-EB017205, NIH-R01-EB001659)
- Topics
- Health Sciences › Medicine › Critical Care and Intensive Care Medicine › Sepsis Diagnosis and Treatment
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Keywords
- reproducibility, critical care research, intensive care unit data, open-source code, electronic health records, sepsis definitions, disease severity index, organ failure scores, data extraction, research data sharing, community collaboration
- MeSH terms
- Critical Care, Humans, ROC Curve, Severity of Illness Index, Therapeutics, Reproducibility of Results, Comorbidity, Databases, Factual, Sepsis, Access to Information, Biomedical Research, Data Mining, Electronic Health Records, Organ Dysfunction Scores
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.