The DOI matches its Crossref record; 39 of the 39 references it cites were found in a registry or an index.
The MIMIC Code Repository: enabling reproducibility in critical care research
Open accessAbstract
Objective: Lack of reproducibility in medical studies is a barrier to the generation of a robust knowledge base to support clinical decision-making. In this paper we outline the Medical Information Mart for Intensive Care (MIMIC) Code Repository, a centralized code base for generating reproducible studies on an openly available critical care dataset. Materials and Methods: Code is provided to load the data into a relational structure, create extractions of the data, and reproduce entire analysis plans including research studies. Results: Concepts extracted include severity of illness scores, comorbid status, administrative definitions of sepsis, physiologic criteria for sepsis, organ failure scores, treatment administration, and more. Executable documents are used for tutorials and reproduce published studies end-to-end, providing a template for future researchers to replicate. The repository’s issue tracker enables community discussion about the data and concepts, allowing users to collaboratively improve the resource. Discussion: The centralized repository provides a platform for users of the data to interact directly with the data generators, facilitating greater understanding of the data. It also provides a location for the community to collaborate on necessary concepts for research progress and share them with a larger audience. Consistent application of the same code for underlying concepts is a key step in ensuring that research studies on the MIMIC database are comparable and reproducible. Conclusion: By providing open source code alongside the freely accessible MIMIC-III database, we enable end-to-end reproducible analysis of electronic health records.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures






Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“The MIMIC Code Repository was originally created for MIMIC-III to enable researchers to share and reuse code24.”
From MIMIC-IV, a freely accessible electronic health record dataset (2023) · cites this paper as [24]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The third international consensus definitions for sepsis and septic shock (Sepsis-3).
Singer, M., Deutschman, C. S., Seymour, C. W., Shankar-Hari, M., Annane, D., Bauer, M., Bellomo, R., Bernard, G. R., Chiche, J.-D., Coopersmith, C. M., et al. (2016) · JAMA · doi:10.1001/jama.2016.0287 · Cited by 28,250 · cited 1×
“21 The precise onset of sepsis is not typically documented in the EHR, and is, in fact, a difficult item to capture clinically.”
BackgroundVincent, J. L., Moreno, R., Takala, J., Willatts, S., De Mendonça, A., Bruining, H., Reinhart, C. K., Suter, P. M., & Thijs, L. G. (1996) · Intensive Care Medicine · doi:10.1007/bf01709751 · Cited by 11,674 · cited 1×
“Some scores assess multiple organ systems: the SOFA score 12 and LODS 13 both assess 6 organ systems for failure.”
BackgroundCoding algorithms for defining comorbidities in ICD-9-CM and ICD-10 administrative data
Quan, H., Sundararajan, V., Halfon, P., Fong, A., Burnand, B., Luthi, J.-C., Saunders, L. D., Beck, C. A., Feasby, T. E., & Ghali, W. A. (2005) · Medical Care · doi:10.1097/01.mlr.0000182534.19832.83 · Cited by 10,669 · cited 1×
“27 Finally, Quan et al. 28 proposed an enhanced ICD-9 coding methodology based on examining inconsistencies among previous definitions.”
MethodComorbidity measures for use with administrative data.
Elixhauser, A., Steiner, C., Harris, D. R., & Coffey, R. M. (1998) · Medical Care · doi:10.1097/00005650-199801000-00004 · Cited by 9,921 · cited 4×
“Van Walraven et al. 29 later aggregated comorbidities codified by Elixhauser et al. 26 into a single point score for in-hospital mortality prediction, which is also available in the repository.”
BackgroundMethodInfluentialAngus, D. C., Linde-Zwirble, W. T., Lidicker, J., Clermont, G., Carcillo, J., & Pinsky, M. R. (2001) · Critical Care Medicine · doi:10.1097/00003246-200107000-00002 · Cited by 8,503 · cited 3×
“The criteria as proposed by Angus et al. 23 were validated in a later study by Iwashyna et al. 25 These 3 criteria – explicit coding, those proposed by Angus et al., 23 and those proposed by Martin et al. 24 – are available in the repository.”
MethodMIMIC-III, a freely accessible critical care database
Johnson, A. E. W., Pollard, T. J., Shen, L., Lehman, L.-W. H., Feng, M., Ghassemi, M., Moody, B., Szolovits, P., Anthony Celi, L., & Mark, R. G. (2016) · Scientific Data · doi:10.1038/sdata.2016.35 · Cited by 8,141 · cited 1×
“3,4 The database comprises detailed clinical information regarding > 60000 stays in ICUs at the Beth Israel Deaconess Medical Center in Boston, Massachusetts, collected as part of routine clinical care.”
BackgroundSelf-citationLe Gall, J.-R. (1993) · JAMA: The Journal of the American Medical Association · doi:10.1001/jama.270.24.2957 · Cited by 7,773
Acute Kidney Injury Network: report of an initiative to improve outcomes in acute kidney injury
Mehta, R. L., Kellum, J. A., Shah, S. V., Molitoris, B. A., Ronco, C., Warnock, D. G., Levin, A., & the Acute Kidney Injury Network (2007) · Critical Care · doi:10.1186/cc5713 · Cited by 7,048 · cited 1×
“Examples include the Model for End-stage Liver Disease, 14 the Risk/Injury/Failure/Loss/End-stage renal disease criteria, 15 the Acute Kidney Injury Network classification, 16 and the Kidney Disease: Improving Global Outcomes criteria.”
BackgroundThe epidemiology of sepsis in the United States from 1979 through 2000
Martin, G. S., Mannino, D. M., Eaton, S., & Moss, M. (2003) · New England Journal of Medicine · doi:10.1056/nejmoa022139 · Cited by 5,841 · cited 4×
“The criteria as proposed by Angus et al. 23 were validated in a later study by Iwashyna et al. 25 These 3 criteria – explicit coding, those proposed by Angus et al., 23 and those proposed by Martin et al. 24 – are available in the repository.”
BackgroundInfluential1,500 scientists lift the lid on reproducibility
Baker, M. (2016) · Nature · doi:10.1038/533452a · Cited by 4,209 · cited 1×
“Concerns about the reproducibility of results in science are becoming increasingly prominent in both scientific and mainstream literature.(1) Some commentators have gone so far as to call the current state a crisis, citing causes such as pressure to publish positive results, the cost of replicating studies such as dou…”
Background
Cited by
MIMIC-IV, a freely accessible electronic health record dataset
Johnson, A. E. W., Bulgarelli, L., Shen, L., Gayles, A., Shammout, A., Horng, S., Pollard, T. J., Hao, S., Moody, B., Gow, B., et al. (2023) · Scientific Data · doi:10.1038/s41597-022-01899-x · Cited by 3,401
“The MIMIC Code Repository was originally created for MIMIC-III to enable researchers to share and reuse code 24 .”
MethodSelf-citationMIMIC-CXR, a de-identified publicly available database of chest radiographs with free-text reports
Johnson, A. E. W., Pollard, T. J., Berkowitz, S. J., Greenbaum, N. R., Lungren, M. P., Deng, C.-Y., Mark, R. G., & Horng, S. (2019) · Scientific Data · doi:10.1038/s41597-019-0322-0 · Cited by 1,816
“We believe that publicly accessible code using publicly available data will accelerate research in the field and ensure reproducibility of future studies 26,27 .”
MethodSelf-citationSerpa Neto, A., Deliberato, R. O., Johnson, A. E. W., Bos, L. D., Amorim, P., Pereira, S. M., Cazati, D. C., Cordioli, R. L., Correa, T. D., Pollard, T. J., et al. (2018) · Intensive Care Medicine · doi:10.1007/s00134-018-5375-6 · Cited by 613
Self-citationOpen science saves lives: lessons from the COVID-19 pandemic
Besançon, L., Peiffer-Smadja, N., Segalas, C., Jiang, H., Masuzzo, P., Smout, C., Billy, E., Deforet, M., & Leyrat, C. (2021) · BMC Medical Research Methodology · doi:10.1186/s12874-021-01304-y · Cited by 249
“An open code repository has been created [90] and many scientific journals are now asking researchers to publish the source code alongside their findings.”
BackgroundTransthoracic echocardiography and mortality in sepsis: analysis of the MIMIC-III database
Feng, M., McSparron, J. I., Kien, D. T., Stone, D. J., Roberts, D. H., Schwartzstein, R. M., Vieillard-Baron, A., & Celi, L. A. (2018) · Intensive Care Medicine · doi:10.1007/s00134-018-5208-7 · Cited by 239
“Discussions around reliability of specific data elements are captured in a community white board to guide future investigators who may be interested in exploring similar concepts [21].”
Self-citation
Metadata
- Authors
- Johnson, A. E. W., Stone, D. J., Celi, L. A., & Pollard, T. J.
- Affiliations
- Massachusetts Institute of Technology (US)
- University of Virginia (US)
- Beth Israel Deaconess Medical Center (US)
- Corresponding author
- Johnson, Alistair E W
- Venue
- Journal of the American Medical Informatics Association
- Publisher
- Oxford University Press (OUP)
- Published
- 2017-09-27
- Volume
- 25
- Issue
- 1
- Pages
- 32-39
- ISSN
- 1067-5027 (ISSN-L)
- 1527-974X
- PMC
- PMC6381763
- OpenAlex
- W2757504960
- Cited by
- 442
- References
- 36
- Citations by year
- 2026: 44 · 2025: 67 · 2024: 56 · 2023: 72 · 2022: 57 · 2021: 62 · 2020: 39 · 2019: 24 · 2018: 17 · 2017: 3
- FWCI
- 17.77
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Journal of the American Medical Informatics Association (publishedVersion, cc-by-nc)
- Europe PMC (PubMed Central) (publishedVersion, cc-by-nc)
- PubMed Central (publishedVersion, cc-by-nc)
- PubMed (publishedVersion)
- Funders
- National Institutes of Health (NIH-R01-EB017205, NIH-R01-EB001659)
- Topics
- Health Sciences › Medicine › Critical Care and Intensive Care Medicine › Sepsis Diagnosis and Treatment
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Keywords
- reproducibility, critical care research, intensive care unit data, open-source code, electronic health records, sepsis definitions, disease severity index, organ failure scores, data extraction, research data sharing, community collaboration
- MeSH terms
- Critical Care, Humans, ROC Curve, Severity of Illness Index, Therapeutics, Reproducibility of Results, Comorbidity, Databases, Factual, Sepsis, Access to Information, Biomedical Research, Data Mining, Electronic Health Records, Organ Dysfunction Scores
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: The MIMIC Code Repository: enabling reproducibility in critical care research. https://app.cochat.ai/papers/10.1093/jamia/ocx084
- The paper
- Johnson, A. E. W., Stone, D. J., Celi, L. A., & Pollard, T. J. (2018). The MIMIC Code Repository: enabling reproducibility in critical care research. Journal of the American Medical Informatics Association. https://doi.org/10.1093/jamia/ocx084
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.