The DOI matches its Crossref record; 67 of the 67 references it cites were found in a registry or an index.
Identifying and Overcoming Threats to Reproducibility, Replicability, Robustness, and Generalizability in Microbiome Research
Open accessAbstract
The “reproducibility crisis” in science affects microbiology as much as any other area of inquiry, and microbiologists have long struggled to make their research reproducible. We need to respect that ensuring that our methods and results are sufficiently transparent is difficult. This difficulty is compounded in interdisciplinary fields such as microbiome research. There are many reasons why a researcher is unable to reproduce a previous result, and even if a result is reproducible, it may not be correct. Furthermore, failures to reproduce previous results have much to teach us about the scientific process and microbial life itself. This Perspective delineates a framework for identifying and overcoming threats to reproducibility, replicability, robustness, and generalizability of microbiome research. Instead of seeing signs of a crisis in others’ work, we need to appreciate the technical and social difficulties that limit reproducibility in the work of others as well as our own.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Schloss, P. D., Westcott, S. L., Ryabin, T., Hall, J. R., Hartmann, M., Hollister, E. B., Lesniewski, R. A., Oakley, B. B., Parks, D. H., Robinson, C. J., et al. (2009) · Applied and Environmental Microbiology · doi:10.1128/aem.01541-09 · Cited by 19,549
Self-citationAn obesity-associated gut microbiome with increased capacity for energy harvest
Turnbaugh, P. J., Ley, R. E., Mahowald, M. A., Magrini, V., Mardis, E. R., & Gordon, J. I. (2006) · Nature · doi:10.1038/nature05414 · Cited by 12,444 · cited 2×
“They observed stark differences in the microbiota of genetically lean and obese mice and found that the microbiota of obese mice could transmit the propensity to gain weight to germfree mice (15).”
BackgroundMethodStructure, function and diversity of the healthy human microbiome
The Human Microbiome Project Consortium (2012) · Nature · doi:10.1038/nature11234 · Cited by 11,726
Greengenes, a Chimera-Checked 16S rRNA Gene Database and Workbench Compatible with ARB
DeSantis, T. Z., Hugenholtz, P., Larsen, N., Rojas, M., Brodie, E. L., Keller, K., Huber, T., Dalevi, D., Hu, P., & Andersen, G. L. (2006) · Applied and Environmental Microbiology · doi:10.1128/aem.03006-05 · Cited by 10,891
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505
Human gut microbes associated with obesity
Ley, R. E., Turnbaugh, P. J., Klein, S., & Gordon, J. I. (2006) · Nature · doi:10.1038/4441022a · Cited by 8,785
A core gut microbiome in obese and lean twins
Turnbaugh, P. J., Hamady, M., Yatsunenko, T., Cantarel, B. L., Duncan, A., Ley, R. E., Sogin, M. L., Jones, W. J., Roe, B. A., Affourtit, J. P., et al. (2009) · Nature · doi:10.1038/nature07540 · Cited by 7,823
Obesity alters gut microbial ecology
Ley, R. E., Bäckhed, F., Turnbaugh, P., Lozupone, C. A., Knight, R. D., & Gordon, J. I. (2005) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0504978102 · Cited by 6,176
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · CHANCE · doi:10.1080/09332480.2005.10722754 · Cited by 4,702 · cited 1×
“In other cases, the original study may have been underpowered, rendering it susceptible to an inflated risk of type I errors (38).”
BackgroundInduction of Intestinal Th17 Cells by Segmented Filamentous Bacteria
Ivanov, I. I., Atarashi, K., Manel, N., Brodie, E. L., Shima, T., Karaoz, U., Wei, D., Goldfarb, K. C., Santee, C. A., Lynch, S. V., et al. (2009) · Cell · doi:10.1016/j.cell.2009.09.033 · Cited by 4,554
Cited by
Microbiome definition re-visited: old concepts and new challenges
Berg, G., Rybakova, D., Fischer, D., Cernava, T., Vergès, M.-C. C., Charles, T., Chen, X., Cocolin, L., Eversole, K., Corral, G. H., et al. (2020) · Microbiome · doi:10.1186/s40168-020-00875-0 · Cited by 2,275
RESCRIPt: Reproducible sequence taxonomy reference database management
(2021) · doi:10.1371/journal.pcbi.1009581 · Cited by 983
Systems biology and multi-omics integration: viewpoints from the metabolomics research community
Pinu, F. R., Beale, D. J., Paten, A. M., Kouremenos, K., Swarup, S., Schirra, H. J., & Wishart, D. (2019) · Metabolites · doi:10.3390/metabo9040076 · Cited by 723
“For instance, data from host-associated microbiomes can also now be integrated with the exometabolome data to better understand the mechanistic bases of their associations [26].”
BackgroundGreengenes2 unifies microbial data in a single reference tree
McDonald, D., Jiang, Y., Balaban, M., Cantrell, K., Zhu, Q., Gonzalez, A., Morton, J. T., Nicolaou, G., Parks, D. H., Karst, S. M., et al. (2024) · Nature Biotechnology · doi:10.1038/s41587-023-01845-1 · Cited by 681
Cullin, N., Azevedo Antunes, C., Straussman, R., Stein-Thoeringer, C. K., & Elinav, E. (2021) · Cancer Cell · doi:10.1016/j.ccell.2021.08.006 · Cited by 578
“However, amplification biases, chimera generation, and methodological variation in amplified regions of the 16S rRNAgene, data curation, and referencedatabases, introduce challenges in analytical outputs, data interpretation, and reproducibility across studies (Schloss, 2018).”
BackgroundReporting guidelines for human microbiome research: the STORMS checklist
Mirzayi, C., Renson, A., Genomic Standards Consortium, Massive Analysis and Quality Control Society, Furlanello, C., Sansone, S.-A., Zohra, F., Elsafoury, S., Geistlinger, L., Kasselman, L. J., et al. (2021) · Nature Medicine · doi:10.1038/s41591-021-01552-x · Cited by 521
Self-citationNext-generation sequencing: insights to advance clinical investigations of the microbiome
Wensel, C. R., Pluznick, J. L., Salzberg, S. L., & Sears, C. L. (2022) · Journal of Clinical Investigation · doi:10.1172/jci154944 · Cited by 437
Metagenome-wide association of gut microbiome features for schizophrenia
Zhu, F., Ju, Y., Wang, W., Wang, Q., Guo, R., Ma, Q., Sun, Q., Fan, Y., Xie, Y., Yang, Z., et al. (2020) · Nature Communications · doi:10.1038/s41467-020-15457-9 · Cited by 409
A Genus Definition for Bacteria and Archaea Based on a Standard Genome Relatedness Index
Barco, R. A., Garrity, G. M., Scott, J. J., Amend, J. P., Nealson, K. H., & Emerson, D. (2020) · mBio · doi:10.1128/mbio.02475-19 · Cited by 347
Reintroducing mothur: 10 Years Later
Schloss, P. D. (2020) · Applied and Environmental Microbiology · doi:10.1128/aem.02343-19 · Cited by 285
Self-citationPotential Role of the Gut Microbiome In Colorectal Cancer Progression
Kim, J., & Lee, H. K. (2022) · Frontiers in Immunology · doi:10.3389/fimmu.2021.807648 · Cited by 259
Identifying biases and their potential solutions in human microbiome studies
Nearing, J. T., Comeau, A. M., & Langille, M. G. I. (2021) · Microbiome · doi:10.1186/s40168-021-01059-0 · Cited by 240
“However, results from different studies often do not match with previous findings [5, 6].”
ResultMa, S., Shungin, D., Mallick, H., Schirmer, M., Nguyen, L. H., Kolde, R., Franzosa, E., Vlamakis, H., Xavier, R., & Huttenhower, C. (2022) · Genome biology · doi:10.1186/s13059-022-02753-4 · Cited by 164
Machine Learning and Artificial Intelligence in the Multi-Omics Approach to Gut Microbiota
Rozera, T., Pasolli, E., Segata, N., & Ianiro, G. (2025) · Gastroenterology · doi:10.1053/j.gastro.2025.02.035 · Cited by 112
Han, D., Gao, P., Li, R., Tan, P., Xie, J., Zhang, R., & Li, J. (2020) · Journal of Advanced Research · doi:10.1016/j.jare.2020.07.010 · Cited by 93
A Systematic Review of Reproducibility Research in Natural Language Processing
Belz, A., Agarwal, S., Shimorina, A., & Reiter, E. (2021) · Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume · doi:10.18653/v1/2021.eacl-main.29 · Cited by 86
Metadata
- Authors
- Schloss, P. D.
- Venue
- mBio
- Publisher
- American Society for Microbiology
- Published
- 2018-07-05
- Volume
- 9
- Issue
- 3
- ISSN
- 2150-7511 (ISSN-L)
- 2161-2129
- OpenAlex
- W2805044645
- Cited by
- 280
- References
- 67
- FWCI
- 36.96
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.