The DOI matches its Crossref record; 71 of the 71 references it cites were found in a registry or an index.
The Power of Microbiome Studies: Some Considerations on Which Alpha and Beta Metrics to Use and How to Report Results
Open accessPublished version of a preprint
Abstract
Background: Since sequencing techniques have become less expensive, larger sample sizes are applicable for microbiota studies. The aim of this study is to show how, and to what extent, different diversity metrics and different compositions of the microbiota influence the needed sample size to observe dissimilar groups. Empirical 16S rRNA amplicon sequence data obtained from animal experiments, observational human data, and simulated data were used to perform retrospective power calculations. A wide variation of alpha diversity and beta diversity metrics were used to compare the different microbiota datasets and the effect on the sample size. Results: Our data showed that beta diversity metrics are the most sensitive to observe differences as compared with alpha diversity metrics. The structure of the data influenced which alpha metrics are the most sensitive. Regarding beta diversity, the Bray–Curtis metric is in general the most sensitive to observe differences between groups, resulting in lower sample size and potential publication bias. Conclusion: We recommend performing power calculations and to use multiple diversity metrics as an outcome measure. To improve microbiota studies, awareness needs to be raised on the sensitivity and bias for microbiota research outcomes created by the used metrics rather than biological differences. We have seen that different alpha and beta diversity metrics lead to different study power: because of this, one could be naturally tempted to try all possible metrics until one or more are found that give a statistically significant test result, i.e., p -value < α. This way of proceeding is one of the many forms of the so-called p -value hacking. To this end, in our opinion, the only way to protect ourselves from (the temptation of) p -hacking would be to publish a statistical plan before experiments are initiated, describing the outcomes of interest and the corresponding statistical analyses to be performed.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Data interpretation must be cautious due to high inter-individual variability, lack of standardization in pipelines, and the absence of a universally defined healthy microbiome [76,77].”
From The Seven Methods for the Evaluation of Nutritional Status—ABCDEFG: Narrative Review (2026) · cites this paper as [76]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Faul, F., Erdfelder, E., Lang, A.-G., & Buchner, A. (2007) · Behavior Research Methods · doi:10.3758/bf03193146 · Cited by 65,971
DADA2: High–resolution sample inference from Illumina amplicon data
Callahan, B. J., McMurdie, P. J., Rosen, M. J., Han, A. W., Johnson, A. J. A., & Holmes, S. P. (2016) · Nature Methods · doi:10.1038/nmeth.3869 · Cited by 35,698
The SILVA ribosomal RNA gene database project: improved data processing and web-based tools
Quast, C., Pruesse, E., Yilmaz, P., Gerken, J., Schweer, T., Yarza, P., Peplies, J., & Glöckner, F. O. (2012) · Nucleic Acids Research · doi:10.1093/nar/gks1219 · Cited by 33,055
Statistical Power Analysis for the Behavioral Sciences
Cohen, J. (2013) · doi:10.4324/9780203771587 · Cited by 25,373
phyloseq: An R Package for Reproducible Interactive Analysis and Graphics of Microbiome Census Data
McMurdie, P. J., & Holmes, S. (2013) · PLoS ONE · doi:10.1371/journal.pone.0061217 · Cited by 21,942
Simpson, E. (1949) · Nature · doi:10.1038/163688a0 · Cited by 14,809
Non‐parametric multivariate analyses of changes in community structure
Clarke, K. R. (1993) · Australian Journal of Ecology · doi:10.1111/j.1442-9993.1993.tb00438.x · Cited by 14,449
Use of Ranks in One-Criterion Variance Analysis
Kruskal, W. H., & Wallis, W. A. (1952) · Journal of the American Statistical Association · doi:10.1080/01621459.1952.10483441 · Cited by 13,912
A new method for non-parametric multivariate analysis of variance
Anderson, M. J. (2001) · Austral Ecology · doi:10.1111/j.1442-9993.2001.01070.pp.x · Cited by 13,828
An Ordination of the Upland Forest Communities of Southern Wisconsin
Bray, J. R., & Curtis, J. T. (1957) · Ecological Monographs · doi:10.2307/1942268 · Cited by 11,736
Structure, function and diversity of the healthy human microbiome
The Human Microbiome Project Consortium (2012) · Nature · doi:10.1038/nature11234 · Cited by 11,726
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505
Lakens, D. (2013) · Frontiers in Psychology · doi:10.3389/fpsyg.2013.00863 · Cited by 10,303
Estimating the reproducibility of psychological science
Open Science Collaboration (2015) · Science · doi:10.1126/science.aac4716 · Cited by 8,515
UniFrac: a New Phylogenetic Method for Comparing Microbial Communities
Lozupone, C., & Knight, R. (2005) · Applied and Environmental Microbiology · doi:10.1128/aem.71.12.8228-8235.2005 · Cited by 8,227
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567
Vegan : community ecology package. R package version 1.17-9
Oksanen, J. (2007) · Medical Entomology and Zoology · Cited by 6,549
Diversity and Evenness: A Unifying Notation and Its Consequences
Hill, M. (1973) · Ecology · doi:10.2307/1934352 · Cited by 6,486
The ASA Statement on p -Values: Context, Process, and Purpose
Wasserstein, R. L., & Lazar, N. A. (2016) · The American Statistician · doi:10.1080/00031305.2016.1154108 · Cited by 6,306 · cited 1×
“…(p-hacking) (Simmons et al., 2011). p-Hacking (also called data dredging, significance chasing, significance questing, or selective inference (Wasserstein and Lazar, 2016)) is the improper use of data (like adding or removing observations) or statistical procedures (like applying many…”
MethodSample size in factor analysis.
MacCallum, R. C., Widaman, K. F., Zhang, S., & Hong, S. (1999) · Psychological Methods · doi:10.1037/1082-989x.4.1.84 · Cited by 5,497
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · CHANCE · doi:10.1080/09332480.2005.10722754 · Cited by 4,702 · cited 3×
“A well-known phenomenon that can hamper progress in every research field concerns publication biases in reporting mainly positive findings (Ioannidis, 2005).”
BackgroundTHE DISTRIBUTION OF THE FLORA IN THE ALPINE ZONE. 1
Jaccard, P. (1912) · New Phytologist · doi:10.1111/j.1469-8137.1912.tb05611.x · Cited by 4,561
Nonparametric estimation of the number of classes in a population
Chao, A. (1984) · Scandinavian Journal of Statistics · Cited by 4,263
Exact sequence variants should replace operational taxonomic units in marker-gene data analysis
Callahan, B. J., McMurdie, P. J., & Holmes, S. (2017) · The ISME Journal · doi:10.1038/ismej.2017.119 · Cited by 3,253
Microbiome Datasets Are Compositional: And This Is Not Optional
Gloor, G. B., Macklaim, J. M., Pawlowsky-Glahn, V., & Egozcue, J. J. (2017) · Frontiers in Microbiology · doi:10.3389/fmicb.2017.02224 · Cited by 3,044 · cited 1×
“This happens because basic assumptions are not met, such as independence of the sample units, the multivariate normality of errors, homogeneity of variance–covariance matrices among the groups, or because the number of variables is larger than the number of samples, making it impossible to apply the test (Hanson and W…”
MethodLozupone, C., Hamady, M., Kelley, S. T., & Knight, R. D. (2007) · Applied and Environmental Microbiology · doi:10.1128/aem.01996-06 · Cited by 2,717
Normalization and microbial differential abundance strategies depend upon data characteristics
Weiss, S. J., Xu, Z. Z., Peddada, S. D., Amir, A., Bittinger, K. L., González, A., Lozupone, C., Zaneveld, J., Vázquez‐Baeza, Y., Birmingham, A., et al. (2017) · Microbiome · doi:10.1186/s40168-017-0237-y · Cited by 2,231
PARTITIONING DIVERSITY INTO INDEPENDENT ALPHA AND BETA COMPONENTS
Jost, L. (2007) · Ecology · doi:10.1890/06-1736.1 · Cited by 2,230
Anderson, M. J., & Walsh, D. C. I. (2013) · Ecological Monographs · doi:10.1890/12-2010.1 · Cited by 2,050
Best practices for analysing microbiomes
Knight, R., Vrbanac, A., Taylor, B. C., Aksenov, A., Callewaert, C., Debelius, J., Gonzalez, A., Kosciolek, T., McCall, L.-I., McDonald, D., et al. (2018) · Nature Reviews Microbiology · doi:10.1038/s41579-018-0029-9 · Cited by 2,030
Counting the Uncountable: Statistical Approaches to Estimating Microbial Diversity
Hughes, J. B., Hellmann, J. J., Ricketts, T. H., & Bohannan, B. J. M. (2001) · Applied and Environmental Microbiology · doi:10.1128/aem.67.10.4399-4406.2001 · Cited by 1,337
Reproducibility in Science: Improving the Standard for Basic and Preclinical Research
Begley, C. G., & Ioannidis, J. P. A. (2015) · Circulation Research · doi:10.1161/circresaha.114.303819 · Cited by 1,035
Deciphering diversity indices for a better understanding of microbial communities
Kim, B.-R., Shin, J., Guevarra, R. B., Lee, J. H., Kim, D. W., Seol, K.-H., Lee, J.-H., Kim, H. B., & Isaacson, R. E. (2017) · Journal of Microbiology and Biotechnology · doi:10.4014/jmb.1709.09027 · Cited by 1,012
Measuring biological diversity
Magurran, A. E. (2021) · Current Biology · doi:10.1016/j.cub.2021.07.049 · Cited by 831
Borcard, D., Gillet, F., & Legendre, P. (2018) · Use R! · doi:10.1007/978-3-319-71404-2 · Cited by 781
Rarefaction, Alpha Diversity, and Statistics
Willis, A. D. (2019) · Frontiers in Microbiology · doi:10.3389/fmicb.2019.02407 · Cited by 705
Guidelines for the Content of Statistical Analysis Plans in Clinical Trials
Gamble, C., Krishan, A., Stocken, D., Lewis, S., Juszczak, E., Doré, C., Williamson, P. R., Altman, D. G., Montgomery, A., Lim, P., et al. (2017) · JAMA · doi:10.1001/jama.2017.18556 · Cited by 536
Koren, O., Knights, D., Gonzalez, A., Waldron, L., Segata, N., Knight, R., Huttenhower, C., & Ley, R. E. (2013) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002863 · Cited by 508
Power and sample-size estimation for microbiome studies using pairwise distances and PERMANOVA
Kelly, B. J., Gross, R., Bittinger, K. L., Sherrill-Mix, S. A., Lewis, J. D., Collman, R. G., Bushman, F. D., & Li, H. (2015) · Bioinformatics · doi:10.1093/bioinformatics/btv183 · Cited by 467
Robust estimation of microbial diversity in theory and in practice
Haegeman, B., Hamelin, J., Moriarty, J., Neal, P. J., Dushoff, J. G., & Weitz, J. S. (2013) · The ISME Journal · doi:10.1038/ismej.2013.10 · Cited by 446
Data dredging, bias, or confounding: They can all get you into the BMJ and the Friday papers
Smith, G. D., & Ebrahim, S. (2002) · BMJ · doi:10.1136/bmj.325.7378.1437 · Cited by 434
Rethinking microbial diversity analysis in the high throughput sequencing era
Lemos, L. N., Fulthorpe, R. R., Triplett, E. W., & Roesch, L. F. W. (2011) · Journal of Microbiological Methods · doi:10.1016/j.mimet.2011.03.014 · Cited by 373
Hypothesis Testing and Power Calculations for Taxonomic-Based Human Microbiome Data
Rosa, P. S. L., Brooks, J. P., Deych, E., Boone, E. L., Edwards, D., Wang, Q., Sodergren, E. J., Weinstock, G. M., & Shannon, W. D. (2012) · PLoS ONE · doi:10.1371/journal.pone.0052078 · Cited by 325
Ramiro‐Garcia, J., Hermes, G. D. A., Giatsis, C., Sipkema, D., Zoetendal, E. G., Schaap, P. J., & Smidt, H. (2016) · F1000Research · doi:10.12688/f1000research.9227.2 · Cited by 238
III.1 Biodiversity: Concepts, Patterns, and Measurement
Colwell, R. K. (2009) · The Princeton Guide to Ecology · doi:10.1515/9781400833023.257 · Cited by 225
Ellison, A. M. (2010) · Ecology · doi:10.1890/09-1692.1 · Cited by 212
Journal of Biopharmaceutical Statistics
Peace, K. E., & Chow, S. (2005) · Encyclopedia of Biostatistics · doi:10.1002/0470011815.b2a17077 · Cited by 196
Ramiro‐Garcia, J., Hermes, G. D. A., Giatsis, C., Sipkema, D., Zoetendal, E. G., Schaap, P. J., & Smidt, H. (2016) · F1000Research · doi:10.12688/f1000research.9227.1 · Cited by 187
Mulder, I. E., Schmidt, B., Lewis, M. C., Delday, M. I., Stokes, C., Bailey, M., Aminov, R. I., Gill, B. P., Pluske, J. R., Mayer, C., et al. (2011) · PLoS ONE · doi:10.1371/journal.pone.0028279 · Cited by 185
An estimate of the science-wise false discovery rate and application to the top medical literature
Jager, L. R., & Leek, J. T. (2014) · Biostatistics · doi:10.1093/biostatistics/kxt007 · Cited by 179
Saccenti, E., & Timmerman, M. E. (2016) · Journal of Proteome Research · doi:10.1021/acs.jproteome.5b01029 · Cited by 140
Self-citationMicrobial Diversity in Clinical Microbiome Studies: Sample Size and Statistical Power Considerations
Casals‐Pascual, C., González, A., Vázquez‐Baeza, Y., Song, S. J., Jiang, L., & Knight, R. D. (2020) · Gastroenterology · doi:10.1053/j.gastro.2019.11.305 · Cited by 103
Optimal sample size planning for the Wilcoxon‐Mann‐Whitney test
Happ, M., Bathke, A. C., & Brunner, E. (2018) · Statistics in Medicine · doi:10.1002/sim.7983 · Cited by 91
Phylogenetic Diversity Measures and Their Decomposition: A Framework Based on Hill Numbers
Chao, A., Chiu, C., & Jost, L. (2016) · Topics in biodiversity and conservation · doi:10.1007/978-3-319-22461-9_8 · Cited by 84
The Role of the Phylogenetic Diversity Measure, PD, in Bio-Informatics: Getting the Definition Right
Faith, D. P. (2006) · Evolutionary Bioinformatics · doi:10.1177/117693430600200008 · Cited by 76
Pipeline for amplifying and analyzing amplicons of the V1–V3 region of the 16S rRNA gene
Allen, H. K., Bayles, D. O., Looft, T. P., Trachsel, J., Bass, B. E., Alt, D. P., Bearson, S. M. D., Nicholson, T. L., & Casey, T. A. (2016) · BMC Research Notes · doi:10.1186/s13104-016-2172-6 · Cited by 71
Measuring metagenome diversity and similarity with Hill numbers
Ma, Z. S., & Li, L. (2018) · Molecular Ecology Resources · doi:10.1111/1755-0998.12923 · Cited by 70
Rearing environment affects development of the immune system in neonates
Inman, C. F., Haverson, K., Konstantinov, S. R., Jones, P. H., Harris, C., Smidt, H., Miller, B. G., Bailey, M., & Stokes, C. (2010) · Clinical & Experimental Immunology · doi:10.1111/j.1365-2249.2010.04090.x · Cited by 70
Faith, D. P. (2007) · PubMed · Cited by 65
Kers, J. G., Velkers, F. C., Fischer, E. A. J., Hermes, G. D. A., Lamot, D. M., Stegeman, A., & Smidt, H. (2019) · Animal Microbiome · doi:10.1186/s42523-019-0009-z · Cited by 61
Self-citationOn Sample Size of the Kruskal-Wallis Test with Application to a Mouse Peritoneal Cavity Study
Fan, C., Zhang, D., & Zhang, C. (2010) · Biometrics · doi:10.1111/j.1541-0420.2010.01407.x · Cited by 52
Williams, S. C. P. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1324049111 · Cited by 52
Power and sample size calculations for high-throughput sequencing-based experiments
Li, C.-I., Samuels, D. C., Zhao, Y., Shyr, Y., & Guo, Y. P. (2017) · Briefings in Bioinformatics · doi:10.1093/bib/bbx061 · Cited by 46
The importance of the microbiome in epidemiologic research
Hanson, B., & Weinstock, G. M. (2016) · Annals of Epidemiology · doi:10.1016/j.annepidem.2016.03.008 · Cited by 44
Considering Horn’s Parallel Analysis from a Random Matrix Theory Point of View
Saccenti, E., & Timmerman, M. E. (2016) · Psychometrika · doi:10.1007/s11336-016-9515-z · Cited by 28
Self-citationFan, C., & Zhang, D. (2012) · Journal of Biopharmaceutical Statistics · doi:10.1080/10543406.2011.578313 · Cited by 20
Measuring Microbiome Diversity and Similarity with Hill Numbers
Ma, Z. S. (2017) · Elsevier eBooks · doi:10.1016/b978-0-08-102268-9.00008-2 · Cited by 19
P-Hacking: A Wake-Up Call for the Scientific Community
Raj, A. T., Patil, S., Sarode, S. C., & Salameh, Z. A. (2017) · Science and Engineering Ethics · doi:10.1007/s11948-017-9984-1 · Cited by 17
microbiomeDASim: Simulating longitudinal differential abundance for microbiome data
Williams, J., Bravo, H. C., Tom, J. A., & Paulson, J. N. (2020) · F1000Research · doi:10.12688/f1000research.20660.2 · Cited by 14
Power and Sample Size Calculations for Microbiome Data
Xia, Y., Sun, J., & Chen, D. (2018) · ICSA book series in statistics · doi:10.1007/978-981-13-1534-3_5 · Cited by 13
microbiomeDASim: Simulating longitudinal differential abundance for microbiome data
Williams, J., Bravo, H. C., Tom, J. A., & Paulson, J. N. (2019) · F1000Research · doi:10.12688/f1000research.20660.1 · Cited by 10
Flagged first, then by citations
Research works printed but not linked: 6
- 1994, Publication Manual of the American Psychological Association: DAR ALMHRER ELADABE.
- Hoffman, 2015, Baic Biostatistics for Medical and Biomedical Practitioners.
- Kolassa, 2020, MultNonParam-Package R Documentation.
- Lahti, 2017, Tools for Microbiome Analysis in R. Microbiome Package Version 1.15.1.
- 2008, R: A Language and Environment for Statistical Computing.
- Shetty Sudarshan, 2020, Microbial Bioinformatics Introductory Course Material 2018 (Version v3.0).
Cited by
Sorted by citations
Puig‐Castellví, F., Pacheco‐Tapia, R., Deslande, M., Jia, M., Andrikopoulos, P., Chechi, K., Bonnefond, A., Froguel, P., & Dumas, M. (2023) · TrAC Trends in Analytical Chemistry · doi:10.1016/j.trac.2023.117248 · Cited by 85
Thin, Z. S., Chew, J., Ong, T. Y. Y., Ali, R. A. R., & Gew, L. T. (2025) · BMC Gastroenterology · doi:10.1186/s12876-025-04140-2 · Cited by 85
Fulcher, J. A., Li, F., Tobin, N. H., Zabih, S., Elliott, J. E., Clark, J. L., D’Aquila, R. T., Mustanski, B. S., Kipke, M. D., SHOPTAW, S. J., et al. (2022) · EBioMedicine · doi:10.1016/j.ebiom.2022.104286 · Cited by 51
Rajguru, B., Shri, M., & Bhatt, V. D. (2024) · 3 Biotech · doi:10.1007/s13205-024-04065-9 · Cited by 47
Semen microbiota are dramatically altered in men with abnormal sperm parameters
Osadchiy, V., Belarmino, A., Kianian, R., Sigalos, J. T., Ancira, J. S., Kanie, T., Mangum, S. F., Tipton, C. D., Hsieh, T.-C. M., Mills, J. N., et al. (2024) · Scientific Reports · doi:10.1038/s41598-024-51686-4 · Cited by 43
Metadata
- Authors
- Kers, J. G., & Saccenti, E.
- Affiliations
- Wageningen University & Research (NL)
- Corresponding author
- Saccenti, Edoardo
- Venue
- Frontiers in Microbiology
- Publisher
- Frontiers Media SA
- Published
- 2022-03-03
- Volume
- 12
- Pages
- 796025-796025
- ISSN
- 1664-302X (ISSN-L)
- PMC
- PMC8928147
- OpenAlex
- W4214875183
- Cited by
- 359
- References
- 70
- Citations by year
- 2026: 121 · 2025: 121 · 2024: 81 · 2023: 31 · 2022: 5
- FWCI
- 19.51
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Frontiers in Microbiology (publishedVersion, cc-by)
- Research Square (acceptedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Socio-Environmental Systems Modeling (acceptedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Data Archiving and Networked Services (DANS) (submittedVersion, cc-by)
- Funders
- Nederlandse Organisatie voor Wetenschappelijk Onderzoek
- European Commission
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Gut microbiota and health
- Physical Sciences › Environmental Science › Ecology › Microbial Community Ecology and Physiology
- Health Sciences › Dentistry › Periodontics › Oral microbiology and periodontitis research
- Keywords
- microbiome, sample size determination, statistical power, 16S rRNA amplicon sequencing, Bray-Curtis distance, p-hacking, publication bias, statistical reporting, pre-analysis plan
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
