The DOI matches its Crossref record; 57 of the 57 references it cites were found in a registry or an index.
Robust estimation of microbial diversity in theory and in practice
Open accessAbstract
Quantifying diversity is of central importance for the study of structure, function and evolution of microbial communities. The estimation of microbial diversity has received renewed attention with the advent of large-scale metagenomic studies. Here, we consider what the diversity observed in a sample tells us about the diversity of the community being sampled. First, we argue that one cannot reliably estimate the absolute and relative number of microbial species present in a community without making unsupported assumptions about species abundance distributions. The reason for this is that sample data do not contain information about the number of rare species in the tail of species abundance distributions. We illustrate the difficulty in comparing species richness estimates by applying Chao’s estimator of species richness to a set of in silico communities: they are ranked incorrectly in the presence of large numbers of rare species. Next, we extend our analysis to a general family of diversity metrics (‘Hill diversities’), and construct lower and upper estimates of diversity values consistent with the sample data. The theory generalizes Chao’s estimator, which we retrieve as the lower estimate of species richness. We show that Shannon and Simpson diversity can be robustly estimated for the in silico communities. We analyze nine metagenomic data sets from a wide range of environments, and show that our findings are relevant for empirically-sampled communities. Hence, we recommend the use of Shannon and Simpson diversity rather than species richness in efforts to quantify and compare microbial diversity.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A Mathematical Theory of Communication
Shannon, C. E. (1948) · Bell System Technical Journal · doi:10.1002/j.1538-7305.1948.tb01338.x · Cited by 88,486
A Mathematical Theory of Communication
Shannon, C. E. (1948) · Bell System Technical Journal · doi:10.1002/j.1538-7305.1948.tb00917.x · Cited by 48,949
SIMPSON, E. H. (1949) · Nature · doi:10.1038/163688a0 · Cited by 14,164
Possible generalization of Boltzmann-Gibbs statistics
Tsallis, C. (1988) · Journal of Statistical Physics · doi:10.1007/bf01016429 · Cited by 9,589
Measuring biological diversity
(2004) · Choice Reviews Online · doi:10.5860/choice.42-1547 · Cited by 6,621
Gotelli, N. J., & Colwell, R. K. (2001) · Ecology Letters · doi:10.1046/j.1461-0248.2001.00230.x · Cited by 6,420
Diversity and Evenness: A Unifying Notation and Its Consequences
Hill, M. O. (1973) · Ecology · doi:10.2307/1934352 · Cited by 6,344
Nonparametric estimation of the number of classes in a population
Chao, A. (1984) · Scandinavian Journal of Statistics · Cited by 5,758
Prokaryotes: The unseen majority
Whitman, W. B., Coleman, D. C., & Wiebe, W. J. (1998) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.95.12.6578 · Cited by 4,871 · cited 2×
“Moreover, the number of organisms in microbial communities is typically many orders of magnitude larger than the number of organisms in plant or animal communities (Whitman et al., 1998).”
Jost, L. (2006) · Oikos · doi:10.1111/j.2006.0030-1299.14714.x · Cited by 4,669
Biodiversity and ecosystem functioning: Current knowledge and future challenges
Loreau, M., Naeem, S., Inchausti, P., Bengtsson, J., Grime, J. P., Hector, A., Hooper, D. U., Huston, M. A., Raffaelli, D., Schmid, B., et al. (2001) · Science · doi:10.1126/science.1064088 · Cited by 4,649
Measuring Biological Diversity
REID, J. R. W. (2005) · Austral Ecology · doi:10.1111/j.1442-9993.2005.01502.x · Cited by 4,442
Microbial diversity in the deep sea and the underexplored “rare biosphere”
Sogin, M. L., Morrison, H. G., Huber, J. A., Welch, D. M., Huse, S. M., Neal, P. R., Arrieta, J. M., & Herndl, G. J. (2006) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0605127103 · Cited by 3,701
Self-citationTHE POPULATION FREQUENCIES OF SPECIES AND THE ESTIMATION OF POPULATION PARAMETERS
GOOD, I. J. (1953) · Biometrika · doi:10.1093/biomet/40.3-4.237 · Cited by 3,437
How Many Species Are There on Earth and in the Ocean?
Mora, C., Tittensor, D. P., Adl, S., Simpson, A. G. B., & Worm, B. (2011) · PLoS Biology · doi:10.1371/journal.pbio.1001127 · Cited by 3,045 · cited 3×
“, 2004), or to assume a particular distribution for the community diversity over taxonomic levels (May, 1988; Mora et al., 2011).”
Tettelin, H., Masignani, V., Cieslewicz, M. J., Donati, C., Medini, D., Ward, N. L., Angiuoli, S. V., Crabtree, J., Jones, A. L., Durkin, A. S., et al. (2005) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0506758102 · Cited by 2,631
Schloss, P. D., & Handelsman, J. (2005) · Applied and Environmental Microbiology · doi:10.1128/aem.71.3.1501-1506.2005 · Cited by 2,501
Wilson, E. O. (1992) · DigitalGeorgetown (Georgetown University Library) · Cited by 2,468
Rusch, D. B., Halpern, A. L., Sutton, G., Heidelberg, K. B., Williamson, S., Yooseph, S., Wu, D., Eisen, J. A., Hoffman, J. M., Remington, K., et al. (2007) · PLoS Biology · doi:10.1371/journal.pbio.0050077 · Cited by 2,060
Pyrosequencing enumerates and contrasts soil microbial diversity
Roesch, L. F. W., Fulthorpe, R. R., Riva, A., Casella, G., Hadwin, A. K. M., Kent, A. D., Daroub, S. H., Camargo, F. A. O., Farmerie, W. G., & Triplett, E. W. (2007) · The ISME Journal · doi:10.1038/ismej.2007.53 · Cited by 1,852
Global patterns in bacterial diversity
Lozupone, C. A., & Knight, R. (2007) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0611525104 · Cited by 1,825
INTERPOLATING, EXTRAPOLATING, AND COMPARING INCIDENCE-BASED SPECIES ACCUMULATION CURVES
Colwell, R. K., Mao, C. X., & Chang, J. (2004) · Ecology · doi:10.1890/03-0557 · Cited by 1,788
Biological Diversity: Frontiers in Measurement and Assessment
Magurran, A. E., & McGill, B. J. (2011) · DigitalCommons (California Polytechnic State University) · Cited by 1,771
Stability and Diversity of Ecosystems
Ives, A. R., & Carpenter, S. R. (2007) · Science · doi:10.1126/science.1133258 · Cited by 1,584
Report of the ad hoc committee for the re-evaluation of the species definition in bacteriology.
Stackebrandt, E., Frederiksen, W., Garrity, G. M., Grimont, P. A. D., Kämpfer, P., Maiden, M. C. J., Nesme, X., Rosselló-Mora, R., Swings, J., Trüper, H. G., et al. (2002) · International Journal of Systematic and Evolutionary Microbiology · doi:10.1099/00207713-52-3-1043 · Cited by 1,540
Counting the Uncountable: Statistical Approaches to Estimating Microbial Diversity
Hughes, J. B., Hellmann, J. J., Ricketts, T. H., & Bohannan, B. J. M. (2001) · Applied and Environmental Microbiology · doi:10.1128/aem.67.10.4399-4406.2001 · Cited by 1,296
Computational Improvements Reveal Great Bacterial Diversity and High Metal Toxicity in Soil
Gans, J., Wolinsky, M., & Dunbar, J. (2005) · Science · doi:10.1126/science.1112665 · Cited by 1,207
How Many Species Are There on Earth?
May, R. M. (1988) · Science · doi:10.1126/science.241.4872.1441 · Cited by 1,187
Estimating prokaryotic diversity and its limits
Curtis, T. P., Sloan, W. T., & Scannell, J. W. (2002) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.142680199 · Cited by 1,153
Report of the ad hoc committee for the re-evaluation of the species definition in bacteriology
Stackebrandt, E. (2002) · INTERNATIONAL JOURNAL OF SYSTEMATIC AND EVOLUTIONARY MICROBIOLOGY · doi:10.1099/ijs.0.02360-0 · Cited by 916
Microbial Population Structures in the Deep Marine Biosphere
Huber, J. A., Mark Welch, D. B., Morrison, H. G., Huse, S. M., Neal, P. R., Butterfield, D. A., & Sogin, M. L. (2007) · Science · doi:10.1126/science.1146689 · Cited by 887
Self-citationEstimating the Number of Species: A Review
Bunge, J., & Fitzpatrick, M. (1993) · Journal of the American Statistical Association · doi:10.1080/01621459.1993.10594330 · Cited by 763
Marine microbial diversity: can it be determined?
Pedrós-Alió, C. (2006) · Trends in Microbiology · doi:10.1016/j.tim.2006.04.007 · Cited by 731
Microbial Biogeography: From Taxonomy to Traits
Green, J. L., Bohannan, B. J. M., & Whitaker, R. J. (2008) · Science · doi:10.1126/science.1153475 · Cited by 594
Bacterial diversity in aquatic and other environments: what 16S rDNA libraries can tell us
Kemp, P. F., & Aller, J. Y. (2004) · FEMS Microbiology Ecology · doi:10.1016/s0168-6496(03)00257-5 · Cited by 558
Sufficient sampling for asymptotic minimum species richness estimators
Chao, A., Colwell, R. K., Lin, C.-W., & Gotelli, N. J. (2009) · Ecology · doi:10.1890/07-2147.1 · Cited by 551
Statistical analysis of denaturing gel electrophoresis (DGE) fingerprinting patterns
Fromin, N., Hamelin, J., Tarnawski, S., Roesti, D., Jourdain‐Miserez, K., Forestier, N., Teyssier‐Cuvelle, S., Gillet, F., Aragno, M., & Rossi, P. (2002) · Environmental Microbiology · doi:10.1046/j.1462-2920.2002.00358.x · Cited by 551
Self-citationToward a census of bacteria in soil
Schloss, P. D., & Handelsman, J. (2006) · PLoS Computational Biology · doi:10.1371/journal.pcbi.0020092 · Cited by 429
ESTIMATING SPECIES RICHNESS: SENSITIVITY TO SAMPLE COVERAGE AND INSENSITIVITY TO SPATIAL PATTERNS
Brose, U., Martinez, N. D., & Williams, R. J. (2003) · Ecology · doi:10.1890/02-0558 · Cited by 383
From Populations to Ecosystems: Theoretical Foundations for a New Ecological Synthesis (MPB-46)
Loreau, M. (2010) · doi:10.1515/9781400834167 · Cited by 366
PHYLOGENETIC CLUSTERING AND OVERDISPERSION IN BACTERIAL COMMUNITIES
Horner-Devine, M. C., & Bohannan, B. J. M. (2006) · Ecology · doi:10.1890/0012-9658(2006)87[100:pcaoib]2.0.co;2 · Cited by 336
Comparison of phenotypic diversity and DNA heterogeneity in a population of soil bacteria
Torsvik, V, Salte, K., Sørheim, R., & Goksøyr, J. (1990) · Applied and Environmental Microbiology · doi:10.1128/aem.56.3.776-781.1990 · Cited by 324
The tragedy of the uncommon: understanding limitations in the analysis of microbial diversity
Bent, S. J., & Forney, L. J. (2008) · The ISME Journal · doi:10.1038/ismej.2008.44 · Cited by 314
Santa Rosalia revisited: Why are there so many species of bacteria?
Dykhuizen, D. E. (1998) · Antonie van Leeuwenhoek · doi:10.1023/a:1000665216662 · Cited by 291
From populations to ecosystems: theoretical foundations for a new ecological synthesis
(2011) · Choice Reviews Online · doi:10.5860/choice.48-3260 · Cited by 251
Predicting microbial species richness
Hong, S.-H., Bunge, J., Jeon, S.-O., & Epstein, S. S. (2006) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0507245102 · Cited by 214
The rational exploration of microbial diversity
Quince, C., Curtis, T. P., & Sloan, W. T. (2008) · The ISME Journal · doi:10.1038/ismej.2008.69 · Cited by 212
It's all relative: ranking the diversity of aquatic bacterial communities
Shaw, A. K., Halpern, A. L., Beeson, K., Tran, B., Venter, J. C., & Martiny, J. B. H. (2008) · Environmental Microbiology · doi:10.1111/j.1462-2920.2008.01626.x · Cited by 176
PREDICTING THE NUMBER OF NEW SPECIES IN FURTHER TAXONOMIC SAMPLING
Shen, T.-J., Chao, A., & Lin, C.-F. (2003) · Ecology · doi:10.1890/0012-9658(2003)084[0798:ptnons]2.0.co;2 · Cited by 171
When species accumulation curves intersect: implications for ranking diversity using small samples
Lande, R., DeVries, P. J., & Walla, T. R. (2000) · Oikos · doi:10.1034/j.1600-0706.2000.890320.x · Cited by 159
Dipping into the Rare Biosphere
Pedrós‐Alió, C. (2007) · Science · doi:10.1126/science.1135933 · Cited by 148
ESTIMATION OF SPECIES RICHNESS: MIXTURE MODELS, THE ROLE OF RARE SPECIES, AND INFERENTIAL CHALLENGES
Mao, C. X., & Colwell, R. K. (2005) · Ecology · doi:10.1890/04-1078 · Cited by 141
New approaches to analyzing microbial biodiversity data
Bohannan, B. J. M., & Hughes, J. (2003) · Current Opinion in Microbiology · doi:10.1016/s1369-5274(03)00055-9 · Cited by 139
Multivariate Cutoff Level Analysis (MultiCoLA) of large community data sets
Gobet, A., Quince, C., & Ramette, A. (2010) · Nucleic Acids Research · doi:10.1093/nar/gkq545 · Cited by 127
Genomic fluidity: an integrative view of gene diversity within microbial populations
Kislyuk, A. O., Haegeman, B., Bergman, N. H., & Weitz, J. S. (2011) · BMC Genomics · doi:10.1186/1471-2164-12-32 · Cited by 120
Self-citationAccessing uncultivated microorganisms: from the environment to organisms and genomes and back
(2008) · Choice Reviews Online · doi:10.5860/choice.46-2057 · Cited by 97
On Turing's formula for word probabilities
Nadas, A. J. (1985) · IEEE Transactions on Acoustics Speech and Signal Processing · doi:10.1109/tassp.1985.1164728 · Cited by 85
Cited by
Microbial diversity drives multifunctionality in terrestrial ecosystems
Delgado-Baquerizo, M., Maestre, F. T., Reich, P. B., Jeffries, T. C., Gaitan, J. J., Encinar, D., Berdugo, M., Campbell, C. D., & Singh, B. K. (2016) · Nature Communications · doi:10.1038/ncomms10541 · Cited by 2,624
Leff, J. W., Jones, S. E., Prober, S. M., Barberán, A., Borer, E. T., Firn, J. L., Harpole, W. S., Hobbie, S. E., Hofmockel, K. S., Knops, J. M. H., et al. (2015) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1508382112 · Cited by 1,452
Ecology and exploration of the rare biosphere
Lynch, M. D. J., & Neufeld, J. D. (2015) · Nature Reviews Microbiology · doi:10.1038/nrmicro3400 · Cited by 1,268
Increasing aridity reduces soil microbial diversity and abundance in global drylands
Maestre, F. T., Delgado-Baquerizo, M., Jeffries, T. C., Eldridge, D. J., Ochoa, V., Gozalo, B., Quero, J. L., García-Gómez, M., Gallardo, A., Ulrich, W., et al. (2015) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1516684112 · Cited by 1,152
Deciphering diversity indices for a better understanding of microbial communities
Kim, B.-R., Shin, J., Guevarra, R. B., Lee, J. H., Kim, D. W., Seol, K.-H., Lee, J.-H., Kim, H. B., & Isaacson, R. E. (2017) · Journal of Microbiology and Biotechnology · doi:10.4014/jmb.1709.09027 · Cited by 1,012
Metadata
- Authors
- Haegeman, B., Hamelin, J., Moriarty, J., Neal, P., Dushoff, J., & Weitz, J. S.
- Venue
- The ISME Journal
- Publisher
- Oxford University Press (OUP)
- Published
- 2013-02-14
- Volume
- 7
- Issue
- 6
- Pages
- 1092-1101
- ISSN
- 1751-7362 (ISSN-L)
- 1751-7370
- OpenAlex
- W1990746682
- Cited by
- 431
- References
- 57
- FWCI
- 30.22
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.