Unraveling the functional dark matter through global metagenomics
Open accessThe DOI matches its Crossref record; 15 of the 15 references it cites were found in a registry or an index.
Abstract
Metagenomes encode an enormous diversity of proteins, reflecting a multiplicity of functions and activities 1,2. Exploration of this vast sequence space has been limited to a comparative analysis against reference microbial genomes and protein families derived from those genomes. Here, to examine the scale of yet untapped functional diversity beyond what is currently possible through the lens of reference genomes, we develop a computational approach to generate reference-free protein families from the sequence space in metagenomes. We analyse 26,931 metagenomes and identify 1.17 billion protein sequences longer than 35 amino acids with no similarity to any sequences from 102,491 reference genomes or the Pfam database 3. Using massively parallel graph-based clustering, we group these proteins into 106,198 novel sequence clusters with more than 100 members, doubling the number of protein families obtained from the reference genomes clustered using the same approach. We annotate these families on the basis of their taxonomic, habitat, geographical and gene neighbourhood distributions and, where sufficient sequence diversity is available, predict protein three-dimensional models, revealing novel structures. Overall, our results uncover an enormously diverse functional space, highlighting the importance of further exploring the microbial functional dark matter.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Highly accurate protein structure prediction with AlphaFold
Jumper, J., Evans, R., Pritzel, A., Green, T., Figurnov, M., Ronneberger, O., Tunyasuvunakool, K., Bates, R., Žídek, A., Potapenko, A., et al. (2021) · Nature · doi:10.1038/s41586-021-03819-2 · Cited by 43,764
Gephi: An Open Source Software for Exploring and Manipulating Networks
Bastian, M., Heymann, S., & Jacomy, M. (2009) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v3i1.13937 · Cited by 11,675 · cited 1×
“Distribution analysis of the protein clusters across ecosystems and NCBI taxa was performed by creating and visualizing networks with Gephi 57 using the Yifan Hu algorithm 58 to generate the layout.”
MethodTensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G. S., Davis, A., Dean, J., Devin, M., et al. (2016) · arXiv · doi:10.48550/arxiv.1603.04467 · Cited by 9,408
KEGG as a reference resource for gene and protein annotation
Kanehisa, M., Sato, Y., Kawashima, M., Furumichi, M., & Tanabe, M. (2016) · Nucleic Acids Research · doi:10.1093/nar/gkv1070 · Cited by 7,387
Accelerated Profile HMM Searches
Eddy, S. R. (2011) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002195 · Cited by 6,975 · cited 1×
“Pfam hits (v.31) were detected with the use of the hmmsearch tool (HMMER v.3.1 package) 34 using the default trusted cut-off.”
MethodInfluentialBiopython: freely available python tools for computational molecular biology and bioinformatics
Cock, P. J. A., Antao, T., Chang, J. T., Chapman, B. A., Cox, C. J., Dalke, A., Friedberg, I., Hamelryck, T., Kauff, F., Wilczynski, B., et al. (2009) · Bioinformatics · doi:10.1093/bioinformatics/btp163 · Cited by 6,080 · cited 1×
“The resulting MSAs were then filtered to produce seed (non-redundant) alignments using a script written in Python and the ProDy/Evol and Biopython modules 43 (90% sequence identity, 75% alignment coverage) (Supplementary Methods).”
MethodMMseqs2 enables sensitive protein sequence searching for the analysis of massive data sets
Steinegger, M., & Söding, J. (2017) · Nature Biotechnology · doi:10.1038/nbt.3988 · Cited by 4,840
TM-align: a protein structure alignment algorithm based on the TM-score
Zhang, Y. (2005) · Nucleic Acids Research · doi:10.1093/nar/gki524 · Cited by 3,774
Announcing the worldwide Protein Data Bank
Berman, H., Henrick, K., & Nakamura, H. (2003) · Nature Structural & Molecular Biology · doi:10.1038/nsb1203-980 · Cited by 3,093
Insights into the phylogeny and coding potential of microbial dark matter
Rinke, C., Schwientek, P., Sczyrba, A., Ivanova, N. N., Anderson, I. J., Cheng, J.-F., Darling, A., Malfatti, S., Swan, B. K., Gies, E. A., et al. (2013) · Nature · doi:10.1038/nature12352 · Cited by 2,394
Self-citation
Research works printed but not linked: 51
- New, F. N. & Brito, I. L. What is metagenomics teaching us, and what is missed? Annu. Rev. Microbiol. 74, 117–135 (2020). · DOI
- Mistry, J. et al. Pfam: the protein families database in 2021. Nucleic Acids Res. 49, D412–D419 (2021). · DOI · Paper page
- Meyer, F. et al. MG-RAST version 4—lessons learned from a decade of low-budget ultra-high-throughput metagenome analysis. Brief. Bioinform. 20, 1151–1159 (2019). · DOI
- Ayling, M., Clark, M. D. & Leggett, R. M. New approaches for metagenome assembly with short reads. Brief. Bioinform. 21, 584–594 (2020). · DOI · Paper page
- Chen, I.-M. A. et al. The IMG/M data management and analysis system v.6.0: new tools and advanced capabilities. Nucleic Acids Res. 49, D751–D763 (2021). · DOI · Paper page
- Galperin, M. Y. et al. COG database update: focus on microbial diversity, model organisms, and widespread pathogens. Nucleic Acids Res. 49, D274–D281 (2021). · DOI · Paper page
- Vanni, C. et al. AGNOSTOS-DB: a resource to unlock the uncharted regions of the coding sequence space. Preprint at bioRxiv https://doi.org/10.1101/2021.06.07.447314 (2021). · DOI
- Rodríguez del Río, Á. et al. Functional and evolutionary significance of unknown genes from uncultivated taxa. Preprint at bioRxiv https://doi.org/10.1101/2022.01.26.477801 (2022). · DOI
- Modha, S., Robertson, D. L., Hughes, J. & Orton, R. J. Quantifying and cataloguing unknown sequences within human microbiomes. mSystems https://doi.org/10.1128/msystems.01468-21 (2022). · DOI
- Azad, A., Pavlopoulos, G. A., Ouzounis, C. A., Kyrpides, N. C. & Buluç, A. HipMCL: a high-performance parallel implementation of the Markov clustering algorithm for large-scale networks. Nucleic Acids Res. 46, e33 (2018). · DOI
- Enright, A. J., Van Dongen, S. & Ouzounis, C. A. An efficient algorithm for large-scale detection of protein families. Nucleic Acids Res. 30, 1575–1584 (2002). · DOI · Paper page
- Mukherjee, S. et al. Genomes OnLine Database (GOLD) v.8: overview and updates. Nucleic Acids Res. 49, D723–D733 (2021). · DOI · Paper page
- Ivanova, N. et al. A call for standardized classification of metagenome projects. Environ. Microbiol. 12, 1803–1805 (2010). · DOI
- Coelho, L. P. et al. Towards the biogeography of prokaryotic genes. Nature 601, 252–256 (2022). · DOI · Paper page
- Clum, A. et al. DOE JGI Metagenome Workflow. mSystems 6, e00804-20 (2021). · DOI
- Baltoumas, F. A. et al. Exploring microbial functional biodiversity at the protein family level-From metagenomic sequence reads to annotated protein clusters. Front. Bioinform. 3, 1157956 (2023). · DOI
- Nayfach, S. et al. A genomic catalog of Earth’s microbiomes. Nat. Biotechnol. 39, 499–509 (2021). · DOI · Paper page
- Overbeek, R., Fonstein, M., D’Souza, M., Pusch, G. D. & Maltsev, N. The use of gene clusters to infer functional coupling. Proc. Natl Acad. Sci. USA 96, 2896–2901 (1999). · DOI · Paper page
- Hou, Q. et al. Using metagenomic data to boost protein structure prediction and discovery. Comput. Struct. Biotechnol. J. 20, 434–442 (2022). · DOI
- Yang, J. et al. Improved protein structure prediction using predicted interresidue orientations. Proc. Natl Acad. Sci. USA 117, 1496–1503 (2020). · DOI · Paper page
- Chandonia, J.-M. et al. SCOPe: improvements to the structural classification of proteins—extended database to facilitate variant interpretation and machine learning. Nucleic Acids Res. 50, D553–D559 (2022). · DOI · Paper page
- Mukherjee, S. et al. 1,003 reference genomes of bacterial and archaeal isolates expand coverage of the tree of life. Nat. Biotechnol. 35, 676–683 (2017). · DOI · Paper page
- Roberts, R. J. et al. COMBREX: a project to accelerate the functional annotation of prokaryotic genomes. Nucleic Acids Res. 39, D11–D14 (2011). · DOI
- Koonin, E. V. & Makarova, K. S. Evolutionary plasticity and functional versatility of CRISPR systems. PLoS Biol. 20, e3001481 (2022). · DOI
- Qin, J. et al. A human gut microbial gene catalogue established by metagenomic sequencing. Nature 464, 59–65 (2010). · DOI · Paper page
- Wyman, S. K., Avila-Herrera, A., Nayfach, S. & Pollard, K. S. A most wanted list of conserved microbial protein families with no known domains. PLoS ONE 13, e0205749 (2018). · DOI
- Frith, M. C. A new repeat-masking method enables specific detection of homologous sequences. Nucleic Acids Res. 39, e23 (2011). · DOI · Paper page
- Kiełbasa, S. M., Wan, R., Sato, K., Horton, P. & Frith, M. C. Adaptive seeds tame genomic sequence comparison. Genome Res. 21, 487–493 (2011). · DOI · Paper page
- Li, W. & Godzik, A. Cd-hit: a fast program for clustering and comparing large sets of protein or nucleotide sequences. Bioinformatics 22, 1658–1659 (2006). · DOI · Paper page
- Edgar, R. C. Search and clustering orders of magnitude faster than BLAST. Bioinformatics 26, 2460–2461 (2010). · DOI · Paper page
- Hauser, M., Mayer, C. E. & Söding, J. kClust: fast and sensitive clustering of large protein sequence databases. BMC Bioinform. 14, 248 (2013). · DOI
- Blondel, V. D., Guillaume, J.-L., Lambiotte, R. & Lefebvre, E. Fast unfolding of communities in large networks. J. Stat. Mech. 2008, P10008 (2008). · DOI · Paper page
- Jiang, P. & Singh, M. SPICi: a fast clustering algorithm for large biological networks. Bioinformatics 26, 1105–1111 (2010). · DOI · Paper page
- Sievers, F. & Higgins, D. G. Clustal Omega for making accurate alignments of many protein sequences. Protein Sci. 27, 135–145 (2018). · DOI · Paper page
- Altschul, S. F., Gish, W., Miller, W., Myers, E. W. & Lipman, D. J. Basic local alignment search tool. J. Mol. Biol. 215, 403–410 (1990). · doi:10.1016/s0022-2836(05)80360-2 · Paper page
- Wickham, H. ggplot2: Elegant Graphics for Data Analysis (Springer-Verlag, 2016). · DOI · Paper page
- Eberhardt, R. Y. et al. AntiFam: a tool to help identify spurious ORFs in protein annotation. Database 2012, bas003 (2012). · DOI · Paper page
- Buttigieg, P. L. et al. The environment ontology in 2016: bridging domains with increased scope, semantic density, and interoperation. J. Biomed. Semantics 7, 57 (2016). · DOI · Paper page
- Thompson, L. R. et al. A communal catalogue reveals Earth’s multiscale microbial diversity. Nature 551, 457–463 (2017). · DOI · Paper page
- Roux, S. et al. IMG/VR v3: an integrated ecological and evolutionary framework for interrogating genomes of uncultivated viruses. Nucleic Acids Res. 49, D764–D775 (2021). · DOI · Paper page
- Ren, J. et al. Identifying viruses from metagenomic data using deep learning. Quant. Biol. 8, 64–77 (2020). · DOI · Paper page
- Pronk, L. J. U. & Medema, M. H. Whokaryote: distinguishing eukaryotic and prokaryotic contigs in metagenomes based on gene structure. Microb. Genomics 8, mgen000823 (2022). · DOI
- West, P. T., Probst, A. J., Grigoriev, I. V., Thomas, B. C. & Banfield, J. F. Genome-reconstruction for eukaryotes from complex natural microbial communities. Genome Res. 28, 569–580 (2018). · DOI · Paper page
- Mirdita, M., Steinegger, M., Breitwieser, F., Söding, J. & Levy Karin, E. Fast and sensitive taxonomic assignment to metagenomic contigs. Bioinformatics 37, 3029–3031 (2021). · DOI · Paper page
- Hu, Y. in Combinatorial Scientific Computing (eds Naumann, U. & Schenk, O.) 525–549 (CRC Press, 2010).
- Flajolet, P. & Noy, M. in Formal Power Series and Algebraic Combinatorics (eds Krob, D. et al.) 191–201 (Springer, 2000); https://doi.org/10.1007/978-3-662-04166-6_17. · doi:10.1007/978-3-662-04166-6_17
- Sievert, C. Interactive Web-Based Data Visualization with R, plotly, and shiny (Chapman and Hall/CRC, 2020). · DOI · Paper page
- Karatzas, E. et al. The network makeup artist (NORMA-2.0): distinguishing annotated groups in a network using innovative layout strategies. Bioinform. Adv. 2, vbac036 (2022). · doi:10.1093/bioadv/vbac036
- Pedregosa, F. et al. Scikit-learn: machine learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011).
- Andreeva, A., Kulesha, E., Gough, J. & Murzin, A. G. The SCOP database in 2020: expanded classification of representative family and superfamily domains of known protein structures. Nucleic Acids Res. 48, D376–D382 (2020). · DOI · Paper page
- Mukherjee, S. & Zhang, Y. MM-align: a quick algorithm for aligning multiple-chain protein complex structures using iterative dynamic programming. Nucleic Acids Res. 37, e83 (2009). · DOI · Paper page
Cited by
AlphaFold predictions of fold-switched conformations are driven by structure memorization
Chakravarty, D., Schafer, J. W., Chen, E. A., Thole, J. F., Ronish, L. A., Lee, M., & Porter, L. L. (2024) · Nature Communications · doi:10.1038/s41467-024-51801-z · Cited by 159
DSSP 4: FAIR annotation of protein secondary structure
Hekkelman, M. L., Salmoral, D. Á., Perrakis, A., & Joosten, R. P. (2025) · Protein Science · doi:10.1002/pro.70208 · Cited by 97
Protein structure-informed bacteriophage genome annotation with Phold
Bouras, G., Grigson, S. R., Mirdita, M., Heinzinger, M., Papudeshi, B., Mallawaarachchi, V., Green, R., Kim, R. S., Mihalia, V., Psaltis, A. J., et al. (2026) · Nucleic Acids Research · doi:10.1093/nar/gkaf1448 · Cited by 71
AI in microbiome‐related healthcare
Probul, N., Huang, Z., Saak, C. C., Baumbach, J., & List, M. (2024) · Microbial Biotechnology · doi:10.1111/1751-7915.70027 · Cited by 36
The role of metabolism in shaping enzyme structures over 400 million years
(2025) · doi:10.1038/s41586-025-09205-6 · Cited by 28
Metadata
- Authors
- Pavlopoulos, G. A., Baltoumas, F. A., Liu, S., Selvitopi, O., Camargo, A. P., Nayfach, S., Azad, A., Roux, S., Call, L., Ivanova, N. N., Chen, I. M., Paez-Espino, D., Karatzas, E., Novel Metagenome Protein Families Consortium, Acinas, S. G., Ahlgren, N., Attwood, G., Baldrian, P., Berry, T., Bhatnagar, J. M., Bhaya, D., Bidle, K. D., Blanchard, J. L., Boyd, E. S., Bowen, J. L., Bowman, J., Brawley, S. H., Brodie, E. L., Brune, A., Bryant, D. A., Buchan, A., Cadillo-Quiroz, H., Campbell, B. J., Cavicchioli, R., Chuckran, P. F., Coleman, M., Crowe, S., Colman, D. R., Currie, C. R., Dangl, J., Delherbe, N., Denef, V. J., Dijkstra, P., Distel, D. D., Eloe-Fadrosh, E., Fisher, K., Francis, C., Garoutte, A., Gaudin, A., Gerwick, L., Godoy-Vitorino, F., Guerra, P., Guo, J., Habteselassie, M. Y., Hallam, S. J., Hatzenpichler, R., Hentschel, U., Hess, M., Hirsch, A. M., Hug, L. A., Hultman, J., Hunt, D. E., Huntemann, M., Inskeep, W. P., James, T. Y., Jansson, J., Johnston, E. R., Kalyuzhnaya, M., Kelly, C. N., Kelly, R. M., Klassen, J. L., Nüsslein, K., Kostka, J. E., Lindow, S., Lilleskov, E., Lynes, M., Mackelprang, R., Martin, F. M., Mason, O. U., McKay, R. M., McMahon, K., Mead, D. A., Medina, M., Meredith, L. K., Mock, T., Mohn, W. W., Moran, M. A., Murray, A., Neufeld, J. D., Neumann, R., Norton, J. M., Partida-Martinez, L. P., Pietrasiak, N., Pelletier, D., Reddy, T. B. K., Reese, B. K., Reichart, N. J., Reiss, R., Saito, M. A., Schachtman, D. P., Seshadri, R., Shade, A., Sherman, D., Simister, R., Simon, H., Stegen, J., Stepanauskas, R., Sullivan, M., Sumner, D. Y., Teeling, H., Thamatrakoln, K., Treseder, K., Tringe, S., Vaishampayan, P., Valentine, D. L., Waldo, N. B., Waldrop, M. P., Walsh, D. A., Ward, D. M., Wilkins, M., Whitman, T., Woolet, J., Woyke, T., Iliopoulos, I., Konstantinidis, K., Tiedje, J. M., Pett-Ridge, J., Baker, D., Visel, A., Ouzounis, C. A., Ovchinnikov, S., Buluç, A., & Kyrpides, N. C.
- Venue
- Nature
- Published
- 2023-10-11
- Volume
- 622
- Issue
- 7983
- Pages
- 594-602
- ISSN
- 0028-0836 (ISSN-L)
- 1476-4687
- OpenAlex
- W4387541035
- Cited by
- 206
- References
- 15
- FWCI
- 15.77
- Type
- journal article
- Language
- en
- Open access
- hybrid
- Licence
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Unraveling the functional dark matter through global metagenomics. https://app.cochat.ai/papers/10.1038/s41586-023-06583-7
- The paper
- Pavlopoulos, G. A., Baltoumas, F. A., Liu, S., Selvitopi, O., Camargo, A. P., Nayfach, S., Azad, A., Roux, S., Call, L., Ivanova, N. N., Chen, I. M., Paez-Espino, D., Karatzas, E., Novel Metagenome Protein Families Consortium, Acinas, S. G., Ahlgren, N., Attwood, G., Baldrian, P., Berry, T., . . . Kyrpides, N. C. (2023). Unraveling the functional dark matter through global metagenomics. Nature. https://doi.org/10.1038/s41586-023-06583-7
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.