MetaPheno: A critical evaluation of deep learning and machine learning in metagenome-based disease prediction
Open accessThe DOI matches its Crossref record; 24 of the 24 references it cites were found in a registry or an index.
Abstract
The human microbiome plays a number of critical roles, impacting almost every aspect of human health and well-being. Conditions in the microbiome have been linked to a number of significant diseases. Additionally, revolutions in sequencing technology have led to a rapid increase in publicly-available sequencing data. Consequently, there have been growing efforts to predict disease status from metagenomic sequencing data, with a proliferation of new approaches in the last few years. Some of these efforts have explored utilizing a powerful form of machine learning called deep learning, which has been applied successfully in several biological domains. Here, we review some of these methods and the algorithms that they are based on, with a particular focus on deep learning methods. We also perform a deeper analysis of Type 2 Diabetes and obesity datasets that have eluded improved results, using a variety of machine learning and feature extraction methods. We conclude by offering perspectives on study design considerations that may impact results and future directions the field can take to improve results and offer more valuable conclusions. The scripts and extracted features for the analyses conducted in this paper are available via GitHub: https://github.com/nlapier2/metapheno
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing
Benjamini, Y., & Hochberg, Y. (1995) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1995.tb02031.x · Cited by 109,482 · cited 1×
“For each k-mer, we calculate the p-value with the Student’s t-test, followed by the Benjamini-Hochberg procedure [64] to control the false discovery rate from multiple hypothesis testings.”
MethodLeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Cortes, C., & Vapnik, V. (1995) · Machine Learning · doi:10.1007/bf00994018 · Cited by 59,421 · cited 1×
“Two classical machine learning methods are commonly used in metagenome-based disease prediction: Support Vector Machines (SVMs) [19] and Random Forests (RFs) [20].”
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
Predictive functional profiling of microbial communities using 16S rRNA marker gene sequences
Langille, M. G. I., Zaneveld, J., Caporaso, J. G., McDonald, D., Knights, D., Reyes, J. A., Clemente, J. C., Burkepile, D. E., Vega Thurber, R. L., Knight, R., et al. (2013) · Nature Biotechnology · doi:10.1038/nbt.2676 · Cited by 9,116 · cited 2×
“One less explored feature extraction approach is attempting to explicitly infer functional characteristics of the microbiome, using methods such as HUMAnN [41] or PICRUSt [42].”
MethodXGBoost: A Scalable Tree Boosting System
Chen, T., & Guestrin, C. (2016) · arXiv · doi:10.48550/arxiv.1603.02754 · Cited by 7,104
Large-Scale Machine Learning with Stochastic Gradient Descent
Bottou, L. (2010) · Proceedings of COMPSTAT'2010 · doi:10.1007/978-3-7908-2604-3_16 · Cited by 5,795 · cited 1×
“5), optimizer (stochastic gradient descent[65]/adagrad[66]/adam[67]), and learning rate (0.01/0.001) for AutoNN.”
BackgroundInfluentialRichness of human gut microbiome correlates with metabolic markers
Le Chatelier, E., Nielsen, T., Qin, J., Prifti, E., Hildebrand, F., Falony, G., Almeida, M., Arumugam, M., Batto, J.-M., Kennedy, S., et al. (2013) · Nature · doi:10.1038/nature12506 · Cited by 4,662
A fast, lock-free approach for efficient parallel counting of occurrences of k -mers
Marçais, G., & Kingsford, C. (2011) · Bioinformatics · doi:10.1093/bioinformatics/btr011 · Cited by 4,661
(2010) · Journal of Machine Learning Research · doi:10.5555/1756006.1953039 · Cited by 4,051
A survey of cross-validation procedures for model selection
Arlot, S., & Celisse, A. (2010) · Statistics Surveys · doi:10.1214/09-ss054 · Cited by 3,522 · cited 2×
“A common way to split the data into training and testing is k-fold cross-validation (k-fold CV) [15].”
MethodGut metagenome in European women with normal, impaired and diabetic glucose control
Karlsson, F. H., Tremaroli, V., Nookaew, I., Bergström, G., Behre, C. J., Fagerberg, B., Nielsen, J., & Bäckhed, F. (2013) · Nature · doi:10.1038/nature12198 · Cited by 3,115 · cited 1×
“They are: liver cirrhosis [51], colorectal cancer [52], inflammatory bowel diseases (IBD) [53], obesity [54], and type 2 diabetes (T2D) (two distinct studies [4] and [55]).”
BackgroundEnvironment dominates over host genetics in shaping human gut microbiota
Rothschild, D., Weissbrod, O., Barkan, E., Kurilshikov, A., Korem, T., Zeevi, D., Costea, P. I., Godneva, A., Kalka, I. N., Bar, N., et al. (2018) · Nature · doi:10.1038/nature25973 · Cited by 2,859 · cited 3×
“This is because genetic and metagenomic data provide complementary information about the host and environment, respectively [74].”
BackgroundInfluentialBias in error estimation when using cross-validation for model selection
Varma, S., & Simon, R. (2006) · BMC Bioinformatics · doi:10.1186/1471-2105-7-91 · Cited by 2,485 · cited 1×
“As for which cross-validation method is ideal for this setting, there is no obvious best choice, but LOOCV has been shown to have low bias and strong generalization to new data [15, 70, 71], with the main drawback being computational cost [15].”
MethodMetaPhlAn2 for enhanced metagenomic taxonomic profiling
Truong, D. T., Franzosa, E. A., Tickle, T. L., Scholz, M., Weingart, G., Pasolli, E., Tett, A., Huttenhower, C., & Segata, N. (2015) · Nature Methods · doi:10.1038/nmeth.3589 · Cited by 2,185
Alterations of the human gut microbiome in liver cirrhosis
Qin, N., Yang, F., Li, A., Prifti, E., Chen, Y., Shao, L., Guo, J., Le Chatelier, E., Yao, J., Wu, L., et al. (2014) · Nature · doi:10.1038/nature13568 · Cited by 2,177
Similarity network fusion for aggregating data types on a genomic scale
Wang, B., Mezlini, A. M., Demir, F., Fiume, M., Tu, Z., Brudno, M., Haibe-Kains, B., & Goldenberg, A. (2014) · Nature Methods · doi:10.1038/nmeth.2810 · Cited by 2,036
A universal SNP and small-indel variant caller using deep neural networks
Poplin, R., Chang, P.-C., Alexander, D., Schwartz, S., Colthurst, T., Ku, A., Newburger, D., Dijamco, J., Nguyen, N., Afshar, P. T., et al. (2018) · Nature Biotechnology · doi:10.1038/nbt.4235 · Cited by 1,840
Potential of fecal microbiota for early-stage detection of colorectal cancer
Zeller, G., Tap, J., Voigt, A. Y., Sunagawa, S., Kultima, J. R., Costea, P. I., Amiot, A., Böhm, J., Brunetti, F., Habermann, N., et al. (2014) · Molecular Systems Biology · doi:10.15252/msb.20145645 · Cited by 1,375
Optimal classifier for imbalanced data using Matthews Correlation Coefficient metric
Boughorbel, S., Jarray, F., & El-Anbari, M. (2017) · PLOS ONE · doi:10.1371/journal.pone.0177678 · Cited by 1,222 · cited 1×
“Some papers also report the Matthews Correlation Coefficient (MCC) which is robust to case-control class imbalances [72].”
BackgroundMachine Learning Meta-analysis of Large Metagenomic Datasets: Tools and Biological Insights
Pasolli, E., Truong, D. T., Malik, F., Waldron, L., & Segata, N. (2016) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1004977 · Cited by 659 · cited 9×
“In the context of metagenome-based disease prediction, these features are the microbes or the functional elements that contribute most to the disease prediction, enhancing the interpretability of the model [5, 9, 7, 21].”
BackgroundMethodResultInfluentialAccessible, curated metagenomic data through ExperimentHub
Pasolli, E., Schiffer, L., Manghi, P., Renson, A., Obenchain, V., Truong, D. T., Beghini, F., Malik, F., Ramos, M., Dowd, J. B., et al. (2017) · Nature Methods · doi:10.1038/nmeth.4468 · Cited by 514
Cited by
Marcos-Zambrano, L. J., Karaduzovic-Hadziabdic, K., Loncar Turukalo, T., Przymus, P., Trajkovik, V., Aasmets, O., Berland, M., Gruca, A., Hasic, J., Hron, K., et al. (2021) · Frontiers in Microbiology · doi:10.3389/fmicb.2021.634511 · Cited by 416
Machine learning and deep learning applications in microbiome research
Hernández Medina, R., Kutuzova, S., Nielsen, K. N., Johansen, J., Hansen, L. H., Nielsen, M., & Rasmussen, S. (2022) · ISME Communications · doi:10.1038/s43705-022-00182-9 · Cited by 341
Gut microbiome, big data and machine learning to promote precision medicine for cancer
Cammarota, G., Ianiro, G., Ahern, A., Carbone, C., Temko, A., Claesson, M. J., Gasbarrini, A., & Tortora, G. (2020) · Nature Reviews Gastroenterology & Hepatology · doi:10.1038/s41575-020-0327-3 · Cited by 324
A survey of k-mer methods and applications in bioinformatics
Moeckel, C., Mareboina, M., Konnaris, M. A., Chan, C. S. Y., Mouratidis, I., Montgomery, A., Chantzi, N., Pavlopoulos, G. A., & Georgakopoulos-Soares, I. (2024) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2024.05.025 · Cited by 131
“K-mers can serve as valuable clinical biomarkers for detecting pathogens [17 – 19], antimicrobial resistance [17, 20, 21], and human diseases [16, 22 – 24].”
BackgroundScientific novelty beyond the experiment
Hallsworth, J. E., Udaondo, Z., Pedrós‐Alió, C., Höfer, J., Benison, K. C., Lloyd, K. G., Cordero, R. J. B., de Campos, C. B. L., Yakimov, M. M., & Amils, R. (2023) · Microbial Biotechnology · doi:10.1111/1751-7915.14222 · Cited by 62
Metadata
- Authors
- LaPierre, N., Ju, C. J.-T., Zhou, G., & Wang, W.
- Venue
- Methods
- Publisher
- Elsevier BV
- Published
- 2019-08
- Volume
- 166
- Pages
- 74-82
- ISSN
- 1046-2023 (ISSN-L)
- OpenAlex
- W2921789835
- Cited by
- 114
- References
- 24
- FWCI
- 4.30
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.