Learning sparse log-ratios for high-throughput sequencing data
Open accessPublished version of a preprint
Abstract
Motivation: The automatic discovery of sparse biomarkers that are associated with an outcome of interest is a central goal of bioinformatics. In the context of high-throughput sequencing (HTS) data, and compositional data (CoDa) more generally, an important class of biomarkers are the log-ratios between the input variables. However, identifying predictive log-ratio biomarkers from HTS data is a combinatorial optimization problem, which is computationally challenging. Existing methods are slow to run and scale poorly with the dimension of the input, which has limited their application to low- and moderate-dimensional metagenomic datasets. Results: Building on recent advances from the field of deep learning, we present CoDaCoRe, a novel learning algorithm that identifies sparse, interpretable and predictive log-ratio biomarkers. Our algorithm exploits a continuous relaxation to approximate the underlying combinatorial optimization problem. This relaxation can then be optimized efficiently using the modern ML toolbox, in particular, gradient descent. As a result, CoDaCoRe runs several orders of magnitude faster than competing methods, all while achieving state-of-the-art performance in terms of predictive accuracy and sparsity. We verify the outperformance of CoDaCoRe across a wide range of microbiome, metabolite and microRNA benchmark datasets, as well as a particularly high-dimensional dataset that is outright computationally intractable for existing sparse log-ratio selection methods. Availability and implementation: The CoDaCoRe package is available at https://github.com/egr95/R-codacore. Code and instructions for reproducing our results are available at https://github.com/cunningham-lab/codacore. Supplementary information: Supplementary data are available at Bioinformatics online.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The normalization constant removed from the original balance definition is unnecessary.
“Note that we have removed the normalization constant from the original definition as it is later shown by ref. 74 to be unnecessary.”
From Longitudinal gut microbiome changes in immune checkpoint blockade-treated advanced melanoma (2024) · cites this paper as [74]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
The Statistical Analysis of Compositional Data
Aitchison, J. (1982) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.1111/j.2517-6161.1982.tb01195.x · Cited by 3,457 · see citing works
Microbiome Datasets Are Compositional: And This Is Not Optional
Gloor, G. B., Macklaim, J. M., Pawlowsky-Glahn, V., & Egozcue, J. J. (2017) · Frontiers in Microbiology · doi:10.3389/fmicb.2017.02224 · Cited by 3,044 · see citing works
Liquid biopsies come of age: Towards implementation of circulating tumour DNA
Wan, J. C. M., Massie, C., Garcia-Corbacho, J., Mouliere, F., Brenton, J. D., Caldas, C., Pacey, S., Baird, R., & Rosenfeld, N. (2017) · Nature Reviews Cancer · doi:10.1038/nrc.2017.7 · Cited by 2,809 · see citing works
The Firmicutes/Bacteroidetes Ratio: A Relevant Marker of Gut Dysbiosis in Obese Patients?
Magne, F., Gotteland, M., Gauthier, L., Zazueta, A., Pesoa, S., Navarrete, P., & Balamurugan, R. (2020) · Nutrients · doi:10.3390/nu12051474 · Cited by 2,335 · see citing works
European Union Regulations on Algorithmic Decision Making and a “Right to Explanation”
Goodman, B., & Flaxman, S. (2017) · AI Magazine · doi:10.1609/aimag.v38i3.2741 · Cited by 2,131 · see citing works
Isometric Logratio Transformations for Compositional Data Analysis
Egozcue, J. J., Pawlowsky‐Glahn, V., Mateu‐Figueras, G., & Barceló-Vidal, C. (2003) · Mathematical Geology · doi:10.1023/a:1023818214614 · Cited by 1,956 · see citing works
VII. Mathematical contributions to the theory of evolution.—III. Regression, heredity, and panmixia
Pearson, K. (1896) · Philosophical Transactions of the Royal Society of London Series A Containing Papers of a Mathematical or Physical Character · doi:10.1098/rsta.1896.0007 · Cited by 1,810 · see citing works
Clinical Applications of Circulating Tumor Cells and Circulating Tumor DNA as Liquid Biopsy
Alix-Panabières, C., & Pantel, K. (2016) · Cancer Discovery · doi:10.1158/2159-8290.cd-15-1483 · Cited by 1,394 · see citing works
Fernandes, A. D., Reid, J. N., Macklaim, J. M., McMurrough, T. A., Edgell, D. R., & Gloor, G. B. (2014) · Microbiome · doi:10.1186/2049-2618-2-15 · Cited by 1,329 · see citing works
Dillies, M., Rau, A., Aubert, J., Hennequet‐Antier, C., Jeanmougin, M., Servant, N., Keime, C., Marot, G., Castel, D., Estellé, J., et al. (2012) · Briefings in Bioinformatics · doi:10.1093/bib/bbs046 · Cited by 1,281 · see citing works
Best, M. G., Sol, N., Kooi, I. E., Tannous, J. M., Westerman, B. A., Rustenburg, F., Schellen, P., Verschueren, H., Post, E., Köster, J., et al. (2015) · Cancer Cell · doi:10.1016/j.ccell.2015.09.018 · Cited by 922 · see citing works
ANOVA-Like Differential Expression (ALDEx) Analysis for Mixed Population RNA-Seq
Fernandes, A. D., Macklaim, J. M., Linn, T. G., Reid, G., & Gloor, G. B. (2013) · PLoS ONE · doi:10.1371/journal.pone.0067019 · Cited by 877 · see citing works
Compositional Data Analysis: Theory and Applications
(2011) · Wiley eBooks · doi:10.1002/9781119976462 · Cited by 704 · cited 4× · see citing works
“The choice of the log-ratio transform offers the necessary property of scale invariance, but in the CoDa literature it holds primacy for a variety of other technical reasons, including subcompositional coherence (Aitchison, 1982; Pawlowsky-Glahn and Buccianti, 2011; Egozcue and Pawlowsky-Glahn, 2019).”
BackgroundMethodInfluentialGroups of Parts and Their Balances in Compositional Data Analysis
Egozcue, J. J., & Pawlowsky‐Glahn, V. (2005) · Mathematical Geology · doi:10.1007/s11004-005-7381-9 · Cited by 642 · see citing works
Establishing microbial composition measurement standards with reference frames
Morton, J. T., Marotz, C., Washburne, A., Silverman, J., Zaramela, L. S., Edlund, A., Zengler, K., & Knight, R. (2019) · Nature Communications · doi:10.1038/s41467-019-10656-5 · Cited by 613 · see citing works
Profile of the gut microbiota of adults with obesity: a systematic review
Crovesy, L., Masterson, D., & Rosado, E. L. (2020) · European Journal of Clinical Nutrition · doi:10.1038/s41430-020-0607-6 · Cited by 610 · see citing works
Univariate statistical analysis of environmental (compositional) data: Problems and possibilities
Filzmoser, P., Hron, K., & Reimann, C. (2009) · The Science of The Total Environment · doi:10.1016/j.scitotenv.2009.08.008 · Cited by 448 · see citing works
Compositional data and their analysis: an introduction
Pawlowsky‐Glahn, V., & Egozcue, J. J. (2006) · Geological Society London Special Publications · doi:10.1144/gsl.sp.2006.264.01.01 · Cited by 447 · see citing works
Compositional analysis: a valid approach to analyze microbiome high-throughput sequencing data
Gloor, G. B., & Reid, G. (2016) · Canadian Journal of Microbiology · doi:10.1139/cjm-2015-0821 · Cited by 375 · see citing works
Understanding sequencing data as compositions: an outlook and review
Quinn, T. P., Erb, I., Richardson, M. F., & Crowley, T. M. (2018) · Bioinformatics · doi:10.1093/bioinformatics/bty175 · Cited by 349 · see citing works
Self-citationA field guide for the compositional analysis of any-omics data
Quinn, T. P., Erb, I., Gloor, G. B., Notredame, C., Richardson, M. F., & Crowley, T. M. (2019) · GigaScience · doi:10.1093/gigascience/giz107 · Cited by 345 · see citing works
Self-citationGut microbiome, big data and machine learning to promote precision medicine for cancer
Cammarota, G., Ianiro, G., Ahern, A., Carbone, C., Temko, A., Claesson, M. J., Gasbarrini, A., & Tortora, G. (2020) · Nature Reviews Gastroenterology & Hepatology · doi:10.1038/s41575-020-0327-3 · Cited by 324 · see citing works
It's all relative: analyzing microbiome data as compositions
Gloor, G. B., Wu, J. R., Pawlowsky-Glahn, V., & Egozcue, J. J. (2016) · Annals of Epidemiology · doi:10.1016/j.annepidem.2016.03.003 · Cited by 323 · see citing works
A phylogenetic transform enhances analysis of compositional microbiota data
Silverman, J. D., Washburne, A., Mukherjee, S., & David, L. A. (2017) · eLife · doi:10.7554/elife.21887 · Cited by 310 · see citing works
Evidence for greater production of colonic short-chain fatty acids in overweight than lean humans
Rahat-Rozenbloom, S., Fernandes, J., Gloor, G. B., & Wolever, T. M. S. (2014) · International Journal of Obesity · doi:10.1038/ijo.2014.46 · Cited by 290 · see citing works
Balances: a New Perspective for Microbiome Analysis
Rivera-Pinto, J., Egozcue, J. J., Pawlowsky‐Glahn, V., Paredes, R., Noguera-Julián, M., & Calle, M. L. (2018) · mSystems · doi:10.1128/msystems.00053-18 · Cited by 290 · see citing works
Proportionality: A Valid Alternative to Correlation for Relative Data
Lovell, D. R., Pawlowsky‐Glahn, V., Egozcue, J. J., Marguerat, S., & Bähler, J. (2015) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1004075 · Cited by 277 · see citing works
Microbiome, Metagenomics, and High-Dimensional Compositional Data Analysis
Li, H. (2015) · Annual Review of Statistics and Its Application · doi:10.1146/annurev-statistics-010814-020351 · Cited by 276 · see citing works
Statistical Analysis of Metagenomics Data
Calle, M. L. (2019) · Genomics & Informatics · doi:10.5808/gi.2019.17.1.e6 · Cited by 262 · see citing works
Balance Trees Reveal Microbial Niche Differentiation
Morton, J. T., Sanders, J., Quinn, R. A., McDonald, D., Gonzalez, A., Vázquez-Baeza, Y., Navas-Molina, J. A., Song, S. J., Metcalf, J. L., Hyde, E. R., et al. (2017) · mSystems · doi:10.1128/msystems.00162-16 · Cited by 254 · see citing works
Quinn, T. P., Richardson, M. F., Lovell, D. R., & Crowley, T. M. (2017) · Scientific Reports · doi:10.1038/s41598-017-16520-0 · Cited by 211 · see citing works
Self-citationWashburne, A., Silverman, J. D., Leff, J. W., Bennett, D. J., Darcy, J. L., Mukherjee, S., Fierer, N., & David, L. A. (2017) · PeerJ · doi:10.7717/peerj.2969 · Cited by 105 · see citing works
What can go wrong at the data normalization step for identification of biomarkers?
Peter Filzmoser, A., & Walczak, B. (2014) · Journal of Chromatography A · doi:10.1016/j.chroma.2014.08.050 · Cited by 98 · see citing works
Variable Selection in Compositional Data Analysis Using Pairwise Logratios
Greenacre, M. J. (2018) · Mathematical Geosciences · doi:10.1007/s11004-018-9754-x · Cited by 94 · see citing works
Variable selection in microbiome compositional data analysis
Susín, A., Wang, Y., Cao, K. L., & Calle, M. L. (2020) · NAR Genomics and Bioinformatics · doi:10.1093/nargab/lqaa029 · Cited by 85 · see citing works
Generalized Linear Models With Linear Constraints for Microbiome Compositional Data
Lu, J., Shi, P., & Li, H. (2018) · Biometrics · doi:10.1111/biom.12956 · Cited by 77 · see citing works
Vangay, P., Hillmann, B., & Knights, D. (2019) · GigaScience · doi:10.1093/gigascience/giz042 · Cited by 72 · see citing works
Identification of tumor-educated platelet biomarkers of non-small-cell lung cancer
Sheng, M., Dong, Z., & Xie, Y. (2018) · OncoTargets and Therapy · doi:10.2147/ott.s177384 · Cited by 68 · see citing works
Identifying and analyzing different cancer subtypes using RNA-seq data of blood platelets
Zhang, Y.-H., Huang, T., Chen, L., Xu, Y., Hu, Y., Hu, L.-D., Cai, Y., & Kong, X. (2017) · Oncotarget · doi:10.18632/oncotarget.20903 · Cited by 54 · see citing works
Interpretable and accurate prediction models for metagenomics data
Prifti, E., Chevaleyre, Y., Hanczar, B., Belda, E., Danchin, A., Clément, K., & Zucker, J. (2020) · GigaScience · doi:10.1093/gigascience/giaa010 · Cited by 46 · see citing works
Greenacre, M. J. (2019) · Applied Computing and Geosciences · doi:10.1016/j.acags.2019.100017 · Cited by 38 · see citing works
Log-Ratio Lasso: Scalable, Sparse Estimation for Log-Ratio Models
Bates, S., & Tibshirani, R. (2018) · Biometrics · doi:10.1111/biom.12995 · Cited by 37 · see citing works
A comparison of isometric and amalgamation logratio balances in compositional data analysis
Greenacre, M. J., Grunsky, E. C., & Bacon‐Shone, J. (2020) · Computers & Geosciences · doi:10.1016/j.cageo.2020.104621 · Cited by 36 · see citing works
Quinn, T. P., & Erb, I. (2020) · mSystems · doi:10.1128/msystems.00230-19 · Cited by 32 · see citing works
Self-citationAmalgams: data-driven amalgamation for the dimensionality reduction of compositional data
Quinn, T. P., & Erb, I. (2020) · NAR Genomics and Bioinformatics · doi:10.1093/nargab/lqaa076 · Cited by 28 · see citing works
Self-citationComments on: Compositional data: the sample space and its structure
Greenacre, M. J. (2019) · Test · doi:10.1007/s11749-019-00673-3 · Cited by 8 · see citing works
A Critique of Differential Abundance Analysis, and Advocacy for an AlternativePreprint
Quinn, T. P., Gordon-Rodríguez, E., & Erb, I. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.07266 · Cited by 1 · see citing works
Self-citation
Flagged first, then by citations
Research works printed but not linked: 8
- Friedman J. et al (2001) The Elements of Statistical Learning, Vol. 1. Springer Series in Statistics, New York.
- He H., Ma Y. (2013) Imbalanced Learning: Foundations, Algorithms, and Applications, Wiley-IEEE Press, New York.
- Jang E. et al (2017) Categorical reparameterization with gumbel-softmax. In: 5th International Conference on Learning Representations, (ICLR) 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings.
- Linderman S. et al (2018) Reparameterizing the Birkhoff polytope for variational permutation inference. In: International Conference on Artificial Intelligence and Statistics, (AISTATS) 2018, 9-11 April 2018, Playa Blanca, Lanzarote, Canary Islands, Spain.
- Maddison C.J. et al (2017) The concrete distribution: a continuous relaxation of discrete random variables. In: 5th International Conference on Learning Representations, (ICLR) 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings.
- Mena G. et al (2018) Learning latent permutations with Gumbel-Sinkhorn networks. In: 6th International Conference on Learning Representations, (ICLR) 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings.
- Potapczynski A. et al (2020) Invertible gaussian reparameterization: revisiting the gumbel-softmax. Advances in Neural Information Processing Systems, 33.
- Quinn T. et al (2020) Deepcoda: personalized interpretability for compositional health data. In: Proceedings of the 37th International Conference on Machine Learning, (ICML) 2020, 13-18 July 2020, Virtual Event.
Cited by
Sorted by citations
Longitudinal gut microbiome changes in immune checkpoint blockade-treated advanced melanoma
Björk, J. R., Bolte, L. A., Maltez Thomas, A., Lee, K. A., Rossi, N., Wind, T. T., Smit, L. M., Armanini, F., Asnicar, F., Blanco-Miguez, A., et al. (2024) · Nature Medicine · doi:10.1038/s41591-024-02803-3 · Cited by 156 · see citing works
- Authors
- Gordon-Rodriguez, E., Quinn, T. P., & Cunningham, J. P.
- Affiliations
- Department of Statistics, Columbia University , New York, NY 10025, USA
- Deakin University (AU)
- Columbia University (US)
- Venue
- Bioinformatics
- Publisher
- Oxford University Press (OUP)
- Published
- 2021-09-08
- Volume
- 38
- Issue
- 1
- Pages
- 157-163
- ISSN
- 1367-4803 (ISSN-L)
- 1367-4811
- PMC
- PMC8696089
- OpenAlex
- W3197666941
- Cited by
- 37
- References
- 47
- Citations by year
- 2026: 4 · 2025: 4 · 2024: 11 · 2023: 9 · 2022: 6 · 2021: 3
- FWCI
- 2.35
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Bioinformatics (publishedVersion, cc-by)
- bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by-nc)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Figshare (submittedVersion, cc-by)
- Funders
- Simons Foundation (542963)
- Sloan Foundation
- McKnight Endowment Fund
- NSF (DBI-1707398)
- Gatsby Charitable Foundation
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Gene expression and cancer classification
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Genomics and Phylogenetic Studies
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Inference
- Keywords
- compositional data, high-throughput sequencing data, metagenomics, microbiome, continuous relaxation, combinatorial optimization, microRNA, metabolomics
- MeSH terms
- Algorithms, Software, Metagenomics, High-Throughput Nucleotide Sequencing, Microbiota
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
