The DOI matches its Crossref record; 64 of the 64 references it cites were found in a registry or an index.
Benchmarking Parametric and Machine Learning Models for Genomic Prediction of Complex Traits
Open accessAbstract
The usefulness of genomic prediction in crop and livestock breeding programs has prompted efforts to develop new and improved genomic prediction algorithms, such as artificial neural networks and gradient tree boosting. However, the performance of these algorithms has not been compared in a systematic manner using a wide range of datasets and models. Using data of 18 traits across six plant species with different marker densities and training population sizes, we compared the performance of six linear and six non-linear algorithms. First, we found that hyperparameter selection was necessary for all non-linear algorithms and that feature selection prior to model training was critical for artificial neural networks when the markers greatly outnumbered the number of training lines. Across all species and trait combinations, no one algorithm performed best, however predictions based on a combination of results from multiple algorithms (i.e., ensemble predictions) performed consistently well. While linear and non-linear algorithms performed best for a similar number of traits, the performance of non-linear algorithms vary more between traits. Although artificial neural networks did not perform best for any trait, we identified strategies (i.e., feature selection, seeded starting weights) that boosted their performance to near the level of other algorithms. Our results highlight the importance of algorithm selection for the prediction of trait values.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538
Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing
Benjamini, Y., & Hochberg, Y. (1995) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1995.tb02031.x · Cited by 109,482
Scikit-learn: Machine Learning in Python
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Müller, A., Nothman, J., Louppe, G., et al. (2012) · arXiv · doi:10.48550/arxiv.1201.0490 · Cited by 71,016
Greedy function approximation: A gradient boosting machine
Friedman, J. H. (2001) · The Annals of Statistics · doi:10.1214/aos/1013203451 · Cited by 30,731
Regularization and Variable Selection Via the Elastic Net
Zou, H., & Hastie, T. (2005) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2005.00503.x · Cited by 20,957
Ensemble methods in machine learning
Dietterich, T. G. (2000) · Lecture Notes in Computer Science · doi:10.1007/3-540-45014-9_1 · Cited by 8,408
Prediction of Total Genetic Value Using Genome-Wide Dense Marker Maps
Meuwissen, T. H. E., Hayes, B. J., & Goddard, M. E. (2001) · Genetics · doi:10.1093/genetics/157.4.1819 · Cited by 7,917
The Pfam protein families database: towards a more sustainable future
Finn, R. D., Coggill, P., Eberhardt, R. Y., Eddy, S. R., Mistry, J., Mitchell, A. L., Potter, S. C., Punta, M., Qureshi, M., Sangrador-Vegas, A., et al. (2016) · Nucleic Acids Research · doi:10.1093/nar/gkv1344 · Cited by 5,975
NWChem: A comprehensive and scalable open-source solution for large scale molecular simulations
Valiev, M., Bylaska, E. J., Govind, N., Kowalski, K., Straatsma, T. P., Van Dam, H. J. J., Wang, D., Nieplocha, J., Apra, E., Windus, T. L., et al. (2010) · Computer Physics Communications · doi:10.1016/j.cpc.2010.04.018 · Cited by 5,779
Choosing Multiple Parameters for Support Vector Machines
Chapelle, O., Vapnik, V., Bousquet, O., & Mukherjee, S. (2002) · Machine Learning · doi:10.1023/a:1012450327387 · Cited by 2,206
Ridge Regression and Other Kernels for Genomic Selection with R Package rrBLUP
Endelman, J. B. (2011) · The Plant Genome · doi:10.3835/plantgenome2011.08.0024 · Cited by 2,094
Genomic Selection for Crop Improvement
Heffner, E. L., Sorrells, M. E., & Jannink, J. (2009) · Crop Science · doi:10.2135/cropsci2008.08.0512 · Cited by 1,652
Prechelt, L. (1998) · Lecture Notes in Computer Science · doi:10.1007/3-540-49430-8_3 · Cited by 1,649
Genome-Wide Regression and Prediction with the BGLR Statistical Package
Pérez, P., & de los Campos, G. (2014) · Genetics · doi:10.1534/genetics.114.164442 · Cited by 1,594
Self-citationDeep learning for computational biology
Angermueller, C., Pärnamaa, T., Parts, L., & Stegle, O. (2016) · Molecular Systems Biology · doi:10.15252/msb.20156651 · Cited by 1,473
Extension of the bayesian alphabet for genomic selection
Habier, D., Fernando, R. L., Kizilkaya, K., & Garrick, D. J. (2011) · BMC Bioinformatics · doi:10.1186/1471-2105-12-186 · Cited by 1,224
Genomic selection: prediction of accuracy and maximisation of long term response
Goddard, M. (2009) · Genetica · doi:10.1007/s10709-008-9308-0 · Cited by 1,089
Data and Theory Point to Mainly Additive Genetic Variance for Complex Traits
Hill, W. G., Goddard, M. E., & Visscher, P. M. (2008) · PLoS Genetics · doi:10.1371/journal.pgen.1000008 · Cited by 1,039
Whole-Genome Regression and Prediction Methods Applied to Plant and Animal Breeding
de los Campos, G., Hickey, J. M., Pong-Wong, R., Daetwyler, H. D., & Calus, M. P. L. (2013) · Genetics · doi:10.1534/genetics.112.143313 · Cited by 893
Self-citationSMURF: Genomic mapping of fungal secondary metabolite clusters
Khaldi, N., Seifuddin, F. T., Turner, G., Haft, D., Nierman, W. C., Wolfe, K. H., & Fedorova, N. D. (2010) · Fungal Genetics and Biology · doi:10.1016/j.fgb.2010.06.003 · Cited by 771
Genomic Selection in Plant Breeding: A Comparison of Models
Heslot, N., Yang, H., Sorrells, M. E., & Jannink, J. (2012) · Crop Science · doi:10.2135/cropsci2011.06.0297 · Cited by 698
Genomic selection: genome-wide prediction in plant improvement
Desta, Z. A., & Ortiz, R. (2014) · Trends in Plant Science · doi:10.1016/j.tplants.2014.05.006 · Cited by 688
Predicting Quantitative Traits With Regression Models for Dense Molecular Markers and Pedigree
de los Campos, G., Naya, H., Gianola, D., Crossa, J., Legarra, A., Manfredi, E., Weigel, K., & Cotes, J. M. (2009) · Genetics · doi:10.1534/genetics.109.101501 · Cited by 599
Self-citationSpindel, J., Begum, H., Akdemir, D., Virk, P., Collard, B., Redoña, E., Atlin, G., Jannink, J.-L., & McCouch, S. R. (2015) · PLOS Genetics · doi:10.1371/journal.pgen.1004982 · Cited by 590
Insights into the Maize Pan-Genome and Pan-Transcriptome
Hirsch, C. N., Foerster, J. M., Johnson, J. M., Sekhon, R. S., Muttoni, G., Vaillancourt, B., Peñagaricano, F., Lindquist, E., Pedraza, M. A., Barry, K., et al. (2014) · The Plant Cell · doi:10.1105/tpc.113.119982 · Cited by 572
Genetic architecture of complex traits in plants
HOLLAND, J. (2007) · Current Opinion in Plant Biology · doi:10.1016/j.pbi.2007.01.003 · Cited by 505
Genomic Selection in Plant Breeding
Lorenz, A. J., Chao, S., Asoro, F. G., Heffner, E. L., Hayashi, T., Iwata, H., Smith, K. P., Sorrells, M. E., & Jannink, J.-L. (2011) · Advances in Agronomy · doi:10.1016/b978-0-12-385531-2.00002-5 · Cited by 477
Genomic-Assisted Prediction of Genetic Value With Semiparametric Procedures
Gianola, D., Fernando, R. L., & Stella, A. (2006) · Genetics · doi:10.1534/genetics.105.049510 · Cited by 450
Genomic Selection in Plant Breeding
Newell, M. A., & Jannink, J.-L. (2014) · Methods in Molecular Biology · doi:10.1007/978-1-4939-0446-4_10 · Cited by 393
Ribaut, J.-M., & Ragot, M. (2006) · Journal of Experimental Botany · doi:10.1093/jxb/erl214 · Cited by 365
DE LOS CAMPOS, G., GIANOLA, D., ROSA, G. J. M., WEIGEL, K. A., & CROSSA, J. (2010) · Genetics Research · doi:10.1017/s0016672310000285 · Cited by 361
Webb, S. (2018) · Nature · doi:10.1038/d41586-018-02174-z · Cited by 353
A deep convolutional neural network approach for predicting phenotypes from genotypes
Ma, W., Qiu, Z., Song, J., Li, J., Cheng, Q., Zhai, J., & Ma, C. (2018) · Planta · doi:10.1007/s00425-018-2976-9 · Cited by 341
Application of high-dimensional feature selection: evaluation for genomic prediction in man
Bermingham, M. L., Pong-Wong, R., Spiliopoulou, A., Hayward, C., Rudan, I., Campbell, H., Wright, A. F., Wilson, J. F., Agakov, F., Navarro, P., et al. (2015) · Scientific Reports · doi:10.1038/srep10312 · Cited by 322
Accuracy of breeding values of 'unrelated' individuals predicted by dense SNP genotyping
Meuwissen, T. H. (2009) · Genetics Selection Evolution · doi:10.1186/1297-9686-41-35 · Cited by 305
Handling limited datasets with neural networks in medical applications: A small-data approach
Shaikhina, T., & Khovanova, N. A. (2017) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2016.12.003 · Cited by 296
Can deep learning improve genomic prediction of complex human traits?
Bellot, P., de los Campos, G., & Pérez-Enciso, M. (2018) · Genetics · doi:10.1534/genetics.118.301298 · Cited by 225
Self-citationNorman, A., Taylor, J., Edwards, J., & Kuchel, H. (2018) · G3 Genes|Genomes|Genetics · doi:10.1534/g3.118.200311 · Cited by 214
Genome-enabled prediction of genetic values using radial basis function neural networks
González-Camacho, J. M., de los Campos, G., Pérez, P., Gianola, D., Cairns, J. E., Mahuku, G., Babu, R., & Crossa, J. (2012) · Theoretical and Applied Genetics · doi:10.1007/s00122-012-1868-9 · Cited by 210
Self-citationMoser, G., Tier, B., Crump, R. E., Khatkar, M. S., & Raadsma, H. W. (2009) · Genetics Selection Evolution · doi:10.1186/1297-9686-41-56 · Cited by 193
LASSO with cross-validation for genomic selection
USAI, M. G., GODDARD, M. E., & HAYES, B. J. (2009) · Genetics Research · doi:10.1017/s0016672309990334 · Cited by 184
Fernandes, S. B., Dias, K. O. G., Ferreira, D. F., & Brown, P. J. (2018) · Theoretical and Applied Genetics · doi:10.1007/s00122-017-3033-y · Cited by 180
Applications of Machine Learning Methods to Genomic Selection in Breeding Wheat for Rust Resistance
González‐Camacho, J. M., Ornella, L., Pérez‐Rodríguez, P., Gianola, D., Dreisigacker, S., & Crossa, J. (2018) · The Plant Genome · doi:10.3835/plantgenome2017.11.0104 · Cited by 168
Does genomic selection have a future in plant breeding?
Jonas, E., & de Koning, D. (2013) · Trends in biotechnology · doi:10.1016/j.tibtech.2013.06.003 · Cited by 167
Genome-wide prediction of discrete traits using bayesian regressions and machine learning
González-Recio, O., & Forni, S. (2011) · Genetics Selection Evolution · doi:10.1186/1297-9686-43-7 · Cited by 166
Genome-Enabled Prediction Models for Yield Related Traits in Chickpea
Roorkiwal, M., Rathore, A., Das, R. R., Singh, M. K., Jain, A., Samineni, S., Gaur, P. M., Bharadwaj, C., Tripathi, S., Li, Y., et al. (2016) · Frontiers in Plant Science · doi:10.3389/fpls.2016.01666 · Cited by 153
Application of support vector regression to genome-assisted prediction of quantitative traits
Long, N., Gianola, D., Rosa, G. J. M., & WEIGEL, K. A. (2011) · Theoretical and Applied Genetics · doi:10.1007/s00122-011-1648-y · Cited by 129
Beaulieu, J., Doerksen, T., MacKay, J. J., Rainville, A., & Bousquet, J. (2014) · BMC Genomics · doi:10.1186/1471-2164-15-1048 · Cited by 126
Genomic selection of agronomic traits in hybrid rice using an NCII population
XU, Y. C., Wang, X., Ding, X., Zheng, X., Yang, Z., Chenwu, X., & Hu, Z. (2018) · Rice · doi:10.1186/s12284-018-0223-4 · Cited by 119
A comparison of statistical methods for genomic selection in a mice population
Neves, H. H. D. R., Carvalheiro, R., & de Queiroz, S. A. (2012) · BMC Genetics · doi:10.1186/1471-2156-13-100 · Cited by 108
A bootstrap evaluation of the effect of data splitting on financial time series
LeBaron, B. D., & Weigend, A. S. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/72.655043 · Cited by 103
Assessing Predictive Properties of Genome-Wide Selection in Soybeans
Xavier, A., Muir, W. M., & Rainey, K. M. (2016) · G3 Genes Genomes Genetics · doi:10.1534/g3.116.032268 · Cited by 100
Riedelsheimer, C., Technow, F., & Melchinger, A. E. (2012) · BMC Genomics · doi:10.1186/1471-2164-13-452 · Cited by 99
Ehret, A., Hochstuhl, D., Gianola, D., & Thaller, G. E. (2015) · Genetics Selection Evolution · doi:10.1186/s12711-015-0097-5 · Cited by 99
A Ranking Approach to Genomic Selection
Blondel, M., Onogi, A., Iwata, H., & Ueda, N. (2015) · PLoS ONE · doi:10.1371/journal.pone.0128570 · Cited by 96
Genetic Diversity of a Maize Association Population with Restricted Phenology
Hansey, C. N., Johnson, J. M., Sekhon, R. S., Kaeppler, S. M., & de León, N. (2011) · Crop Science · doi:10.2135/cropsci2010.03.0178 · Cited by 94
Prediction of body mass index in mice using dense molecular markers and a regularized neural network
Okut, H., Gianola, D., Rosa, G. J. M., & WEIGEL, K. A. (2011) · Genetics Research · doi:10.1017/s0016672310000662 · Cited by 88
Genome-enabled prediction using probabilistic neural network classifiers
González‐Camacho, J. M., Crossa, J. L., Pérez‐Rodríguez, P., Ornella, L., & Gianola, D. (2016) · BMC Genomics · doi:10.1186/s12864-016-2553-1 · Cited by 86
Vazquez, A. I., Rosa, G. J. M., WEIGEL, K. A., Campos, G. D. L., Daniel Gianola, B., & Allison, D. B. (2010) · Journal of Dairy Science · doi:10.3168/jds.2010-3335 · Cited by 84
Accelerating the Switchgrass (Panicum virgatum L.) Breeding Cycle Using Genomic Selection Approaches
Lipka, A. E., Lü, F., Cherney, J. H., Buckler, E. S., Casler, M. D., & Costich, D. E. (2014) · PLoS ONE · doi:10.1371/journal.pone.0112227 · Cited by 79
González-Recio, Ó., Jiménez-Montero, J. A., & Alenda, R. (2012) · Journal of Dairy Science · doi:10.3168/jds.2012-5630 · Cited by 61
Ramstein, G. P., Evans, J., Kaeppler, S. M., Mitchell, R. B., Vogel, K. P., Buell, C. R., & Casler, M. D. (2016) · G3 Genes Genomes Genetics · doi:10.1534/g3.115.024950 · Cited by 49
Evans, J., Crisovan, E., Barry, K., Daum, C. G., Jenkins, J. W., Kunde‐Ramamoorthy, G., Nandety, A., Ngan, C. Y., Vaillancourt, B., Wei, C., et al. (2015) · The Plant Journal · doi:10.1111/tpj.13041 · Cited by 37
Extensive Genetic Diversity is Present within North American Switchgrass Germplasm
Evans, J., Sanciangco, M. D. A., Lau, K. H., Crisovan, E., Barry, K., Daum, C. G., Hundley, H. N., Jenkins, J. W., Kennedy, M., Kunde‐Ramamoorthy, G., et al. (2017) · The Plant Genome · doi:10.3835/plantgenome2017.06.0055 · Cited by 35
Research works printed but not linked: 3
- Kasnavi, 2017, Performance evaluation of support vector machine (SVM)-based predictors in genomic selection., Indian J. Anim. Sci., 87, 1226, 10.56093/ijans.v87i10.75270 · doi:10.56093/ijans.v87i10.75270
- Monir, 2018, Dominance and Epistasis Interactions Revealed as Important Variants for Leaf Traits of Maize NAM Population., Front. Plant Sci., 9, 627, 10.3389/fpls.2018.00627 · doi:10.3389/fpls.2018.00627
- BMC Genomics, 452
Cited by
A review of deep learning applications for genomic selection
Montesinos-López, O. A., Montesinos-López, A., Pérez-Rodríguez, P., Barrón-López, J. A., Martini, J. W. R., Fajardo-Flores, S. B., Gaytan-Lugo, L. S., Santana-Mancilla, P. C., & Crossa, J. (2021) · BMC Genomics · doi:10.1186/s12864-020-07319-x · Cited by 361
“[77], using data of six plant species and 18 traits across different training population sizes and marker densities, compared the performance of six linear and five non-linear machine learning models, including DL models.”
MethodXu, Y., Zhang, X., Li, H., Zheng, H., Zhang, J., Olsen, M. S., Varshney, R. K., Prasanna, B. M., & Qian, Q. (2022) · Molecular Plant · doi:10.1016/j.molp.2022.09.001 · Cited by 363
Genomic selection in plant breeding: Key factors shaping two decades of progress
Alemu, A., Åstrand, J., Montesinos-López, O. A., Isidro y Sánchez, J., Fernández-Gónzalez, J., Tadesse, W., Vetukuri, R. R., Carlsson, A. S., Ceplitis, A., Crossa, J., et al. (2024) · Molecular Plant · doi:10.1016/j.molp.2024.03.007 · Cited by 344
Machine learning in plant science and plant breeding
van Dijk, A. D. J., Kootstra, G., Kruijer, W., & de Ridder, D. (2021) · iScience · doi:10.1016/j.isci.2020.101890 · Cited by 320
LightGBM: accelerated genomically designed crop breeding through ensemble learning
Yan, J., Xu, Y., Cheng, Q., Jiang, S., Wang, Q., Xiao, Y., Ma, C., Yan, J., & Wang, X. (2021) · Genome Biology · doi:10.1186/s13059-021-02492-y · Cited by 320
Metadata
- Authors
- Azodi, C. B., Bolger, E., McCarren, A., Roantree, M., de los Campos, G., & Shiu, S.-H.
- Venue
- G3 Genes|Genomes|Genetics
- Publisher
- Oxford University Press (OUP)
- Published
- 2019-11-01
- Volume
- 9
- Issue
- 11
- Pages
- 3691-3702
- ISSN
- 2160-1836 (ISSN-L)
- OpenAlex
- W2974740838
- Cited by
- 259
- References
- 64
- FWCI
- 8.80
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.