Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 38 of the 38 references it cites were found in a registry or an index; 4 of the 16 sources its claims cite were checked for support.

Multimodal deep learning methods enhance genomic prediction of wheat breeding

Open access
Montesinos-López, Abelardo, Rivera, Carolina, Pinto, Francisco, Piñera, Francisco, Gonzalez, David, Reynolds, Mathew, Pérez-Rodríguez, Paulino, Li, Huihui, Montesinos-López, Osval A, Crossa, Jose · 2023 · G3: Genes, Genomes, Genetics

Abstract

While several statistical machine learning methods have been developed and studied for assessing the genomic prediction (GP) accuracy of unobserved phenotypes in plant breeding research, few methods have linked genomics and phenomics (imaging). Deep learning (DL) neural networks have been developed to increase the GP accuracy of unobserved phenotypes while simultaneously accounting for the complexity of genotype–environment interaction (GE); however, unlike conventional GP models, DL has not been investigated for when genomics is linked with phenomics. In this study we used 2 wheat data sets (DS1 and DS2) to compare a novel DL method with conventional GP models. Models fitted for DS1 were GBLUP, gradient boosting machine (GBM), support vector regression (SVR) and the DL method. Results indicated that for 1 year, DL provided better GP accuracy than results obtained by the other models. However, GP accuracy obtained for other years indicated that the GBLUP model was slightly superior to the DL. DS2 is comprised only of genomic data from wheat lines tested for 3 years, 2 environments (drought and irrigated) and 2–4 traits. DS2 results showed that when predicting the irrigated environment with the drought environment, DL had higher accuracy than the GBLUP model in all analyzed traits and years. When predicting drought environment with information on the irrigated environment, the DL model and GBLUP model had similar accuracy. The DL method used in this study is novel and presents a strong degree of generalization as several modules can potentially be incorporated and concatenated to produce an output for a multi-input data structure.

Overview

CoChat Summary

TL;DR

This study compares a novel multimodal deep learning (MMDL) method with conventional genomic prediction models (GBLUP, GBM, SVR) in two wheat datasets, using genomic, year, and NDVI image data for DS1 and genomic-only data for DS2. MMDL improved accuracy in some scenarios, especially predicting irrigated from drought environments and some leave-one-year-out predictions, but GBLUP was often slightly superior under 5-fold cross-validation.

Key findings

  • In DS1 (350 wheat lines, 2 years, traits YLD and TGW), DL provided better GP accuracy than the other models for one year, but for other years GBLUP was slightly superior to DL. Under 5FCV, GBLUP was slightly superior to DL when genomic and various NDVI measurements were included; for example, GBLUP-TGW with Year + G had Cor = 0.8116 (SD 0.0195) versus MMDL-TGW Cor = 0.7608 (SD 0.0158), and GBLUP-YLD had Cor = 0.7364 (SD 0.0567) versus MMDL-YLD Cor = 0.6305 (SD 0.1198).
  • Under leave-one-year-out (LOO) in DS1, the authors report that DL outperformed the other models for both YLD and TGW, for NDVI average and NDVI dates aligned, and for both metrics (NRMSEP and Cor Mean).
  • In DS2 (genomic-only, 3 years, drought and irrigated environments, traits BM, HI, PMG, YLD), when predicting the irrigated environment with the drought environment, DL had higher accuracy than GBLUP in all analyzed traits and years, with the most considerable differences in YLD and BM. When predicting drought with irrigated information, DL and GBLUP had similar accuracy except in the first year (2015–2016), where a more significant difference was obtained.
  • The correlation metric in DS2 was more ambiguous: for PMG, GBLUP was better than DL in all cases but with marginal differences when predicting drought; for YLD, DL was less accurate than GBLUP when predicting drought, but in 2 of 3 years the difference was small, while DL was better in 2 of 3 years when predicting irrigated.

Methods

  • DS1: 350 wheat cultivars/accessions from CIMMYT evaluated over 2 years (2015–2016 and 2016–2017) for grain yield (YLD) and thousand grains weight (TGW) under fully irrigated conditions; alpha lattice design with 2 replicates and 6 incomplete blocks of size 5. Genotyping used the 35 K Axiom Wheat Breeders array, leaving 10,560 SNPs after QC. Aerial multispectral UAV imagery provided NDVI measured 4 times during vegetative and 2 times during grain filling, used as average or dates-aligned covariates. Models compared were GBLUP, MMDL, GBM, and SVR, assessed by 5-fold cross-validation (5FCV) and leave-one-year-out (LOO) using NRMSEP and correlation between observed and predicted values.
  • DS2: synthetic hexaploid wheat lines evaluated over 3 consecutive years (2015–2016, 2016–2017, 2017–2018), with 145–155 lines per year in drought and irrigated environments and 2 or 4 traits (BM, HI, PMG, YLD). Genomic-only data with 10,064 SNPs after QC; GBLUP and DL were compared by predicting one entire environment from the other (LOO).
  • Analysis: GBLUP was fitted with the BGLR R package; MMDL used three modalities (year, genomic, NDVI) with separate neural networks whose outputs were concatenated, implemented in TensorFlow with batch size 32, 128 epochs, Adam optimizer, early stopping, dropout, and L2 regularization. Hyperparameters were tuned with Bayesian optimization (`bayes_opt`) over 150 iterations, requiring an average of 8 hours to train a model.

Limitations

  • Stated by the authors: validation on additional data is needed; larger datasets with denser NDVI data should be used; different neural network types beyond the multilayer perceptron and other late fusion strategies (e.g., uncertainty-based weighting or additional layers after combining sub-model outputs) need to be explored.
  • Evident from the text: DS2 lacks image/phenomics data, so the multimodal DL reduces to standard DL for that dataset; MMDL was only used under 5FCV in DS1, not under LOO; results were mixed, with GBLUP often slightly superior under 5FCV; and the study uses only two wheat datasets with small-to-medium sample sizes, while training each DL model took about 8 hours.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1093/g3journal/jkad045

Cites

38 of 38 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Fig. 1. Left diagram: Multimodal deep learning model (MMDL) with 3 modalities (type of input). Right diagram: stacked residuals network (RestNet) composed of 2 sequence dense layers (FCL) applied in each MP neural network. FCL(L2)+ +BN + RELU denote the successive application of a full connected layer (FCL) with L2 regularization, batch normalization layer and relu activation. The meaning is similar for FCL(L2)+BN, while DO indicate the application of the dropout regularization. Batch Normalization (BN) also works as a regularizer to speed up the training process (He et al. 2016). To produce the final output, the concatenated outputs of three networks are used as an input in the output layer with 1 neuron, linear activation function, and L2 regularization for its weights (concatenate outputs of all 3 (multilayer perceptron deep neural network) MP neural networks + FCL + L2).
Fig. 1. Left diagram: Multimodal deep learning model (MMDL) with 3 modalities (type of input). Right diagram: stacked residuals network (RestNet) composed of 2 sequence dense layers (FCL) applied in each MP neural network. FCL(L2)+ +BN + RELU denote the successive application of a full connected layer (FCL) with L2 regularization, batch normalization layer and relu activation. The meaning is similar for FCL(L2)+BN, while DO indicate the application of the dropout regularization. Batch Normalization (BN) also works as a regularizer to speed up the training process (He et al. 2016). To produce the final output, the concatenated outputs of three networks are used as an input in the output layer with 1 neuron, linear activation function, and L2 regularization for its weights (concatenate outputs of all 3 (multilayer perceptron deep neural network) MP neural networks + FCL + L2).CC BY · PubMed Central
Fig. 2. DS1.Normalized Root Mean Squared Error of Prediction (NRMSEP) for the prediction of a complete year using information from the other year (leave one environment out, LOO) for traits TGW and YLD using the models fitted with GBLUP with BGLR (R software) and with DL (Python software) with the predictors G (red Type of NDVI__, no use of NDVI), which correspond to the genomic matrix, G + NDVIs_gf (green), G + NDVIs_vg (blue), and G + NDVIs_vg_gf (purple) for the NDVIs covariate in any of its 2 types NDVI averages and dates aligned.
Fig. 2. DS1.Normalized Root Mean Squared Error of Prediction (NRMSEP) for the prediction of a complete year using information from the other year (leave one environment out, LOO) for traits TGW and YLD using the models fitted with GBLUP with BGLR (R software) and with DL (Python software) with the predictors G (red Type of NDVI__, no use of NDVI), which correspond to the genomic matrix, G + NDVIs_gf (green), G + NDVIs_vg (blue), and G + NDVIs_vg_gf (purple) for the NDVIs covariate in any of its 2 types NDVI averages and dates aligned.CC BY · PubMed Central
Fig. 3. DS1.Correlation Mean (Cor Mean) for the prediction of a complete year using information from the other year (leave one environment out, LOO) for traits TGW and YLD using the models fitted GBLUP with BGLR (R software) and with DL (Python software) with the predictors G (red Type of NDVI__, no use of NDVI), which correspond to the genomic matrix, G + NDVIs_gf (green), G + NDVIs_vg (blue), and G + NDVIs_vg_gf (purple) for the NDVIs covariate in any of its 2 types NDVI averages and dates aligned.
Fig. 3. DS1.Correlation Mean (Cor Mean) for the prediction of a complete year using information from the other year (leave one environment out, LOO) for traits TGW and YLD using the models fitted GBLUP with BGLR (R software) and with DL (Python software) with the predictors G (red Type of NDVI__, no use of NDVI), which correspond to the genomic matrix, G + NDVIs_gf (green), G + NDVIs_vg (blue), and G + NDVIs_vg_gf (purple) for the NDVIs covariate in any of its 2 types NDVI averages and dates aligned.CC BY · PubMed Central
Fig. 4. DS1. Normalized Root Mean Squared Error of Prediction (NRMSEP) under a 5-fold cross-validation for traits TGW and YLD, for models fitted under BGLR (R software) and multimodal DL (Python software) with the predictors being year + G, which correspond to the year and the genomic matrix, year + G + NVVDIs_gf, and year + G + NDVIs_vg, adding the covariate NDVIs as average or aligned dates (Type_NDVIs). For the GBLUP the following predictors are also used, year + G + year × G, making use of the interaction between the genomic matrix and the year, year + G + year × GID + NDVIs_gf, year + GID + year × GID + NDVIs_vg, and year + G + year × GID + NDVIs_vg_gf that adds the NDVIs covariate in 1 of its 2 types (Type of NDVIs), averages and dates aligned.
Fig. 4. DS1. Normalized Root Mean Squared Error of Prediction (NRMSEP) under a 5-fold cross-validation for traits TGW and YLD, for models fitted under BGLR (R software) and multimodal DL (Python software) with the predictors being year + G, which correspond to the year and the genomic matrix, year + G + NVVDIs_gf, and year + G + NDVIs_vg, adding the covariate NDVIs as average or aligned dates (Type_NDVIs). For the GBLUP the following predictors are also used, year + G + year × G, making use of the interaction between the genomic matrix and the year, year + G + year × GID + NDVIs_gf, year + GID + year × GID + NDVIs_vg, and year + G + year × GID + NDVIs_vg_gf that adds the NDVIs covariate in 1 of its 2 types (Type of NDVIs), averages and dates aligned.CC BY · PubMed Central
Fig. 5. DS1. Correlation Mean (Cor Mean) under a 5-fold cross-validation for the prediction of traits TGW and YLD, for models fitted under GBLUP with BGLR (R software) and multimodal DL (Python software) with the predictors being year + G, which corresponds to the year and the genomic matrix, year + G + NVVDIs_gf, and year + G + NDVIs_vg, adding the covariate NDVIs as average or dates aligned (Type of NDVIs). For the GBLUP the following predictors are also used, year + G + year × G, making use of the interaction between the genomic matrix and the year, year + G + year × GID + NDVIs_gf, year + GID + year × GID + NDVIs_vg, and year + G + year × GID + NDVIs_vg_gf that adds the NDVIs covariate in 1 of its 2 types (Type of NDVIs), averages and dates aligned.
Fig. 5. DS1. Correlation Mean (Cor Mean) under a 5-fold cross-validation for the prediction of traits TGW and YLD, for models fitted under GBLUP with BGLR (R software) and multimodal DL (Python software) with the predictors being year + G, which corresponds to the year and the genomic matrix, year + G + NVVDIs_gf, and year + G + NDVIs_vg, adding the covariate NDVIs as average or dates aligned (Type of NDVIs). For the GBLUP the following predictors are also used, year + G + year × G, making use of the interaction between the genomic matrix and the year, year + G + year × GID + NDVIs_gf, year + GID + year × GID + NDVIs_vg, and year + G + year × GID + NDVIs_vg_gf that adds the NDVIs covariate in 1 of its 2 types (Type of NDVIs), averages and dates aligned.CC BY · PubMed Central
Fig. 6. DS2. Normalized Root Mean Squared Error of Prediction (NRMSEP) for the prediction of a trait (BM, HI, PMG, and YLD) in drought (irrigated) environment using information from an irrigated (drought) environment (leave one environment out, LOO) using a sub-model of (1) fitted with GBLUP and with a sub-model of DL model (2), both with the predictor G. Each of these sub-models were fitted for each year (2015–2016, 2016–2017, and 2017–2018) and each trait.
Fig. 6. DS2. Normalized Root Mean Squared Error of Prediction (NRMSEP) for the prediction of a trait (BM, HI, PMG, and YLD) in drought (irrigated) environment using information from an irrigated (drought) environment (leave one environment out, LOO) using a sub-model of (1) fitted with GBLUP and with a sub-model of DL model (2), both with the predictor G. Each of these sub-models were fitted for each year (2015–2016, 2016–2017, and 2017–2018) and each trait.CC BY · PubMed Central
Fig. 7. DS2. Correlation Mean (Cor Mean) for the prediction of a trait (BM, HI, PMG, and YLD) in drought (irrigated) environment using information from an irrigated (drought) environment (leave one environment out, LOO) using a sub-model of 1 fitted with GBLUP and with a sub-model of DL model 2, both with the predictor G. Each of these sub-models were fitted for each year (2015–2016, 2016–2017, and 2017–2018) and each trait.
Fig. 7. DS2. Correlation Mean (Cor Mean) for the prediction of a trait (BM, HI, PMG, and YLD) in drought (irrigated) environment using information from an irrigated (drought) environment (leave one environment out, LOO) using a sub-model of 1 fitted with GBLUP and with a sub-model of DL model 2, both with the predictor G. Each of these sub-models were fitted for each year (2015–2016, 2016–2017, and 2017–2018) and each trait.CC BY · PubMed Central

Verification

Claims the paper makes

4 Valid · 1 Not valid · 12 Not checked · 71 Own results and interpretations

40 more claims

Create a free account to see all 88 claims and to verify the 12 references still open.

See all claims — free account
  • Probably not a substantial claim. 20
  • It may state two separate claims as one. 26

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

3 of 3

Citations

Cites

Show all 38 references

Research works printed but not linked: 1

  1. Wang, 2021, –, 1169

Cited by

See all 41

Metadata

Authors
Montesinos-López, A., Rivera, C., Pinto, F., Piñera, F., Gonzalez, D., Reynolds, M., Pérez-Rodríguez, P., Li, H., Montesinos-López, O. A., & Crossa, J.
Affiliations
  • Departamento de Matemáticas, Centro Universitario de Ciencias Exactas e Ingenierías (CUCEI), Universidad de Guadalajara , 44430, Guadalajara, Jalisco , Mexico
  • Centro Internacional de Mejoramiento de Maíz Y Trigo (MX)
  • International Maize and Wheat Improvement Center (CIMMYT) , Carretera México- Veracruz Km. 45, El Batán, CP 56237, Texcoco, Edo. de México , Mexico
  • Colegio de Postgraduados , Montecillos, Edo. de México, CP 56230 , Mexico
  • Chinese Academy of Agricultural Sciences (CN)
  • Universidad de Colima (MX)
Corresponding author
Montesinos-López, Osval A
Venue
G3: Genes, Genomes, Genetics
Published
2023-02-27
Volume
13
Issue
5
ISSN
2160-1836 (ISSN-L)
OpenAlex
W4323067038
Cited by
41
References
38
Citations by year
2026: 14 · 2025: 11 · 2024: 11 · 2023: 5
FWCI
6.80
Type
journal article
Language
en
Open access
Locations
  • G3 Genes Genomes Genetics (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
  • Bill & Melinda Gates Foundation (INV-003439)
  • BMGF/FCDO
  • Accelerating Genetic Gains in Maize and Wheat for Improved Livelihoods
  • USAID
  • AGG
  • CIMMYT CRP
  • Foundation for Research Levy on Agricultural Products
  • Agricultural Agreement Research Fund
  • Research Council of Norway (301835, 320090)
  • International Wheat Yield Partnership
  • Heat and Drought Wheat Improvement Consortium
  • Foundation for Food and Agriculture Research (DFs-19-0000000013)
Topics
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Genetics › Genetic and phenotypic traits in livestock
  • Life Sciences › Agricultural and Biological Sciences › Plant Science › Wheat and Barley Genetics and Pathology
  • Life Sciences › Agricultural and Biological Sciences › Plant Science › Genetics and Plant Breeding
Keywords
genomic prediction, wheat breeding, deep learning, multimodal data, phenomics, gene-environment interaction, GBLUP, gradient boosting, support vector regression, drought tolerance, multi-input neural network
MeSH terms
Plant Breeding, Deep Learning, Genotype, Models, Genetic, Phenotype, Triticum, Genomics

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

A language model drew the claims from the paper on 2026-10-05, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-05 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (qwen3.8-omni-flash).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Multimodal deep learning methods enhance genomic prediction of wheat breeding. https://app.cochat.ai/papers/10.1093/g3journal/jkad045
The paper
Montesinos-López, A., Rivera, C., Pinto, F., Piñera, F., Gonzalez, D., Reynolds, M., Pérez-Rodríguez, P., Li, H., Montesinos-López, O. A., & Crossa, J. (2023). Multimodal deep learning methods enhance genomic prediction of wheat breeding. G3: Genes, Genomes, Genetics. https://doi.org/10.1093/g3journal/jkad045

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.