Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 68 of the 68 references it cites were found in a registry or an index; 24 of the 92 sources its claims cite were checked for support.

A review of multimodal deep learning methods for genomic-enabled prediction in plant breeding

Open access
Montesinos-López, Osval A, Chavira-Flores, Moises, Kiasmiantini, Crespo-Herrera, Leo, Saint Piere, Carolina, Li, HuiHui, Fritsche-Neto, Roberto, Al-Nowibet, Khalid, Montesinos-López, Abelardo, Crossa, José · 2024 · GENETICS

Abstract

Deep learning methods have been applied when working to enhance the prediction accuracy of traditional statistical methods in the field of plant breeding. Although deep learning seems to be a promising approach for genomic prediction, it has proven to have some limitations, since its conventional methods fail to leverage all available information. Multimodal deep learning methods aim to improve the predictive power of their unimodal counterparts by introducing several modalities (sources) of input information. In this review, we introduce some theoretical basic concepts of multimodal deep learning and provide a list of the most widely used neural network architectures in deep learning, as well as the available strategies to fuse data from different modalities. We mention some of the available computational resources for the practical implementation of multimodal deep learning problems. We finally performed a review of applications of multimodal deep learning to genomic selection in plant breeding and other related fields. We present a meta-picture of the practical performance of multimodal deep learning methods to highlight how these tools can help address complex problems in the field of plant breeding. We discussed some relevant considerations that researchers should keep in mind when applying multimodal deep learning methods. Multimodal deep learning holds significant potential for various fields, including genomic selection. While multimodal deep learning displays enhanced prediction capabilities over unimodal deep learning and other machine learning methods, it demands more computational resources. Multimodal deep learning effectively captures intermodal interactions, especially when integrating data from different sources. To apply multimodal deep learning in genomic selection, suitable architectures and fusion strategies must be chosen. It is relevant to keep in mind that multimodal deep learning, like unimodal deep learning, is a powerful tool but should be carefully applied. Given its predictive edge over traditional methods, multimodal deep learning is valuable in addressing challenges in plant breeding and food security amid a growing global population.

Overview

CoChat Summary

TL;DR

This review introduces multimodal deep learning (MMDL) concepts, architectures, and fusion strategies for genomic-enabled prediction in plant breeding, then surveys applications integrating genomic, environmental, phenotypic, and other omics data. It concludes MMDL often improves prediction over unimodal DL and traditional ML, but requires more computational resources and careful choices of architecture, fusion strategy, and hyperparameters.

Key findings

  • MMDL can integrate genomic, phenotypic, environmental, pedigree, transcriptomic, epigenomic, proteomic, and image modalities; early, intermediate, and late fusion each have tradeoffs in computational cost, robustness, and ability to capture cross-modality interactions.
  • Reviewed applications often show MMDL advantages: Shook et al. (2021) LSTM models surpassed SVR–RBF and Lasso on RMSE, MAE, and R²; Måløy et al. (2021) performer-based models significantly outperformed traditional DL and BGLR-RKHS; Togninalli et al. (2023) PheGeMIL achieved PCC 0.754 ± 0.024 versus RF 0.707 ± 0.027 and Lasso 0.708 ± 0.029, and 0.767 ± 0.019 with all data channels.
  • MMDL is not always superior: Kick et al. (2023) reported MMDL RMSE 0.948 and NRMSE 14.554%, closely matching BLUP RMSE 0.937 and NRMSE 14.388%; Montesinos-López et al. (2023) found GBLUP nRMSE 0.0663 ± 0.0089 for thousand-grain weight versus MMDL 0.0744 ± 0.0081.
  • MMDL demands more computational resources, data acquisition/preprocessing, tuning, and storage; transfer learning and interpretability are highlighted as future directions.

Methods

  • Design: narrative review covering neural network basics, activation functions, advanced architectures (MLP, CNN, RNN/LSTM, ResNet, Transformers, GNN, Autoencoders), and data fusion strategies (early, intermediate, late).
  • Data/sample: review of publications in Table 3 with actual or potential applications to genomic prediction, focusing on molecular marker (SNP) data and comparisons of MMDL with unimodal DL; includes crops/species, models, modalities, responses, and comparators.
  • Main analysis: qualitative synthesis of reported prediction performance, plus a simple MMDL example using 4,464 wheat lines across 4 environments in 2021/2022 at Ciudad Obregón, Mexico, with 18,239 SNPs and 5 traits; no formal meta-analysis is reported.

Limitations

  • Stated: MMDL application in GS/PB is relatively nascent with limited literature; selecting an optimal fusion strategy is challenging; computational costs are high for high-dimensional genomic data; additional modalities may not justify computational expense; data from each modality must be available in training and testing; caution is needed with small training-testing datasets and hyperparameter selection.
  • Stated: some studies, such as Kick et al. (2023) and Togninalli et al. (2023), show no clear superiority of MMDL over unimodal DL in prediction accuracy.
  • Evident from the text: the review is narrative/qualitative rather than a systematic meta-analysis, and performance comparisons rely on heterogeneous studies, metrics, and datasets, so no pooled effect sizes are reported.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1093/genetics/iyae161

Cites

68 of 68 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Fig. 1. Basic anatomy and working of an artificial neuron.
Fig. 1. Basic anatomy and working of an artificial neuron.CC BY · PubMed Central
Fig. 2. a) Early data fusion with raw data for all modalities; b) early data fusion with some modalities preprocessed separately; and c) early data fusion with all modalities preprocessed jointly.
Fig. 2. a) Early data fusion with raw data for all modalities; b) early data fusion with some modalities preprocessed separately; and c) early data fusion with all modalities preprocessed jointly.CC BY · PubMed Central
Fig. 3. Example of a model with intermediate data fusion. Note that after the fusion module where representations for all modalities are combined, we have a final sub-model to produce the definite output. The Sub-output means a marginal representation (marginal latent factor), that is, representations of data features specific to each modality.
Fig. 3. Example of a model with intermediate data fusion. Note that after the fusion module where representations for all modalities are combined, we have a final sub-model to produce the definite output. The Sub-output means a marginal representation (marginal latent factor), that is, representations of data features specific to each modality.CC BY · PubMed Central
Fig. 4. General form of late data fusion in MMDL. Every modality has its own separated DL model. The resulting outputs are concatenated and finally combined by an aggregation function to get the final output.
Fig. 4. General form of late data fusion in MMDL. Every modality has its own separated DL model. The resulting outputs are concatenated and finally combined by an aggregation function to get the final output.CC BY · PubMed Central

Verification

Claims the paper makes

21 Valid · 10 Not valid · 60 Not checked · 55 Own results and interpretations

58 more claims · 9 with their cited sources checked

Create a free account to see all 146 claims and to verify the 68 references still open.

See all claims — free account
  • It may state two separate claims as one. 39
  • Probably not a substantial claim. 63
  • The quote only partly states this claim. 3

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

1 of 1

Citations

Cites

Show all 67 references

Research works printed but not linked: 4

  1. Chen, 2021, A review of multi-view deep learning methods for multimodal data fusion, Neural Comput, 33, 67
  2. Chollet, 2017, Deep Learning with R
  3. Goodfellow, 2016, Deep Learning
  4. Henderson, 1984, Applications of Linear Models in Animal Breeding Models

Cited by

See all 47

Metadata

Authors
Montesinos-López, O. A., Chavira-Flores, M., Kiasmiantini, Crespo-Herrera, L., Saint Piere, C., Li, H., Fritsche-Neto, R., Al-Nowibet, K., Montesinos-López, A., & Crossa, J.
Affiliations
  • Universidad de Colima (MX)
  • Instituto de Investigaciones en Matemáticas Aplicadas y Sistemas (IIMAS), Universidad Nacional Autónoma de México (UNAM) , Ciudad de México 04510 ,
  • Yogyakarta State University (ID)
  • International Maize and Wheat Improvement Center (CIMMYT) , Km 45, Carretera Mexico-Veracruz, Texcoco, CP 52640 Edo. de México ,
  • Chinese Academy of Agricultural Sciences (CN)
  • Louisiana State University , Baton Rouge, LA 70803 ,
  • Department of Statistics and Operations Research, King Saud University , Riyah 11451 ,
  • Departamento de Matematicas, Centro Universitario de Ciencias Exactas e Ingenierías (CUCEI), Universidad de Guadalajara , 44430, Guadalajara, Jalisco ,
  • Louisiana State University (US)
Corresponding author
Crossa, José
Venue
GENETICS
Published
2024-11-05
ISSN
  • 0016-6731 (ISSN-L)
  • 1943-2631
OpenAlex
W4404061905
Cited by
47
References
67
Citations by year
2026: 32 · 2025: 14 · 2024: 1
FWCI
12.43
Type
journal article
Language
en
Open access
Locations
  • Genetics (publishedVersion)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • CGSPace A Repository of Agricultural Research Outputs (Consultative Group for International Agricultural Research) (submittedVersion, other-oa)
Funders
  • Bill & Melinda Gates Foundation (INV-003439 BMGF/FCDO)
  • Accelerating Genetic Gains in Maize and Wheat for Improved Livelihoods
  • USAID projects
  • Foundation for Research Levy on Agricultural Products
  • Agricultural Agreement Research Fund
  • Research Council of Norway (301835, 320090)
Topics
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Genetics › Genetic and phenotypic traits in livestock
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Genetics › Genetic Mapping and Diversity in Plants and Animals
  • Life Sciences › Agricultural and Biological Sciences › Plant Science › Smart Agriculture and AI
Keywords
multimodal deep learning, genomic selection, plant breeding, genomic prediction, neural network architecture, computational resources, food security

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-05, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-05 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol and qwen3.8-omni-flash).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: A review of multimodal deep learning methods for genomic-enabled prediction in plant breeding. https://app.cochat.ai/papers/10.1093/genetics/iyae161
The paper
Montesinos-López, O. A., Chavira-Flores, M., Kiasmiantini, Crespo-Herrera, L., Saint Piere, C., Li, H., Fritsche-Neto, R., Al-Nowibet, K., Montesinos-López, A., & Crossa, J. (2024). A review of multimodal deep learning methods for genomic-enabled prediction in plant breeding. GENETICS. https://doi.org/10.1093/genetics/iyae161

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.