Multi-modal deep learning improves grain yield prediction in wheat breeding by fusing genomics and phenomics
Open accessThe DOI matches its Crossref record; 38 of the 38 references it cites were found in a registry or an index.
Abstract
Motivation: Developing new crop varieties with superior performance is highly important to ensure robust and sustainable global food security. The speed of variety development is limited by long field cycles and advanced generation selections in plant breeding programs. While methods to predict yield from genotype or phenotype data have been proposed, improved performance and integrated models are needed. Results: We propose a machine learning model that leverages both genotype and phenotype measurements by fusing genetic variants with multiple data sources collected by unmanned aerial systems. We use a deep multiple instance learning framework with an attention mechanism that sheds light on the importance given to each input during prediction, enhancing interpretability. Our model reaches 0.754 ± 0.024 Pearson correlation coefficient when predicting yield in similar environmental conditions; a 34.8% improvement over the genotype-only linear baseline (0.559 ± 0.050). We further predict yield on new lines in an unseen environment using only genotypes, obtaining a prediction accuracy of 0.386 ± 0.010, a 13.5% improvement over the linear baseline. Our multi-modal deep learning architecture efficiently accounts for plant health and environment, distilling the genetic contribution and providing excellent predictions. Yield prediction algorithms leveraging phenotypic observations during training therefore promise to improve breeding programs, ultimately speeding up delivery of improved varieties. Availability and implementation: Available at https://github.com/BorgwardtLab/PheGeMIL (code) and https://doi.org/doi:10.5061/dryad.kprr4xh5p (data).
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures




Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“In some instances, studies such as Kick et al. (2023) and Togninalli et al. (2023) have shown no clear superiority of MMDL over unimodal DL in terms of prediction accuracy.”
From A review of multimodal deep learning methods for genomic-enabled prediction in plant breeding (2024) · cites this paper as Togninalli et al. (2023)
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“The integration of all data channels results in a PCC of 0.767 ± 0.019, showcasing the model's superior predictive capability.”
From A review of multimodal deep learning methods for genomic-enabled prediction in plant breeding (2024) · cites this paper as Togninalli et al. (2023)
In the citing paper
“PheGeMIL significantly outperforms benchmarks like Standard, Lasso, Ridge, Elastic Net regression, Gradient Boosting, and RF.”
From A review of multimodal deep learning methods for genomic-enabled prediction in plant breeding (2024) · cites this paper as Togninalli et al. (2023)
In the citing paper
“With the inclusion of genotypic information, the model achieves a PCC of 0.754 ± 0.024, compared to 0.707 ± 0.027 for RF and 0.708 ± 0.029 for Lasso.”
From A review of multimodal deep learning methods for genomic-enabled prediction in plant breeding (2024) · cites this paper as Togninalli et al. (2023)
In the citing paper
“Togninalli et al. (2023) introduce Phenotype–Genotype Mutual Information Learning (PheGeMIL), a DL framework incorporating multichannel temporal inputs to enhance grain yield prediction using UAV images.”
From A review of multimodal deep learning methods for genomic-enabled prediction in plant breeding (2024) · cites this paper as Togninalli et al. (2023)
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Scikit-learn: Machine Learning in Python
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Müller, A., Nothman, J., Louppe, G., et al. (2012) · arXiv · doi:10.48550/arxiv.1201.0490 · Cited by 71,016 · cited 1×
“Hyperparameters for the other baseline models are tuned via internal cross-validation on the training set (90% of the dataset) using the dedicated scikit-learn python library (Pedregosa et al. 2011).”
Advances in neural information processing systems 7
(1996) · Computers & Mathematics with Applications · doi:10.1016/s0898-1221(96)90463-0 · Cited by 14,281
Proceedings of the 25th international conference on Machine learning
Cohen, W. W., McCallum, A., & Roweis, S. T. (2008) · Cited by 5,691
Sagi, O., & Rokach, L. (2018) · WIREs Data Mining and Knowledge Discovery · doi:10.1002/widm.1249 · Cited by 3,103 · cited 1×
“Given that we have five trained models from the cross-validation scheme, we also ensemble the predictions as this is known to yield better performance (Sagi and Rokach 2018), finally obtaining a 17.6% improvement for PheGeMIL over the linear baseline.”
InfluentialUnmanned aerial systems for photogrammetry and remote sensing: A review
Colomina, I., & Molina, P. (2014) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2014.02.013 · Cited by 2,954 · cited 1×
“Both satellite imagery and low altitude Unmanned Aerial Vehicle (UAV) embedded sensors are used, but the latter are preferred for their finer-grained resolution and high-throughput pheno-typing abilities (Colomina and Molina 2014, Haghighattalab et al. 2016).”
Breeding Technologies to Increase Crop Production in a Changing World
Tester, M., & Langridge, P. (2010) · Science · doi:10.1126/science.1183700 · Cited by 2,322 · cited 1×
“…advances such as high-throughput phenotyping and genomic selection have enabled a constant growth in yield for various crops across the years (Tester and Langridge 2010): for example, breeders of wheat ( Triticum aestivum ) manage to obtain average annual gains of 1% (Tadesse et al. 2019).”
Crop yield prediction using machine learning: A systematic literature review
van Klompenburg, T., Kassahun, A., & Catal, C. (2020) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2020.105709 · Cited by 1,858
Genomic Selection in Plant Breeding: Methods, Models, and Perspectives
Crossa, J., Pérez-Rodríguez, P., Cuevas, J., Montesinos-López, O., Jarquín, D., de los Campos, G., Burgueño, J., González-Camacho, J. M., Pérez-Elizalde, S., Beyene, Y., et al. (2017) · Trends in Plant Science · doi:10.1016/j.tplants.2017.08.011 · Cited by 1,839
Self-citationA chromosome-based draft sequence of the hexaploid bread wheat ( Triticum aestivum ) genome
The International Wheat Genome Sequencing Consortium (IWGSC), Mayer, K. F. X., Rogers, J., Doležel, J., Pozniak, C., Eversole, K., Feuillet, C., Gill, B., Friebe, B., Lukaszewski, A. J., et al. (2014) · Science · doi:10.1126/science.1251788 · Cited by 1,583
Self-citationSoybean yield prediction from UAV using multimodal data fusion and deep learning
Maimaitijiang, M., Sagan, V., Sidike, P., Hartling, S., Esposito, F., & Fritschi, F. B. (2020) · Remote Sensing of Environment · doi:10.1016/j.rse.2019.111599 · Cited by 971 · cited 2×
“More recently, deep learning techniques leveraged the potential of high-dimensional image data both across data sources (Herrero-Huerta et al. 2020, Maimaitijiang et al. 2020) and time (Khaki et al. 2019, Nevavuori et al. 2020).”
Research works printed but not linked: 1
- Falcon, 2019, PyTorch Lightning
Cited by
Artificial intelligence in plant breeding
Farooq, M. A., Gao, S., Hassan, M. A., Huang, Z., Rasheed, A., Hearne, S., Prasanna, B., Li, X., & Li, H. (2024) · Trends in Genetics · doi:10.1016/j.tig.2024.07.001 · Cited by 288
Big data and artificial intelligence‐aided crop breeding: Progress and prospects
Zhu, W., Li, W., Zhang, H., & Li, L. (2024) · Journal of Integrative Plant Biology · doi:10.1111/jipb.13791 · Cited by 93
“…instance learning (PheGeMIL) uses an MLP to capture fea - tures from genotypes and employs a small residual network architecture to extract features from multispectral images and thermal images, reaching a 34.8% improvement over a genotype ‐ only linear baseline approach (Togninalli et al., 2023).”
InfluentialYuan, J., Zhang, Y., Zheng, Z., Yao, W., Wang, W., & Guo, L. (2024) · Drones · doi:10.3390/drones8100559 · Cited by 95
Kaushal, S., Gill, H. S., Billah, M. M., Khan, S. N., Halder, J., Bernardo, A., Amand, P. S., Bai, G., Glover, K. D., Maimaitijiang, M., et al. (2024) · Frontiers in Plant Science · doi:10.3389/fpls.2024.1410249 · Cited by 72
A review of multimodal deep learning methods for genomic-enabled prediction in plant breeding
Montesinos-López, O. A., Chavira-Flores, M., Kiasmiantini, Crespo-Herrera, L., Saint Piere, C., Li, H., Fritsche-Neto, R., Al-Nowibet, K., Montesinos-López, A., & Crossa, J. (2024) · GENETICS · doi:10.1093/genetics/iyae161 · Cited by 47
Self-citation
Metadata
- Authors
- Togninalli, M., Wang, X., Kucera, T., Shrestha, S., Juliana, P., Mondal, S., Pinto, F., Govindan, V., Crespo-Herrera, L., Huerta-Espino, J., Singh, R. P., Borgwardt, K., & Poland, J.
- Affiliations
- SIB Swiss Institute of Bioinformatics (CH)
- Kansas State University (US)
- Centro Internacional de Mejoramiento de Maíz Y Trigo (MX)
- Global Wheat Program, International Maize and Wheat Improvement Center , Texcoco, Estado de Mexico, Mexico
- Corresponding author
- Togninalli, Matteo
- Borgwardt, Karsten
- Poland, Jesse
- ORCID
- Venue
- Bioinformatics
- Publisher
- Oxford University Press (OUP)
- Published
- 2023-05-23
- Volume
- 39
- Issue
- 6
- ISSN
- 1367-4803 (ISSN-L)
- 1367-4811
- PMC
- PMC10246581
- OpenAlex
- W4377691294
- Cited by
- 49
- References
- 38
- Citations by year
- 2026: 22 · 2025: 17 · 2024: 10
- FWCI
- 7.72
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Bioinformatics (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- King Abdullah University of Science and Technology Repository (King Abdullah University of Science and Technology) (submittedVersion, cc-by)
- Repository for Publications and Research Data (ETH Zurich) (submittedVersion, cc-by)
- ETH Zürich Research Collection
- Funders
- National Science Foundation (1238187)
- Topics
- Life Sciences › Agricultural and Biological Sciences › Plant Science › Smart Agriculture and AI
- Life Sciences › Agricultural and Biological Sciences › Plant Science › Genetics and Plant Breeding
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Genetics › Genetic and phenotypic traits in livestock
- Keywords
- wheat breeding, grain yield prediction, multimodal deep learning, unmanned aerial systems, plant phenotyping, attention mechanism, interpretability, plant health, cultivar development
- MeSH terms
- Plant Breeding, Deep Learning, Phenomics, Edible Grain, Genotype, Phenotype, Selection, Genetic, Triticum, Genomics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Multi-modal deep learning improves grain yield prediction in wheat breeding by fusing genomics and phenomics. https://app.cochat.ai/papers/10.1093/bioinformatics/btad336
- The paper
- Togninalli, M., Wang, X., Kucera, T., Shrestha, S., Juliana, P., Mondal, S., Pinto, F., Govindan, V., Crespo-Herrera, L., Huerta-Espino, J., Singh, R. P., Borgwardt, K., & Poland, J. (2023). Multi-modal deep learning improves grain yield prediction in wheat breeding by fusing genomics and phenomics. Bioinformatics. https://doi.org/10.1093/bioinformatics/btad336
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.