The DOI matches its Crossref record; 14 of the 14 references it cites were found in a registry or an index.
On Some Limitations of Current Machine Learning Weather Prediction Models
Open accessAbstract
Machine Learning (ML) is having a profound impact in the domain of Weather and Climate Prediction. A recent development in this area has been the emergence of fully data‐driven ML prediction models which routinely claim superior performance to that of traditional physics‐based models. We examine some aspects of the forecasts produced by three of the leading current ML models, Pangu‐Weather, FourCastNet and GraphCast, with a focus on their fidelity and physical consistency. The main conclusion is that these ML models are not able to properly reproduce sub‐synoptic and mesoscale weather phenomena and lack the fidelity and physical consistency of physics‐based models and this has impacts on the interpretation of their forecasts and their perceived skill. Balancing forecast skill and physical realism will be an important consideration for future ML models.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Unlike physical models, machine-learning models misrepresent derived (diagnostic) variables such as geostrophic wind16.”
From Neural general circulation models for weather and climate (2024) · cites this paper as [16]
Citations
Cites
Hersbach, H., Bell, B., Berrisford, P., Hirahara, S., Horányi, A., Muñoz‐Sabater, J., Nicolas, J., Peubey, C., Radu, R., Schepers, D., et al. (2020) · Quarterly Journal of the Royal Meteorological Society · doi:10.1002/qj.3803 · Cited by 31,882
Self-citationAn Introduction to Dynamic Meteorology
Holton, J. R., & Staley, D. O. (1973) · American Journal of Physics · doi:10.1119/1.1987371 · Cited by 3,425
Learning skillful medium-range global weather forecasting
Lam, R., Sanchez-Gonzalez, A., Willson, M., Wirnsberger, P., Fortunato, M., Alet, F., Ravuri, S., Ewalds, T., Eaton-Rosen, Z., Hu, W., et al. (2023) · Science · doi:10.1126/science.adi2336 · Cited by 1,466
MetNet: A Neural Weather Model for Precipitation ForecastingPreprint
Sønderby, C. K., Espeholt, L., Heek, J., Dehghani, M., Oliver, A., Salimans, T., Agrawal, S., Hickey, J. J., & Kalchbrenner, N. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.12140 · Cited by 507
Complete ERA5 global atmospheric reanalysis
Service, C. C. C. (2023) · European Centre for Medium-Range Weather Forecasts · doi:10.24381/cds.143582cf · Cited by 238
The evolution of the ECMWF hybrid data assimilation system
Bonavita, M., Hólm, E. V., Isaksen, L., & Fisher, M. (2015) · Quarterly Journal of the Royal Meteorological Society · doi:10.1002/qj.2652 · Cited by 222
Self-citationThe Ceaseless Wind: An Introduction to the Theory of Atmospheric Motion
Dutton, J. A. (1976) · Cited by 197
Can Artificial Intelligence‐Based Weather Prediction Models Simulate the Butterfly Effect?
Selz, T., & Craig, G. C. (2023) · Geophysical Research Letters · doi:10.1029/2023gl105747 · Cited by 109
Chen, L., Fei, D., Hu, Y., Wang, Z., & Wang, F. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i1.25105 · Cited by 72
Numerical Weather Prediction Basics: Models, Numerical Methods, and Data Assimilation
Pu, Z., & Kalnay, E. (2018) · Springer eBooks · doi:10.1007/978-3-642-40457-3_11-1 · Cited by 55
Research works printed but not linked: 17
- Ben‐Bouallegue Z. Clare M. C. Magnusson L. Gascon E. Maier‐Gerber M. Janoušek M. et al. (2023).The rise of data‐driven weather forecasting. arXiv preprint arXiv:2307.1012.
- Bi K. Xie L. Zhang H. Chen X. Gu X. &Tian Q.(2022).Pangu‐weather: A 3D high‐resolution model for fast and accurate global weather forecast. arXiv preprint arXiv:2211.02556. · arXiv
- 10.1038/s41586‐023‐06185‐3 · DOI · Paper page
- 10.1038/s41612‐023‐00387‐2 · DOI
- Chen K. Han T. Gong J. Bai L. Ling F. Luo J.‐J. et al. (2023).FengWu: Pushing the skillful global medium‐range weather forecast beyond 10 days lead. arXiv preprint arXiv:2304.02948. · arXiv
- Dosovitskyi A. Beyer L. Kolesnikov A. Weissenborn D. Zhai X. Unterthiner T. et al. (2021).An image is worth 16×16 words: Transformers for image recognition at scale. arXiv:2010.11929. · arXiv
- Keisler R.(2022).Forecasting global weather with graph neural networks. arXiv preprint arXiv:2202.07575. · arXiv
- Knapp K. R., 2018, International best track archive for climate stewardship (IBTrACS) project, version 4
- Krasnopolsky V., 2023, Review: Using machine learning for data assimilation, model physics, and post‐processing model outputs, 513
- 10.1175/1520‐0493(1974)102<0409:tsomcf>2.0.co;2 · DOI
- Pathak J. Subramanian S. Harrington P. Raja S. Chattopadhyay A. Mardani M. et al. (2022).Fourcastnet: A global data‐driven high‐resolution weather model using adaptive Fourier neural operators. arXiv preprint arXiv:2202.11214. · arXiv
- Sanchez‐Gonzalez A., 2020, International Conference on Machine Learning, 8459
- 10.1038/s41612‐022‐00269‐z · DOI
- 10.22541/essoar.169111361.10601606/v1 · DOI
- 10.1175/15200493(1997)125<3297:EFANAT>2.0.CO;2 · DOI
- Tung N. Brandstetter J. Kapoor A. Gupta J. K. &Grover A.(2023).ClimaX: A foundation model for weather and climate. arXiv preprint arXiv:2301.10343. · arXiv
- Vaswani A. Shazeer N. Parmar N. Uszkoreit J. Jones L. Gomez A. N. et al. (2017).Attention is all you need. arXiv:1706.03762. · arXiv
Cited by
Sorted by citations
Neural general circulation models for weather and climate
Kochkov, D., Yuval, J., Langmore, I., Norgaard, P., Smith, J., Mooers, G., Klöwer, M., Lottes, J., Rasp, S., Düben, P., et al. (2024) · Nature · doi:10.1038/s41586-024-07744-y · Cited by 454
“A recent study 16 highlighted that Pangu misrepresents the vertical structure of the geostrophic and ageostrophic winds and noted a deterioration at longer lead times.”
BackgroundMachine learning for the physics of climate
Bracco, A., Brajard, J., Dijkstra, H. A., Hassanzadeh, P., Lessig, C., & Monteleoni, C. (2024) · Nature Reviews Physics · doi:10.1038/s42254-024-00776-3 · Cited by 79
Forecasting the eddying ocean with a deep neural network
Cui, Y., Wu, R., Zhang, X., Zhu, Z., Liu, B., Shi, J., Chen, J., Liu, H., Zhou, S., Liang, S., et al. (2025) · Nature Communications · doi:10.1038/s41467-025-57389-2 · Cited by 84
Can AI weather models predict out-of-distribution gray swan tropical cyclones?
Sun, Y. Q., Hassanzadeh, P., Zand, M., Chattopadhyay, A., Weare, J., & Abbot, D. S. (2025) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2420914122 · Cited by 83
Lawal, Z. K., Yassin, H., Lai, D. T. C., & Idris, A. C. (2025) · Measurement · doi:10.1016/j.measurement.2025.117383 · Cited by 48
Metadata
- Authors
- Bonavita, M.
- Affiliations
- European Centre for Medium-Range Weather Forecasts (GB)
- Corresponding author
- Bonavita, Massimo
- ORCID
- Bonavita, Massimo
- Venue
- Geophysical Research Letters
- Publisher
- American Geophysical Union (AGU)
- Published
- 2024-06-14
- Volume
- 51
- Issue
- 12
- ISSN
- 0094-8276 (ISSN-L)
- 1944-8007
- OpenAlex
- W4399668152
- Cited by
- 155
- References
- 13
- Citations by year
- 2026: 68 · 2025: 73 · 2024: 14
- FWCI
- 28.29
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- Geophysical Research Letters (publishedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Topics
- Physical Sciences › Earth and Planetary Sciences › Atmospheric Science › Meteorological Phenomena and Simulations
- Physical Sciences › Environmental Science › Environmental Engineering › Hydrological Forecasting Using AI
- Physical Sciences › Physics and Astronomy › Statistical and Nonlinear Physics › Model Reduction and Neural Networks
- Keywords
- machine learning weather prediction, Pangu-Weather, FourCastNet, GraphCast, physical consistency, forecast skill, physics-based modeling, data-driven prediction models
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: On Some Limitations of Current Machine Learning Weather Prediction Models. https://app.cochat.ai/papers/10.1029/2023gl107377
- The paper
- Bonavita, M. (2024). On Some Limitations of Current Machine Learning Weather Prediction Models. Geophysical Research Letters. https://doi.org/10.1029/2023gl107377
