The DOI matches its Crossref record; 54 of the 54 references it cites were found in a registry or an index.
WeatherBench 2: A Benchmark for the Next Generation of Data‐Driven Global Weather Models
Open accessAbstract
WeatherBench 2 is an update to the global, medium‐range (1–14 days) weather forecasting benchmark proposed by (Rasp et al., 2020, https://doi.org/10.1029/2020ms002203 ), designed with the aim to accelerate progress in data‐driven weather modeling. WeatherBench 2 consists of an open‐source evaluation framework, publicly available training, ground truth and baseline data as well as a continuously updated website with the latest metrics and state‐of‐the‐art models: https://sites.research.google/weatherbench . This paper describes the design principles of the evaluation framework and presents results for current state‐of‐the‐art physical and data‐driven weather models. The metrics are based on established practices for evaluating weather forecasts at leading operational weather centers. We define a set of headline scores to provide an overview of model performance. In addition, we also discuss caveats in the current evaluation setup and challenges for the future of data‐driven weather forecasting.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Our evaluation set-up focuses on quantifying accuracy and physical consistency, following WeatherBench212. We regrid all forecasts to a 1.5° grid using conservative regridding, and average over all 732 forecasts made at noon and midnight UTC in the year 2020, which was held-out from training data for all machine-learning models.”
From Neural general circulation models for weather and climate (2024) · cites this paper as [12]
In the citing paper
“We use ECMWF’s ensemble (ENS) model as a reference baseline as it achieves the best performance across the majority of lead times12.”
From Neural general circulation models for weather and climate (2024) · cites this paper as [12]
In the citing paper
“Furthermore, previous work has shown that the performance of MLWP models that are trained on reanalysis can be further improved by fine-tuning using operational data, such as HRES analysis inputs and targets30.”
From Probabilistic weather forecasting with machine learning (2025) · cites this paper as [30]
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Hersbach, H., Bell, B., Berrisford, P., Hirahara, S., Horányi, A., Muñoz‐Sabater, J., Nicolas, J., Peubey, C., Radu, R., Schepers, D., et al. (2020) · Quarterly Journal of the Royal Meteorological Society · doi:10.1002/qj.3803 · Cited by 31,882
Strictly Proper Scoring Rules, Prediction, and Estimation
Gneiting, T., & Raftery, A. E. (2007) · Journal of the American Statistical Association · doi:10.1198/016214506000001437 · Cited by 5,793
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding
Wang, A., Singh, A., Michael, J., Hill, F., Levy, O., & Bowman, S. (2018) · Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP · doi:10.18653/v1/w18-5446 · Cited by 4,222
Atmospheric Modeling, Data Assimilation and Predictability
Kalnay, E. (2002) · doi:10.1017/cbo9780511802270 · Cited by 3,391 · cited 1×
“Current NWP models are based on discretizations of the governing equations describing fluid flow and thermodynamics (Kalnay, 2002).”
BackgroundThe quiet revolution of numerical weather prediction
Bauer, P., Thorpe, A., & Brunet, G. (2015) · Nature · doi:10.1038/nature14956 · Cited by 2,729
Accurate medium-range global weather forecasting with 3D neural networks
Bi, K., Xie, L., Zhang, H., Chen, X., Gu, X., & Tian, Q. (2023) · Nature · doi:10.1038/s41586-023-06185-3 · Cited by 1,670
Pathak, J., Subramanian, S., Harrington, P., Raja, S., Chattopadhyay, A., Mardani, M., Kurth, T., Hall, D., Li, Z., Azizzadenesheli, K., et al. (2022) · arXiv · doi:10.48550/arxiv.2202.11214 · Cited by 1,547
Statistical Methods in the Atmospheric Sciences
Briggs, W. M. (2007) · Journal of the American Statistical Association · doi:10.1198/jasa.2007.s163 · Cited by 1,542
Learning skillful medium-range global weather forecasting
Lam, R., Sanchez-Gonzalez, A., Willson, M., Wirnsberger, P., Fortunato, M., Alet, F., Ravuri, S., Ewalds, T., Eaton-Rosen, Z., Hu, W., et al. (2023) · Science · doi:10.1126/science.adi2336 · Cited by 1,466
Self-citationGneiting, T., Raftery, A. E., Westveld, A. H., & Goldman, T. (2005) · Monthly Weather Review · doi:10.1175/mwr2904.1 · Cited by 1,142
Stochastic representation of model uncertainties in the ECMWF ensemble prediction system
Buizza, R., Milleer, M., & Palmer, T. N. (1999) · Quarterly Journal of the Royal Meteorological Society · doi:10.1002/qj.49712556006 · Cited by 1,103
The predictability of a flow which possesses many scales of motion
Lorenz, E. N. (1969) · Tellus · doi:10.1111/j.2153-3490.1969.tb00444.x · Cited by 1,009
WeatherBench: A Benchmark Data Set for Data‐Driven Weather Forecasting
Rasp, S., Dueben, P. D., Scher, S., Weyn, J. A., Mouatadid, S., & Thuerey, N. (2020) · Journal of Advances in Modeling Earth Systems · doi:10.1029/2020ms002203 · Cited by 663
Self-citationClimaX: A foundation model for weather and climatePreprint
Nguyen, T. T., Brandstetter, J., Kapoor, A., Gupta, J. K., & Grover, A. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2301.10343 · Cited by 573
Neural Networks for Postprocessing Ensemble Weather Forecasts
Rasp, S., & Lerch, S. (2018) · Monthly Weather Review · doi:10.1175/mwr-d-18-0187.1 · Cited by 554
Self-citationAn evaluation of ERA5 precipitation for climate monitoring
Lavers, D. A., Simmons, A., Vamborg, F., & Rodwell, M. J. (2022) · Quarterly Journal of the Royal Meteorological Society · doi:10.1002/qj.4351 · Cited by 488
Neural general circulation models for weather and climate
Kochkov, D., Yuval, J., Langmore, I., Norgaard, P., Smith, J., Mooers, G., Klöwer, M., Lottes, J., Rasp, S., Düben, P., et al. (2024) · Nature · doi:10.1038/s41586-024-07744-y · Cited by 454
Self-citationChallenges and design choices for global weather and climate models based on machine learning
Dueben, P. D., & Bauer, P. (2018) · Geoscientific model development · doi:10.5194/gmd-11-3999-2018 · Cited by 401
Self-citationLearning Mesh-Based Simulation with Graph NetworksPreprint
Pfaff, T., Fortunato, M., Sánchez‐González, Á., & Battaglia, P. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.03409 · Cited by 400
Self-citationFengWu: Pushing the Skillful Global Medium-range Weather Forecast beyond 10 Days Lead
Chen, K., Han, T., Gong, J., Bai, L., Ling, F., Luo, J.-J., Chen, X., Ma, L., Zhang, T., Su, R., et al. (2023) · arXiv · doi:10.48550/arxiv.2304.02948 · Cited by 396
Weyn, J. A., Durran, D. R., & Caruana, R. (2019) · Journal of Advances in Modeling Earth Systems · doi:10.1029/2019ms001705 · Cited by 378
Weyn, J. A., Durran, D. R., & Caruana, R. (2020) · Journal of Advances in Modeling Earth Systems · doi:10.1029/2020ms002109 · Cited by 368
FuXi: a cascade machine learning forecasting system for 15-day global weather forecast
Chen, L., Zhong, X., Zhang, F., Cheng, Y., Xu, Y., Qi, Y., & Li, H. (2023) · npj Climate and Atmospheric Science · doi:10.1038/s41612-023-00512-1 · Cited by 367
Adaptive Fourier Neural Operators: Efficient Token Mixers for Transformers
Guibas, J., Mardani, M., Li, Z., Tao, A., Anandkumar, A., & Catanzaro, B. (2021) · CaltechAUTHORS (California Institute of Technology) · doi:10.48550/arxiv.2111.13587 · Cited by 358
Stensrud, D. J. (2007) · Cambridge University Press eBooks · doi:10.1017/cbo9780511812590 · Cited by 348
Bouallègue, Z. B., Clare, M., Magnusson, L., Gascón, E., Maier‐Gerber, M., Janoušek, M., Rodwell, M. J., Pinault, F., Dramsch, J. S., Lang, S. T. K., et al. (2024) · Bulletin of the American Meteorological Society · doi:10.1175/bams-d-23-0162.1 · Cited by 314
Self-citationWhat Is the Predictability Limit of Midlatitude Weather?
Zhang, F., Sun, Y. Q., Magnusson, L., Buizza, R., Lin, S., Chen, J., & Emanuel, K. (2019) · Journal of the Atmospheric Sciences · doi:10.1175/jas-d-18-0269.1 · Cited by 312
Scher, S. (2018) · Geophysical Research Letters · doi:10.1029/2018gl080704 · Cited by 304
Zhang, F., Bei, N., Rotunno, R., Snyder, C., & Epifanio, C. C. (2007) · Journal of the Atmospheric Sciences · doi:10.1175/jas4028.1 · Cited by 292
Rasp, S., & Thuerey, N. (2021) · Journal of Advances in Modeling Earth Systems · doi:10.1029/2020ms002405 · Cited by 276
Self-citationWhy Should Ensemble Spread Match the RMSE of the Ensemble Mean?
Fortin, V., Abaza, M., Anctil, F., & Turcotte, R. (2014) · Journal of Hydrometeorology · doi:10.1175/jhm-d-14-0008.1 · Cited by 263
Characteristics of Occasional Poor Medium-Range Weather Forecasts for Europe
Rodwell, M. J., Magnusson, L., Bauer, P., Bechtold, P., Bonavita, M., Cardinali, C., Diamantakis, M., Earnshaw, P. D., Garcia-Mendez, A., Isaksen, L., et al. (2013) · Bulletin of the American Meteorological Society · doi:10.1175/bams-d-12-00099.1 · Cited by 226
Deep learning for post-processing ensemble weather forecasts
Grönquist, P., Yao, C., Ben‐Nun, T., Dryden, N., Dueben, P. D., Li, S., & Hoefler, T. (2021) · Philosophical Transactions of the Royal Society A Mathematical Physical and Engineering Sciences · doi:10.1098/rsta.2020.0092 · Cited by 225
Self-citationZamo, M., & Naveau, P. (2018) · Mathematical Geosciences · doi:10.1007/s11004-017-9709-7 · Cited by 199
Deep Learning for Day Forecasts from Sparse ObservationsPreprint
Andrychowicz, M., Espeholt, L., Li, D., Merchant, S., Merose, A., Zyda, F., Agrawal, S., & Kalchbrenner, N. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2306.06079 · Cited by 159
Self-citationSpherical Fourier Neural Operators: Learning Stable Dynamics on the SpherePreprint
Bonev, B., Kurth, T., Hundt, C., Pathak, J., Baust, M., Kashinath, K., & Anandkumar, A. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2306.03838 · Cited by 147
Forecasting Global Weather with Graph Neural Networks
Keisler, R. (2022) · arXiv · doi:10.48550/arxiv.2202.07575 · Cited by 120
Factors Influencing Skill Improvements in the ECMWF Forecasting System
Magnusson, L., & Källén, E. (2013) · Monthly Weather Review · doi:10.1175/mwr-d-12-00318.1 · Cited by 120
Scaling transformer neural networks for skillful and reliable medium-range weather forecasting
Nguyen, T. T., Shah, R., Bansal, H., Arcomano, T., Maulik, R., Kotamarthi, V. R., Foster, I. T., Madireddy, S., & Grover, A. (2024) · neural information processing systems · doi:10.52202/079017-2196 · Cited by 119
A new equitable score suitable for verifying precipitation in numerical weather prediction
Rodwell, M. J., Richardson, D. S., Hewson, T., & Haiden, T. (2010) · Quarterly Journal of the Royal Meteorological Society · doi:10.1002/qj.656 · Cited by 94
Dueben, P. D., Schultz, M. G., Chantry, M., Gagne, D. J., Hall, D. M., & McGovern, A. (2022) · Artificial Intelligence for the Earth Systems · doi:10.1175/aies-d-21-0002.1 · Cited by 64
Self-citationThe Transition from Practical to Intrinsic Predictability of Midlatitude Weather
Selz, T., Riemer, M., & Craig, G. C. (2022) · Journal of the Atmospheric Sciences · doi:10.1175/jas-d-21-0271.1 · Cited by 59
Combining distribution‐based neural networks to predict weather forecast probabilities
Clare, M., Jamil, O., & Morcrette, C. J. (2021) · Quarterly Journal of the Royal Meteorological Society · doi:10.1002/qj.4180 · Cited by 55
Scale‐dependent verification of ensemble forecasts
Jung, T. J., & Leutbecher, M. (2008) · Quarterly Journal of the Royal Meteorological Society · doi:10.1002/qj.255 · Cited by 54
Garg, S., Rasp, S., & Thuerey, N. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2205.00865 · Cited by 45
Self-citationImproving Medium-Range Ensemble Weather Forecasts with Hierarchical Ensemble Transformers
Bouallègue, Z. B., Weyn, J. A., Clare, M., Dramsch, J. S., Dueben, P. D., & Chantry, M. (2023) · Artificial Intelligence for the Earth Systems · doi:10.1175/aies-d-23-0027.1 · Cited by 42
Self-citationEstimating the Intrinsic Limit of Predictability Using a Stochastic Convection Scheme
Selz, T. (2018) · Journal of the Atmospheric Sciences · doi:10.1175/jas-d-17-0373.1 · Cited by 42
Scaling Spherical CNNsPreprint
Esteves, C., Slotine, J.-J. E., & Makadia, A. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2306.05420 · Cited by 24
Finn, T. S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.13924 · Cited by 16
The EUPPBench postprocessing benchmark dataset v1.0Preprint
Demaeyer, J., Bhend, J., Lerch, S., Primo, C., Schaeybroeck, B. V., Atencia, A., Bouallègue, Z. B., Chen, J., Dabernig, M., Evans, G., et al. (2023) · doi:10.5194/essd-2022-465 · Cited by 8
ENS-10: A Dataset For Post-Processing Ensemble Weather ForecastsPreprint
Ashkboos, S., Huang, L., Dryden, N., Ben‐Nun, T., Dueben, P. D., Gianinazzi, L., Kummer, L., & Hoefler, T. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2206.14786 · Cited by 8
Self-citation
Research works printed but not linked: 7
- Ben‐Bouallegue Z. Clare M. C. A. Magnusson L. Gascon E. Maier‐Gerber M. Janousek M. et al. (2023).The rise of data‐driven weather forecasting(arXiv:2307.10128 [physics]).https://doi.org/10.48550/arXiv.2307.10128 · DOI
- 10.48448/zn7f‐fc64 · DOI
- Kochkov D. Yuval J. Langmore I. Norgaard P. Smith J. Mooers G. et al. (2023).Neural general circulation models(arXiv:2311.07222 [physics]).https://doi.org/10.48550/arXiv.2311.07222 · DOI
- 10.1175/1520‐0469(1963)020〈0130:DNF〉2.0.CO;2 · DOI
- Palmer T. N., 1993, Ensemble prediction. Proc. ECMWF seminar on validation of models over Europe
- 10.1175/1520‐0477(1993)074〈2317:EFANTG〉2.0.CO;2 · DOI
- (WMO), W. M. O, 2019, Manual on the global data‐processing and forecasting system (WMO‐No. 485): Annex IV to the WMO technical regulations
Cited by
Sorted by citations
Neural general circulation models for weather and climate
Kochkov, D., Yuval, J., Langmore, I., Norgaard, P., Smith, J., Mooers, G., Klöwer, M., Lottes, J., Rasp, S., Düben, P., et al. (2024) · Nature · doi:10.1038/s41586-024-07744-y · Cited by 454
“Our evaluation set-up focuses on quantifying accuracy and physical consistency, following WeatherBench2 12 .”
BackgroundMethodSelf-citationProbabilistic weather forecasting with machine learning
Price, I., Sanchez-Gonzalez, A., Alet, F., Andersson, T. R., El-Kadi, A., Masters, D., Ewalds, T., Stott, J., Mohamed, S., Battaglia, P., et al. (2025) · Nature · doi:10.1038/s41586-024-08252-9 · Cited by 434
Self-citationA foundation model for the Earth system
Bodnar, C., Bruinsma, W. P., Lucic, A., Stanley, M., Allen, A., Brandstetter, J., Garvan, P., Riechert, M., Weyn, J. A., Dong, H., et al. (2025) · Nature · doi:10.1038/s41586-025-09005-y · Cited by 267
Early warning of complex climate risk with integrated artificial intelligence
Reichstein, M., Benson, V., Blunk, J., Camps‐Valls, G., Creutzig, F., Fearnley, C. J., Han, B., Kornhuber, K., Rahaman, N., Schölkopf, B., et al. (2025) · Nature Communications · doi:10.1038/s41467-025-57640-w · Cited by 189
Materia, S., García, L. P., van Straaten, C., O, S., Mamalakis, A., Cavicchia, L., Coumou, D., de Luca, P., Kretschmer, M., & Donat, M. (2024) · WIREs Climate Change · doi:10.1002/wcc.914 · Cited by 130
“Efforts to build benchmark datasets are ongoing for weather-scale forecasts where the atmosphere is still deterministic (WeatherBENCH; Rasp et al., 2020, 2023), or multi-decadal timescale where the climate response is largely driven by socioeconomic scenarios (ClimateBENCH; Watson-Parris et al.,…”
Metadata
- Authors
- Rasp, S., Hoyer, S., Merose, A., Langmore, I., Battaglia, P., Russell, T., Sanchez‐Gonzalez, A., Yang, V., Carver, R., Agrawal, S., Chantry, M., Ben Bouallegue, Z., Dueben, P., Bromberg, C., Sisk, J., Barrington, L., Bell, A., & Sha, F.
- Affiliations
- Google (United States) (US)
- Google DeepMind (United Kingdom) (GB)
- Google DeepMind London UK
- Google Research Mountain View CA USA
- European Centre for Medium-Range Weather Forecasts (GB)
- European Centre for Medium‐Range Weather Forecasts Reading UK
- Corresponding author
- Rasp, Stephan
- ORCID
- Venue
- Journal of Advances in Modeling Earth Systems
- Publisher
- American Geophysical Union (AGU)
- Published
- 2024-06
- Volume
- 16
- Issue
- 6
- ISSN
- 1942-2466 (ISSN-L)
- OpenAlex
- W4399779937
- Cited by
- 245
- References
- 54
- Citations by year
- 2026: 111 · 2025: 95 · 2024: 39
- FWCI
- 44.90
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Journal of Advances in Modeling Earth Systems (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Earth and Planetary Sciences › Atmospheric Science › Meteorological Phenomena and Simulations
- Physical Sciences › Environmental Science › Environmental Engineering › Hydrological Forecasting Using AI
- Physical Sciences › Physics and Astronomy › Statistical and Nonlinear Physics › Model Reduction and Neural Networks
- Keywords
- medium-range weather forecasting, forecast evaluation metrics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
