The DOI matches its Crossref record; 54 of the 54 references it cites were found in a registry or an index.
WeatherBench 2: A Benchmark for the Next Generation of Data‐Driven Global Weather Models
Open accessAbstract
WeatherBench 2 is an update to the global, medium‐range (1–14 days) weather forecasting benchmark proposed by (Rasp et al., 2020, https://doi.org/10.1029/2020ms002203 ), designed with the aim to accelerate progress in data‐driven weather modeling. WeatherBench 2 consists of an open‐source evaluation framework, publicly available training, ground truth and baseline data as well as a continuously updated website with the latest metrics and state‐of‐the‐art models: https://sites.research.google/weatherbench . This paper describes the design principles of the evaluation framework and presents results for current state‐of‐the‐art physical and data‐driven weather models. The metrics are based on established practices for evaluating weather forecasts at leading operational weather centers. We define a set of headline scores to provide an overview of model performance. In addition, we also discuss caveats in the current evaluation setup and challenges for the future of data‐driven weather forecasting.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Our evaluation set-up focuses on quantifying accuracy and physical consistency, following WeatherBench212. We regrid all forecasts to a 1.5° grid using conservative regridding, and average over all 732 forecasts made at noon and midnight UTC in the year 2020, which was held-out from training data for all machine-learning models.”
From Neural general circulation models for weather and climate (2024) · cites this paper as [12]
In the citing paper
“We use ECMWF’s ensemble (ENS) model as a reference baseline as it achieves the best performance across the majority of lead times12.”
From Neural general circulation models for weather and climate (2024) · cites this paper as [12]
In the citing paper
“Furthermore, previous work has shown that the performance of MLWP models that are trained on reanalysis can be further improved by fine-tuning using operational data, such as HRES analysis inputs and targets30.”
From Probabilistic weather forecasting with machine learning (2025) · cites this paper as [30]
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Hersbach, H., Bell, B., Berrisford, P., Hirahara, S., Horányi, A., Muñoz‐Sabater, J., Nicolas, J., Peubey, C., Radu, R., Schepers, D., et al. (2020) · Quarterly Journal of the Royal Meteorological Society · doi:10.1002/qj.3803 · Cited by 31,882
Strictly Proper Scoring Rules, Prediction, and Estimation
Gneiting, T., & Raftery, A. E. (2007) · Journal of the American Statistical Association · doi:10.1198/016214506000001437 · Cited by 5,793
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding
Wang, A., Singh, A., Michael, J., Hill, F., Levy, O., & Bowman, S. (2018) · Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP · doi:10.18653/v1/w18-5446 · Cited by 4,222
Atmospheric Modeling, Data Assimilation and Predictability
Kalnay, E. (2002) · doi:10.1017/cbo9780511802270 · Cited by 3,391 · cited 1×
“Current NWP models are based on discretizations of the governing equations describing fluid flow and thermodynamics (Kalnay, 2002).”
BackgroundThe quiet revolution of numerical weather prediction
Bauer, P., Thorpe, A., & Brunet, G. (2015) · Nature · doi:10.1038/nature14956 · Cited by 2,729
Accurate medium-range global weather forecasting with 3D neural networks
Bi, K., Xie, L., Zhang, H., Chen, X., Gu, X., & Tian, Q. (2023) · Nature · doi:10.1038/s41586-023-06185-3 · Cited by 1,670
Pathak, J., Subramanian, S., Harrington, P., Raja, S., Chattopadhyay, A., Mardani, M., Kurth, T., Hall, D., Li, Z., Azizzadenesheli, K., et al. (2022) · arXiv · doi:10.48550/arxiv.2202.11214 · Cited by 1,547
Research works printed but not linked: 7
- Ben‐Bouallegue Z. Clare M. C. A. Magnusson L. Gascon E. Maier‐Gerber M. Janousek M. et al. (2023).The rise of data‐driven weather forecasting(arXiv:2307.10128 [physics]).https://doi.org/10.48550/arXiv.2307.10128 · DOI
- 10.48448/zn7f‐fc64 · DOI
- Kochkov D. Yuval J. Langmore I. Norgaard P. Smith J. Mooers G. et al. (2023).Neural general circulation models(arXiv:2311.07222 [physics]).https://doi.org/10.48550/arXiv.2311.07222 · DOI
- 10.1175/1520‐0469(1963)020〈0130:DNF〉2.0.CO;2 · DOI
- Palmer T. N., 1993, Ensemble prediction. Proc. ECMWF seminar on validation of models over Europe
- 10.1175/1520‐0477(1993)074〈2317:EFANTG〉2.0.CO;2 · DOI
- (WMO), W. M. O, 2019, Manual on the global data‐processing and forecasting system (WMO‐No. 485): Annex IV to the WMO technical regulations
Cited by
Sorted by citations
Neural general circulation models for weather and climate
Kochkov, D., Yuval, J., Langmore, I., Norgaard, P., Smith, J., Mooers, G., Klöwer, M., Lottes, J., Rasp, S., Düben, P., et al. (2024) · Nature · doi:10.1038/s41586-024-07744-y · Cited by 454
“Our evaluation set-up focuses on quantifying accuracy and physical consistency, following WeatherBench2 12 .”
BackgroundMethodSelf-citationProbabilistic weather forecasting with machine learning
Price, I., Sanchez-Gonzalez, A., Alet, F., Andersson, T. R., El-Kadi, A., Masters, D., Ewalds, T., Stott, J., Mohamed, S., Battaglia, P., et al. (2025) · Nature · doi:10.1038/s41586-024-08252-9 · Cited by 434
Self-citationA foundation model for the Earth system
Bodnar, C., Bruinsma, W. P., Lucic, A., Stanley, M., Allen, A., Brandstetter, J., Garvan, P., Riechert, M., Weyn, J. A., Dong, H., et al. (2025) · Nature · doi:10.1038/s41586-025-09005-y · Cited by 267
Early warning of complex climate risk with integrated artificial intelligence
Reichstein, M., Benson, V., Blunk, J., Camps‐Valls, G., Creutzig, F., Fearnley, C. J., Han, B., Kornhuber, K., Rahaman, N., Schölkopf, B., et al. (2025) · Nature Communications · doi:10.1038/s41467-025-57640-w · Cited by 189
Materia, S., García, L. P., van Straaten, C., O, S., Mamalakis, A., Cavicchia, L., Coumou, D., de Luca, P., Kretschmer, M., & Donat, M. (2024) · WIREs Climate Change · doi:10.1002/wcc.914 · Cited by 130
“Efforts to build benchmark datasets are ongoing for weather-scale forecasts where the atmosphere is still deterministic (WeatherBENCH; Rasp et al., 2020, 2023), or multi-decadal timescale where the climate response is largely driven by socioeconomic scenarios (ClimateBENCH; Watson-Parris et al.,…”
Metadata
- Authors
- Rasp, S., Hoyer, S., Merose, A., Langmore, I., Battaglia, P., Russell, T., Sanchez‐Gonzalez, A., Yang, V., Carver, R., Agrawal, S., Chantry, M., Ben Bouallegue, Z., Dueben, P., Bromberg, C., Sisk, J., Barrington, L., Bell, A., & Sha, F.
- Affiliations
- Google (United States) (US)
- Google DeepMind (United Kingdom) (GB)
- Google DeepMind London UK
- Google Research Mountain View CA USA
- European Centre for Medium-Range Weather Forecasts (GB)
- European Centre for Medium‐Range Weather Forecasts Reading UK
- Corresponding author
- Rasp, Stephan
- ORCID
- Venue
- Journal of Advances in Modeling Earth Systems
- Publisher
- American Geophysical Union (AGU)
- Published
- 2024-06
- Volume
- 16
- Issue
- 6
- ISSN
- 1942-2466 (ISSN-L)
- OpenAlex
- W4399779937
- Cited by
- 245
- References
- 54
- Citations by year
- 2026: 111 · 2025: 95 · 2024: 39
- FWCI
- 44.90
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Journal of Advances in Modeling Earth Systems (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Earth and Planetary Sciences › Atmospheric Science › Meteorological Phenomena and Simulations
- Physical Sciences › Environmental Science › Environmental Engineering › Hydrological Forecasting Using AI
- Physical Sciences › Physics and Astronomy › Statistical and Nonlinear Physics › Model Reduction and Neural Networks
- Keywords
- medium-range weather forecasting, forecast evaluation metrics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
