Cited by 246Meteorological Phenomena and SimulationsHydrological Forecasting Using AIModel Reduction and Neural Networks

WeatherBench 2: A Benchmark for the Next Generation of Data‐Driven Global Weather Models

Open access
Rasp, Stephan, Hoyer, Stephan, Merose, Alexander, Langmore, Ian, Battaglia, Peter, Russell, Tyler, Sanchez‐Gonzalez, Alvaro, Yang, Vivian +10 more·2024·Journal of Advances in Modeling Earth Systems

Abstract

WeatherBench 2 is an update to the global, medium‐range (1–14 days) weather forecasting benchmark proposed by (Rasp et al., 2020, https://doi.org/10.1029/2020ms002203 ), designed with the aim to accelerate progress in data‐driven weather modeling. WeatherBench 2 consists of an open‐source evaluation framework, publicly available training, ground truth and baseline data as well as a continuously updated website with the latest metrics and state‐of‐the‐art models: https://sites.research.google/weatherbench . This paper describes the design principles of the evaluation framework and presents results for current state‐of‐the‐art physical and data‐driven weather models. The metrics are based on established practices for evaluating weather forecasts at leading operational weather centers. We define a set of headline scores to provide an overview of model performance. In addition, we also discuss caveats in the current evaluation setup and challenges for the future of data‐driven weather forecasting.

Integrity signals

Registry

Crossref · 10.1029/2023ms004019

Cites

54 of 54 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The evaluation set-up follows WeatherBench2, regrids all forecasts to a 1.5° grid using conservative regridding, and averages over all 732 forecasts made at noon and midnight UTC in 2020, which was held out from training data for all machine-learning models.

    “Our evaluation set-up focuses on quantifying accuracy and physical consistency, following WeatherBench212. We regrid all forecasts to a 1.5° grid using conservative regridding, and average over all 732 forecasts made at noon and midnight UTC in the year 2020, which was held-out from training data for all machine-learning models.”

    From Neural general circulation models for weather and climate (2024) · cites this paper as [12]

  • ECMWF's ensemble model achieves the best performance across the majority of lead times.

    “We use ECMWF’s ensemble (ENS) model as a reference baseline as it achieves the best performance across the majority of lead times12.”

    From Neural general circulation models for weather and climate (2024) · cites this paper as [12]

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Authors
Rasp, S., Hoyer, S., Merose, A., Langmore, I., Battaglia, P., Russell, T., Sanchez‐Gonzalez, A., Yang, V., Carver, R., Agrawal, S., Chantry, M., Ben Bouallegue, Z., Dueben, P., Bromberg, C., Sisk, J., Barrington, L., Bell, A., & Sha, F.
Affiliations
  • Google (United States) (US)
  • Google DeepMind (United Kingdom) (GB)
  • Google DeepMind London UK
  • Google Research Mountain View CA USA
  • European Centre for Medium-Range Weather Forecasts (GB)
  • European Centre for Medium‐Range Weather Forecasts Reading UK
Corresponding author
Rasp, Stephan
Venue
Journal of Advances in Modeling Earth Systems
Published
2024-06
Volume
16
Issue
6
ISSN
1942-2466 (ISSN-L)
OpenAlex
W4399779937
Cited by
245
References
54
Citations by year
2026: 111 · 2025: 95 · 2024: 39
FWCI
44.90
Type
journal article
Language
en
Open access
Locations
  • Journal of Advances in Modeling Earth Systems (publishedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
  • arXiv (Cornell University) (cc-by)
Topics
  • Physical Sciences › Earth and Planetary Sciences › Atmospheric Science › Meteorological Phenomena and Simulations
  • Physical Sciences › Environmental Science › Environmental Engineering › Hydrological Forecasting Using AI
  • Physical Sciences › Physics and Astronomy › Statistical and Nonlinear Physics › Model Reduction and Neural Networks
Keywords
medium-range weather forecasting, forecast evaluation metrics

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review