WeatherBench: A Benchmark Data Set for Data‐Driven Weather Forecasting
Open accessAbstract
Data‐driven approaches, most prominently deep learning, have become powerful tools for prediction in many domains. A natural question to ask is whether data‐driven methods could also be used to predict global weather patterns days in advance. First studies show promise but the lack of a common data set and evaluation metrics make intercomparison between studies difficult. Here we present a benchmark data set for data‐driven medium‐range weather forecasting (specifically 3–5 days), a topic of high scientific interest for atmospheric and computer scientists alike. We provide data derived from the ERA5 archive that has been processed to facilitate the use in machine learning models. We propose simple and clear evaluation metrics which will enable a direct comparison between different methods. Further, we provide baseline scores from simple linear regression techniques, deep learning models, as well as purely physical forecasting models. The data set is publicly available at https://github.com/pangeo‐data/WeatherBench and the companion code is reproducible with tutorials for getting started. We hope that this data set will accelerate research in data‐driven weather forecasting.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Well-curated benchmark datasets enable collaboration between researchers with different expertise, like climate and computer scientists.
“Well-curated benchmark datasets enable collaboration between researchers with different expertise, like climate and computer scientists (Rasp et al., 2020).”
From Artificial intelligence for climate prediction of extremes: State of the art, challenges, and future perspectives (2024) · cites this paper as Rasp et al., 2020
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · see citing works
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · see citing works
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404 · see citing works
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126 · see citing works
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380 · see citing works
Hersbach, H., Bell, B., Berrisford, P., Hirahara, S., Horányi, A., Muñoz‐Sabater, J., Nicolas, J., Peubey, C., Radu, R., Schepers, D., et al. (2020) · Quarterly Journal of the Royal Meteorological Society · doi:10.1002/qj.3803 · Cited by 31,882 · see citing works
Eyring, V., Bony, S., Meehl, G. A., Senior, C. A., Stevens, B., Stouffer, R. J., & Taylor, K. E. (2016) · Geoscientific Model Development · doi:10.5194/gmd-9-1937-2016 · Cited by 11,981 · see citing works
Spatially Extended Tests of a Neural Network Parametrization Trained by Coarse‐Graining
Brenowitz, N. D., & Bretherton, C. S. (2019) · Journal of Advances in Modeling Earth Systems · doi:10.1029/2019ms001711 · Cited by 231 · see citing works
Perraudin, N., Defferrard, M., Kacprzak, T., & Sgier, R. (2019) · Astronomy and Computing · doi:10.1016/j.ascom.2019.03.004 · Cited by 184 · cited 1× · see citing works
“A possible solution in the horizontal could be to use spherical convolutions (Cohen et al., 2018; Perraudin et al., 2019; Jiang et al., 2019) or to feed in latitude information to the network.”
Background
Flagged first, then by citations
Cited by
Sorted by citations
Accurate medium-range global weather forecasting with 3D neural networks
Bi, K., Xie, L., Zhang, H., Chen, X., Gu, X., & Tian, Q. (2023) · Nature · doi:10.1038/s41586-023-06185-3 · Cited by 1,670 · see citing works
Learning skillful medium-range global weather forecasting
Lam, R., Sanchez-Gonzalez, A., Willson, M., Wirnsberger, P., Fortunato, M., Alet, F., Ravuri, S., Ewalds, T., Eaton-Rosen, Z., Hu, W., et al. (2023) · Science · doi:10.1126/science.adi2336 · Cited by 1,466 · see citing works
Towards neural Earth system modelling by integrating artificial intelligence in Earth system science
Irrgang, C., Boers, N., Sonnewald, M., Barnes, E. A., Kadow, C., Staneva, J., & Saynisch-Wagner, J. (2021) · Nature Machine Intelligence · doi:10.1038/s42256-021-00374-3 · Cited by 308 · see citing works
A foundation model for the Earth system
Bodnar, C., Bruinsma, W. P., Lucic, A., Stanley, M., Allen, A., Brandstetter, J., Garvan, P., Riechert, M., Weyn, J. A., Dong, H., et al. (2025) · Nature · doi:10.1038/s41586-025-09005-y · Cited by 267 · see citing works
Self-citationWeatherBench 2: A Benchmark for the Next Generation of Data‐Driven Global Weather Models
Rasp, S., Hoyer, S., Merose, A., Langmore, I., Battaglia, P., Russell, T., Sanchez‐Gonzalez, A., Yang, V., Carver, R., Agrawal, S., et al. (2024) · Journal of Advances in Modeling Earth Systems · doi:10.1029/2023ms004019 · Cited by 245 · see citing works
Self-citationLi, W., & Hsu, C.-Y. (2022) · ISPRS International Journal of Geo-Information · doi:10.3390/ijgi11070385 · Cited by 147 · see citing works
Materia, S., García, L. P., van Straaten, C., O, S., Mamalakis, A., Cavicchia, L., Coumou, D., de Luca, P., Kretschmer, M., & Donat, M. (2024) · WIREs Climate Change · doi:10.1002/wcc.914 · Cited by 130 · see citing works
“Well-curated benchmark datasets enable collaboration between researchers with different expertise, like climate and computer scientists (Rasp et al., 2020).”
Machine Learning Methods for Weather Forecasting: A Survey
Zhang, H., Liu, Y., Zhang, C., & Li, N. (2025) · Atmosphere · doi:10.3390/atmos16010082 · Cited by 113 · see citing works
Dueben, P. D., Schultz, M. G., Chantry, M., Gagne, D. J., Hall, D. M., & McGovern, A. (2022) · Artificial Intelligence for the Earth Systems · doi:10.1175/aies-d-21-0002.1 · Cited by 64 · see citing works
Self-citationSwinVRNN: A Data‐Driven Ensemble Forecasting Model via Learned Distribution Perturbation
Hu, Y., Chen, L., Wang, Z., & Li, H. (2023) · Journal of Advances in Modeling Earth Systems · doi:10.1029/2022ms003211 · Cited by 63 · see citing works
Global extreme heat forecasting using neural weather models
Lopez-Gomez, I., McGovern, A., Agrawal, S., & Hickey, J. (2023) · Artificial Intelligence for the Earth Systems · doi:10.1175/aies-d-22-0035.1 · Cited by 41 · see citing works
On the foundations of Earth foundation models
Zhu, X. X., Xiong, Z., Wang, Y., Stewart, A. J., Heidler, K., Wang, Y., Yuan, Z., Dujardin, T., Xu, Q., & Shi, Y. (2026) · Communications Earth & Environment · doi:10.1038/s43247-025-03127-x · Cited by 26 · see citing works
A Bayesian Deep Learning Approach to Near‐Term Climate Prediction
Luo, X., Nadiga, B. T., Park, J. H., Ren, Y., Xu, W., & Yoo, S. (2022) · Journal of Advances in Modeling Earth Systems · doi:10.1029/2022ms003058 · Cited by 18 · see citing works
Chishtie, F. A., Brunet, D., White, R. H., Michelson, D., Jiang, J., Lucas, V., Ruboonga, E., Imaash, S., Westland, M., Chui, T., et al. (2024) · arXiv · doi:10.48550/arxiv.2411.13496
“Future research will also aim to explore the integration of DI-GNN with other machine learning architectures, including Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), and Transformers, to enhance predictive performance for diverse spatiotemporal datasets [15, 13].”
- Authors
- Rasp, S., Dueben, P. D., Scher, S., Weyn, J. A., Mouatadid, S., & Thuerey, N.
- Affiliations
- Technical University of Munich (DE)
- European Centre for Medium-Range Weather Forecasts (GB)
- Stockholm University (SE)
- University of Washington (US)
- University of Toronto (CA)
- Corresponding author
- Rasp, Stephan
- Venue
- Journal of Advances in Modeling Earth Systems
- Publisher
- American Geophysical Union (AGU)
- Published
- 2020-11
- Volume
- 12
- Issue
- 11
- ISSN
- 1942-2466 (ISSN-L)
- OpenAlex
- W3003709258
- Cited by
- 663
- References
- 9
- Citations by year
- 2026: 149 · 2025: 154 · 2024: 107 · 2023: 82 · 2022: 91 · 2021: 63 · 2020: 17
- FWCI
- 20.73
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Journal of Advances in Modeling Earth Systems (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Geo-Leo e-docs (Deutsche Initiative für Netzwerkinformation) (submittedVersion, cc-by-nd)
- mediaTUM – the media and publications repository of the Technical University Munich (Technical University Munich) (submittedVersion, cc-by)
- mediaTUM – the media and publications repository of the Technical University Munich (Technical University Munich) (submittedVersion, other-oa)
- Funders
- Deutsche Forschungsgemeinschaft
- Seventh Framework Programme (823988)
- Topics
- Physical Sciences › Earth and Planetary Sciences › Atmospheric Science › Meteorological Phenomena and Simulations
- Physical Sciences › Environmental Science › Environmental Engineering › Hydrological Forecasting Using AI
- Physical Sciences › Engineering › Building and Construction › Traffic Prediction and Management Techniques
- Keywords
- medium-range weather forecasting, benchmark dataset, ERA5 reanalysis, forecast evaluation metrics, model intercomparison
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
