A machine learning model that outperforms conventional global subseasonal forecast models
Open accessPublished version of a preprint
The DOI matches its Crossref record; 102 of the 102 references it cites were found in a registry or an index; 24 of the 74 sources its claims cite were checked for support.
Abstract
Skillful subseasonal forecasts are crucial for various sectors of society but pose a grand scientific challenge. Recently, machine learning-based weather forecasting models outperform the most successful numerical weather predictions generated by the European Centre for Medium-Range Weather Forecasts (ECMWF), but have not yet surpassed conventional models at subseasonal timescales. This paper introduces FuXi Subseasonal-to-Seasonal (FuXi-S2S), a machine learning model that provides global daily mean forecasts up to 42 days, encompassing five upper-air atmospheric variables at 13 pressure levels and 11 surface variables. FuXi-S2S, trained on 72 years of daily statistics from ECMWF ERA5 reanalysis data, outperforms the ECMWF's state-of-the-art Subseasonal-to-Seasonal model in ensemble mean and ensemble forecasts for total precipitation and outgoing longwave radiation, notably enhancing global precipitation forecast. The improved performance of FuXi-S2S can be primarily attributed to its superior capability to capture forecast uncertainty and accurately predict the Madden-Julian Oscillation (MJO), extending the skillful MJO prediction from 30 days to 36 days. Moreover, FuXi-S2S not only captures realistic teleconnections associated with the MJO but also emerges as a valuable tool for discovering precursor signals, offering researchers insights and potentially establishing a new paradigm in Earth system science research.
Overview
CoChat Summary
TL;DR
FuXi-S2S is a 51-member machine learning subseasonal forecasting system trained on ERA5 that extends global daily forecasts to 42 days and is evaluated against ECMWF S2S reforecasts. The paper reports better ensemble-mean and probabilistic skill for total precipitation and outgoing longwave radiation, plus improved Madden–Julian Oscillation prediction and an earlier forecast of the 2022 Pakistan floods.
Key findings
- FuXi-S2S outperforms ECMWF S2S for ensemble-mean forecasts of total precipitation (TP) and outgoing longwave radiation (OLR), with significantly higher temporal anomaly correlation coefficients (TCC) for TP and OLR across lead times of 3, 4, 5, 6, 3–4, and 5–6 weeks.
- For MJO prediction, FuXi-S2S keeps bivariate correlation above the skill threshold of 0.5 for up to 36 days, extending skillful prediction from 30 days for ECMWF S2S.
- For the 2022 Pakistan floods, FuXi-S2S predicted rainfall of at least 4 standard deviations above climatology for forecasts initialized on July 21, 2022, about 4 weeks in advance; the observed rainfall anomaly was about 6 standard deviations above climatology.
- The paper attributes much of the improved skill to flow-dependent perturbations for ensemble generation and reports that increasing the ensemble size from 51 to 101 members further improves performance.
Methods
- The model is trained on 67 years of ERA5 daily statistics (1950–2016) and evaluated on 2017–2021 testing data, with 2022 used for the Pakistan flood case study and comparison with ECMWF S2S real-time forecasts.
- FuXi-S2S is an autoregressive encoder-perturbation-decoder model that generates 51-member ensemble forecasts of 76 variables (5 upper-air variables at 13 pressure levels and 11 surface variables) at 1.5° spatial resolution and daily temporal resolution, out to 42 days.
- Skill is assessed with detrended TCC for deterministic forecasts, RPSS for probabilistic forecasts, BSS for extreme precipitation, RMM bivariate COR for MJO, and saliency maps to interpret the Pakistan flood prediction.
Limitations
- The paper states that FuXi-S2S currently has coarser spatial resolution than ECMWF S2S (1.5° vs 36 km) and predicts daily mean values only up to 50 hPa, lacking some useful variables such as daily maximum and minimum temperatures.
- The authors note that their Pakistan flood interpretability analysis focuses on gradients with respect to SST, and that other precursor signals such as U, V, and Z anomalies are not examined in detail; they say this is for future research.
- Evident from the text: evaluation is limited to 2017–2021 testing data for most metrics and a single highlighted extreme event in 2022, so broader out-of-sample generalization beyond these periods is not directly shown.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures





Verification
Claims the paper makes
10 Valid · 8 Not valid · 21 Not checked · 92 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“However, these systems often exhibit considerable biases13–17, particularly in predicting extreme events18.”
§ A machine learning model that outperforms conventional global subseasonal forecast models > Introduction Subseasonal forecasting, which predicts weather patterns from 2 to 6 weeks in advance, bridges a critical gap between short-term weather forecasts, typically up to 15 days, and longer-term climate forecasts that extend to seasonal and longer timescales1. Forecasting at this intermediate subseasonal timescale is indispensable for a variety of applications, including agricultural planning, disaster preparedness, mitigating impacts of extreme events such as heatwaves, droughts, floods, and cold spells, and water resource management2–5. Despite its significant socioeconomic benefits, subseasonal forecasting has historically not received sufficient attention compared to medium-range weather and climate predictions. This gap existed because accurate subseasonal forecasts were once considered nearly impossible. Subseasonal forecasts are particularly challenging as they rely on both atmospheric initial conditions, essential in short-term weather forecasts, and boundary conditions at the Earth’s surface, key to seasonal and climate forecasts6,7. However, neither of these conditions provides sufficient predictability, leaving subseasonal forecasts in a so-called predictability desert. Despite these challenges, recent advances in both physical and statistical modeling have enabled the regular production of subseasonal forecasts globally. Nonetheless, there remains a ongoing, strong demand for their further development to support informed decision-making across various sectors. Developing an ensemble prediction system (EPS) based on traditional physics-based numerical weather prediction (NWP) models is a widely acknowledged and effective method for enhancing subseasonal forecast accuracy8,9. Major forecasting centers have implemented such EPS for subseasonal forecasts3,10–12. However, these systems often exhibit considerable biases13–17, particularly in predicting extreme events18. The two primary challenges in this field are ensuring an adequate ensemble size within computational constraints and designing ensemble perturbations that accurately reflect uncertainty in key atmospheric and oceanic variability19. Enlarging the ensemble size is beneficial for forecast performance20–22, but the substantial computational costs typically limit ensemble sizes to between 4 and 51 members across 11 international forecasting centers12. Given these computational limitations, machine learning model emerges as a promising alternative for direct subseasonal forecasting23. Machine learning models have the advantages of significantly higher computational efficiency, facilitating the generation of a large number of ensemble members which are crucial for prediction skill and reliability24. Recent advancements in machine learning for medium-range weather forecasting25–31 have demonstrated that machine learning models can outperform the high-resolution forecasts (HRES) generated by the European Centre for Medium-Range Weather Forecasts (ECMWF), widely considered as the most accurate global weather forecasts32. Machine learning models have achieved made significant strides in medium-range weather forecasting and seasonal forecasting33, but their success in subseasonal forecasting has been less pronounced8,34,35. This shortfall primarily stems from the limited range of variables incorporated into the models, and more importantly, from the inadequate methods employed for ensemble generation. Conventional machine learning techniques for ensemble forecasting, such as introducing random perturbations into initial conditions and altering model structures, overlook the background flow and consequently lead to rapid reduction in ensemble spread. The inadequate representation of the complexities limits the performance of these prior machine learning-based subseasonal forecasting models, which do not yet rival that of traditional EPS based on NWP models. To overcome these challenges, we introduce the FuXi Subseasonal-to-Seasonal (FuXi-S2S) model, representing a significant advancement in machine learning for subseasonal forecasting. This model is designed to generate global daily mean forecasts for 42 days from initialization. Unlike previous models that incorporated a limited set of variables, it incorporates a comprehensive suite of variables, instead of a couple of variables in previous models: 5 upper-air atmospheric variables at 13 pressure levels and 11 surface variables. Furthermore, it features a innovative perturbation module specifically designed to generate flow-dependent perturbations for ensemble forecasting. This module leverages vast amounts of historical data to learn probability distributions, thereby introducing flow-dependent perturbations directly into the model’s hidden features. Compared to conventional NWP ensemble forecasting methods, which often struggle with constructing initial condition perturbations due to the complexities of multivariate interactions and the need to maintain dynamic balance and ensemble spread in simulations36, our approach of introducing perturbations directly into the model’s latent space, presenting an effective alternative. This perturbation module significantly enhances the performance of the FuXi-S2S forecasts, as demonstrated in Supplementary Fig. 1. More details about the FuXi-S2S model architecture are available in “Methods”. Remarkably, FuXi-S2S outperforms the ECMWF Subseasonal to Seasonal (S2S) ensemble, which is recognized as the most skillful S2S modeling system, in producing both the ensemble mean and probabilistic forecasts5,37. Its efficacy is particularly evident in extreme total precipitation (TP) forecasting, as exemplified by its accurate forecasts for the 2022 Pakistan floods. Such capability is closely related to FuXi-S2S’s improved prediction of the Madden–Julian Oscillation (MJO)38,39, a key driver of global climate patterns, extending the skillful MJO prediction from 30 days to 36 days. These results further confirm that the notable improvement in FuXi-S2S’s performance can be primarily attributed to the innovative perturbation module for ensemble generation. Another promising result is the ability of the FuXi-S2S model to identify potential precursor signals to physical processes. Beyond mere accuracy, in many applications involving machine learning forecasts, it is imperative to understand and validate the decision-making mechanisms of these models. Such understanding not only leads to enhanced trust in the models’ predictions but also increases the likelihood of implementing effective actions, particularly in mitigating the risks associated with extreme events. Therefore, interpreting machine learning models to align their reasoning with established knowledge becomes crucial. Recent developments in explainable machine learning (XML)40–45 methods have facilitated this interpretation. This study delves into the 2022 Pakistan floods, investigating the FuXi-S2S model’s predictions to identify key geographic regions that significantly impact its predictive accuracy. This is achieved through the generation and analysis of saliency maps46, wherein the identified regions in close alignment with insights from previous studies47. Therefore, we argue that FuXi-S2S transcends traditional NWP models in terms of accuracy and speed, potentially unveiling previously unrecognized processes within Earth’s system in subseasonal forecasting48,49.
What the sources say
- Not matchedcites [13] Nowak, K., Webb, R., Cifelli, R. & Brekke, L. Sub-seasonal climate forecast rodeo. In Proc. 2017 AGU Fall Meeting, 11–15 (New Orleans, LA, 2017).
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Partly supportedcites [14] Monhart, S. et al. Skill of subseasonal forecasts in Europe: Effect of bias correction and downscaling using surface observations. J. Geophys. Res. Atmos. 123, 7999–8016 (2018). sourceExperimental — accuracy not yet measured
Our copy clearly supports the bias half of the claim: the Introduction states that 'Despite the recent improvements of subseasonal forecasts mentioned above, the forecast models still exhibit systematic biases and are not capable to capture all local characteristics,' and Section 4.1/4.2 document negative temperature biases, a general wet precipitation bias, and U-shaped rank histograms indicating overconfident (not fully reliable) ensembles in the ECMWF subseasonal system. What our copy does not support is the qualifier 'particularly in predicting extreme events': searches for extremes, high quantiles/tails and severe-event skill returned only upper-tercile ROC skill for weekly aggregates (Section 4.1) and the statement that QM corrects 'the frequency and intensity of the target variable' (Section 3.1), never a claim that biases are especially large for extremes. The verification is also limited to weekly means/sums at 1,637 European stations for lead times 5-32 days, not to extreme-event prediction. (LLM, conf=0.72)
- Partly supportedcites [15] Hwang, J., Orenstein, P., Cohen, J., Pfeiffer, K. & Mackey, L. Improving subseasonal forecasting in the western us with machine learning. In Proc. of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2325–2335 (2019). sourceExperimental — accuracy not yet measured
Our copy touches bias in an operational subseasonal dynamical ensemble only procedurally: Section 5.1 (p. 5) reconstructs a 'debiased CFSv2 forecast' by subtracting long-term CFS reforecast averages and adding observed 1999-2010 averages, and Appendix B (p. 11) details the debiasing of CFSv2 forecasts (the official contest baseline 'averages the forecasts of four model initializations'), which supports that such systems carry bias requiring removal. Nothing in our copy states that these biases are 'considerable', and no passage addresses extremes being especially biased; the paper instead frames subseasonal difficulty as limited skillful forecast horizon from chaos (Introduction, p. 1) and a 'predictability desert' more difficult than short-term weather or long-term climate forecasting (Section 1.1, p. 2). The 'particularly in predicting extreme events' element of the claim is therefore not supported by this source. (LLM, conf=0.65)
- Could not verifycites [16] Vitart, F. et al. Outcomes of the WMO prize challenge to improve subseasonal to seasonal predictions using artificial intelligence. Bull. Am. Meteorol Soc. 103, E2878–E2886 (2022). sourceExperimental — accuracy not yet measured
Our copy of the WMO S2S-AI Challenge paper never characterises the biases of subseasonal ensemble prediction systems, let alone their behaviour for extreme events: the Abstract (p. 1) says only that 'the skill at this time range is often low, or only marginally better than climatology or persistence forecasts', and the extremes are mentioned purely as a motivation ('changes in risks of extreme events'). The nearest statements are procedural — the 'Results from the competition' section notes the ECMWF benchmark 'had been bias corrected by computing the tercile thresholds from the model's reforecast climatology', and the Abstract/Conclusions credit the winning entries with 'improved calibration of the ECMWF raw forecast outputs' — which imply the raw output needed correction but do not assert considerable biases or any extreme-event focus. Searches for 'biases/bias correction of ensemble prediction systems', 'extreme events forecast skill', 'systematic errors/poor calibration' and 'why S2S prediction is difficult' returned no passage stating the claim, and none contradicting it. (LLM, conf=0.68)
- Could not verifycites [17] Mouatadid, S. et al. Adaptive bias correction for improved subseasonal forecasting. Nat. Commun. 14, 3482 (2023). sourceExperimental — accuracy not yet measured
The Introduction reports persistent systematic errors in subseasonal temperature and precipitation forecasts and mentions the importance of forecasting extreme weather events. The Results describe average model biases, but neither section says that biases are particularly pronounced when predicting extreme events. Searched for extreme-event and extreme-precipitation forecast bias. (LLM, conf=0.70)
- Partly supportedcites [18] Domeisen, D. I. et al. Advances in the subseasonal prediction of extreme events: relevant case studies across the globe. Bull. Am. Meteorol Soc. 103, 1473–1501 (2022). sourceExperimental — accuracy not yet measured
Our copy does address bias in subseasonal forecasting of extremes: the Discussion (lines 677–690) states the authors had to use anomalies 'to correct the systematic bias and model drift' and that 'region- and process-specific biases and drifts are likely still present', and the following paragraph (lines 691–705) calls bias-correction at subseasonal timescales 'non-trivial'. The Conclusions (lines 643–646) also note heatwaves are anticipated 3–4 weeks ahead 'though often with a reduced amplitude'. However, the paper characterises the shortfall mostly as limited predictability/skill of a single ECMWF S2S system rather than asserting that subseasonal ensemble systems generally 'often exhibit considerable biases, particularly in predicting extreme events'; the magnitude and the cross-system generality of the claim are not stated in our copy. (LLM, conf=0.68)
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Recent advancements in machine learning for medium-range weather forecasting25–31 have demonstrated that machine learning models can outperform the high-resolution forecasts (HRES) generated by the European Centre for Medium-Range Weather Forecasts (ECMWF), widely considered as the most accurate global weather forecasts32.”
§ A machine learning model that outperforms conventional global subseasonal forecast models > Introduction Subseasonal forecasting, which predicts weather patterns from 2 to 6 weeks in advance, bridges a critical gap between short-term weather forecasts, typically up to 15 days, and longer-term climate forecasts that extend to seasonal and longer timescales1. Forecasting at this intermediate subseasonal timescale is indispensable for a variety of applications, including agricultural planning, disaster preparedness, mitigating impacts of extreme events such as heatwaves, droughts, floods, and cold spells, and water resource management2–5. Despite its significant socioeconomic benefits, subseasonal forecasting has historically not received sufficient attention compared to medium-range weather and climate predictions. This gap existed because accurate subseasonal forecasts were once considered nearly impossible. Subseasonal forecasts are particularly challenging as they rely on both atmospheric initial conditions, essential in short-term weather forecasts, and boundary conditions at the Earth’s surface, key to seasonal and climate forecasts6,7. However, neither of these conditions provides sufficient predictability, leaving subseasonal forecasts in a so-called predictability desert. Despite these challenges, recent advances in both physical and statistical modeling have enabled the regular production of subseasonal forecasts globally. Nonetheless, there remains a ongoing, strong demand for their further development to support informed decision-making across various sectors. Developing an ensemble prediction system (EPS) based on traditional physics-based numerical weather prediction (NWP) models is a widely acknowledged and effective method for enhancing subseasonal forecast accuracy8,9. Major forecasting centers have implemented such EPS for subseasonal forecasts3,10–12. However, these systems often exhibit considerable biases13–17, particularly in predicting extreme events18. The two primary challenges in this field are ensuring an adequate ensemble size within computational constraints and designing ensemble perturbations that accurately reflect uncertainty in key atmospheric and oceanic variability19. Enlarging the ensemble size is beneficial for forecast performance20–22, but the substantial computational costs typically limit ensemble sizes to between 4 and 51 members across 11 international forecasting centers12. Given these computational limitations, machine learning model emerges as a promising alternative for direct subseasonal forecasting23. Machine learning models have the advantages of significantly higher computational efficiency, facilitating the generation of a large number of ensemble members which are crucial for prediction skill and reliability24. Recent advancements in machine learning for medium-range weather forecasting25–31 have demonstrated that machine learning models can outperform the high-resolution forecasts (HRES) generated by the European Centre for Medium-Range Weather Forecasts (ECMWF), widely considered as the most accurate global weather forecasts32. Machine learning models have achieved made significant strides in medium-range weather forecasting and seasonal forecasting33, but their success in subseasonal forecasting has been less pronounced8,34,35. This shortfall primarily stems from the limited range of variables incorporated into the models, and more importantly, from the inadequate methods employed for ensemble generation. Conventional machine learning techniques for ensemble forecasting, such as introducing random perturbations into initial conditions and altering model structures, overlook the background flow and consequently lead to rapid reduction in ensemble spread. The inadequate representation of the complexities limits the performance of these prior machine learning-based subseasonal forecasting models, which do not yet rival that of traditional EPS based on NWP models. To overcome these challenges, we introduce the FuXi Subseasonal-to-Seasonal (FuXi-S2S) model, representing a significant advancement in machine learning for subseasonal forecasting. This model is designed to generate global daily mean forecasts for 42 days from initialization. Unlike previous models that incorporated a limited set of variables, it incorporates a comprehensive suite of variables, instead of a couple of variables in previous models: 5 upper-air atmospheric variables at 13 pressure levels and 11 surface variables. Furthermore, it features a innovative perturbation module specifically designed to generate flow-dependent perturbations for ensemble forecasting. This module leverages vast amounts of historical data to learn probability distributions, thereby introducing flow-dependent perturbations directly into the model’s hidden features. Compared to conventional NWP ensemble forecasting methods, which often struggle with constructing initial condition perturbations due to the complexities of multivariate interactions and the need to maintain dynamic balance and ensemble spread in simulations36, our approach of introducing perturbations directly into the model’s latent space, presenting an effective alternative. This perturbation module significantly enhances the performance of the FuXi-S2S forecasts, as demonstrated in Supplementary Fig. 1. More details about the FuXi-S2S model architecture are available in “Methods”. Remarkably, FuXi-S2S outperforms the ECMWF Subseasonal to Seasonal (S2S) ensemble, which is recognized as the most skillful S2S modeling system, in producing both the ensemble mean and probabilistic forecasts5,37. Its efficacy is particularly evident in extreme total precipitation (TP) forecasting, as exemplified by its accurate forecasts for the 2022 Pakistan floods. Such capability is closely related to FuXi-S2S’s improved prediction of the Madden–Julian Oscillation (MJO)38,39, a key driver of global climate patterns, extending the skillful MJO prediction from 30 days to 36 days. These results further confirm that the notable improvement in FuXi-S2S’s performance can be primarily attributed to the innovative perturbation module for ensemble generation. Another promising result is the ability of the FuXi-S2S model to identify potential precursor signals to physical processes. Beyond mere accuracy, in many applications involving machine learning forecasts, it is imperative to understand and validate the decision-making mechanisms of these models. Such understanding not only leads to enhanced trust in the models’ predictions but also increases the likelihood of implementing effective actions, particularly in mitigating the risks associated with extreme events. Therefore, interpreting machine learning models to align their reasoning with established knowledge becomes crucial. Recent developments in explainable machine learning (XML)40–45 methods have facilitated this interpretation. This study delves into the 2022 Pakistan floods, investigating the FuXi-S2S model’s predictions to identify key geographic regions that significantly impact its predictive accuracy. This is achieved through the generation and analysis of saliency maps46, wherein the identified regions in close alignment with insights from previous studies47. Therefore, we argue that FuXi-S2S transcends traditional NWP models in terms of accuracy and speed, potentially unveiling previously unrecognized processes within Earth’s system in subseasonal forecasting48,49.
What the sources say
- Partly supportedcites [25] Hu, Y., Chen, L., Wang, Z. & Li, H. SwinVRNN: A data-driven ensemble forecasting model via learned distribution perturbation. J. Adv. Model. Earth Syst. 15, e2022MS003211 (2023). sourceExperimental — accuracy not yet measured
Our copy does report a data-driven model beating ECMWF's operational high-resolution model — Section 3.2 (p. 6) identifies the IFS baseline as 'the state-of-the-art operational NWP model of the European Center for Medium-range Weather Forecasting (ECMWF), and runs at 9 km horizontal resolution with 137 vertical levels', and Section 3.5 (p. 13) states 'our models outperform IFS on the two surface fields T2M and TP with a clear margin at all the forecasting lead times up to 5 days' (echoed in the Introduction p. 2-3 and Conclusions p. 16). But the outperformance is bounded: only T2M and TP, only within 5-day lead, only on WeatherBench at 5.625° resolution, with T850 merely comparable and 'slightly surpasses it at 5-day forecast', and no claimed advantage on Z500 or at longer leads (14-day results are extrapolations without IFS comparison). The unqualified claim that ML models 'can outperform the high-resolution forecasts generated by ECMWF' therefore overstates the source's scope; the accompanying 'most accurate global weather forecasts' characterization is credited to a different citation (32) and is not made in our copy. (LLM, conf=0.78)
- Partly supportedcites [26] Kurth, T. et al. FourCastNet: Accelerating Global High-Resolution Weather Forecasting Using Adaptive Fourier Neural Operators. In Proc. of the Platform for Advanced Scientific Computing Conference (2022). sourceExperimental — accuracy not yet measured
Section 3.2.2 'Data-driven surrogates' is the closest passage: it states FourCastNet 'was the first DL model to close the resolution disparity significantly and has demonstrated superior forecast accuracy compared to operational weather models at short lead times on certain atmospheric variables', which supports the general thrust but only with those qualifiers (short lead times, certain variables, 'operational weather models' rather than ECMWF HRES specifically) that the citing sentence drops. The Abstract frames the paper's own result as 'approaching state-of-the-art accuracy' while being five orders-of-magnitude faster, and the Figure 1 caption's explicit outperformance is against the baseline DLWP model, not against the downsampled IFS. So the unclaimed-equal statement that ML models 'can outperform the high-resolution forecasts (HRES) generated by ECMWF' overstates what this source says. (LLM, conf=0.78)
- Supportedcites [27] Lam, R. et al. Learning skillful medium-range global weather forecasting. Science 382, eadi2336 (2023). source
The Forecast verification results (p. 5) report that the machine-learning model GraphCast outperformed ECMWF’s HRES on 90.3% of 1,380 evaluated targets. The Introduction identifies HRES as ECMWF’s top operational deterministic forecast system. This supports the claim that a machine-learning model can outperform HRES. (LLM, conf=0.94)
- Supportedcites [28] Bi, K. et al. Accurate medium-range global weather forecasting with 3D neural networks. Nature 619, 533–538 (2023). source
The paper’s ‘Tracking tropical cyclones’ section reports that its machine-learning model, Pangu-Weather, tracked 88 cyclones more accurately than ECMWF-HRES. It cautions that the comparison used different initial-condition data: reanalysis data for Pangu-Weather and IFS data for HRES. This supports the claim that a machine-learning model can outperform HRES, within that comparison. (LLM, conf=0.90)
- Supportedcites [29] Chen, L. et al. Fuxi: a cascade machine learning forecasting system for 15-day global weather forecast. Npj Clim. Atmos. Sci. 6, 1–11 (2023). source
The Abstract (p.1) states that 'state-of-the-art ML models ... have superior performance compared to the European Centre for Medium-Range Weather Forecasts (ECMWF)'s high-resolution forecast (HRES)' and that 'ML models have outperformed HRES in 10-day forecasts with a spatial resolution of 0.25°'. The Introduction (p.1) reinforces this with SwinRDM, Pangu-Weather and GraphCast each described as outperforming ECMWF HRES, and the Results ('Deterministic forecast metrics comparison') report that 'both FuXi and GraphCast significantly outperform ECMWF HRES'. The framing clause about HRES being the most accurate global forecast also appears in our copy (Introduction, citing Haiden et al.), so nothing attributed here exceeds the source. (LLM, conf=0.92)
- Not checked — no full text of the sourcecites [30] Zhong, X. et al. Fuxi-extreme: improving extreme rainfall and wind forecasts with diffusion model (2023). source
- Supportedcites [31] Nguyen, T. et al. Scaling transformer neural networks for skillful and reliable medium-range weather forecasting. Preprint at https://arxiv.org/abs/2312.03876 (2023). source
The Introduction (p. 2) states that Pangu-Weather 'was the first model to outperform operational IFS' and that GraphCast improved further, and Section 4.1 (p. 8) explicitly frames its comparison against 'IFS HRES, the state-of-the-art numerical forecasting system'. Appendix C.1 (p. 18) reports the outcome: 'Stormer achieves lower RMSE compared to the IFS model for most variables... To the best of our knowledge, Stormer is the first model trained on 1.40625° data to surpass IFS.' The claim's weak, capability-level form ('can outperform') matches what our copy demonstrates; the only caveat is that the paper's own model loses to IFS ENS on some variables/short lead times, but that ensemble comparison is not what the citing sentence attributes. (LLM, conf=0.85)
- Not checked — no full text of the sourcecites [32] Haiden, T. et al. Evaluation of ECMWF forecasts, including the 2021 upgrade. ECMWF Technical Memorandum No. 884 (European Centre for Medium-Range Weather Forecasts, 2021). https://doi.org/10.21957/90pgicjk4. source
boostedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“The inadequate representation of the complexities limits the performance of these prior machine learning-based subseasonal forecasting models, which do not yet rival that of traditional EPS based on NWP models.”
§ A machine learning model that outperforms conventional global subseasonal forecast models > Introduction Subseasonal forecasting, which predicts weather patterns from 2 to 6 weeks in advance, bridges a critical gap between short-term weather forecasts, typically up to 15 days, and longer-term climate forecasts that extend to seasonal and longer timescales1. Forecasting at this intermediate subseasonal timescale is indispensable for a variety of applications, including agricultural planning, disaster preparedness, mitigating impacts of extreme events such as heatwaves, droughts, floods, and cold spells, and water resource management2–5. Despite its significant socioeconomic benefits, subseasonal forecasting has historically not received sufficient attention compared to medium-range weather and climate predictions. This gap existed because accurate subseasonal forecasts were once considered nearly impossible. Subseasonal forecasts are particularly challenging as they rely on both atmospheric initial conditions, essential in short-term weather forecasts, and boundary conditions at the Earth’s surface, key to seasonal and climate forecasts6,7. However, neither of these conditions provides sufficient predictability, leaving subseasonal forecasts in a so-called predictability desert. Despite these challenges, recent advances in both physical and statistical modeling have enabled the regular production of subseasonal forecasts globally. Nonetheless, there remains a ongoing, strong demand for their further development to support informed decision-making across various sectors. Developing an ensemble prediction system (EPS) based on traditional physics-based numerical weather prediction (NWP) models is a widely acknowledged and effective method for enhancing subseasonal forecast accuracy8,9. Major forecasting centers have implemented such EPS for subseasonal forecasts3,10–12. However, these systems often exhibit considerable biases13–17, particularly in predicting extreme events18. The two primary challenges in this field are ensuring an adequate ensemble size within computational constraints and designing ensemble perturbations that accurately reflect uncertainty in key atmospheric and oceanic variability19. Enlarging the ensemble size is beneficial for forecast performance20–22, but the substantial computational costs typically limit ensemble sizes to between 4 and 51 members across 11 international forecasting centers12. Given these computational limitations, machine learning model emerges as a promising alternative for direct subseasonal forecasting23. Machine learning models have the advantages of significantly higher computational efficiency, facilitating the generation of a large number of ensemble members which are crucial for prediction skill and reliability24. Recent advancements in machine learning for medium-range weather forecasting25–31 have demonstrated that machine learning models can outperform the high-resolution forecasts (HRES) generated by the European Centre for Medium-Range Weather Forecasts (ECMWF), widely considered as the most accurate global weather forecasts32. Machine learning models have achieved made significant strides in medium-range weather forecasting and seasonal forecasting33, but their success in subseasonal forecasting has been less pronounced8,34,35. This shortfall primarily stems from the limited range of variables incorporated into the models, and more importantly, from the inadequate methods employed for ensemble generation. Conventional machine learning techniques for ensemble forecasting, such as introducing random perturbations into initial conditions and altering model structures, overlook the background flow and consequently lead to rapid reduction in ensemble spread. The inadequate representation of the complexities limits the performance of these prior machine learning-based subseasonal forecasting models, which do not yet rival that of traditional EPS based on NWP models. To overcome these challenges, we introduce the FuXi Subseasonal-to-Seasonal (FuXi-S2S) model, representing a significant advancement in machine learning for subseasonal forecasting. This model is designed to generate global daily mean forecasts for 42 days from initialization. Unlike previous models that incorporated a limited set of variables, it incorporates a comprehensive suite of variables, instead of a couple of variables in previous models: 5 upper-air atmospheric variables at 13 pressure levels and 11 surface variables. Furthermore, it features a innovative perturbation module specifically designed to generate flow-dependent perturbations for ensemble forecasting. This module leverages vast amounts of historical data to learn probability distributions, thereby introducing flow-dependent perturbations directly into the model’s hidden features. Compared to conventional NWP ensemble forecasting methods, which often struggle with constructing initial condition perturbations due to the complexities of multivariate interactions and the need to maintain dynamic balance and ensemble spread in simulations36, our approach of introducing perturbations directly into the model’s latent space, presenting an effective alternative. This perturbation module significantly enhances the performance of the FuXi-S2S forecasts, as demonstrated in Supplementary Fig. 1. More details about the FuXi-S2S model architecture are available in “Methods”. Remarkably, FuXi-S2S outperforms the ECMWF Subseasonal to Seasonal (S2S) ensemble, which is recognized as the most skillful S2S modeling system, in producing both the ensemble mean and probabilistic forecasts5,37. Its efficacy is particularly evident in extreme total precipitation (TP) forecasting, as exemplified by its accurate forecasts for the 2022 Pakistan floods. Such capability is closely related to FuXi-S2S’s improved prediction of the Madden–Julian Oscillation (MJO)38,39, a key driver of global climate patterns, extending the skillful MJO prediction from 30 days to 36 days. These results further confirm that the notable improvement in FuXi-S2S’s performance can be primarily attributed to the innovative perturbation module for ensemble generation. Another promising result is the ability of the FuXi-S2S model to identify potential precursor signals to physical processes. Beyond mere accuracy, in many applications involving machine learning forecasts, it is imperative to understand and validate the decision-making mechanisms of these models. Such understanding not only leads to enhanced trust in the models’ predictions but also increases the likelihood of implementing effective actions, particularly in mitigating the risks associated with extreme events. Therefore, interpreting machine learning models to align their reasoning with established knowledge becomes crucial. Recent developments in explainable machine learning (XML)40–45 methods have facilitated this interpretation. This study delves into the 2022 Pakistan floods, investigating the FuXi-S2S model’s predictions to identify key geographic regions that significantly impact its predictive accuracy. This is achieved through the generation and analysis of saliency maps46, wherein the identified regions in close alignment with insights from previous studies47. Therefore, we argue that FuXi-S2S transcends traditional NWP models in terms of accuracy and speed, potentially unveiling previously unrecognized processes within Earth’s system in subseasonal forecasting48,49.
What the sources say
- Partly supportedcites [8] Weyn, J. A., Durran, D. R., Caruana, R. & Cresswell-Clay, N. Sub-seasonal forecasting with a large ensemble of deep-learning weather prediction models. J. Adv. Model. Earth Syst. 13, e2021MS002502 (2021). sourceExperimental — accuracy not yet measured
Weyn et al. (2021) support the inferiority half of the statement only for their own DLWP system: the Abstract (p. 2) says the DLWP ensemble is "only modestly inferior" to the ECMWF S2S ensemble at 4 and 5-6 weeks and that "at shorter lead times, the ECMWF ensemble performs better than DLWP", and the Conclusions (p. 26) state "our current data-driven DLWP model is worse than operational NWP models for the deterministic prediction of synoptic-scale weather patterns" (see also §5.2.2, p. 21). What the paper does not support is the generalization to "prior machine learning-based subseasonal forecasting models" as a class: its Introduction (p. 3) surveys earlier ML work (Dueben & Bauer 2018; Scher & Messori 2019; Weyn et al. 2019) without ranking it against NWP ensembles, and it explicitly recognizes "other successful machine-learning approaches to S2S forecasting (Hwang et al., 2019)". The attribution of the shortfall to "inadequate representation of the complexities" is the citing authors' framing, not a comparative claim about prior models in this source. (LLM, conf=0.72)
- Partly supportedcites [34] He, S., Li, X., DelSole, T., Ravikumar, P. & Banerjee, A. Sub-seasonal climate forecasting via machine learning: challenges, analysis, and advances. Proc. AAAI Conf. Artif. Intell. 35, 169–177 (2021). sourceExperimental — accuracy not yet measured
The Introduction says that high-dimensional, spatially correlated climate data and limited training data challenge machine-learning subseasonal forecasts. The Results (§6.1) compare the models with climatology and other climate baselines, but our copy does not establish that they perform worse than traditional NWP-based ensemble prediction systems. (LLM, conf=0.81)
- Could not verifycites [35] Kiefer, S. M., Lerch, S., Ludwig, P. & Pinto, J. G. Can machine learning models be a suitable tool for predicting central European cold winter weather on subseasonal to seasonal time scales? Artif. Intell. Earth Syst. 2, 1–16 (2023). sourceExperimental — accuracy not yet measured
Our copy of DOI 10.1175/aies-23masthead is the journal masthead for Artificial Intelligence for the Earth Systems: it contains only the journal's scope statement (Artificial Intelligence for the Earth Systems, p. 1) and lists of editors and AMS staff (pp. 1-2). Searching for 'subseasonal forecasting machine learning ensemble prediction system skill' returned no passage discussing ML-based subseasonal forecasts or their skill relative to NWP ensemble prediction systems. Nothing in our copy supports or contradicts the claim, so it cannot be assessed against this source. (LLM, conf=0.80)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources that could be checked couldn’t confirm or dispute this claim.
In the paper
“This is achieved through the generation and analysis of saliency maps46, wherein the identified regions in close alignment with insights from previous studies47.”
§ A machine learning model that outperforms conventional global subseasonal forecast models > Introduction Subseasonal forecasting, which predicts weather patterns from 2 to 6 weeks in advance, bridges a critical gap between short-term weather forecasts, typically up to 15 days, and longer-term climate forecasts that extend to seasonal and longer timescales1. Forecasting at this intermediate subseasonal timescale is indispensable for a variety of applications, including agricultural planning, disaster preparedness, mitigating impacts of extreme events such as heatwaves, droughts, floods, and cold spells, and water resource management2–5. Despite its significant socioeconomic benefits, subseasonal forecasting has historically not received sufficient attention compared to medium-range weather and climate predictions. This gap existed because accurate subseasonal forecasts were once considered nearly impossible. Subseasonal forecasts are particularly challenging as they rely on both atmospheric initial conditions, essential in short-term weather forecasts, and boundary conditions at the Earth’s surface, key to seasonal and climate forecasts6,7. However, neither of these conditions provides sufficient predictability, leaving subseasonal forecasts in a so-called predictability desert. Despite these challenges, recent advances in both physical and statistical modeling have enabled the regular production of subseasonal forecasts globally. Nonetheless, there remains a ongoing, strong demand for their further development to support informed decision-making across various sectors. Developing an ensemble prediction system (EPS) based on traditional physics-based numerical weather prediction (NWP) models is a widely acknowledged and effective method for enhancing subseasonal forecast accuracy8,9. Major forecasting centers have implemented such EPS for subseasonal forecasts3,10–12. However, these systems often exhibit considerable biases13–17, particularly in predicting extreme events18. The two primary challenges in this field are ensuring an adequate ensemble size within computational constraints and designing ensemble perturbations that accurately reflect uncertainty in key atmospheric and oceanic variability19. Enlarging the ensemble size is beneficial for forecast performance20–22, but the substantial computational costs typically limit ensemble sizes to between 4 and 51 members across 11 international forecasting centers12. Given these computational limitations, machine learning model emerges as a promising alternative for direct subseasonal forecasting23. Machine learning models have the advantages of significantly higher computational efficiency, facilitating the generation of a large number of ensemble members which are crucial for prediction skill and reliability24. Recent advancements in machine learning for medium-range weather forecasting25–31 have demonstrated that machine learning models can outperform the high-resolution forecasts (HRES) generated by the European Centre for Medium-Range Weather Forecasts (ECMWF), widely considered as the most accurate global weather forecasts32. Machine learning models have achieved made significant strides in medium-range weather forecasting and seasonal forecasting33, but their success in subseasonal forecasting has been less pronounced8,34,35. This shortfall primarily stems from the limited range of variables incorporated into the models, and more importantly, from the inadequate methods employed for ensemble generation. Conventional machine learning techniques for ensemble forecasting, such as introducing random perturbations into initial conditions and altering model structures, overlook the background flow and consequently lead to rapid reduction in ensemble spread. The inadequate representation of the complexities limits the performance of these prior machine learning-based subseasonal forecasting models, which do not yet rival that of traditional EPS based on NWP models. To overcome these challenges, we introduce the FuXi Subseasonal-to-Seasonal (FuXi-S2S) model, representing a significant advancement in machine learning for subseasonal forecasting. This model is designed to generate global daily mean forecasts for 42 days from initialization. Unlike previous models that incorporated a limited set of variables, it incorporates a comprehensive suite of variables, instead of a couple of variables in previous models: 5 upper-air atmospheric variables at 13 pressure levels and 11 surface variables. Furthermore, it features a innovative perturbation module specifically designed to generate flow-dependent perturbations for ensemble forecasting. This module leverages vast amounts of historical data to learn probability distributions, thereby introducing flow-dependent perturbations directly into the model’s hidden features. Compared to conventional NWP ensemble forecasting methods, which often struggle with constructing initial condition perturbations due to the complexities of multivariate interactions and the need to maintain dynamic balance and ensemble spread in simulations36, our approach of introducing perturbations directly into the model’s latent space, presenting an effective alternative. This perturbation module significantly enhances the performance of the FuXi-S2S forecasts, as demonstrated in Supplementary Fig. 1. More details about the FuXi-S2S model architecture are available in “Methods”. Remarkably, FuXi-S2S outperforms the ECMWF Subseasonal to Seasonal (S2S) ensemble, which is recognized as the most skillful S2S modeling system, in producing both the ensemble mean and probabilistic forecasts5,37. Its efficacy is particularly evident in extreme total precipitation (TP) forecasting, as exemplified by its accurate forecasts for the 2022 Pakistan floods. Such capability is closely related to FuXi-S2S’s improved prediction of the Madden–Julian Oscillation (MJO)38,39, a key driver of global climate patterns, extending the skillful MJO prediction from 30 days to 36 days. These results further confirm that the notable improvement in FuXi-S2S’s performance can be primarily attributed to the innovative perturbation module for ensemble generation. Another promising result is the ability of the FuXi-S2S model to identify potential precursor signals to physical processes. Beyond mere accuracy, in many applications involving machine learning forecasts, it is imperative to understand and validate the decision-making mechanisms of these models. Such understanding not only leads to enhanced trust in the models’ predictions but also increases the likelihood of implementing effective actions, particularly in mitigating the risks associated with extreme events. Therefore, interpreting machine learning models to align their reasoning with established knowledge becomes crucial. Recent developments in explainable machine learning (XML)40–45 methods have facilitated this interpretation. This study delves into the 2022 Pakistan floods, investigating the FuXi-S2S model’s predictions to identify key geographic regions that significantly impact its predictive accuracy. This is achieved through the generation and analysis of saliency maps46, wherein the identified regions in close alignment with insights from previous studies47. Therefore, we argue that FuXi-S2S transcends traditional NWP models in terms of accuracy and speed, potentially unveiling previously unrecognized processes within Earth’s system in subseasonal forecasting48,49.
What the sources say
- Could not verifycites [47] Dunstone, N. et al. Windows of opportunity for predicting seasonal climate extremes highlighted by the Pakistan floods of 2022. Nat. Commun. 14, 6544 (2023). sourceExperimental — accuracy not yet measured
The Results section (‘Using physical understanding to build forecast confidence’; page unavailable) reports agreement between predicted atmospheric circulation anomalies and historical teleconnections and skill maps. The Discussion describes maps of forecast rainfall anomalies, not saliency maps. Searches for saliency maps and attribution maps found no passage assessing whether regions identified by saliency maps align with previous studies. (LLM, conf=0.76)
plainstates an associationSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Could not verify
Every source the paper cites for this claim was checked and supports it.
In the paper
“Similar to FuXi, we utilize an autoregressive, multi-step loss function to mitigate cumulative errors over long lead times, as outlined in Lam et al.27.”
§ A machine learning model that outperforms conventional global subseasonal forecast models > Methods > FuXi-S2S model Most state-of-the-art machine learning models utilized in medium-range weather forecasting are built upon encoder-decoder84 architectures27–29,85. These structures are favored due to their proficiency in processing and generating sequential and spatial data. Within these architectures, the encoder processes key features from the input data and transforms them into a compressed and abstract representation in the latent space. The decoder then utilizes this representation to generate weather forecasts. The primary objective of training these models is to minimize differences between the model’s output and the target data. However, the standard encoder-decoder structures are inherently deterministic, producing identical forecasts for the same inputs, which limits their applicability in generating ensemble forecasts. To overcome this limitation, we introduce the FuXi-S2S model, drawing inspiration from Variational Autoencoders (VAEs)86–88. VAEs are inherently probabilistic, making them well-suited for tasks that require uncertainty quantification. Like VAEs, the FuXi-S2S model’s encoder does not merely generate a static hidden feature from input data. Instead, it transforms input data into a Gaussian distribution in the latent space, which captures the probabilistic characteristics of the data, along with a static hidden feature. Then, the decoder combines samples from the Gaussian distribution with the static hidden feature to generate forecasts. This methodology effectively captures the inherent uncertainty in the data, thereby enabling the generation of ensemble predictions under identical input conditions by repeatedly sampling from the Gaussian distribution. For better understanding, we draw analogies between these machine learning techniques and the conventional terminology in ensemble weather/subseasonal forecasting. In our model, the static hidden feature forms the basis for deterministic forecasts, while sampling from the Gaussian distribution serves as a perturbation module. This module introduces flow-dependent perturbations into the model’s hidden feature, facilitating the generation of ensemble forecasts. The FuXi-S2S model, illustrated in Fig. 5a, consists of three primary components: an encoder P, a perturbation module, and a decoder. The encoder, processing predicted weather parameters from two preceding time steps, with each time step representing 1 day, as FuXi-S2S is designed to forecast daily mean values. Specifically, it takes and as inputs into a two-dimensional (2D) convolution layer with a kernel size of two, which reduces the dimensions of the input data by half. Following this, the hidden feature ht (with dimensions of 1536 × 60 × 120) is derived from 12 repeated transformer blocks. The input to the encoder is a data cube that combines both upper-air and surface variables, with dimensions of 2 × 76 × 121 × 240. These dimensions represent two preceding time steps (t − 1 and t), the number of input variables, and the latitude (H) and longitude (W) grid points, respectively. To account for the accumulation of forecast error over time, the forecast lead time (t) is also included in the encoder’s input. Besides ht, the encoder also generates a low-rank multivariate Gaussian distribution, , characterized by a mean vector μt (128 × 60 × 120), a covariance matrix σt (1536 × 60 × 120), and a diagonal covariance matrix diagt (128 × 60 × 120). Intermediate perturbation vectors (, dimension: 128 × 60 × 120) are sampled from this Gaussian distribution (). These vectors, after being weighted by a learned weight vector, yield the final perturbation vectors zt (dimension: 1536 × 60 × 120). The decoder then processes the perturbed hidden features () through 24 transformer blocks and a fully connected layer, resulting in the final ensemble output . The number of ensemble members generated equals the number of samples drawn from the Gaussian distribution . The FuXi-S2S model’s training primarily focuses on constructing a Gaussian distribution that accurately represents the uncertainty in the model’s predictions. A significant challenge in this process is the deviation of the Gaussian distribution derived from the model’s predictions from the Gaussian distribution based on the target data, largely attributable to prediction errors. This challenge is addressed through knowledge distillation, which enables the transfer of information from real-world distributions to those predicted by the model. Within this framework, the encoder Q plays a crucial role, converting the target data into a Gaussian distribution. This distribution serves as a supervisor for the distribution generated by the encoder P, aiming to align both distributions closely by minimizing the Kullback–Leibler (KL) divergence loss (LKL). This KL loss measures the discrepancy between the distributions predicted by both encoders. As illustrated in the Fig. 5b, during the training phase of the FuXi-S2S model, the encoder Q, which shares the network structure with the encoder P, processes a data cube containing target weather parameters from a preceding and the current time steps: Xt and Xt+1. It predicts a low-rank multivariate Gaussian distribution () similar to the encoder P. Intermediate perturbation vectors are sampled from the encoder Q’s distribution () during training (see Fig. 5b), and from the encoder P’s distribution () during testing (see Fig. 5a). These vectors have dimensions of 128 × 60 × 120. Additionally, an L1 loss is computed between the model’s output () and the target Xt+1. Therefore, the overall loss function at each autoregressive step is thus determined by the following equation:1where λ, a tune-able coefficient balancing LKL and L1, is set to 1 × 10−4 in this study. The design of this loss function serves two purposes: the first term ensures the perturbation vector closely approximates the true data distribution, while the second term ensures the prediction unaffected by any perturbation vectors zt. In this study, we employ 51 ensemble members for subseasonal ensemble forecasting. As illustrated in Supplementary Fig. 15, the FuXi-S2S model, when enhanced with flow-dependent perturbations incorporated into its hidden features, demonstrates considerably improved forecast performance compared to the FuXi-S2S model that combines Perlin noise in the initial conditions with fixed perturbations added to the hidden features. Notably, the addition of Perlin noise results in only marginal improvements in forecast accuracy when the ensemble size is small. However, with larger ensemble sizes, such as the 51 members in this study, the addition of Perlin noise does not enhance forecast accuracy. Similar to FuXi, we utilize an autoregressive, multi-step loss function to mitigate cumulative errors over long lead times, as outlined in Lam et al.27. The training process follows an autoregressive training regime and a curriculum training schedule, incrementally increasing the number of autoregressive steps from 1 to 17. Each autoregressive step undergoes 1000 gradient descent updates, resulting in a total number of 17,000 training steps. The training process utilizes 8 Nvidia A100 graphics processing units (GPUs), each employing a batch size of 1. Optimization is performed using the AdamW89,90 optimizer with the following parameters: β1 = 0.9 and β2 = 0.95, an initial learning rate of 2.5 × 10−4, and a weight decay coefficient of 0.1. The optimization hyperparameters used for training are summarized in Supplementary Table 1.
What the sources say
- Supportedcites Lam et al.27 Lam, R. et al. Learning skillful medium-range global weather forecasting. Science 382, eadi2336 (2023). source
Section 4.3 (p. 32) says GraphCast used autoregressive training to improve forecasts beyond one step, computing a loss at each forecast step. Section 7.3.2 (p. 65) reports that training with fewer autoregressive steps traded longer-lead-time accuracy for shorter-lead-time accuracy. Together, these support the cited description of a multi-step loss used to limit errors at longer lead times. (LLM, conf=0.91)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
51 more claims
Create a free account to see all 131 claims and to verify the 28 references still open.
- It may state two separate claims as one. 47
- Probably not a substantial claim. 33
- Borderline: one of the tests for a substantial claim is unclear. 1
- The quote only partly states this claim. 1
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ A machine learning model that outperforms conventional global subseasonal forecast models > Abstract
“FuXi-S2S, trained on 72 years of daily statistics from ECMWF ERA5 reanalysis data, outperforms the ECMWF’s state-of-the-art Subseasonal-to-Seasonal model”
§ A machine learning model that outperforms conventional global subseasonal forecast models > Methods > Data
“The model’s training relies on 67 years of data spanning from 1950 to 2016, while evaluation involves a 5-year dataset from 2017 to 2021.”
Cited works that were retracted
Works this paper cites that were later retracted, from the retraction notices publishers deposit. Major when the notice came out before this paper was published.
A note, not an issue: for example, a cited work that was retracted only after this paper was published, which its authors could not have known.
In the paper
“Inrecent years, increasing efforts have focused on applying machine learning models to improve MJO forecasts, either by post-processing dynamical forecasts 61 – 63 or through direct forecasting 44,64,65 .”
Cited work 10.1038/s41612-021-00214-6 · retracted 2025-02-14 · notice 10.1038/s41612-025-00937-w · this paper published 2024-07-30
Citations
Cites
1 retracted
RETRACTED ARTICLE: Machine learning prediction of the Madden-Julian oscillation
Silini, R., Barreiro, M., & Masoller, C. (2021) · npj Climate and Atmospheric Science · doi:10.1038/s41612-021-00214-6 · Cited by 21 · cited 1×
“Inrecent years, increasing efforts have focused on applying machine learning models to improve MJO forecasts, either by post-processing dynamical forecasts 61 – 63 or through direct forecasting 44,64,65 .”
BackgroundRetractedBreiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538
Hersbach, H., Bell, B., Berrisford, P., Hirahara, S., Horányi, A., Muñoz‐Sabater, J., Nicolas, J., Peubey, C., Radu, R., Schepers, D., et al. (2020) · Quarterly Journal of the Royal Meteorological Society · doi:10.1002/qj.3803 · Cited by 31,882 · cited 1×
“It operates at a remarkable temporal resolution of 1h and a horizontal resolution of approximately 31 km, covering data from January 1950 to the present day 78 .”
BackgroundAdvances in neural information processing systems 7
Cios, K., & Shields, M. (1997) · Neurocomputing · doi:10.1016/s0925-2312(97)90018-7 · Cited by 22,059
Decoupled Weight Decay Regularization
Loshchilov, I., & Hutter, F. (2017) · arXiv · doi:10.48550/arxiv.1711.05101 · Cited by 12,522
On the Properties of Neural Machine Translation: Encoder–Decoder Approaches
Cho, K., van Merrienboer, B., Bahdanau, D., & Bengio, Y. (2014) · Proceedings of SSST-8, Eighth Workshop on Syntax, Semantics and Structure in Statistical Translation · doi:10.3115/v1/w14-4012 · Cited by 7,099
On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation
Bach, S., Binder, A., Montavon, G., Klauschen, F., Müller, K.-R., & Samek, W. (2015) · PLOS ONE · doi:10.1371/journal.pone.0130140 · Cited by 4,524
The NCEP Climate Forecast System Version 2
Saha, S., Moorthi, S., Wu, X., Wang, J., Nadiga, S., Tripp, P., Behringer, D., Hou, Y.-T., Chuang, H.-Y., Iredell, M., et al. (2014) · Journal of Climate · doi:10.1175/jcli-d-12-00823.1 · Cited by 3,354
Wheeler, M. C., & Hendon, H. H. (2004) · Monthly Weather Review · doi:10.1175/1520-0493(2004)132<1917:aarmmi>2.0.co;2 · Cited by 2,891
An Introduction to Variational Autoencoders
Kingma, D. P., & Welling, M. (2019) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000056 · Cited by 2,609
Research works printed but not linked: 8
- National Academies of Sciences. Next generation earth system prediction: strategies for subseasonal to seasonal forecasts (National Academies Press, Washington, DC, 2016).
- Nowak, K., Webb, R., Cifelli, R. & Brekke, L. Sub-seasonal climate forecast rodeo. In Proc. 2017 AGU Fall Meeting, 11–15 (New Orleans, LA, 2017).
- Domeisen, D. I. et al. Advances in the subseasonal prediction of extreme events: relevant case studies across the globe. Bull. Am. Meteorol Soc. 103, 1473–1501 (2022). · DOI · Paper page
- Simonyan, K., Vedaldi, A. & Zisserman, A. Deep inside convolutional networks: Visualising image classification models and saliency maps. In Proc. International Conference on Learning Representations (2013).
- Chattopadhyay, A. & Hassanzadeh, P. Long-term instability of deep learning-based digital twins of the climate system: Cause and solution. In Proc. APS March Meeting Abstracts (2023).
- Walker, G. T. Correlations in seasonal variations of weather. viii, a further study of world weather. Men. Indian Meteor. Dept. 24, 275–332 (1924).
- Huang, R. H. Influence of the heat source anomaly over the western tropical Pacific for the subtropical high over East Asia. In Proc. International Conference on the General Circulation of East Asia. Chendu, China, April 10–15, 1987, 40–50 (1987).
- Kingma, D. P. & Ba, J. Adam: A method for stochastic optimization. In Proc. International Conference on Learning Representations (2015).
Cited by
Guo, Q., He, Z., Wang, Z., Qiao, S., Zhu, J., & Chen, J. (2024) · Water · doi:10.3390/w16192870 · Cited by 155
Machine Learning Methods for Weather Forecasting: A Survey
Zhang, H., Liu, Y., Zhang, C., & Li, N. (2025) · Atmosphere · doi:10.3390/atmos16010082 · Cited by 113
Machine learning for the physics of climate
Bracco, A., Brajard, J., Dijkstra, H. A., Hassanzadeh, P., Lessig, C., & Monteleoni, C. (2024) · Nature Reviews Physics · doi:10.1038/s42254-024-00776-3 · Cited by 79
A global estimate of multiecosystem photosynthesis losses under microplastic pollution
Zhu, R., Zhang, Z., Zhang, N., Zhong, H., Zhou, F., Zhang, X., Liu, C., Huang, Y., Yuan, Y., Wang, Y., et al. (2025) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2423957122 · Cited by 118
“…highlight the previously unrecognized role of MPs in threatening seafood security and contributing to hidden hunger particularly in coastal communities that rely heavily on seafood ( 34 , 37 ), thereby jeop-ardizing SDG goals 2 and 14, i.e., zero hunger and the sustainable use of marine resources.”
A machine learning model for hub-height short-term wind speed prediction
Zhang, Z., Lin, L., Gao, S., Wang, J., Zhao, H., & Yu, H. (2025) · Nature Communications · doi:10.1038/s41467-025-58456-4 · Cited by 98
“…Center for Medium-Range Weather Forecasts - High Resolution) is one of the most accurate numerical weather prediction models in the world 60,61 , and it is widely used in global meteorological researchand weather forecasting byits high resolution, advanced data assimilation techniques, and…”
Metadata
- Authors
- Chen, L., Zhong, X., Li, H., Wu, J., Lu, B., Chen, D., Xie, S.-P., Wu, L., Chao, Q., Lin, C., Hu, Z., & Qi, Y.
- Affiliations
- Fudan University (CN)
- China Meteorological Administration (CN)
- University of Gothenburg (SE)
- Scripps Institution of Oceanography (US)
- Corresponding author
- Li, Hao
- Qi, Yuan
- ORCID
- Venue
- Nature Communications
- Published
- 2024-07-30
- Volume
- 15
- Issue
- 1
- Pages
- 6425-6425
- ISSN
- 2041-1723 (ISSN-L)
- PMC
- PMC11289088
- OpenAlex
- W4401112323
- Semantic Scholar
- 6727224b0a153a953665af558f00afa4032b2b99
- Cited by
- 105
- References
- 89
- Citations by year
- 2026: 41 · 2025: 56 · 2024: 8
- FWCI
- 19.17
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by-nc-nd
- creativecommons.org/licenses/by-nc-nd/4.0 (tdm)
- creativecommons.org/licenses/by-nc-nd/4.0 (vor)
- Locations
- Nature Communications (publishedVersion, cc-by-nc-nd)
- Research Square (acceptedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- eScholarship (California Digital Library) (submittedVersion, cc-by-nc-nd)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- National Natural Science Foundation of China (42175052)
- Topics
- Physical Sciences › Earth and Planetary Sciences › Atmospheric Science › Meteorological Phenomena and Simulations
- Physical Sciences › Environmental Science › Global and Planetary Change › Climate variability and models
- Physical Sciences › Environmental Science › Environmental Engineering › Hydrological Forecasting Using AI
- Keywords
- subseasonal-to-seasonal forecasting, Madden-Julian Oscillation, MJO prediction, precipitation forecasting, outgoing longwave radiation, forecast uncertainty, teleconnections, precursor signals, ERA5 reanalysis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (qwen3.8-omni-flash and gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: A machine learning model that outperforms conventional global subseasonal forecast models. https://app.cochat.ai/papers/10.1038/s41467-024-50714-1
- The paper
- Chen, L., Zhong, X., Li, H., Wu, J., Lu, B., Chen, D., Xie, S.-P., Wu, L., Chao, Q., Lin, C., Hu, Z., & Qi, Y. (2024). A machine learning model that outperforms conventional global subseasonal forecast models. Nature Communications. https://doi.org/10.1038/s41467-024-50714-1
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.