CoChat

The suitability of differentiable, physics-informed machine learning hydrologic models for ungauged regions and climate change impact assessment

Open access
Feng, Dapeng, Beck, Hylke, Lawson, Kathryn, Shen, Chaopeng · 2023 · Hydrology and Earth System Sciences

Published version of a preprint

The DOI matches its Crossref record; 67 of the 67 references it cites were found in a registry or an index; 4 of the 27 sources its claims cite were checked for support.

Abstract

As a genre of physics-informed machine learning, differentiable process-based hydrologic models (abbreviated as δ or delta models) with regionalized deep-network-based parameterization pipelines were recently shown to provide daily streamflow prediction performance closely approaching that of state-of-the-art long short-term memory (LSTM) deep networks. Meanwhile, δ models provide a full suite of diagnostic physical variables and guaranteed mass conservation. Here, we ran experiments to test (1) their ability to extrapolate to regions far from streamflow gauges and (2) their ability to make credible predictions of long-term (decadal-scale) change trends. We evaluated the models based on daily hydrograph metrics (Nash–Sutcliffe model efficiency coefficient, etc.) and predicted decadal streamflow trends. For prediction in ungauged basins (PUB; randomly sampled ungauged basins representing spatial interpolation), δ models either approached or surpassed the performance of LSTM in daily hydrograph metrics, depending on the meteorological forcing data used. They presented a comparable trend performance to LSTM for annual mean flow and high flow but worse trends for low flow. For prediction in ungauged regions (PUR; regional holdout test representing spatial extrapolation in a highly data-sparse scenario), δ models surpassed LSTM in daily hydrograph metrics, and their advantages in mean and high flow trends became prominent. In addition, an untrained variable, evapotranspiration, retained good seasonality even for extrapolated cases. The δ models' deep-network-based parameterization pipeline produced parameter fields that maintain remarkably stable spatial patterns even in highly data-scarce scenarios, which explains their robustness. Combined with their interpretability and ability to assimilate multi-source observations, the δ models are strong candidates for regional and global-scale hydrologic simulations and climate change impact assessment.

Overview

CoChat Summary

TL;DR

The paper evaluates differentiable, physics-informed hydrologic models (δ models) against LSTM and a traditional regionalized model for spatial interpolation/extrapolation and long-term trend prediction on the CAMELS dataset. It finds that δ models generalize especially well in ungauged regions, with the two-parameter dynamic version δ(β_t, γ_t) performing best overall and retaining good evapotranspiration seasonality even under extrapolation.

Key findings

  • On the randomized PUB test (531 basins), under Maurer forcings δ(β_t, γ_t) had median NSE 0.64 and LSTM had 0.65; for KGE, δ(β_t) and δ(β_t, γ_t) were 0.66 and 0.65, “essentially the same as LSTM,” but with a smaller spread.
  • Under Daymet forcings in PUB, δ(β_t) and δ(β_t, γ_t) reached median NSE/KGE of 0.68/0.68 and 0.69/0.67, exceeding LSTM’s 0.66/0.66.
  • In the regional holdout PUR test, δ models outperformed LSTM in most daily metrics; with Maurer forcings, median NSE/KGE were 0.55/0.52 for LSTM, 0.56/0.59 for δ, and 0.59/0.61 for δ(β_t, γ_t); with Daymet, they were 0.55/0.51, 0.60/0.61, and 0.62/0.63, respectively.
  • For decadal trends in PUR, LSTM dropped sharply while δ models remained stable: LSTM annual-mean-flow R² fell from 0.87 (PUB) to 0.64 (PUR), and high-flow Q98 R² from 0.76 to 0.27, whereas δ(β_t, γ_t) stayed around 0.88 for annual mean discharge and about 0.77 for Q98 trends.
  • For evapotranspiration, extrapolation had a moderate impact on seasonality and a muted impact on annual mean ET; under PUB/PUR the median correlation between simulated ET and MODIS ET stayed around 0.84 and RMSE around 6.4 mm per 8 d, with δ(γ_t, β_t) showing the best ET seasonality.

Methods

  • The study used differentiable hydrologic models with an HBV backbone and neural-network parameterization; some versions included dynamic parameterization (DP) where β and γ became time-dependent (β_t, γ_t).
  • Experiments were run on CAMELS basins across the contiguous United States: 671 basins for training and a 531-basin subset for reported test performance; PUB used 10 random folds, and PUR divided CONUS into seven regions with seven holdouts.
  • Performance was evaluated using daily hydrograph metrics (NSE, KGE, FLV, FHV) and decadal streamflow trends (Sen’s slope for Q10, Q50, Q98, and mean annual flow); ET was independently checked against MODIS 8-day integrated ET.

Limitations

  • The paper states that the comparison to an existing process-based model with parameter regionalization was only possible on the temporal test, because systematic PUB/PUR results for that model were not available.
  • The authors note that the PUB setup was computationally economical and therefore less demanding than prior work: only 10 years of training were used, no ensemble was used, and a lower k-fold was used; they say stronger setups could be studied in future.
  • Evident from the text: the study trains and evaluates on CONUS/CAMELS, so the conclusions about global applicability and data-poor continents are suggested rather than directly tested here.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.5194/hess-27-2357-2023

Cites

67 of 67 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Figure 1 A comparison of spatial generalization tests. (a) Prediction in ungauged basins (PUB). (b) Prediction in ungauged regions (PUR) tests. The black dots are the training basins, while the red stars are the test basins for one fold. In this study, we conducted cross-validation to obtain the spatial out-of-sample predictions for basins in the CAMELS dataset.
Figure 1 A comparison of spatial generalization tests. (a) Prediction in ungauged basins (PUB). (b) Prediction in ungauged regions (PUR) tests. The black dots are the training basins, while the red stars are the test basins for one fold. In this study, we conducted cross-validation to obtain the spatial out-of-sample predictions for basins in the CAMELS dataset.CC-BY · From the PDF
Loss backpropagation & gradient descent to train gA and embedded neural networks
Loss backpropagation & gradient descent to train gA and embedded neural networksCC-BY · From the PDF
Figure 3 Performance of simulated daily hydrographs from the models for (a) the randomized PUB experiment using Maurer meteorological forcing data. (b) Comparison of PUB results using either Maurer or Daymet forcing data. Each box summarizes 531 values (one for each CAMELS basin) obtained as a result of cross-validation. All models except those denoted as 'in sample' (which means all sites are included in the training set and thus is at an advantage in testing) were evaluated out-of-sample spatially; i.e., they were trained on some basins and tested on other holdout basins. For MPR C mHM (Rakovec et al., 2019), all test basins were included in the training dataset. NSE is the Nash-Sutcliffe model efficiency coefficient, KGE is the Kling-Gupta efficiency, FLV is the low flow percent bias, and FHV is the high flow percent bias. , . t ), or . t , t ) refer to the differentiable, learnable HBV models with static, one-parameter dynamic, and two-parameter dynamic parameterizations, respectively. The horizontal line in each box represents the median, and the bottom and top of the box represent the first and third quantiles, respectively, while the whiskers extend to 1.5 times the interquartile range from the first and third quantiles, respectively. The PUB was run in a computationally economic manner to be comparable to other models, while also reducing computational demand; we used only 10 years of training period, did not use an ensemble, and used a lower k fold. When we previously ran the experiments using the same settings as Kratzert et al. (2019a), our LSTM was able to match the PUB performance in their work (Feng et al., 2021).
Figure 3 Performance of simulated daily hydrographs from the models for (a) the randomized PUB experiment using Maurer meteorological forcing data. (b) Comparison of PUB results using either Maurer or Daymet forcing data. Each box summarizes 531 values (one for each CAMELS basin) obtained as a result of cross-validation. All models except those denoted as 'in sample' (which means all sites are included in the training set and thus is at an advantage in testing) were evaluated out-of-sample spatially; i.e., they were trained on some basins and tested on other holdout basins. For MPR C mHM (Rakovec et al., 2019), all test basins were included in the training dataset. NSE is the Nash-Sutcliffe model efficiency coefficient, KGE is the Kling-Gupta efficiency, FLV is the low flow percent bias, and FHV is the high flow percent bias. , . t ), or . t , t ) refer to the differentiable, learnable HBV models with static, one-parameter dynamic, and two-parameter dynamic parameterizations, respectively. The horizontal line in each box represents the median, and the bottom and top of the box represent the first and third quantiles, respectively, while the whiskers extend to 1.5 times the interquartile range from the first and third quantiles, respectively. The PUB was run in a computationally economic manner to be comparable to other models, while also reducing computational demand; we used only 10 years of training period, did not use an ensemble, and used a lower k fold. When we previously ran the experiments using the same settings as Kratzert et al. (2019a), our LSTM was able to match the PUB performance in their work (Feng et al., 2021).CC-BY · From the PDF
Figure 4 Decadal trends (m 3 s 1 yr 1 ) of streamflow for different flow percentiles for the randomized PUB cross-validation experiment (using Maurer forcing data), as compared to the observed trends. Chart columns are organized by flow percentile. Q 10 , Q 50 , and Q 98 mean that the trends were evaluated in the annual 10th, 50th, and 98th percentile flows, respectively (or more simply, low, median, and high flows). Chart rows are organized by model, and results for LSTM are shown in pink, results for are in red, . t ) are in blue, and . t , t ) are in black. , . t ), or . t , t ) refer to the differentiable, learnable HBV models with static, one-parameter dynamic, and two-parameter dynamic parameterizations, respectively. For each flow percentile, a corresponding value was extracted from each year's daily data, and Sen's slope was estimated and evaluated between hydrologic years 1989 and 1999.
Figure 4 Decadal trends (m 3 s 1 yr 1 ) of streamflow for different flow percentiles for the randomized PUB cross-validation experiment (using Maurer forcing data), as compared to the observed trends. Chart columns are organized by flow percentile. Q 10 , Q 50 , and Q 98 mean that the trends were evaluated in the annual 10th, 50th, and 98th percentile flows, respectively (or more simply, low, median, and high flows). Chart rows are organized by model, and results for LSTM are shown in pink, results for are in red, . t ) are in blue, and . t , t ) are in black. , . t ), or . t , t ) refer to the differentiable, learnable HBV models with static, one-parameter dynamic, and two-parameter dynamic parameterizations, respectively. For each flow percentile, a corresponding value was extracted from each year's daily data, and Sen's slope was estimated and evaluated between hydrologic years 1989 and 1999.CC-BY · From the PDF
Figure 5 Observed vs. simulated decadal trends (m 3 s 1 yr 1 ) of streamflow for the temporal test for 447 basins where MPR C mHM has predictions (all models trained with Maurer forcing data from 1999 to 2008 and tested from 1989 to 1999 of hydrologic years on the same basins). We could only compare the trends with an existing process-based model with a parameter regionalization scheme on the temporal test because we did not have their systematic PUB results on the same dataset. Chart columns are organized by flow percentile. Q 10 , Q 50 , and Q 98 mean that the trends were evaluated in the annual 10th, 50th, and 98th percentile flows, respectively (or more simply, low, median, and high flows). Chart rows are organized by model, and results for MPR C mHMare shown in green, LSTM are in pink, results for are in red, . t ) are in blue, and . t , t ) are in black. , . t ), or . t , t ) refer to the differentiable, learnable HBV models with static, one-parameter dynamic, and two-parameter dynamic parameterizations, respectively. For each flow percentile, a corresponding value was extracted from each year's daily data, and Sen's slope was estimated and evaluated between hydrologic years 1989 and 1999.
Figure 5 Observed vs. simulated decadal trends (m 3 s 1 yr 1 ) of streamflow for the temporal test for 447 basins where MPR C mHM has predictions (all models trained with Maurer forcing data from 1999 to 2008 and tested from 1989 to 1999 of hydrologic years on the same basins). We could only compare the trends with an existing process-based model with a parameter regionalization scheme on the temporal test because we did not have their systematic PUB results on the same dataset. Chart columns are organized by flow percentile. Q 10 , Q 50 , and Q 98 mean that the trends were evaluated in the annual 10th, 50th, and 98th percentile flows, respectively (or more simply, low, median, and high flows). Chart rows are organized by model, and results for MPR C mHMare shown in green, LSTM are in pink, results for are in red, . t ) are in blue, and . t , t ) are in black. , . t ), or . t , t ) refer to the differentiable, learnable HBV models with static, one-parameter dynamic, and two-parameter dynamic parameterizations, respectively. For each flow percentile, a corresponding value was extracted from each year's daily data, and Sen's slope was estimated and evaluated between hydrologic years 1989 and 1999.CC-BY · From the PDF
Figure 6 Same as Fig. 3 but for the regional holdout (PUR) test. Performance of simulated daily hydrographs from the models for (a) the regionalized PUR experiment using Maurer meteorological forcing data. (b) Comparison of PUR results using either Maurer or Daymet forcing data. Each box summarizes the metrics of 531 basins obtained in a regional cross-validation fashion. We see the clear outperformance of LSTM by the models for these daily hydrograph metrics (NSE, KGE, and FHV).
Figure 6 Same as Fig. 3 but for the regional holdout (PUR) test. Performance of simulated daily hydrographs from the models for (a) the regionalized PUR experiment using Maurer meteorological forcing data. (b) Comparison of PUR results using either Maurer or Daymet forcing data. Each box summarizes the metrics of 531 basins obtained in a regional cross-validation fashion. We see the clear outperformance of LSTM by the models for these daily hydrograph metrics (NSE, KGE, and FHV).CC-BY · From the PDF
Image 7
CC-BY · From the PDF
Figure 9 Comparison of the agreement of simulated ET and the MODIS satellite product for different models under the temporal test (in sample), PUB, and PUR scenarios using two different metrics, namely (a) correlation and (b) root mean square error (RMSE). All models were trained only with streamflow as the target. LSTM is not shown, as it is unable to output physical variables on which it has not been explicitly trained.
Figure 9 Comparison of the agreement of simulated ET and the MODIS satellite product for different models under the temporal test (in sample), PUB, and PUR scenarios using two different metrics, namely (a) correlation and (b) root mean square error (RMSE). All models were trained only with streamflow as the target. LSTM is not shown, as it is unable to output physical variables on which it has not been explicitly trained.CC-BY · From the PDF
Figure B1. Division of the CAMELS dataset into seven large regions for the PUR cross validation test. For every fold, the models were trained on six of the seven regions and tested on the one held out. We ran the experiments for seven rounds so that each region would be the test region once. The results for the test basins were then collected, and the test metrics were reported for this collection.
Figure B1. Division of the CAMELS dataset into seven large regions for the PUR cross validation test. For every fold, the models were trained on six of the seven regions and tested on the one held out. We ran the experiments for seven rounds so that each region would be the test region once. The results for the test basins were then collected, and the test metrics were reported for this collection.CC-BY · From the PDF
Figure B2. The PUR performance comparison in different regions (shown in Fig. B1) in terms of (a) NSE and (b) KGE.
Figure B2. The PUR performance comparison in different regions (shown in Fig. B1) in terms of (a) NSE and (b) KGE.CC-BY · From the PDF

Verification

Claims the paper makes

3 Valid · 2 Not valid · 17 Not checked · 110 Own results and interpretations

48 more claims

Create a free account to see all 132 claims and to verify the 2 references still open.

See all claims — free account
  • It may state two separate claims as one. 55
  • Probably not a substantial claim. 33
  • The quote only partly states this claim. 6

Create a free account to see these claims.

Citations

Cites

Show all 67 references

Research works printed but not linked: 2

  1. 8. Bergström, S.: Development and application of a concep- tual runoff model for Scandinavian catchments, PhD Thesis, Swedish Meteorological and Hydrological Institute (SMHI), Norköping, Sweden, http://urn.kb.se/resolve?urn=urn:nbn:se: smhi:diva-5738 (last access: 8 June 2022), 1976.
  2. 50. Paszke, A., Gross, S., Chintala, S., Chanan, G., Yang, E., DeVito, Z., Lin, Z., Desmaison, A., Antiga, L., and Lerer, A.: Automatic dif- ferentiation in PyTorch, in: 31st Conference on Neural Informa- tion Processing Systems (NIPS 2017), Long Beach, CA, USA, 9 December 2017, https://openreview.net/forum?id=BJJsrmfCZ (last access: 29 June 2023), 2017.

Cited by

See all 197

Metadata

Authors
Feng, D., Beck, H., Lawson, K., & Shen, C.
Affiliations
  • Pennsylvania State University (US)
  • Joint Research Centre (IT)
Corresponding author
  • Beck, Hylke
  • Shen, Chaopeng
Venue
Hydrology and Earth System Sciences
Published
2023-06-30
Volume
27
Issue
12
Pages
2357-2373
ISSN
  • 1027-5606 (ISSN-L)
  • 1607-7938
OpenAlex
W4382866390
Cited by
197
References
67
Citations by year
2026: 73 · 2025: 57 · 2024: 42 · 2023: 22
FWCI
16.12
Type
journal article
Language
en
Open access
Locations
  • Hydrology and earth system sciences (publishedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • NOAA Institutional Repository (submittedVersion, cc-by)
  • OSTI OAI (U.S. Department of Energy Office of Scientific and Technical Information) (submittedVersion)
  • King Abdullah University of Science and Technology Repository (King Abdullah University of Science and Technology) (submittedVersion, cc-by)
Funders
  • National Science Foundation (EAR 1832294, OAC 1940190)
  • U.S. Department of Energy (DE‐SC0016605)
Topics
  • Physical Sciences › Environmental Science › Environmental Engineering › Hydrological Forecasting Using AI
  • Physical Sciences › Environmental Science › Water Science and Technology › Hydrology and Watershed Management Studies
  • Physical Sciences › Environmental Science › Management, Monitoring, Policy and Law › Environmental Monitoring and Data Management
Keywords
physics-informed machine learning, ungauged basins, streamflow forecasting, climate impact assessment, spatial extrapolation, data-scarce regions, evapotranspiration, mass conservation, LSTM comparison

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: The suitability of differentiable, physics-informed machine learning hydrologic models for ungauged regions and climate change impact assessment. https://app.cochat.ai/papers/10.5194/hess-27-2357-2023
The paper
Feng, D., Beck, H., Lawson, K., & Shen, C. (2023). The suitability of differentiable, physics-informed machine learning hydrologic models for ungauged regions and climate change impact assessment. Hydrology and Earth System Sciences. https://doi.org/10.5194/hess-27-2357-2023

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.