The DOI matches its Crossref record; 145 of the 145 references it cites were found in a registry or an index.
Differentiable modelling to unify machine learning and physical models for geosciences
Open accessAbstract
Process-based modelling offers interpretability and physical consistency in many domains of geosciences but struggles to leverage large datasets efficiently. Machine-learning methods, especially deep networks, have strong predictive skills yet are unable to answer specific scientific questions. In this Perspective, we explore differentiable modelling as a pathway to dissolve the perceived barrier between process-based modelling and machine learning in the geosciences and demonstrate its potential with examples from hydrological modelling. ‘Differentiable’ refers to accurately and efficiently calculating gradients with respect to model variables or parameters, enabling the discovery of high-dimensional unknown relationships. Differentiable modelling involves connecting (flexible amounts of) prior physical knowledge to neural networks, pushing the boundary of physics-informed machine learning. It offers better interpretability, generalizability, and extrapolation capabilities than purely data-driven machine learning, achieving a similar level of accuracy while requiring less training data. Additionally, the performance and efficiency of differentiable models scale well with increasing data volumes. Under data-scarce scenarios, differentiable models have outperformed machine-learning models in producing short-term dynamics and decadal-scale trends owing to the imposed physical constraints. Differentiable modelling approaches are primed to enable geoscientists to ask questions, test hypotheses, and discover unrecognized physical relationships. Future work should address computational challenges, reduce uncertainty, and verify the physical significance of outputs. Differentiable modelling is an approach that flexibly integrates the learning capability of machine learning with the interpretability of process-based models. This Perspective highlights the potential of differentiable modelling to improve the representation of processes, parameter estimation, and predictive accuracy in the geosciences.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
A fast and elitist multiobjective genetic algorithm: NSGA-II
Deb, K., Pratap, A., Agarwal, S., & Meyarivan, T. (2002) · IEEE Transactions on Evolutionary Computation · doi:10.1109/4235.996017 · Cited by 50,053
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
Multilayer feedforward networks are universal approximators
Hornik, K., Stinchcombe, M., & White, H. (1989) · Neural Networks · doi:10.1016/0893-6080(89)90020-8 · Cited by 21,728
Raissi, M., Perdikaris, P., & Karniadakis, G. E. (2019) · Journal of Computational Physics · doi:10.1016/j.jcp.2018.10.045 · Cited by 20,384
Eyring, V., Bony, S., Meehl, G. A., Senior, C. A., Stevens, B., Stouffer, R. J., & Taylor, K. E. (2016) · Geoscientific Model Development · doi:10.5194/gmd-9-1937-2016 · Cited by 11,981
Physics-informed machine learning
Karniadakis, G. E., Kevrekidis, I. G., Lu, L., Perdikaris, P., Wang, S., & Yang, L. (2021) · Nature Reviews Physics · doi:10.1038/s42254-021-00314-5 · Cited by 8,718
Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting
Shi, X., Chen, Z., Wang, H., Yeung, D.-Y., Wong, W.-K., & Woo, W.-C. (2015) · arXiv · doi:10.48550/arxiv.1506.04214 · Cited by 7,709
Approximation capabilities of multilayer feedforward networks
Hornik, K. (1991) · Neural Networks · doi:10.1016/0893-6080(91)90009-t · Cited by 6,091
Research works printed but not linked: 29
- Lepore, C., Arnone, E., Noto, L. V., Sivandran, G. & Bras, R. L. Physically based modeling of rainfall-triggered landslides: a case study in the Luquillo forest, Puerto Rico. Hydrol. Earth Syst. Sci. 17, 3371–3387 (2013). · doi:10.5194/hess-17-3371-2013
- Weyant, J. et al. in Climate Change 1995: Social and Economic Dimensions of Climate Change (eds Bruce, J. P., Lee, H. & Haites, E. F.) 367–396 (IPCC, Cambridge Univ. Press, 1996).
- IPCC. Climate Change 2021: The Physical Science Basis (eds Masson-Delmotte, V. et al.) (Cambridge Univ. Press, 2021).
- van Kampenhout, L. et al. Improving the representation of polar snow and firn in the Community Earth System Model. J. Adv. Modeling Earth Syst. 9, 2583–2600 (2017). · doi:10.1002/2017ms000988
- Hunt, R. J., Fienen, M. N. & White, J. T. Revisiting ‘An exercise in groundwater model calibration and prediction’ after 30 years: insights and new directions. Groundwater 58, 168–182 (2020). · doi:10.1111/gwat.12907
- Wagener, T., Wheater, H. S. & Gupta, H. V. in Calibration of Watershed Models 29–47 (Wiley, 2003). · doi:10.1029/ws006p0029
- Trancoso, R., Larsen, J. R., McVicar, T. R., Phinn, S. R. & McAlpine, C. A. CO2–vegetation feedbacks and other climate changes implicated in reducing base flow. Geophys. Res. Lett. 44, 2310–2318 (2017). · doi:10.1002/2017gl072759
- Krizhevsky, A., Sutskever, I. & Hinton, G. E. in Advances in Neural Information Processing Systems vol. 25 (eds Bartlett, P. et al.) 1097–1105 (Curran Associates, 2012).
- Lecun, Y. & Bengio, Y. Convolutional networks for images, speech, and time-series. in The Handbook of Brain Theory and Neural Networks (ed. Arbib, M. A.) 255–258 (MIT Press, 1995).
- Bhowmik, M., Singh, M., Rao, S. & Paul, S. DeepClouds.ai: deep learning enabled computationally cheap direct numerical simulations. Preprint at https://doi.org/10.48550/arXiv.2208.08956 (2022). · arXiv · doi:10.48550/arxiv.2208.08956
- Lin, G.-Y., Chen, H.-W., Chen, B.-J. & Yang, Y.-C. Characterization of temporal PM2.5, nitrate, and sulfate using deep learning techniques. Atmos. Pollut. Res. 13, 101260 (2022). · doi:10.1016/j.apr.2021.101260
- Afzaal, H., Farooque, A. A., Abbas, F., Acharya, B. & Esau, T. Groundwater estimation from major physical hydrology components using artificial neural networks and deep learning. Water 12, 5 (2020). · doi:10.3390/w12010005
- Fleming, S. W., Watson, J. R., Ellenson, A., Cannon, A. J. & Vesselinov, V. C. Machine learning in Earth and environmental science requires education and research policy reforms. Nat. Geosci. 14, 878–880 (2021). · doi:10.1038/s41561-021-00865-3
- Zitzler, E., Laumanns, M. & Thiele, L. SPEA2: Improving the Strength Pareto Evolutionary Algorithm. TIK Report vol. 103 (ETH Zurich, 2001); https://www.research-collection.ethz.ch/handle/20.500.11850/145755.
- Baydin, A. G., Pearlmutter, B. A., Radul, A. A. & Siskind, J. M. Automatic differentiation in machine learning: a survey. J. Mach. Learn. Res. 18, 1–43 (2018).
- Goodfellow, I., Bengio, Y. & Courville, A. in Deep Learning (ed. Goodfellow, I.) Ch. 4 (MIT Press, 2016).
- Paszke, A. et al. Automatic differentiation in PyTorch. In 31st Conference on Neural Information Processing Systems (NIPS 2017) (NIPS, 2017).
- Abadi, M. et al. Tensorflow: a system for large-scale machine learning. In 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 16) 265–283 (USENIX Association, 2016).
- Johnson, S. G. Notes on Adjoint Methods for 18.335 (2021). · doi:10.1038/s41592-021-01122-6
- Pal, A., Edelman, A. & Rackauckas, C. Mixing implicit and explicit deep learning with skip DEQs and infinite time neural ODEs (continuous DEQs). Preprint at https://doi.org/10.48550/arXiv.2201.12240 (2022). · arXiv · doi:10.48550/arxiv.2201.12240
- Chen, R. T. Q., Rubanova, Y., Bettencourt, J. & Duvenaud, D. Neural ordinary differential equations. In Proceedings of the 32nd International Conference on Neural Information Processing Systems, 6572–6583 (Curran Associates, 2018).
- O’Neill, P. E. et al. SMAP enhanced L3 radiometer global and polar grid daily 9 km EASE-grid soil moisture, Version 5 (SPL3SMP_E) (2021); https://doi.org/10.5067/4DQ54OUIJ9DL. · doi:10.5067/4dq54ouij9dl
- Feng, D., Beck, H., Lawson, K. & Shen, C. The suitability of differentiable, learnable hydrologic models for ungauged regions and climate change impact assessment. Hydrology and Earth System Sciences Discussions 1–28 (European Geoscience Union, 2022); https://doi.org/10.5194/hess-2022-245. · doi:10.5194/hess-2022-245
- Onken, D. & Ruthotto, L. Discretize-optimize vs. optimize-discretize for time-series regression and continuous normalizing flows. Preprint at https://arxiv.org/abs/2005.13420 (2020). · arXiv
- Neupauer, R. M. & Wilson, J. L. Adjoint-derived location and travel time probabilities for a multidimensional groundwater system. Water Resour. Res. 37, 1657–1668 (2001). · doi:10.1029/2000wr900388
- Aboelyazeed, D. et al. A differentiable ecosystem modeling framework for large-scale inverse problems: demonstration with photosynthesis simulations. Biogeosci. Discuss. https://doi.org/10.5194/bg-2022-211 (2022). · doi:10.5194/bg-2022-211
- Khandelwal, A. et al. Physics guided machine learning methods for hydrology. https://arxiv.org/abs/2012.02854 (2020). · arXiv
- Hochreiter, S., Bengio, Y., Frasconi, P. & Jürgen S. in A Field Guide to Dynamical Recurrent Neural Networks (eds Kremer, S. C. & Kolen, J. F.) 237–244 (IEEE, 2001).
- Krapu, C. & Borsuk, M. A differentiable hydrology approach for modeling with time-varying parameters. Water Resour. Res. 58, e2021WR031377 (2022). · doi:10.1029/2021wr031377
Cited by
Deep learning for water quality
Zhi, W., Appling, A. P., Golden, H. E., Podgorski, J., & Li, L. (2024) · Nature Water · doi:10.1038/s44221-024-00202-z · Cited by 364
Self-citationLi, P., Song, Y., Pan, M., Lawson, K., & Shen, C. (2025) · Hydrology and Earth System Sciences · doi:10.5194/hess-29-6829-2025 · Cited by 7
Self-citation
Metadata
- Authors
- Shen, C., Appling, A. P., Gentine, P., Bandai, T., Gupta, H., Tartakovsky, A., Baity-Jesi, M., Fenicia, F., Kifer, D., Li, L., Liu, X., Ren, W., Zheng, Y., Harman, C. J., Clark, M., Farthing, M., Feng, D., Kumar, P., Aboelyazeed, D., Rahmani, F., Song, Y., Beck, H. E., Bindas, T., Dwivedi, D., Fang, K., Höge, M., Rackauckas, C., Mohanty, B., Roy, T., Xu, C., & Lawson, K.
- Venue
- Nature Reviews Earth & Environment
- Published
- 2023-07-11
- Volume
- 4
- Issue
- 8
- Pages
- 552-567
- ISSN
- 2662-138X (ISSN-L)
- OpenAlex
- W4383955976
- Cited by
- 468
- References
- 145
- FWCI
- 10.69
- Type
- journal article
- Language
- en
- Open access
- green
- Licence
- www.springernature.com/gp/researchers/text-and-data-mining (tdm)
- www.springernature.com/gp/researchers/text-and-data-mining (vor)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Differentiable modelling to unify machine learning and physical models for geosciences. https://app.cochat.ai/papers/10.1038/s43017-023-00450-9
- The paper
- Shen, C., Appling, A. P., Gentine, P., Bandai, T., Gupta, H., Tartakovsky, A., Baity-Jesi, M., Fenicia, F., Kifer, D., Li, L., Liu, X., Ren, W., Zheng, Y., Harman, C. J., Clark, M., Farthing, M., Feng, D., Kumar, P., Aboelyazeed, D., . . . Lawson, K. (2023). Differentiable modelling to unify machine learning and physical models for geosciences. Nature Reviews Earth & Environment. https://doi.org/10.1038/s43017-023-00450-9
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.