CoChat
Sign Up
Cited by 8,718Model Reduction and Neural NetworksNeural Networks and ApplicationsMachine Learning in Materials Science

Physics-informed machine learning

Open access
Karniadakis, George Em, Kevrekidis, Ioannis G., Lu, Lu, Perdikaris, Paris, Wang, Sifan, Yang, Liu·2021·Nature Reviews Physics
DOI 10.1038/s42254-021-00314-5
Open this paper in CoChat — free accountOpen PDF

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1038/s42254-021-00314-5

Cites

18 of 18 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The distinction between first principles-based and data-driven models is not binary; many models lie within a spectrum with pure data-driven and pure first principles based models at the extremes, and hybrid methods are possible.

    “This is not a binary categorization. Rather, many models lie within a spectrum with pure data-driven and pure first principles based models at the extremes [20], and hybrid methods are possible, e.g., [21].”

    From Credibility assessment of in silico clinical trials for medical devices (2024) · cites this paper as [21]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Part of this paper’s reference list could not be split into single entries.

Flagged first, then by citations

Research works printed but not linked: 49

  1. Kurth, T. et al. Exascale deep learning for climate analytics (IEEE, 2018).
  2. Iten, R., Metger, T., Wilming, H., Del Rio, L. & Renner, R. Discovering physical concepts with neural networks. Phys. Rev. Lett. 124, 010508 (2020).
  3. Brunton, S. L., Proctor, J. L. & Kutz, J. N. Discovering governing equations from data by sparse identification of nonlinear dynamical systems. Proc. Natl Acad. Sci. USA 113, 3932-3937 (2016).
  4. Jasak, H. et al. OpenFOAM: A C++ library for complex physics simulations. Int. Workshop Coupled Methods Numer. Dyn. 1000, 1-20 (2007).
  5. Kashefi, A., Rempe, D. & Guibas, L. J. A point-cloud deep learning framework for prediction of fluid flow fields on irregular geometries. Phys. Fluids 33, 027104 (2021).
  6. Li, Z. et al. Fourier neural operator for parametric partial differential equations. in Int. Conf. Learn. Represent. (2021).
  7. LeCun, Y. & Bengio, Y. et al. Convolutional networks for images, speech, and time series. Handb. Brain Theory Neural Netw. 3361, 1995 (1995).
  8. Mallat, S. Understanding deep convolutional networks. Phil. Trans. R. Soc. A 374, 20150203 (2016).
  9. Cohen, T., Weiler, M., Kicanaoglu, B. & Welling, M. Gauge equivariant convolutional networks and the icosahedral CNN. Proc. Machine Learn. Res. 97, 1321-1330 (2019).
  10. Hamzi, B. & Owhadi, H. Learning dynamical systems from data: a simple cross-validation perspective, part I: parametric kernel flows. Physica D 421, 132817 (2021).
  11. Winkens, J., Linmans, J., Veeling, B. S., Cohen, T. S. & Welling, M. Improved semantic segmentation for histopathology using rotation equivariant convolutional networks. in Conf. Med. Imaging Deep Learn. (2018).
  12. Bruna, J. & Mallat, S. Invariant scattering convolution networks. IEEE Trans. Pattern Anal. Mach. Intell. 35, 1872-1886 (2013).
  13. Tai, K. S., Bailis, P. & Valiant, G. Equivariant transformer networks. Proc. Int. Conf. Mach. Learn. 97, 6086-6095 (2019).
  14. Pfau, D., Spencer, J. S., Matthews, A. G. & Foulkes, W. M. C. Ab initio solution of the many- electron Schrödinger equation with deep neural networks. Phys. Rev. Res. 2, 033429 (2020).
  15. Lusch, B., Kutz, J. N. & Brunton, S. L. Deep learning for universal linear embeddings of nonlinear dynamics. Nat. Commun. 9, 4950 (2018).
  16. Lagaris, I. E., Likas, A. & Fotiadis, D. I. Artificial neural networks for solving ordinary and partial differential equations. IEEE Trans. Neural Netw. 9, 987-1000 (1998).
  17. Sheng, H. & Yang, C. PFNN: A penalty-free neural network method for solving a class of second-order boundary-value problems on complex geometries. J. Comput. Phys. 428, 110085 (2021). 0123456789();:
  18. McFall, K. S. & Mahan, J. R. Artificial neural network method for solution of boundary value problems with exact satisfaction of arbitrary boundary conditions. IEEE Transac. Neural Netw. 20, 1221-1233 (2009).
  19. Lagari, P. L., Tsoukalas, L. H., Safarkhani, S. & Lagaris, I. E. Systematic construction of neural forms for solving partial differential equations inside rectangular domains, subject to initial, boundary and interface conditions. Int. J. Artif. Intell. Tools 29, 2050009 (2020).
  20. Dong, S. & Ni, N. A method for representing periodic functions and enforcing exactly periodic boundary conditions with deep neural networks. J. Comput. Phys. 435, 110242 (2021).
  21. Darbon, J. & Meng, T. On some neural network architectures that can represent viscosity solutions of certain high dimensional Hamilton-Jacobi partial differential equations. J. Comput. Phys. 425, 109907 (2021).
  22. Kissas, G. et al. Machine learning in cardiovascular flows modeling: predicting arterial blood pressure from non-invasive 4D flow MRI data using physics- informed neural networks. Comput. Methods Appl. Mech. Eng. 358, 112623 (2020).
  23. Geneva, N. & Zabaras, N. Modeling the dynamics of PDE systems with physics-constrained deep auto-regressive networks. J. Comput. Phys. 403, 109056 (2020).
  24. Wu, J. L. et al. Enforcing statistical constraints in generative adversarial networks for modeling chaotic dynamical systems. J. Comput. Phys. 406, 109209 (2020).
  25. Shah, V. et al. Encoding invariances in deep generative models. Preprint at arXiv https://arxiv.org/abs/ 1906.01626 (2019).
  26. Geneva, N. & Zabaras, N. Transformers for modeling physical systems. Preprint at arXiv https://arxiv.org/ abs/2010.03957 (2020). · arXiv
  27. Li, Z. et al. Multipole graph neural operator for parametric partial differential equations. in Adv. Neural Inf. Process. Syst. (2020).
  28. Nelsen, N. H. & Stuart, A. M. The random feature model for input-output maps between Banach spaces. Preprint at arXiv https://arxiv.org/abs/2005.10224 (2020). · arXiv
  29. Cai, S., Wang, Z., Lu, L., Zaki, T. A. & Karniadakis, G. E. DeepM&Mnet: inferring the electroconvection multiphysics fields based on operator approximation by neural networks. J. Comput. Phys. 436, 110296 (2020).
  30. Mao, Z., Lu, L., Marxen, O., Zaki, T. A. & Karniadakis, G. E. DeepM&Mnet for hypersonics: predicting the coupled flow and finite-rate chemistry behind a normal shock using neural-network approximation of operators. Preprint at arXiv https://arxiv.org/abs/2011.03349 (2020). · arXiv
  31. Meng, X. & Karniadakis, G. E. A composite neural network that learns from multi-fidelity data: application to function approximation and inverse PDE problems. J. Comput. Phys. 401, 109020 (2020).
  32. Sirignano, J., MacArt, J. F. & Freund, J. B. DPM: a deep learning PDE augmentation method with application to large-eddy simulation. J. Comput. Phys. 423, 109811 (2020).
  33. Lu, L. et al. Extraction of mechanical properties of materials through deep learning from instrumented indentation. Proc. Natl Acad. Sci. USA 117, 7052-7062 (2020).
  34. Reyes, B., Howard, A. A., Perdikaris, P. & Tartakovsky, A. M. Learning unknown physics of non-Newtonian fluids. Preprint at arXiv https://arxiv.org/ abs/2009.01658 (2020). · arXiv
  35. Rico-Martinez, R., Anderson, J. & Kevrekidis, I. Continuous-time nonlinear signal processing: a neural network based approach for gray box identification (IEEE, 1994).
  36. Xu, K., Huang, D. Z. & Darve, E. Learning constitutive relations using symmetric positive definite neural networks. Preprint at arXiv https://arxiv.org/abs/ 2004.00265 (2020).
  37. Huang, D. Z., Xu, K., Farhat, C. & Darve, E. Predictive modeling with learned constitutive laws from indirect observations. Preprint at arXiv https://arxiv.org/abs/ 1905.12530 (2019).
  38. Yang, Y., Bhouri, M. A. & Perdikaris, P. Bayesian differential programming for robust systems identification under uncertainty. Proc. R. Soc. A 476, 20200290 (2020).
  39. Wang, S., Wang, H. & Perdikaris, P. On the eigenvector bias of Fourier feature networks: from regression to solving multi-scale PDEs with physics-informed neural networks. Preprint at arXiv https://arxiv.org/abs/ 2012.10047 (2020).
  40. Wilson, A. G., Hu, Z., Salakhutdinov, R. & Xing, E. P. Deep kernel learning. Proc. Int. Conf. Artif. Intell. Stat. 51, 370-378 (2016).
  41. Owhadi, H. Do ideas have shape? Plato's theory of forms as the continuous limit of artificial neural networks. Preprint at arXiv https://arxiv.org/abs/ 2008.03920 (2020).
  42. Owhadi, H. & Scovel, C. Operator-Adapted Wavelets, Fast Solvers, and Numerical Homogenization: From a Game Theoretic Approach to Numerical Approximation and Algorithm Design (Cambridge Univ. Press, 2019).
  43. Micchelli, C. A. & Rivlin, T. J. in Optimal Estimation in Approximation Theory (eds. Micchelli, C. A. & Rivlin, T. J.) 1-54 (Springer, 1977).
  44. Long, Z., Lu, Y., Ma, X. & Dong, B. PDE-Net: learning PDEs from data. Proc. Int. Conf. Mach. Learn. 80, 3208-3216 (2018).
  45. Chen, T. Q., Rubanova, Y., Bettencourt, J. & Duvenaud, D. K. Neural ordinary differential equations. Adv. Neural Inf. Process. Syst. 31, 6571-6583 (2018).
  46. Jia, J. & Benson, A. R. Neural jump stochastic differential equations. Adv. Neural Inf. Process. Syst. 32, 9847-9858 (2019).
  47. Rico-Martinez, R., Kevrekidis, I. & Krischer, K. in Neural Networks for Chemical Engineers (ed. Bulsari, A. B.) 409-442 (Elsevier, 1995). 100. He, J., Li, L., Xu, J. & Zheng, C. ReLU deep neural networks and linear finite elements. J. Comput. Math. 38, 502-527 (2020). 101. Jagtap, A. D., Kharazmi, E. & Karniadakis, G. E. Conservative physics-informed neural networks on discrete domains for conservation laws: applications to forward and inverse problems. Comput. Methods Appl. Mech. Eng. 365, 113028 (2020). 102. Yang, L., Zhang, D. & Karniadakis, G. E. Physics-informed generative adversarial networks for stochastic differential equations. SIAM J. Sci. Comput. 42, A292-A317 (2020). 103. Pang, G., Lu, L. & Karniadakis, G. E. fPINNs: fractional physics-informed neural networks. SIAM J. Sci. Comput. 41, A2603-A2626 (2019). 104. Kharazmi, E., Zhang, Z. & Karniadakis, G. E. hp-VPINNs: variational physics-informed neural networks with domain decomposition. Comput. Methods Appl. Mech. Eng. 374, 113547 (2021). 105. Jagtap, A. D. & Karniadakis, G. E. Extended physics- informed neural networks (XPINNs): a generalized space-time domain decomposition based deep learning framework for nonlinear partial differential equations. Commun. Comput. Phys. 28, 2002-2041 (2020). 106. Raissi, M., Yazdani, A. & Karniadakis, G. E. Hidden fluid mechanics: learning velocity and pressure fields from flow visualizations. Science 367, 1026-1030 (2020). 107. Yang, L., Meng, X. & Karniadakis, G. E. B-PINNs: Bayesian physics-informed neural networks for forward and inverse PDE problems with noisy data. J. Comput. Phys. 415, 109913 (2021). 108. Wang, S. & Perdikaris, P. Deep learning of free boundary and Stefan problems. J. Comput. Phys. 428, 109914 (2020). 109. Spigler, S. et al. A jamming transition from under-to over-parametrization affects generalization in deep learning. J. Phys. A 52, 474001 (2019). 110. Geiger, M. et al. Scaling description of generalization with number of parameters in deep learning. J. Stat. Mech. Theory Exp. 2020, 023401 (2020). 0123456789();: 111.
  48. Bianco, S., Cadene, R., Celona, L. & Napoletano, P. Benchmark analysis of representative deep neural www.nature.com/natrevphys R e v i e w s 0123456789();: network architectures. IEEE Access 6, 64270-64277 (2018). 187. Vlachas, P. R. et al. Backpropagation algorithms and reservoir computing in recurrent neural networks for the forecasting of complex spatiotemporal dynamics. Neural Networks (2020). 188.
  49. Shin, Y., Darbon, J. & Karniadakis, G. E. On the convergence of physics informed neural networks for linear second-order elliptic and parabolic type PDEs. Commun. Comput. Phys. 28, 2042-2074 (2020). 189. Mishra, S. & Molinaro, R. Estimates on the generalization error of physics informed neural networks (PINNs) for approximating PDEs. Preprint at arXiv https://arxiv.org/abs/2006.16144 (2020). 190. Mishra, S. & Molinaro, R. Estimates on the generalization error of physics informed neural networks (PINNs) for approximating PDEs II: a class of inverse problems. Preprint at arXiv https://arxiv.org/ abs/2007.01138 (2020). 191. Shin, Y., Zhang, Z. & Karniadakis, G.E. Error estimates of residual minimization using neural networks for linear PDEs. Preprint at arXiv https://arxiv.org/abs/ 2010.08019 (2020). 192. Kharazmi, E., Zhang, Z. & Karniadakis, G. Variational physics-informed neural networks for solving partial differential equations. Preprint at arXiv https://arxiv.org/abs/1912.00873 (2019). 193. Jo, H., Son, H., Hwang, H. Y. & Kim, E. Deep neural network approach to forward-inverse problems. Netw. Heterog. Media 15, 247-259 (2020). 194. Guo, M. & Haghighat, E. An energy-based error bound of physics-informed neural network solutions in elasticity. Preprint at arXiv https://arxiv.org/abs/ 2010.09088 (2020). 195. Lee, J. Y., Jang, J. W. & Hwang, H. J. The model reduction of the Vlasov-Poisson-Fokker-Planck system to the Poisson-Nernst-Planck system via the deep neural network approach. Preprint at arXiv https://arxiv.org/abs/2009.13280 (2020). 196. Kingma, D. P. & Ba, J. Adam: a method for stochastic optimization. in Int. Conf. Learn. Represent. (2015). 197. Luo, T. & Yang, H. Two-layer neural networks for partial differential equations: optimization and generalization theory. Preprint at arXiv https://arxiv.org/abs/ 2006.15733 (2020). 198. Jacot, A., Gabriel, F. & Hongler, C. Neural tangent kernel: convergence and generalization in neural networks. Adv. Neural Inf. Process. Syst. 31, 8571-8580 (2018). 199. Alnæs, M. et al. The FEniCS project version 1.5. Arch. Numer. Softw. 3, 9-23 (2015). 200. Kemeth, F. P. et al. An emergent space for distributed data with hidden internal order through manifold learning. IEEE Access 6, 77402-77413 (2018). 201. Kemeth, F. P. et al. Learning emergent PDEs in a learned emergent space. Preprint at arXiv https:// arxiv.org/abs/2012.12738 (2020). 202. Defense Advanced Research Projects Agency. DARPA shredder challenge rules. DARPA https://web.archive. org/web/20130221190250/http://archive.darpa.mil/ shredderchallenge/Rules.aspx (2011). 203. Rovelli, C. Forget time. Found. Phys. 41, 1475 (2011). 204. Hy, T. S., Trivedi, S., Pan, H., Anderson, B. M. & Kondor, R. Predicting molecular properties with covariant compositional networks. J. Chem. Phys. 148, 241745 (2018). 205. Hachmann, J. et al. The Harvard clean energy project: large-scale computational screening and design of organic photovoltaics on the world community grid. J. Phys. Chem. Lett. 2, 2241-2251 (2011). 206. Byrd, R. H., Lu, P., Nocedal, J. & Zhu, C. A limited memory algorithm for bound constrained optimization. SIAM J. Sci. Comput. 16, 1190-1208 (1995). ReLAted Links ADCMe: https://kailaix.github.io/ADCME.jl/latest DeepXDe: https://deepxde.readthedocs.io/ GPyTorch: https://gpytorch.ai/ NeuroDiffeq: https://github.com/NeuroDiffGym/neurodiffeq NeuralPDe: https://neuralpde.sciml.ai/dev/ Neural Tangents: https://github.com/google/neural-tangents PyDens: https://github.com/analysiscenter/pydens PyTorch: https://pytorch.org sciANN: https://www.sciann.com/ simNet: https://developer.nvidia.com/simnet TensorFlow: www.tensorflow.org · arXiv

Cited by

Sorted by citations

Authors
Karniadakis, G. E., Kevrekidis, I. G., Lu, L., Perdikaris, P., Wang, S., & Yang, L.
Affiliations
  • Brown University (US)
  • Johns Hopkins University (US)
  • Massachusetts Institute of Technology (US)
  • University of Pennsylvania (US)
ORCID
Venue
Nature Reviews Physics
Publisher
Springer Science and Business Media LLC
Published
2021-05-24
Volume
3
Issue
6
Pages
422-440
ISSN
2522-5820 (ISSN-L)
DOI
10.1038/s42254-021-00314-5
OpenAlex
W3163993681
Cited by
8,718
References
18
Citations by year
2026: 3514 · 2025: 1998 · 2024: 1477 · 2023: 1040 · 2022: 591 · 2021: 94 · 2020: 1 · 2012: 2
FWCI
34.75
Type
journal article
Language
en
Open access
Licence
Locations
  • Nature Reviews Physics (publishedVersion)
  • OSTI OAI (U.S. Department of Energy Office of Scientific and Technical Information) (submittedVersion)
Funders
  • U.S. Department of Energy (DESC0019116, DESC0019453)
  • California Institute of Technology (FA9550-20-1-0358)
  • Defense Advanced Research Projects Agency (HR00111890034)
  • University of California, Santa Barbara
  • Advanced Research Projects Agency
  • Multidisciplinary University Research Initiative (FA9550-20–1-0358)
  • Air Force Office of Scientific Research (FA9550-, FA9550‐20‐1‐0358, FA9550-20-1, FA9550-20, FA9550-20-1-0060, 1-0358)
Topics
  • Physical Sciences › Physics and Astronomy › Statistical and Nonlinear Physics › Model Reduction and Neural Networks
  • Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
  • Physical Sciences › Materials Science › Materials Chemistry › Machine Learning in Materials Science
Keywords
physics-informed machine learning, physics-informed neural networks, inverse problems, parameterized partial differential equations, noisy data, high-dimensional problems, kernel regression, physical invariants, forward problem, multiphysics simulation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review