CoChat

fPINNs: Fractional Physics-Informed Neural Networks

Open access
Pang, Guofei, Lu, Lu, Karniadakis, George Em · 2019 · SIAM Journal on Scientific Computing

The DOI matches its Crossref record; 43 of the 43 references it cites were found in a registry or an index.

Abstract

Physics-informed neural networks (PINNs), introduced in [M. Raissi, P. Perdikaris, and G. Karniadakis, J. Comput. Phys., 378 (2019), pp. 686--707], are effective in solving integer-order partial differential equations (PDEs) based on scattered and noisy data. PINNs employ standard feedforward neural networks (NNs) with the PDEs explicitly encoded into the NN using automatic differentiation, while the sum of the mean-squared PDE residuals and the mean-squared error in initial-boundary conditions is minimized with respect to the NN parameters. Here we extend PINNs to fractional PINNs (fPINNs) to solve space-time fractional advection-diffusion equations (fractional ADEs), and we study systematically their convergence, hence explaining both fPINNs and PINNs for the first time. Specifically, we demonstrate their accuracy and effectiveness in solving multidimensional forward and inverse problems with forcing terms whose values are only known at randomly scattered spatio-temporal coordinates ( black-box (BB) forcing terms). A novel element of the fPINNs is the hybrid approach that we introduce for constructing the residual in the loss function using both automatic differentiation for the integer-order operators and numerical discretization for the fractional operators. This approach bypasses the difficulties stemming from the fact that automatic differentiation is not applicable to fractional operators because the standard chain rule in integer calculus is not valid in fractional calculus. To discretize the fractional operators, we employ the Grünwald--Letnikov (GL) formula in one-dimensional fractional ADEs and the vector GL formula in conjunction with the directional fractional Laplacian in two- and three-dimensional fractional ADEs. We first consider the one-dimensional fractional Poisson equation and compare the convergence of the fPINNs against the finite difference method (FDM). We present the solution convergence using both the mean $L^2$ error as well as the standard deviation due to sensitivity to NN parameter initializations. Using different GL formulas we observe first-, second-, and third-order convergence rates for small size training sets but the error saturates for larger training sets. We explain these results by analyzing the four sources of numerical errors due to discretization, sampling, NN approximation, and optimization. The total error decays monotonically (below $10^{-5}$ for a third-order GL formula) but it saturates beyond that point due to the optimization error. We also analyze the relative balance between discretization and sampling errors and observe that the sampling size and the number of discretization points ( auxiliary points) should be comparable to achieve the highest accuracy. As we increase the depth of the NN up to certain value, the mean error decreases and the standard deviation increases, whereas the width has essentially no effect unless its value is either too small or too large. We next consider time-dependent fractional ADEs and compare white-box (WB) and BB forcing. We observe that for the WB forcing, our results are similar to the aforementioned cases; however, for the BB forcing fPINNs outperform FDM. Subsequently, we consider multidimensional time-, space-, and space-time-fractional ADEs using the directional fractional Laplacian and we observe relative errors of $10^{-3}\sim10^{-4}$. Finally, we solve several inverse problems in one, two, and three dimensions to identify the fractional orders, diffusion coefficients, and transport velocities and obtain accurate results given proper initializations even in the presence of significant noise.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1137/18m1229845

Cites

43 of 43 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 43 references

Research works printed but not linked: 31

  1. [62] Avrim Blum and Ronald L Rivest. Training a 3-node neural network is np-complete. In Advances in neural information processing systems , pages 494-501, 1989. -
  2. [13] Samuel H Rudy, Steven L Brunton, Joshua L Proctor, and J Nathan Kutz. Data-driven discovery of partial differential equations. Science Advances , 3(4):e1602614, 2017. -
  3. [14] Zichao Long, Yiping Lu, Xianzhong Ma, and Bin Dong. PDE-net: Learning PDEs from data. In Jennifer Dy and Andreas Krause, editors, Proceedings of the 35th International Conference on Machine Learning , volume 80 of Proceedings of Machine Learning Research , pages 32083216, Stockholmsmässan, Stockholm Sweden, 10-15 Jul 2018. PMLR. -
  4. [11] Francesco Mainardi. Fractional calculus and waves in linear viscoelasticity: an introduction to mathematical models . World Scientific, 2010. -
  5. [5] Bradley E Treeby and BT Cox. Modeling power law absorption and dispersion for acoustic propagation using the fractional laplacian. The Journal of the Acoustical Society of America , 127(5):2741-2748, 2010. -
  6. 10.1007/978-3-642-14574-2 · doi:10.1007/978-3-642-14574-2
  7. [57] Rudolf Gorenflo, Anatoli˘ ı Aleksandrovich Kilbas, Francesco Mainardi, Sergei V Rogosin, et al. Mittag-Leffler functions, related topics and applications , volume 2. Springer, 2014. -
  8. [19] Vladimir I Gorbachenko, Tatiana V Lazovskaya, Dmitriy A Tarkhov, Alexander N Vasilyev, and Maxim V Zhukov. Neural network technique in some inverse problems of mathematical physics. In International Symposium on Neural Networks , pages 310-316. Springer, 2016. -
  9. [59] Xavier Glorot and Yoshua Bengio. Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the thirteenth international conference on artificial intelligence and statistics , pages 249-256, 2010. -
  10. 10.1007/978-3-319-40663-3_36 · doi:10.1007/978-3-319-40663-3_36
  11. 10.1007/978-3-662-43930-2 · doi:10.1007/978-3-662-43930-2
  12. [47] A Anatolii Aleksandrovich Kilbas, Hari Mohan Srivastava, and Juan J Trujillo. Theory and applications of fractional differential equations , volume 204. Elsevier Science Limited, 2006. -
  13. [23] Mamikon Gulian, Maziar Raissi, Paris Perdikaris, and George Karniadakis. Machine learning of space-fractional differential equations. arXiv preprint arXiv:1808.00931 , 2018. - · arXiv
  14. [24] Tianping Chen and Hong Chen. Approximations of continuous functionals by neural networks with application to dynamic systems. IEEE Transactions on Neural Networks , 4(6):910-918, 1993. -
  15. [7] Tieyuan Zhu and Jerry M Harris. Modeling acoustic wave propagation in heterogeneous attenuating media using decoupled fractional laplacians. Geophysics , 79(3):T105-T116, 2014. -
  16. [54] Bangti Jin, Raytcho Lazarov, and Zhi Zhou. An analysis of the l1 scheme for the subdiffusion equation with nonsmooth data. IMA Journal of Numerical Analysis , 36(1):197-221, 2015. -
  17. Kilbas A. A. A., 2006, North-Holland Math. Stud. 204
  18. [61] Jason D Lee, Max Simchowitz, Michael I Jordan, and Benjamin Recht. Gradient descent only converges to minimizers. In Conference on Learning Theory , pages 1246-1257, 2016. -
  19. [67] Lihua Lei, Cheng Ju, Jianbo Chen, and Michael I Jordan. Non-convex finite-sum optimization via scsg methods. In Advances in Neural Information Processing Systems , pages 2348-2358, 2017. -
  20. [68] Chenxin Ma, Jakub Konečn` y, Martin Jaggi, Virginia Smith, Michael I Jordan, Peter Richtárik, and Martin Takáč. Distributed optimization with arbitrary local solvers. Optimization Methods and Software , 32(4):813-848, 2017.
  21. 10.1142/p614 · doi:10.1142/p614
  22. Meerschaert M. M., 2004, Fract. Calc. Appl. Anal., 7, 61
  23. [41] Ying Zhang and Xiang Xu. Inverse source problem for a fractional diffusion equation. Inverse problems , 27(3):035010, 2011. -
  24. [42] Bangti Jin and William Rundell. An inverse problem for a one-dimensional time-fractional diffusion problem. Inverse Problems , 28(7):075010, 2012. -
  25. [50] Igor Podlubny. Fractional differential equations: an introduction to fractional derivatives, fractional differential equations, to methods of their solution and some of their applications. Mathematics in science and engineering , 198:261-300, 1999. -
  26. [51] Kai Diethelm. The analysis of fractional differential equations: An application-oriented exposition using differential operators of Caputo type . Springer Science & Business Media, 2010. -
  27. [32] Yoshua Bengio, Ian J Goodfellow, and Aaron Courville. Deep learning. Nature , 521(7553):436444, 2015. -
  28. Samko S. G., 1993, Gordon and Breach
  29. [64] John Paul Roop. Computational aspects of fem approximation of fractional advection dispersion equations on bounded domains in r2. Journal of Computational and Applied Mathematics , 193(1):243-268, 2006. -
  30. [9] Vladimir V Uchaikin. Fractional derivatives for physicists and engineers , volume 2. Springer, 2013. -
  31. [35] Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: a simple way to prevent neural networks from overfitting. The Journal of Machine Learning Research , 15(1):1929-1958, 2014. -

Cited by

Items 301–400 of 1,000

Metadata

Authors
Pang, G., Lu, L., & Karniadakis, G. E.
Venue
SIAM Journal on Scientific Computing
Published
2019-01
Volume
41
Issue
4
Pages
A2603-A2626
ISSN
  • 1064-8275 (ISSN-L)
  • 1095-7197
OpenAlex
W2900501930
Cited by
1,105
References
38
Citations by year
2026: 253 · 2025: 260 · 2024: 195 · 2023: 155 · 2022: 119 · 2021: 82 · 2020: 35 · 2019: 6
FWCI
28.01
Type
journal article
Language
en
Open access
Locations
  • SIAM Journal on Scientific Computing (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • OSTI OAI (U.S. Department of Energy Office of Scientific and Technical Information) (submittedVersion)
  • arXiv (Cornell University)
Funders
  • Army Research Office (W911NF-18-1-0301, W911NF-15-1-0562)
  • U.S. Department of Energy (DE-SC0019434, DE-SC0019453)
Topics
  • Physical Sciences › Physics and Astronomy › Statistical and Nonlinear Physics › Model Reduction and Neural Networks
  • Physical Sciences › Mathematics › Modeling and Simulation › Fractional Differential Equations Solutions
  • Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
Keywords
physics-informed neural networks, fractional advection-diffusion equation, fractional Poisson equation, convergence analysis, inverse problems, sampling error, optimization error

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.