The DOI matches its Crossref record; 73 of the 73 references it cites were found in a registry or an index.
Efficient training of physics‐informed neural networks via importance sampling
Open accessAbstract
Physics‐informed neural networks (PINNs) are a class of deep neural networks that are trained, using automatic differentiation, to compute the response of systems governed by partial differential equations (PDEs). The training of PINNs is simulation free, and does not require any training data set to be obtained from numerical PDE solvers. Instead, it only requires the physical problem description, including the governing laws of physics, domain geometry, initial/boundary conditions, and the material properties. This training usually involves solving a nonconvex optimization problem using variants of the stochastic gradient descent method, with the gradient of the loss function approximated on a batch of collocation points, selected randomly in each iteration according to a uniform distribution. Despite the success of PINNs in accurately solving a wide variety of PDEs, the method still requires improvements in terms of computational efficiency. To this end, in this paper, we study the performance of an importance sampling approach for efficient training of PINNs. Using numerical examples together with theoretical evidences, we show that in each training iteration, sampling the collocation points according to a distribution proportional to the loss function will improve the convergence behavior of the PINNs training. Additionally, we show that providing a piecewise constant approximation to the loss function for faster importance sampling can further improve the training efficiency. This importance sampling approach is straightforward and easy to implement in the existing PINN codes, and also does not introduce any new hyperparameter to calibrate. The numerical examples include elasticity, diffusion, and plane stress problems, through which we numerically verify the accuracy and efficiency of the importance sampling approach compared to the predominant uniform sampling approach.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Raissi, M., Perdikaris, P., & Karniadakis, G. E. (2019) · Journal of Computational Physics · doi:10.1016/j.jcp.2018.10.045 · Cited by 20,384
TensorFlow: A system for large-scale machine learning
Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., Devin, M., Ghemawat, S., Irving, G., Isard, M., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.08695 · Cited by 11,822
(2016) · Cited by 11,679
(2010) · Cited by 8,261
Large-Scale Machine Learning with Stochastic Gradient Descent
Bottou, L. (2010) · Proceedings of COMPSTAT'2010 · doi:10.1007/978-3-7908-2604-3_16 · Cited by 5,795
ADADELTA: An Adaptive Learning Rate Method
Zeiler, M. D. (2012) · arXiv · doi:10.48550/arxiv.1212.5701 · Cited by 5,322
An overview of gradient descent optimization algorithms
Ruder, S. (2016) · arXiv · doi:10.48550/arxiv.1609.04747 · Cited by 4,962
In-Datacenter Performance Analysis of a Tensor Processing Unit
Jouppi, N. P., Young, C., Patil, N., Patterson, D., Agrawal, G., Bajwa, R., Bates, S., Bhatia, S., Boden, N., Borchers, A., et al. (2017) · Proceedings of the 44th Annual International Symposium on Computer Architecture · doi:10.1145/3079856.3080246 · Cited by 4,537
Voronoi diagrams—a survey of a fundamental geometric data structure
Aurenhammer, F. (1991) · ACM Computing Surveys · doi:10.1145/116873.116880 · Cited by 4,340
Hammersley, J. M., & Handscomb, D. C. (1964) · Springer eBooks · doi:10.1007/978-94-009-5819-7 · Cited by 4,054
(2013) · Cited by 3,740
Numerical Recipes 3rd Edition: The Art of Scientific Computing
Press, W. H., Teukolsky, S. A., Vetterling, W. T., & Flannery, B. P. (2007) · Cambridge University Press eBooks · Cited by 3,328
Deep Learning‐Based Crack Damage Detection Using Convolutional Neural Networks
Cha, Y., Choi, W., & Büyüköztürk, O. (2017) · Computer-Aided Civil and Infrastructure Engineering · doi:10.1111/mice.12263 · Cited by 3,295
On the distribution of points in a cube and the approximate evaluation of integrals
Sobol', I. M. (1967) · USSR Computational Mathematics and Mathematical Physics · doi:10.1016/0041-5553(67)90144-9 · Cited by 2,975
Artificial neural networks for solving ordinary and partial differential equations
Lagaris, I. E., Likas, A., & Fotiadis, D. I. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/72.712178 · Cited by 2,412
DGM: A deep learning algorithm for solving partial differential equations
Sirignano, J., & Spiliopoulos, K. (2018) · Journal of Computational Physics · doi:10.1016/j.jcp.2018.08.029 · Cited by 2,208
Automatic Differentiation in Machine Learning: a Survey
Baydin, A. G., Pearlmutter, B. A., Radul, A., & Siskind, J. M. (2015) · Maynooth University ePrints and eTheses Archive (Maynooth University) · Cited by 2,026
Halton, J. H. (1960) · Numerische Mathematik · doi:10.1007/bf01386213 · Cited by 1,985
Stochastic Gradient Descent Tricks
Bottou, L. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-35289-8_25 · Cited by 1,929
The Deep Ritz Method: A Deep Learning-Based Numerical Algorithm for Solving Variational Problems
E, W., & Yu, B. (2018) · Communications in Mathematics and Statistics · doi:10.1007/s40304-018-0127-z · Cited by 1,418
Adaptive activation functions accelerate convergence in deep and physics-informed neural networks
Jagtap, A. D., Kawaguchi, K., & Karniadakis, G. E. (2020) · Journal of Computational Physics · doi:10.1016/j.jcp.2019.109136 · Cited by 1,127
Machine Learning: An Algorithmic Perspective
Marsland, S. (2009) · Cited by 1,068
Zhu, Y., Zabaras, N., Koutsourelakis, P.-S., & Perdikaris, P. (2019) · Journal of Computational Physics · doi:10.1016/j.jcp.2019.05.024 · Cited by 1,030
A hybrid neural network‐first principles approach to process modeling
Psichogios, D. C., & Ungar, L. H. (1992) · AIChE Journal · doi:10.1002/aic.690381003 · Cited by 982
Artificial Neural Networks for Solving Ordinary and Partial Differential Equations
Lagaris, I. E., Likas, A., & Fotiadis, D. I. (1998) · Cited by 944
Raissi, M., Perdikaris, P. G., & Karniadakis, G. E. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.10561 · Cited by 925
Transfer learning enhanced physics informed neural network for phase-field modeling of fracture
Goswami, S., Anitescu, C., Chakraborty, S., & Rabczuk, T. (2020) · Theoretical and Applied Fracture Mechanics · doi:10.1016/j.tafmec.2019.102447 · Cited by 900
Automatic differentiation in machine learning: a survey
Baydin, A. G., Pearlmutter, B. A., Radul, A., & Siskind, J. M. (2015) · arXiv (Cornell University) · doi:10.4230/lipics.ecoop.2019.25 · Cited by 823
E, W., Han, J., & Jentzen, A. (2017) · Communications in Mathematics and Statistics · doi:10.1007/s40304-017-0117-6 · Cited by 822
Meng, X., & Karniadakis, G. E. (2020) · Journal of Computational Physics · doi:10.1016/j.jcp.2019.109020 · Cited by 713
Berg, J., & Nyström, K. (2018) · Neurocomputing · doi:10.1016/j.neucom.2018.06.056 · Cited by 657
Kissas, G., Yang, Y., Hwuang, E., Witschey, W. R., Detre, J. A., & Perdikaris, P. (2020) · Computer Methods in Applied Mechanics and Engineering · doi:10.1016/j.cma.2019.112623 · Cited by 647
Structural Damage Detection with Automatic Feature‐Extraction through Deep Learning
Lin, Y., Nie, Z., & Ma, H. (2017) · Computer-Aided Civil and Infrastructure Engineering · doi:10.1111/mice.12313 · Cited by 634
Neural‐network‐based approximations for solving partial differential equations
Dissanayake, M. W. M. G., & Phan‐Thien, N. (1994) · Communications in Numerical Methods in Engineering · doi:10.1002/cnm.1640100303 · Cited by 622
MONTE CARLO METHODS FOR SOLVING MULTIVARIABLE PROBLEMS
Hammersley, J. M. (1960) · Annals of the New York Academy of Sciences · doi:10.1111/j.1749-6632.1960.tb42846.x · Cited by 516
Physics-Informed Generative Adversarial Networks for Stochastic Differential Equations
Yang, L., Zhang, D., & Karniadakis, G. E. (2020) · SIAM Journal on Scientific Computing · doi:10.1137/18m1225409 · Cited by 482
A Deep Collocation Method for the Bending Analysis of Kirchhoff Plate
Guo, H., Zhuang, X., & Rabczuk, T. (2019) · Computers, Materials & Continua · doi:10.32604/cmc.2019.06660 · Cited by 466
Large-Scale Machine Learning with Stochastic Gradient Descent
Bottou, L. (2011) · Chapman & Hall/CRC Computer Science & Data Analysis · doi:10.1201/b11429-4 · Cited by 460
Deep Hidden Physics Models: Deep Learning of Nonlinear Partial Differential Equations
Raissi, M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1801.06637 · Cited by 419
Adversarial uncertainty quantification in physics-informed neural networks
Yang, Y., & Perdikaris, P. (2019) · Journal of Computational Physics · doi:10.1016/j.jcp.2019.05.027 · Cited by 414
Deep learning of vortex-induced vibrations
Raissi, M., Wang, Z., Triantafyllou, M. S., & Karniadakis, G. E. (2019) · Journal of Fluid Mechanics · doi:10.1017/jfm.2018.872 · Cited by 404
Rafiei, M. H., & Adeli, H. (2018) · Engineering Structures · doi:10.1016/j.engstruct.2017.10.070 · Cited by 401
Liang, X. (2019) · Computer-Aided Civil and Infrastructure Engineering · doi:10.1111/mice.12425 · Cited by 398
Constructing Sobol Sequences with Better Two-Dimensional Projections
Joe, S., & Kuo, F. Y. (2008) · SIAM Journal on Scientific Computing · doi:10.1137/070709359 · Cited by 386
Deep learning of vortex-induced vibrationsPreprint
Raissi, M., Wang, Z., Triantafyllou, M. S., & Karniadakis, G. E. (2019) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 361
Machine Learning: An Algorithmic Perspective
Marsland, S. (2014) · Chapman & Hall/CRC eBooks · doi:10.1201/b17476 · Cited by 353
A novel machine learning‐based algorithm to detect damage in high‐rise building structures
Rafiei, M. H., & Adeli, H. (2017) · The Structural Design of Tall and Special Buildings · doi:10.1002/tal.1400 · Cited by 346
Data driven governing equations approximation using deep neural networks
Qin, T., Wu, K., & Xiu, D. (2019) · Journal of Computational Physics · doi:10.1016/j.jcp.2019.06.042 · Cited by 284
Supervised Deep Restricted Boltzmann Machine for Estimation of Concrete
(2017) · ACI Materials Journal · doi:10.14359/51689560 · Cited by 278
Raissi, M., Yazdani, A. Z. K., & Karniadakis, G. E. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1808.04327 · Cited by 235
Structural health monitoring using extremely compressed data through deep learning
Azimi, M., & Pekcan, G. (2020) · Computer-Aided Civil and Infrastructure Engineering · doi:10.1111/mice.12517 · Cited by 214
Concrete crack detection using context‐aware deep semantic segmentation network
Zhang, X., Rajan, D., & Story, B. (2019) · Computer-Aided Civil and Infrastructure Engineering · doi:10.1111/mice.12477 · Cited by 210
Variance Reduction in SGD by Distributed Importance SamplingPreprint
Alain, G., Lamb, A., Sankar, C., Courville, A. C., & Bengio, Y. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.06481 · Cited by 199
PDE-Net: Learning PDEs from Data
Long, Z., Lu, Y., Ma, X., & Dong, B. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1710.09668 · Cited by 184
Short‐Term Traffic Speed Forecasting Based on Attention Convolutional Neural Network for Arterials
Liu, Q., Wang, B., & Zhu, Y. (2018) · Computer-Aided Civil and Infrastructure Engineering · doi:10.1111/mice.12417 · Cited by 162
Koziarski, M., & Cyganek, B. (2017) · Integrated Computer-Aided Engineering · doi:10.3233/ica-170551 · Cited by 160
NVIDIA SimNet™: An AI-Accelerated Multi-Physics Simulation Framework
Hennigh, O., Narasimhan, S., Nabian, M. A., Subramaniam, A., Tangsali, K., Fang, Z., Rietmann, M., Byeon, W., & Choudhry, S. (2021) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-77977-1_36 · Cited by 159
Self-citationDeep Learning for Accelerated Seismic Reliability Analysis of Transportation Networks
Nabian, M. A., & Meidani, H. (2018) · Computer-Aided Civil and Infrastructure Engineering · doi:10.1111/mice.12359 · Cited by 156
Self-citationRaissi, M. (2023) · WORLD SCIENTIFIC eBooks · doi:10.1142/9789811280306_0018 · Cited by 148
A Deep Neural Network Surrogate for High-Dimensional Random Partial Differential Equations
Nabian, M. A., & Meidani, H. (2018) · Probabilistic Engineering Mechanics · doi:10.1016/j.probengmech.2019.05.001 · Cited by 112
Self-citationA graph deep learning method for short-term traffic forecasting on large road networks
Zhang, Y., Cheng, T., & Ren, Y. (2019) · Computer-Aided Civil and Infrastructure Engineering · doi:10.1111/mice.12450 · Cited by 100
Physics-Driven Regularization of Deep Neural Networks for Enhanced Engineering Design and Analysis
Nabian, M. A., & Meidani, H. (2019) · Journal of Computing and Information Science in Engineering · doi:10.1115/1.4044507 · Cited by 97
Self-citationSolving stochastic differential equations and Kolmogorov equations by means of deep learningPreprint
Beck, C., Becker, S. B. S., Grohs, P., Jaafari, N., & Jentzen, A. (2018) · arXiv (Cornell University) · Cited by 82
Competitive probabilistic neural network
Zeinali, Y., & STORY, B. A. (2017) · Integrated Computer-Aided Engineering · doi:10.3233/ica-170540 · Cited by 78
NVIDIA SimNet^{TM}: an AI-accelerated multi-physics simulation frameworkPreprint
Hennigh, O., Narasimhan, S., Nabian, M. A., Subramaniam, A., Tangsali, K., Rietmann, M., Ferrandis, J. D. Á., Byeon, W., Fang, Z., & Choudhry, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.07938 · Cited by 76
Self-citationGeneralized Halton sequences in 2008
Faure, H., & Lemieux, C. (2009) · ACM Transactions on Modeling and Computer Simulation · doi:10.1145/1596519.1596520 · Cited by 71
End‐to‐End Deep Learning Methodology for Real‐Time Traffic Network Management
Hashemi, H., & Abdelghany, K. F. (2018) · Computer-Aided Civil and Infrastructure Engineering · doi:10.1111/mice.12376 · Cited by 63
OrbitalBrain: A Distributed Framework for Training ML Models in Space
Chabra, O., Li, C., Hsieh, K., Segarra, S., Arzani, B., Olsen, P., & Chandra, R. (2026) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/oasics.nines.2026.5 · Cited by 60
Biased Importance Sampling for Deep Neural Network TrainingPreprint
Katharopoulos, A., & Fleuret, F. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.00043 · Cited by 46
The Neural Network Approach to Inverse Problems in Differential EquationsPreprint
Xu, K., & Darve, E. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.07758 · Cited by 41
Deep convolutional neural networks for uncertainty propagation in random fields
Luo, X., & Kareem, A. (2019) · Computer-Aided Civil and Infrastructure Engineering · doi:10.1111/mice.12510 · Cited by 26
Kissas, G., Yang, Y., Hwuang, E., et al. (2019) · arXiv (Cornell University) · Cited by 5
Cited by
Scientific Machine Learning Through Physics–Informed Neural Networks: Where we are and What’s Next
Cuomo, S., Di Cola, V. S., Giampaolo, F., Rozza, G., Raissi, M., & Piccialli, F. (2022) · Journal of Scientific Computing · doi:10.1007/s10915-022-01939-z · Cited by 2,784
Wu, C., Zhu, M., Tan, Q., Kartha, Y., & Lu, L. (2023) · Computer Methods in Applied Mechanics and Engineering · doi:10.1016/j.cma.2022.115671 · Cited by 723
Gradient-enhanced physics-informed neural networks for forward and inverse PDE problems
Yu, J., Lu, L., Meng, X., & Karniadakis, G. E. (2022) · Computer Methods in Applied Mechanics and Engineering · doi:10.1016/j.cma.2022.114823 · Cited by 722
Yuan, L., Ni, Y.-Q., Deng, X.-Y., & Hao, S. (2022) · Journal of Computational Physics · doi:10.1016/j.jcp.2022.111260 · Cited by 422
Respecting causality for training physics-informed neural networks
Wang, S., Sankaran, S., & Perdikaris, P. (2024) · Computer Methods in Applied Mechanics and Engineering · doi:10.1016/j.cma.2024.116813 · Cited by 370
Metadata
- Authors
- Nabian, M. A., Gladstone, R. J., & Meidani, H.
- Venue
- Computer-Aided Civil and Infrastructure Engineering
- Publisher
- Elsevier BV
- Published
- 2021-04-09
- Volume
- 36
- Issue
- 8
- Pages
- 962-977
- ISSN
- 1093-9687 (ISSN-L)
- 1467-8667
- OpenAlex
- W3154575637
- Cited by
- 330
- References
- 73
- FWCI
- 14.89
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.