The DOI matches its Crossref record; 37 of the 37 references it cites were found in a registry or an index.
A comprehensive study of non-adaptive and residual-based adaptive sampling for physics-informed neural networks
Open accessAbstract
Physics-informed neural networks (PINNs) have shown to be an effective tool for solving forward and inverse problems of partial differential equations (PDEs). PINNs embed the PDEs into the loss of the neural network, and this PDE loss is evaluated at a set of scattered residual points. The distribution of these points are highly important to the performance of PINNs. However, in the existing studies on PINNs, only a few simple residual point sampling methods have mainly been used. Here, we present a comprehensive study of two categories of sampling: non-adaptive uniform sampling and adaptive nonuniform sampling. We consider six uniform sampling, including (1) equispaced uniform grid, (2) uniformly random sampling, (3) Latin hypercube sampling, (4) Halton sequence, (5) Hammersley sequence, and (6) Sobol sequence. We also consider a resampling strategy for uniform sampling. To improve the sampling efficiency and the accuracy of PINNs, we propose two new residual-based adaptive sampling methods: residual-based adaptive distribution (RAD) and residual-based adaptive refinement with distribution (RAR-D), which dynamically improve the distribution of residual points based on the PDE residuals during training. Hence, we have considered a total of 10 different sampling methods, including six non-adaptive uniform sampling, uniform sampling with resampling, two proposed adaptive sampling, and an existing adaptive sampling. We extensively tested the performance of these sampling methods for four forward problems and two inverse problems in many setups. Our numerical results presented in this study are summarized from more than 6000 simulations of PINNs. We show that the proposed adaptive sampling methods of RAD and RAR-D significantly improve the accuracy of PINNs with fewer residual points. The results obtained in this study can also be used as a practical guideline in choosing sampling methods.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Raissi, M., Perdikaris, P., & Karniadakis, G. E. (2019) · Journal of Computational Physics · doi:10.1016/j.jcp.2018.10.045 · Cited by 20,384
Labaca-Castro, R. (2023) · Machine Learning under Malware Attack · doi:10.1007/978-3-658-40442-0_9 · Cited by 19,272
Physics-informed machine learning
Karniadakis, G. E., Kevrekidis, I. G., Lu, L., Perdikaris, P., Wang, S., & Yang, L. (2021) · Nature Reviews Physics · doi:10.1038/s42254-021-00314-5 · Cited by 8,718 · cited 2×
“Despite the past success, addressing a wide range of PDE problems with increasing levels of complexity can be theoretically and practically challenging, and thus many aspects of PINNs still require further improvements to achieve more accurate prediction, higher computational efficiency, and training robustness [4].”
BackgroundSelf-citationOn the distribution of points in a cube and the approximate evaluation of integrals
Sobol', I. M. (1967) · USSR Computational Mathematics and Mathematical Physics · doi:10.1016/0041-5553(67)90144-9 · Cited by 2,975
Mckay, M. D., Beckman, R. J., & Conover, W. J. (2000) · Technometrics · doi:10.1080/00401706.2000.10485979 · Cited by 2,833
DeepXDE: A Deep Learning Library for Solving Differential Equations
Lu, L., Meng, X., Mao, Z., & Karniadakis, G. E. (2021) · SIAM Review · doi:10.1137/19m1274067 · Cited by 2,512
Self-citationLarge Sample Properties of Simulations Using Latin Hypercube Sampling
Stein, M. (1987) · Technometrics · doi:10.1080/00401706.1987.10488205 · Cited by 2,471
Hidden fluid mechanics: Learning velocity and pressure fields from flow visualizations
Raissi, M., Yazdani, A., & Karniadakis, G. E. (2020) · Science · doi:10.1126/science.aaw4741 · Cited by 2,103
Understanding and Mitigating Gradient Flow Pathologies in Physics-Informed Neural Networks
Wang, S., Teng, Y., & Perdikaris, P. (2021) · SIAM Journal on Scientific Computing · doi:10.1137/20m1318043 · Cited by 2,048
Halton, J. H. (1960) · Numerische Mathematik · doi:10.1007/bf01386213 · Cited by 1,985
When and why PINNs fail to train: A neural tangent kernel perspective
Wang, S., Yu, X., & Perdikaris, P. (2022) · Journal of Computational Physics · doi:10.1016/j.jcp.2021.110768 · Cited by 1,496
Physics-informed neural networks for high-speed flows
Mao, Z., Jagtap, A. D., & Karniadakis, G. E. (2020) · Computer Methods in Applied Mechanics and Engineering · doi:10.1016/j.cma.2019.112789 · Cited by 1,282
D. Jagtap, A., & Em Karniadakis, G. (2020) · Communications in Computational Physics · doi:10.4208/cicp.oa-2020-0164 · Cited by 1,130
fPINNs: Fractional Physics-Informed Neural Networks
Pang, G., Lu, L., & Karniadakis, G. E. (2019) · SIAM Journal on Scientific Computing · doi:10.1137/18m1229845 · Cited by 1,105 · cited 3×
“In addition to standard PDEs, PINNs have also been extended to solve other types of PDEs, including integro-differential equations [3], fractional PDEs [9], and stochastic PDEs [10].”
BackgroundMethodSelf-citationPhysics-Informed Neural Networks with Hard Constraints for Inverse Design
Lu, L., Pestourie, R., Yao, W., Wang, Z., Verdugo, F., & Johnson, S. G. (2021) · SIAM Journal on Scientific Computing · doi:10.1137/21m1397908 · Cited by 771
Self-citationGradient-enhanced physics-informed neural networks for forward and inverse PDE problems
Yu, J., Lu, L., Meng, X., & Karniadakis, G. E. (2022) · Computer Methods in Applied Mechanics and Engineering · doi:10.1016/j.cma.2022.114823 · Cited by 722
Self-citationPhysics-informed neural networks for inverse problems in nano-optics and metamaterials
Chen, Y., Lu, L., Karniadakis, G. E., & Dal Negro, L. (2020) · Optics Express · doi:10.1364/oe.384875 · Cited by 701
Self-citationPPINN: Parareal physics-informed neural network for time-dependent PDEs
Meng, X., Li, Z., Zhang, D., & Karniadakis, G. E. (2020) · Computer Methods in Applied Mechanics and Engineering · doi:10.1016/j.cma.2020.113250 · Cited by 654
Zhang, D., Lu, L., Guo, L., & Karniadakis, G. E. (2019) · Journal of Computational Physics · doi:10.1016/j.jcp.2019.07.048 · Cited by 513
Self-citationSelf-adaptive loss balanced Physics-informed neural networks
Xiang, Z., Peng, W., Liu, X., & Yao, W. (2022) · Neurocomputing · doi:10.1016/j.neucom.2022.05.015 · Cited by 507
Parallel physics-informed neural networks via domain decomposition
Shukla, K., Jagtap, A. D., & Karniadakis, G. E. (2021) · Journal of Computational Physics · doi:10.1016/j.jcp.2021.110683 · Cited by 431
Efficient training of physics‐informed neural networks via importance sampling
Nabian, M. A., Gladstone, R. J., & Meidani, H. (2021) · Computer-Aided Civil and Infrastructure Engineering · doi:10.1111/mice.12685 · Cited by 330
Systems biology informed deep learning for inferring parameters and hidden dynamics
Yazdani, A., Lu, L., Raissi, M., & Karniadakis, G. E. (2020) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1007575 · Cited by 329
Self-citationSolving Allen-Cahn and Cahn-Hilliard Equations Using the Adaptive Physics Informed Neural Networks
L. Wight, C., & Zhao, J. (2021) · Communications in Computational Physics · doi:10.4208/cicp.oa-2020-0086 · Cited by 294
Mattey, R., & Ghosh, S. (2022) · Computer Methods in Applied Mechanics and Engineering · doi:10.1016/j.cma.2021.114474 · Cited by 278
Meta-learning PINN loss functions
Psaros, A. F., Kawaguchi, K., & Karniadakis, G. E. (2022) · Journal of Computational Physics · doi:10.1016/j.jcp.2022.111121 · Cited by 195
Dong, S. S., & Ni, N. (2021) · Journal of Computational Physics · doi:10.1016/j.jcp.2021.110242 · Cited by 192
Hanna, J. M., Aguado, J. V., Comas-Cardona, S., Askri, R., & Borzacchiello, D. (2022) · Computer Methods in Applied Mechanics and Engineering · doi:10.1016/j.cma.2022.115100 · Cited by 146
Self-Adaptive Physics-Informed Neural Networks using a Soft Attention MechanismPreprint
McClenny, L. D., & Braga-Neto, U. (2020) · arXiv (Cornell University) · doi:10.1016/j.jcp.2022.111722 · Cited by 114
Daneker, M., Zhang, Z., Karniadakis, G. E., & Lu, L. (2023) · Methods in molecular biology · doi:10.1007/978-1-0716-3008-2_4 · Cited by 98
Self-citationAdaptive deep neural networks methods for high-dimensional partial differential equations
Zeng, S., Zhang, Z., & Zou, Q. (2022) · Journal of Computational Physics · doi:10.1016/j.jcp.2022.111232 · Cited by 97
Active learning based sampling for high-dimensional nonlinear partial differential equations
Gao, W., & Wang, C. (2022) · Journal of Computational Physics · doi:10.1016/j.jcp.2022.111848 · Cited by 92
Investigating molecular transport in the human brain from MRI with physics-informed neural networks
Zapf, B., Haubner, J., Kuchta, M., Ringstad, G. A., Eide, P. K., & Mardal, K. (2022) · Scientific Reports · doi:10.1038/s41598-022-19157-w · Cited by 78
SelectNet: Self-paced learning for high-dimensional partial differential equations
Gu, Y., Yang, H., & Zhou, C. (2021) · Journal of Computational Physics · doi:10.1016/j.jcp.2021.110444 · Cited by 68
Lagari, P. L., Tsoukalas, L. H., Safarkhani, S., & Lagaris, I. E. (2020) · International Journal of Artificial Intelligence Tools · doi:10.1142/s0218213020500098 · Cited by 67
Improved Training of Physics-Informed Neural Networks with Model Ensembles
Haitsiukevich, K., & Ilin, A. (2023) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn54540.2023.10191822 · Cited by 47
Guo, H., Zhuang, X., Meng, X., & Rabczuk, T. (2020) · arXiv (Cornell University) · Cited by 3
Cited by
Qi, X., de Almeida, G. A. M., & Maldonado, S. (2024) · Journal of Hydrology · doi:10.1016/j.jhydrol.2024.131263 · Cited by 78
Du, Z., & Lu, R. (2025) · Batteries · doi:10.3390/batteries11060204 · Cited by 58
Metadata
- Authors
- Wu, C., Zhu, M., Tan, Q., Kartha, Y., & Lu, L.
- Venue
- Computer Methods in Applied Mechanics and Engineering
- Publisher
- Elsevier BV
- Published
- 2023-01
- Volume
- 403
- Pages
- 115671
- ISSN
- 0045-7825 (ISSN-L)
- OpenAlex
- W4307154444
- Cited by
- 723
- References
- 37
- FWCI
- 43.62
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.