The DOI matches its Crossref record; 89 of the 89 references it cites were found in a registry or an index.
Physics-constrained deep learning for high-dimensional surrogate modeling and uncertainty quantification without labeled data
Open accessAbstract
Abstract Surrogate modeling and uncertainty quantification tasks for PDE systems are most often considered as supervised learning problems where input and output data pairs are used for training. The construction of such emulators is by definition a small data problem which poses challenges to deep learning approaches that have been developed to operate in the big data regime. Even in cases where such models have been shown to have good predictive capability in high dimensions, they fail to address constraints in the data implied by the PDE model. This paper provides a methodology that incorporates the governing equations of the physical model in the loss/likelihood functions. The resulting physics-constrained, deep learning models are trained without any labeled data (e.g. employing only input data) and provide comparable predictive responses with data-driven models while obeying the constraints of the problem at hand. This work employs a convolutional encoder-decoder neural network approach as well as a conditional flow-based generative model for the solution of PDEs, surrogate model construction, and uncertainty quantification tasks. The methodology is posed as a minimization problem of the reverse Kullback-Leibler (KL) divergence between the model predictive density and the reference conditional density, where the later is defined as the Boltzmann-Gibbs distribution at a given inverse temperature with the underlying potential relating to the PDE system of interest. The generalization capability of these models to out-of-distribution input is considered. Quantification and interpretation of the predictive uncertainty is provided for a number of problems.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
Densely Connected Convolutional Networks
Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K. Q. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.243 · Cited by 48,566
Unpaired image-to-image translation using cycle-consistent adversarial networks
Zhu, J.-Y., Park, T., Isola, P., & Efros, A. A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.244 · Cited by 23,602
Raissi, M., Perdikaris, P., & Karniadakis, G. E. (2019) · Journal of Computational Physics · doi:10.1016/j.jcp.2018.10.045 · Cited by 20,384
Self-citationLabaca-Castro, R. (2023) · Machine Learning under Malware Attack · doi:10.1007/978-3-658-40442-0_9 · Cited by 19,272
GAN(Generative Adversarial Nets)
(2017) · Journal of Japan Society for Fuzzy Theory and Intelligent Informatics · doi:10.3156/jsoft.29.5_177_2 · Cited by 19,270
Advances in neural information processing systems 7
(1996) · Computers & Mathematics with Applications · doi:10.1016/s0898-1221(96)90463-0 · Cited by 14,281
Advances in Neural Information Processing Systems 29
Teymur, O., Zygalakis, K., & Calderhead, B. (2016) · Cited by 13,887
Conditional Generative Adversarial Nets
Mirza, M., & Osindero, S. (2014) · arXiv · doi:10.48550/arxiv.1411.1784 · Cited by 8,863
Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
Finn, C., Abbeel, P., & Levine, S. (2017) · arXiv · doi:10.48550/arxiv.1703.03400 · Cited by 8,512
Proceedings of the 25th international conference on Machine learning - ICML '08
(2008) · doi:10.1145/1390156 · Cited by 6,680
Dropout as a bayesian approximation: Representing model uncertainty in deep learning
Gal, Y., & Ghahramani, Z. (2015) · arXiv · doi:10.48550/arxiv.1506.02142 · Cited by 6,471
On calibration of modern neural networks
Guo, C., Pleiss, G., Sun, Y., & Weinberger, K. Q. (2017) · arXiv · doi:10.48550/arxiv.1706.04599 · Cited by 3,435
Evolving Neural Networks through Augmenting Topologies
Stanley, K. O., & Miikkulainen, R. (2002) · Evolutionary Computation · doi:10.1162/106365602320169811 · Cited by 3,258
2017 IEEE International Conference on Computer Vision (ICCV)
(2017) · doi:10.1109/iccv37128.2017 · Cited by 3,056
A practical Bayesian framework for backpropagation networks
MacKay, D. J. C. (1992) · Neural Computation · doi:10.1162/neco.1992.4.3.448 · Cited by 2,917
Advances in Neural Information Processing Systems 32 (NeurIPS 2019)
Wallach, H., Larochelle, H., Beygelzimer, A., d’Alché–Buc, F., Fox, E. B., & Garnett, R. (2020) · HAL · Cited by 2,451
Artificial neural networks for solving ordinary and partial differential equations
Lagaris, I. E., Likas, A., & Fotiadis, D. I. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/72.712178 · Cited by 2,412
DGM: A deep learning algorithm for solving partial differential equations
Sirignano, J., & Spiliopoulos, K. (2018) · Journal of Computational Physics · doi:10.1016/j.jcp.2018.08.029 · Cited by 2,208
Density estimation using Real NVP
Dinh, L., Sohl-Dickstein, J., & Bengio, S. (2016) · arXiv · Cited by 2,067
A Tutorial on Energy-Based Learning
LeCun, Y., Chopra, S., Hadsell, R. T., Ranzato, A., & Huang, F. J. (2006) · Cited by 1,931
The FEniCS Project Version 1.5
Alnæs, M. S., Blechta, J., Hake, J. E., Johansson, A., Kehlet, B., Logg, A., Richardson, C., Ring, J., Rognes, M. E., & Wells, G. N. (2015) · Department of Earth Sciences EPrints Repository · doi:10.11588/ans.2015.100.20553 · Cited by 1,877
Conditional Image Generation with PixelCNN Decoders
van den Oord, A., Kalchbrenner, N., Vinyals, O., Espeholt, L., Graves, A., & Kavukcuoglu, K. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.05328 · Cited by 1,833
Solving high-dimensional partial differential equations using deep learning
Han, J., Jentzen, A., & E, W. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1718942115 · Cited by 1,722
Learning a variational network for reconstruction of accelerated MRI data
Hammernik, K., Klatzer, T., Kobler, E., Recht, M. P., Sodickson, D. K., Pock, T., & Knoll, F. (2018) · Magnetic Resonance in Medicine · doi:10.1002/mrm.26977 · Cited by 1,713
Reynolds averaged turbulence modelling using deep neural networks with embedded invariance
Ling, J., Kurzawski, A., & Templeton, J. (2016) · Journal of Fluid Mechanics · doi:10.1017/jfm.2016.615 · Cited by 1,624
Predicting the output from a complex computer code when fast approximations are available
Kennedy, M. (2000) · Biometrika · doi:10.1093/biomet/87.1.1 · Cited by 1,605
2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
(2016) · doi:10.1109/cvpr33180.2016 · Cited by 1,534
Deep learning for universal linear embeddings of nonlinear dynamics
Lusch, B., Kutz, J. N., & Brunton, S. L. (2018) · Nature Communications · doi:10.1038/s41467-018-07210-0 · Cited by 1,424
The Deep Ritz Method: A Deep Learning-Based Numerical Algorithm for Solving Variational Problems
E, W., & Yu, B. (2018) · Communications in Mathematics and Statistics · doi:10.1007/s40304-018-0127-z · Cited by 1,418
Variational Inference with Normalizing Flows
Rezende, D. J., & Mohamed, S. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1505.05770 · Cited by 1,253
A hybrid neural network‐first principles approach to process modeling
Psichogios, D. C., & Ungar, L. H. (1992) · AIChE Journal · doi:10.1002/aic.690381003 · Cited by 982
Artificial Neural Networks for Solving Ordinary and Partial Differential Equations
Lagaris, I. E., Likas, A., & Fotiadis, D. I. (1998) · Cited by 944
Auto-Encoding Variational Bayes
Chen, Y., Liu, J., Peng, L., Wu, Y., Xu, Y., & Zhang, Z. (2024) · Cambridge Explorations in Arts and Sciences · doi:10.61603/ceas.v2i1.33 · Cited by 934
Raissi, M., Perdikaris, P. G., & Karniadakis, G. E. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.10561 · Cited by 925
Self-citationToward Multimodal Image-to-Image Translation
Zhu, J.-Y., Zhang, R., Pathak, D., Darrell, T. J., Efros, A. A., Wang, O., & Shechtman, E. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.11586 · Cited by 906
Boltzmann generators: Sampling equilibrium states of many-body systems with deep learning
Noé, F., Olsson, S., Köhler, J., & Wu, H. (2019) · Science · doi:10.1126/science.aaw1147 · Cited by 744
Ulyanov, D., Vedaldi, A., & Lempitsky, V. (2020) · International Journal of Computer Vision · doi:10.1007/s11263-020-01303-4 · Cited by 719
Compositional pattern producing networks: A novel abstraction of development
Stanley, K. O. (2007) · Genetic Programming and Evolvable Machines · doi:10.1007/s10710-007-9028-8 · Cited by 713
Zhu, Y., & Zabaras, N. (2018) · Journal of Computational Physics · doi:10.1016/j.jcp.2018.04.018 · Cited by 704
Self-citationBerg, J., & Nyström, K. (2018) · Neurocomputing · doi:10.1016/j.neucom.2018.06.056 · Cited by 657
The Comparison and Evaluation of Forecasters
DeGroot, M. H., & Fienberg, S. E. (1983) · The Statistician · doi:10.2307/2987588 · Cited by 604 · cited 1×
“Here, the predictive uncertainty is calibrated using the reliability diagram [59].”
MethodWang, T.-C., Liu, M.-Y., Zhu, J.-Y., Liu, G., Tao, A., Kautz, J., & Catanzaro, B. (2018) · arXiv · doi:10.48550/arxiv.1808.06601 · Cited by 594
Do Deep Generative Models Know What They Don't Know?Preprint
Nalisnick, E., Matsukawa, A., Teh, Y. W., Görür, D., & Lakshminarayanan, B. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.09136 · Cited by 591
Solving ill-posed inverse problems using iterative deep neural networks
Adler, J., & Öktem, O. (2017) · Inverse Problems · doi:10.1088/1361-6420/aa9581 · Cited by 529
(2007) · The MIT Press eBooks · doi:10.7551/mitpress/7443.001.0001 · Cited by 525
GPyTorch: Blackbox Matrix-Matrix Gaussian Process Inference with GPU Acceleration
Gardner, J. R., Pleiss, G., Bindel, D., Weinberger, K. Q., & Wilson, A. G. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.11165 · Cited by 496
Physics-Informed Generative Adversarial Networks for Stochastic Differential Equations
Yang, L., Zhang, D., & Karniadakis, G. E. (2020) · SIAM Journal on Scientific Computing · doi:10.1137/18m1225409 · Cited by 482
Tripathy, R. K., & Bilionis, I. (2018) · Journal of Computational Physics · doi:10.1016/j.jcp.2018.08.036 · Cited by 464
Variational Dropout and the Local Reparameterization Trick
Kingma, D. P., Salimans, T., & Welling, M. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.02557 · Cited by 418
Adversarial uncertainty quantification in physics-informed neural networks
Yang, Y., & Perdikaris, P. (2019) · Journal of Computational Physics · doi:10.1016/j.jcp.2019.05.027 · Cited by 414
Self-citationTraining‐Image Based Geostatistical Inversion Using a Spatial Generative Adversarial Neural Network
Laloy, E., Hérault, R., Jacques, D., & Linde, N. (2018) · Water Resources Research · doi:10.1002/2017wr022148 · Cited by 382
Deep Fluids: A Generative Network for Parameterized Fluid Simulations
Kim, B., Azevedo, V. C., Thuerey, N., Kim, T., Gross, M., & Solenthaler, B. (2019) · Computer Graphics Forum · doi:10.1111/cgf.13619 · Cited by 372
Mo, S., Zhu, Y., Zabaras, N., Shi, X., & Wu, J. (2019) · Water Resources Research · doi:10.1029/2018wr023528 · Cited by 364
Self-citationAccurate Uncertainties for Deep Learning Using Calibrated Regression
Kuleshov, V., Fenner, N., & Ermon, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.00263 · Cited by 337
Parallel WaveNet: Fast High-Fidelity Speech Synthesis
van den Oord, A., Li, Y., Babuschkin, I., Simonyan, K., Vinyals, O., Kavukcuoglu, K., van den Driessche, G., Lockhart, E., Cobo, L. C., Stimberg, F., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.10433 · Cited by 330
Beck, C., E, W., & Jentzen, A. (2019) · Journal of Nonlinear Science · doi:10.1007/s00332-018-9525-3 · Cited by 308
Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithm
Liu, Q., & Wang, D. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1608.04471 · Cited by 292
Accelerating Eulerian Fluid Simulation With Convolutional Networks
Tompson, J., Schlachter, K., Sprechmann, P., & Perlin, K. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1607.03597 · Cited by 280
Label-Free Supervision of Neural Networks with Physics and Domain Knowledge
Stewart, R., & Ermon, S. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.10934 · Cited by 274
Probabilistic numerics and uncertainty in computations
Hennig, P., Osborne, M. A., & Girolami, M. (2015) · Proceedings of the Royal Society A Mathematical Physical and Engineering Sciences · doi:10.1098/rspa.2015.0142 · Cited by 265
The numerical solution of linear ordinary differential equations by feedforward neural networks
Meade, A. J., & Fernandez, A. A. (1994) · Mathematical and Computer Modelling · doi:10.1016/0895-7177(94)90095-7 · Cited by 256
Diversity-Sensitive Conditional Generative Adversarial NetworksPreprint
Yang, D., Hong, S., Jang, Y., Zhao, T., & Lee, H. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.09024 · Cited by 233
Probabilistic numerics and uncertainty in computations
Ma, O., Numerics, G. M. P., Osborne, M. A., & Girolami, M. (2015) · Cited by 224
Nonlinear corrections to Darcy's law at low Reynolds numbers
FIRDAOUSS, M., GUERMOND, J.-L., & LE QUÉRÉ, P. (1997) · Journal of Fluid Mechanics · doi:10.1017/s0022112097005843 · Cited by 221
Lagging Inference Networks and Posterior Collapse in Variational Autoencoders
He, J., Spokoyny, D., Neubig, G., & Berg-Kirkpatrick, T. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.05534 · Cited by 189
Kernel Interpolation for Scalable Structured Gaussian Processes (KISS-GP)
Wilson, A. G., & Nickisch, H. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1503.01057 · Cited by 184
PDE-Net: Learning PDEs from Data
Long, Z., Lu, Y., Ma, X., & Dong, B. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1710.09668 · Cited by 184
Bilionis, I., Zabaras, N., Konomi, B. A., & Lin, G. (2013) · Journal of Computational Physics · doi:10.1016/j.jcp.2013.01.011 · Cited by 167
Self-citationGrohs, P., Hornung, F., Jentzen, A., & von Wurstemberger, P. (2023) · Memoirs of the American Mathematical Society · doi:10.1090/memo/1410 · Cited by 165
A deep learning solution approach for high-dimensional random differential equations
Nabian, M. A., & Meidani, H. (2019) · Probabilistic Engineering Mechanics · doi:10.1016/j.probengmech.2019.05.001 · Cited by 164
Geneva, N., & Zabaras, N. J. (2019) · Journal of Computational Physics · doi:10.1016/j.jcp.2019.01.021 · Cited by 158
Self-citationRaissi, M. (2023) · WORLD SCIENTIFIC eBooks · doi:10.1142/9789811280306_0018 · Cited by 148
Neural Network Renormalization Group
Li, S.-H., & Wang, L. (2018) · Physical Review Letters · doi:10.1103/physrevlett.121.260601 · Cited by 140
Spherical CNNs on Unstructured GridsPreprint
Jiang, C. M., Huang, J., Kashinath, K., Prabhat, Marcus, P., & Nießner, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.02039 · Cited by 128
Data‐driven projection method in fluid simulation
Yang, C., Yang, X., & Xiao, X. (2016) · Computer Animation and Virtual Worlds · doi:10.1002/cav.1695 · Cited by 121
tempoGAN: A Temporally Coherent, Volumetric GAN for Super-resolution Fluid FlowPreprint
Xie, Y. X., Franz, E., Chu, M., & Thuerey, N. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1801.09710 · Cited by 100
Deep multiscale model learning
Wang, Y., Cheung, S. W., Chung, E. T., Efendiev, Y. R., & Wang, M. (2019) · Journal of Computational Physics · doi:10.1016/j.jcp.2019.109071 · Cited by 97
Bayesian probabilistic numerical methods
Cockayne, J., Oates, C., Sullivan, T., Girolami, M., et al. (2019) · Warwick Research Archive Portal (University of Warwick) · Cited by 90
A Multiscale Neural Network Based on Hierarchical Matrices
Fan, Y., Lin, L., Ying, L., & Zepeda-Núñez, L. (2019) · Multiscale Modeling and Simulation · doi:10.1137/18m1203602 · Cited by 84
Convolutional Gaussian ProcessesPreprint
van der Wilk, M., Rasmussen, C. E., & Hensman, J. J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1709.01894 · Cited by 67
Generative Ensembles for Robust Anomaly Detection
Choi, H., & Jang, E. (2018) · Cited by 51
Atkinson, S., & Zabaras, N. J. (2019) · Journal of Computational Physics · doi:10.1016/j.jcp.2018.12.037 · Cited by 45
Self-citationConvolutional Gaussian Processes
van der Wilk, M., Rasmussen, C. E., & Hensman, J. J. (2017) · Apollo (University of Cambridge) · doi:10.17863/cam.21271 · Cited by 44
Probabilistic Meshless Methods for Partial Differential Equations and Bayesian Inverse Problems.Preprint
Cockayne, J., Oates, C. J., Sullivan, T. J., et al. (2016) · arXiv (Cornell University) · Cited by 31
Bayesian Model and Dimension Reduction for Uncertainty Propagation: Applications in Random Media
Grigo, C., & Koutsourelakis, P. (2019) · SIAM/ASA Journal on Uncertainty Quantification · doi:10.1137/17m1155867 · Cited by 28
Self-citationBeck, C., Weinan, E., & Jentzen, A. (2017) · Repository for Publications and Research Data (ETH Zurich) · doi:10.3929/ethz-b-000314918 · Cited by 14
Cited by
Physics-informed machine learning
Karniadakis, G. E., Kevrekidis, I. G., Lu, L., Perdikaris, P., Wang, S., & Yang, L. (2021) · Nature Reviews Physics · doi:10.1038/s42254-021-00314-5 · Cited by 8,718
Self-citationLearning nonlinear operators via DeepONet based on the universal approximation theorem of operators
Lu, L., Jin, P., Pang, G., Zhang, Z., & Karniadakis, G. E. (2021) · Nature Machine Intelligence · doi:10.1038/s42256-021-00302-5 · Cited by 3,418
Scientific Machine Learning Through Physics–Informed Neural Networks: Where we are and What’s Next
Cuomo, S., Di Cola, V. S., Giampaolo, F., Rozza, G., Raissi, M., & Piccialli, F. (2022) · Journal of Scientific Computing · doi:10.1007/s10915-022-01939-z · Cited by 2,784
A review of uncertainty quantification in deep learning: Techniques, applications and challenges
Abdar, M., Pourpanah, F., Hussain, S., Rezazadegan, D., Liu, L., Ghavamzadeh, M., Fieguth, P., Cao, X., Khosravi, A., Acharya, U. R., et al. (2021) · Information Fusion · doi:10.1016/j.inffus.2021.05.008 · Cited by 3,111
“[392] devised a physics-constrained deep learning model for high-dimensional UQ and surrogate modeling without labeled data.”
BackgroundDeepXDE: A Deep Learning Library for Solving Differential Equations
Lu, L., Meng, X., Mao, Z., & Karniadakis, G. E. (2021) · SIAM Review · doi:10.1137/19m1274067 · Cited by 2,512
Metadata
- Authors
- Zhu, Y., Zabaras, N., Koutsourelakis, P.-S., & Perdikaris, P.
- Venue
- Journal of Computational Physics
- Publisher
- Elsevier BV
- Published
- 2019-10
- Volume
- 394
- Pages
- 56-81
- ISSN
- 0021-9991 (ISSN-L)
- OpenAlex
- W2908541468
- Cited by
- 1,030
- References
- 89
- FWCI
- 46.43
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.