The DOI matches its Crossref record; 71 of the 71 references it cites were found in a registry or an index.
Modeling the dynamics of PDE systems with physics-constrained deep auto-regressive networks
Open accessAbstract
In recent years, deep learning has proven to be a viable methodology for surrogate modeling and uncertainty quantification for a vast number of physical systems. However, in their traditional form, such models can require a large amount of training data. This is of particular importance for various engineering and scientific applications where data may be extremely expensive to obtain. To overcome this shortcoming, physics-constrained deep learning provides a promising methodology as it only utilizes the governing equations. In this work, we propose a novel auto-regressive dense encoder-decoder convolutional neural network to solve and model non-linear dynamical systems without training data at a computational cost that is potentially magnitudes lower than standard numerical solvers. This model includes a Bayesian framework that allows for uncertainty quantification of the predicted quantities of interest at each time-step. We rigorously test this model on several non-linear transient partial differential equation systems including the turbulence of the Kuramoto-Sivashinsky equation, multi-shock formation and interaction with 1D Burgers' equation and 2D wave dynamics with coupled Burgers' equations. For each system, the predictive results and uncertainty are presented and discussed together with comparisons to the results obtained from traditional numerical analysis methods.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 2×
“Among the most classical methods are standard recurrent neural networks which tend to be difficult to train [27], as well as long-short term memory (LSTM) architectures [28].”
BackgroundAdam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Exploiting Generative AI to Scale up Intelligent Tutoring Systems
Jakubův, J., Chvalovský, K., Goertzel, Z., Kaliszyk, C., Olšák, M., Piotrowski, B., Schulz, S., Suda, M., & Urban, J. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,658
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818
Raissi, M., Perdikaris, P., & Karniadakis, G. E. (2019) · Journal of Computational Physics · doi:10.1016/j.jcp.2018.10.045 · Cited by 20,384
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S. (2024) · arXiv (Cornell University) · doi:10.57702/o9raffed · Cited by 14,214
Decoupled Weight Decay Regularization
Loshchilov, I., & Hutter, F. (2017) · arXiv · doi:10.48550/arxiv.1711.05101 · Cited by 12,522
(2016) · Cited by 11,679
(2017) · Cited by 11,625
GMRES: A Generalized Minimal Residual Algorithm for Solving Nonsymmetric Linear Systems
Saad, Y., & Schultz, M. H. (1986) · SIAM Journal on Scientific and Statistical Computing · doi:10.1137/0907058 · Cited by 11,154
Whitham, G. B., & Fowler, R. G. (1975) · Physics Today · doi:10.1063/1.3069011 · Cited by 8,179
(2006) · Springer eBooks · Cited by 8,123
(2011) · Cited by 6,993
Dropout as a bayesian approximation: Representing model uncertainty in deep learning
Gal, Y., & Ghahramani, Z. (2015) · arXiv · doi:10.48550/arxiv.1506.02142 · Cited by 6,471
A calculation procedure for heat, mass and momentum transfer in three-dimensional parabolic flows
Patankar, S. V., & SPALDING, D. B. (1972) · International Journal of Heat and Mass Transfer · doi:10.1016/0017-9310(72)90054-3 · Cited by 6,006
On the difficulty of training recurrent neural networks
Pascanu, R., Mikolov, T., & Bengio, Y. (2012) · arXiv · doi:10.48550/arxiv.1211.5063 · Cited by 4,310
Bayesian Learning for Neural Networks
Neal, R. M. (1996) · Lecture Notes in Statistics · doi:10.1007/978-1-4612-0745-0 · Cited by 4,276
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1512.03385 · Cited by 4,196
Geometric Deep Learning: Going beyond Euclidean data
Bronstein, M. M., Bruna, J., LeCun, Y., Szlam, A., & Vandergheynst, P. (2017) · IEEE Signal Processing Magazine · doi:10.1109/msp.2017.2693418 · Cited by 3,661
What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?
Kendall, A., & Gal, Y. (2017) · arXiv · doi:10.48550/arxiv.1703.04977 · Cited by 3,088
A practical Bayesian framework for backpropagation networks
MacKay, D. J. C. (1992) · Neural Computation · doi:10.1162/neco.1992.4.3.448 · Cited by 2,917
A First Course in Numerical Analysis
Maisel, H., & Ralston, A. (1966) · Technometrics · doi:10.1080/00401706.1966.10490393 · Cited by 2,559
Artificial neural networks for solving ordinary and partial differential equations
Lagaris, I. E., Likas, A., & Fotiadis, D. I. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/72.712178 · Cited by 2,412
DGM: A deep learning algorithm for solving partial differential equations
Sirignano, J., & Spiliopoulos, K. (2018) · Journal of Computational Physics · doi:10.1016/j.jcp.2018.08.029 · Cited by 2,208
Density estimation using Real NVP
Dinh, L., Sohl-Dickstein, J., & Bengio, S. (2016) · arXiv · Cited by 2,067
Fixing Weight Decay Regularization in Adam
Loshchilov, I., & Hutter, F. (2018) · Cited by 1,986
Hierarchical recurrent neural network for skeleton based action recognition
Du, Y., Wang, W., & Wang, L. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298714 · Cited by 1,895
The FEniCS Project Version 1.5
Alnæs, M. S., Blechta, J., Hake, J. E., Johansson, A., Kehlet, B., Logg, A., Richardson, C., Ring, J., Rognes, M. E., & Wells, G. N. (2015) · Department of Earth Sciences EPrints Repository · doi:10.11588/ans.2015.100.20553 · Cited by 1,877
Acceleration of Stochastic Approximation by Averaging
Polyak, B. T., & Juditsky, A. B. (1992) · SIAM Journal on Control and Optimization · doi:10.1137/0330046 · Cited by 1,827 · cited 1×
“Bayesian method that builds upon the Stochastic Weight Averaging (SWA), an optimization methodology where running averages of model parameters are kept during the stochastic gradient descent (SGD) procedure [56, 57, 58].”
Method2014 IEEE Conference on Computer Vision and Pattern Recognition
(2014) · doi:10.1109/cvpr20399.2014 · Cited by 1,774
Reynolds averaged turbulence modelling using deep neural networks with embedded invariance
Ling, J., Kurzawski, A., & Templeton, J. (2016) · Journal of Fluid Mechanics · doi:10.1017/jfm.2016.615 · Cited by 1,624
LeVeque, R. J. (2007) · Society for Industrial and Applied Mathematics eBooks · doi:10.1137/1.9780898717839 · Cited by 1,526
The Deep Ritz Method: A Deep Learning-Based Numerical Algorithm for Solving Variational Problems
E, W., & Yu, B. (2018) · Communications in Mathematics and Statistics · doi:10.1007/s40304-018-0127-z · Cited by 1,418
Averaging Weights Leads to Wider Optima and Better Generalization
Izmailov, P., Podoprikhin, D., Garipov, T., Vetrov, D., & Wilson, A. G. (2018) · arXiv · doi:10.48550/arxiv.1803.05407 · Cited by 1,409
Weight Uncertainty in Neural Networks
Blundell, C., Cornebise, J., Kavukcuoglu, K., & Wierstra, D. (2015) · arXiv · doi:10.48550/arxiv.1505.05424 · Cited by 1,342
Persistent Propagation of Concentration Waves in Dissipative Media Far from Thermal Equilibrium
Kuramoto, Y., & Tsuzuki, T. (1976) · Progress of Theoretical Physics · doi:10.1143/ptp.55.356 · Cited by 1,032
Zhu, Y., Zabaras, N., Koutsourelakis, P.-S., & Perdikaris, P. (2019) · Journal of Computational Physics · doi:10.1016/j.jcp.2019.05.024 · Cited by 1,030
Self-citationNeural Network Classifiers Estimate Bayesian a posteriori Probabilities
Richard, M. D., & Lippmann, R. P. (1991) · Neural Computation · doi:10.1162/neco.1991.3.4.461 · Cited by 986
A hybrid neural network‐first principles approach to process modeling
Psichogios, D. C., & Ungar, L. H. (1992) · AIChE Journal · doi:10.1002/aic.690381003 · Cited by 982
(2008) · Choice Reviews Online · doi:10.5860/choice.45-4438 · Cited by 969
Artificial Neural Networks for Solving Ordinary and Partial Differential Equations
Lagaris, I. E., Likas, A., & Fotiadis, D. I. (1998) · Cited by 944
VIII. The deferred approach to the limit
Richardson, L. F., & Gaunt, J. A. (1927) · Philosophical Transactions of the Royal Society of London. Series A, Containing Papers of a Mathematical or Physical Character · doi:10.1098/rsta.1927.0008 · Cited by 839
A first course in the numerical analysis of differential equations
(2010) · Choice Reviews Online · doi:10.5860/choice.47-2622 · Cited by 779
Zhu, Y., & Zabaras, N. (2018) · Journal of Computational Physics · doi:10.1016/j.jcp.2018.04.018 · Cited by 704
Self-citationBerg, J., & Nyström, K. (2018) · Neurocomputing · doi:10.1016/j.neucom.2018.06.056 · Cited by 657
Using machine learning to replicate chaotic attractors and calculate Lyapunov exponents from data
Pathak, J., Lu, Z., Hunt, B. R., Girvan, M., & Ott, E. (2017) · Chaos: An Interdisciplinary Journal of Nonlinear Science · doi:10.1063/1.5010300 · Cited by 620
Neural-network methods for boundary value problems with irregular boundaries
Lagaris, I. E., Likas, A. C., & Papageorgiou, D. G. (2000) · IEEE Transactions on Neural Networks · doi:10.1109/72.870037 · Cited by 521
Tripathy, R. K., & Bilionis, I. (2018) · Journal of Computational Physics · doi:10.1016/j.jcp.2018.08.036 · Cited by 464
A Simple Baseline for Bayesian Uncertainty in Deep Learning
Maddox, W., Garipov, T., Izmailov, P., Vetrov, D., & Wilson, A. G. (2019) · arXiv · doi:10.48550/arxiv.1902.02476 · Cited by 429 · cited 5×
“yesian dropout approximation [51] and Steins variational gradient decent [52]. For sampling the posterior of BAR-DenseED, we will use a recently proposed Stochastic Weighted Averaging Gaussian (SWAG) [53]. SWAG is an approximate Bayesian method that builds upon the Stochastic Weight Averages (SWA), an optimization met…”
BackgroundMethodInfluentialDeep Hidden Physics Models: Deep Learning of Nonlinear Partial Differential Equations
Raissi, M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1801.06637 · Cited by 419
Adversarial uncertainty quantification in physics-informed neural networks
Yang, Y., & Perdikaris, P. (2019) · Journal of Computational Physics · doi:10.1016/j.jcp.2019.05.027 · Cited by 414
Kim, Y., Denton, C., Hoang, L., & Rush, A. M. (2017) · arXiv · doi:10.48550/arxiv.1702.00887 · Cited by 381 · cited 2×
“t term memory (LSTM) architectures [27]. However, such models tend to be dicult and slow to train [28]. Recent advances of modeling time series include hierarchical networks [29], attention networks [30] and transformer networks [31]. For modeling dynamical systems, we propose the following autoregressive dense encode…”
MethodStein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithm
Liu, Q., & Wang, D. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1608.04471 · Cited by 292
Efficient Estimations from a Slowly Convergent Robbins-Monro Process
Ruppert, D. (1988) · eCommons (Cornell University) · Cited by 277
Ensemble learning in Bayesian neural networks
Barber, D., & Bishop, C. (1998) · Cited by 256
The numerical solution of linear ordinary differential equations by feedforward neural networks
Meade, A. J., & Fernandez, A. A. (1994) · Mathematical and Computer Modelling · doi:10.1016/0895-7177(94)90095-7 · Cited by 256
Order and complexity in the Kuramoto-Sivashinsky model of weakly turbulent interfaces
Hyman, J. M., Nicolaenko, B., & Zaleski, S. (1986) · Physica D: Nonlinear Phenomena · doi:10.1016/0167-2789(86)90136-3 · Cited by 212
Solution of nonlinear ordinary differential equations by feedforward neural networks
Meade, A. J., & Fernandez, A. A. (1994) · Mathematical and Computer Modelling · doi:10.1016/0895-7177(94)00160-x · Cited by 172
Grohs, P., Hornung, F., Jentzen, A., & von Wurstemberger, P. (2023) · Memoirs of the American Mathematical Society · doi:10.1090/memo/1410 · Cited by 165
Solving for high-dimensional committor functions using artificial neural networks
Khoo, Y., Lu, J., & Ying, L. (2018) · Research in the Mathematical Sciences · doi:10.1007/s40687-018-0160-2 · Cited by 165
Estimating uncertainties in statistics computed from direct numerical simulation
Oliver, T., Malaya, N., Ulerich, R., & Moser, R. (2014) · Physics of Fluids · doi:10.1063/1.4866813 · Cited by 164
A deep learning solution approach for high-dimensional random differential equations
Nabian, M. A., & Meidani, H. (2019) · Probabilistic Engineering Mechanics · doi:10.1016/j.probengmech.2019.05.001 · Cited by 164
Karumuri, S., Tripathy, R., Bilionis, I., & Panchal, J. (2020) · Journal of Computational Physics · doi:10.1016/j.jcp.2019.109120 · Cited by 159
Geneva, N., & Zabaras, N. J. (2019) · Journal of Computational Physics · doi:10.1016/j.jcp.2019.01.021 · Cited by 158
Self-citationOn the State Space Geometry of the Kuramoto–Sivashinsky Flow in a Periodic Domain
Cvitanović, P., Davidchack, R. L., & Siminos, E. (2010) · SIAM Journal on Applied Dynamical Systems · doi:10.1137/070705623 · Cited by 140
Data‐driven projection method in fluid simulation
Yang, C., Yang, X., & Xiao, X. (2016) · Computer Animation and Virtual Worlds · doi:10.1002/cav.1695 · Cited by 121
Long‐Time Predictive Modeling of Nonlinear Dynamical Systems Using Neural Networks
Pan, S., & Duraisamy, K. (2018) · Complexity · doi:10.1155/2018/4801012 · Cited by 119
Predictive collective variable discovery with deep Bayesian models
Schöberl, M., Zabaras, N. J., & Koutsourelakis, P. (2019) · The Journal of Chemical Physics · doi:10.1063/1.5058063 · Cited by 40
Self-citation
Cited by
Physics-informed machine learning
Karniadakis, G. E., Kevrekidis, I. G., Lu, L., Perdikaris, P., Wang, S., & Yang, L. (2021) · Nature Reviews Physics · doi:10.1038/s42254-021-00314-5 · Cited by 8,718
Scientific Machine Learning Through Physics–Informed Neural Networks: Where we are and What’s Next
Cuomo, S., Di Cola, V. S., Giampaolo, F., Rozza, G., Raissi, M., & Piccialli, F. (2022) · Journal of Scientific Computing · doi:10.1007/s10915-022-01939-z · Cited by 2,784
Understanding and Mitigating Gradient Flow Pathologies in Physics-Informed Neural Networks
Wang, S., Teng, Y., & Perdikaris, P. (2021) · SIAM Journal on Scientific Computing · doi:10.1137/20m1318043 · Cited by 2,048
Transfer learning enhanced physics informed neural network for phase-field modeling of fracture
Goswami, S., Anitescu, C., Chakraborty, S., & Rabczuk, T. (2020) · Theoretical and Applied Fracture Mechanics · doi:10.1016/j.tafmec.2019.102447 · Cited by 900
Integrating Scientific Knowledge with Machine Learning for Engineering and Environmental Systems
Willard, J., Jia, X., Xu, S., Steinbach, M., & Kumar, V. (2023) · ACM Computing Surveys · doi:10.1145/3514228 · Cited by 878
Metadata
- Authors
- Geneva, N., & Zabaras, N.
- Venue
- Journal of Computational Physics
- Publisher
- Elsevier BV
- Published
- 2020-02
- Volume
- 403
- Pages
- 109056
- ISSN
- 0021-9991 (ISSN-L)
- OpenAlex
- W2951392159
- Cited by
- 339
- References
- 71
- FWCI
- 13.95
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.