The DOI matches its Crossref record; 45 of the 45 references it cites were found in a registry or an index.
A comprehensive and fair comparison of two neural operators (with practical extensions) based on FAIR data
Open accessAbstract
Neural operators can learn nonlinear mappings between function spaces and offer a new simulation paradigm for real-time prediction of complex dynamics for realistic diverse applications as well as for system identification in science and engineering. Herein, we investigate the performance of two neural operators, which have shown promising results so far, and we develop new practical extensions that will make them more accurate and robust and importantly more suitable for industrial-complexity applications. The first neural operator, DeepONet, was published in 2019 (Lu et al., 2019), and its original architecture was based on the universal approximation theorem of Chen & Chen (1995). The second one, named Fourier Neural Operator or FNO, was published in 2020, and it is based on parameterizing the integral kernel in the Fourier space. DeepONet is represented by a summation of products of neural networks (NNs), corresponding to the branch NN for the input function and the trunk NN for the output function; both NNs are general architectures, e.g., the branch NN can be replaced with a CNN or a ResNet. According to Kovachki et al. (2021), FNO in its continuous form can be viewed conceptually as a DeepONet with a specific architecture of the branch NN and a trunk NN represented by a trigonometric basis. In order to compare FNO with DeepONet computationally for realistic setups, we develop several extensions of FNO that can deal with complex geometric domains as well as mappings where the input and output function spaces are of different dimensions. We also develop an extended DeepONet with special features that provide inductive bias and accelerate training, and we present a faster implementation of DeepONet with cost comparable to the computational cost of FNO, which is based on the Fast Fourier Transform. Here we consider 16 different benchmarks to demonstrate the relative performance of the two neural operators, including instability wave analysis in hypersonic boundary layers, prediction of the vorticity field of a flapping airfoil, porous media simulations in complex-geometry domains, etc. We follow the guiding principles of FAIR (Findability, Accessibility, Interoperability, and Reusability) for scientific data management and stewardship. The performance of DeepONet and FNO is comparable for relatively simple settings, but for complex geometries the performance of FNO deteriorates greatly. We also compare theoretically the two neural operators and obtain similar error estimates for DeepONet and FNO under the same regularity assumptions.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Multilayer feedforward networks are universal approximators
Hornik, K., Stinchcombe, M., & White, H. (1989) · Neural Networks · doi:10.1016/0893-6080(89)90020-8 · Cited by 21,728
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.123 · Cited by 19,741
The FAIR Guiding Principles for scientific data management and stewardship
Wilkinson, M. D., Dumontier, M., Aalbersberg, I. J., Appleton, G., Axton, M., Baak, A., Blomberg, N., Boiten, J.-W., da Silva Santos, L. B., Bourne, P. E., et al. (2016) · Scientific Data · doi:10.1038/sdata.2016.18 · Cited by 18,423
(2010) · Cited by 14,134
Approximation by superpositions of a sigmoidal function
Cybenko, G. (1989) · Mathematics of Control, Signals, and Systems · doi:10.1007/bf02551274 · Cited by 13,883
Physics-informed machine learning
Karniadakis, G. E., Kevrekidis, I. G., Lu, L., Perdikaris, P., Wang, S., & Yang, L. (2021) · Nature Reviews Physics · doi:10.1038/s42254-021-00314-5 · Cited by 8,718 · cited 1×
“…in the last 20 years and there is currently plenty of euphoria and admiration about the so-called “unreasonable effectiveness of deep learning in artificial intelligence” [5], the development of physics-informed machine learning and its application to scientific applications is relatively recent [6].”
BackgroundSelf-citationLearning nonlinear operators via DeepONet based on the universal approximation theorem of operators
Lu, L., Jin, P., Pang, G., Zhang, Z., & Karniadakis, G. E. (2021) · Nature Machine Intelligence · doi:10.1038/s42256-021-00302-5 · Cited by 3,418
Self-citationDeepXDE: A Deep Learning Library for Solving Differential Equations
Lu, L., Meng, X., Mao, Z., & Karniadakis, G. E. (2021) · SIAM Review · doi:10.1137/19m1274067 · Cited by 2,512
Self-citationPhysics-informed neural networks (PINNs) for fluid mechanics: a review
Cai, S., Mao, Z., Wang, Z., Yin, M., & Karniadakis, G. E. (2021) · Acta Mechanica Sinica · doi:10.1007/s10409-021-01148-1 · Cited by 2,084
Self-citationPerdikaris, P. G., Wang, S., & Perdikaris, P. G. (2021) · Science Advances · doi:10.26226/morressier.612f6736bc98103724100850 · Cited by 1,439
Chen, T., & Chen, H. (1995) · IEEE Transactions on Neural Networks · doi:10.1109/72.392253 · Cited by 1,230
Yang, L., Meng, X., & Karniadakis, G. E. (2021) · Journal of Computational Physics · doi:10.1016/j.jcp.2020.109913 · Cited by 1,169
Self-citationD. Jagtap, A., & Em Karniadakis, G. (2020) · Communications in Computational Physics · doi:10.4208/cicp.oa-2020-0164 · Cited by 1,130
fPINNs: Fractional Physics-Informed Neural Networks
Pang, G., Lu, L., & Karniadakis, G. E. (2019) · SIAM Journal on Scientific Computing · doi:10.1137/18m1229845 · Cited by 1,105 · cited 2×
“For example, using the DeepONet as a building block in [43] for modeling hypersonic flows, we were able to obtain very accurate results outside the Mach number training range [8 , 10], e.g., at Mach number 7 and 11, by assimilating only up to five new measurements of velocity and temperature.”
MethodResultSelf-citationFourier Neural Operator for Parametric Partial Differential Equations
Li, Z., Kovachki, N., Azizzadenesheli, K., Liu, B., Bhattacharya, K., Stuart, A., & Anandkumar, A. (2020) · arXiv · doi:10.48550/arxiv.2010.08895 · Cited by 1,021
Raissi, M., Perdikaris, P. G., & Karniadakis, G. E. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.10561 · Cited by 925
Self-citationPhysics-Informed Neural Networks with Hard Constraints for Inverse Design
Lu, L., Pestourie, R., Yao, W., Wang, Z., Verdugo, F., & Johnson, S. G. (2021) · SIAM Journal on Scientific Computing · doi:10.1137/21m1397908 · Cited by 771
Self-citationGradient-enhanced physics-informed neural networks for forward and inverse PDE problems
Yu, J., Lu, L., Meng, X., & Karniadakis, G. E. (2022) · Computer Methods in Applied Mechanics and Engineering · doi:10.1016/j.cma.2022.114823 · Cited by 722
Self-citationMeng, X., & Karniadakis, G. E. (2020) · Journal of Computational Physics · doi:10.1016/j.jcp.2019.109020 · Cited by 713
Self-citationApproximation by superpositions of a sigmoidal function
Cybenko, G. (1992) · Mathematics of Control, Signals, and Systems · doi:10.1007/bf02134016 · Cited by 703
PPINN: Parareal physics-informed neural network for time-dependent PDEs
Meng, X., Li, Z., Zhang, D., & Karniadakis, G. E. (2020) · Computer Methods in Applied Mechanics and Engineering · doi:10.1016/j.cma.2020.113250 · Cited by 654
Self-citationZhang, D., Lu, L., Guo, L., & Karniadakis, G. E. (2019) · Journal of Computational Physics · doi:10.1016/j.jcp.2019.07.048 · Cited by 513
Self-citationA physics-informed variational DeepONet for predicting crack path in quasi-brittle materials
Goswami, S., Yin, M., Yu, Y., & Karniadakis, G. E. (2022) · Computer Methods in Applied Mechanics and Engineering · doi:10.1016/j.cma.2022.114587 · Cited by 425
Self-citationThe unreasonable effectiveness of deep learning in artificial intelligence
Sejnowski, T. J. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1907373117 · Cited by 352 · cited 1×
“While there have been rapid developments in machine learning in the last 20 years and there is currently plenty of euphoria and admiration about the so-called “unreasonable effectiveness of deep learning in artificial intelligence” [5], the development of physics-informed machine learning and its application to scient…”
BackgroundExtraction of mechanical properties of materials through deep learning from instrumented indentation
Lu, L., Dao, M., Kumar, P., Ramamurty, U., Karniadakis, G. E., & Suresh, S. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1922210117 · Cited by 352
Self-citationModel Reduction And Neural Networks For Parametric PDEs
Bhattacharya, K., Hosseini, B., Kovachki, N. B., & Stuart, A. M. (2021) · The SMAI Journal of computational mathematics · doi:10.5802/smai-jcm.74 · Cited by 329
Systems biology informed deep learning for inferring parameters and hidden dynamics
Yazdani, A., Lu, L., Raissi, M., & Karniadakis, G. E. (2020) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1007575 · Cited by 329
Self-citationRaissi, M., Perdikaris, P. G., & Karniadakis, G. E. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.10566 · Cited by 312
Self-citationZhang, D., Guo, L., & Karniadakis, G. E. (2020) · SIAM Journal on Scientific Computing · doi:10.1137/19m1260141 · Cited by 308
Self-citationDying ReLU and Initialization: Theory and Numerical Examples
Lu, L., Shin, Y., Su, Y., & Em Karniadakis, G. (2020) · Communications in Computational Physics · doi:10.4208/cicp.oa-2020-0165 · Cited by 281
Self-citationCai, S., Wang, Z., Lu, L., Zaki, T. A., & Karniadakis, G. E. (2021) · Journal of Computational Physics · doi:10.1016/j.jcp.2021.110296 · Cited by 214
Self-citationMao, Z., Lu, L., Marxen, O., Zaki, T. A., & Karniadakis, G. E. (2021) · Journal of Computational Physics · doi:10.1016/j.jcp.2021.110698 · Cited by 212
Self-citationMulti-fidelity Bayesian neural networks: Algorithms and applications
Meng, X., Babaee, H., & Karniadakis, G. E. (2021) · Journal of Computational Physics · doi:10.1016/j.jcp.2021.110361 · Cited by 196
Self-citationDong, S. S., & Ni, N. (2021) · Journal of Computational Physics · doi:10.1016/j.jcp.2021.110242 · Cited by 192
Bayesian neural networks for uncertainty quantification in data-driven materials modeling
Olivier, A., Shields, M. D., & Graham-Brady, L. (2021) · Computer Methods in Applied Mechanics and Engineering · doi:10.1016/j.cma.2021.114079 · Cited by 191
Error estimates for DeepONets: a deep learning framework in infinite dimensions
Lanthaler, S., Mishra, S., & Karniadakis, G. E. (2022) · Transactions of Mathematics and Its Applications · doi:10.1093/imatrm/tnac001 · Cited by 173
Self-citationA physics-informed operator regression framework for extracting data-driven continuum models
Patel, R. G., Trask, N., Wood, M. A., & Cyr, E. C. (2020) · Computer Methods in Applied Mechanics and Engineering · doi:10.1016/j.cma.2020.113500 · Cited by 159
Learning functional priors and posteriors from data and physics
Meng, X., Yang, L., Mao, Z., Ferrandis, J. D. Á., & Karniadakis, G. E. (2022) · Journal of Computational Physics · doi:10.1016/j.jcp.2022.111073 · Cited by 81
Self-citationData-driven learning of nonlocal physics from high-fidelity synthetic data
You, H., Yu, Y., Trask, N., Gulian, M., & D’Elia, M. (2020) · Computer Methods in Applied Mechanics and Engineering · doi:10.1016/j.cma.2020.113553 · Cited by 79
Yin, M., Ban, E., Rego, B. V., Zhang, E., Cavinato, C., Humphrey, J. D., & Karniadakis, G. E. (2022) · Journal of The Royal Society Interface · doi:10.1098/rsif.2021.0670 · Cited by 60
Self-citationGMLS-Nets: A Framework for Learning from Unstructured Data
Sandia National Laboratories (SNL), A., Trask, N., Patel, R., Gross, B., & Atzberger, P. J. (2019) · doi:10.2172/1763256 · Cited by 35
Exponential Convergence of Deep Operator Networks for Elliptic Partial Differential Equations
Marcati, C., & Schwab, C. (2023) · SIAM Journal on Numerical Analysis · doi:10.1137/21m1465718 · Cited by 34
Generalizing universal function approximators
Higgins, I. (2021) · Nature Machine Intelligence · doi:10.1038/s42256-021-00318-x · Cited by 28
Arbitrary-Depth Universal Approximation Theorems for Operator Neural NetworksPreprint
Yu, A., Becquey, C., Halikias, D., Mallory, M. E., & Townsend, A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.11354 · Cited by 17
Convergence rate of DeepONets for learning operators arising from advection-diffusion equationsPreprint
Deng, B., Shin, Y., Lu, L., Zhang, Z., & Karniadakis, G. E. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.10621 · Cited by 11
Self-citation
Research works printed but not linked: 8
- You, 2021, Nonlocal kernel network (NKN): a stable and resolution-independent deep neural network, Preprint
- Lin, 2021, Operator learning for predicting multiscale bubble growth dynamics, J. Chem. Phys., 154, 10.1063/5.0041203 · DOI
- Lin, 2021, A seamless multiscale operator neural network for inferring bubble dynamics, J. Fluid Mech., 929, 10.1017/jfm.2021.866 · DOI
- Opschoor, 2021, Exponential reLU DNN expression of holomorphic maps in high dimension, Constr. Approx.
- Wang, 2009, High-order well-balanced schemes and applications to non-equilibrium flow, J. Comput. Phys., 228, 6682, 10.1016/j.jcp.2009.05.028 · doi:10.1016/j.jcp.2009.05.028
- Zhang, 2011, Positivity-preserving high order discontinuous Galerkin schemes for compressible Euler equations with source terms, J. Comput. Phys., 230, 1238, 10.1016/j.jcp.2010.10.036 · doi:10.1016/j.jcp.2010.10.036
- Meng, 2015, Multiple-relaxation-time lattice Boltzmann model for incompressible miscible flow with large viscosity ratio and high Péclet number, Phys. Rev. E, 92, 10.1103/PhysRevE.92.043305 · doi:10.1103/physreve.92.043305
- Guo, 2002, Non-equilibrium extrapolation method for velocity and pressure boundary conditions in the lattice Boltzmann method, Chin. Phys., 11, 366, 10.1088/1009-1963/11/4/310 · doi:10.1088/1009-1963/11/4/310
Cited by
A physics-informed variational DeepONet for predicting crack path in quasi-brittle materials
Goswami, S., Yin, M., Yu, Y., & Karniadakis, G. E. (2022) · Computer Methods in Applied Mechanics and Engineering · doi:10.1016/j.cma.2022.114587 · Cited by 425
Self-citationPhysics-Informed Neural Operator for Learning Partial Differential Equations
Li, Z., Zheng, H., Kovachki, N., Jin, D., Chen, H., Liu, B., Azizzadenesheli, K., & Anandkumar, A. (2024) · ACM / IMS Journal of Data Science · doi:10.1145/3648506 · Cited by 393
Neural operators for accelerating scientific simulations and design
Azizzadenesheli, K., Kovachki, N., Li, Z., Liu-Schiaffini, M., Kossaifi, J., & Anandkumar, A. (2024) · Nature Reviews Physics · doi:10.1038/s42254-024-00712-5 · Cited by 314
Faroughi, S. A., Pawar, N. M., Fernandes, C., Raissi, M., Das, S., Kalantari, N. K., & Kourosh Mahjour, S. (2024) · Journal of Computing and Information Science in Engineering · doi:10.1115/1.4064449 · Cited by 257
Bhan, L., Krstic, M., & Shi, Y. (2025) · arXiv · doi:10.48550/arxiv.2509.26443 · Cited by 4
Metadata
- Authors
- Lu, L., Meng, X., Cai, S., Mao, Z., Goswami, S., Zhang, Z., & Karniadakis, G. E.
- Venue
- Computer Methods in Applied Mechanics and Engineering
- Publisher
- Elsevier BV
- Published
- 2022-04
- Volume
- 393
- Pages
- 114778
- ISSN
- 2331-8422 (ISSN-L)
- 0045-7825
- OpenAlex
- W4221019179
- Cited by
- 514
- References
- 45
- Type
- posted content
- Language
- en
- Open access
- green
- Licence
- www.elsevier.com/tdm/userlicense/1.0/ (tdm)
- www.elsevier.com/legal/tdmrep-license (tdm)
- www.elsevier.com/open-access/userlicense/1.0/ (am, 344 d)
- doi.org/10.15223/policy-017 (stm-asf)
- doi.org/10.15223/policy-037 (stm-asf)
- doi.org/10.15223/policy-012 (stm-asf)
- doi.org/10.15223/policy-029 (stm-asf)
- doi.org/10.15223/policy-004 (stm-asf)
- Funders
- DARPA
- Office of the Secretary of Defense
- Air Force Office of Scientific Research (FA9550-20-1-0056, FA9550-20-1-0358)
- Department of Energy (DE-SC0019453)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.