CoChat
Sign Up
PreprintCited by 36Model Reduction and Neural NetworksNeural Networks and ApplicationsGaussian Processes and Bayesian Inference

Nonlocality and Nonlinearity Implies Universality in Operator Learning

Open access
Lanthaler, Samuel, Li, Zongyi, Stuart, Andrew M.·2023·arXiv
DOI 10.48550/arxiv.2304.13221arXiv 2304.13221
Open this paper in CoChat — free accountOpen PDF

Abstract

Neural operator architectures approximate operators between infinite-dimensional Banach spaces of functions. They are gaining increased attention in computational science and engineering, due to their potential both to accelerate traditional numerical methods and to enable data-driven discovery. As the field is in its infancy basic questions about minimal requirements for universal approximation remain open. It is clear that any general approximation of operators between spaces of functions must be both nonlocal and nonlinear. In this paper we describe how these two attributes may be combined in a simple way to deduce universal approximation. In so doing we unify the analysis of a wide range of neural operator architectures and open up consideration of new ones. A popular variant of neural operators is the Fourier neural operator (FNO). Previous analysis proving universal operator approximation theorems for FNOs resorts to use of an unbounded number of Fourier modes, relying on intuition from traditional analysis of spectral methods. The present work challenges this point of view: (i) the work reduces FNO to its core essence, resulting in a minimal architecture termed the ``averaging neural operator'' (ANO); and (ii) analysis of the ANO shows that even this minimal ANO architecture benefits from universal approximation. This result is obtained based on only a spatial average as its only nonlocal ingredient (corresponding to retaining only a \emph{single} Fourier mode in the special case of the FNO). The analysis paves the way for a more systematic exploration of nonlocality, both through the development of new operator learning architectures and the analysis of existing and new architectures. Numerical results are presented which give insight into complexity issues related to the roles of channel width (embedding dimension) and number of Fourier modes.

Integrity signals

Registry

DataCite · 10.48550/arxiv.2304.13221

Cites

30 of 30 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Many architectures (e.g., FNO, Laplace neural operator, DeepONet) contain kernel functions that recover the averaging kernel and thus are universal under Theorem 1.

    “The key corollary to Theorem 1 is that, many of the architectures aforementioned (e.g. FNO, Laplace neural operator, DeepONet) contain kernel functions K l that recover the averaging kernel and thus are universal under the setting of Theorem 1.”

    From Stabilization of nonlinear systems with unknown delays via delay-adaptive neural operator approximate predictors (2025) · cites this paper as [35]

Cites

Flagged first, then by citations

1 of the 31 references couldn’t be matched to a known paper.

Cited by

Sorted by citations

Related

Authors
Lanthaler, S., Li, Z., & Stuart, A. M.
Affiliations
  • University of Vienna (AT)
  • California Institute of Technology (US)
Corresponding author
Lanthaler, Samuel
ORCID
Venue
arXiv
Publisher
arXiv
Published
2023
Volume
62
Issue
2
Pages
261-303
ISSN
0176-4276 (ISSN-L)
DOI
10.48550/arxiv.2304.13221
arXiv
2304.13221
OpenAlex
W4413127858
Cited by
25
References
31
Citations by year
2026: 6 · 2025: 11 · 2024: 6 · 2023: 2
FWCI
6.99
Type
journal article
Language
en
Open access
Licence
Locations
  • Constructive Approximation (publishedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
  • National Science Foundation
  • U.S. Department of Defense
  • Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung (206737)
  • Universität Wien
  • Multidisciplinary University Research Initiative (FA9550-20–1-0358)
Topics
  • Physical Sciences › Physics and Astronomy › Statistical and Nonlinear Physics › Model Reduction and Neural Networks
  • Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
  • Physical Sciences › Computer Science › Artificial Intelligence › Gaussian Processes and Bayesian Inference
Keywords
neural operators, universal approximation, nonlocality, nonlinearity, Fourier neural operator, operator learning, infinite-dimensional Banach spaces, spatial averaging, Fourier modes, embedding dimension, channel width

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review