The DOI matches its Crossref record; 24 of the 24 references it cites were found in a registry or an index.
A Stochastic Newton MCMC Method for Large-Scale Statistical Inverse Problems with Application to Seismic Inversion
Abstract
We address the solution of large-scale statistical inverse problems in the framework of Bayesian inference. The Markov chain Monte Carlo (MCMC) method is the most popular approach for sampling the posterior probability distribution that describes the solution of the statistical inverse problem. MCMC methods face two central difficulties when applied to large-scale inverse problems: first, the forward models (typically in the form of partial differential equations) that map uncertain parameters to observable quantities make the evaluation of the probability density at any point in parameter space very expensive; and second, the high-dimensional parameter spaces that arise upon discretization of infinite-dimensional parameter fields make the exploration of the probability density function prohibitive. The challenge for MCMC methods is to construct proposal functions that simultaneously provide a good approximation of the target density while being inexpensive to manipulate. Here we present a so-called Stochastic Newton method in which MCMC is accelerated by constructing and sampling from a proposal density that builds a local Gaussian approximation based on local gradient and Hessian (of the log posterior) information. Thus, the method exploits tools (adjoint-based gradients and Hessians) that have been instrumental for fast (often mesh-independent) solution of deterministic inverse problems. Hessian manipulations (inverse, square root) are made tractable by a low-rank approximation that exploits the compact nature of the data misfit operator. This is analogous to a reduced model of the parameter-to-observable map. The method is applied to the Bayesian solution of an inverse medium problem governed by 1D seismic wave propagation. We compare the Stochastic Newton method with a reference black box MCMC method as well as a gradient-based Langevin MCMC method, and observe at least two orders of magnitude improvement in convergence for problems with up to 65 parameters. Numerical evidence suggests that a 1025 parameter problem converges at the same rate as the 65 parameter problem.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Equation of State Calculations by Fast Computing Machines
Metropolis, N., Rosenbluth, A. W., Rosenbluth, M. N., Teller, A. H., & Teller, E. (1953) · The Journal of Chemical Physics · doi:10.1063/1.1699114 · Cited by 37,192
General methods for monitoring convergence of iterative simulations
Brooks, S. P., & Gelman, A. (1998) · Journal of Computational and Graphical Statistics · doi:10.1080/10618600.1998.10474787 · Cited by 6,045
Bayesian calibration of computer models
Kennedy, M. C., & O'Hagan, A. (2001) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/1467-9868.00294 · Cited by 4,222
Markov Chains for Exploring Posterior Distributions
Tierney, L. (1994) · The Annals of Statistics · doi:10.1214/aos/1176325750 · Cited by 3,452
Inverse problems: A Bayesian perspective
Stuart, A. M. (2010) · Acta Numerica · doi:10.1017/s0962492910000061 · Cited by 1,791
Haario, H., Laine, M., Mira, A., & Saksman, E. (2006) · Statistics and Computing · doi:10.1007/s11222-006-9438-0 · Cited by 1,498
Riemann manifold langevin and hamiltonian monte carlo methods
Girolami, M., & Calderhead, B. (2011) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2010.00765.x · Cited by 1,473
Optimal scaling for various Metropolis-Hastings algorithms
Roberts, G. O., & Rosenthal, J. S. (2001) · Statistical Science · doi:10.1214/ss/1015346320 · Cited by 1,103
Exponential Convergence of Langevin Distributions and Their Discrete Approximations
Roberts, G. O., & Tweedie, R. L. (1996) · Bernoulli · doi:10.2307/3318418 · Cited by 998
Computer Model Calibration Using High-Dimensional Output
Higdon, D., Gattiker, J., Williams, B., & Rightley, M. (2008) · Journal of the American Statistical Association · doi:10.1198/016214507000000888 · Cited by 867
Stochastic spectral methods for efficient Bayesian solution of inverse problems
Marzouk, Y. M., Najm, H. N., & Rahn, L. A. (2007) · Journal of Computational Physics · doi:10.1016/j.jcp.2006.10.010 · Cited by 470
Dimensionality reduction and polynomial chaos acceleration of Bayesian inference in inverse problems
Marzouk, Y. M., & Najm, H. N. (2009) · Journal of Computational Physics · doi:10.1016/j.jcp.2008.11.024 · Cited by 418
A Stochastic Collocation Approach to Bayesian Inference in Inverse Problems
Marzouk, Y., & Xiu, D. (2009) · Communications in Computational Physics · doi:10.4208/cicp.2009.v6.p826 · Cited by 284
Parameter and State Model Reduction for Large-Scale Statistical Inverse Problems
Lieberman, C., Willcox, K., & Ghattas, O. (2010) · SIAM Journal on Scientific Computing · doi:10.1137/090775622 · Cited by 254
Self-citationApproximation errors and model reduction with an application in optical diffusion tomography
Arridge, S. R., Kaipio, J. P., Kolehmainen, V, Schweiger, M., Somersalo, E., Tarvainen, T., & Vauhkonen, M. (2006) · Inverse Problems · doi:10.1088/0266-5611/22/1/010 · Cited by 221
Non‐linear model reduction for uncertainty quantification in large‐scale inverse problems
Galbally, D., Fidkowski, K., Willcox, K., & Ghattas, O. (2010) · International Journal for Numerical Methods in Engineering · doi:10.1002/nme.2746 · Cited by 209
Self-citationParametric Reduced-Order Models for Probabilistic Analysis of Unsteady Aerodynamic Applications
Bui–Thanh, T., Willcox, K. E., & Ghattas, O. N. (2008) · AIAA Journal · doi:10.2514/1.35850 · Cited by 181
Self-citationA general purpose sampling algorithm for continuous distributions (the t-walk)
Christen, J. A., & Fox, C. (2010) · Bayesian Analysis · doi:10.1214/10-ba60 · Cited by 173
Using Bayesian statistics in the estimation of heat source in radiation
Wang, J. B., & Zabaras, N. J. (2004) · International Journal of Heat and Mass Transfer · doi:10.1016/j.ijheatmasstransfer.2004.08.009 · Cited by 134
Sampling the posterior: An approach to non-Gaussian data assimilation
Apte, A., Hairer, M., Stuart, A. M., et al. (2006) · Physica D Nonlinear Phenomena · doi:10.1016/j.physd.2006.06.009 · Cited by 133
Conditional Path Sampling of SDEs and the Langevin MCMC Method
Stuart, A. M., Voß, J., & Wilberg, P. (2004) · Communications in Mathematical Sciences · doi:10.4310/cms.2004.v2.n4.a7 · Cited by 91
Bui–Thanh, T., & Ghattas, O. N. (2012) · Inverse Problems · doi:10.1088/0266-5611/28/5/055002 · Cited by 52
Self-citationOn markov chain monte carlo methods for nonlinear and non-gaussian state-space models
Geweke, J. F., & Tanizaki, H. (1999) · Communications in Statistics - Simulation and Computation · doi:10.1080/03610919908813583 · Cited by 51
Note on the Sampling Distribution for the Metropolis-Hastings Algorithm
Geweke, J. F., & Tanizaki, H. (2003) · Communication in Statistics- Theory and Methods · doi:10.1081/sta-120018828 · Cited by 20
Cited by
Survey of Multifidelity Methods in Uncertainty Propagation, Inference, and Optimization
Peherstorfer, B., Willcox, K., & Gunzburger, M. (2018) · SIAM Review · doi:10.1137/16m1082469 · Cited by 1,061
Bui-Thanh, T., Ghattas, O., Martin, J., & Stadler, G. (2013) · SIAM Journal on Scientific Computing · doi:10.1137/12089586x · Cited by 377
Self-citationBayesian inference with optimal maps
El Moselhy, T. A., & Marzouk, Y. M. (2012) · Journal of Computational Physics · doi:10.1016/j.jcp.2012.07.022 · Cited by 267
Petra, N., Martin, J., Stadler, G., & Ghattas, O. (2014) · SIAM Journal on Scientific Computing · doi:10.1137/130934805 · Cited by 252
Self-citation
Metadata
- Authors
- Martin, J., Wilcox, L. C., Burstedde, C., & Ghattas, O.
- Venue
- SIAM Journal on Scientific Computing
- Published
- 2012-01
- Volume
- 34
- Issue
- 3
- Pages
- A1460-A1487
- ISSN
- 1064-8275 (ISSN-L)
- 1095-7197
- OpenAlex
- W2082261407
- Cited by
- 424
- References
- 24
- FWCI
- 20.67
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.