The DOI matches its Crossref record; 24 of the 24 references it cites were found in a registry or an index.
Curse-of-dimensionality revisited: Collapse of the particle filter in very large scale systems
Open accessAbstract
<!-- *** Custom HTML *** --> It has been widely realized that Monte Carlo methods (approximation via a sample ensemble) may fail in large scale systems. This work offers some theoretical insight into this phenomenon in the context of the particle filter. We demonstrate that the maximum of the weights associated with the sample ensemble converges to one as both the sample size and the system dimension tends to infinity. Specifically, under fairly weak assumptions, if the ensemble size grows sub-exponentially in the cube root of the system dimension, the convergence holds for a single update step in state-space models with independent and identically distributed kernels. Further, in an important special case, more refined arguments show (and our simulations suggest) that the convergence to unity occurs unless the ensemble grows super-exponentially in the system dimension. The weight singularity is also established in models with more general multivariate likelihoods, e.g. Gaussian and Cauchy. Although presented in the context of atmospheric data assimilation for numerical weather prediction, our results are generally valid for high-dimensional particle filters.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
An Introduction to Multivariate Statistical Analysis
Muirhead, R. J., & Anderson, T. W. (1986) · Journal of Business & Economic Statistics · doi:10.2307/1391399 · Cited by 9,082
Novel approach to nonlinear/non-Gaussian Bayesian state estimation
Gordon, N. J., Salmond, D. J., & Smith, A. F. M. (1993) · IEE Proceedings F Radar and Signal Processing · doi:10.1049/ip-f-2.1993.0015 · Cited by 7,550
Sequential Monte Carlo Methods in Practice
(2001) · Springer eBooks · doi:10.1007/978-1-4757-3437-9 · Cited by 7,366
Mathematical Methods of Statistics.
Geary, R. C., & Cramer, H. (1947) · The Economic Journal · doi:10.2307/2226151 · Cited by 5,927
Evensen, G. (1994) · Journal of Geophysical Research: Oceans · doi:10.1029/94jc00572 · Cited by 5,371
An Introduction to Multivariate Statistical Analysis.
Skinner, C. J., & Anderson, T. W. (1985) · Journal of the Royal Statistical Society. Series A (General) · doi:10.2307/2981950 · Cited by 3,245
Monte Carlo Strategies in Scientific Computing
Liu, J. S. (2004) · Springer Series in Statistics · doi:10.1007/978-0-387-76371-2 · Cited by 2,690
Sequential Monte Carlo Methods for Dynamic Systems
Liu, J. S., & Chen, R. (1998) · Journal of the American Statistical Association · doi:10.1080/01621459.1998.10473765 · Cited by 2,191
Analysis Scheme in the Ensemble Kalman Filter
Burgers, G., Jan van Leeuwen, P., & Evensen, G. (1998) · Monthly Weather Review · doi:10.1175/1520-0493(1998)126<1719:asitek>2.0.co;2 · Cited by 1,869
Data Assimilation Using an Ensemble Kalman Filter Technique
Houtekamer, P. L., & Mitchell, H. L. (1998) · Monthly Weather Review · doi:10.1175/1520-0493(1998)126<0796:dauaek>2.0.co;2 · Cited by 1,864
The ECMWF Ensemble Prediction System: Methodology and validation
Molteni, F., Buizza, R., Palmer, T. N., & Petroliagis, T. (1996) · Quarterly Journal of the Royal Meteorological Society · doi:10.1002/qj.49712252905 · Cited by 1,616
A Sequential Ensemble Kalman Filter for Atmospheric Data Assimilation
Houtekamer, P. L., & Mitchell, H. L. (2001) · Monthly Weather Review · doi:10.1175/1520-0493(2001)129<0123:asekff>2.0.co;2 · Cited by 1,492
Monte Carlo Strategies in Scientific Computing
Hesterberg, T. (2002) · Technometrics · doi:10.1198/tech.2002.s85 · Cited by 1,409
Distance-Dependent Filtering of Background Error Covariance Estimates in an Ensemble Kalman Filter
Hamill, T. M., Whitaker, J. S., & Snyder, C. (2001) · Monthly Weather Review · doi:10.1175/1520-0493(2001)129<2776:ddfobe>2.0.co;2 · Cited by 1,112
Ensemble Forecasting at NCEP and the Breeding Method
Toth, Z., & Kalnay, E. (1997) · Monthly Weather Review · doi:10.1175/1520-0493(1997)125<3297:efanat>2.0.co;2 · Cited by 1,063
Anderson, J. L., & Anderson, S. L. (1999) · Monthly Weather Review · doi:10.1175/1520-0493(1999)127<2741:amciot>2.0.co;2 · Cited by 1,042
Tippett, M. K., Anderson, J. L., Bishop, C. H., Hamill, T. M., & Whitaker, J. S. (2003) · Monthly Weather Review · doi:10.1175/1520-0493(2003)131<1485:esrf>2.0.co;2 · Cited by 796
Following a Moving Target—Monte Carlo Inference for Dynamic Bayesian Models
Gilks, W. R., & Berzuini, C. (2001) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/1467-9868.00280 · Cited by 756
Limit theorems of probability theory
(2011) · Translations of Mathematical Monographs · doi:10.1090/mmono/163/05 · Cited by 627
Particle filters for state-space models with the presence of unknown static parameters
Storvik, G. (2002) · IEEE Transactions on Signal Processing · doi:10.1109/78.978383 · Cited by 444
Estimation of high-dimensional prior and posterior covariance matrices in Kalman filter variants
Furrer, R., & Bengtsson, T. (2007) · Journal of Multivariate Analysis · doi:10.1016/j.jmva.2006.08.003 · Cited by 414
Self-citationSpatiotemporal Hierarchical Bayesian Modeling Tropical Ocean Surface Winds
Wikle, C. K., Milliff, R. F., Nychka, D., & Berliner, L. M. (2001) · Journal of the American Statistical Association · doi:10.1198/016214501753168109 · Cited by 335
Toward a nonlinear ensemble filter for high‐dimensional systems
Bengtsson, T., Snyder, C., & Nychka, D. (2003) · Journal of Geophysical Research: Atmospheres · doi:10.1029/2002jd002900 · Cited by 173
Self-citationThe ECMWF Ensemble Prediction System: Methodology and validation
Molteni, F., Buizza, R., Palmer, T. N., & Petroliagis, T. I. (1996) · Quarterly Journal of the Royal Meteorological Society · doi:10.1256/smsqj.52904 · Cited by 165
Cited by
Inverse problems: A Bayesian perspective
Stuart, A. M. (2010) · Acta Numerica · doi:10.1017/s0962492910000061 · Cited by 1,791
Obstacles to High-Dimensional Particle Filtering
Snyder, C., Bengtsson, T., Bickel, P., & Anderson, J. (2008) · Monthly Weather Review · doi:10.1175/2008mwr2529.1 · Cited by 696
Self-citationParticle Filtering in Geophysical Systems
van Leeuwen, P. J. (2009) · Monthly Weather Review · doi:10.1175/2009mwr2835.1 · Cited by 678
Probabilistic Forecasting and Bayesian Data Assimilation
Reich, S., & Cotter, C. (2015) · Cambridge University Press eBooks · doi:10.1017/cbo9781107706804 · Cited by 334
A Review of Modern Computational Algorithms for Bayesian Optimal Design
Ryan, E. G., Drovandi, C. C., McGree, J. M., & Pettitt, A. N. (2016) · International Statistical Review · doi:10.1111/insr.12107 · Cited by 312
Metadata
- Authors
- Bengtsson, T., Bickel, P., & Li, B.
- Venue
- Institute of Mathematical Statistics Collections
- Publisher
- Institute of Mathematical Statistics
- Published
- 2008
- Pages
- 316-334
- OpenAlex
- W1628268552
- Cited by
- 397
- References
- 24
- FWCI
- 37.32
- Type
- book chapter
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.