The DOI matches its Crossref record; 35 of the 35 references it cites were found in a registry or an index.
Obstacles to High-Dimensional Particle Filtering
Open accessAbstract
Particle filters are ensemble-based assimilation schemes that, unlike the ensemble Kalman filter, employ a fully nonlinear and non-Gaussian analysis step to compute the probability distribution function (pdf) of a system’s state conditioned on a set of observations. Evidence is provided that the ensemble size required for a successful particle filter scales exponentially with the problem size. For the simple example in which each component of the state vector is independent, Gaussian, and of unit variance and the observations are of each state component separately with independent, Gaussian errors, simulations indicate that the required ensemble size scales exponentially with the state dimension. In this example, the particle filter requires at least 1011 members when applied to a 200-dimensional state. Asymptotic results, following the work of Bengtsson, Bickel, and collaborators, are provided for two cases: one in which each prior state component is independent and identically distributed, and one in which both the prior pdf and the observation errors are Gaussian. The asymptotic theory reveals that, in both cases, the required ensemble size scales exponentially with the variance of the observation log likelihood rather than with the state dimension per se.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deterministic Nonperiodic Flow
Lorenz, E. N. (1963) · Journal of the Atmospheric Sciences · doi:10.1175/1520-0469(1963)020<0130:dnf>2.0.co;2 · Cited by 19,398
Density Estimation for Statistics and Data Analysis
Silverman, B. W. (2018) · Routledge eBooks · doi:10.1201/9781315140919 · Cited by 15,796
Density Estimation for Statistics and Data Analysis
(1986) · Springer eBooks · doi:10.1007/978-1-4899-3324-9 · Cited by 8,525
Novel approach to nonlinear/non-Gaussian Bayesian state estimation
Gordon, N. J., Salmond, D. J., & Smith, A. F. M. (1993) · IEE Proceedings F Radar and Signal Processing · doi:10.1049/ip-f-2.1993.0015 · Cited by 7,550
Advanced Mathematical Methods for Scientists and Engineers I
Bender, C. M., & Orszag, S. A. (1999) · doi:10.1007/978-1-4757-3069-2 · Cited by 4,906
Probability: Theory and Examples.
Prewitt, K., & Durrett, R. (1992) · Journal of the American Statistical Association · doi:10.2307/2290301 · Cited by 3,085
Monte Carlo Strategies in Scientific Computing
Liu, J. S. (2004) · Springer Series in Statistics · doi:10.1007/978-0-387-76371-2 · Cited by 2,690
Filtering via Simulation: Auxiliary Particle Filters
Pitt, M. K., & Shephard, N. (1999) · Journal of the American Statistical Association · doi:10.1080/01621459.1999.10474153 · Cited by 2,438
David, H. A., & Nagaraja, H. N. (2003) · Wiley Series in Probability and Statistics · doi:10.1002/0471722162 · Cited by 2,370
Data Assimilation Using an Ensemble Kalman Filter Technique
Houtekamer, P. L., & Mitchell, H. L. (1998) · Monthly Weather Review · doi:10.1175/1520-0493(1998)126<0796:dauaek>2.0.co;2 · Cited by 1,864
An Ensemble Adjustment Kalman Filter for Data Assimilation
Anderson, J. L. (2001) · Monthly Weather Review · doi:10.1175/1520-0493(2001)129<2884:aeakff>2.0.co;2 · Cited by 1,749
Self-citationA Sequential Ensemble Kalman Filter for Atmospheric Data Assimilation
Houtekamer, P. L., & Mitchell, H. L. (2001) · Monthly Weather Review · doi:10.1175/1520-0493(2001)129<0123:asekff>2.0.co;2 · Cited by 1,492
Monte Carlo Strategies in Scientific Computing
Hesterberg, T. (2002) · Technometrics · doi:10.1198/tech.2002.s85 · Cited by 1,409
Regularized estimation of large covariance matrices
Bickel, P. J., & Levina, E. (2006) · Cited by 1,166
Self-citationAn Introduction to Sequential Monte Carlo Methods
Doucet, A., Freitas, N., & Gordon, N. (2001) · Sequential Monte Carlo Methods in Practice · doi:10.1007/978-1-4757-3437-9_1 · Cited by 1,119
Distance-Dependent Filtering of Background Error Covariance Estimates in an Ensemble Kalman Filter
Hamill, T. M., Whitaker, J. S., & Snyder, C. (2001) · Monthly Weather Review · doi:10.1175/1520-0493(2001)129<2776:ddfobe>2.0.co;2 · Cited by 1,112
Self-citationPredictability – a problem partly solved
Lorenz, E. N. (2006) · Predictability of Weather and Climate · doi:10.1017/cbo9780511617652.004 · Cited by 1,046
Anderson, J. L., & Anderson, S. L. (1999) · Monthly Weather Review · doi:10.1175/1520-0493(1999)127<2741:amciot>2.0.co;2 · Cited by 1,042
Self-citationRegularized estimation of large covariance matrices
Bickel, P. J., & Levina, E. (2008) · The Annals of Statistics · doi:10.1214/009053607000000758 · Cited by 934
Self-citationInterpretation of Rank Histograms for Verifying Ensemble Forecasts
Hamill, T. M. (2001) · Monthly Weather Review · doi:10.1175/1520-0493(2001)129<0550:iorhfv>2.0.co;2 · Cited by 892
Hydrologic Data Assimilation with the Ensemble Kalman Filter
Reichle, R. H., McLaughlin, D. B., & Entekhabi, D. (2002) · Monthly Weather Review · doi:10.1175/1520-0493(2002)130<0103:hdawte>2.0.co;2 · Cited by 874
Probability: Theory and Examples.
Durrett, R. (1993) · Biometrics · doi:10.2307/2532227 · Cited by 857
Ide, K., Courtier, P., Ghil, M., & Lorenc, A. C. (1997) · Journal of the Meteorological Society of Japan. Ser. II · doi:10.2151/jmsj1965.75.1b_181 · Cited by 821
Moradkhani, H., Hsu, K., Gupta, H., & Sorooshian, S. (2005) · Water Resources Research · doi:10.1029/2004wr003604 · Cited by 777
An Ensemble Kalman Smoother for Nonlinear Dynamics
Evensen, G., & van Leeuwen, P. J. (2000) · Monthly Weather Review · doi:10.1175/1520-0493(2000)128<1852:aeksfn>2.0.co;2 · Cited by 578
Assimilation of Simulated Doppler Radar Observations with an Ensemble Kalman Filter*
Snyder, C., & Zhang, F. (2003) · Monthly Weather Review · doi:10.1175//2555.1 · Cited by 466
Self-citationEstimation of high-dimensional prior and posterior covariance matrices in Kalman filter variants
Furrer, R., & Bengtsson, T. (2007) · Journal of Multivariate Analysis · doi:10.1016/j.jmva.2006.08.003 · Cited by 414
Self-citationCurse-of-dimensionality revisited: Collapse of the particle filter in very large scale systems
Bengtsson, T., Bickel, P., & Li, B. (2008) · Institute of Mathematical Statistics Collections · doi:10.1214/193940307000000518 · Cited by 397
Self-citationReanalysis without Radiosondes Using Ensemble Data Assimilation
Whitaker, J. S., Compo, G. P., Wei, X., & Hamill, T. M. (2004) · Monthly Weather Review · doi:10.1175/1520-0493(2004)132<1190:rwrued>2.0.co;2 · Cited by 242
Toward a nonlinear ensemble filter for high‐dimensional systems
Bengtsson, T., Snyder, C., & Nychka, D. (2003) · Journal of Geophysical Research: Atmospheres · doi:10.1029/2002jd002900 · Cited by 173
Self-citationMerging particle filter for sequential data assimilation
Nakano, S., Ueno, G., & Higuchi, T. (2007) · Nonlinear processes in geophysics · doi:10.5194/npg-14-395-2007 · Cited by 134
Assessing the Performance of the Ensemble Kalman Filter for Land Surface Data Assimilation
Zhou, Y., McLaughlin, D. B., & Entekhabi, D. (2006) · Monthly Weather Review · doi:10.1175/mwr3153.1 · Cited by 114
Keppenne, C. L., Rienecker, M. M., Kurkowski, N. P., & Adamec, D. (2005) · Nonlinear processes in geophysics · doi:10.5194/npg-12-491-2005 · Cited by 65
A Note on the Particle Filter with Posterior Gaussian Resampling
Xiong, X., Navon, I. M., & Uzunoğlu, B. (2006) · Tellus A Dynamic Meteorology and Oceanography · doi:10.3402/tellusa.v58i4.14798 · Cited by 61
Cluster ensemble Kalman filter
Smith, K. W. (2007) · Tellus A Dynamic Meteorology and Oceanography · doi:10.3402/tellusa.v59i5.15148 · Cited by 51
Cited by
Li, R., Pei, S., Chen, B., Song, Y., Zhang, T., Yang, W., & Shaman, J. (2020) · Science · doi:10.1126/science.abb3221 · Cited by 3,633
Inverse problems: A Bayesian perspective
Stuart, A. M. (2010) · Acta Numerica · doi:10.1017/s0962492910000061 · Cited by 1,791
Data assimilation in the geosciences: An overview of methods, issues, and perspectives
Carrassi, A., Bocquet, M., Bertino, L., & Evensen, G. (2018) · WIREs Climate Change · doi:10.1002/wcc.535 · Cited by 867
Particle Filtering in Geophysical Systems
van Leeuwen, P. J. (2009) · Monthly Weather Review · doi:10.1175/2009mwr2835.1 · Cited by 678
Liu, Y., Weerts, A. H., Clark, M., Hendricks Franssen, H.-J., Kumar, S., Moradkhani, H., Seo, D.-J., Schwanenberg, D., Smith, P., van Dijk, A. I. J. M., et al. (2012) · Hydrology and Earth System Sciences · doi:10.5194/hess-16-3863-2012 · Cited by 487
Metadata
- Authors
- Snyder, C., Bengtsson, T., Bickel, P., & Anderson, J.
- Venue
- Monthly Weather Review
- Publisher
- American Meteorological Society
- Published
- 2008-12-01
- Volume
- 136
- Issue
- 12
- Pages
- 4629-4640
- ISSN
- 0027-0644 (ISSN-L)
- 1520-0493
- OpenAlex
- W2150951085
- Cited by
- 696
- References
- 35
- FWCI
- 13.10
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.