The DOI matches its Crossref record; 13 of the 13 references it cites were found in a registry or an index.
On “Field Significance” and the False Discovery Rate
Abstract
The conventional approach to evaluating the joint statistical significance of multiple hypothesis tests (i.e., “field,” or “global,” significance) in meteorology and climatology is to count the number of individual (or “local”) tests yielding nominally significant results and then to judge the unusualness of this integer value in the context of the distribution of such counts that would occur if all local null hypotheses were true. The sensitivity (i.e., statistical power) of this approach is potentially compromised both by the discrete nature of the test statistic and by the fact that the approach ignores the confidence with which locally significant tests reject their null hypotheses. An alternative global test statistic that has neither of these problems is the minimum p value among all of the local tests. Evaluation of field significance using the minimum local p value as the global test statistic, which is also known as the Walker test, has strong connections to the joint evaluation of multiple tests in a way that controls the “false discovery rate” (FDR, or the expected fraction of local null hypothesis rejections that are incorrect). In particular, using the minimum local p value to evaluate field significance at a level αglobal is nearly equivalent to the slightly more powerful global test based on the FDR criterion. An additional advantage shared by Walker’s test and the FDR approach is that both are robust to spatial dependence within the field of tests. The FDR method not only provides a more broadly applicable and generally more powerful field significance test than the conventional counting procedure but also allows better identification of locations with significant differences, because fewer than αglobal × 100% (on average) of apparently significant local tests will have resulted from local null hypotheses that are true.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing
Benjamini, Y., & Hochberg, Y. (1995) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1995.tb02031.x · Cited by 109,482
Applied Multivariate Statistical Analysis.
Johnson, A. R., & Wichern, D. W. (1988) · Biometrics · doi:10.2307/2531616 · Cited by 11,353
Statistical Methods in the Atmospheric Sciences
Ramsey, P. H. (1996) · Technometrics · doi:10.1080/00401706.1996.10484553 · Cited by 7,601
Gumbel, E. J. (1958) · Columbia University Press eBooks · doi:10.7312/gumb92958 · Cited by 5,567
Applied Multivariate Statistical Analysis
Härdle, W., & Simar, L. (2003) · Springer eBooks · doi:10.1007/978-3-662-05802-2 · Cited by 2,263
Statistical Methods in the Atmospheric Sciences
Briggs, W. (. (2007) · Journal of the American Statistical Association · doi:10.1198/jasa.2007.s163 · Cited by 1,531
Statistical Field Significance and its Determination by Monte Carlo Techniques
Livezey, R. E., & Chen, W. Y. (1983) · Monthly Weather Review · doi:10.1175/1520-0493(1983)111<0046:sfsaid>2.0.co;2 · Cited by 1,153
Statistical methods in the atmospheric sciences
Wilks, D. S. (2006) · Cited by 1,046
Self-citationTests of significance in harmonic analysis
Fisher, R. A. (1929) · Proceedings of the Royal Society of London. Series A, Containing Papers of a Mathematical and Physical Character · doi:10.1098/rspa.1929.0151 · Cited by 672
Ventura, V., Paciorek, C. J., & Risbey, J. S. (2004) · Journal of Climate · doi:10.1175/3199.1 · Cited by 236
Estimating Changing Extremes Using Empirical Ranking Methods
Folland, C. K., & Anderson, C. W. (2002) · Journal of Climate · Cited by 72
A Remark on Chervin-Schneider's Algorithm to Test Significance of Climate Experiments with GCM's
von STORCH, H. (1982) · Journal of the Atmospheric Sciences · Cited by 50
Beljaars (1997) · doi:10.1175/1520-0442(1997)010
Cited by
Wilks, D. S. (2016) · Bulletin of the American Meteorological Society · doi:10.1175/bams-d-15-00267.1 · Cited by 895
Self-citationObserved increases in extreme fire weather driven by atmospheric humidity and temperature
Jain, P., Castellanos-Acuna, D., Coogan, S. C. P., Abatzoglou, J. T., & Flannigan, M. D. (2022) · Nature Climate Change · doi:10.1038/s41558-021-01224-1 · Cited by 509
Khaliq, M. N., Ouarda, T. B. M. J., Gachon, P., Sushama, L., & St-Hilaire, A. (2009) · Journal of Hydrology · doi:10.1016/j.jhydrol.2009.01.035 · Cited by 357
Evaluation of CMIP5 palaeo-simulations to improve climate projections
Harrison, S. P., Bartlein, P. J., Izumi, K., Li, G., Annan, J., Hargreaves, J., Braconnot, P., & Kageyama, M. (2015) · Nature Climate Change · doi:10.1038/nclimate2649 · Cited by 297
Saft, M., Western, A. W., Zhang, L., Peel, M. C., & Potter, N. J. (2015) · Water Resources Research · doi:10.1002/2014wr015348 · Cited by 287
Metadata
- Authors
- Wilks, D. S.
- Venue
- Journal of Applied Meteorology and Climatology
- Publisher
- American Meteorological Society
- Published
- 2006-09-01
- Volume
- 45
- Issue
- 9
- Pages
- 1181-1189
- ISSN
- 1558-8424 (ISSN-L)
- 1558-8432
- OpenAlex
- W2156464996
- Cited by
- 531
- References
- 13
- FWCI
- 3.66
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.