The DOI matches its Crossref record; 25 of the 25 references it cites were found in a registry or an index.
Statistics, Handle with Care: Detecting Multiple Model Components with the Likelihood Ratio Test
Open accessAbstract
The likelihood ratio test (LRT) and the related F -test, popularized in astrophysics by Eadie and coworkers in 1971, Bevington in 1969, Lampton, Margon, & Bowyer, in 1976, Cash in 1979, and Avni in 1978, do not (even asymptotically) adhere to their nominal χ 2 and F -distributions in many statistical tests common in astrophysics, thereby casting many marginal line or source detections and nondetections into doubt. Although the above authors illustrate the many legitimate uses of these statistics, in some important cases it can be impossible to compute the correct false positive rate. For example, it has become common practice to use the LRT or the F -test to detect a line in a spectral model or a source above background despite the lack of certain required regularity conditions. (These applications were not originally suggested by Cash or by Bevington.) In these and other settings that involve testing a hypothesis that is on the boundary of the parameter space, contrary to common practice, the nominal χ 2 distribution for the LRT or the F-distribution for the F-test should not be used . In this paper, we characterize an important class of problems in which the LRT and the F -test fail and illustrate this nonstandard behavior. We briefly sketch several possible acceptable alternatives, focusing on Bayesian posterior predictive probability values. We present this method in some detail since it is a simple, robust, and intuitive approach. This alternative method is illustrated using the gamma-ray burst of 1997 May 8 (GRB 970508) to investigate the presence of an Fe K emission line during the initial phase of the observation. There are many legitimate uses of the LRT and the F -test in astrophysics, and even when these tests are inappropriate, there remain several statistical alternatives (e.g., judicious use of error bars and Bayes factors). Nevertheless, there are numerous cases of the inappropriate use of the LRT and similar tests in the literature, bringing substantive scientific results into question.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Estimating the Dimension of a Model
Schwarz, G. (1978) · The Annals of Statistics · doi:10.1214/aos/1176344136 · Cited by 47,033
Gelman, A., Carlin, J. B., Stern, H. S., & Rubin, D. B. (2003) · doi:10.1201/9780429258480 · Cited by 27,276
Inference from iterative simulation using multiple sequences
Gelman, A., & Rubin, D. B. (1992) · Statistical Science · doi:10.1214/ss/1177011136 · Cited by 16,627
Data Reduction and Error Analysis for the Physical Sciences
Bevington, P. R., Robinson, D. K., Blair, J. M., Mallinckrodt, A. J., & McKay, S. (1993) · Computers in Physics · doi:10.1063/1.4823194 · Cited by 14,824
Kass, R. E., & Raftery, A. E. (1995) · Journal of the American Statistical Association · doi:10.1080/01621459.1995.10476572 · Cited by 12,558
Approximation Theorems of Mathematical Statistics
Serfling, R. J. (1980) · Wiley Series in Probability and Statistics · doi:10.1002/9780470316481 · Cited by 5,851
Parameter estimation in astronomy through application of the likelihood ratio
Cash, W. (1979) · The Astrophysical Journal · doi:10.1086/156922 · Cited by 3,083
Statistical Analysis of Finite Mixture Distributions.
Lindsay, B. G., Titterington, D. M., Smith, A. F. M., & Makov, U. E. (1987) · Journal of the American Statistical Association · doi:10.2307/2289504 · Cited by 2,905
Kass, R. E., & Raftery, A. E. (1995) · Journal of the American Statistical Association · doi:10.2307/2291091 · Cited by 2,312
Bayes and Empirical Bayes Methods for Data Analysis
(2001) · Technometrics · doi:10.1198/tech.2001.s608 · Cited by 2,098
Mixture models : inference and applications to clustering
McLachlan, G. J., & Basford, K. E. (1988) · M. Dekker eBooks · Cited by 2,094
Everitt, B. S. (2005) · Encyclopedia of Statistics in Behavioral Science · doi:10.1002/0470013192.bsa231 · Cited by 1,263
The Likelihood Analysis of EGRET Data
Mattox, J. R., Bertsch, D. L., Chiang, J., Dingus, B. L., Digel, S. W., Eposito, J. A., Fierro, J. M., Hartman, R. C., Hunter, S. D., Kanbach, G., et al. (1996) · The Astrophysical Journal · doi:10.1086/177068 · Cited by 1,199
Mixture Models: Theory, Geometry and Applications
Lindsay, B. G. (1995) · NSF-CBMS Regional Conference Series in Probability and Statistics · doi:10.1214/cbms/1462106013 · Cited by 1,084
Everitt, B. S., & Hand, D. J. (1981) · Springer eBooks · doi:10.1007/978-94-009-5897-5 · Cited by 1,058
Statistical Methods in Experimental Physics
James, F. (2006) · WORLD SCIENTIFIC eBooks · doi:10.1142/6096 · Cited by 969
Posterior Predictive $p$-Values
Meng, X.-L. (1994) · The Annals of Statistics · doi:10.1214/aos/1176325622 · Cited by 785
On the Distribution of the Likelihood Ratio
Chernoff, H. (1954) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177728725 · Cited by 783
Parameter estimation in X-ray astronomy
Lampton, M., Margon, B., & Bowyer, S. (1976) · The Astrophysical Journal · doi:10.1086/154592 · Cited by 690
Ockham's Razor and Bayesian Analysis
Jefferys, W. H., & Berger, J. O. (1992) · American Scientist · Cited by 542
Bayes Factors and Choice Criteria for Linear Models
Smith, A. F. M., & Spiegelhalter, D. J. (1980) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1980.tb01122.x · Cited by 333
Piro, L., Costa, E., Feroci, M., Frontera, F., Amati, L., Dal Fiume, D., Antonelli, L. A., Heise, J., Zand, J. I. '., Owens, A., et al. (1999) · The Astrophysical Journal · doi:10.1086/311946 · Cited by 162
Analysis of Energy Spectra with Low Photon Counts via Bayesian Posterior Simulation
van Dyk, D. A., Connors, A., Kashyap, V. L., & Siemiginowska, A. (2001) · The Astrophysical Journal · doi:10.1086/318656 · Cited by 130
Self-citationInterpretations of multiple absorption features in a gamma-ray burst spectrum
Fenimore, E. E., Conner, J. P., Epstein, R. I., Klebesadel, R. W., Laros, J. G., Yoshida, A., Fujii, M., Hayashida, K., Itoh, M., Murakami, T., et al. (1988) · The Astrophysical Journal · doi:10.1086/185342 · Cited by 87
Analysis of energy spectra with low photon counts via Bayesian posterior simulation
van Dyk, D. A., & Connors, A. (2001) · Cited by 83
Self-citation
Cited by
Buchner, J., Georgakakis, A., Nandra, K., Hsu, L., Rangel, C., Brightman, M., Merloni, A., Salvato, M., Donley, J., & Kocevski, D. (2014) · Astronomy & Astrophysics · doi:10.1051/0004-6361/201322971 · Cited by 1,644
FERMI LARGE AREA TELESCOPE FIRST SOURCE CATALOG
Abdo, A. A., Ackermann, M., Ajello, M., Allafort, A., Antolini, E., Atwood, W. B., Axelsson, M., Baldini, L., Ballet, J., Barbiellini, G., et al. (2010) · The Astrophysical Journal Supplement Series · doi:10.1088/0067-0049/188/2/405 · Cited by 1,262
Bayes in the sky: Bayesian inference and model selection in cosmology
Trotta, R. (2008) · Contemporary Physics · doi:10.1080/00107510802066753 · Cited by 1,253
DETECTION OF AN UNIDENTIFIED EMISSION LINE IN THE STACKED X-RAY SPECTRUM OF GALAXY CLUSTERS
Bulbul, E., Markevitch, M., Foster, A., Smith, R. K., Loewenstein, M., & Randall, S. W. (2014) · The Astrophysical Journal · doi:10.1088/0004-637x/789/1/13 · Cited by 792
Tombesi, F., Cappi, M., Reeves, J. N., Palumbo, G. G. C., Yaqoob, T., Braito, V., & Dadina, M. (2010) · Astronomy and Astrophysics · doi:10.1051/0004-6361/200913440 · Cited by 682
Metadata
- Authors
- Protassov, R., van Dyk, D. A., Connors, A., Kashyap, V. L., & Siemiginowska, A.
- Venue
- The Astrophysical Journal
- Publisher
- American Astronomical Society
- Published
- 2002-05-20
- Volume
- 571
- Issue
- 1
- Pages
- 545-559
- ISSN
- 0004-637X (ISSN-L)
- 1538-4357
- OpenAlex
- W2154359271
- Cited by
- 678
- References
- 25
- FWCI
- 22.67
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.