Evaluation of the Potential Excess of Statistically Significant Findings in Published Genetic Association Studies: Application to Alzheimer's Disease
Open accessThe DOI matches its Crossref record; 15 of the 15 references it cites were found in a registry or an index.
Abstract
The authors evaluated whether there is an excess of statistically significant results in studies of genetic associations with Alzheimer's disease reflecting either between-study heterogeneity or bias. Among published articles on genetic associations entered into the comprehensive AlzGene database (www.alzgene.org) through January 31, 2007, 1,348 studies included in 175 meta-analyses with 3 or more studies each were analyzed. The number of observed studies (O) with statistically significant results (P = 0.05 threshold) was compared with the expected number (E) under different assumptions for the magnitude of the effect size. In the main analysis, the plausible effect size of each association was the summary effect presented in the respective meta-analysis. Overall, 19 meta-analyses (all with eventually nonsignificant summary effects) had a documented excess of O over E: Typically single studies had significant effects pointing in opposite directions and early summary effects were dissipated over time. Across the whole domain, O was 235 (17.4%), while E was 164.8 (12.2%) (P < 10(-6)). The excess showed a predilection for meta-analyses with nonsignificant summary effects and between-study heterogeneity. The excess was seen for all levels of statistical significance and also for studies with borderline P values (P = 0.05-0.10). The excess of significant findings may represent significance-chasing biases in a setting of massive testing.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Genome-wide association study identifies novel breast cancer susceptibility loci
Easton, D. F., Pooley, K. A., Dunning, A. M., Pharoah, P. D. P., Thompson, D., Ballinger, D. G., Struewing, J. P., Morrison, J., Field, H., Luben, R., et al. (2007) · Nature · doi:10.1038/nature05887 · Cited by 2,319
Publication and related bias in meta-analysis
Sterne, J. A. C., Gavaghan, D., & Egger, M. (2000) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(00)00242-0 · Cited by 2,282
Replication of Genome-Wide Association Signals in UK Samples Reveals Risk Loci for Type 2 Diabetes
Zeggini, E., Weedon, M. N., Lindgren, C. M., Frayling, T. M., Elliott, K. S., Lango, H., Timpson, N. J., Perry, J. R. B., Rayner, N. W., Freathy, R. M., et al. (2007) · Science · doi:10.1126/science.1142364 · Cited by 2,194
Systematic meta-analyses of Alzheimer disease genetic association studies: the AlzGene database
Bertram, L., McQueen, M. B., Mullin, K., Blacker, D., & Tanzi, R. E. (2007) · Nature Genetics · doi:10.1038/ng1934 · Cited by 1,857 · cited 5×
“AlzGene studies have been scrutinized for overlapping data sets ( 33 ).”
BackgroundMethodSelf-citationChan, A.-W., Hróbjartsson, A., Haahr, M. T., Gøtzsche, P. C., & Altman, D. G. (2004) · JAMA · doi:10.1001/jama.291.20.2457 · Cited by 1,845
The case of the misleading funnel plot
Lau, J., Ioannidis, J. P. A., Terrin, N., Schmid, C. H., & Olkin, I. (2006) · BMJ · doi:10.1136/bmj.333.7568.597 · Cited by 1,766
Self-citationA comprehensive review of genetic association studies
Hirschhorn, J. N., Lohmueller, K., Byrne, E., & Hirschhorn, K. (2002) · Genetics in Medicine · doi:10.1097/00125817-200203000-00002 · Cited by 1,674 · cited 1×
“This situation is consistent with the well-described lack of replication documented in diverse fields of genetic associations ( 2 , 41).”
ResultThe Existence of Publication Bias and Risk Factors for Its Occurrence
Dickersin, K. (1990) · JAMA: The Journal of the American Medical Association · doi:10.1001/jama.1990.03440100097014 · Cited by 1,417
Replicating genotype–phenotype associations
NCI-NHGRI Working Group on Replication in Association Studies, Chanock, S. J., Manolio, T., Boehnke, M., Boerwinkle, E., Hunter, D. J., Thomas, G., Hirschhorn, J. N., Abecasis, G., Altshuler, D., et al. (2007) · Nature · doi:10.1038/447655a · Cited by 1,352 · cited 1×
“With the advent of massive-testing technologies, new challenges arise (3, 5 ), and significance-chasing biases may still erode the credibility of accumulated evidence (48).”
BackgroundA comparison of methods to detect publication bias in meta‐analysis
Macaskill, P., Walter, S. D., & Irwig, L. (2001) · Statistics in Medicine · doi:10.1002/sim.698 · Cited by 1,286 · cited 1×
“Inferences derived from single meta-analyses are limited by the fact that usually few studies are available; thus, these tests are underpowered (25, 26 ).”
Background
Cited by
Why Most Discovered True Associations Are Inflated
Ioannidis, J. P. A. (2008) · Epidemiology · doi:10.1097/ede.0b013e31818131e7 · Cited by 1,495
Self-citationTheodoratou, E., Tzoulaki, I., Zgaga, L., & Ioannidis, J. P. A. (2014) · BMJ · doi:10.1136/bmj.g2035 · Cited by 956
Self-citationDunn, W. B., Broadhurst, D. I., Atherton, H. J., Goodacre, R., & Griffin, J. L. (2011) · Chem. Soc. Rev. · doi:10.1039/b906712b · Cited by 860
Long-term outcomes of childhood sexual abuse: an umbrella review
Hailes, H. P., Yu, R., Danese, A., & Fazel, S. (2019) · The Lancet Psychiatry · doi:10.1016/s2215-0366(19)30286-x · Cited by 827
Ioannidis, J. P. A. (2009) · Canadian Medical Association Journal · doi:10.1503/cmaj.081086 · Cited by 716
“Finally, domain analyses can be useful in understanding biases in observational research.(49,50) There are also examples of meta-epidemiologic research in the prognostic(51) and diagnostic(52,53) literature.”
BackgroundSelf-citation
Metadata
- Authors
- Kavvoura, F. K., McQueen, M. B., Khoury, M. J., Tanzi, R. E., Bertram, L., & Ioannidis, J. P. A.
- Venue
- American Journal of Epidemiology
- Publisher
- Oxford University Press (OUP)
- Published
- 2008-09-08
- Volume
- 168
- Issue
- 8
- Pages
- 855-865
- ISSN
- 0002-9262 (ISSN-L)
- 1476-6256
- OpenAlex
- W2124001429
- Cited by
- 48
- References
- 15
- FWCI
- 5.99
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Evaluation of the Potential Excess of Statistically Significant Findings in Published Genetic Association Studies: Application to Alzheimer's Disease. https://app.cochat.ai/papers/10.1093/aje/kwn206
- The paper
- Kavvoura, F. K., McQueen, M. B., Khoury, M. J., Tanzi, R. E., Bertram, L., & Ioannidis, J. P. A. (2008). Evaluation of the Potential Excess of Statistically Significant Findings in Published Genetic Association Studies: Application to Alzheimer's Disease. American Journal of Epidemiology. https://doi.org/10.1093/aje/kwn206
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.