Obstacles and opportunities in meta-analysis of genetic association studies
Open accessAbstract
Genetic association studies have the potential to advance our understanding of genotype-phenotype relationships, especially for common, complex diseases where other approaches, such as linkage, are less powerful. Unfortunately, many reported studies are not replicated or corroborated. This lack of reproducibility has many potential causes, relating to study design, sample size, and power issues, and from sources of true variability among populations. Genetic association studies can be considered as more similar to randomized trials than other types of observational epidemiological studies because of "Mendelian randomization" (Mendel's second law). The rationale and methodology for synthesizing randomized trials is highly relevant to the meta-analysis of genetic association studies. Nevertheless, there are a number of obstacles to overcome when performing such meta-analyses. In this review, the impacts of Type I error, lack of power, and publication and reporting biases are explored, and the role of multiple testing is discussed. A number of special features of association studies are especially pertinent, because they may lead to true variability among study results. These include population dynamics and structure, linkage disequilibrium, conformity to Hardy-Weinberg Equilibrium, bias, population stratification, statistical heterogeneity, epistatic and environmental interactions, and the choice of statistical models used in the analysis. Approaches to dealing with these issues are outlined. The supreme importance of complete and consistent study reporting and of making data readily available is also highlighted as a prerequisite for sound meta-analysis. We believe that systematic review and meta-analysis has an important role to play in understanding genetic association studies and should help us to separate the wheat from the chaff.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The need for complete reporting of all relevant methodological aspects as a prerequisite to assessing the risk of bias of studies was highlighted in 10 articles, and was the focus of recommendations in a publication from the MOOSE group.
“The need for complete reporting of all relevant methodological aspects as a prerequisite to assessing the risk of bias of studies was highlighted in 10 articles, [16, 33, 35, 47, 50, 56, 86, 87, 90, 91] and was the focus of recommendations in a publication from the Meta-analysis Of Observational Studies in Epidemiology (MOOSE) group [12].”
From Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations (2018) · cites this paper as [87]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Measuring inconsistency in meta-analyses
Higgins, J. P. T., Thompson, S. G., Deeks, J. J., & Altman, D. G. (2003) · BMJ · doi:10.1136/bmj.327.7414.557 · Cited by 63,139 · see citing works
Self-citationDavey Smith, G., & Ebrahim, S. (2003) · International Journal of Epidemiology · doi:10.1093/ije/dyg070 · Cited by 5,882 · see citing works
Moher, D., Schulz, K. F., & Altman, D. G. (2001) · The Lancet · doi:10.1016/s0140-6736(00)04337-3 · Cited by 5,373 · see citing works
Assessing the quality of controlled clinical trials
Jüni, P., Altman, D. G., & Egger, M. (2001) · BMJ · doi:10.1136/bmj.323.7303.42 · Cited by 3,309 · see citing works
How should meta‐regression analyses be undertaken and interpreted?
Thompson, S. G., & Higgins, J. P. T. (2002) · Statistics in Medicine · doi:10.1002/sim.1187 · Cited by 3,026 · see citing works
Self-citationCharbonneau, D. H. (2005) · Europe PMC (PubMed Central) · Cited by 2,041 · see citing works
Replication validity of genetic association studies
Ioannidis, J., Ntzani, E., Trikalinos, T., & Contopoulos-Ioannidis, D. (2001) · Nature Genetics · doi:10.1038/ng749 · Cited by 1,926 · see citing works
Lohmueller, K. E., Pearce, C. L., Pike, M., Lander, E. S., & Hirschhorn, J. N. (2003) · Nature Genetics · doi:10.1038/ng1071 · Cited by 1,900 · see citing works
Chan, A.-W., Hróbjartsson, A., Haahr, M. T., Gøtzsche, P. C., & Altman, D. G. (2004) · JAMA · doi:10.1001/jama.291.20.2457 · Cited by 1,845 · see citing works
Searching for genetic determinants in the new millennium
Risch, N. J. (2000) · Nature · doi:10.1038/35015718 · Cited by 1,839 · see citing works
(2000) · Queen Mary Research Online (Queen Mary University of London) · Cited by 1,795 · see citing works
Systematic Reviews in Health Care: Meta‐Analysis in Context
(2001) · doi:10.1002/9780470693926 · Cited by 1,693 · see citing works
A comprehensive review of genetic association studies
Hirschhorn, J. N., Lohmueller, K., Byrne, E., & Hirschhorn, K. (2002) · Genetics in Medicine · doi:10.1097/00125817-200203000-00002 · Cited by 1,674 · see citing works
Association study designs for complex diseases
Cardon, L. R., & Bell, J. I. (2001) · Nature Reviews Genetics · doi:10.1038/35052543 · Cited by 1,380 · see citing works
Problems of reporting genetic associations with complex outcomes
Colhoun, H. M., McKeigue, P., & Smith, G. D. (2003) · The Lancet · doi:10.1016/s0140-6736(03)12715-8 · Cited by 1,203 · see citing works
Use of Unlinked Genetic Markers to Detect Population Stratification in Association Studies
Pritchard, J. K., & Rosenberg, N. A. (1999) · The American Journal of Human Genetics · doi:10.1086/302449 · Cited by 1,198 · see citing works
Population stratification and spurious allelic association
Cardon, L. R., & Palmer, L. J. (2003) · The Lancet · doi:10.1016/s0140-6736(03)12520-2 · Cited by 1,189 · see citing works
Meta‐Analysis Of Controlled Clinical Trials
Whitehead, A. (2002) · doi:10.1002/0470854200 · Cited by 827 · see citing works
Higgins, J. P. T., Thompson, S. G., Deeks, J. J., & Altman, D. G. (2002) · Journal of Health Services Research & Policy · doi:10.1258/1355819021927674 · Cited by 755 · see citing works
Self-citationFundamentals of Genetic Epidemiology
Khoury, M. J., Beaty, T. H., & Cohen, B. H. (1993) · doi:10.1093/oso/9780195052886.001.0001 · Cited by 699 · see citing works
Genetic associations in large versus small studies: an empirical assessment
Ioannidis, J. P. A., Trikalinos, T. A., Ntzani, E., & Contopoulos‐Ioannidis, D. G. (2003) · The Lancet · doi:10.1016/s0140-6736(03)12516-0 · Cited by 573 · see citing works
Hein, D. W. (2002) · Mutation Research - Fundamental and Molecular Mechanisms of Mutagenesis · doi:10.1016/s0027-5107(02)00153-7 · Cited by 518 · see citing works
Wacholder, S. (2000) · JNCI Journal of the National Cancer Institute · doi:10.1093/jnci/92.14.1151 · Cited by 442 · see citing works
The nature and identification of quantitative trait loci: a community's view
Members of the Complex Trait Consortium (2003) · Nature Reviews Genetics · doi:10.1038/nrg1206 · Cited by 437 · see citing works
Detection of genotyping errors by Hardy–Weinberg equilibrium testing
Hosking, L. K., Lumsden, S. M., Lewis, K. F., Yeo, A. J. M., McCarthy, L. C., Bansal, A. T., Riley, J., Purvis, I. J., & Xu, C. (2004) · European Journal of Human Genetics · doi:10.1038/sj.ejhg.5201164 · Cited by 405 · see citing works
Reporting, Appraising, and Integrating Data on Genotype Prevalence and Gene-Disease Associations
Little, J., Bradley, L., Bray, M. S., Clyne, M., Dorman, J. S., Ellsworth, D. L., Hanson, J., Khoury, M. J., Lau, J., OʼBrien, T. R., et al. (2002) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a000179 · Cited by 366 · see citing works
Meta-analysis of genetic association studies.
Munafo, M., & Flint, J. (2004) · Trends in Genetics · doi:10.1016/j.tig.2004.06.014 · Cited by 346 · see citing works
Meta-analyses of molecular association studies: Methodologic lessons for genetic epidemiology
Attia, J. (2003) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(03)00011-8 · Cited by 311 · see citing works
Genetic associations: false or true?
Ioannidis, J. P. A. (2003) · Trends in Molecular Medicine · doi:10.1016/s1471-4914(03)00030-3 · Cited by 289 · see citing works
Modelling publication bias in meta-analysis: a review
Sutton, A. J., Song, F., Gilbody, S. M., & Abrams, K. R. (2000) · Statistical Methods in Medical Research · doi:10.1177/096228020000900503 · Cited by 288 · see citing works
for the CONSORT Group*, Moher, D., Schulz, K. F., & Altman, D. G. (2003) · Clinical Oral Investigations · doi:10.1007/s00784-002-0188-x · Cited by 259 · see citing works
Methods for Exploring Heterogeneity in Meta-Analysis
Song, F., Sheldon, T., Sutton, A. J., Abrams, K. R., & Jones, D. R. (2001) · Evaluation & the Health Professions · doi:10.1177/016327870102400203 · Cited by 253 · see citing works
Bias in meta‐analysis due to outcome variable selection within studies
Hutton, J. L., & Williamson, P. (2000) · Journal of the Royal Statistical Society Series C (Applied Statistics) · doi:10.1111/1467-9876.00197 · Cited by 236 · see citing works
22q11 deletion syndrome in adults with schizophrenia
Bassett, A. S., Hodgkinson, K. A., Chow, E. W. C., Correia, S., Scutt, L. E., & Weksberg, R. (1998) · American Journal of Medical Genetics · doi:10.1002/(sici)1096-8628(19980710)81:4 · Cited by 228 · see citing works
Witte, J. S., Gauderman, W. J., & Thomas, D. C. (1999) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a009877 · Cited by 211 · see citing works
Association studies of genetic polymorphisms and complex disease
Cheung, B. M. Y., & Kumana, C. R. (2000) · The Lancet · doi:10.1016/s0140-6736(05)74708-5 · Cited by 188 · see citing works
Clinical Epidemiological Quality in Molecular Genetic Research
Bogardus, S. T., Concato, J., & Feinstein, A. R. (1999) · JAMA · doi:10.1001/jama.281.20.1919 · Cited by 175 · see citing works
GT Repeats Are Associated with Recombination on Human Chromosome 22Preprint
Majewski, J., & Ott, J. (2000) · Genome Research · doi:10.1101/gr.10.8.1108 · Cited by 127 · see citing works
Hardy–Weinberg quality control
Gomes, I. B., Collins, A. R., Lonjou, C., et al. (1999) · Annals of Human Genetics · doi:10.1046/j.1469-1809.1999.6360535.x · Cited by 126 · see citing works
Interpretation of genetic association studies in complex disease
Campbell, H., & Rudan, I. (2002) · The Pharmacogenomics Journal · doi:10.1038/sj.tpj.6500132 · Cited by 124 · see citing works
Fundamentals of genetic epidemiology
Young, I. D. (1994) · Neuromuscular Disorders · doi:10.1016/0960-8966(94)90036-1 · Cited by 123 · see citing works
Heath, K. E. (2001) · Atherosclerosis · doi:10.1016/s0021-9150(00)00647-x · Cited by 121 · see citing works
Commentary: Meta-analysis of Individual Participants' Data in Genetic Epidemiology
Ioannidis, J. P. A. (2002) · American Journal of Epidemiology · doi:10.1093/aje/kwf031 · Cited by 117 · see citing works
Hahn, S., Williamson, P. R., & Hutton, J. L. (2002) · Journal of Evaluation in Clinical Practice · doi:10.1046/j.1365-2753.2002.00314.x · Cited by 97 · see citing works
Establishment of genetic associations for complex diseases is independent of early study findings
Trikalinos, T. A., Ntzani, E., Contopoulos‐Ioannidis, D. G., & Ioannidis, J. P. A. (2004) · European Journal of Human Genetics · doi:10.1038/sj.ejhg.5201227 · Cited by 93 · see citing works
Human Genome Epidemiologic Reviews: The Beginning of Something HuGE
Khoury, M. J., & Little, J. (2000) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a010117 · Cited by 73 · see citing works
Mendelian randomisation: a new spin or real progress?
Little, J., & Khoury, M. J. (2003) · The Lancet · doi:10.1016/s0140-6736(03)14396-6 · Cited by 70 · see citing works
The Human Genome Project Is Complete. How Do We Develop a Handle for the Pump?
Little, J. (2003) · American Journal of Epidemiology · doi:10.1093/aje/kwg048 · Cited by 66 · see citing works
Methodological issues in pooled analysis of biomarker studies
Taioli, E. (2002) · Mutation Research/Reviews in Mutation Research · doi:10.1016/s1383-5742(02)00027-3 · Cited by 37 · see citing works
Genetic association studies in Alzheimer's disease research: challenges and opportunities
Edland, S. D., Slager, S. L., & Farrer, M. J. (2004) · Statistics in Medicine · doi:10.1002/sim.1706 · Cited by 20 · see citing works
Flagged first, then by citations
Research works printed but not linked: 2
- Little J, Khoury MJ. Mendelian randomisation: a new spin or real progress? Lancet 2003;362:930 -931.
- Bellivier F, Schurhoff F, Nosten-Bertrand M et al. Methodological problems in meta-analysis of association studies between bipolar affective disorders and the ty- rosine hydroxylase gene. Am J Med Genet 1998;81:349 -352.
Cited by
Sorted by citations
Mueller, M., D’Addario, M., Egger, M., Cevallos, M., Dekkers, O., Mugglin, C., & Scott, P. (2018) · BMC Medical Research Methodology · doi:10.1186/s12874-018-0495-9 · Cited by 400 · see citing works
Barnett, J. H., Scoriels, L., & Munafò, M. R. (2008) · Biological Psychiatry · doi:10.1016/j.biopsych.2008.01.005 · Cited by 412 · see citing works
Comprehensive literature review and statistical considerations for GWAS meta-analysis
Begum, F., Ghosh, D., Tseng, G. C., & Feingold, E. (2012) · Nucleic Acids Research · doi:10.1093/nar/gkr1255 · Cited by 179 · see citing works
Nakaoka, H., & Inoue, I. (2009) · Journal of Human Genetics · doi:10.1038/jhg.2009.95 · Cited by 111 · see citing works
Lahey, B. B., Rathouz, P. J., Lee, S. S., Chronis-Tuscano, A., Pelham, W. E., Waldman, I. D., & Cook, E. H. (2011) · Journal of Abnormal Psychology · doi:10.1037/a0021133 · Cited by 63 · see citing works
Genetic Risk Factors for Placental Abruption: A HuGE Review and Meta-Analysis
Zdoukopoulos, N., & Zintzaras, E. (2008) · Epidemiology · doi:10.1097/ede.0b013e3181635694 · Cited by 56 · see citing works
- Authors
- Salanti, G., Sanderson, S., & Higgins, J. P. T.
- Affiliations
- MRC Biostatistics Unit (GB)
- University of Cambridge (GB)
- Venue
- Genetics in Medicine
- Publisher
- Elsevier BV
- Published
- 2005-01
- Volume
- 7
- Issue
- 1
- Pages
- 13-20
- ISSN
- 1098-3600 (ISSN-L)
- OpenAlex
- W2011987261
- Cited by
- 196
- References
- 50
- Citations by year
- 2026: 2 · 2025: 1 · 2023: 1 · 2022: 8 · 2021: 4 · 2020: 4 · 2019: 3 · 2018: 4 · 2017: 4 · 2016: 8 · 2015: 18 · 2014: 10 · 2013: 12 · 2012: 12
- FWCI
- 3.33
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- Locations
- Genetics in Medicine (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Genetics › Genetic Associations and Epidemiology
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Keywords
- genetic association studies, meta-analysis, reproducibility, replication, type I error, statistical power, publication bias, reporting bias, multiple comparisons, population stratification, linkage disequilibrium, Hardy-Weinberg equilibrium, statistical heterogeneity, epistasis, Mendelian randomization, research design, data availability
- MeSH terms
- Epistasis, Genetic, False Positive Reactions, Genetics, Population, Genotype, Humans, Phenotype, Meta-Analysis as Topic, Linkage Disequilibrium, Randomized Controlled Trials as Topic, Publication Bias
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
