Replication and heterogeneity in gene×environment interaction studies
Open accessThe DOI matches its Crossref record; 4 of the 4 references it cites were found in a registry or an index.
Abstract
There has been considerable interest in gene×environment (G×E) interactions in psychiatric genetics since studies by Caspi and colleagues (Caspi et al.2002, 2003) suggested that these may illuminate the aetiology and genetic architecture of behavioural phenotypes. However, as with earlier candidate gene studies (Ioannidis et al.2001), initial promise was subsequently followed by failures to replicate and increasing inconsistency in the rapidly growing G×E literature (Munafò et al.2009a). One reason for this is that the greater number of potential statistical tests that are possible when interaction effects are included in any analysis (e.g. by investigating multiple markers of environmental exposures) greatly increases the risk of false positives in studies of this kind (Munafò et al.2009a). However, another possibility is that there is genuine heterogeneity between studies, which reflects the sampling of different populations across which the genetic effects of interest may vary in strength or direction. Laucht and colleagues (Laucht et al.2009) report evidence that individuals homozygous for the L allele of the 5-HTTLPR displayed higher rates of depressive or anxiety disorders when exposed to environmental adversity, contrary to the original report by Caspi and colleagues in which possession of the S allele increased susceptibility to depression under stressful conditions (Caspi et al.2003). Laucht and colleagues argue that a ‘possible source for the conflicting findings might be attributed to heterogeneity in depression phenotypes and environmental adversity’. An alternative and perhaps more parsimonious explanation is that their result is further evidence that the literature to date is compatible with chance findings, perhaps when seen through the filter of selective reporting of the most promising findings obtained from multiple unreported statistical tests. A recent simulation study by Sullivan (2007) illustrates the ease with which potentially publishable ‘findings’ may be obtained from random data. Sullivan simulated a candidate gene association dataset and showed that in over 90% of cases some potentially publishable (i.e. nominally significant) correlation between a genetic variant and a phenotype might be obtained, given multiple possible groupings of genotype groups, multiple polymorphisms tested, and so on. Furthermore, Sullivan showed that in the majority of cases these ‘findings’ can be replicated, given a weak definition of ‘replication’, again using random data. It is noteworthy that this simulation focused on simple gene–disease association studies, and the degree of unreported multiple testing in G×E studies is expected to be greater still. It is of course theoretically possible that genetic associations (and, by extension, G×E effects) might depend on population structure, which may be specific to the sample analysed, or on the phenotype measure used. Indeed, there are precedents for this, where the widely reported association between the 5-HTTLPR and anxiety-related traits has been suggested to depend on the specific measurement instrument employed (Munafò et al.2009b). However, few robust gene–disease associations differ as a function of sample ancestry (Ioannidis et al.2004), and many investigations of apparent sex differences in genetic associations appear to be conducted post hoc (Patsopoulos et al.2007), highlighting again the potential for multiple statistical tests to be conducted on datasets of this kind. Can we discriminate between genuine heterogeneity which may inform and advance our understanding of the aetiology of these phenotypes, and stochastic variation across studies which reflects nothing more than variation that might be expected to be observed by chance (and subsequently filtered through various reporting and publication biases)? Clayton & McKeigue (2001) argue that, while it is theoretically possible that G×E studies might afford increased statistical power, by allowing us to focus on specific subgroups where any genetic effects might be expected to be largest, this is difficult in practice because it is rare that we will have strong a priori reasons for expecting any effect to be restricted to any specific subgroup. However, in the case of an initial finding we might specify a subgroup on the basis of this finding in our subsequent attempts at replication. That is, following the initial report of an interaction between 5-HTTLPR genotype and stressful life events on risk of subsequent depression, we might design our replication studies to focus exclusively on individuals exposed to stressful life events (defined in the same way as in the original study). This would substantially reduce the multiple statistical testing burden. However, it might be argued that this approach would substantially restrict the scope for exploratory analyses in subsequent analyses, and might prevent existing datasets from being used if they did not include variables sufficiently comparabl
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Martinson, B. C., Anderson, M. S., & de Vries, R. (2005) · Nature · doi:10.1038/435737a · Cited by 1,096
Claims of Sex Differences: An Empirical Assessment in Genetic Associations
Patsopoulos, N. A., Tatsioni, A., & Ioannidis, J. P. A. (2007) · JAMA · doi:10.1001/jama.298.8.880 · Cited by 150
Bias in genetic association studies and impact factor
Munafò, M. R., Stothart, G., & Flint, J. (2009) · Molecular Psychiatry · doi:10.1038/mp.2008.77 · Cited by 60
Self-citationMUNAFÒ, M. R., ATTWOOD, A. S., & FLINT, J. (2008) · Psychological Medicine · doi:10.1017/s003329170800353x · Cited by 32
Self-citation
Research works printed but not linked: 10
- Ioannidis, 2005, Public Library of Science Medicine, 2, e124
- 10.1016/S0140-6736(01)06418-2 · DOI · Paper page
- 10.1126/science.1083968 · DOI · Paper page
- 10.1126/science.1072290 · DOI · Paper page
- 10.1016/j.biopsych.2006.11.010 · DOI · Paper page
- 10.1038/ng1474 · DOI · Paper page
- 10.1002/ajmg.b.30808 · DOI · Paper page
- 10.1016/j.biopsych.2008.06.009 · DOI · Paper page
- Laucht, 2009, International Journal of Neuropsychopharmacology
- 10.1038/ng749 · DOI · Paper page
Cited by
Duncan, L. E., & Keller, M. C. (2011) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2011.11020191 · Cited by 1,032
“Critics worry about the multiple testing problem combined with publication bias against null results (6).”
Keller, M. C. (2014) · Biological Psychiatry · doi:10.1016/j.biopsych.2013.09.006 · Cited by 625
Candidate Gene–Environment Interaction Research: Reflections and Recommendations
Dick, D. M., Agrawal, A., Keller, M. C., Adkins, A., Aliev, F., Monroe, S., Hewitt, J. K., Kendler, K. S., & Sher, K. J. (2015) · Perspectives on Psychological Science · doi:10.1177/1745691614556682 · Cited by 361
Culverhouse, R. C., Saccone, N. L., Horton, A. C., Ma, Y., Anstey, K. J., Banaschewski, T., Burmeister, M., Cohen-Woods, S., Etain, B., Fisher, H. L., et al. (2018) · Molecular Psychiatry · doi:10.1038/mp.2017.44 · Cited by 321
Self-citationManuck, S. B., & McCaffery, J. M. (2014) · Annual Review of Psychology · doi:10.1146/annurev-psych-010213-115100 · Cited by 277
Metadata
- Authors
- Munafò, M. R., & Flint, J.
- Venue
- The International Journal of Neuropsychopharmacology
- Publisher
- Oxford University Press (OUP)
- Published
- 2009-05-28
- Volume
- 12
- Issue
- 06
- Pages
- 727
- ISSN
- 1461-1457 (ISSN-L)
- 1469-5111
- OpenAlex
- W2051222611
- Cited by
- 64
- References
- 4
- FWCI
- 4.63
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Replication and heterogeneity in gene×environment interaction studies. https://app.cochat.ai/papers/10.1017/s1461145709000479
- The paper
- Munafò, M. R., & Flint, J. (2009). Replication and heterogeneity in gene×environment interaction studies. The International Journal of Neuropsychopharmacology. https://doi.org/10.1017/s1461145709000479
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.