Guidelines for performing Mendelian randomization investigations

In the paper

“and the mode-based method assumes more variants estimate the true causal effect than estimate any other quantity (plurality valid assumption).”

What the sources say

• ?Could not verify

  The Discussion section (“Relation to existing literature”; page unavailable) describes a median-based method that requires at least half the variants to be valid, but does not describe a mode-based method or its plurality-valid assumption. Searches for the mode-based method and plurality-valid assumption found no passage addressing that claim in our copy. (LLM, conf=0.74)

  cites [19] J Bowden, 2015, Mendelian randomization with invalid instruments: effect estimation and bias detection through Egger regression., Int J Epidemiol. sourceExperimental — accuracy not yet measured
• ?Could not verify

  The Methods section describes simple and weighted median estimators, including the requirement that more than half of the variants be valid for the simple median. The passages found for “mode,” “modal,” and “plurality” do not describe a mode-based method or its plurality-valid assumption. Our copy therefore does not verify the claim attributed to this paper. (LLM, conf=0.73)

  cites [20] J Bowden, 2016, Consistent Estimation in Mendelian Randomization with Some Invalid Instruments Using a Weighted Median Estimator., Genet Epidemiol. sourceExperimental — accuracy not yet measured
• ?Could not verify

  The Introduction mentions mode-based MR as an approach for analyses with invalid instruments, but does not describe its plurality-valid assumption. Searches for that assumption and for modal effect estimates found no passage in our copy stating the claimed requirement. (LLM, conf=0.72)

  cites [92] E Porcu, 2019, Mendelian randomization integrating GWAS and eQTL data reveals genetic determinants of complex and clinical traits., Nat Commun. sourceExperimental — accuracy not yet measured

In the paper

“Summarized data are often available for larger sample sizes, meaning that power to detect a causal effect is increased.”

In the paper

“Alternatively, the two samples could differ substantially according to population characteristics such as age, sex, socio-economic background, and so on 32 . Such differences can affect not only the interpretation of causal estimates, but also the validity of causal inferences 33 .”

What the sources say

• Not checked yetcites [32] J Bowden, 2017, A framework for the investigation of pleiotropy in two-sample summary data Mendelian randomization., Stat Med. source
• Not checked yetcites [33] J Staley, 2016, PhenoScanner: a database of human genotype-phenotype associations., Bioinformatics. source

In the paper

“Moreover, two-sample approaches can be used with individual-level data (such as the use of externally-derived weights), and summarized data approaches can be used with one-sample data (if necessary, by creating the summarized data from the individual-level data 37 ).”

What the sources say

• Not checked yetcites [37] S Burgess, 2011, Avoiding bias from weak instruments in Mendelian randomization studies., Int J Epidemiol. source

In the paper

“Individual-level data are required to conduct analyses in specific subgroups or strata of the population, or to choose which variables to adjust for when generating the summarized data.”

In the paper

“One-sample analyses do not suffer from these concerns, nor do they require harmonization of the genetic variants across the datasets (see Section 4).”

In the paper

“A specific advantage of publicly available summarized data is transparency, as the analysis can be reproduced by a third party with access to the same data.”

In the paper

“Finally, dynastic effects occur when a parent’s genotype affects their child’s outcome by a causal pathway not acting via the child’s phenotype (for example, due to an effect of the parent’s phenotype on the child’s outcome). This can induce associations between the offspring’s genotype and outcomes that do not reflect the effect of the exposure in the offspring.”

In the paper

“For the special case of a one-sample analysis with a binary disease outcome, if the genetic associations with the exposure are estimated in the controls only, then genetic associations with the exposure and outcome will not be correlated, and bias will follow the pattern of the two-sample setting 42 .”

What the sources say

• Not checked yetcites [42] L Mokry, 2015, Vitamin D and Risk of Multiple Sclerosis: A Mendelian Randomization Study., PLoS Med. source

In the paper

“However, within-family analyses typically have lower power than analyses in unrelated individuals, as families where all individuals have the same genotype (such as where both parents are major homozygotes) would not contribute any information to the analysis.”

In the paper

“This results in instrumental variable estimates that are biased in the direction of the confounded association, and inflated false positive (type 1 error) rates, particularly when more than one variant is included in the analysis 41 .”

What the sources say

• Not checked yetcites [41] M Brion, 2013, Calculating statistical power in Mendelian randomization studies., Int J Epidemiol. source

In the paper

“In a two-sample setting without sample overlap, bias due to weak instruments is in the direction of the null, and does not lead to false positive findings.”

In the paper

“The key consequence of this randomization is that genetic variants are independently distributed from traits that they do not affect, an implication of Mendel’s laws of segregation and independent assortment.”

In the paper

“Various statistical methods have been proposed to reduce weak instrument bias due to sample overlap and bias due to winner’s curse (see Section 3) 43– 45 .”

What the sources say

• Not checked yetcites [43] D Swerdlow, 2012, The interleukin-6 receptor as a target for prevention of coronary heart disease: a mendelian randomisation analysis., Lancet. source
• Not checked yetcites [44] Q Zhao, 2019, Powerful three-sample genome-wide design and robust statistical inference in summary-data Mendelian randomization., Int J Epidemiol. source
• Not checked yetcites [45] G Davey Smith, 2014, Mendelian randomization: genetic anchors for causal inference in epidemiological studies., Hum Mol Genet. source

In the paper

“In practice, distinguishing between horizontal pleiotropy and vertical pleiotropy requires knowledge of the relationships between the variables in the analysis.”

In the paper

“For a polygenic analysis, there are two main strategies for selecting variants: either a biologically driven approach or a statistically driven approach.”

In the paper

“As an example, although genetic variants in the IL6R gene region are associated with increased circulating levels of interleukin-6, they in fact decrease interleukin-6 signalling, leading to opposite directions of association with disease outcomes to those expected based on serum interleukin-6 measurements 65 .”

What the sources say

• Not checked yetcites [65] S Burgess, 2017, Interpreting findings from Mendelian randomization using the MR-Egger method., Eur J Epidemiol. source

In the paper

“However, for complex multifactorial exposures such as body mass index or blood pressure, there is no single relevant gene, and so a more agnostic polygenic analysis may be necessary.”

In the paper

“When genetic variants are all valid instruments, power depends on the proportion of variance in the exposure explained by the variants 56 – hence a polygenic Mendelian randomization investigation will typically have greater power than one including variants only from a single gene region.”

What the sources say

• Not checked yetcites [56] M Greco, 2015, Detecting pleiotropy in Mendelian randomisation studies with summary data and a continuous outcome., Stat Med. source

In the paper

“Variants in the same gene region are likely to either all be valid instruments or all invalid.”

In the paper

“The two approaches are not mutually exclusive, and the overall decision of which variants to include may comprise elements from both approaches.”

In the paper

“Similar two-stage analyses can be performed with binary variables using logistic regression 73”

What the sources say

• Not checked yetcites [73] J Smith, 2014, Association of low-density lipoprotein cholesterol-related genetic variants with aortic valve calcium and incident aortic stenosis., JAMA. source

In the paper

“Scenario e reflects “horizontal pleiotropy” (also called “direct pleiotropy”) that violates the instrumental variable assumptions.”

In the paper

“Hence a null finding in a liberal analysis is more convincing evidence of a true null relationship – there is little evidence for a causal relationship even when potentially pleiotropic genetic variants are included in the analysis.”

In the paper

“If all genetic variants are valid instruments and the relationships between all variables (genetic variants, exposure and outcome) are linear and homogeneous for all individuals in the population, then we would expect the variant-specific estimates (that is, the ratio estimates based on each variant in turn) to all target the same causal parameter, and for there to be no more heterogeneity between”

What the sources say

• Not checked yetcites [13] B Bulik-Sullivan, 2015, LD Score regression distinguishes confounding from polygenicity in genome-wide association studies., Nat Genet. source

In the paper

“If there is a reverse causal effect of the outcome on the exposure, then genetic predictors of the outcome could be identified as hits in a genome-wide association study for the exposure. However, such variants would not be valid instrumental variables.”

In the paper

“Allele and strand information can be double-checked by comparing allele frequency information – if the allele frequencies are similar for the A and T alleles, then the researcher can be more confident that this is a strand mismatch.”

In the paper

“Causal conclusions will be more reliable when the instrumental variable assumptions are more plausible.”

In the paper

“Scenario f reflects a reverse causation situation where the genetic variant has been incorrectly identified as primarily affecting the exposure.”

In the paper

“Estimates from the MR-Egger method are particularly affected by outlying and influential datapoints 90 , and are prone to be imprecise, particularly when the variant—exposure associations are all similar in magnitude.”

What the sources say

• Not checked yetcites [90] J White, 2016, Plasma urate concentration and risk of coronary heart disease: a Mendelian randomisation analysis., Lancet Diabetes Endocrinol. source

In the paper

“Examples of exposure sets where multivariable Mendelian randomization has been used include lipid fractions (such as high-density lipoprotein cholesterol, LDL-cholesterol, and triglycerides) 82 , and body composition measures (such as fat mass and fat-free mass) 83 .”

What the sources say

• Not checked yetcites [82] N Solovieff, 2013, Pleiotropy in complex traits: challenges and strategies., Nat Rev Genet. source
• Not checked yetcites [83] M Benn, 2017, Low LDL cholesterol, PCSK9 and HMGCR genetic variation, and risk of Alzheimer's disease and Parkinson's disease: Mendelian randomisation study., BMJ. source

In the paper

“If there is no more heterogeneity between the ratio estimates for the individual variants than would be expected by chance alone, then the random-effect analysis is equivalent to the fixed-effect analysis, and there is no loss of precision in making the weaker random-effects assumption.”

In the paper

“The instrumental variable assumptions in multivariable Mendelian randomization require each variant to be associated with at least one of the exposures, not associated with the outcome via confounding, and not to affect the outcome except potentially via its association with one or more of the exposures included in the analysis model.”

In the paper

“The most commonly used robust methods are MR-Egger, median- and mode-based methods, and MR-PRESSO.”

In the paper

“However, the robust methods are more likely to be effective for addressing instrument invalidity that arises due to issues such as pleiotropy or linkage disequilibrium with a variant influencing a confounder, which affect specific variants in a sporadic way, and less effective for instrument invalidity that arises due to issues such as population stratification or dynastic effects, which affect”

What the sources say

• Not checked yetcites [48] D Lawlor, 2008, Mendelian randomization: using genes as instruments for making causal inferences in epidemiology., Stat Med. source

In the paper

“A causal model is evidenced if the predominance of variants that associate with the exposure also associate with the outcome in a proportional way.”

In the paper

“Another method making the InSIDE assumption is the MR-RAPS (robust adjusted profile score) method, which first excludes strongly pleiotropic variants, and then assumes all remaining variants follow the InSIDE assumption 18 .”

What the sources say

• Not checked yetcites [18] S Burgess, 2013, Mendelian randomization analysis with multiple genetic variants using summarized data., Genet Epidemiol. source

In the paper

“The MR-PRESSO method is a variation on the IVW method that first sequentially removes genetic variants from the analysis whose variant-specific causal estimate differs substantially from those of other variants 21 .”

What the sources say

• Not checked yetcites [21] F Hartwig, 2017, Robust inference in summary data Mendelian randomization via the zero modal pleiotropy assumption., Int J Epidemiol. source

In the paper

“Leave-one-out analyses (i.e. remove one variant from the analysis and re-estimate the causal effect) can be valuable in assessing the reliance of a Mendelian randomization analysis on a particular variant 100 .”

What the sources say

• Not checked yetcites [100] M Holmes, 2017, Mendelian randomization in cardiometabolic disease: challenges in evaluating causality., Nat Rev Cardiol. source

In the paper

“It is unlikely that one approach will perform best in all settings”

In the paper

“If estimates from all methods are similar, then any causal claim is more credible.”

In the paper

“Conclusions are more reliable when multiple genetic variants provide concordant evidence for a causal effect, and particularly when there is no more heterogeneity between the variant-specific causal estimates than expected by chance.”

In the paper

“Combining this with orthogonal validation – such as through the use of positive and negative controls ( Section 7) – is the most powerful way of addressing causal questions, within the triangulation of evidence framework 96, 97 .”

What the sources say

• Not checked yetcites [96] S Burgess, 2015, Network Mendelian randomization: using genetic variants as instrumental variables to investigate mediation in causal pathways., Int J Epidemiol. source
• Not checked yetcites [97] J Bowden, 2018, Improving the visualization, interpretation and analysis of two-sample summary data Mendelian randomization via the Radial plot and Radial regression., Int J Epidemiol. source

In the paper

“Removing a variant from the analysis is more justified when a pleiotropic association of the variant has been identified 103 .”

What the sources say

• Not checked yetcites [103] D Lawlor, 2016, Triangulation in aetiological epidemiology., Int J Epidemiol. source

In the paper

“A further approach for identifying variants to remove from the analysis is Steiger filtering, which removes variants from the analysis if their association with the outcome is stronger than that with the exposure 102 .”

What the sources say

• Not checked yetcites [102] M Munafò, 2018, Robust research needs many lines of evidence., Nature. source

In the paper

“For example, childhood levels of vitamin D have been used as a negative control outcome for the effect of adulthood BMI; childhood BMI was shown to affect childhood vitamin D levels in a Mendelian randomization investigation, but adulthood BMI did not 106 .”

What the sources say

• Not checked yetcites [106] S Burgess, 2019, How humans can contribute to Mendelian randomization analyses., Int J Epidemiol. source

In the paper

“Other approaches for assessing robustness include varying the dataset and choice of genetic variants in the analysis (including the suggestion of liberal and conservative variant sets in Section 3), the use of positive and negative control outcomes and/or samples, colocalization, subgroup analyses, and examining associations with potentially pleiotropic variables.”

In the paper

“If there are many variants in an analysis, leaving one variant out at a time is unlikely to change the estimate substantially, and leaving out subsets of the variants (say, a randomly chosen 30% at a time 101 ) may be more appropriate.”

What the sources say

• Not checked yetcites [101] T Richardson, 2019, Can the impact of childhood adiposity on disease risk be reversed? A Mendelian randomization study., medRxiv. source

In the paper

“This is particularly important for investigations based on a single gene region, as several of the methods discussed above are not applicable in this case.”

In the paper

“Hence, while in some cases colocalization methods will provide helpful evidence supporting or questioning the Mendelian randomization assumptions, in other cases they may provide no strong evidence for or against colocalization 112 .”

What the sources say

• Not checked yetcites [112]

In the paper

“A colocalization analysis revealed distinct causal variants for LDL-cholesterol and Alzheimer’s disease at the APOE locus, indicating that variants in this gene region are not valid instruments assessing the effect of LDL-cholesterol on Alzheimer’s disease 112 .”

What the sources say

• Not checked yetcites [112]

In the paper

“A further possible sensitivity analysis is to check the genetic associations with other variables associated with the outcome, and which are thought not to lie on the causal pathway through the exposure (i.e. are not mediators).”

In the paper

“As sex cannot be affected by autosomal genetic variants, sex cannot be a collider, and so stratification on sex will not induce collider bias 119 .”

What the sources say

• Not checked yetcites [119]

In the paper

“A further possibility in this case is to perform a multivariable Mendelian randomization, including the putative pleiotropic variables as additional exposures in the analysis model 120 .”

What the sources say

• Not checked yetcites [120]

In the paper

“Simulation studies have shown that selection bias can have a severe impact on Mendelian randomization estimates, but only when the selection effects are quite strong 72, 122 .”

What the sources say

• Not checked yetcites [72] L Corbin, 2016, BMI as a Modifiable Risk Factor for Type 2 Diabetes: Refining and Understanding Causal Estimates Using Mendelian Randomization., Diabetes. source
• Not checked yetcites [122]

In the paper

“Collider bias is not unique to Mendelian randomization, but it is particularly relevant as some published genetic association estimates have been adjusted for potential colliders 123 .”

What the sources say

• Not checked yetcites [123]

In the paper

“Selection bias is a specific example of collider bias which occurs when selection into a study sample depends on a collider.”

In the paper

“Specific weights to reduce selection bias have been proposed for the UK Biobank study 131 .”

What the sources say

• Not checked yetcites [131]

In the paper

“Selection bias can potentially be addressed using inverse-probability weighting 130 , although this requires estimation of the probability of selection into the study sample for all participants.”

What the sources say

• Not checked yetcites [130]

In the paper

“This analysis will estimate the direct effect of the exposure on the outcome keeping these variables constant.”

In the paper

“Each of these steps can be performed using standard Mendelian randomization, although separate instrumental variables are required for the exposure and mediator.”

In the paper

“A weakness of these investigations is that statistical power to detect an interaction is often low, in which case a null finding should not be interpreted as strong evidence for lack of interaction.”

In the paper

“Researchers should be cautious when performing such analyses to ensure that the values of the exposure at different time periods genuinely represent biologically distinct risk factors, and not simply measures taken at different times but capturing the same essential risk factor 144 .”

What the sources say

• Not checked yetcites [144]

In the paper

“The doubly-ranked stratification method has been proposed that may be less sensitive to variability in these genetic effects 139 .”

What the sources say

• Not checked yetcites [139]

In the paper

“Indeed, variability in the effect of genetic variants on the exposure is evident for several exposures; this variability can lead to highly misleading estimates 137, 138 .”

What the sources say

• Not checked yetcites [137]
• Not checked yetcites [138]

In the paper

“The genetic code is fixed at conception, and so Mendelian randomization investigations typically compare groups of the population having different trajectories in their distribution of the exposure over time 161 .”

What the sources say

• Not checked yetcites [161]

In the paper

“For example, in a polygenic analysis, a scatter plot of the genetic associations with the outcome against the genetic associations with the exposure reveals much about the analysis – whether different genetic variants provide similar estimates of the causal effect or if there is considerable heterogeneity, and whether the analysis is dominated by a single genetic variant or not 31 .”

What the sources say

• Not checked yetcites [31] J Labrecque, 2019, Interpretation and Potential Biases of Mendelian Randomization Estimates With Time-Varying Exposures., Am J Epidemiol. source

In the paper

“Plots allow the investigators and readers to assess the reliability of the analysis method and its underlying assumptions, and we strongly recommend their inclusion in a manuscript.”

In the paper

“For multivariable Mendelian randomization, the conditional F statistic is a more relevant measure of instrument strength, and assesses the strength of association of the variants with each exposure in turn after accounting for the other exposures in the model 158 .”

What the sources say

• Not checked yetcites [158]

In the paper

“Even if a Mendelian randomization finding is replicated in a separate dataset, there is still intrinsic uncertainty in the instrumental variable assumptions, meaning that uncertainty in a causal conclusion remains.”

In the paper

“That said, results from Mendelian randomization investigations have often been shown to qualitatively agree with the results from randomized trials, suggesting that a causal interpretation for Mendelian randomization findings is often reasonable 121 .”

What the sources say

• Not checked yetcites [121]

In the paper

“The appropriate degree of caution will depend on the plausibility of the instrumental variable assumptions, the concordance of estimates from different methods and different analytical approaches, the results from sensitivity and supplementary analyses, and so on.”

In the paper

“Power calculations are often performed post hoc, as sample sizes are rarely determined based on a proposed Mendelian randomization analysis.”

In the paper

“Power to detect a causal effect depends on the proportion of variance in the exposure explained by the genetic variants, proposed size of causal effect, sample size (for the genetic associations with the outcome), and (with a binary outcome) proportion of individuals with an outcome event.”

In the paper

“Other important information to report include the first-stage R 2 statistic (when the exposure is continuous), which is a measure of the variance in the exposure explained by the genetic variants, and (particularly in a one-sample setting) the related F statistic, which is a measure of instrument strength and can be used to judge the extent of weak instrument bias 157 .”

What the sources say

• Not checked yetcites [157]

In the paper

“Power calculations are more meaningful when performed prior to the analysis, and can guide investigators which exposure/outcome pairs to consider, and so focus on analyses that have a better chance of giving meaningful results.”

In the paper

“Other important information to report include the first-stage R 2 statistic (when the exposure is continuous), which is a measure of the variance in the exposure explained by the genetic variants, and (particularly in a one-sample setting) the related F statistic, which is a measure of instrument strength and can be used to judge the extent of weak instrument bias 157 .”

What the sources say

• Not checked yetcites [157]

In the paper

“Alternatives are forest plots, funnel plots, and radial plots – each of these assesses heterogeneity in the variant-specific causal estimates 156 .”

What the sources say

• Not checked yetcites [156]

In the paper

“Particularly with the advent of summarized data and the two-sample setting, performing a Mendelian randomization analysis has become more straightforward 30 .”

What the sources say

• Not checked yetcites [30] S Burgess, 2016, Combining information on multiple instrumental variables in Mendelian randomization: comparison of allele score and summarized data methods., Stat Med. source

In the paper

“Mendelian randomization investigations are worthwhile in providing an alternative line of aetiological evidence even though the instrumental variable assumptions can never be proved beyond all doubt 96, 97 .”

What the sources say

• Not checked yetcites [96] S Burgess, 2015, Network Mendelian randomization: using genetic variants as instrumental variables to investigate mediation in causal pathways., Int J Epidemiol. source
• Not checked yetcites [97] J Bowden, 2018, Improving the visualization, interpretation and analysis of two-sample summary data Mendelian randomization via the Radial plot and Radial regression., Int J Epidemiol. source

In the paper

“The difficulty is not in performing a Mendelian randomization analysis, but rather in performing a credible analysis 167 and providing a reasoned interpretation 164 .”

What the sources say

• Not checked yetcites [167]
• Not checked yetcites [164]

In the paper

“But even when a Mendelian randomization investigation is performed primarily to assess the causal role of an exposure, causal estimates can still be useful, for example to assess heterogeneity in estimates from different variants as a test of instrument validity, or to compare results from different analysis methods as an assessment of robustness.”

In the paper

“The estimate from a Mendelian randomization investigation is therefore better interpreted as a test statistic for a causal hypothesis and an indicator of the direction of the effect, rather than the estimated impact of a well-defined intervention at a specific point in time.”

In the paper

“These elements are necessary for the reader to judge the reliability of a Mendelian randomization investigation.”