Nested by design: model fitting and interpretation in a mixed model era

Biological sources of interaction variance cannot disappear when a model omits interaction because total variance in the response is unaffected by model modification.

“However, biological sources of interaction variance cannot disappear because the total variance in the response values is unaffected by our modification to the model.”

For a nested design, it is impossible to tell whether variation among sites reflects average site differences, site-specific treatment effects, or both.

“We cannot decide if the between-site variation that we estimate is caused by average differences among sites (e.g.productivity differences) and/or by differences among sites in the effect of the treatment.”

The experimental design, whether nested or crossed, affects how results are interpreted.

“But the experimental design (either nested or crossed) affects the interpretation of the results.”

A two-way factorial design allows up to four biological variance components to be estimated.

“In a two-way factorial design (such as a classical two-way ANOVA scenario), there are four biological sources of variance that can potentially be estimated (Fig. 2).”

If the interaction term is removed from a crossed model, the interaction variance is not estimated by the model.

“If we decide to remove the interaction term from the model, because we consider it of little biological interest, then the interaction variance will not be estimated by the model.”

The conflation of main-effect and interaction variance underlies the standard caution against dropping main effects while interactions remain.

“The pooling of the main effect variance and the interaction variance is the reason for the well-appreciated warning against the removal of main effects in the presence of interactions (Aiken & West 1991).”

• cites Aiken & West 1991Not checked yet

  Not checked in the free run.

Removing the interaction term from a crossed model increases residual variance because interaction variance is pooled with residual.

“As a consequence, the removal of the interaction term will lead to an increase in the residual variance component, because the interaction variance will be pooled with the residual variance (Table 3).”

In nested data structures, interaction variance is pooled with the main effect variance of the nested factor.

“With nested data structures, the interaction variance is pooled with the main effect variance of the nested factor.”

• Also stated in: § Introduction · plain

  “Our main aim is to show that the key consequence of nesting is the pooling of the interaction variance with the main effect variance of the nested factor.”

Balanced sampling increases power to separate variance components.

“Balanced sampling increases the power to separate variance components and is therefore an important aim in data”

Whether data are nested or crossed is determined by experimental design, not by the coding of the statistical model.

“This difference between nested and crossed data is determined by the experimental design (thus by the nature of data sets) and not by the coding of the statistical model.”

In that model, the interaction estimate also absorbs the main effect variance of the omitted factor.

“The estimate for the interaction term no longer represents the interaction variance only but is conflated with the main effect variance of the removed factor (Table 3).”

Data can be nested by design when a crossed design would have been technically feasible and biologically relevant.

“Data can be nested by design in the sense that it would have been technically feasible and biologically relevant to collect the data in a crossed design.”

In mixed models, the central distinction between nested and crossed effects concerns how interaction variance is estimated and interpreted.

“The key difference between nested and crossed effects in mixed models is the estimation and interpretation of the interaction variance.”

Differences between natural nesting and nesting by design are important for interpreting variance explained by the nested factor.

“The differences between natural nesting and nesting by design will become important when it comes to the interpretation of the variance explained by the nested factor.”

Crossed designs are required to separate interaction variance from the main effect variance of the nested factor.

“Crossed designs are required to separate the two components.”

Nested effects can usually be fitted using crossed-effects syntax in mixed models if coding reflects implicit nesting.

“Nested effects can usually be fitted using the syntax for crossed effects in mixed models, provided that the coding reflects implicit nesting.”

• Also stated in: § Introduction · plain

  “With mixed-effects models, two categorical predictors can be fitted using the syntax for crossed effects even if the design is nested.”

When a crossed design is analysed with an interaction term but one main effect removed, the interaction term is difficult to interpret.

“When data from a crossed design are analysed in a model with an interaction term, but with one of the main effects removed, then the interaction term becomes very difficult to interpret.”

• cites Aiken & West 1991Not checked yet

  Not checked in the free run.

• cites Engqvist 2005Not checked yet

  Not checked in the free run.

In a nested design, the interaction effect cannot be separated from the main effect variance of the nested factor.

“Indeed, the interaction effect and the nested factor main effect variance are completely confounded.”

In a nested design, the estimate of between-site variance is inflated by interaction variance.

“The estimate that we obtain for the between-site variance is inflated by the interaction variance.”

Breeding designs for quantitative genetics are classic instances of nested and crossed experiments.

“Classical examples of nested and crossed experimental designs are the breeding designs employed for quantitative genetic analyses (Comstock & Robinson 1952).”

• cites Comstock & Robinson 1952Not checked yet

  Not checked in the free run.

Giving each level of a nested factor a unique label makes the nesting evident to software or collaborators without extra documentation.

“A main advantage of this way of coding is that it implicitly describes a nested data structure that a computer programme or a colleague would recognize the structure as nested without further explanations.”

In a crossed design, all four variance components can be estimated.

“In the crossed design that we describe above, it is possible to estimate all four variance components (Table 3).”

If two treatments are applied in a nested manner, the nested treatment is completely confounded with the higher-level treatment effect, making meaningful main effects impossible to estimate.

“This is because if we are applying two treatments (each with multiple levels) in a nested manner, the nested treatment is completely confounded with the higher-level treatment effect and it is impossible to estimate meaningful main effects.”

A nested design is not suited for separating main effect and interaction variance.

“In particular, a nested design is not suited for separating main effect and interaction variance.”

Nested effects are more easily fitted as random effects because this avoids potential pitfalls with choosing the wrong residual variance.

“In the context of this paper, it is important that nested effects are more easily fitted as random effects, because this avoids potential pitfalls with choosing the wrong residual variance (Gelman 2005).”

• cites Gelman 2005Not checked yet

  Not checked in the free run.

The interaction variance is pooled with the random-effect variance.

“The interaction variance is pooled with the random-effect variance.”

Between-group variance can be standardized by total phenotypic variance to give repeatability (an intraclass correlation), allowing comparison across studies.

“A critical quantity is the between-group variance that can be standardized by the total phenotypic variance to give the repeatability (a form of an intraclass correlation), such that it can be compared across studies (Nakagawa & Schielzeth 2010).”

• cites Nakagawa & Schielzeth 2010Not checked yet

  Not checked in the free run.

With only one observation per plot x data-level predictor combination, interaction variance is pooled with residual variance.

“With only one observation per plot 9 datalevel predictor combination, the plot 9 data-level predictor interaction variance is pooled with the residual variance.”

Strong main effects at the grouping level that are not explicitly modelled tend to produce bi- or multimodal distributions of group means that normal distributions do not adequately capture.

“Strong main effects at the grouping level (such as age classes, sexes, morphs) that are not explicitly modelled will tend to produce bi-or multimodal distributions of group means that are not adequately captured by normal distributions.”

The issue of degrees of freedom for nested fixed effects can be avoided by routinely fitting nested factors as random effects.

“This issue can be avoided by routinely fitting nested factors as random effects.”

If there are replications within block-treatment combinations, block x treatment interaction variance can be estimated.

“If there are replications within block-treatment combinations, it is possible to estimate the block 9 treatment interaction variance, that is, whether blocks differ in their treatment effects.”

Nested effects can still be fitted as fixed effects in a linear model.

“Although we stated that nested effects are typically (conceptually) random effects, they can still be fitted as fixed effects in a linear model (e.g.Quinn & Keough 2002;Gelman 2005;Kirk 2009).”

• cites Quinn & Keough 2002Not checked yet

  Not checked in the free run.

• cites Gelman 2005Not checked yet

  Not checked in the free run.

• cites Kirk 2009Not checked yet

  Not checked in the free run.

When random factors are fitted as fixed factors, inferences are limited to the particular sample of groups.

“In such cases, however, inferences will be limited only on the particular sample of groups.”

Significant group-level predictors tend to reduce the random-effect variance by explaining part of it and reducing unexplained group-level variance.

“Signifi-cant (important) group-level predictors will tend to reduce the random-effect variance, as they explain the part of it and hence reduce the unexplained group-level variance.”

In the typical case of normally distributed random effects, the group-level model is a normal distribution with mean zero and a group-level variance estimated from the data.

“In the typical case of normally distributed random effects, the group-level model consists of a normal distribution with a mean of zero and a group-level variance, which is estimated from the data.”

Unless the random-effect variance is very large, considering data-level predictors can still be useful because they can reduce residual variance.

“Unless the random-effect variance is very large, it might still be useful to consider data-level predictors, which can potentially reduce residual variance (Snijders & Bosker 2011; Fig. 3).”

• cites Snijders & Bosker 2011Not checked yet

  Not checked in the free run.

The genetic variance is inflated by genotype by environment interactions if the data are nested by design.

“The genetic variance is inflated by G 9 E if the data are nested by design.”

It is not possible to fit two nested fixed effects in a straightforward way as two crossed effects because such a model would be overparameterized.

“In this case, it is not possible to fit them in a straightforward way as two crossed effects, because such a model would be overparameterized.”

Nested factors are typically conceptually random factors.

“Nested factors are typically conceptually random factors (Quinn & Keough 2002).”

• cites Quinn & Keough 2002Not checked yet

  Not checked in the free run.

In a model with multiple grouping levels, a statement about a group-level predictor is ambiguous unless the level is specified.

“For example, if there are observations clustered in subjects that are nested in families, there are two grouping levels and a statement about a group-level predictor will be ambiguous.”

Outliers tend to impair the fit of the group-level model and thus the fit of the mixed model as a whole.

“Outliers (as may be produced by a class of 'others' that include a diverse group of features) will tend to impair the fit of the group-level model and thus the fit of the mixed model as a whole.”

For data-level predictors in a split-plot design, marginal effect and interaction variance can be estimated separately if there is replication of plot x data-level predictor combinations.

“whereas for the data-level predictors, it is possible to estimate the marginal effect and the interaction variance separately, if there is replication of plot 9 data-level predictor combinations.”

Confidence intervals for variance components in mixed models can be estimated by parametric bootstrapping.

“Confidence intervals for variance components in mixed models can be estimated, for example, by (parametric) bootstrapping (Faraway 2006;Nakagawa & Schielzeth 2010).”

• cites Faraway 2006Not checked yet

  Not checked in the free run.

• cites Nakagawa & Schielzeth 2010Not checked yet

  Not checked in the free run.

The estimators for random-effect variances allow important biological insight.

“This practice is unfortunate, because the estimators for the randomeffect variances allow important biological insight (Merlo et al. 2005a(Merlo et al. ,b,c, 2006)).”

• cites Merlo et al. 2005aNot checked yet

  Not checked in the free run.

• cites Merlo et al. 2005bNot checked yet

  Not checked in the free run.

• cites Merlo et al. 2005cNot checked yet

  Not checked in the free run.

The key difference between nested and crossed effects lies in the interpretation of model estimates, not in the model fitting itself.

“As we have described above, the key difference lies in the interpretation of the model estimates and not in the model fitting itself.”

• AssumptionRests on an assumption: Statistical software converts whatever syntax is used into design matrices, and these are blind to whether data collection was nested or crossed.

Traditional repeated-measures ANOVA does not allow missing values.

“Traditionally, data from such designs are analysed by repeated-measures ANOVAs, which unfortunately does not allow any missing values.”

Fitting nested effects as fixed effects is particularly useful when the number of groups is low, because estimating the group-level variance with mixed models is difficult.

“This will be particularly useful if the number of groups is low, which makes it difficult to estimate the grouplevel variance using mixed models.”

Nested effects can usually be fitted using the syntax for crossed effects if the coding reflects implicit nesting.

“Nested effects can usually be fitted using the syntax for crossed effects if the coding reflects implicit nesting.”

• AssumptionRests on an assumption: Statistical software converts whatever syntax is used into design matrices, and these are blind to whether data collection was nested or crossed.

In a split-plot design, the plot-by-treatment interaction variance is unavoidably merged into the main effect of the group-level predictor.

“The plot 9 treatment interaction variance is necessarily pooled with the main effect of the group-level predictors”

When fitting nested fixed effects, the degrees of freedom for estimating the standard error of the higher-level factor should be based on the number of levels of the nested factor, not on the total number of observations.

“The complication when fitting nested fixed effects lies in the fact that the degrees of freedom for estimating the standard error of the higher-level factor (treatment) should be based on the number of levels of the nested factor (block), not on the total number of observations (Gelman 2005).”

• cites Gelman 2005Not checked yet

  Not checked in the free run.

A nested design leaves fewer degrees of freedom to model all effects that are potentially of interest.

“A nested design simply leaves fewer 'degrees of freedom' to model all the effects that are potentially of interest.”