Misunderstanding analysis of covariance.

Controlling for differences inherent to group status is inappropriate.

“Controlling for differences inherent to group status is inappropriate22 and, therefore, we tested for group differences on our hypothesised variables without including established risk factors for psychosis (eg, IQ, ethnicity, and sex) or factors inherent to need-for-care status (eg, education, impaired functioning, anxiety, and depression) as covariates.14, 15, 23”

Controlling a third confounding variable can lead to Simpson's paradox, the diminution or enhancement of an association between two variables.

“This situation can lead to statistical anomalies including Simpon's paradox, which refers to the diminution or enhancement of an association between two variables (e.g. HRV and wellbeing) when a third confounding variable is “corrected” or “controlled” (e.g., age, sex, psychopathology) (Miller and Chapman, 2001; Tu et al., 2008).”

Adequate control of potential confounding is a methodological consideration in HRV research, especially when variables cannot be randomly allocated to groups.

“There are also methodological considerations when conducting HRV research, including adequate control of potential confounding, especially when certain variables cannot be randomly allocated to groups (Miller and Chapman, 2001).”

The inclusion of variables associated with both dialogue and outcome, i.e. confounds, means that dialogue-outcome relations are consistently underestimated.

“This is partly to avoid missing trends, but mainly because the inclusion of variables associated with both dialogue and outcome, i.e. the confounds, means that dialogue-outcome relations are consistently underestimated (Miller & Chapman, 2001).”

Analysis of covariance is suboptimal at adjusting for factors when groups differ significantly in their covariates.

“analysis of covariance is suboptimal at adjusting for factors when groups differ significantly in their covariates”

Literature suggests that biases may occur when group differences might differ by controlling variables, but the features important for classification are not associated with those variables.

“There is some literature101 that suggests such biases may occur when the differences in groups might differ by the controlling variables, but the features important for classification (i.e. in this case the behavioral measures) are not associated with those variables (i.e., here, age or gender).”

Similar concerns have been found when using parametric tests like ANCOVA in psychiatric research.

“Similar concerns have been found when using parametric tests like analysis of covariance (ANCOVA) in psychiatric research101.”

Age and gender were entered as covariates to reduce error variance, justified because they did not differ between groups.

“Age and gender were entered as covariates of no interest in order to reduce error variance”

Using ANCOVAs may create spurious group effects or remove an actual effect of interest.

“The use of ANCOVAs may also create spurious group effects or remove an actual effect of interest”