influence.ME: Tools for Detecting Influential Data in Mixed Effects Models

May combine two claims.

influence.ME calculates standardized measures of influential data for the point estimates of generalized mixed effects models, including DFBETAS, Cook's distance, percentile change, and a test for changing levels of significance.

“influence.ME calculates standardized measures of influential data for the point estimates of generalized mixed effects models, such as DFBETAS, Cook's distance, as well as percentile change and a test for changing levels of significance.”

Classified as probably not a substantial claim.

influence.ME provides tools for detecting influential data in mixed effects models.

“influence.ME provides tools for detecting influential data in mixed effects models.”

• Also stated in: § Additional information · plain

  “influence.ME provides tools for detecting influential data in mixed effects models.”

• AssumptionRests on an assumption: The application of mixed effects models has become common practice, but the development of diagnostic tools lags behind.

Classified as probably not a substantial claim. May combine two claims.

The paper introduces the measure of percentile change and a test for changing levels of significance of the fixed parameters.

“In addition, we introduce the measure of percentile change and a test for changing levels of significance of the fixed parameters.”

Probably not a substantial claim: may not pass S3 (load-bearing).

In influence.ME, values for Cook's distance in mixed effects models are calculated using the function cooks.distance, which takes the object returned from influence as input.

“In influence.ME, values for Cook's distance in mixed effects models are calculated using the function cooks.distance,which takes the object returned from influence as input.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims. The quote only partly states this claim.

influence.ME offers four standardized measures of influence: DFBETAS, Cook's distance, percentile change, and a test for changes in significance.

“influence.ME offers four such measures, which are detailed in this section.”

Probably not a substantial claim: may not pass S4 (based).

Detecting influence of lower-level observations is supported for applications in various disciplines where mixed effects models are typically applied to only a limited number of observations per group.

“Detecting influence of lower-level observations is supported for applications in various disciplines where mixed effects models are typically applied to only a limited number of observations per group.”

Classified as probably not a substantial claim.

The current installment of influence.ME emphasizes detecting the influence of cases on the point estimates of generalized mixed effect models.

“Consequently, the current installment of influence.ME emphasizes detecting the influence of cases on the point estimates of generalized mixed effect models.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The school23 data.frame provides information on the performance of 519 students in 23 schools.

“The influence.ME package contains the school23 data.frame, that provides information on the performance of 519 students in 23 schools.”

Probably not a substantial claim: may not pass S3 (load-bearing).

influence.ME provides the Langford and Lewis (1998) functionality, accessed by specifying delete=FALSE as an option to the influence function.

“influence.ME provides this functionality, which is accessed by specifying delete=FALSE as an option to the influence function.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The plot function takes the output of the influence function to create a dotplot of a selected measure of influence.

“The plot function takes the output of the influence function to create a dotplot of a selected measure of influence (cf.Sarkar, 2008).”

• cites Sarkar, 2008Not checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

influence.ME allows for detecting the influence of lower-level cases in the mixed effects model.

“Finally, influence.ME also allows for detecting the influence of lower-level cases in the mixed effects model.”

Classified as probably not a substantial claim.

The sigtest function tests whether excluding the influence of a single case changes the statistical significance of any of the variables in the model.

“The sigtest function tests whether excluding the influence of a single case changes the statistical significance of any of the variables in the model.”

Classified as probably not a substantial claim.

The returned object of class estex can be passed to functions calculating standardized measures of influence such as DFBETAS and Cook's Distance.

“The returned object of class”

Probably not a substantial claim: may not pass S3 (load-bearing).

All four measures of influence discussed above can be plotted.

“All four measures of influence discussed above, can also be plotted.”

Probably not a substantial claim: may not pass S4 (based).

The influence function returns an object with information on how much the parameter estimates in a mixed effects model change after the observations of higher-level groups and their individual-level observations were removed iteratively.

“The influence function described above returns an object with information on how much the parameter estimates in a mixed effects model change, after the (influence of) observations of higher-level groups and their individual-level observations were removed from it iteratively.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Student's class and school are equivalent in the school23 data because only one class per school is available.

“Student's class and school are equivalent in this data, since only one class per school is available.”

Probably not a substantial claim: may not pass S3 (load-bearing).

In influence.ME, values for DFBETAS in mixed effects models can be calculated using the function dfbetas, which takes the object returned from influence as input.

“In influence.ME, values for DFBETAS in mixed effects models can be calculated using the function dfbetas, which takes the object returned from influence as input.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The default option of dfbetas is to calculate DFBETAS for estimates of all fixed effects in the model.

“The default option of dfbetas is to calculate DFBETAS for estimates of all fixed effects in the model.”

Classified as probably not a substantial claim. May combine two claims.

The influence function in influence.ME performs the procedure of deleting all observations nested within a single higher level group, re-estimating the regression model, and evaluating the change in estimated regression parameters automatically, returning an object with parameter estimates excluding the influence of each higher level group separately.

“The influence function in the influence.ME package performs this procedure automatically, and returns an object containing information on the parameter estimates excluding the influence of each higher level group separately.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The significance test in sigtest is based on the test statistic provided by the lme4 package.

“This test of significance is based on the test statistic provided by the lme4 package.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

The influence function can either determine the influence of higher-level cases, or of single observations, but not both at the same time.

“The influence function can either determine the influence of higher-level cases, or of single observations, but not both at the same time.”

May combine two claims.

Cook's distance and sigtest results both indicate that school 7472 overly influences the regression outcomes regarding the school-level structure variable.

“In the case of our example, the results for Cook's Distance and the results of this test for changing levels of significance both indicate that school number 7472 overly influences the regression outcomes regarding the school-level structure variable.”

May combine two claims.

Both the structure and homework variables are highly susceptible to the influence of single schools.

“Based on Figure 2, it is clear that both the structure and the homework variables are highly susceptible to the influence of single schools.”

Probably not a substantial claim: may not pass S3 (load-bearing).

For the structure variable, the estimate is influenced most strongly by observations in school 7472.

“The estimate for class structure in this model seems to be influenced most strongly by observations in school number 7472.”

May combine two claims. The quote only partly states this claim.

Deleting school 7472 from the model makes the structure variable's effect statistically significant, with the test statistic changing from -1.583 to -2.72.

“More importantly, though, the negative effect of the structure variable now is statistically significant, whereas it was not in the original model.”

May combine two claims.

Visual examination of aggregated school-level data suggests one school with class structure level 2 is an outlier and may overly influence the linear regression line.

“In this plot, it is clear that one single school represented in the lower-left corner of the graph seems to be an outlier, and -more importantly -the non-linear curve shown in this graph clearly indicates this single school with class structure level of 2 may overly influence a linear regression line estimated based on these data.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Deletion of none of the schools changed the level of significance of the homework variable.

“These results are not shown here, but indicated that the deletion of none of the schools changed the level of significance of the homework variable.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

Students 88 and 89 exert too much influence on the estimate of the structure variable.

“The output is not shown here, but the reader can verify that students with numbers 88 and 89 exert too much influence on the estimate of the structure variable.”

Classified as probably not a substantial claim.

Testing for influential cases in mixed effects regression models is important because influential data negatively influence the statistical fit and generalizability of the model.

“Testing for influential cases in mixed effects regression models is important, because influential data negatively influence the statistical fit and generalizability of the model.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

The application of mixed effects regression models has become common practice in the field of social sciences.

“The application of mixed effects regression models has become common practice in the field of social sciences.”

Probably not a substantial claim: may not pass S4 (based).

If a single case has extremely high or low scores on the dependent variable relative to its expected value, or on one or more independent variables, it may overly influence the regression parameters by pulling the estimated regression line towards itself.

“However, if a single case has extremely high or low scores on the dependent variable relative to its expected value -given other variables in the model, one or more of the independent variables, or both -this case may overly influence the regression parameters by 'pulling' the estimated regression line towards itself.”

• AssumptionRests on an assumption: All cases used to estimate a regression model exert some level of influence on the regression parameters.

Classified as probably not a substantial claim.

As used in the social sciences, mixed effects regression models take into account that observations on individual respondents are nested within higher-level groups such as schools, classrooms, states, and countries.

“As used in the social sciences, mixed effects regression models take into account that observations on individual respondents are nested within higherlevel groups such as schools, classrooms, states, and countries (Snijders and Bosker, 1999), and are often referred to as multilevel regression models.”

• cites Snijders and Bosker, 1999Not checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

These base R plots are not available for mixed effects models.

“These plots, however, are not available for mixed effects models.”

Classified as probably not a substantial claim.

Values exceeding the DFBETAS cut-off value are regarded as overly influencing the regression outcomes for that specific estimate.

“Values exceeding this cut-off value are regarded as overly influencing the regression outcomes for that specific estimate.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Regarding mixed models, DFBETAS relates to the influence a higher-level unit has on the parameter estimate.

“Regarding mixed models, this relates to the influence a higher-level unit has on the parameter estimate.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

A higher leverage means more extreme scores on the independent variables and a greater potential of overly influencing the regression outcomes.

“A higher leverage means more extreme scores on the independent variable(s), and a greater potential of overly influencing the regression outcomes.”

Probably not a substantial claim: may not pass S4 (based).

In detecting influential data in mixed effects models, the key focus is on changes in the estimates of variables measured at the group-level.

“In detecting influential data in mixed effects models, the key focus is on changes in the estimates of variables measured at the group-level.”

Probably not a substantial claim: may not pass S4 (based).

The measure of leverage was not developed for generalized mixed effects models.

“The limitation is that, unfortunately, as far as we are aware, the measure of leverage was not developed for generalized mixed effects models.”

Classified as probably not a substantial claim.

Cook's distance provides a summary measure for the influence a higher level unit exerts on all parameter estimates simultaneously, or a selection thereof.

“Cook's distance provides a summary measure for the influence a higher level unit exerts on all parameter estimates simultaneously, or a selection thereof.”

Classified as probably not a substantial claim.

To apply the logic of detecting influential data to generalized mixed effects models, one has to measure the influence of a particular higher level group on the estimates of a predictor measured at that level.

“To apply the logic of detecting influential data to generalized mixed effects models, one has to measure the influence of a particular higher level group on the estimates of a predictor measured at that level.”

Probably not a substantial claim: may not pass S4 (based).

Most mixed effects regression models estimate the effects of both lower-level and higher-level variables simultaneously.

“However, most mixed effects regression models estimate the effects of both lower-level and higher-level variables simultaneously.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Since the procedure of the influence function entails re-estimating mixed effects models several times, it can be computationally intensive.

“Since the procedure of the influence function entails re-estimating mixed effects models several times, this can be computationally intensive.”

Probably not a substantial claim: may not pass S3 (load-bearing).

DFBETAS is a standardized measure that indicates the level of influence observations have on single parameter estimates.

“DFBETAS is a standardized measure that indicates the level of influence observations have on single parameter estimates (Fox, 2002).”

• cites Fox, 2002Not checked yet

  Not checked in the free run.

May combine two claims.

Since neither outliers nor cases with high leverage are necessarily influential, a different procedure is required for detecting influential cases.

“Since neither outliers, nor cases with a high leverage, are necessarily influential, a different procedure is required for detecting influential cases.”

Probably not a substantial claim: may not pass S3 (load-bearing).

As a rule of thumb, a cut-off value for DFBETAS is 2/√n, where n is the number of groups in the grouping factor under evaluation.

“As a rule of thumb, a cut-off value is given for DFBETAS (Belsley et al., 1980): 2/ √ n in which n, the number of observations, refers to the number of groups in the grouping factor under evaluation (and not to the number of observations nested within the group under investigation).”

• cites Belsley et al., 1980Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

The LMERConvenienceFunctions package provides model criticism plots, including the density of the model residuals and the fitted values versus the standardized residuals.

“The LMERConvenienceFunctions package provides model criticism plots, including the density of the model residuals and the fitted values versus the standardized residuals (Tremblay, 2012).”

• cites Tremblay, 2012Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

DFBETAS is a standardized measure of the absolute difference between the estimate with a particular case included and the estimate without that particular case.

“First, DFBETAS is a standardized measure of the absolute difference between the estimate with a particular case included and the estimate without that particular case (Belsley, Kuh, and Welsch, 1980).”

• cites Belsley, Kuh, and Welsch, 1980Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

Cases with high leverage but a low deleted residual compress standard errors, while cases with low leverage and a high deleted residual inflate standard errors.

“Cases with high leverage but a low deleted residual compress standard errors, while cases with low leverage and a high deleted residual inflate standard errors.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Cook's distance provides an overall measurement of the change in all parameter estimates, or a selection thereof.

“Second, Cook's distance provides an overall measurement of the change in all parameter estimates, or a selection thereof (Cook, 1977).”

• cites Cook, 1977Not checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

The basic rationale behind measuring influential cases is that when single cases are iteratively omitted from the data, models based on these data should not produce substantially different estimates.

“The basic rationale behind measuring influential cases is based on the principle that when single cases are iteratively omitted from the data, models based on these data should not produce substantially different estimates.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

If Cook's distance is calculated based on a single parameter, the Cook's distance equals the squared value of DFBETAS for that parameter.

“As a final note, it is pointed out that if Cook's distance is calculated based on a single parameter, the Cook's distance equals the squared value of DFBE-TAS for that parameter.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The nature of the test statistic varies between different distributional families in generalized mixed effects models.

“The nature of this statistic varies between different distributional families in the generalized mixed effects models.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

The base R installation provides various plots for regression models, including residuals versus fitted scores, Cook's distances, and leverage versus deleted residuals.

“The base R installation provides various plots for regression models, including but not limited to plots showing residuals versus the fitted scores, Cook's distances, and the leverage versus the deleted residuals.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The LMERConvenienceFunctions package only is applicable for linear mixed effects models.

“However, while this package works with the lme4 package, it only is applicable for linear mixed effects models.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The school23 data are a subset of the NELS-88 data.

“These data are a subset of the NELS-88 data (National Education Longitudinal Study of 1988).”

• cites NELS-88Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

No cut-off value is provided for percentile change because determining what percent change is too large depends on the goal for which the model was estimated and the nature of the hypotheses tested.

“No cut-off value is provided, for determining what percent change in parameter estimate is considered too large will primarily depend on the goal for which the model was estimated and, more specifically, the nature of the hypotheses that are tested.”

Classified as probably not a substantial claim.

If the model parameters change substantially after a single case is excluded, this case may be regarded as too influential.

“If the model parameters change substantially after a single case is excluded, this case may be regarded as too influential.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

In social science applications of mixed effects models, with a great number of lower-level observations nested in a limited number of groups, detecting influence of lower-level observations will not always be feasible.

“In social science applications of mixed effects models, with a great number of lower-level observations nested in a limited number of groups, this will not always be feasible.”

Classified as probably not a substantial claim. May combine two claims.

influence.ME calculates standardized measures of influential data such as DFBETAS and Cook's distance, as well as percentile change and a test for changing statistical significance of fixed parameter estimates.

“influence.ME calculates standardized measures of influential data such as DFBETAS and Cook's distance, as well as percentile change and a test for changing in statistical significance of fixed parameter estimates.”

• AssumptionRests on an assumption: The application of mixed effects models has become common practice, but the development of diagnostic tools lags behind.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The cut-off value of DFBETAS equals 2/√n, which in this case equals .41.

“The cut-off value of DFBETAS equals 2/ √ n (Belsley et al., 1980), which in this case equals 2/ √ 23 = .41.”

• cites Belsley et al., 1980Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

The presence of outliers may indicate that one or more important variables were omitted from the model.

“He argues that the presence of outliers may indicate that one or more important variables were omitted from the model.”

• cites Lieberman (2005)Not checked yet

  Not checked in the free run.

May combine two claims.

Cases can influence point estimates without affecting their level of significance, or affect the level of significance without overly affecting the point estimate itself.

“Cases can influence the point estimates without affecting their level of significance, or affect the level of significance without overly affecting the point estimate itself.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

An influential case is not necessarily an outlier in a regression model.

“We discussed above that an influential case is not necessarily an outlier in a regression model.”