The DOI matches its Crossref record; 16 of the 18 references it cites were found in a registry or an index; none of the 16 sources its claims cite is checked for support yet.
influence.ME: Tools for Detecting Influential Data in Mixed Effects Models
Open accessAbstract
influence.ME provides tools for detecting influential data in mixed effects models.The application of these models has become common practice, but the development of diagnostic tools has lagged behind.influence.ME calculates standardized measures of influential data for the point estimates of generalized mixed effects models, such as DFBETAS, Cook's distance, as well as percentile change and a test for changing levels of significance.influence.ME calculates these measures of influence while accounting for the nesting structure of the data.The package and measures of influential data are introduced, a practical example is given, and strategies for dealing with influential data are suggested.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
The paper introduces influence.ME, an R package for detecting influential cases in generalized mixed effects models estimated with lme4, providing DFBETAS, Cook’s distance, percentile change, and a test for changing significance while accounting for nesting. In a 23-school example, deleting school 7472 changed the school-level structure effect from non-significant to significant.
Key findings
- influence.ME calculates standardized influence measures for mixed effects models: DFBETAS, Cook’s distance, percentile change, and a significance-change test (`sigtest`), plus plotting and `exclude.influence` for deleting identified cases.
- In the example with 519 students in 23 schools, the original model estimated the structure effect as -2.0950 (SE 1.3237, t = -1.583, not significant) and the homework effect as 2.3938 (SE 0.2771, t = 8.640).
- The DFBETAS cut-off was 2/√23 = .41; school 7472 had a structure DFBETAS of 1.254990963, exceeding the cut-off. The Cook’s distance cut-off was 4/23 = .17; school 7472 had a Cook’s distance of 1.575002.
- After excluding school 7472, the model used 496 observations in 22 groups and estimated the structure effect as -3.8949 (SE 1.4342, t = -2.716), now statistically significant. For lower-level observations, the Cook’s distance cut-off was 4/519 = .0077; students 88 and 89 exceeded it for the structure estimate, but `sigtest` showed no significance changes.
Methods
- The paper describes the influence.ME package and demonstrates it with the `school23` data, a subset of NELS-88 containing 519 students nested in 23 schools, using a linear mixed model: `math ~ homework + structure + (1 | school.ID)` estimated with lme4.
- The `influence` function iteratively deletes each higher-level group, or with `obs=TRUE` each lower-level observation, and re-estimates the model; the resulting `estex` object is passed to `dfbetas`, `cooks.distance`, `pchange`, `sigtest`, and plotting functions.
- DFBETAS measures change in a single parameter standardized by the excluding-case standard error; Cook’s distance summarizes change across selected parameters; percentile change reports percentage change; `sigtest` compares test statistics after deletion to a user-specified value.
Limitations
- Stated: the leverage measure was not developed for generalized mixed effects models, so the package emphasizes influence on point estimates; the Langford and Lewis `delete=FALSE` procedure does not work with factor variables; the `influence` function can evaluate higher-level cases or single observations but not both at once; lower-level detection is often infeasible with many lower-level observations; the procedure can be computationally intensive; no cut-off is provided for percentile change.
- Evident: the worked example assumes math, homework, and structure are correctly measured at the interval level, and the paper does not report simulation or benchmark comparisons of the package’s performance.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
41 No basis given · 24 Own interpretation · 16 Not checked yet · 12 Own result
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“influence.ME calculates these measures of influence while accounting for the nesting structure of the data.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Abstract
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“We introduce influence.ME (Nieuwenhuis, Pelzer, and te Grotenhuis, 2012), an R-package that provides tools for detecting influential cases in mixed effects regression models estimated with lme4 (Bates and Maechler, 2010).”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Abstract
What the sources say
- Not checked yetcites Nieuwenhuis, Pelzer, and te Grotenhuis, 2012
- Not checked yetcites Bates and Maechler, 2010
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Based on these estimates, we may conclude that students spending more time on their math homework score better on a math test.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Example: students in 23 schools
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“It does, however, provide a basic test for detecting whether single cases change the level of significance of an estimate, and therefore the ability to make inferences from the estimated model.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data in Mixed Effects Models
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The influence.ME package introduced here contributes to these existing options, by providing several measures of influential data for generalized mixed effects models.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data in Mixed Effects Models
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Although influence.ME cannot provide the leverage measure to detect this, it provides a test for changes in the statistical significance of the fixed parameters in the mixed effects model.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Test for changes in significance
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The DFBETAS for the structure variable clearly exceeds the cut-off value of .41.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Calculating measures of influence
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Regarding class structure, however, we do not find a statistically significant association with math test scores.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Example: students in 23 schools
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Using the sigtest function, however, showed that the deletion of none of the students from the data affected the level of significance of the structure variable, nor of any of the other variables in the model.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Measuring the influence of lower-level observations
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Also, the estimates of the homework variable changes substantially with the deletion of several schools, as indicated by the high values of DF-BETAS.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Calculating measures of influence
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“The analysis of residuals cannot be used for the detection of influential cases (Crawley, 2007).”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data
What the sources say
- Not checked yetcites Crawley, 2007
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“For instance, Van der Meer, te Grotenhuis, and Pelzer (2010) were unable to find any country-level comparative studies involving more than 54 countries.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Abstract
What the sources say
- Not checked yetcites Van der Meer, te Grotenhuis, and Pelzer (2010)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“With such a relatively low number of countries, a single country can easily be overly influential on the parameter estimates of one or more of the country-level variables.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Abstract
plainconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As a result, these existing procedures fail to detect that higher-level cases may be influential on estimates of variables measured at specifically that level.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Abstract
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: Most existing procedures for detecting influential data do not account for the nesting structure of the data.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This runs against distributional assumptions associated with regression models, and as a result limits the validity and generalizability of regression models in which influential cases are present.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In social science applications of mixed models the testing for influential data is especially important, since these models are frequently based on large numbers of observations at the individual level while the number of higher level groups is relatively small.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Abstract
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The simple inclusion or exclusion of such a single case may then lead to substantially different regression estimates.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“However, an influential case is not always an outlier.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“Langford and Lewis (1998) developed a procedure in which the mixed effects model is modified to neutralize the group's influence on the higher-level estimate, while at the same time allowing the lower-level observations nested within that group to help estimate the effects of the lowerlevel predictors in the model.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data in Mixed Effects Models
What the sources say
- Not checked yetcites Langford and Lewis (1998)
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“As discussed above, even when cases are not influential on the point estimates (BETAS) of the regression model, cases can still influence the standard errors of these estimates.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Test for changes in significance
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: unclear — It is unclear why the paper says it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The reason for this is that the amount of influence a case exerts on the regression slope is not only determined by how well its (observed) score is fitted by the specified regression model, but also by its score(s) on the independent variable(s).”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Single cases, however, can also bias the confidence intervals of these estimates.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As a result of the specific modification of the model-specification, this specific procedure suggested by Langford and Lewis (1998) does not work when factor-variables are used in the regression model.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data in Mixed Effects Models
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Inferences made to the population from models in which such cases are present may be incorrect.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, if a case has very extreme scores on the independent variable(s) but is fitted very well by a regression model, and if this case has a low deleted (standardized) residual, this case is not necessarily overly influencing the outcomes of the regression model.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Just as influential cases are not necessarily outliers, outliers are not necessarily influential cases.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“Thus, adding important variables to the model may solve the problem of influential data.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Dealing with Influential Data
What the sources say
- Not checked yetcites Lieberman (2005)
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The observation that high values of DFBETAS were found for the homework variable, suggests that substantial differences between these schools exist in terms of how much time students spend on average on their homework.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Calculating measures of influence
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“So, now these model outcomes indicate that higher levels of class structure indeed are associated with lower math test scores, even when controlled for the students' homework efforts.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Dealing with Influential Data
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Nevertheless, if additional variables in the model can account for the fact that an observation has extreme scores on one or more variables, the case may no longer be an influential one.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Dealing with Influential Data
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- It may state two separate claims as one. 15
- Probably not a substantial claim. 56
- The quote only partly states this claim. 2
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“influence.ME calculates standardized measures of influential data for the point estimates of generalized mixed effects models, such as DFBETAS, Cook's distance, as well as percentile change and a test for changing levels of significance.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“influence.ME provides tools for detecting influential data in mixed effects models.”
- Also stated in: § Additional information · plain
“influence.ME provides tools for detecting influential data in mixed effects models.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
- Rests on an assumption: The application of mixed effects models has become common practice, but the development of diagnostic tools lags behind.
- Also stated in: § Additional information · plain
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In addition, we introduce the measure of percentile change and a test for changing levels of significance of the fixed parameters.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In influence.ME, values for Cook's distance in mixed effects models are calculated using the function cooks.distance,which takes the object returned from influence as input.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Cook's Distance
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“influence.ME offers four such measures, which are detailed in this section.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > The Outcome Measures
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one. The quote only partly states this claim.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Detecting influence of lower-level observations is supported for applications in various disciplines where mixed effects models are typically applied to only a limited number of observations per group.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data in Mixed Effects Models
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Consequently, the current installment of influence.ME emphasizes detecting the influence of cases on the point estimates of generalized mixed effect models.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data in Mixed Effects Models
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The influence.ME package contains the school23 data.frame, that provides information on the performance of 519 students in 23 schools.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Example: students in 23 schools
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“influence.ME provides this functionality, which is accessed by specifying delete=FALSE as an option to the influence function.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data in Mixed Effects Models
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“The plot function takes the output of the influence function to create a dotplot of a selected measure of influence (cf.Sarkar, 2008).”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Plots
What the sources say
- Not checked yetcites Sarkar, 2008
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Finally, influence.ME also allows for detecting the influence of lower-level cases in the mixed effects model.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data in Mixed Effects Models
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The sigtest function tests whether excluding the influence of a single case changes the statistical significance of any of the variables in the model.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Test for changes in significance
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The returned object of class”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data in Mixed Effects Models
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“All four measures of influence discussed above, can also be plotted.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Plots
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The influence function described above returns an object with information on how much the parameter estimates in a mixed effects model change, after the (influence of) observations of higher-level groups and their individual-level observations were removed from it iteratively.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > The Outcome Measures
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Student's class and school are equivalent in this data, since only one class per school is available.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Example: students in 23 schools
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In influence.ME, values for DFBETAS in mixed effects models can be calculated using the function dfbetas, which takes the object returned from influence as input.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > DFBETAS
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The default option of dfbetas is to calculate DFBETAS for estimates of all fixed effects in the model.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > DFBETAS
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The influence function in the influence.ME package performs this procedure automatically, and returns an object containing information on the parameter estimates excluding the influence of each higher level group separately.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data in Mixed Effects Models
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“This test of significance is based on the test statistic provided by the lme4 package.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Test for changes in significance
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The influence function can either determine the influence of higher-level cases, or of single observations, but not both at the same time.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data in Mixed Effects Models
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In the case of our example, the results for Cook's Distance and the results of this test for changing levels of significance both indicate that school number 7472 overly influences the regression outcomes regarding the school-level structure variable.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Testing for Changes in Statistical Significance (sigtest)
hedgeddirectSingle case ·- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Based on Figure 2, it is clear that both the structure and the homework variables are highly susceptible to the influence of single schools.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Calculating measures of influence
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The estimate for class structure in this model seems to be influenced most strongly by observations in school number 7472.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Calculating measures of influence
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“More importantly, though, the negative effect of the structure variable now is statistically significant, whereas it was not in the original model.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Dealing with Influential Data
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one. The quote only partly states this claim.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In this plot, it is clear that one single school represented in the lower-left corner of the graph seems to be an outlier, and -more importantly -the non-linear curve shown in this graph clearly indicates this single school with class structure level of 2 may overly influence a linear regression line estimated based on these data.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Visual Examination
boostedconditionalSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“These results are not shown here, but indicated that the deletion of none of the schools changed the level of significance of the homework variable.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Testing for Changes in Statistical Significance (sigtest)
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The output is not shown here, but the reader can verify that students with numbers 88 and 89 exert too much influence on the estimate of the structure variable.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Measuring the influence of lower-level observations
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Testing for influential cases in mixed effects regression models is important, because influential data negatively influence the statistical fit and generalizability of the model.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Abstract
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The application of mixed effects regression models has become common practice in the field of social sciences.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, if a single case has extremely high or low scores on the dependent variable relative to its expected value -given other variables in the model, one or more of the independent variables, or both -this case may overly influence the regression parameters by 'pulling' the estimated regression line towards itself.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
- Rests on an assumption: All cases used to estimate a regression model exert some level of influence on the regression parameters.
In the paper
“As used in the social sciences, mixed effects regression models take into account that observations on individual respondents are nested within higherlevel groups such as schools, classrooms, states, and countries (Snijders and Bosker, 1999), and are often referred to as multilevel regression models.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Abstract
What the sources say
- Not checked yetcites Snijders and Bosker, 1999
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“These plots, however, are not available for mixed effects models.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data in Mixed Effects Models
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
None of the sources cited for this claim have been checked yet.
In the paper
“Values exceeding this cut-off value are regarded as overly influencing the regression outcomes for that specific estimate.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > DFBETAS
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Regarding mixed models, this relates to the influence a higher-level unit has on the parameter estimate.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > DFBETAS
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“A higher leverage means more extreme scores on the independent variable(s), and a greater potential of overly influencing the regression outcomes.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data
plainconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In detecting influential data in mixed effects models, the key focus is on changes in the estimates of variables measured at the group-level.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data in Mixed Effects Models
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The limitation is that, unfortunately, as far as we are aware, the measure of leverage was not developed for generalized mixed effects models.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data in Mixed Effects Models
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Cook's distance provides a summary measure for the influence a higher level unit exerts on all parameter estimates simultaneously, or a selection thereof.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Cook's Distance
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“To apply the logic of detecting influential data to generalized mixed effects models, one has to measure the influence of a particular higher level group on the estimates of a predictor measured at that level.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data in Mixed Effects Models
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“However, most mixed effects regression models estimate the effects of both lower-level and higher-level variables simultaneously.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data in Mixed Effects Models
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Since the procedure of the influence function entails re-estimating mixed effects models several times, this can be computationally intensive.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data in Mixed Effects Models
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“DFBETAS is a standardized measure that indicates the level of influence observations have on single parameter estimates (Fox, 2002).”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > DFBETAS
What the sources say
- Not checked yetcites Fox, 2002
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Since neither outliers, nor cases with a high leverage, are necessarily influential, a different procedure is required for detecting influential cases.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“As a rule of thumb, a cut-off value is given for DFBETAS (Belsley et al., 1980): 2/ √ n in which n, the number of observations, refers to the number of groups in the grouping factor under evaluation (and not to the number of observations nested within the group under investigation).”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > DFBETAS
What the sources say
- Not checked yetcites Belsley et al., 1980
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“The LMERConvenienceFunctions package provides model criticism plots, including the density of the model residuals and the fitted values versus the standardized residuals (Tremblay, 2012).”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data in Mixed Effects Models
What the sources say
- Not checked yetcites Tremblay, 2012
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“First, DFBETAS is a standardized measure of the absolute difference between the estimate with a particular case included and the estimate without that particular case (Belsley, Kuh, and Welsch, 1980).”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data
What the sources say
- Not checked yetcites Belsley, Kuh, and Welsch, 1980
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Cases with high leverage but a low deleted residual compress standard errors, while cases with low leverage and a high deleted residual inflate standard errors.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one.
In the paper
“Second, Cook's distance provides an overall measurement of the change in all parameter estimates, or a selection thereof (Cook, 1977).”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data
What the sources say
- Not checked yetcites Cook, 1977
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The basic rationale behind measuring influential cases is based on the principle that when single cases are iteratively omitted from the data, models based on these data should not produce substantially different estimates.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As a final note, it is pointed out that if Cook's distance is calculated based on a single parameter, the Cook's distance equals the squared value of DFBE-TAS for that parameter.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Cook's Distance
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The nature of this statistic varies between different distributional families in the generalized mixed effects models.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Test for changes in significance
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The base R installation provides various plots for regression models, including but not limited to plots showing residuals versus the fitted scores, Cook's distances, and the leverage versus the deleted residuals.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data in Mixed Effects Models
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“However, while this package works with the lme4 package, it only is applicable for linear mixed effects models.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data in Mixed Effects Models
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“These data are a subset of the NELS-88 data (National Education Longitudinal Study of 1988).”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Example: students in 23 schools
What the sources say
- Not checked yetcites NELS-88
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“No cut-off value is provided, for determining what percent change in parameter estimate is considered too large will primarily depend on the goal for which the model was estimated and, more specifically, the nature of the hypotheses that are tested.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Percentile Change
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“If the model parameters change substantially after a single case is excluded, this case may be regarded as too influential.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In social science applications of mixed effects models, with a great number of lower-level observations nested in a limited number of groups, this will not always be feasible.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data in Mixed Effects Models
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“influence.ME calculates standardized measures of influential data such as DFBETAS and Cook's distance, as well as percentile change and a test for changing in statistical significance of fixed parameter estimates.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Additional information
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
- Rests on an assumption: The application of mixed effects models has become common practice, but the development of diagnostic tools lags behind.
In the paper
“The cut-off value of DFBETAS equals 2/ √ n (Belsley et al., 1980), which in this case equals 2/ √ 23 = .41.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Calculating measures of influence
What the sources say
- Not checked yetcites Belsley et al., 1980
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“He argues that the presence of outliers may indicate that one or more important variables were omitted from the model.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Dealing with Influential Data
What the sources say
- Not checked yetcites Lieberman (2005)
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Cases can influence the point estimates without affecting their level of significance, or affect the level of significance without overly affecting the point estimate itself.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Testing for Changes in Statistical Significance (sigtest)
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: unclear — It is unclear why the paper says it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We discussed above that an influential case is not necessarily an outlier in a regression model.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Dealing with Influential Data
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
The paper relies on this without support.
Diagnostic tools to evaluate fitted mixed effects models lag behind their increasing popularity.
“Despite these models' increasing popularity, diagnostic tools to evaluate fitted models lag behind.”
The paper relies on this without support.
Most existing procedures for detecting influential data do not account for the nesting structure of the data.
“However, most existing procedures do not account for the nesting structure of the data.”
The paper relies on this without support.
All cases used to estimate a regression model exert some level of influence on the regression parameters.
“All cases used to estimate a regression model exert some level of influence on the regression parameters.”
The authors state this as an assumption.
The math, homework, and structure variables were correctly measured at the interval level.
“For the purpose of our example, we assume here that the math, homework, and structure variables were correctly measured at the interval level.”
The paper relies on this without support.
The application of mixed effects models has become common practice, but the development of diagnostic tools lags behind.
“The application of these models has become common practice, but the development of diagnostic tools lag behind.”
The paper relies on this without support.
Overly influential data are a problem especially encountered in models based on a limited number of cases.
“Since overly influential data are a problem especially encountered in models based on a limited number of cases, a straightforward remedy would be to observe more cases in the population of interest.”
The paper relies on this without support.
If data are used to estimate too complex models, or if parameterization is incorrect, influential cases are more likely to occur.
“If the data are used to estimate too complex models, or if parameterization is incorrect, influential cases are more likely to occur.”
The paper relies on this without support.
In social science applications of mixed effects models, higher-level groups are often readily accessible cases such as schools and countries, making it possible to obtain additional measurements to remedy influential data.
“However, in social science applications of mixed effects models, the higher-level groups are often readily accessible cases such as schools and countries.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“We screened for outliers within the multilevel modeling framework using the R package influence.ME (Nieuwenhuis, Pelzer, & Grotenhuis, 2012) and found no evidence of influential outlying data (all Cook’s distance < 1; Tabachnick & Fidell, 2007).”
From Enhancing social connectedness in anxiety and depression through amplification of positivity: Preliminary treatment outcomes and process of change (2020) · cites this paper as Nieuwenhuis, Pelzer, & Grotenhuis, 2012
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Cook's Distance
“In addition, the user can specify sort=TRUE to have the values for Cook's distance returned in descending order.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Calculating measures of influence
“[,1] 24371 6.825871e-06 72292 1.524927e-05 ... 54344 2.256612e-01 7829 3.081222e-01 7472 1.575002e+00”
Citations
Cites
Regression Modeling Strategies
Harrell, F. E. (2001) · Springer series in statistics · doi:10.1007/978-1-4757-3462-1 · Cited by 9,270
Regression Diagnostics: Identifying Influential Data and Sources of Collinearity
Belsley, D. A., Kuh, E., & Welsch, R. E. (1980) · Wiley Series in Probability and Statistics · doi:10.1002/0471725153 · Cited by 7,297
Harrell (2015) · Springer Series in Statistics · doi:10.1007/978-3-319-19425-7 · Cited by 6,878
Regression Diagnostics -- Identifying Influential Data and Sources of Collinearity
Beaumont, C. D., Belsley, D. A., Kuh, E., & Welsch, R. E. (1981) · Journal of the Operational Research Society · doi:10.2307/2581267 · Cited by 6,362
Crawley, M. J. (2007) · Medical Entomology and Zoology · Cited by 5,431
Introducing Multilevel Modeling
Kreft, I., & de Leeuw, J. (1998) · doi:10.4135/9781849209366 · Cited by 2,997
Detection of Influential Observation in Linear Regression
Cook, R. D. (2000) · Technometrics · doi:10.2307/1271434 · Cited by 2,277
Sarkar, D. (2008) · Springer eBooks · doi:10.1007/978-0-387-75969-2 · Cited by 1,792
Nested Analysis as a Mixed-Method Strategy for Comparative Research
LIEBERMAN, E. S. (2005) · American Political Science Review · doi:10.1017/s0003055405051762 · Cited by 1,515 · cited 2×
“As a result of the specific modification of the model-specification, this specific procedure suggested by Langford and Lewis (1998) does not work when factor-variables are used in the regression model.”
MethodRegression Diagnostics: Identifying Influential Data and Sources of Collinearity.
Patterson, H. D., Belsley, D. A., Kuh, E., & Welsch, R. E. (1981) · Biometrics · doi:10.2307/2530185 · Cited by 1,370
Research works printed but not linked: 4
- R. Cook. Detection of influential observations in lin- ear regression. Technometrics, 19(1):15-18, 1977.
- M. J. Crawley. The R Book. Wiley, 2007.
- I. Kreft and J. De Leeuw. Introducing Multilevel Mod- eling. Sage Publications, 1998.
- T. Van der Meer, M. te Grotenhuis, and B. Pelzer. In- fluential cases in multilevel modeling. A method- ological comment. American Sociological Review, 75: 173-178, 2010. [p38, 40, 46]
Cited by
Sorted by citations
Fitting Linear Mixed-Effects Models Using lme4
Bates, D., Mächler, M., Bolker, B., & Walker, S. (2015) · Journal of Statistical Software · doi:10.18637/jss.v067.i01 · Cited by 86,835
Linear models and linear mixed effects models in R with linguistic applicationsPreprint
Winter, B. (2013) · arXiv.org · doi:10.48550/arxiv.1308.5499 · Cited by 658
Gotham, K., Brunwasser, S. M., & Lord, C. (2015) · Journal of the American Academy of Child & Adolescent Psychiatry · doi:10.1016/j.jaac.2015.02.005 · Cited by 310
Ten-month-old infants infer the value of goals from the costs of actions
Liu, S., Ullman, T. D., Tenenbaum, J. B., & Spelke, E. S. (2017) · Science · doi:10.1126/science.aag2132 · Cited by 230
Palma, A. D., Kuhlmann, M. T., Roberts, S. P. M., Potts, S. G., Börger, L., Hudson, L. N., Lysenko, I., Newbold, T., & Purvis, A. (2015) · Journal of Applied Ecology · doi:10.1111/1365-2664.12524 · Cited by 186
Metadata
- Authors
- Nieuwenhuis, R., Grotenhuis, M., & Pelzer, B.
- Venue
- The R Journal
- Publisher
- The R Foundation
- Published
- 2012
- Volume
- 4
- Issue
- 2
- Pages
- 38
- ISSN
- 2073-4859 (ISSN-L)
- OpenAlex
- W1549371580
- Semantic Scholar
- c6d41ed109eb79b026ba2d1fb769b6ebf525ebd4
- Cited by
- 442
- References
- 18
- Citations by year
- 2026: 29 · 2025: 36 · 2024: 45 · 2023: 38 · 2022: 64 · 2021: 63 · 2020: 33 · 2019: 28 · 2018: 24 · 2017: 26 · 2016: 22 · 2015: 17 · 2014: 12 · 2013: 5
- FWCI
- 10.39
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- The R Journal (publishedVersion, cc-by)
- Lincoln (University of Nebraska) (submittedVersion, cc-by)
- Radboud Repository (Radboud University) (submittedVersion)
- University of Twente Research Information (submittedVersion)
- Radboud Repository (Radboud University) (submittedVersion, other-oa)
- University of Twente Research Information (submittedVersion, other-oa)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Statistical Methods and Models
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Physical Sciences › Mathematics › Statistics and Probability › Data Analysis with R
- Keywords
- mixed models, generalized linear mixed models, Cook's distance, nested data, influence diagnostics, R package
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-09, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-09 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
