The DOI matches its Crossref record; 16 of the 18 references it cites were found in a registry or an index; none of the 16 sources its claims cite is checked for support yet.
influence.ME: Tools for Detecting Influential Data in Mixed Effects Models
Open accessAbstract
influence.ME provides tools for detecting influential data in mixed effects models.The application of these models has become common practice, but the development of diagnostic tools has lagged behind.influence.ME calculates standardized measures of influential data for the point estimates of generalized mixed effects models, such as DFBETAS, Cook's distance, as well as percentile change and a test for changing levels of significance.influence.ME calculates these measures of influence while accounting for the nesting structure of the data.The package and measures of influential data are introduced, a practical example is given, and strategies for dealing with influential data are suggested.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
The paper introduces influence.ME, an R package for detecting influential cases in generalized mixed effects models estimated with lme4, providing DFBETAS, Cook’s distance, percentile change, and a test for changing significance while accounting for nesting. In a 23-school example, deleting school 7472 changed the school-level structure effect from non-significant to significant.
Key findings
- influence.ME calculates standardized influence measures for mixed effects models: DFBETAS, Cook’s distance, percentile change, and a significance-change test (`sigtest`), plus plotting and `exclude.influence` for deleting identified cases.
- In the example with 519 students in 23 schools, the original model estimated the structure effect as -2.0950 (SE 1.3237, t = -1.583, not significant) and the homework effect as 2.3938 (SE 0.2771, t = 8.640).
- The DFBETAS cut-off was 2/√23 = .41; school 7472 had a structure DFBETAS of 1.254990963, exceeding the cut-off. The Cook’s distance cut-off was 4/23 = .17; school 7472 had a Cook’s distance of 1.575002.
- After excluding school 7472, the model used 496 observations in 22 groups and estimated the structure effect as -3.8949 (SE 1.4342, t = -2.716), now statistically significant. For lower-level observations, the Cook’s distance cut-off was 4/519 = .0077; students 88 and 89 exceeded it for the structure estimate, but `sigtest` showed no significance changes.
Methods
- The paper describes the influence.ME package and demonstrates it with the `school23` data, a subset of NELS-88 containing 519 students nested in 23 schools, using a linear mixed model: `math ~ homework + structure + (1 | school.ID)` estimated with lme4.
- The `influence` function iteratively deletes each higher-level group, or with `obs=TRUE` each lower-level observation, and re-estimates the model; the resulting `estex` object is passed to `dfbetas`, `cooks.distance`, `pchange`, `sigtest`, and plotting functions.
- DFBETAS measures change in a single parameter standardized by the excluding-case standard error; Cook’s distance summarizes change across selected parameters; percentile change reports percentage change; `sigtest` compares test statistics after deletion to a user-specified value.
Limitations
- Stated: the leverage measure was not developed for generalized mixed effects models, so the package emphasizes influence on point estimates; the Langford and Lewis `delete=FALSE` procedure does not work with factor variables; the `influence` function can evaluate higher-level cases or single observations but not both at once; lower-level detection is often infeasible with many lower-level observations; the procedure can be computationally intensive; no cut-off is provided for percentile change.
- Evident: the worked example assumes math, homework, and structure are correctly measured at the interval level, and the paper does not report simulation or benchmark comparisons of the package’s performance.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
41 No basis given · 24 Own interpretation · 16 Not checked yet · 12 Own result
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“influence.ME calculates these measures of influence while accounting for the nesting structure of the data.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Abstract
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“We introduce influence.ME (Nieuwenhuis, Pelzer, and te Grotenhuis, 2012), an R-package that provides tools for detecting influential cases in mixed effects regression models estimated with lme4 (Bates and Maechler, 2010).”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Abstract
What the sources say
- Not checked yetcites Nieuwenhuis, Pelzer, and te Grotenhuis, 2012
- Not checked yetcites Bates and Maechler, 2010
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Based on these estimates, we may conclude that students spending more time on their math homework score better on a math test.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Example: students in 23 schools
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“It does, however, provide a basic test for detecting whether single cases change the level of significance of an estimate, and therefore the ability to make inferences from the estimated model.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data in Mixed Effects Models
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The influence.ME package introduced here contributes to these existing options, by providing several measures of influential data for generalized mixed effects models.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data in Mixed Effects Models
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Although influence.ME cannot provide the leverage measure to detect this, it provides a test for changes in the statistical significance of the fixed parameters in the mixed effects model.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Test for changes in significance
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The DFBETAS for the structure variable clearly exceeds the cut-off value of .41.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Calculating measures of influence
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Regarding class structure, however, we do not find a statistically significant association with math test scores.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Example: students in 23 schools
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Using the sigtest function, however, showed that the deletion of none of the students from the data affected the level of significance of the structure variable, nor of any of the other variables in the model.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Measuring the influence of lower-level observations
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Also, the estimates of the homework variable changes substantially with the deletion of several schools, as indicated by the high values of DF-BETAS.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Calculating measures of influence
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- It may state two separate claims as one. 15
- Probably not a substantial claim. 56
- The quote only partly states this claim. 2
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“influence.ME calculates standardized measures of influential data for the point estimates of generalized mixed effects models, such as DFBETAS, Cook's distance, as well as percentile change and a test for changing levels of significance.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“influence.ME provides tools for detecting influential data in mixed effects models.”
- Also stated in: § Additional information · plain
“influence.ME provides tools for detecting influential data in mixed effects models.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
- Rests on an assumption: The application of mixed effects models has become common practice, but the development of diagnostic tools lags behind.
- Also stated in: § Additional information · plain
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In addition, we introduce the measure of percentile change and a test for changing levels of significance of the fixed parameters.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In influence.ME, values for Cook's distance in mixed effects models are calculated using the function cooks.distance,which takes the object returned from influence as input.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Cook's Distance
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“influence.ME offers four such measures, which are detailed in this section.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > The Outcome Measures
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one. The quote only partly states this claim.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Detecting influence of lower-level observations is supported for applications in various disciplines where mixed effects models are typically applied to only a limited number of observations per group.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data in Mixed Effects Models
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Consequently, the current installment of influence.ME emphasizes detecting the influence of cases on the point estimates of generalized mixed effect models.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data in Mixed Effects Models
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The influence.ME package contains the school23 data.frame, that provides information on the performance of 519 students in 23 schools.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Example: students in 23 schools
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“influence.ME provides this functionality, which is accessed by specifying delete=FALSE as an option to the influence function.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data in Mixed Effects Models
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“The plot function takes the output of the influence function to create a dotplot of a selected measure of influence (cf.Sarkar, 2008).”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Plots
What the sources say
- Not checked yetcites Sarkar, 2008
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Finally, influence.ME also allows for detecting the influence of lower-level cases in the mixed effects model.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data in Mixed Effects Models
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The sigtest function tests whether excluding the influence of a single case changes the statistical significance of any of the variables in the model.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Test for changes in significance
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The returned object of class”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data in Mixed Effects Models
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“All four measures of influence discussed above, can also be plotted.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Plots
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The influence function described above returns an object with information on how much the parameter estimates in a mixed effects model change, after the (influence of) observations of higher-level groups and their individual-level observations were removed from it iteratively.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > The Outcome Measures
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Student's class and school are equivalent in this data, since only one class per school is available.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Example: students in 23 schools
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In influence.ME, values for DFBETAS in mixed effects models can be calculated using the function dfbetas, which takes the object returned from influence as input.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > DFBETAS
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The default option of dfbetas is to calculate DFBETAS for estimates of all fixed effects in the model.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > DFBETAS
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The influence function in the influence.ME package performs this procedure automatically, and returns an object containing information on the parameter estimates excluding the influence of each higher level group separately.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data in Mixed Effects Models
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“This test of significance is based on the test statistic provided by the lme4 package.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Test for changes in significance
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The influence function can either determine the influence of higher-level cases, or of single observations, but not both at the same time.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data in Mixed Effects Models
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In the case of our example, the results for Cook's Distance and the results of this test for changing levels of significance both indicate that school number 7472 overly influences the regression outcomes regarding the school-level structure variable.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Testing for Changes in Statistical Significance (sigtest)
hedgeddirectSingle case ·- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Based on Figure 2, it is clear that both the structure and the homework variables are highly susceptible to the influence of single schools.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Calculating measures of influence
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The estimate for class structure in this model seems to be influenced most strongly by observations in school number 7472.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Calculating measures of influence
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“More importantly, though, the negative effect of the structure variable now is statistically significant, whereas it was not in the original model.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Dealing with Influential Data
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one. The quote only partly states this claim.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In this plot, it is clear that one single school represented in the lower-left corner of the graph seems to be an outlier, and -more importantly -the non-linear curve shown in this graph clearly indicates this single school with class structure level of 2 may overly influence a linear regression line estimated based on these data.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Visual Examination
boostedconditionalSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“These results are not shown here, but indicated that the deletion of none of the schools changed the level of significance of the homework variable.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Testing for Changes in Statistical Significance (sigtest)
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The output is not shown here, but the reader can verify that students with numbers 88 and 89 exert too much influence on the estimate of the structure variable.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Measuring the influence of lower-level observations
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Testing for influential cases in mixed effects regression models is important, because influential data negatively influence the statistical fit and generalizability of the model.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Abstract
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The application of mixed effects regression models has become common practice in the field of social sciences.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, if a single case has extremely high or low scores on the dependent variable relative to its expected value -given other variables in the model, one or more of the independent variables, or both -this case may overly influence the regression parameters by 'pulling' the estimated regression line towards itself.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
- Rests on an assumption: All cases used to estimate a regression model exert some level of influence on the regression parameters.
In the paper
“As used in the social sciences, mixed effects regression models take into account that observations on individual respondents are nested within higherlevel groups such as schools, classrooms, states, and countries (Snijders and Bosker, 1999), and are often referred to as multilevel regression models.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Abstract
What the sources say
- Not checked yetcites Snijders and Bosker, 1999
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“These plots, however, are not available for mixed effects models.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data in Mixed Effects Models
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
None of the sources cited for this claim have been checked yet.
In the paper
“Values exceeding this cut-off value are regarded as overly influencing the regression outcomes for that specific estimate.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > DFBETAS
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Regarding mixed models, this relates to the influence a higher-level unit has on the parameter estimate.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > DFBETAS
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“A higher leverage means more extreme scores on the independent variable(s), and a greater potential of overly influencing the regression outcomes.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data
plainconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In detecting influential data in mixed effects models, the key focus is on changes in the estimates of variables measured at the group-level.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data in Mixed Effects Models
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The limitation is that, unfortunately, as far as we are aware, the measure of leverage was not developed for generalized mixed effects models.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data in Mixed Effects Models
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Cook's distance provides a summary measure for the influence a higher level unit exerts on all parameter estimates simultaneously, or a selection thereof.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Cook's Distance
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“To apply the logic of detecting influential data to generalized mixed effects models, one has to measure the influence of a particular higher level group on the estimates of a predictor measured at that level.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data in Mixed Effects Models
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“However, most mixed effects regression models estimate the effects of both lower-level and higher-level variables simultaneously.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data in Mixed Effects Models
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Since the procedure of the influence function entails re-estimating mixed effects models several times, this can be computationally intensive.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data in Mixed Effects Models
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“DFBETAS is a standardized measure that indicates the level of influence observations have on single parameter estimates (Fox, 2002).”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > DFBETAS
What the sources say
- Not checked yetcites Fox, 2002
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Since neither outliers, nor cases with a high leverage, are necessarily influential, a different procedure is required for detecting influential cases.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“As a rule of thumb, a cut-off value is given for DFBETAS (Belsley et al., 1980): 2/ √ n in which n, the number of observations, refers to the number of groups in the grouping factor under evaluation (and not to the number of observations nested within the group under investigation).”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > DFBETAS
What the sources say
- Not checked yetcites Belsley et al., 1980
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“The LMERConvenienceFunctions package provides model criticism plots, including the density of the model residuals and the fitted values versus the standardized residuals (Tremblay, 2012).”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data in Mixed Effects Models
What the sources say
- Not checked yetcites Tremblay, 2012
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“First, DFBETAS is a standardized measure of the absolute difference between the estimate with a particular case included and the estimate without that particular case (Belsley, Kuh, and Welsch, 1980).”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data
What the sources say
- Not checked yetcites Belsley, Kuh, and Welsch, 1980
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Cases with high leverage but a low deleted residual compress standard errors, while cases with low leverage and a high deleted residual inflate standard errors.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one.
In the paper
“Second, Cook's distance provides an overall measurement of the change in all parameter estimates, or a selection thereof (Cook, 1977).”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data
What the sources say
- Not checked yetcites Cook, 1977
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The basic rationale behind measuring influential cases is based on the principle that when single cases are iteratively omitted from the data, models based on these data should not produce substantially different estimates.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As a final note, it is pointed out that if Cook's distance is calculated based on a single parameter, the Cook's distance equals the squared value of DFBE-TAS for that parameter.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Cook's Distance
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The nature of this statistic varies between different distributional families in the generalized mixed effects models.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Test for changes in significance
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The base R installation provides various plots for regression models, including but not limited to plots showing residuals versus the fitted scores, Cook's distances, and the leverage versus the deleted residuals.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data in Mixed Effects Models
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“However, while this package works with the lme4 package, it only is applicable for linear mixed effects models.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data in Mixed Effects Models
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“These data are a subset of the NELS-88 data (National Education Longitudinal Study of 1988).”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Example: students in 23 schools
What the sources say
- Not checked yetcites NELS-88
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“No cut-off value is provided, for determining what percent change in parameter estimate is considered too large will primarily depend on the goal for which the model was estimated and, more specifically, the nature of the hypotheses that are tested.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Percentile Change
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“If the model parameters change substantially after a single case is excluded, this case may be regarded as too influential.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In social science applications of mixed effects models, with a great number of lower-level observations nested in a limited number of groups, this will not always be feasible.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data in Mixed Effects Models
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“influence.ME calculates standardized measures of influential data such as DFBETAS and Cook's distance, as well as percentile change and a test for changing in statistical significance of fixed parameter estimates.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Additional information
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
- Rests on an assumption: The application of mixed effects models has become common practice, but the development of diagnostic tools lags behind.
In the paper
“The cut-off value of DFBETAS equals 2/ √ n (Belsley et al., 1980), which in this case equals 2/ √ 23 = .41.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Calculating measures of influence
What the sources say
- Not checked yetcites Belsley et al., 1980
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“He argues that the presence of outliers may indicate that one or more important variables were omitted from the model.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Dealing with Influential Data
What the sources say
- Not checked yetcites Lieberman (2005)
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Cases can influence the point estimates without affecting their level of significance, or affect the level of significance without overly affecting the point estimate itself.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Testing for Changes in Statistical Significance (sigtest)
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: unclear — It is unclear why the paper says it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We discussed above that an influential case is not necessarily an outlier in a regression model.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Dealing with Influential Data
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
The paper relies on this without support.
Diagnostic tools to evaluate fitted mixed effects models lag behind their increasing popularity.
“Despite these models' increasing popularity, diagnostic tools to evaluate fitted models lag behind.”
The paper relies on this without support.
Most existing procedures for detecting influential data do not account for the nesting structure of the data.
“However, most existing procedures do not account for the nesting structure of the data.”
The paper relies on this without support.
All cases used to estimate a regression model exert some level of influence on the regression parameters.
“All cases used to estimate a regression model exert some level of influence on the regression parameters.”
The authors state this as an assumption.
The math, homework, and structure variables were correctly measured at the interval level.
“For the purpose of our example, we assume here that the math, homework, and structure variables were correctly measured at the interval level.”
The paper relies on this without support.
The application of mixed effects models has become common practice, but the development of diagnostic tools lags behind.
“The application of these models has become common practice, but the development of diagnostic tools lag behind.”
The paper relies on this without support.
Overly influential data are a problem especially encountered in models based on a limited number of cases.
“Since overly influential data are a problem especially encountered in models based on a limited number of cases, a straightforward remedy would be to observe more cases in the population of interest.”
The paper relies on this without support.
If data are used to estimate too complex models, or if parameterization is incorrect, influential cases are more likely to occur.
“If the data are used to estimate too complex models, or if parameterization is incorrect, influential cases are more likely to occur.”
The paper relies on this without support.
In social science applications of mixed effects models, higher-level groups are often readily accessible cases such as schools and countries, making it possible to obtain additional measurements to remedy influential data.
“However, in social science applications of mixed effects models, the higher-level groups are often readily accessible cases such as schools and countries.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“We screened for outliers within the multilevel modeling framework using the R package influence.ME (Nieuwenhuis, Pelzer, & Grotenhuis, 2012) and found no evidence of influential outlying data (all Cook’s distance < 1; Tabachnick & Fidell, 2007).”
From Enhancing social connectedness in anxiety and depression through amplification of positivity: Preliminary treatment outcomes and process of change (2020) · cites this paper as Nieuwenhuis, Pelzer, & Grotenhuis, 2012
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Cook's Distance
“In addition, the user can specify sort=TRUE to have the values for Cook's distance returned in descending order.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Calculating measures of influence
“[,1] 24371 6.825871e-06 72292 1.524927e-05 ... 54344 2.256612e-01 7829 3.081222e-01 7472 1.575002e+00”
Citations
Cites
Regression Modeling Strategies
Harrell, F. E. (2001) · Springer series in statistics · doi:10.1007/978-1-4757-3462-1 · Cited by 9,270
Regression Diagnostics: Identifying Influential Data and Sources of Collinearity
Belsley, D. A., Kuh, E., & Welsch, R. E. (1980) · Wiley Series in Probability and Statistics · doi:10.1002/0471725153 · Cited by 7,297
Harrell (2015) · Springer Series in Statistics · doi:10.1007/978-3-319-19425-7 · Cited by 6,878
Regression Diagnostics -- Identifying Influential Data and Sources of Collinearity
Beaumont, C. D., Belsley, D. A., Kuh, E., & Welsch, R. E. (1981) · Journal of the Operational Research Society · doi:10.2307/2581267 · Cited by 6,362
Crawley, M. J. (2007) · Medical Entomology and Zoology · Cited by 5,431
Introducing Multilevel Modeling
Kreft, I., & de Leeuw, J. (1998) · doi:10.4135/9781849209366 · Cited by 2,997
Detection of Influential Observation in Linear Regression
Cook, R. D. (2000) · Technometrics · doi:10.2307/1271434 · Cited by 2,277
Sarkar, D. (2008) · Springer eBooks · doi:10.1007/978-0-387-75969-2 · Cited by 1,792
Nested Analysis as a Mixed-Method Strategy for Comparative Research
LIEBERMAN, E. S. (2005) · American Political Science Review · doi:10.1017/s0003055405051762 · Cited by 1,515 · cited 2×
“As a result of the specific modification of the model-specification, this specific procedure suggested by Langford and Lewis (1998) does not work when factor-variables are used in the regression model.”
MethodRegression Diagnostics: Identifying Influential Data and Sources of Collinearity.
Patterson, H. D., Belsley, D. A., Kuh, E., & Welsch, R. E. (1981) · Biometrics · doi:10.2307/2530185 · Cited by 1,370
Research works printed but not linked: 4
- R. Cook. Detection of influential observations in lin- ear regression. Technometrics, 19(1):15-18, 1977.
- M. J. Crawley. The R Book. Wiley, 2007.
- I. Kreft and J. De Leeuw. Introducing Multilevel Mod- eling. Sage Publications, 1998.
- T. Van der Meer, M. te Grotenhuis, and B. Pelzer. In- fluential cases in multilevel modeling. A method- ological comment. American Sociological Review, 75: 173-178, 2010. [p38, 40, 46]
Cited by
Sorted by citations
Items 101–200 of 449
Touitou, S., Heistermann, M., Schülke, O., & Ostner, J. (2021) · Behavioral Ecology and Sociobiology · doi:10.1007/s00265-021-03058-5 · Cited by 27
Wilke, C., Lahiff, N. J., Badihi, G., Donnellan, E., Hobaiter, C., Machanda, Z. P., Mundry, R., Pika, S., Soldati, A., Wrangham, R. W., et al. (2022) · Animal Behaviour · doi:10.1016/j.anbehav.2022.04.007 · Cited by 26
Holden, E., Buryn-Weitzel, J. C., Atim, S., Biroch, H., Donnellan, E., Graham, K. E., Hoffman, M., Jurua, M., Knapper, C. V., Lahiff, N. J., et al. (2022) · PLoS ONE · doi:10.1371/journal.pone.0278378 · Cited by 31
Santos, R. V. S., Novoa‐Pabon, A., Silva, H. M., & Pinho, M. R. (2020) · Marine Biology Research · doi:10.1080/17451000.2020.1718713 · Cited by 26
Muffato, V., Simonetto, A., Pellegrini, M., Tortora, C., & Meneghetti, C. (2022) · Journal of Environmental Psychology · doi:10.1016/j.jenvp.2022.101820 · Cited by 26
Sensorimotor synchronization to music reduces pain
Werner, L. M., Skouras, S., Bechtold, L., Pallesen, S., & Koelsch, S. (2023) · PLoS ONE · doi:10.1371/journal.pone.0289302 · Cited by 27
Wintering space use and site fidelity in a nomadic species, the snowy owl
Robillard, A., Gauthier, G., Therrien, J., & Bêty, J. (2018) · Journal of Avian Biology · doi:10.1111/jav.01707 · Cited by 26
Kim, H. H., & Kim, H.-J. K. (2021) · Research in Social Stratification and Mobility · doi:10.1016/j.rssm.2021.100640 · Cited by 26
Male chimpanzees communicate to mediate competition and cooperation during feeding
Bouchard, A. L., & Zuberbühler, K. (2022) · Animal Behaviour · doi:10.1016/j.anbehav.2022.01.009 · Cited by 25
Auger, C. A., Perosa, V., Greenberg, S. M., van Veluw, S. J., & Kozberg, M. G. (2023) · Journal of Neuroinflammation · doi:10.1186/s12974-023-02872-0 · Cited by 25
Nieuwenhuis, R. (2022) · Social Policy and Administration · doi:10.1111/spol.12806 · Cited by 25
Self-citationWaterhouse, M. D., Sjodin, B. M. F., Ray, C., Erb, L. P., Wilkening, J. L., & Russello, M. A. (2017) · Ecology and Evolution · doi:10.1002/ece3.3009 · Cited by 26
Christie, A. I., Colefax, A. P., & Cagnazzi, D. (2021) · Remote Sensing · doi:10.3390/rs14010021 · Cited by 24
Linguistic dissimilarity increases age-related decline in adult language learning
Schepens, J. J., van Hout, R. W. N. M., & der Slik, F. W. P. V. (2022) · Studies in Second Language Acquisition · doi:10.1017/s0272263122000067 · Cited by 30
Villain, A. S., Fernandez, M., Bouchut, C., Soula, H. A., & Vignal, C. (2016) · Animal Behaviour · doi:10.1016/j.anbehav.2016.03.009 · Cited by 24
Retrieval and Encoding Interference: Cross-Linguistic Evidence from Anaphor Processing
Laurinavichyute, A., Jäger, L. A., Akinina, Y. S., Roß, J., & Dragoy, O. (2017) · Frontiers in Psychology · doi:10.3389/fpsyg.2017.00965 · Cited by 24
Interactions between predation and disturbances shape prey communities
Karakoç, C., Radchuk, V., Harms, H., & Chatzinotas, A. (2018) · Scientific Reports · doi:10.1038/s41598-018-21219-x · Cited by 24
Profiling children's reading comprehension: A dynamic approach
Gruhn, S., Segers, E., Keuning, J., & Verhoeven, L. T. W. (2020) · Learning and Individual Differences · doi:10.1016/j.lindif.2020.101923 · Cited by 24
Cheng, L., Samuni, L., Lucchesi, S., Deschner, T., & Surbeck, M. (2022) · Animal Behaviour · doi:10.1016/j.anbehav.2022.02.014 · Cited by 28
Pickren, S. E., Stacy, M. E., Tufo, S. N. D., Spencer, M., & Cutting, L. E. (2021) · Reading Research Quarterly · doi:10.1002/rrq.431 · Cited by 32
Taxing the development structure of open source communities: An information processing view
Almarzouq, M. N., Grover, V., & Thatcher, J. B. (2015) · Decision Support Systems · doi:10.1016/j.dss.2015.09.004 · Cited by 27
Multicomponent and multisensory communicative acts in orang-utans may serve different functions
Fröhlich, M., Bartolotta, N., Fryns, C., Wagner, C., Momon, L., Jaffrezic, M., Setia, T. M., van Noordwijk, M. A., & van Schaik, C. P. (2021) · Communications Biology · doi:10.1038/s42003-021-02429-y · Cited by 24
Pupil dilation and skin conductance as measures of prediction error in aversive learning
Stemerding, L., van Ast, V. A., Gerlicher, A. M. V., & Kindt, M. (2022) · Behaviour Research and Therapy · doi:10.1016/j.brat.2022.104164 · Cited by 24
What You See Is What You Remember: Visual Chunking by Temporal Integration Enhances Working Memory
Akyürek, E. G., Kappelmann, N., Volkert, M., & van Rijn, H. (2017) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_01175 · Cited by 22
Schwab, S., & Dellwo, V. (2017) · The Journal of the Acoustical Society of America · doi:10.1121/1.5008849 · Cited by 22
Madden, V. J., Kamerman, P. R., Bellan, V., Catley, M. J., Russek, L. N., Camfferman, D., & Moseley, G. L. (2018) · Journal of Pain · doi:10.1016/j.jpain.2018.10.006 · Cited by 22
Dixon, C., Thomson, J., & Fricke, S. (2020) · Journal of Research in Reading · doi:10.1111/1467-9817.12305 · Cited by 30
Provost, E. J., Coleman, M. A., Butcher, P. A., Colefax, A. P., Schlacher, T. A., Bishop, M. J., Connolly, R. M., Gilby, B. L., Henderson, C. J., Jones, A. R., et al. (2021) · Ocean & Coastal Management · doi:10.1016/j.ocecoaman.2021.105750 · Cited by 22
Meta-analysis of diagnostic cell-free circulating microRNAs for breast cancer detection
Šehović, E., Urru, S., Chiorino, G., & Doebler, P. (2022) · BMC Cancer · doi:10.1186/s12885-022-09698-8 · Cited by 22
The Cerebellum Contributes to Prediction Error Coding in Reinforcement Learning in Humans
Huvermann, D. M., Berlijn, A. M., Thieme, A. G., Erdlenbruch, F., Groiss, S. J., Deistung, A., Mittelstaedt, M., Wondzinski, E., Sievers, H., Frank, B., et al. (2025) · Journal of Neuroscience · doi:10.1523/jneurosci.1972-24.2025 · Cited by 23
Thompson, J. S., Hudson, C. D., Huxley, J. N., Kaler, J., Robinson, R. S., Woad, K. J., Bollard, N. J., Gibbons, J. M., & Green, M. J. (2022) · Scientific Reports · doi:10.1038/s41598-022-07826-9 · Cited by 21
Liebert, J. A., Cherney, J. H., Ketterings, Q. M., Mirsky, S. B., Pelzer, C. J., & Ryan, M. R. (2023) · Frontiers in Sustainable Food Systems · doi:10.3389/fsufs.2023.1067506 · Cited by 21
Roberts, D. R., Wood, W. H., & Marshall, S. J. (2019) · International Journal of Climatology · doi:10.1002/joc.6005 · Cited by 21
Isbell, D. R., Park, O.-S., & Lee, K. (2019) · Journal of Second Language Pronunciation · doi:10.1075/jslp.17010.isb · Cited by 21
God Values the Lives of My Out-Group More Than I Do: Evidence From Fiji and Israel
Pasek, M. H., Shackleford, C. M., Smith, J. M., Vishkin, A., Lehner, A., & Ginges, J. (2020) · Social Psychological and Personality Science · doi:10.1177/1948550620904516 · Cited by 22
Löwenbrück, F., & Heß, U. (2021) · Biological Psychology · doi:10.1016/j.biopsycho.2021.108120 · Cited by 23
What's worth the effort: Ten-month-old infants infer the value of goals from the costs of actions.
Liu, S., Ullman, T. D., Tenenbaum, J. B., & Spelke, E. S. (2017) · Cognitive Science · Cited by 20
Maternal deprivation affects goat kids’ stress coping behaviour
Toinon, C., Waiblinger, S., & Rault, J. (2021) · Physiology & Behavior · doi:10.1016/j.physbeh.2021.113494 · Cited by 20
Causes and consequences of variation in female mate search investment in a lekking bird
DuVal, E. H., & Kapoor, J. A. (2015) · Behavioral Ecology · doi:10.1093/beheco/arv110 · Cited by 20
KNOWLAND, V. C. P., Berens, S. C., GASKELL, M. G., WALKER, S. A., & HENDERSON, L.-M. (2021) · Journal of Child Language · doi:10.1017/s0305000920000677 · Cited by 20
Emotional food craving across the eating disorder spectrum: an ecological momentary assessment study
Arend, A., Blechert, J., Yanagida, T., Voderholzer, U., & Reichenberger, J. (2024) · Eating and Weight Disorders - Studies on Anorexia Bulimia and Obesity · doi:10.1007/s40519-024-01690-4 · Cited by 20
Chartrand, K., Szabó, M., Sinutok, S., Rasheed, M. A., & Ralph, P. J. (2018) · Marine Environmental Research · doi:10.1016/j.marenvres.2018.02.006 · Cited by 19
Risk-promoting effects of reward-paired cues in human sign- and goal-trackers
Cherkasova, M. V., Clark, L., Barton, J. J. S., Stoessl, A. J., & Winstanley, C. A. (2024) · Behavioural Brain Research · doi:10.1016/j.bbr.2024.114865 · Cited by 23
Palamara, D. R., Neeman, T. M., Golab, A. N., & Sheppard, A. P. (2015) · International journal of greenhouse gas control · doi:10.1016/j.ijggc.2015.09.007 · Cited by 19
Vervet monkeys greet adult males during high-risk situations
Mercier, S., Neumann, C., van de Waal, E., Chollet, E., de Bellefon, J. M., & Zuberbühler, K. (2017) · Animal Behaviour · doi:10.1016/j.anbehav.2017.07.021 · Cited by 19
Gran, O., & Götmark, F. (2019) · Biodiversity and Conservation · doi:10.1007/s10531-019-01736-5 · Cited by 19
Thompson, M. J., & Morand‐Ferron, J. (2019) · Animal Cognition · doi:10.1007/s10071-019-01271-4 · Cited by 19
Sleep Promotes Phonological Learning in Children Across Language and Autism Spectra
Knowland, V. C. P., Fletcher, F. E., Henderson, L. M., Walker, S., Norbury, C., & Gaskell, M. G. (2019) · Journal of Speech Language and Hearing Research · doi:10.1044/2019_jslhr-s-19-0098 · Cited by 23
Bufford, J. L., & Hulme, P. E. (2021) · Biological Invasions · doi:10.1007/s10530-021-02532-5 · Cited by 19
Detecting influential data in multivariate survival models
Kaombe, T. M., & Manda, S. (2021) · Communication in Statistics- Theory and Methods · doi:10.1080/03610926.2021.1982983 · Cited by 18
Arseneau‐Robar, T. J. M., Anderson, K. A., Vasey, E. N., Sicotte, P., & Teichroeb, J. A. (2022) · Frontiers in Ecology and Evolution · doi:10.3389/fevo.2022.775288 · Cited by 18
Food mobility and the evolution of grasping behaviour: a case study in strepsirrhine primates
Peckre, L. R., Lowie, A., Brewer, D., Ehmke, E. E., Welser, K. H., Shaw, E., Wall, C. E., Pouydebat, E., & Fabre, A. (2019) · Journal of Experimental Biology · doi:10.1242/jeb.207688 · Cited by 18
Timing storytime to maximize children’s ability to retain new vocabulary
Henderson, L. M., van Rijn, E., James, E., Walker, S., Knowland, V. C. P., & Gaskell, M. G. (2021) · Journal of Experimental Child Psychology · doi:10.1016/j.jecp.2021.105207 · Cited by 22
A novel theory of Asian elephant high-frequency squeak production
Beeck, V. C., Heilmann, G., Kerscher, M., & Stoeger, A. S. (2021) · BMC Biology · doi:10.1186/s12915-021-01026-z · Cited by 19
Seed size–number trade‐offs are absent in the introduced range for three congeneric plant invaders
Bufford, J. L., & Hulme, P. E. (2021) · Journal of Ecology · doi:10.1111/1365-2745.13761 · Cited by 18
Liew, S., Zavaliangos‐Petropulu, A., Schweighofer, N., Jahanshad, N., Lang, C. E., Lohse, K. R., Banaj, N., Barisano, G., Baugh, L. A., Bhattacharya, A. K., et al. (2021) · Brain Communications · doi:10.1093/braincomms/fcab254 · Cited by 18
Time-dependent effects of psychosocial stress on the contextualization of neutral memories
Sep, M. S. C., van Ast, V. A., Gorter, R., Joëls, M., & Geuze, E. (2019) · Psychoneuroendocrinology · doi:10.1016/j.psyneuen.2019.06.021 · Cited by 19
Election-related internet-shutdowns in autocracies and hybrid regimes
Eichhorn, K., & Linhart, E. (2022) · Journal of Elections Public Opinion and Parties · doi:10.1080/17457289.2022.2090950 · Cited by 19
Social tolerance and interactional opportunities as drivers of gestural redoings in orang-utans
Fröhlich, M., & van Schaik, C. P. (2022) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2021.0106 · Cited by 17
van Dijk, C. N., & Unsworth, S. (2023) · Open Mind · doi:10.1162/opmi_a_00105 · Cited by 17
Innovative problem solving by wild falcons
Harrington, K. J., Folkertsma, R., Auersperg, A. M. I., Biondi, L. M., & Lambert, M. L. (2023) · Current Biology · doi:10.1016/j.cub.2023.10.061 · Cited by 17
Gambling disorder is associated with reduced sensitivity to expected value during risky choice
Limbrick‐Oldfield, E. H., Cherkasova, M. V., Kennedy, D., Goshko, C., Griffin, D., Barton, J. J. S., & Clark, L. (2020) · Journal of Behavioral Addictions · doi:10.1556/2006.2020.00088 · Cited by 17
Number and type of toys affect joint attention of mothers and infants
Koşkulu‐Sancar, S., Küntay, A. C., Liszkowski, U., & Uzundağ, B. A. (2021) · Infant Behavior and Development · doi:10.1016/j.infbeh.2021.101589 · Cited by 17
Biotic resistance to tropical ornamental invasion
Bufford, J. L., Lurie, M. H., & Daehler, C. C. (2015) · Journal of Ecology · doi:10.1111/1365-2745.12534 · Cited by 17
Interactive vocal communication at the nest by parent Great Tits Parus major
Boucaud, I. C. A., Valère, P. A., Smith, M. L. N. A., Doligez, B., Cauchard, L., Rybak, F., & Vignal, C. (2016) · Ibis · doi:10.1111/ibi.12374 · Cited by 17
Flexible tool set transport in Goffin’s cockatoos
Osuna‐Mascaró, A. J., O’Hara, M., Folkertsma, R., Tebbich, S., Beck, S. R., & Auersperg, A. M. I. (2023) · Current Biology · doi:10.1016/j.cub.2023.01.023 · Cited by 16
Roberts, B. H., Morrongiello, J. R., Morgan, D. L., King, A. J., Saunders, T. M., Banks, S. C., & Crook, D. A. (2023) · Journal of Animal Ecology · doi:10.1111/1365-2656.14019 · Cited by 16
Karlsson, Ø., Laaksonen, M. S., & McGawley, K. (2021) · PLoS ONE · doi:10.1371/journal.pone.0250088 · Cited by 17
Maternal deprivation affects goat kids’ social behavior before and after weaning
Toinon, C., Waiblinger, S., & Rault, J. (2022) · Developmental Psychobiology · doi:10.1002/dev.22269 · Cited by 16
Hessel, A., & Murphy, V. A. (2018) · Journal of Child Language · doi:10.1017/s0305000918000399 · Cited by 21
Mocha, Y. B., & Pika, S. (2019) · Frontiers in Ecology and Evolution · doi:10.3389/fevo.2019.00087 · Cited by 15
Prior territorial responses and home range size predict territory defense in radio-tagged great tits
Naguib, M., Titulaer, M., Waas, J. R., van Oers, K., Sprau, P., & Snijders, L. (2022) · Behavioral Ecology and Sociobiology · doi:10.1007/s00265-022-03143-3 · Cited by 15
The association of fear of movement and postural sway in people with low back pain
Meinke, A., Maschio, C., Meier, M. L., Karlen, W., & Swanenburg, J. (2022) · Frontiers in Psychology · doi:10.3389/fpsyg.2022.1006034 · Cited by 15
Aelenei, C., Jury, M., Darnon, C., Sicard, A., Maggio, S., & Martinot, D. (2022) · British Journal of Educational Psychology · doi:10.1111/bjep.12571 · Cited by 16
Steen, D. A., McClure, C. J. W., Sutton, W. B., Rudolph, D. C., Pierce, J. B., Lee, J. R., Smith, L. L., Gregory, B. B., Baxley, D. L., Stevenson, D. J., et al. (2014) · Herpetologica · doi:10.1655/herpetologica-d-13-00064 · Cited by 15
Mühlenbeck, C., Liebal, K., Pritsch, C., & Jacobsen, T. (2015) · PLoS ONE · doi:10.1371/journal.pone.0139894 · Cited by 16
Modality-specific Changes in Motor Cortex Excitability After Visuo-proprioceptive Realignment
Munoz‐Rubke, F., Mirdamadi, J. L., Lynch, A. K., & Block, H. J. (2017) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_01171 · Cited by 15
Hajdu, D., Sedova, A., Datlinger, F., Häfner, J., Steiner, I., Kriechbaum, K., Scholda, C. D., Sacu, S., Schmidt-Erfurth, U. M., & Pollreisz, A. (2020) · International Journal of Retina and Vitreous · doi:10.1186/s40942-020-00253-w · Cited by 15
Illusion of control affects ERP amplitude reductions for auditory outcomes of self‐generated actions
Seidel, A., Ghio, M., Studer, B., & Bellebaum, C. (2021) · Psychophysiology · doi:10.1111/psyp.13792 · Cited by 15
Wyss, A. M., Baumgärtner, T., Rosales, E. G., Soutschek, A., & Knoch, D. (2024) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2024.1395426 · Cited by 14
Fallon, B., Neumann, C., Byrne, R. W., & Zuberbühler, K. (2016) · Animal Behaviour · doi:10.1016/j.anbehav.2015.12.028 · Cited by 14
The impact of social anxiety on feedback-based go and nogo learning
Peterburs, J., Albrecht, C., & Bellebaum, C. (2021) · Psychological Research · doi:10.1007/s00426-021-01479-5 · Cited by 16
Social dominance and rainfall predict telomere dynamics in a cooperative arid‐zone bird
Wood, E. M., Capilla‐Lasheras, P., Cram, D. L., Walker, L. A., York, J. E., Lange, A., Hamilton, P. B., Tyler, C. R., & Young, A. J. (2021) · Molecular Ecology · doi:10.1111/mec.15868 · Cited by 14
Schwarck, S., Busse, N., Ziegler, G., Glanz, W., Becke, A., & Düzel, E. (2021) · Frontiers in Sports and Active Living · doi:10.3389/fspor.2021.684089 · Cited by 14
Autocratic heterogeneity in the provision of environmental protection
Eichhorn, K., & Linhart, E. (2022) · Zeitschrift für Vergleichende Politikwissenschaft · doi:10.1007/s12286-022-00519-7 · Cited by 14
Avoid or Embrace? Practice Effects in Alzheimer’s Disease Prevention Trials
Aschenbrenner, A. J., Hassenstab, J. J., Wang, G., Li, Y., Xiong, C., McDade, E., Clifford, D. B., Salloway, S., Farlow, M. R., Yaari, R., et al. (2022) · Frontiers in Aging Neuroscience · doi:10.3389/fnagi.2022.883131 · Cited by 14
Alawneh, J. I., Stevenson, M. A., Williamson, N. B., Lopez-Villalobos, N., & Otley, T. (2013) · Preventive Veterinary Medicine · doi:10.1016/j.prevetmed.2013.10.010 · Cited by 14
Thibodeau, P. S., Steinberg, D. K., & Maas, A. E. (2020) · Journal of Experimental Marine Biology and Ecology · doi:10.1016/j.jembe.2020.151412 · Cited by 14
Nutt, K. S., Burslem, D. F. R. P., Maycock, C. R., Ghazoul, J., Khoo, E., Hastie, A. Y. L., & Kettle, C. J. (2016) · Plant Ecology & Diversity · doi:10.1080/17550874.2016.1262923 · Cited by 14
Luo, T., Steeneveld, W., Nielen, M., Zanini, L., & Zecconi, A. (2022) · Animals · doi:10.3390/ani13010080 · Cited by 13
Herfindal, A. M., Nilsen, M., Aspholm, T. E., Schultz, G. I. G., Valeur, J., Rudi, K., Thoresen, M., Lundin, K. E. A., Henriksen, C., & Bøhn, S. K. (2024) · BMC Medicine · doi:10.1186/s12916-024-03562-1 · Cited by 13
Lev‐Ari, S. (2018) · Language Cognition and Neuroscience · doi:10.1080/23273798.2018.1508733 · Cited by 13
Santos, R. V. S., Novoa‐Pabon, A., Silva, H. M., & Pinho, M. R. (2019) · Journal of Fish Biology · doi:10.1111/jfb.13937 · Cited by 13
Menghini, L., & Balducci, C. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph18063263 · Cited by 17
Li, M. (2020) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2021.104278 · Cited by 13
Nuttall, H. E., Moore, D. R., Barry, J. G., Krumbholz, K., & de Boer, J. (2015) · Journal of Neurophysiology · doi:10.1152/jn.00548.2014 · Cited by 13
Evaluating Effort: Influences of Evaluation Mode on Judgments of Task‐specific Efforts
Dunn, T. L., Koehler, D. J., & Risko, E. F. (2017) · Journal of Behavioral Decision Making · doi:10.1002/bdm.2018 · Cited by 15
Spezie, G., & Fusani, L. (2022) · Behavioral Ecology and Sociobiology · doi:10.1007/s00265-022-03200-x · Cited by 12
Kim, H. H. (2023) · International Journal of Comparative Sociology · doi:10.1177/00207152231173307 · Cited by 12
Metadata
- Authors
- Nieuwenhuis, R., Grotenhuis, M., & Pelzer, B.
- Venue
- The R Journal
- Publisher
- The R Foundation
- Published
- 2012
- Volume
- 4
- Issue
- 2
- Pages
- 38
- ISSN
- 2073-4859 (ISSN-L)
- OpenAlex
- W1549371580
- Semantic Scholar
- c6d41ed109eb79b026ba2d1fb769b6ebf525ebd4
- Cited by
- 442
- References
- 18
- Citations by year
- 2026: 29 · 2025: 36 · 2024: 45 · 2023: 38 · 2022: 64 · 2021: 63 · 2020: 33 · 2019: 28 · 2018: 24 · 2017: 26 · 2016: 22 · 2015: 17 · 2014: 12 · 2013: 5
- FWCI
- 10.39
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- The R Journal (publishedVersion, cc-by)
- Lincoln (University of Nebraska) (submittedVersion, cc-by)
- Radboud Repository (Radboud University) (submittedVersion)
- University of Twente Research Information (submittedVersion)
- Radboud Repository (Radboud University) (submittedVersion, other-oa)
- University of Twente Research Information (submittedVersion, other-oa)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Statistical Methods and Models
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Physical Sciences › Mathematics › Statistics and Probability › Data Analysis with R
- Keywords
- mixed models, generalized linear mixed models, Cook's distance, nested data, influence diagnostics, R package
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-09, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-09 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
