The DOI matches its Crossref record; 16 of the 18 references it cites were found in a registry or an index; none of the 16 sources its claims cite is checked for support yet.
influence.ME: Tools for Detecting Influential Data in Mixed Effects Models
Open accessAbstract
influence.ME provides tools for detecting influential data in mixed effects models.The application of these models has become common practice, but the development of diagnostic tools has lagged behind.influence.ME calculates standardized measures of influential data for the point estimates of generalized mixed effects models, such as DFBETAS, Cook's distance, as well as percentile change and a test for changing levels of significance.influence.ME calculates these measures of influence while accounting for the nesting structure of the data.The package and measures of influential data are introduced, a practical example is given, and strategies for dealing with influential data are suggested.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
The paper introduces influence.ME, an R package for detecting influential cases in generalized mixed effects models estimated with lme4, providing DFBETAS, Cook’s distance, percentile change, and a test for changing significance while accounting for nesting. In a 23-school example, deleting school 7472 changed the school-level structure effect from non-significant to significant.
Key findings
- influence.ME calculates standardized influence measures for mixed effects models: DFBETAS, Cook’s distance, percentile change, and a significance-change test (`sigtest`), plus plotting and `exclude.influence` for deleting identified cases.
- In the example with 519 students in 23 schools, the original model estimated the structure effect as -2.0950 (SE 1.3237, t = -1.583, not significant) and the homework effect as 2.3938 (SE 0.2771, t = 8.640).
- The DFBETAS cut-off was 2/√23 = .41; school 7472 had a structure DFBETAS of 1.254990963, exceeding the cut-off. The Cook’s distance cut-off was 4/23 = .17; school 7472 had a Cook’s distance of 1.575002.
- After excluding school 7472, the model used 496 observations in 22 groups and estimated the structure effect as -3.8949 (SE 1.4342, t = -2.716), now statistically significant. For lower-level observations, the Cook’s distance cut-off was 4/519 = .0077; students 88 and 89 exceeded it for the structure estimate, but `sigtest` showed no significance changes.
Methods
- The paper describes the influence.ME package and demonstrates it with the `school23` data, a subset of NELS-88 containing 519 students nested in 23 schools, using a linear mixed model: `math ~ homework + structure + (1 | school.ID)` estimated with lme4.
- The `influence` function iteratively deletes each higher-level group, or with `obs=TRUE` each lower-level observation, and re-estimates the model; the resulting `estex` object is passed to `dfbetas`, `cooks.distance`, `pchange`, `sigtest`, and plotting functions.
- DFBETAS measures change in a single parameter standardized by the excluding-case standard error; Cook’s distance summarizes change across selected parameters; percentile change reports percentage change; `sigtest` compares test statistics after deletion to a user-specified value.
Limitations
- Stated: the leverage measure was not developed for generalized mixed effects models, so the package emphasizes influence on point estimates; the Langford and Lewis `delete=FALSE` procedure does not work with factor variables; the `influence` function can evaluate higher-level cases or single observations but not both at once; lower-level detection is often infeasible with many lower-level observations; the procedure can be computationally intensive; no cut-off is provided for percentile change.
- Evident: the worked example assumes math, homework, and structure are correctly measured at the interval level, and the paper does not report simulation or benchmark comparisons of the package’s performance.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
41 No basis given · 24 Own interpretation · 16 Not checked yet · 12 Own result
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“influence.ME calculates these measures of influence while accounting for the nesting structure of the data.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Abstract
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“We introduce influence.ME (Nieuwenhuis, Pelzer, and te Grotenhuis, 2012), an R-package that provides tools for detecting influential cases in mixed effects regression models estimated with lme4 (Bates and Maechler, 2010).”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Abstract
What the sources say
- Not checked yetcites Nieuwenhuis, Pelzer, and te Grotenhuis, 2012
- Not checked yetcites Bates and Maechler, 2010
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Based on these estimates, we may conclude that students spending more time on their math homework score better on a math test.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Example: students in 23 schools
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“It does, however, provide a basic test for detecting whether single cases change the level of significance of an estimate, and therefore the ability to make inferences from the estimated model.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data in Mixed Effects Models
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The influence.ME package introduced here contributes to these existing options, by providing several measures of influential data for generalized mixed effects models.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data in Mixed Effects Models
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Although influence.ME cannot provide the leverage measure to detect this, it provides a test for changes in the statistical significance of the fixed parameters in the mixed effects model.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Test for changes in significance
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The DFBETAS for the structure variable clearly exceeds the cut-off value of .41.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Calculating measures of influence
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Regarding class structure, however, we do not find a statistically significant association with math test scores.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Example: students in 23 schools
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Using the sigtest function, however, showed that the deletion of none of the students from the data affected the level of significance of the structure variable, nor of any of the other variables in the model.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Measuring the influence of lower-level observations
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Also, the estimates of the homework variable changes substantially with the deletion of several schools, as indicated by the high values of DF-BETAS.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Calculating measures of influence
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- It may state two separate claims as one. 15
- Probably not a substantial claim. 56
- The quote only partly states this claim. 2
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“influence.ME calculates standardized measures of influential data for the point estimates of generalized mixed effects models, such as DFBETAS, Cook's distance, as well as percentile change and a test for changing levels of significance.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“influence.ME provides tools for detecting influential data in mixed effects models.”
- Also stated in: § Additional information · plain
“influence.ME provides tools for detecting influential data in mixed effects models.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
- Rests on an assumption: The application of mixed effects models has become common practice, but the development of diagnostic tools lags behind.
- Also stated in: § Additional information · plain
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In addition, we introduce the measure of percentile change and a test for changing levels of significance of the fixed parameters.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In influence.ME, values for Cook's distance in mixed effects models are calculated using the function cooks.distance,which takes the object returned from influence as input.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Cook's Distance
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“influence.ME offers four such measures, which are detailed in this section.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > The Outcome Measures
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one. The quote only partly states this claim.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Detecting influence of lower-level observations is supported for applications in various disciplines where mixed effects models are typically applied to only a limited number of observations per group.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data in Mixed Effects Models
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Consequently, the current installment of influence.ME emphasizes detecting the influence of cases on the point estimates of generalized mixed effect models.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data in Mixed Effects Models
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The influence.ME package contains the school23 data.frame, that provides information on the performance of 519 students in 23 schools.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Example: students in 23 schools
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“influence.ME provides this functionality, which is accessed by specifying delete=FALSE as an option to the influence function.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data in Mixed Effects Models
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“The plot function takes the output of the influence function to create a dotplot of a selected measure of influence (cf.Sarkar, 2008).”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Plots
What the sources say
- Not checked yetcites Sarkar, 2008
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Finally, influence.ME also allows for detecting the influence of lower-level cases in the mixed effects model.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data in Mixed Effects Models
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The sigtest function tests whether excluding the influence of a single case changes the statistical significance of any of the variables in the model.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Test for changes in significance
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The returned object of class”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data in Mixed Effects Models
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“All four measures of influence discussed above, can also be plotted.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Plots
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The influence function described above returns an object with information on how much the parameter estimates in a mixed effects model change, after the (influence of) observations of higher-level groups and their individual-level observations were removed from it iteratively.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > The Outcome Measures
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Student's class and school are equivalent in this data, since only one class per school is available.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Example: students in 23 schools
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In influence.ME, values for DFBETAS in mixed effects models can be calculated using the function dfbetas, which takes the object returned from influence as input.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > DFBETAS
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The default option of dfbetas is to calculate DFBETAS for estimates of all fixed effects in the model.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > DFBETAS
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The influence function in the influence.ME package performs this procedure automatically, and returns an object containing information on the parameter estimates excluding the influence of each higher level group separately.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data in Mixed Effects Models
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“This test of significance is based on the test statistic provided by the lme4 package.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Test for changes in significance
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The influence function can either determine the influence of higher-level cases, or of single observations, but not both at the same time.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data in Mixed Effects Models
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In the case of our example, the results for Cook's Distance and the results of this test for changing levels of significance both indicate that school number 7472 overly influences the regression outcomes regarding the school-level structure variable.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Testing for Changes in Statistical Significance (sigtest)
hedgeddirectSingle case ·- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Based on Figure 2, it is clear that both the structure and the homework variables are highly susceptible to the influence of single schools.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Calculating measures of influence
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The estimate for class structure in this model seems to be influenced most strongly by observations in school number 7472.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Calculating measures of influence
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“More importantly, though, the negative effect of the structure variable now is statistically significant, whereas it was not in the original model.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Dealing with Influential Data
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one. The quote only partly states this claim.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In this plot, it is clear that one single school represented in the lower-left corner of the graph seems to be an outlier, and -more importantly -the non-linear curve shown in this graph clearly indicates this single school with class structure level of 2 may overly influence a linear regression line estimated based on these data.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Visual Examination
boostedconditionalSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“These results are not shown here, but indicated that the deletion of none of the schools changed the level of significance of the homework variable.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Testing for Changes in Statistical Significance (sigtest)
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The output is not shown here, but the reader can verify that students with numbers 88 and 89 exert too much influence on the estimate of the structure variable.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Measuring the influence of lower-level observations
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Testing for influential cases in mixed effects regression models is important, because influential data negatively influence the statistical fit and generalizability of the model.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Abstract
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The application of mixed effects regression models has become common practice in the field of social sciences.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, if a single case has extremely high or low scores on the dependent variable relative to its expected value -given other variables in the model, one or more of the independent variables, or both -this case may overly influence the regression parameters by 'pulling' the estimated regression line towards itself.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
- Rests on an assumption: All cases used to estimate a regression model exert some level of influence on the regression parameters.
In the paper
“As used in the social sciences, mixed effects regression models take into account that observations on individual respondents are nested within higherlevel groups such as schools, classrooms, states, and countries (Snijders and Bosker, 1999), and are often referred to as multilevel regression models.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Abstract
What the sources say
- Not checked yetcites Snijders and Bosker, 1999
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“These plots, however, are not available for mixed effects models.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data in Mixed Effects Models
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
None of the sources cited for this claim have been checked yet.
In the paper
“Values exceeding this cut-off value are regarded as overly influencing the regression outcomes for that specific estimate.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > DFBETAS
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Regarding mixed models, this relates to the influence a higher-level unit has on the parameter estimate.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > DFBETAS
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“A higher leverage means more extreme scores on the independent variable(s), and a greater potential of overly influencing the regression outcomes.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data
plainconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In detecting influential data in mixed effects models, the key focus is on changes in the estimates of variables measured at the group-level.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data in Mixed Effects Models
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The limitation is that, unfortunately, as far as we are aware, the measure of leverage was not developed for generalized mixed effects models.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data in Mixed Effects Models
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Cook's distance provides a summary measure for the influence a higher level unit exerts on all parameter estimates simultaneously, or a selection thereof.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Cook's Distance
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“To apply the logic of detecting influential data to generalized mixed effects models, one has to measure the influence of a particular higher level group on the estimates of a predictor measured at that level.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data in Mixed Effects Models
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“However, most mixed effects regression models estimate the effects of both lower-level and higher-level variables simultaneously.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data in Mixed Effects Models
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Since the procedure of the influence function entails re-estimating mixed effects models several times, this can be computationally intensive.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data in Mixed Effects Models
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“DFBETAS is a standardized measure that indicates the level of influence observations have on single parameter estimates (Fox, 2002).”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > DFBETAS
What the sources say
- Not checked yetcites Fox, 2002
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Since neither outliers, nor cases with a high leverage, are necessarily influential, a different procedure is required for detecting influential cases.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“As a rule of thumb, a cut-off value is given for DFBETAS (Belsley et al., 1980): 2/ √ n in which n, the number of observations, refers to the number of groups in the grouping factor under evaluation (and not to the number of observations nested within the group under investigation).”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > DFBETAS
What the sources say
- Not checked yetcites Belsley et al., 1980
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“The LMERConvenienceFunctions package provides model criticism plots, including the density of the model residuals and the fitted values versus the standardized residuals (Tremblay, 2012).”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data in Mixed Effects Models
What the sources say
- Not checked yetcites Tremblay, 2012
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“First, DFBETAS is a standardized measure of the absolute difference between the estimate with a particular case included and the estimate without that particular case (Belsley, Kuh, and Welsch, 1980).”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data
What the sources say
- Not checked yetcites Belsley, Kuh, and Welsch, 1980
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Cases with high leverage but a low deleted residual compress standard errors, while cases with low leverage and a high deleted residual inflate standard errors.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one.
In the paper
“Second, Cook's distance provides an overall measurement of the change in all parameter estimates, or a selection thereof (Cook, 1977).”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data
What the sources say
- Not checked yetcites Cook, 1977
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The basic rationale behind measuring influential cases is based on the principle that when single cases are iteratively omitted from the data, models based on these data should not produce substantially different estimates.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As a final note, it is pointed out that if Cook's distance is calculated based on a single parameter, the Cook's distance equals the squared value of DFBE-TAS for that parameter.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Cook's Distance
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The nature of this statistic varies between different distributional families in the generalized mixed effects models.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Test for changes in significance
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The base R installation provides various plots for regression models, including but not limited to plots showing residuals versus the fitted scores, Cook's distances, and the leverage versus the deleted residuals.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data in Mixed Effects Models
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“However, while this package works with the lme4 package, it only is applicable for linear mixed effects models.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data in Mixed Effects Models
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“These data are a subset of the NELS-88 data (National Education Longitudinal Study of 1988).”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Example: students in 23 schools
What the sources say
- Not checked yetcites NELS-88
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“No cut-off value is provided, for determining what percent change in parameter estimate is considered too large will primarily depend on the goal for which the model was estimated and, more specifically, the nature of the hypotheses that are tested.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Percentile Change
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“If the model parameters change substantially after a single case is excluded, this case may be regarded as too influential.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In social science applications of mixed effects models, with a great number of lower-level observations nested in a limited number of groups, this will not always be feasible.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Detecting Influential Data in Mixed Effects Models
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“influence.ME calculates standardized measures of influential data such as DFBETAS and Cook's distance, as well as percentile change and a test for changing in statistical significance of fixed parameter estimates.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Additional information
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
- Rests on an assumption: The application of mixed effects models has become common practice, but the development of diagnostic tools lags behind.
In the paper
“The cut-off value of DFBETAS equals 2/ √ n (Belsley et al., 1980), which in this case equals 2/ √ 23 = .41.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Calculating measures of influence
What the sources say
- Not checked yetcites Belsley et al., 1980
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“He argues that the presence of outliers may indicate that one or more important variables were omitted from the model.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Dealing with Influential Data
What the sources say
- Not checked yetcites Lieberman (2005)
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Cases can influence the point estimates without affecting their level of significance, or affect the level of significance without overly affecting the point estimate itself.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Testing for Changes in Statistical Significance (sigtest)
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: unclear — It is unclear why the paper says it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We discussed above that an influential case is not necessarily an outlier in a regression model.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Dealing with Influential Data
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
The paper relies on this without support.
Diagnostic tools to evaluate fitted mixed effects models lag behind their increasing popularity.
“Despite these models' increasing popularity, diagnostic tools to evaluate fitted models lag behind.”
The paper relies on this without support.
Most existing procedures for detecting influential data do not account for the nesting structure of the data.
“However, most existing procedures do not account for the nesting structure of the data.”
The paper relies on this without support.
All cases used to estimate a regression model exert some level of influence on the regression parameters.
“All cases used to estimate a regression model exert some level of influence on the regression parameters.”
The authors state this as an assumption.
The math, homework, and structure variables were correctly measured at the interval level.
“For the purpose of our example, we assume here that the math, homework, and structure variables were correctly measured at the interval level.”
The paper relies on this without support.
The application of mixed effects models has become common practice, but the development of diagnostic tools lags behind.
“The application of these models has become common practice, but the development of diagnostic tools lag behind.”
The paper relies on this without support.
Overly influential data are a problem especially encountered in models based on a limited number of cases.
“Since overly influential data are a problem especially encountered in models based on a limited number of cases, a straightforward remedy would be to observe more cases in the population of interest.”
The paper relies on this without support.
If data are used to estimate too complex models, or if parameterization is incorrect, influential cases are more likely to occur.
“If the data are used to estimate too complex models, or if parameterization is incorrect, influential cases are more likely to occur.”
The paper relies on this without support.
In social science applications of mixed effects models, higher-level groups are often readily accessible cases such as schools and countries, making it possible to obtain additional measurements to remedy influential data.
“However, in social science applications of mixed effects models, the higher-level groups are often readily accessible cases such as schools and countries.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“We screened for outliers within the multilevel modeling framework using the R package influence.ME (Nieuwenhuis, Pelzer, & Grotenhuis, 2012) and found no evidence of influential outlying data (all Cook’s distance < 1; Tabachnick & Fidell, 2007).”
From Enhancing social connectedness in anxiety and depression through amplification of positivity: Preliminary treatment outcomes and process of change (2020) · cites this paper as Nieuwenhuis, Pelzer, & Grotenhuis, 2012
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Cook's Distance
“In addition, the user can specify sort=TRUE to have the values for Cook's distance returned in descending order.”
§ influence.ME: Tools for Detecting Influential Data in Mixed Effects Models > Calculating measures of influence
“[,1] 24371 6.825871e-06 72292 1.524927e-05 ... 54344 2.256612e-01 7829 3.081222e-01 7472 1.575002e+00”
Citations
Cites
Regression Modeling Strategies
Harrell, F. E. (2001) · Springer series in statistics · doi:10.1007/978-1-4757-3462-1 · Cited by 9,270
Regression Diagnostics: Identifying Influential Data and Sources of Collinearity
Belsley, D. A., Kuh, E., & Welsch, R. E. (1980) · Wiley Series in Probability and Statistics · doi:10.1002/0471725153 · Cited by 7,297
Harrell (2015) · Springer Series in Statistics · doi:10.1007/978-3-319-19425-7 · Cited by 6,878
Regression Diagnostics -- Identifying Influential Data and Sources of Collinearity
Beaumont, C. D., Belsley, D. A., Kuh, E., & Welsch, R. E. (1981) · Journal of the Operational Research Society · doi:10.2307/2581267 · Cited by 6,362
Crawley, M. J. (2007) · Medical Entomology and Zoology · Cited by 5,431
Introducing Multilevel Modeling
Kreft, I., & de Leeuw, J. (1998) · doi:10.4135/9781849209366 · Cited by 2,997
Detection of Influential Observation in Linear Regression
Cook, R. D. (2000) · Technometrics · doi:10.2307/1271434 · Cited by 2,277
Sarkar, D. (2008) · Springer eBooks · doi:10.1007/978-0-387-75969-2 · Cited by 1,792
Nested Analysis as a Mixed-Method Strategy for Comparative Research
LIEBERMAN, E. S. (2005) · American Political Science Review · doi:10.1017/s0003055405051762 · Cited by 1,515 · cited 2×
“As a result of the specific modification of the model-specification, this specific procedure suggested by Langford and Lewis (1998) does not work when factor-variables are used in the regression model.”
MethodRegression Diagnostics: Identifying Influential Data and Sources of Collinearity.
Patterson, H. D., Belsley, D. A., Kuh, E., & Welsch, R. E. (1981) · Biometrics · doi:10.2307/2530185 · Cited by 1,370
Research works printed but not linked: 4
- R. Cook. Detection of influential observations in lin- ear regression. Technometrics, 19(1):15-18, 1977.
- M. J. Crawley. The R Book. Wiley, 2007.
- I. Kreft and J. De Leeuw. Introducing Multilevel Mod- eling. Sage Publications, 1998.
- T. Van der Meer, M. te Grotenhuis, and B. Pelzer. In- fluential cases in multilevel modeling. A method- ological comment. American Sociological Review, 75: 173-178, 2010. [p38, 40, 46]
Cited by
Sorted by citations
Fitting Linear Mixed-Effects Models Using lme4
Bates, D., Mächler, M., Bolker, B., & Walker, S. (2015) · Journal of Statistical Software · doi:10.18637/jss.v067.i01 · Cited by 86,835
Linear models and linear mixed effects models in R with linguistic applicationsPreprint
Winter, B. (2013) · arXiv.org · doi:10.48550/arxiv.1308.5499 · Cited by 658
Gotham, K., Brunwasser, S. M., & Lord, C. (2015) · Journal of the American Academy of Child & Adolescent Psychiatry · doi:10.1016/j.jaac.2015.02.005 · Cited by 310
Ten-month-old infants infer the value of goals from the costs of actions
Liu, S., Ullman, T. D., Tenenbaum, J. B., & Spelke, E. S. (2017) · Science · doi:10.1126/science.aag2132 · Cited by 230
Palma, A. D., Kuhlmann, M. T., Roberts, S. P. M., Potts, S. G., Börger, L., Hudson, L. N., Lysenko, I., Newbold, T., & Purvis, A. (2015) · Journal of Applied Ecology · doi:10.1111/1365-2664.12524 · Cited by 186
Wessel, M. J., Beanato, E., Popa, T., Windel, F., Vassiliadis, P., Menoud, P., Beliaeva, V., Violante, I. R., Abderrahmane, H., Dzialecka, P., et al. (2023) · Nature Neuroscience · doi:10.1038/s41593-023-01457-7 · Cited by 173
Low-fat, plant-based diet in multiple sclerosis: A randomized controlled trial
Yadav, V., Marracci, G., Kim, E., Spain, R., Cameron, M. H., Overs, S. E., Riddehough, A., Li, D. K. B., McDougall, J., Lovera, J. F., et al. (2016) · Multiple Sclerosis and Related Disorders · doi:10.1016/j.msard.2016.07.001 · Cited by 159
Destructive disinfection of infected brood prevents systemic disease spread in ant colonies
Pull, C. D., Ugelvig, L. V., Wiesenhofer, F. M., Grasse, A. V., Tragust, S., Schmitt, T., Brown, M. J. F., & Cremer, S. (2017) · eLife · doi:10.7554/elife.32073 · Cited by 147
Oscillatory brain activity in spontaneous and induced sleep stages in flies
Yap, M., Grabowska, M. J., Rohrscheib, C. E., Jeans, R., Troup, M., Paulk, A. C., van Alphen, B., Shaw, P. J., & van Swinderen, B. (2017) · Nature Communications · doi:10.1038/s41467-017-02024-y · Cited by 142
Li, Y.-X., Kortner, T., Chikwati, E., Belghit, I., Lock, E., & Krogdahl, Å. (2020) · Aquaculture · doi:10.1016/j.aquaculture.2020.734967 · Cited by 118
Six-month-old infants expect agents to minimize the cost of their actions
Liu, S., & Spelke, E. S. (2016) · Cognition · doi:10.1016/j.cognition.2016.12.007 · Cited by 118
Oelze, V. M., Head, J. S., Robbins, M. M., Richards, M. P., & Boesch, C. (2013) · Journal of Human Evolution · doi:10.1016/j.jhevol.2013.10.003 · Cited by 111
de Dios, V. R., Loik, M. E., Smith, R., Aspinwall, M. J., & Tissue, D. T. (2015) · Plant Cell & Environment · doi:10.1111/pce.12598 · Cited by 109
Reliability of marine faunal detections in drone-based monitoring
Colefax, A. P., Butcher, P. A., Pagendam, D., & Kelaher, B. P. (2019) · Ocean & Coastal Management · doi:10.1016/j.ocecoaman.2019.03.008 · Cited by 107
Bittencourt, P., Oliveira, R. S., da Costa, A. C. L., Giles, A. L., Coughlin, I., Costa, P. D. B., Bartholomew, D. C., Ferreira, L. V., Vasconcelos, S. S., Barros, F., et al. (2020) · Global Change Biology · doi:10.1111/gcb.15040 · Cited by 110
Wicki, M., Huber, R. A., & Bernauer, T. (2019) · Journal of Public Policy · doi:10.1017/s0143814x19000205 · Cited by 101
Goldberg, S. B., Imhoff-Smith, T., Bolt, D. M., Wilson-Mendenhall, C. D., Dahl, C. J., Davidson, R. J., & Rosenkranz, M. A. (2020) · JMIR Mental Health · doi:10.2196/23825 · Cited by 97
“Sensitivity analyses were also conducted with outliers (ie, 3 SD from the mean) and each participant sequentially removed [84].”
MethodNon-invasive stimulation of the human striatum disrupts reinforcement learning of motor skills
Vassiliadis, P., Beanato, E., Popa, T., Windel, F., Morishita, T., Neufeld, E., Duqué, J., Derosière, G., Wessel, M. J., & Hummel, F. C. (2024) · Nature Human Behaviour · doi:10.1038/s41562-024-01901-z · Cited by 97
Goldberg, S. B., Imhoff-Smith, T., Bolt, D. M., Wilson-Mendenhall, C. D., Dahl, C. J., Davidson, R. J., & Rosenkranz, M. A. (2020) · doi:10.2196/preprints.23825 · Cited by 97
The impact of ambient air pollution on the human blood metabolome
Vlaanderen, J. J., Janssen, N., Hoek, G., Keski‐Rahkonen, P., Barupal, D. K., Cassee, F. R., Gosens, I., Strak, M., Steenhof, M., Lan, Q., et al. (2017) · Environmental Research · doi:10.1016/j.envres.2017.03.042 · Cited by 97
A populist paradise? Examining populists’ Twitter adoption and use
Jacobs, K., & Spierings, N. (2018) · Information Communication & Society · doi:10.1080/1369118x.2018.1449883 · Cited by 94
Stoddard, J., Tseng, W., Kim, P., Chen, G., Yi, J. Y., Donahue, L. M., Brotman, M. A., Towbin, K. E., Pine, D. S., & Leibenluft, E. (2016) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2016.3282 · Cited by 93
Origins of the concepts cause, cost, and goal in prereaching infants
Liu, S., Brooks, N. B., & Spelke, E. S. (2019) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1904410116 · Cited by 88
Acute respiratory response to traffic-related air pollution during physical activity performance
Matt, F., Cole‐Hunter, T., Donaire-González, D., Kubesch, N. J., Martinez, D. R., Carrasco‐Turigas, G., & Nieuwenhuijsen, M. J. (2016) · Environment International · doi:10.1016/j.envint.2016.10.011 · Cited by 84
Taylor, C. T., Pearlstein, S. L., Kakaria, S., Lyubomirsky, S., & Stein, M. B. (2020) · Cognitive Therapy and Research · doi:10.1007/s10608-020-10102-7 · Cited by 81
Latent semantic analysis cosines as a cognitive similarity measure: Evidence from priming studies
Günther, F., Dudschig, C., & Kaup, B. (2015) · Quarterly Journal of Experimental Psychology · doi:10.1080/17470218.2015.1038280 · Cited by 79
Douglas, P. H., Hohmann, G., Murtagh, R., Thiessen-Bock, R., & Deschner, T. (2016) · BMC Evolutionary Biology · doi:10.1186/s12862-016-0691-3 · Cited by 78
Marks, E. A. N., Mattana, S., Alcañiz, J. M., Pérez-Herrero, E., & Domene, X. (2015) · Agriculture Ecosystems & Environment · doi:10.1016/j.agee.2015.09.004 · Cited by 78
Alm, S., Nelson, K., & Nieuwenhuis, R. (2019) · European Sociological Review · doi:10.1093/esr/jcz053 · Cited by 86
Self-citationCan videos affect learning outcomes? Evidence from an actual learning environment
Tani, M., Manuguerra, M., & Khan, S. A. (2022) · Educational Technology Research and Development · doi:10.1007/s11423-022-10147-3 · Cited by 72
Anderson, S. R., Gianola, M., Perry, J. M., & Losin, E. A. R. (2020) · Pain Medicine · doi:10.1093/pm/pnaa258 · Cited by 69
HLMdiag: A Suite of Diagnostics for Hierarchical Linear Models inR
Loy, A., & Hofmann, H. (2014) · Journal of Statistical Software · doi:10.18637/jss.v056.i05 · Cited by 69
Cooperation-based concept formation in male bottlenose dolphins
King, S. L., Connor, R. C., Krützen, M., & Allen, S. J. (2021) · Nature Communications · doi:10.1038/s41467-021-22668-1 · Cited by 71
Global coral reef ecosystems exhibit declining calcification and increasing primary productivity
Davis, K. L., Colefax, A. P., Tucker, J. P., Kelaher, B. P., & dos Santos, I. R. (2021) · Communications Earth & Environment · doi:10.1038/s43247-021-00168-w · Cited by 66
Moderated Online Data-Collection for Developmental Research: Methods and Replications
Chuey, A., Asaba, M., Bridgers, S., Carrillo, B. A., Dietz, G., Garcia, T., Leonard, J., Liu, S., Merrick, M., Radwan, S. F., et al. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.734398 · Cited by 66
Dainese, M., Riedinger, V., Holzschuh, A., Kleijn, D., Scheper, J. A., & Steffan-Dewenter, I. D. (2017) · Journal of Applied Ecology · doi:10.1111/1365-2664.12930 · Cited by 66
Separating EEG correlates of stress: Cognitive effort, time pressure, and social‐evaluative threat
Ehrhardt, N. M., Fietz, J., Kopf‐Beck, J., Kappelmann, N., & Brem, A. (2021) · European Journal of Neuroscience · doi:10.1111/ejn.15211 · Cited by 65
Integrated framework for characterization of spatial variability of geological profiles
Liu, W. F., Leung, Y. F., & Lo, M. K. (2016) · Canadian Geotechnical Journal · doi:10.1139/cgj-2016-0189 · Cited by 64
Brochu, K. K., van Dyke, M., Milano, N. J., Petersen, J. D., McArt, S. H., Nault, B. A., Keßler, A., & Danforth, B. N. (2020) · Scientific Reports · doi:10.1038/s41598-020-58274-2 · Cited by 67
Kölln, A. (2014) · Party Politics · doi:10.1177/1354068814550432 · Cited by 62
Aggression by male bonobos against immature individuals does not fit with predictions of infanticide
Hohmann, G., Vigilant, L., Mundry, R., Behringer, V., & Surbeck, M. (2019) · Aggressive Behavior · doi:10.1002/ab.21819 · Cited by 57
Modelling biodiversity responses to land use in areas of cocoa cultivation
Maney, C., Sassen, M., & Hill, S. L. L. (2021) · Agriculture Ecosystems & Environment · doi:10.1016/j.agee.2021.107712 · Cited by 57
Syntactic mixing across generations in an environment of community-wide bilingualism
Stoll, S., Zakharko, T., Moran, S., Schikowski, R., & Bickel, B. (2015) · Frontiers in Psychology · doi:10.3389/fpsyg.2015.00082 · Cited by 56
Abrahão, A., Lambers, H., Sawaya, A. C. H. F., Mazzafera, P., & Oliveira, R. S. (2014) · Oecologia · doi:10.1007/s00442-014-3033-4 · Cited by 56
STATISTICAL SENSITIVITY, COGNITIVE APTITUDES, AND PROCESSING OF COLLOCATIONS
Yi, W. (2018) · Studies in Second Language Acquisition · doi:10.1017/s0272263118000141 · Cited by 56
Bonobos modify communication signals according to recipient familiarity
Genty, É., Neumann, C., & Zuberbühler, K. (2015) · Scientific Reports · doi:10.1038/srep16442 · Cited by 53
Losin, E. A. R., Anderson, S. R., & Wager, T. D. (2017) · Journal of Pain · doi:10.1016/j.jpain.2017.02.428 · Cited by 56
Endogenous oxytocin, cortisol, and testosterone in response to group singing
Bowling, D. L., Gahr, J., Ancochea, P. G., Hoeschele, M., Canoine, V., Fusani, L., & Fitch, W. T. (2022) · Hormones and Behavior · doi:10.1016/j.yhbeh.2021.105105 · Cited by 51
Ebensperger, L. A., Correa, L. A., León, C., Ramírez‐Estrada, J., Abades, S. R., Villegas, Á., & Hayes, L. D. (2016) · Journal of Animal Ecology · doi:10.1111/1365-2656.12566 · Cited by 51
Lazzaroni, M., Marshall‐Pescini, S., Manzenreiter, H., Gosch, S., Přibilová, L., Darc, L., McGetrick, J., & Range, F. (2020) · Animal Cognition · doi:10.1007/s10071-020-01345-8 · Cited by 54
How foreign pressure affects mass mobilization in favor of authoritarian regimes
Hellmeier, S. (2020) · European Journal of International Relations · doi:10.1177/1354066120934527 · Cited by 39
The form and function of chimpanzee buttress drumming
Eleuteri, V., Henderson, M., Soldati, A., Badihi, G., Zuberbühler, K., & Hobaiter, C. (2022) · Animal Behaviour · doi:10.1016/j.anbehav.2022.07.013 · Cited by 54
Ecology dictates the value of memory for foraging bees
Pull, C. D., Petkova, I., Watrobska, C., Pasquier, G., Fernandez, M. P., & Leadbeater, E. (2022) · Current Biology · doi:10.1016/j.cub.2022.07.062 · Cited by 49
Hayward, K. S., Schmidt, J., Lohse, K. R., Peters, S., Bernhardt, J. A., Lannin, N. A., & Boyd, L. A. (2016) · NeuroImage Clinical · doi:10.1016/j.nicl.2016.09.015 · Cited by 49
Wild Goffin’s cockatoos flexibly manufacture and use tool sets
O’Hara, M., Mioduszewska, B., Mundry, R., Yohanna, Y., Haryoko, T., Rachmatika, R., Prawiradilaga, D. M., Huber, L., & Auersperg, A. M. I. (2021) · Current Biology · doi:10.1016/j.cub.2021.08.009 · Cited by 49
Andringa, W., Nieuwenhuis, R., & van Gerven, M. (2015) · International Journal of Sociology and Social Policy · doi:10.1108/ijssp-10-2014-0073 · Cited by 46
Self-citationLuncz, L. V., & Boesch, C. (2014) · American Journal of Physical Anthropology · doi:10.1002/ajpa.22628 · Cited by 48
The role of ants, birds and bats for ecosystem functions and yield in oil palm plantations
Denmead, L. H., Darras, K., Clough, Y., Diaz, P., Graß, I., Hoffmann, M. P., Nurdiansyah, F., Fardiansah, R., & Tscharntke, T. (2017) · Ecology · doi:10.1002/ecy.1882 · Cited by 45
Menghini, L., Yuksel, D., Goldstone, A., Baker, F. C., & de Zambotti, M. (2021) · Chronobiology International · doi:10.1080/07420528.2021.1903481 · Cited by 45
Prenatal Antidepressant Exposure and Offspring Brain Morphologic Trajectory
Koc, D., Tiemeier, H., Stricker, B. H., Muetzel, R. L., Hillegers, M., & El Marroun, H. (2023) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2023.3161 · Cited by 44
A Very Basic Tutorial for Performing Linear Mixed Effects Analyses: Tutorial 2
Winter, B. (2018) · QUBES · doi:10.25334/q4w716 · Cited by 43
Griep, Y., Lukic, A., Kraak, J. M., Bohle, S. L., Jiang, L., Elst, T. V., & Witte, H. D. (2021) · Journal of Business Research · doi:10.1016/j.jbusres.2020.12.045 · Cited by 49
Limited heat tolerance in a cold-adapted seabird: implications of a warming Arctic
Choy, E. S., O’Connor, R. S., Gilchrist, H. G., Hargreaves, A. L., Love, O. P., Vézina, F., & Elliott, K. H. (2021) · Journal of Experimental Biology · doi:10.1242/jeb.242168 · Cited by 43
Sheppard, J. P., Ong, V., Lagman, C., Udawatta, M., Duong, C., Nguyen, T. P., Prashant, G. N., Plurad, D. S., Kim, D. Y., & Yang, I. (2018) · Neurosurgery · doi:10.1093/neuros/nyy522 · Cited by 41
Wilcox, T. M., Young, M. K., McKelvey, K. S., Isaak, D. J., Horan, D. L., & Schwartz, M. K. (2018) · Ecosphere · doi:10.1002/ecs2.2500 · Cited by 42
Within‐individual repeatability in telomere length: A meta‐analysis in nonmammalian vertebrates
Kärkkäinen, T., Briga, M., Laaksonen, T., & Stier, A. (2021) · Molecular Ecology · doi:10.1111/mec.16155 · Cited by 44
Audience effect on domestic dogs’ behavioural displays and facial expressions
Pedretti, G., Canori, C., Marshall‐Pescini, S., Palme, R., Pelosi, A., & Valsecchi, P. (2022) · Scientific Reports · doi:10.1038/s41598-022-13566-7 · Cited by 40
Kelly, T. R., MacGillivray, H. L., Sarquis‐Adamson, Y., Watson, M. J., Hobson, K. A., & MacDougall‐Shackleton, E. A. (2016) · Behavioral Ecology and Sociobiology · doi:10.1007/s00265-016-2191-2 · Cited by 40
Garamszegi, L. Z., Mueller, J. C., Markó, G., Szász, E., Zsebők, S., Herczeg, G., Eens, M., & Török, J. (2014) · Ecology and Evolution · doi:10.1002/ece3.1041 · Cited by 39
Dröscher, I., & Kappeler, P. M. (2014) · Behavioral Ecology and Sociobiology · doi:10.1007/s00265-014-1810-z · Cited by 38
Bufford, J. L., Hulme, P. E., Sikes, B. A., Cooper, J., Johnston, P. R., & Duncan, R. P. (2016) · New Phytologist · doi:10.1111/nph.14077 · Cited by 37
Luke, S. G., Darowski, E. S., & Gale, S. D. (2018) · Memory & Cognition · doi:10.3758/s13421-018-0798-4 · Cited by 37
Andersson, E., Crowley, J. J., Lindefors, N., Ljótsson, B., Hedman‐Lagerlöf, E., Boberg, J., Alaoui, S. E., Karlsson, R., Lu, Y., Mattheisen, M., et al. (2018) · Molecular Psychiatry · doi:10.1038/s41380-018-0289-9 · Cited by 37
Sanchez, M., Milà, C., Sreekanth, V., Balakrishnan, K., Sambandam, S., Nieuwenhuijsen, M. J., Kinra, S., Marshall, J. D., & Tonne, C. C. (2019) · Journal of Exposure Science & Environmental Epidemiology · doi:10.1038/s41370-019-0150-5 · Cited by 35
Below the canopy: global trends in forest vertebrate populations and their drivers
Green, E. J., McRae, L., Freeman, R., Harfoot, M. B. J., Hill, S. L. L., Baldwin-Cantello, W., & Simonson, W. D. (2020) · Proceedings of the Royal Society B Biological Sciences · doi:10.1098/rspb.2020.0533 · Cited by 38
Clemente, A., Vila‐Vidal, M., Pearce, M. T., Aguiló, G., Corradi, G. B., & Nadal, M. (2020) · Behavior Research Methods · doi:10.3758/s13428-019-01329-8 · Cited by 35
Honey bees increase social distancing when facing the ectoparasite Varroa destructor
Pusceddu, M., Cini, A., Alberti, S., Salaris, E., Theodorou, P., Floris, I., & Satta, A. (2021) · Science Advances · doi:10.1126/sciadv.abj1398 · Cited by 34
Is there such a thing as too long childcare leave?
Nieuwenhuis, R., Need, A., & van der Kolk, H. (2017) · International Journal of Sociology and Social Policy · doi:10.1108/ijssp-07-2015-0074 · Cited by 34
Self-citationAge and social affinity effects on contact call interactions in free-ranging spider monkeys
Jaramillo, M. B., Ramos‐Fernández, G., Palacios-Romo, T. M., Sosa-López, J. R., & Lemasson, A. (2018) · Behavioral Ecology and Sociobiology · doi:10.1007/s00265-018-2615-2 · Cited by 34
Klaus, J., Mädebach, A., Oppermann, F., & Jescheniak, J. D. (2016) · Quarterly Journal of Experimental Psychology · doi:10.1080/17470218.2016.1167926 · Cited by 33
Talking to Cows: Reactions to Different Auditory Stimuli During Gentle Human-Animal Interactions
Lange, A., Bauer, L., Futschik, A., Waiblinger, S., & Lürzel, S. (2020) · Frontiers in Psychology · doi:10.3389/fpsyg.2020.579346 · Cited by 36
Hood, A. S. C., Aryawan, A. A. K., Advento, A. D., Purnomo, D., Wahyuningsih, R., Luke, S. H., Ps, S., Snaddon, J. L., Foster, W. A., Caliman, J., et al. (2019) · Frontiers in Forests and Global Change · doi:10.3389/ffgc.2019.00051 · Cited by 32
O’Connor, R. S., Pogam, A. L., Young, K. G., Robitaille, F., Choy, E. S., Love, O. P., Elliott, K. H., Hargreaves, A. L., Berteaux, D., Tam, A., et al. (2021) · Ecology and Evolution · doi:10.1002/ece3.7141 · Cited by 32
Hedgerows as a habitat for forest plant species in the agricultural landscape of Europe
Litza, K., Alignier, A., Closset‐Kopp, D., Ernoult, A., Mony, C., Osthaus, M., Staley, J. T., Berge, S. V. D., Vanneste, T., & Diekmann, M. (2021) · Agriculture Ecosystems & Environment · doi:10.1016/j.agee.2021.107809 · Cited by 32
Dominance, gender, and season influence food patch use in a group-living, solitary foraging canid
Dorning, J., & Harris, S. (2017) · Behavioral Ecology · doi:10.1093/beheco/arx092 · Cited by 32
Social immunity modulates competition between coinfecting pathogens
Milutinović, B., Stock, M., Grasse, A. V., Naderlinger, E., Hilbe, C., & Cremer, S. (2020) · Ecology Letters · doi:10.1111/ele.13458 · Cited by 31
Snijders, L., van Oers, K., & Naguib, M. (2017) · Ecology and Evolution · doi:10.1002/ece3.2686 · Cited by 31
Accelerated seed dispersal along linear disturbances in the Canadian oil sands region
Roberts, D. R., Ciuti, S., Barber, Q. E., Willier, C. N., & Nielsen, S. E. (2018) · Scientific Reports · doi:10.1038/s41598-018-22678-y · Cited by 31
Kunz, J. A., Duvot, G. J., van Noordwijk, M. A., Willems, E. P., Townsend, M., Mardianah, N., Utami-Atmoko, S. S., Vogel, E. R., Nugraha, R. T. P., Heistermann, M., et al. (2020) · Behavioral Ecology and Sociobiology · doi:10.1007/s00265-020-02948-4 · Cited by 31
Malaria parasites, immune challenge, MHC variability, and predator avoidance in a passerine bird
Garamszegi, L. Z., Zagalska‐Neubauer, M., Cañal, D., Markó, G., Szász, E., Zsebők, S., Szöllősi, E., Herczeg, G., & Török, J. (2015) · Behavioral Ecology · doi:10.1093/beheco/arv077 · Cited by 31
Eleuteri, V., Bates, L. A., Rendle-Worthington, J., Hobaiter, C., & Stoeger, A. S. (2024) · Communications Biology · doi:10.1038/s42003-024-06133-5 · Cited by 30
The Foraging Costs of Mating Effort in Male Chimpanzees (Pan troglodytes schweinfurthii)
Georgiev, A. V., Russell, A. F., Thompson, M. E., Otali, E., Muller, M. N., & Wrangham, R. W. (2014) · International Journal of Primatology · doi:10.1007/s10764-014-9788-y · Cited by 30
Shared syntax and cross-linguistic influence in bilingual children
Unsworth, S. (2023) · Linguistic Approaches to Bilingualism · doi:10.1075/lab.22093.uns · Cited by 30
Melis, M., Schroyen, G., Leenaerts, N., Smeets, A., Sunaert, S. G., der Gucht, K. V., & Deprez, S. (2023) · Cancer · doi:10.1002/cncr.34640 · Cited by 29
Kopp, K. S., & Liebal, K. (2016) · Behavioral Ecology and Sociobiology · doi:10.1007/s00265-016-2130-2 · Cited by 31
Male–male social bonding, coalitionary support and reproductive success in wild Guinea baboons
Pesco, F. D., Trede, F., Zinner, D. P., & Fischer, J. (2022) · Proceedings of the Royal Society B Biological Sciences · doi:10.1098/rspb.2022.0347 · Cited by 29
Touitou, S., Heistermann, M., Schülke, O., & Ostner, J. (2021) · Hormones and Behavior · doi:10.1016/j.yhbeh.2021.104968 · Cited by 29
Effects of Different Stroking Styles on Behaviour and Cardiac Parameters in Heifers
Lange, A., Franzmayr, S., Wisenöcker, V., Futschik, A., Waiblinger, S., & Lürzel, S. (2020) · Animals · doi:10.3390/ani10030426 · Cited by 30
Mühlenbeck, C., Liebal, K., Pritsch, C., & Jacobsen, T. (2016) · Frontiers in Psychology · doi:10.3389/fpsyg.2016.00408 · Cited by 28
Isbell, D. R., Winke, P. M., & Gass, S. M. (2018) · Language Testing · doi:10.1177/0265532218798139 · Cited by 28
Metadata
- Authors
- Nieuwenhuis, R., Grotenhuis, M., & Pelzer, B.
- Venue
- The R Journal
- Publisher
- The R Foundation
- Published
- 2012
- Volume
- 4
- Issue
- 2
- Pages
- 38
- ISSN
- 2073-4859 (ISSN-L)
- OpenAlex
- W1549371580
- Semantic Scholar
- c6d41ed109eb79b026ba2d1fb769b6ebf525ebd4
- Cited by
- 442
- References
- 18
- Citations by year
- 2026: 29 · 2025: 36 · 2024: 45 · 2023: 38 · 2022: 64 · 2021: 63 · 2020: 33 · 2019: 28 · 2018: 24 · 2017: 26 · 2016: 22 · 2015: 17 · 2014: 12 · 2013: 5
- FWCI
- 10.39
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- The R Journal (publishedVersion, cc-by)
- Lincoln (University of Nebraska) (submittedVersion, cc-by)
- Radboud Repository (Radboud University) (submittedVersion)
- University of Twente Research Information (submittedVersion)
- Radboud Repository (Radboud University) (submittedVersion, other-oa)
- University of Twente Research Information (submittedVersion, other-oa)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Statistical Methods and Models
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Physical Sciences › Mathematics › Statistics and Probability › Data Analysis with R
- Keywords
- mixed models, generalized linear mixed models, Cook's distance, nested data, influence diagnostics, R package
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-09, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-09 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
