The DOI matches its Crossref record; 96 of the 96 references it cites were found in a registry or an index; none of the 79 sources its claims cite is checked for support yet.
An introduction to recursive partitioning: Rationale, application, and characteristics of classification and regression trees, bagging, and random forests.
Open accessAbstract
Recursive partitioning methods have become popular and widely used tools for nonparametric regression and classification in many scientific fields. Especially random forests, which can deal with large numbers of predictor variables even in the presence of complex interactions, have been applied successfully in genetics, clinical medicine, and bioinformatics within the past few years. High-dimensional problems are common not only in genetics, but also in some areas of psychological research, where only a few subjects can be measured because of time or cost constraints, yet a large amount of data is generated for each subject. Random forests have been shown to achieve a high prediction accuracy in such applications and to provide descriptive variable importance measures reflecting the impact of each variable in both main effects and interactions. The aim of this work is to introduce the principles of the standard recursive partitioning methods as well as recent methodological improvements, to illustrate their usage for low and high-dimensional data exploration, but also to point out limitations of the methods and potential pitfalls in their practical application. Application of the methods is illustrated with freely available implementations in the R system for statistical computing.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
This review introduces recursive partitioning methods—classification and regression trees, bagging, and random forests—for psychological and other data, emphasizing their use in low- and high-dimensional problems. It illustrates their application in R, explains variable importance, and warns about practical pitfalls such as instability, biased variable selection, and misleading significance tests.
Key findings
- Recursive partitioning recursively splits the feature space into rectangular regions with similar response values; ensembles such as bagging and random forests average many trees, and random forests additionally randomly preselect predictors at each split, making them suitable for “small n large p” problems and complex interactions.
- In the artificial smoking example with 200 adolescents, the first split on `friends_smoke` isolates 92 adolescents with “none” smoking friends (about 15% intending to smoke), while the remaining 108 are split by `alcohol_per_month` into groups with about 30% and more than 60% “yes” responses; predicted probabilities are approximately 15%, 30%, and 65%.
- A random forest with `ntree=500` and `mtry=2` on the smoking data achieved 74.5% standard prediction accuracy and 71.5% out-of-bag accuracy, with the out-of-bag estimate described as more conservative; in that artificial example, bagging and even a single tree would perform equally well because the only induced effect was the interaction already identified.
- In the gene-expression example, 61 samples (30 cases, 31 controls) from 22,283 genes were reduced to 100 randomly selected genes plus two artificially modified genes and three covariates (105 variables); permutation importance clearly identified the two modified genes, while learning-sample accuracy was over-optimistic at 90.16% versus out-of-bag accuracy of 67.21%, and logistic regression would not converge even on the reduced data set.
Methods
- The paper is an instructive review/tutorial of recursive partitioning, covering CART/C4.5 trees, bagging, random forests, model-based recursive partitioning, prediction, variable importance, and software in R.
- Illustrative examples include an artificial smoking data set generated to resemble key results of Kitsantas et al. (2007) with 200 adolescents and four predictors; a model-based partitioning example using sleep-deprivation reaction-time data from four subjects; and a gene-expression example based on Ryan et al. (2006) with 61 bipolar-disorder case-control samples.
- Main analysis is conceptual and example-based: it explains splitting, stopping, impurity reduction, ensemble construction, out-of-bag error, permutation importance, and practical pitfalls, with R code provided as a supplement.
Limitations
- Stated: random forests were not developed in a stringent statistical framework, so their properties are less predictable than standard parametric methods and some parts remain “under construction”; variable selection and importance can be biased with mixed predictor types; permutation importance is not defined for variables with missing values; tests for variable importance based on the z-score have poor statistical properties; tree probability estimates lack confidence intervals unless bootstrapping/refitting is used; and the functional shape of random forests is a “black-box” that cannot be interpreted analytically.
- Stated: overfitting of random forests is not fully excluded by real-data case studies, and tuning parameters (`ntree`, `mtry`) affect stability and variable-importance reliability; results can vary between runs unless the random seed is fixed.
- Evident from the text: the numerical examples are illustrative rather than generalizable—the smoking data are artificial, the gene data are reduced to 100 randomly selected genes with two genes artificially modified, and the sleep example uses only four subjects—so the reported accuracies and importance rankings are demonstrations, not population estimates.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
0 of 79 cited sources checked · 0 supported · 0 flagged
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“it is illustrated in Figure 9 that the effect of sleep deprivation on reaction time differs between subjects.”
§ An Introduction to Recursive Partitioning: Rationale, Application and Characteristics of Classification and Regression Trees, Bagging and Random Forests > Further Application Examples > Model-Based Recursive Partitioning
Tests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesThe paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In our artificial example, bagging, and even a single tree, would actually perform equally well, because the interaction of friends_smoke and alcohol_per_month, that was already correctly identified by the single tree, is the only effect that was induced in the data.”
§ An Introduction to Recursive Partitioning: Rationale, Application and Characteristics of Classification and Regression Trees, Bagging and Random Forests > The Methods > Predictions from Ensembles of Trees
Strength:hedged· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesThe paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“a logistic regression model would not be applicable in a sample of this size, even for the reduced data set with only 102 genes – not even if forward selection was employed and only main effects were considered, disregarding the possibility of interaction effects – because the estimation algorithm does not converge.”
§ An Introduction to Recursive Partitioning: Rationale, Application and Characteristics of Classification and Regression Trees, Bagging and Random Forests > Further Application Examples > Random Forests
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The standard and out-of-bag prediction accuracy of a random forests with ntree=500 and mtry=2 for our smoking data example is 74.5% and 71.5% respectively”
§ An Introduction to Recursive Partitioning: Rationale, Application and Characteristics of Classification and Regression Trees, Bagging and Random Forests > The Methods > Predictions from Ensembles of Trees
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“in our simple example the two relevant predictor variables friends_smoke and alcohol_per_month are correctly identified by the permutation variable importance of both bagging and random forests”
§ An Introduction to Recursive Partitioning: Rationale, Application and Characteristics of Classification and Regression Trees, Bagging and Random Forests > The Methods > Variable Importance
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The effects of the two artificially modified genes can be clearly identified.”
§ An Introduction to Recursive Partitioning: Rationale, Application and Characteristics of Classification and Regression Trees, Bagging and Random Forests > Further Application Examples > Random Forests
Strength:boosted· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesA conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, even in studies with much lower numbers of predictor variables, the combination of all main and interaction effects of interest – especially in the case of categorical predictor variables – may well lead to cell counts too sparse for parameter convergence.”
§ An Introduction to Recursive Partitioning: Rationale, Application and Characteristics of Classification and Regression Trees, Bagging and Random Forests > Scope of This Work
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesNone of the sources cited for this claim have been checked yet.
In the paper
“For detecting relevant predictor variables, Rossi et al. (2005) point out that the random forest variable importance ranking proves to be more stable than stepwise variable selection approaches available for logistic regression, that are known to be affected by order effects”
§ An Introduction to Recursive Partitioning: Rationale, Application and Characteristics of Classification and Regression Trees, Bagging and Random Forests > Scope of This Work
What the sources say
- Not checked yetcites Rossi et al. (2005)
- Not checked yetcites Freedman 1983
- Not checked yetcites Derksen and Keselman 1992
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesA conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“One situation where parametric approaches are no longer applicable is the so called ”small n large p” case, where the number of predictor variables p is greater than the number of subjects n.”
§ An Introduction to Recursive Partitioning: Rationale, Application and Characteristics of Classification and Regression Trees, Bagging and Random Forests > Scope of This Work
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesA conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, in certain situations these classical methods can be subject to severe limitations.”
§ An Introduction to Recursive Partitioning: Rationale, Application and Characteristics of Classification and Regression Trees, Bagging and Random Forests > Scope of This Work
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
- It may state two separate claims as one. 39
- Probably not a substantial claim. 40
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“due to the random restriction, the trees have become even more diverse; for example the strong predictor variable friends_smoke is no longer chosen for the first split in every single tree.”
§ An Introduction to Recursive Partitioning: Rationale, Application and Characteristics of Classification and Regression Trees, Bagging and Random Forests > The Methods > How Do Ensemble Methods Work?
Tests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“in the original smoking data, those adolescents who drank alcohol in more occasions are more likely to intend to smoke. Randomly permuting the values of alcohol_per_month over all subjects, however, destroys this association. Accordingly, when the permuted variable, together with the remaining unpermuted predictor variables, is now used to predict the response, the prediction accuracy decreases su”
§ An Introduction to Recursive Partitioning: Rationale, Application and Characteristics of Classification and Regression Trees, Bagging and Random Forests > The Methods > Variable Importance
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“We find that the reactions to sleep deprivation of subjects 308 and 350 can be represented by one joint model, while the reactions of the other subjects are represented by distinct models.”
§ An Introduction to Recursive Partitioning: Rationale, Application and Characteristics of Classification and Regression Trees, Bagging and Random Forests > Further Application Examples > Model-Based Recursive Partitioning
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For the entire learning sample the prediction accuracy estimate is over-optimistic (90.16%), while the estimate based on the out-of-bag sample is more conservative (67.21%).”
§ An Introduction to Recursive Partitioning: Rationale, Application and Characteristics of Classification and Regression Trees, Bagging and Random Forests > Further Application Examples > Random Forests
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As opposed to that, random forests can process large numbers of predictor variables simultaneously and provide individual measures of variable importance.”
§ An Introduction to Recursive Partitioning: Rationale, Application and Characteristics of Classification and Regression Trees, Bagging and Random Forests > Scope of This Work
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“even though recent theoretical results disprove this and indicate that other tuning parameters may be responsible for overfitting in random forests.”
§ An Introduction to Recursive Partitioning: Rationale, Application and Characteristics of Classification and Regression Trees, Bagging and Random Forests > Scope of This Work
Strength:hedged· not yet calibratedCausal:conditional· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Random forests have been shown to achieve a high prediction accuracy in such applications, and provide descriptive variable importance measures reflecting the impact of each variable in both main effects and interactions.”
§ An Introduction to Recursive Partitioning: Rationale, Application and Characteristics of Classification and Regression Trees, Bagging and Random Forests > Abstract
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. The paper may give no reason why it holds.
In the paper
“Some of these pitfalls are promoted by the fact that random forests were not developed in a stringent statistical framework, so that their properties are less predictable than those of standard parametric methods, and some parts of random forests are still “under construction””
§ An Introduction to Recursive Partitioning: Rationale, Application and Characteristics of Classification and Regression Trees, Bagging and Random Forests > Scope of This Work
What the sources say
- Not checked yetcites Polikar 2006
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Additional limitations of many standard approaches include the restricted functional form of the association pattern (with the linear model as the most common and most restrictive case)”
§ An Introduction to Recursive Partitioning: Rationale, Application and Characteristics of Classification and Regression Trees, Bagging and Random Forests > Scope of This Work
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. The paper may give no reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“In the past years, recursive partitioning methods have gained popularity as a means of multivariate data exploration in various scientific fields, including, e.g., the analysis of microarray data, DNA sequencing and many other applications in genetics, epidemiology and medicine”
§ An Introduction to Recursive Partitioning: Rationale, Application and Characteristics of Classification and Regression Trees, Bagging and Random Forests > Scope of This Work
What the sources say
- Not checked yetcites Gunther et al. 2003
- Not checked yetcites Lunetta et al. 2004
- Not checked yetcites Segal et al. 2004
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“the fact that ordinally scaled variables, which are particularly common in psychological applications, are often treated as if they were measured on an interval or ratio scale”
§ An Introduction to Recursive Partitioning: Rationale, Application and Characteristics of Classification and Regression Trees, Bagging and Random Forests > Scope of This Work
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. The paper may give no reason why it holds.
In the paper
“As another advantage, Marinic, Supek, Kovacic, Rukavina, Jendricko, and Kozaric-Kovacic (2007) point out in an application to the diagnosis of posttraumatic stress disorder, that random forests can be used to automatically generate realistic estimates of the prediction accuracy on test data by means of repeated random sampling from the learning data.”
§ An Introduction to Recursive Partitioning: Rationale, Application and Characteristics of Classification and Regression Trees, Bagging and Random Forests > Scope of This Work
What the sources say
- Not checked yetcites Marinic et al. (2007)
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“and that measures of variable importance are only available for a small range of methods.”
§ An Introduction to Recursive Partitioning: Rationale, Application and Characteristics of Classification and Regression Trees, Bagging and Random Forests > Scope of This Work
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Especially random forests, that can deal with large numbers of predictor variables even in the presence of complex interactions, have been applied successfully in genetics, clinical medicine and bioinformatics within the past few years.”
§ An Introduction to Recursive Partitioning: Rationale, Application and Characteristics of Classification and Regression Trees, Bagging and Random Forests > Abstract
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. The paper may give no reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“classification trees are instable but on average produce the right prediction (i.e. trees are unbiased predictors), which has been supported by several empirical as well as simulation studies”
§ An Introduction to Recursive Partitioning: Rationale, Application and Characteristics of Classification and Regression Trees, Bagging and Random Forests > The Methods > How Do Ensemble Methods Work?
What the sources say
- Not checked yetcites Breiman 1996a, 1998
- Not checked yetcites Bauer and Kohavi 1999
- Not checked yetcites Dietterich 2000
Load-bearingStrength:boosted· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
In the paper
“Pruning is not discussed in detail here, because the unbiased classification tree algorithm of Hothorn et al. (2006), that is used here for illustration, employs p-values for variable selection and as a stopping criterion and therefore does not rely on pruning.”
§ An Introduction to Recursive Partitioning: Rationale, Application and Characteristics of Classification and Regression Trees, Bagging and Random Forests > The Methods > How Do Classification and Regression Trees Work?
What the sources say
- Not checked yetcites Hothorn et al. (2006)
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“While this kind of “asymmetric” interaction is very common in classification trees, it is extremely unlikely to actually discover a symmetric interaction pattern as that in Figure 4 (right) or even a main effect pattern as that in Figure 4 (left) in real data.”
§ An Introduction to Recursive Partitioning: Rationale, Application and Characteristics of Classification and Regression Trees, Bagging and Random Forests > The Methods > How Do Classification and Regression Trees Work?
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The rationale behind ensemble methods is to base the prediction on a whole set of classification or regression trees, rather than a single tree.”
§ An Introduction to Recursive Partitioning: Rationale, Application and Characteristics of Classification and Regression Trees, Bagging and Random Forests > The Methods > How Do Ensemble Methods Work?
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The related methods bagging and random forests vary only in the way this set of trees is constructed.”
§ An Introduction to Recursive Partitioning: Rationale, Application and Characteristics of Classification and Regression Trees, Bagging and Random Forests > The Methods > How Do Ensemble Methods Work?
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, one should be aware that in this case significance tests do not work in the same way as in a designed study, where a limited number of hypotheses to be tested are specified in advance. In common forward and/or backward stepwise regression it is not known beforehand how many significance tests will have to be conducted. Therefore, it is hard to control the overall significance level”
§ An Introduction to Recursive Partitioning: Rationale, Application and Characteristics of Classification and Regression Trees, Bagging and Random Forests > The Methods > How Do Classification and Regression Trees Work?
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We will see later that ensemble methods, by combining the predictions of many single trees, can approximate functions more smoothly, too.”
§ An Introduction to Recursive Partitioning: Rationale, Application and Characteristics of Classification and Regression Trees, Bagging and Random Forests > The Methods > How Do Classification and Regression Trees Work?
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Therefore it is stated in the literature that classification trees cannot (or rather, are extremely unlikely to) represent additive functions that consist only of main effects, while they are perfectly well suited for representing complex interactions.”
§ An Introduction to Recursive Partitioning: Rationale, Application and Characteristics of Classification and Regression Trees, Bagging and Random Forests > The Methods > How Do Classification and Regression Trees Work?
Strength:hedged· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
In the paper
“Most modern classification tree algorithms rely on this strategy, and employ the p-values of association tests for variable and cutpoint selection. This approach has additional advantages over the original impurity reduction approach, as outlined below.”
§ An Introduction to Recursive Partitioning: Rationale, Application and Characteristics of Classification and Regression Trees, Bagging and Random Forests > The Methods > How Do Classification and Regression Trees Work?
What the sources say
- Not checked yetcites Hothorn, Hornik, and Zeileis 2006
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Note, however, that no confidence intervals are available for the estimates, unless, e.g., bootstrapping is used in combination with refitting to assess the variability of the prediction.”
§ An Introduction to Recursive Partitioning: Rationale, Application and Characteristics of Classification and Regression Trees, Bagging and Random Forests > The Methods > How Do Classification and Regression Trees Work?
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Even though the idea of piecewise constant functions may appear very inflexible, such functions can be used to approximate any functional form, in particular nonlinear and nonmonotone functions.”
§ An Introduction to Recursive Partitioning: Rationale, Application and Characteristics of Classification and Regression Trees, Bagging and Random Forests > The Methods > How Do Classification and Regression Trees Work?
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This is in strong contrast to classical linear or additive regression, where the effects of predictors are restricted to the additive form – the interpretation of which may appear easier, but which may also produce severe artifacts, since in many complex applications the true data generating mechanism is neither linear nor additive.”
§ An Introduction to Recursive Partitioning: Rationale, Application and Characteristics of Classification and Regression Trees, Bagging and Random Forests > The Methods > How Do Classification and Regression Trees Work?
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
- Rests on an assumption: In many complex applications, the true data generating mechanism is neither linear nor additive.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In both bagging and random forests a set of trees is built on random samples of the learning sample: In each step of the algorithm, either a bootstrap sample (of the same size, drawn with replacement) or a subsample (of smaller size, drawn without replacement) of the learning sample is drawn randomly, and an individual tree is grown on each sample.”
§ An Introduction to Recursive Partitioning: Rationale, Application and Characteristics of Classification and Regression Trees, Bagging and Random Forests > The Methods > How Do Ensemble Methods Work?
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
- Rests on an assumption: Each random sample drawn from the learning sample reflects the same data generating process as the original learning sample.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In addition to this, ensemble methods, that are our main focus here, usually employ unpruned trees.”
§ An Introduction to Recursive Partitioning: Rationale, Application and Characteristics of Classification and Regression Trees, Bagging and Random Forests > The Methods > How Do Classification and Regression Trees Work?
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
In the paper
“Lin and Jeon (2006) explore the statistical properties of random forests by means of placing them in a k-nearest neighbor (k-NN) framework, where random forests can be viewed as adaptively weighted k-NN with the terminal node size determining the size of neighborhood.”
§ An Introduction to Recursive Partitioning: Rationale, Application and Characteristics of Classification and Regression Trees, Bagging and Random Forests > The Methods > How Do Ensemble Methods Work?
What the sources say
- Not checked yetcites Lin and Jeon (2006)
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In an ensemble of trees the predictions of all individual trees need to be combined. This is usually accomplished by means of (weighted or unweighted) averaging in regression or voting in classification.”
§ An Introduction to Recursive Partitioning: Rationale, Application and Characteristics of Classification and Regression Trees, Bagging and Random Forests > The Methods > Predictions from Ensembles of Trees
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“there is no diagnostic for the unanimity of the vote.”
§ An Introduction to Recursive Partitioning: Rationale, Application and Characteristics of Classification and Regression Trees, Bagging and Random Forests > The Methods > Predictions from Ensembles of Trees
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“random forests include bagging as the special case where the number of randomly preselected splitting variables is equal to the overall number of variables.”
§ An Introduction to Recursive Partitioning: Rationale, Application and Characteristics of Classification and Regression Trees, Bagging and Random Forests > The Methods > How Do Ensemble Methods Work?
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“in order to be able to mathematically grasp a computationally complex method like random forests, involving several degrees of random sampling, several simplifying assumptions are necessary.”
§ An Introduction to Recursive Partitioning: Rationale, Application and Characteristics of Classification and Regression Trees, Bagging and Random Forests > The Methods > How Do Ensemble Methods Work?
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“It has been shown to outperform single trees and even to reach the predictive performance of bagging, but in general cannot compete because it becomes computationally infeasible for large sets of trees that are standard in today’s ensemble methods.”
§ An Introduction to Recursive Partitioning: Rationale, Application and Characteristics of Classification and Regression Trees, Bagging and Random Forests > The Methods > How Do Ensemble Methods Work?
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
In the paper
“variable selection in a single tree is affected by order effects similar to those present in stepwise variable selection approaches for parametric regression (that is also instable against random variation of the learning data”
§ An Introduction to Recursive Partitioning: Rationale, Application and Characteristics of Classification and Regression Trees, Bagging and Random Forests > The Methods > How Do Ensemble Methods Work?
What the sources say
- Not checked yetcites Austin and Tu 2004
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“The same effect can be achieved not only by means of bootstrap sampling as in standard bagging, but also by means of subsampling (Grandvalet 2004), that is preferable in many applications because it guarantees unbiased variable selection”
§ An Introduction to Recursive Partitioning: Rationale, Application and Characteristics of Classification and Regression Trees, Bagging and Random Forests > The Methods > How Do Ensemble Methods Work?
What the sources say
- Not checked yetcites Grandvalet 2004
- Not checked yetcites Strobl, Boulesteix, Zeileis, and Hothorn 2007
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
In the paper
“the improvement in the prediction is achieved by means of smoothing the hard cut decision boundaries created by splitting in single classification trees, which in return reduces the variance of the prediction”
§ An Introduction to Recursive Partitioning: Rationale, Application and Characteristics of Classification and Regression Trees, Bagging and Random Forests > The Methods > How Do Ensemble Methods Work?
What the sources say
- Not checked yetcites Bühlmann and Yu (2002)
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“Ensemble construction can also be viewed in the context of Bayesian model averaging”
§ An Introduction to Recursive Partitioning: Rationale, Application and Characteristics of Classification and Regression Trees, Bagging and Random Forests > The Methods > How Do Ensemble Methods Work?
What the sources say
- Not checked yetcites Domingos 1997
- Not checked yetcites Hoeting, Madigan, Raftery, and Volinsky 1999
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“the random selection of splitting variables in random forests allows predictor variables, that were otherwise outplayed by a stronger competitor, to enter the ensemble: If the stronger competitor cannot be selected, a new variable has a chance to be included in the model – and may reveal interaction effects with other variables that otherwise would have been missed.”
§ An Introduction to Recursive Partitioning: Rationale, Application and Characteristics of Classification and Regression Trees, Bagging and Random Forests > The Methods > How Do Ensemble Methods Work?
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
In the paper
“the key effect of bagging is that it equalizes the influence of particular observations – which proves beneficial in the case of “bad” leverage points, but may be harmful when “good” leverage points, that could improve the model fit, are downweighted.”
§ An Introduction to Recursive Partitioning: Rationale, Application and Characteristics of Classification and Regression Trees, Bagging and Random Forests > The Methods > How Do Ensemble Methods Work?
What the sources say
- Not checked yetcites Grandvalet (2004)
Strength:hedged· not yet calibratedCausal:conditional· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“single classification trees are easily interpretable, both intuitively at first glance and descriptively when looking in detail at the tree structure.”
§ An Introduction to Recursive Partitioning: Rationale, Application and Characteristics of Classification and Regression Trees, Bagging and Random Forests > The Methods > Variable Importance
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“the visual inspection of the model-based partition, that requires no further assumptions, can provide a helpful first glance impression of different response patterns present in the sample”
§ An Introduction to Recursive Partitioning: Rationale, Application and Characteristics of Classification and Regression Trees, Bagging and Random Forests > Further Application Examples > Model-Based Recursive Partitioning
Strength:plain· not yet calibratedTests✓ Asserted: passes✗ Checkable: fails✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“ensembles of trees are not easy to interpret at all, because the individual trees in them are not nested in any way: Each variable may appear at different positions, if at all, in different trees, as depicted in Figures 6 and 7, so that there is no such thing as an “average tree” that could be visualized for interpretation.”
§ An Introduction to Recursive Partitioning: Rationale, Application and Characteristics of Classification and Regression Trees, Bagging and Random Forests > The Methods > Variable Importance
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“using the subject ID as a pseudo covariate as in this example resembles a latent class approach for identifying groups of subjects with similar response patterns.”
§ An Introduction to Recursive Partitioning: Rationale, Application and Characteristics of Classification and Regression Trees, Bagging and Random Forests > Further Application Examples > Model-Based Recursive Partitioning
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“More elaborate variable importance measures incorporate a (weighted) mean of the individual trees’ improvement in the splitting criterion produced by each variable”
§ An Introduction to Recursive Partitioning: Rationale, Application and Characteristics of Classification and Regression Trees, Bagging and Random Forests > The Methods > Variable Importance
What the sources say
- Not checked yetcites Friedman 2001
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“It describes the average improvement in the “Gini gain” splitting criterion that a variable has achieved in all of its positions in the forest.”
§ An Introduction to Recursive Partitioning: Rationale, Application and Characteristics of Classification and Regression Trees, Bagging and Random Forests > The Methods > Variable Importance
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“variable selection should not be conducted before applying another statistical method on the same learning data”
§ An Introduction to Recursive Partitioning: Rationale, Application and Characteristics of Classification and Regression Trees, Bagging and Random Forests > Features and Pitfalls > Nonlinear Function Approximation
What the sources say
- Not checked yetcites Ambroise and McLachlan 2002
- Not checked yetcites Leeb and Pötscher 2006
- Not checked yetcites Boulesteix et al. 2008
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In real data applications, however, the random forest variable importance may reveal higher importance scores for variables working in complex interactions, that may have gone unnoticed in single trees and bagging”
§ An Introduction to Recursive Partitioning: Rationale, Application and Characteristics of Classification and Regression Trees, Bagging and Random Forests > The Methods > Variable Importance
Strength:hedged· not yet calibratedCausal:conditional· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“This approach, where for p predictor variables 2p − 1 models are fitted in each bootstrap iteration, has the great advantage that it provides sound statistical inference. However, it is computationally prohibitive for problems with many predictor variables of interest”
§ An Introduction to Recursive Partitioning: Rationale, Application and Characteristics of Classification and Regression Trees, Bagging and Random Forests > The Methods > Variable Importance
What the sources say
- Not checked yetcites Azen, Budescu, and Reiser (2001)
- Not checked yetcites Azen and Budescu (2003)
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
In the paper
“random forest variable importances reflect the effect of a variable in complex interactions as outlined above, while the approach of Azen et al. (2001) reflects the main effects – at least as long as interactions are not explicitly included in the candidate models.”
§ An Introduction to Recursive Partitioning: Rationale, Application and Characteristics of Classification and Regression Trees, Bagging and Random Forests > The Methods > Variable Importance
What the sources say
- Not checked yetcites Azen et al. (2001)
Strength:hedged· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“This property is often used for a statistical test, that, however, shows very poor statistical properties as outlined in the section on “Features and pitfalls”.”
§ An Introduction to Recursive Partitioning: Rationale, Application and Characteristics of Classification and Regression Trees, Bagging and Random Forests > The Methods > Variable Importance
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“tree-based methods like bagging and random forests only consider one predictor variable at a time, and can thus deal with high numbers of variables sequentially.”
§ An Introduction to Recursive Partitioning: Rationale, Application and Characteristics of Classification and Regression Trees, Bagging and Random Forests > Features and Pitfalls > “Small n Large p” Applicability
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The most advanced variable importance measure available in random forests is the “permutation accuracy importance” measure”
§ An Introduction to Recursive Partitioning: Rationale, Application and Characteristics of Classification and Regression Trees, Bagging and Random Forests > The Methods > Variable Importance
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“the resulting functional shape cannot be interpreted or grasped analytically, and (aside from measures of overall variable importance) can only serve as a “black-box” for prediction.”
§ An Introduction to Recursive Partitioning: Rationale, Application and Characteristics of Classification and Regression Trees, Bagging and Random Forests > Features and Pitfalls > Nonlinear Function Approximation
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
In the paper
“When the central limit theorem is applied to the mean importance VI (xj), Breiman and Cutler (2008) argue that the z-score is asymptotically standard normal.”
§ An Introduction to Recursive Partitioning: Rationale, Application and Characteristics of Classification and Regression Trees, Bagging and Random Forests > The Methods > Variable Importance
What the sources say
- Not checked yetcites Breiman and Cutler (2008)
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
- Rests on an assumption: The individual tree importance scores are independent given the original sample and identically distributed.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“in most real data applications – especially in cases where many predictor variables work in complex interactions – the prediction accuracy of random forests is found to be higher than for bagging, and both ensemble methods usually highly outperform single trees.”
§ An Introduction to Recursive Partitioning: Rationale, Application and Characteristics of Classification and Regression Trees, Bagging and Random Forests > The Methods > Predictions from Ensembles of Trees
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes? Based: unclearIt may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“By randomly permuting the values of a predictor variable, its original association with the response is broken.”
§ An Introduction to Recursive Partitioning: Rationale, Application and Characteristics of Classification and Regression Trees, Bagging and Random Forests > The Methods > Variable Importance
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The differences induced by random variations are, however, negligible – as long as the parameters of a random forest have been chosen such as to guarantee stable results:”
§ An Introduction to Recursive Partitioning: Rationale, Application and Characteristics of Classification and Regression Trees, Bagging and Random Forests > Features and Pitfalls > Randomness and Stability
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Secondly, for a “scaled” measure one may assume that its values are comparable over different studies – which is not the case for the z-score in random forests, that heavily depends on the choice of tuning parameters, as outlined in the next section.”
§ An Introduction to Recursive Partitioning: Rationale, Application and Characteristics of Classification and Regression Trees, Bagging and Random Forests > Features and Pitfalls > Scaled and Unscaled Importance Measures
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Due to these random processes, a random forest is only exactly reproducible when the random seed, a number that can be set by the user and determines the internal random number generation of the computer, is fixed. Otherwise, the results will vary between two runs of the same code.”
§ An Introduction to Recursive Partitioning: Rationale, Application and Characteristics of Classification and Regression Trees, Bagging and Random Forests > Features and Pitfalls > Randomness and Stability
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“due to the lack of a marginally detectable main effect, none of the variables may be selected in the first split of a classification tree, and the interaction may never be discovered.”
§ An Introduction to Recursive Partitioning: Rationale, Application and Characteristics of Classification and Regression Trees, Bagging and Random Forests > Features and Pitfalls > The “XOR”-Problem and Order Effects
Strength:hedged· not yet calibratedCausal:conditional· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Firstly, the variable importance does not depend on the scaling or variance of the predictor variables in the first place (in fact, the whole method is invariant against the scaling of numeric variables).”
§ An Introduction to Recursive Partitioning: Rationale, Application and Characteristics of Classification and Regression Trees, Bagging and Random Forests > Features and Pitfalls > Scaled and Unscaled Importance Measures
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes? Based: unclearIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The number of trees ntree highly affects the stability of the model. In general, the higher the number of trees the more reliable is the prediction and the interpretability of the variable importance.”
§ An Introduction to Recursive Partitioning: Rationale, Application and Characteristics of Classification and Regression Trees, Bagging and Random Forests > Features and Pitfalls > Randomness and Stability
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“a logistic regression model would not be able to identify an effect in any of the variables either, if the interaction was not explicitly included in the logistic regression model – and in that case a tree model, where an interaction effect of two variables can also be explicitly added as a potential predictor variable, would do equally well.”
§ An Introduction to Recursive Partitioning: Rationale, Application and Characteristics of Classification and Regression Trees, Bagging and Random Forests > Features and Pitfalls > The “XOR”-Problem and Order Effects
Strength:hedged· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
In the paper
“The bias is particularly pronounced for the Gini importance, that is based on the biased Gini gain split selection criterion (Strobl et al. 2007), but can also affect the permutation importance.”
§ An Introduction to Recursive Partitioning: Rationale, Application and Characteristics of Classification and Regression Trees, Bagging and Random Forests > Features and Pitfalls > Bias in Variable Selection and Variable Importance
What the sources say
- Not checked yetcites Strobl et al. 2007
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
In the paper
“In a permutation test framework Strobl et al. (2008) show that only a conditional permutation scheme reflects the desired null hypothesis, and the resulting conditional importance describes the actual effect of a variable in the presence of correlations more reliably.”
§ An Introduction to Recursive Partitioning: Rationale, Application and Characteristics of Classification and Regression Trees, Bagging and Random Forests > Features and Pitfalls > Tests for Variable Importance and Variable Selection
What the sources say
- Not checked yetcites Strobl et al. 2008
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“The particular data sets investigated by Breiman (2001b) seem to enhance the impression that random forests would not overfit, but this notion is heavily criticized by Segal (2004).”
§ An Introduction to Recursive Partitioning: Rationale, Application and Characteristics of Classification and Regression Trees, Bagging and Random Forests > Features and Pitfalls > Do Random Forests Overfit?
What the sources say
- Not checked yetcites Breiman 2001b
- Not checked yetcites Segal 2004
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The two sources of randomness that are responsible for these possible differences are (a) the bootstrap samples (or subsamples) that are randomly drawn in bagging and random forests and (b) the random preselection of predictor variables in random forests.”
- Also stated in: § Randomness and Stability · plain
“When When the permutation importance is computed, another source of of variability is the variability is the random permutation of the predictor vectors.”
§ An Introduction to Recursive Partitioning: Rationale, Application and Characteristics of Classification and Regression Trees, Bagging and Random Forests > Features and Pitfalls > Randomness and Stability
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
- Also stated in: § Randomness and Stability · plain
In the paper
“This bias is carried forward to ensembles of trees: Especially the variable importance can be biased when a data set contains predictor variables of different types (Strobl et al. 2007).”
§ An Introduction to Recursive Partitioning: Rationale, Application and Characteristics of Classification and Regression Trees, Bagging and Random Forests > Features and Pitfalls > Bias in Variable Selection and Variable Importance
What the sources say
- Not checked yetcites Strobl et al. 2007
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“First applications in psychology show that random forests can be of use in a wide variety of applications in this field as well.”
§ An Introduction to Recursive Partitioning: Rationale, Application and Characteristics of Classification and Regression Trees, Bagging and Random Forests > Discussion and Conclusion
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Therefore random forests can be considered as a valuable “off the shelf” tool for exploring complex data sets, that may in a few years from now become as popular in psychology as it is now in the fields of genetics and bioinformatics.”
§ An Introduction to Recursive Partitioning: Rationale, Application and Characteristics of Classification and Regression Trees, Bagging and Random Forests > Discussion and Conclusion
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✗ Checkable: fails✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim. It may be too vague to check against evidence.
- Rests on an assumption: Random forests often produce good results even off the shelf with default settings.
None of the sources cited for this claim have been checked yet.
In the paper
“Of course other recent statistical learning methods, such as boosting (Freund and Schapire 1997) and support vector machines (cf. Vapnik 1995, for an introduction) can also be applied to the scope of problems we suggested for the application of random forests.”
§ An Introduction to Recursive Partitioning: Rationale, Application and Characteristics of Classification and Regression Trees, Bagging and Random Forests > Discussion and Conclusion
What the sources say
- Not checked yetcites Freund and Schapire 1997
- Not checked yetcites Vapnik 1995
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“With respect to the theoretical foundations and practical application of random forests, Segal (2004) implies that the depth of the trees in random forests, rather than the number of trees as suspected, e.g., by Luellen et al. (2005), may regulate overfitting.”
§ An Introduction to Recursive Partitioning: Rationale, Application and Characteristics of Classification and Regression Trees, Bagging and Random Forests > Features and Pitfalls > Do Random Forests Overfit?
What the sources say
- Not checked yetcites Segal 2004
- Not checked yetcites Luellen et al. 2005
Strength:hedged· not yet calibratedCausal:conditional· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“Besides the applications to regression and classification problems covered here, the function cforest (Hothorn et al. 2008, 2006) used in the application example can even be applied to survival data with a censored response and can thus serve as a means of data exploration in a broad range of longitudinal studies, too.”
§ An Introduction to Recursive Partitioning: Rationale, Application and Characteristics of Classification and Regression Trees, Bagging and Random Forests > Discussion and Conclusion
What the sources say
- Not checked yetcites Hothorn et al. 2008
- Not checked yetcites Hothorn et al. 2006
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
The paper relies on this without support.
High dimensional problems are common in some areas of psychological research, where few subjects can be measured due to time or cost constraints, yet a large amount of data is generated for each subject.
“High dimensional problems are common not only in genetics, but also in some areas of psychological research, where only few subjects can be measured due to time or cost constraints, yet a large amount of data is generated for each subject.”
The paper relies on this without support.
The small n large p case is common in genetics, where thousands of genes are considered as potential predictors of a disease.
“This case is common, e.g., in genetics, where thousands of genes are considered as potential predictors of a disease.”
The paper relies on this without support.
In many complex applications, the true data generating mechanism is neither linear nor additive.
“since in many complex applications the true data generating mechanism is neither linear nor additive.”
The paper relies on this without support.
Each random sample drawn from the learning sample reflects the same data generating process as the original learning sample.
“each random sample reflects the same data generating process but differs slightly from the original learning sample due to random variation.”
The paper relies on this without support.
The individual tree importance scores are independent given the original sample and identically distributed.
“The individual importance scores VI (t)(xj) are computed from ntree bootstrap samples, that are independent given the original sample, and are identically distributed.”
The paper relies on this without support.
In many disciplines, intensive model validation is not common practice.
“However, in many disciplines intensive model validation is not common practice.”
The paper relies on this without support.
The importance of irrelevant variables varies randomly around zero.
“The rationale for this rule of thumb is that the importance of irrelevant variables varies randomly around zero.”
The paper relies on this without support.
Random forests often produce good results even off the shelf with default settings.
“and random forests often produce good results even “off the shelf” with default settings.”
The paper relies on this without support.
A variety of studies exploring the characteristics of machine learning tools such as random forests are based on only a few real data sets that happen to be freely available in some data repository.
“A variety of studies exploring the characteristics of machine learning tools such as random forests are based on only a few, real data sets, that happen to be freely available in some data repository.”
The paper relies on this without support.
Random forests with random preselection perform better than bagging with no random preselection at all.
“In general, random forests with random preselection perform better than bagging with no random preselection at all”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Random forests are a recursive partitioning method particularly well-suited to small n large p problems.44”
From Oxytocin efficacy is modulated by dosage and oxytocin receptor genotype in young adults with high-functioning autism: a 24-week randomized clinical trial (2016) · cites this paper as [44]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Smoking prediction rule uses the wrong predictor
MinorThe stated prediction rule groups adolescents by whether they lied to their parents, but the described tree splits first on whether their friends smoke. The paper also identifies lied_to_parents as an irrelevant predictor in this example.
“Subjects who have not lied to their parents as well as those who have lied to their parents but drank alcohol in one or less occasions are not likely to intend to smoke, while those who have lied to their parents and drank alcohol in 2 or more occasions are likely to intend to smoke within the next year.”
“The first split in the variable friends_smoke partitions the entire sample, while the second split in the variable alcohol_per_month further partitions only those subjects whose value for the variable friends_smoke is ”one or more”.”
Citations
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538
The Nature of Statistical Learning Theory
Vapnik, V. N. (1995) · doi:10.1007/978-1-4757-2440-0 · Cited by 39,138
Greedy function approximation: A gradient boosting machine
Friedman, J. H. (2001) · The Annals of Statistics · doi:10.1214/aos/1013203451 · Cited by 30,731
The Elements of Statistical Learning
Hastie, T., Friedman, J., & Tibshirani, R. (2001) · Springer Series in Statistics · doi:10.1007/978-0-387-21606-5 · Cited by 24,162
Classification and Regression Trees.
Gordon, A. D., Breiman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (1984) · Biometrics · doi:10.2307/2530946 · Cited by 23,867
(1992) · Cited by 23,431
Quinlan, J. R. (1986) · Machine Learning · doi:10.1007/bf00116251 · Cited by 21,439
Classification and Regression Trees.
Ryzin, J. V., Breiman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (1986) · Journal of the American Statistical Association · doi:10.2307/2288003 · Cited by 21,030
Regularization and Variable Selection Via the Elastic Net
Zou, H., & Hastie, T. (2005) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2005.00503.x · Cited by 20,957
A decision-theoretic generalization of on-line learning and an application to boosting
Freund, Y., & Schapire, R. E. (1997) · Journal of Computer and System Sciences · doi:10.1006/jcss.1997.1504 · Cited by 20,466
Flagged first, then by citations
Research works printed but not linked: 8
- Breiman L 1998 Arcing classifiers The Annals of Statistics 26 3 801 849
- Bühlmann P Yu B 2002 Analyzing bagging The Annals of Statistics 30 4 927 961
- Bühlmann P Yu B 2003 Boosting with the L2 loss: Regression and classification Journal of the American Statistical Association 98 324 339
- Friedman J Hastie T Tibshirani R 2000 Additive logistic regression: a statistical view of boosting The Annals of Statistics 28 2 337 407
- Hastie T Tibshirani R Friedman JH 2009 The Elements of Statistical Learning 2 New York Springer
- Potapov S Theus M Urbanek S 2006 TWIX: Trees WIth eXtra Splits Presentation slides from the Third Ensemble Workshop of the Statistical Computing task group of the German Section of the International Biometric Society Munich, Germany
- Sanchez-Espigares J Marco L Brito P 2008 Rasch model-based recursive partitioning for statistical survey analysis Abstract Book of the 18th International Conference on Computational Statistics Porto, Portugal
- van Os BJ Meulman J Azen S Kontoghiorghes E Lee JC 2005 Globally optimal tree models Abstract Book of the 3rd World Conference on Computational Statistics & Data Analysis of the International Association for Statistical Computing Cyprus, Greece 79 Matrix Computations and Statistics Group
Cited by
Sorted by citations
Items 1,001–1,004 of 1,004
Blankers, M., Koeter, M. W., & Schippers, G. M. (2013) · BMC Public Health · doi:10.1186/1471-2458-13-455 · Cited by 14
Pairing Machine Learning and Clinical Psychology: How You Evaluate Predictive Performance Matters
Jacobucci, R., Littlefield, A. K., Millner, A. J., Kleiman, E., & Steinley, D. (2020) · doi:10.31234/osf.io/2yber · Cited by 12
Bhondoekhan, F., Li, Y., Hallowell, B. D., Mahoney, L., Daly, M. M., Goulet, J., Beaudoin, F. L., Chambers, L. C., & Marshall, B. D. L. (2024) · Drug and Alcohol Dependence Reports · doi:10.1016/j.dadr.2024.100287 · Cited by 4
Parsimonious statistical learning models for low flow estimation
Laimighofer, J., Melcher, M., & Laaha, G. (2021) · doi:10.5194/hess-2021-481 · Cited by 30
“The third method (cfrank) uses conditional forests (Hothorn et al., 2006; Strobl et al., 2007, 2009) for variable ranking, where the standardized coefficients are a sum of the main and interaction effects for each variable.”
Method
Metadata
- Authors
- Strobl, C., Malley, J., & Tutz, G.
- Affiliations
- Ludwig-Maximilians-Universität München (DE)
- National Institutes of Health (US)
- Venue
- Psychological Methods
- Published
- 2009-12
- Volume
- 14
- Issue
- 4
- Pages
- 323-348
- ISSN
- 1082-989X (ISSN-L)
- 1939-1463
- PMC
- PMC2927982
- OpenAlex
- W2070230130
- Semantic Scholar
- 4544ec96d5ec6e71f7795fa5498f542cb9985c7a
- Cited by
- 2,583
- References
- 94
- Citations by year
- 2026: 127 · 2025: 189 · 2024: 221 · 2023: 188 · 2022: 198 · 2021: 272 · 2020: 208 · 2019: 232 · 2018: 228 · 2017: 156 · 2016: 159 · 2015: 144 · 2014: 114 · 2013: 74 · 2012: 41
- FWCI
- 19.69
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Psychological Methods (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- Open access LMU (Ludwid Maxmilian's Universitat Munchen) (submittedVersion)
- Universitätsbibliothek der LMU
- Funders
- National Institutes of Health (ZIACT000271)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Inference
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Applications
- Physical Sciences › Computer Science › Information Systems › Data Mining Algorithms and Applications
- Keywords
- recursive partitioning, classification and regression trees, bagging, random forest, high-dimensional data, feature importance, predictor interactions, nonparametric regression, classification, R software, prediction accuracy
- MeSH terms
- Data Interpretation, Statistical, Humans, Models, Psychological, Psychology, Regression Analysis, Statistics, Nonparametric
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-09, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-09 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
