Cited by 89Data Mining Algorithms and ApplicationsMachine Learning and AlgorithmsNeural Networks and Applications

Decision trees do not generalize to new variations

Bengio, Yoshua, Delalleau, Olivier, Simard, Clarence·2010·Computational Intelligence

Abstract

The family of decision tree learning algorithms is among the most widespread and studied. Motivated by the desire to develop learning algorithms that can generalize when learning highly varying functions such as those presumably needed to achieve artificial intelligence, we study some theoretical limitations of decision trees. We demonstrate formally that they can be seriously hurt by the curse of dimensionality in a sense that is a bit different from other nonparametric statistical methods, but most importantly, that they cannot generalize to variations not seen in the training set. This is because a decision tree creates a partition of the input space and needs at least one example in each of the regions associated with a leaf to make a sensible prediction in that region. A better understanding of the fundamental reasons for this limitation suggests that one should use forests or even deeper architectures instead of trees, which provide a form of distributed representation and can generalize to variations not encountered in the training data.

Integrity signals

Registry

Crossref · 10.1111/j.1467-8640.2010.00366.x

Cites

42 of 42 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Authors
Bengio, Y., Delalleau, O., & Simard, C.
Affiliations
Université de Montréal (CA)
Corresponding author
Delalleau, Olivier
Venue
Computational Intelligence
Publisher
Wiley
Published
2010-11
Volume
26
Issue
4
Pages
449-467
ISSN
  • 0824-7935 (ISSN-L)
  • 1467-8640
OpenAlex
W2106004777
Cited by
89
References
41
Citations by year
2026: 2 · 2025: 5 · 2024: 9 · 2023: 6 · 2022: 8 · 2021: 10 · 2020: 9 · 2019: 4 · 2018: 3 · 2017: 3 · 2016: 5 · 2015: 4 · 2014: 4 · 2013: 6 · 2012: 5
FWCI
14.31
Type
journal article
Language
en
Open access
closed
Locations
Computational Intelligence (publishedVersion)
Funders
Canadian Institute for Advanced Research
Topics
  • Physical Sciences › Computer Science › Information Systems › Data Mining Algorithms and Applications
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Algorithms
  • Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
Keywords
decision tree, generalization, curse of dimensionality, input space partitioning, random forest, distributed representations, nonparametric learning

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review