Dynamic Trees for Learning and Design
Abstract
Dynamic regression trees are an attractive option for automatic regression and classification with complicated response surfaces in online application settings. We create a sequential tree model whose state changes in time with the accumulation of new data, and provide particle learning algorithms that allow for the efficient online posterior filtering of tree states. A major advantage of tree regression is that it allows for the use of very simple models within each partition. The model also facilitates a natural division of labor in our sequential particle-based inference: tree dynamics are defined through a few potential changes that are local to each newly arrived observation, while global uncertainty is captured by the ensemble of particles. We consider both constant and linear mean functions at the tree leaves, along with multinomial leaves for classification problems, and propose default prior specifications that allow for prediction to be integrated over all model parameters conditional on a given tree. Inference is illustrated in some standard nonparametric regression examples, as well as in the setting of sequential experiment design, including both active learning and optimization applications, and in online classification. We detail implementation guidelines and problem specific methodology for each of these motivating applications. Throughout, it is demonstrated that our practical approach is able to provide better results compared to commonly used methods at a fraction of the cost.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Kelly and Ó Gráda show that England outside London saw the disappearance of the positive check during the early seventeenth century, while London after 1665 showed a continued strong positive check until the early eighteenth century.
“Kelly and Ó Gráda show that England, outside London, saw the disappearance of the positive check during the early seventeenth century, while London after 1665 showed a con-”
From Living standards and plague in <scp>L</scp>ondon, 1560–1665 (2016) · cites this paper as [57]
Cites
Classification and Regression Trees.
Gordon, A. D., Breiman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (1984) · Biometrics · doi:10.2307/2530946 · Cited by 23,834 · see citing works
Classification and Regression Trees.
Ryzin, J. V., Breiman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (1986) · Journal of the American Statistical Association · doi:10.2307/2288003 · Cited by 21,028 · see citing works
Classification and Regression by randomForest
Liaw, A. I., & Wiener, M. C. (2007) · Cited by 18,594 · see citing works
Multivariate Adaptive Regression Splines
Friedman, J. H. (1991) · The Annals of Statistics · doi:10.1214/aos/1176347963 · Cited by 8,108 · see citing works
Efficient Global Optimization of Expensive Black-Box Functions
Jones, D. R., Schonlau, M., & Welch, W. J. (1998) · Journal of Global Optimization · doi:10.1023/a:1008306431147 · Cited by 8,070 · see citing works
Design and Analysis of Computer Experiments
Sacks, J., Welch, W. J., Mitchell, T. J., & Wynn, H. P. (1989) · Statistical Science · doi:10.1214/ss/1177012413 · Cited by 6,960 · see citing works
Pattern Recognition and Neural Networks
Ripley, B. D. (1996) · Cambridge University Press eBooks · doi:10.1017/cbo9780511812651 · Cited by 6,446 · see citing works
Kass, R. E., & Raftery, A. E. (1995) · Journal of the American Statistical Association · doi:10.2307/2291091 · Cited by 2,359 · see citing works
Filtering via Simulation: Auxiliary Particle Filters
Pitt, M. K., & Shephard, N. (1999) · Journal of the American Statistical Association · doi:10.1080/01621459.1999.10474153 · Cited by 2,283 · see citing works
BART: Bayesian additive regression trees
Chipman, H. A., George, E. I., & McCulloch, R. E. (2010) · The Annals of Applied Statistics · doi:10.1214/09-aoas285 · Cited by 1,875 · see citing works
Information-based objective functions for active data selection
MacKay, D. J. C. (1992) · Neural Computation · doi:10.1162/neco.1992.4.4.590 · Cited by 1,221 · see citing works
Sequential Imputations and Bayesian Missing Data Problems
Kong, A., Liu, J. S., & Wong, W. H. (1994) · Journal of the American Statistical Association · doi:10.1080/01621459.1994.10476469 · Cited by 1,061 · see citing works
BART: Bayesian Additive Regression Trees
Chipman, H., George, E. I., & McCulloch, R. E. (2006) · ScholarlyCommons (University of Pennsylvania) · Cited by 889 · see citing works
Filtering via Simulation: Auxiliary Particle Filters
Pitt, M. K., & Shephard, N. (1999) · Journal of the American Statistical Association · doi:10.2307/2670179 · Cited by 834 · see citing works
Bayesian Treed Gaussian Process Models With an Application to Computer Modeling
Gramacy, R. B., & Lee, H. K. H. (2008) · Journal of the American Statistical Association · doi:10.1198/016214508000000689 · Cited by 666 · see citing works
Self-citationGlobal versus local search in constrained optimization of computer models
Schonlau, M., Welch, W. J., & Jones, D. R. (1998) · Lecture notes-monograph series · doi:10.1214/lnms/1215456182 · Cited by 444 · see citing works
Particle Learning and Smoothing
Carvalho, C. M., Johannes, M. S., Lopes, H. F., & Polson, N. G. (2010) · Statistical Science · doi:10.1214/10-sts325 · Cited by 338 · see citing works
Self-citationSequential Imputations and Bayesian Missing Data Problems
Kong, A., Liu, J. S., & Wong, W. H. (1994) · Journal of the American Statistical Association · doi:10.2307/2291224 · Cited by 328 · see citing works
Adaptive Design and Analysis of Supercomputer Experiments
Gramacy, R. B., & Lee, H. K. H. (2009) · Technometrics · doi:10.1198/tech.2009.0015 · Cited by 239 · see citing works
Self-citationChipman, H., George, E. I., & McCulloch, R. E. (2002) · Machine Learning · doi:10.1023/a:1013916107446 · Cited by 177 · see citing works
Posterior probabilities for choosing a regression model
Atkinson, A. C. (1978) · Biometrika · doi:10.1093/biomet/65.1.39 · Cited by 151 · see citing works
Sequential design of computer experiments to minimize integrated response functions
Notz, W. I., Santner, T. J., & Williams, B. J. (2000) · Cited by 135 · see citing works
Chipman, H., George, E. I., & McCulloch, R. E. (1998) · Journal of the American Statistical Association · doi:10.2307/2669832 · Cited by 85 · see citing works
Bayesian Guided Pattern Search for Robust Local Optimization
Taddy, M. A., Lee, H. K. H., Gray, G. A., & Griffin, J. D. (2009) · Technometrics · doi:10.1198/tech.2009.08007 · Cited by 78 · see citing works
Self-citationParticle learning for general mixtures
Carvalho, C. M., Lopes, H. F., Polson, N. G., et al. (2010) · Bayesian Analysis · doi:10.1214/10-ba525 · Cited by 59 · see citing works
Self-citationClassification and Categorical Inputs with Treed Gaussian Process Models
Broderick, T., & Gramacy, R. B. (2011) · Journal of Classification · doi:10.1007/s00357-011-9083-y · Cited by 13 · see citing works
Self-citationParticle learning of Gaussian process models for sequential design and optimizationPreprint
Gramacy, R. B., & Polson, N. G. (2009) · arXiv (Cornell University) · doi:10.48550/arxiv.0909.5262 · Cited by 2 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Letham, B., Rudin, C., McCormick, T. H., & Madigan, D. (2015) · The Annals of Applied Statistics · doi:10.1214/15-aoas848 · Cited by 766 · see citing works
Living standards and plague in <scp>L</scp>ondon, 1560–1665
Cummins, N., Kelly, M., & Ó Gráda, C. (2016) · The Economic History Review · doi:10.1111/ehr.12098 · Cited by 139 · see citing works
- Authors
- Taddy, M. A., Gramacy, R. B., & Polson, N. G.
- Affiliations
- University of Cambridge (GB)
- ORCID
- Gramacy, Robert B.
- Venue
- Journal of the American Statistical Association
- Publisher
- Informa UK Limited
- Published
- 2011-03
- Volume
- 106
- Issue
- 493
- Pages
- 109-123
- ISSN
- 0162-1459 (ISSN-L)
- 1537-274X
- OpenAlex
- W2094352577
- Cited by
- 113
- References
- 27
- Citations by year
- 2026: 1 · 2025: 6 · 2024: 3 · 2023: 8 · 2022: 5 · 2021: 5 · 2020: 9 · 2019: 7 · 2018: 7 · 2017: 10 · 2016: 7 · 2015: 12 · 2014: 13 · 2013: 13 · 2012: 3
- FWCI
- 9.43
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of the American Statistical Association (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- Funders
- Engineering and Physical Sciences Research Council (EP/D065704/1)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Gaussian Processes and Bayesian Inference
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Inference
- Physical Sciences › Mathematics › Statistics and Probability › Markov Chains and Monte Carlo Methods
- Keywords
- particle learning, nonparametric regression, classification, active learning (machine learning), optimization, online classification, default priors
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
