The DOI matches its Crossref record; 59 of the 59 references it cites were found in a registry or an index.
Incremental Online Learning in High Dimensions
Open accessAbstract
Locally weighted projection regression (LWPR) is a new algorithm for incremental nonlinear function approximation in high-dimensional spaces with redundant and irrelevant input dimensions. At its core, it employs nonparametric regression with locally linear models. In order to stay computationally efficient and numerically robust, each local model performs the regression analysis with a small number of univariate regressions in selected directions in input space in the spirit of partial least squares regression. We discuss when and how local learning techniques can successfully work in high-dimensional spaces and review the various techniques for local dimensionality reduction before finally deriving the LWPR algorithm. The properties of LWPR are that it (1) learns rapidly with second-order learning methods based on incremental training, (2) uses statistically sound stochastic leave-one-out cross validation for learning without the need to memorize training data, (3) adjusts its weighting kernels based on only local information in order to minimize the danger of negative interference of incremental learning, (4) has a computational complexity that is linear in the number of inputs, and (5) can deal with a large number of—possibly redundant—inputs, as shown in various empirical evaluations with up to 90 dimensional data sets. For a probabilistic interpretation, predictive variance and confidence intervals are derived. To our knowledge, LWPR is the first truly incremental spatially localized learning method that can successfully and efficiently operate in very high-dimensional spaces.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Gelman, A., Carlin, J. B., Stern, H. S., & Rubin, D. B. (2003) · doi:10.1201/9780429258480 · Cited by 27,276
Horn, R. A., & Johnson, C. R. (1985) · Cited by 22,203
Numerical recipes in C: the art of scientific computing
(1993) · Choice Reviews Online · doi:10.5860/choice.30-5638 · Cited by 17,933
Press, W. H., Teukolsky, S. A., Vetterling, W. T., & Flannery, B. P. (1994) · Cited by 15,411
Nonlinear Dimensionality Reduction by Locally Linear Embedding
Roweis, S. T., & Saul, L. K. (2000) · Science · doi:10.1126/science.290.5500.2323 · Cited by 14,965
A Global Geometric Framework for Nonlinear Dimensionality Reduction
Tenenbaum, J. B., Silva, V. D., & Langford, J. C. (2000) · Science · doi:10.1126/science.290.5500.2319 · Cited by 13,585
A tutorial on support vector regression
Smola, A. J., & Schölkopf, B. (2004) · Statistics and Computing · doi:10.1023/b:stco.0000035301.49549.88 · Cited by 13,387
Numerical recipes in Pascal: the art of scientific computing
(1990) · Choice Reviews Online · doi:10.5860/choice.27-5158 · Cited by 11,811
Brown, R. A., Hastie, T. J., & Tibshirani, R. J. (1991) · Biometrics · doi:10.2307/2532174 · Cited by 8,176
Advances in kernel methods: support vector learning
Schölkopf, B., Burges, C. J. C., & Smola, A. J. (1999) · doi:10.5555/299094 · Cited by 5,789
Emergence of simple-cell receptive field properties by learning a sparse code for natural images
Olshausen, B. A., & Field, D. J. (1996) · Nature · doi:10.1038/381607a0 · Cited by 5,694
Numerical Recipes--The Art of Scientific Computing.
Fritsch, F. N., Press, W. H., Flannery, B. P., Teukolsky, S. A., & Vetterling, W. T. (1988) · Mathematics of Computation · doi:10.2307/2007938 · Cited by 4,845
Numerical Recipes in C: The Art of Scientific Computing.
Birchenhall, C., Press, W. H., Teukolsky, S. A., Vetterling, W. T., & Flannery, B. P. (1994) · The Economic Journal · doi:10.2307/2234664 · Cited by 4,355
Lohrli, A. (1985) · Notes and Queries · doi:10.1093/notesj/32.3.377-a · Cited by 4,352
Theory and Practice of Recursive Identification
Lennart Ljung, A., & Söderström, T. (1983) · Lund University Publications (Lund University) · Cited by 3,989
Multivariate Density Estimation: Theory, Practice, and Visualization
Scott, D. W. (1992) · Wiley Series in Probability and Statistics · doi:10.1002/9780470316849 · Cited by 3,696
Classical and Modern Regression With Applications.
Steinhorst, R. K., & Myers, R. H. (1988) · Journal of the American Statistical Association · doi:10.2307/2288958 · Cited by 3,690
Probabilistic Principal Component Analysis
Tipping, M. E., & Bishop, C. M. (1999) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/1467-9868.00196 · Cited by 3,679
Aircraft Control and Simulation
Stevens, B., & Lewis, F. L. (2004) · Aircraft Engineering and Aerospace Technology · doi:10.1108/aeat.2004.12776eae.001 · Cited by 3,167
Hastie, T., & Tibshirani, R. (2014) · Wiley StatsRef: Statistics Reference Online · doi:10.1002/9781118445112.stat03141 · Cited by 3,156
(1999) · doi:10.1162/089976600300015565 · Cited by 2,799
Hierarchical Mixtures of Experts and the EM Algorithm
Jordan, M. I., & Jacobs, R. A. (1994) · Neural Computation · doi:10.1162/neco.1994.6.2.181 · Cited by 2,673
The Cascade-Correlation learning architecture
Fahlman, S. E., & Lebiere, C. J. (2018) · doi:10.1184/r1/6610403 · Cited by 2,372
Friedman, J. H., & Stuetzle, W. (1981) · Journal of the American Statistical Association · doi:10.1080/01621459.1981.10477729 · Cited by 2,216
A Statistical View of Some Chemometrics Regression Tools
Frank, L. E., & Friedman, J. H. (1993) · Technometrics · doi:10.1080/00401706.1993.10485033 · Cited by 2,213
The “independent components” of natural scenes are edge filters
Bell, A. J., & Sejnowski, T. J. (1997) · Vision Research · doi:10.1016/s0042-6989(97)00121-1 · Cited by 2,201
Atkeson, C. G., Moore, A. W., & Schaal, S. (1997) · Artificial Intelligence Review · doi:10.1023/a:1006559212014 · Cited by 1,671
Self-citationUsing the Nyström Method to Speed Up Kernel Machines
Williams, C. K. I., & Seeger, M. (2000) · Cited by 1,646
Optimal unsupervised learning in a single-layer linear feedforward neural network
Sanger, T. D. (1989) · Neural Networks · doi:10.1016/0893-6080(89)90044-0 · Cited by 1,509
A Resource-Allocating Network for Function Interpolation
Platt, J. (1991) · Neural Computation · doi:10.1162/neco.1991.3.2.213 · Cited by 1,354
Gaussian Processes for Regression
Williams, C. K. I., & Rasmussen, C. E. (1995) · Aston Publications Explorer (Aston University) · Cited by 1,246
Principal Components Regression in Exploratory Statistical Research
Massy, W. F. (1965) · Journal of the American Statistical Association · doi:10.1080/01621459.1965.10480787 · Cited by 830
When networks disagree: Ensemble methods for hybrid neural networks
PERRONE, M. P., & COOPER, L. N. (1995) · World Scientific Series in 20th Century Physics · doi:10.1142/9789812795885_0025 · Cited by 755
EM Algorithms for ML Factor Analysis
Rubin, D. B., & Thayer, D. T. (1982) · Psychometrika · doi:10.1007/bf02293851 · Cited by 603
Model-based control of a robot manipulator
(1988) · Choice Reviews Online · doi:10.5860/choice.26-1542 · Cited by 553
Locally Weighted Learning for Control
Atkeson, C. G., Moore, A. W., & Schaal, S. (1997) · Artificial Intelligence Review · doi:10.1023/a:1006511328852 · Cited by 527
Self-citationAn Introduction to Latent Variable Models
Everitt, B. S. (1984) · Springer eBooks · doi:10.1007/978-94-009-5564-6 · Cited by 523
Constructive Incremental Learning from Only Local Information
Schaal, S., & Atkeson, C. G. (1998) · Neural Computation · doi:10.1162/089976698300016963 · Cited by 517
Self-citationSoft Modelling by Latent Variables: The Non-Linear Iterative Partial Least Squares (NIPALS) Approach
Wold, H. (1975) · Journal of Applied Probability · doi:10.1017/s0021900200047604 · Cited by 504
Variational Inference for Bayesian Mixtures of Factor Analysers
Ghahramani, Z., & Beal, M. J. (1999) · neural information processing systems · Cited by 432
Local Regression: Automatic Kernel Carpentry
Hastie, T., & Loader, C. (1993) · Statistical Science · doi:10.1214/ss/1177011002 · Cited by 415
SMEM Algorithm for Mixture Models
Ueda, N., Nakano, R., Ghahramani, Z., & Hinton, G. E. (2000) · Neural Computation · doi:10.1162/089976600300015088 · Cited by 409
A Statistical View of Some Chemometrics Regression Tools
Frank, I. E., & Friedman, J. H. (1993) · Technometrics · doi:10.2307/1269656 · Cited by 395
Perspectives in Probability and Statistics.
Ness, J. W. V., & Gani, J. (1980) · Journal of the American Statistical Association · doi:10.2307/2287484 · Cited by 300
(2000) · Cited by 296
Tutorial on Variational Approximation Methods
Jaakkola, T. S. (2001) · Advanced Mean Field Methods · doi:10.7551/mitpress/1100.003.0014 · Cited by 269
Time and tide in cerebellar memory formation
De Zeeuw, C. I., & Yeo, C. H. (2005) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2005.10.008 · Cited by 233
An Alternative Model for Mixtures of Experts
Xu, L., Jordan, M. I., & Hinton, G. E. (1994) · Cited by 215
Origins and violations of the 2/3 power law in rhythmic three-dimensional arm movements
Schaal, S., & Sternad, D. (2001) · Experimental Brain Research · doi:10.1007/s002210000505 · Cited by 196
Self-citationEfficient implementation of gaussian processes
Gibbs, M. (1997) · Cited by 151
Support Vector Machine Reference Manual
Saunders, C. J., Mo, W. S., Schölkopf Bottou, L., & Smola, A. J. (1998) · ePrints Soton (University of Southampton) · Cited by 148
Composite adaptive control with locally weighted statistical learning
Nakanishi, J., Farrell, J. A., & Schaal, S. (2004) · Neural Networks · doi:10.1016/j.neunet.2004.08.009 · Cited by 108
Self-citationReal-time robot learning with locally weighted statistical learning
Schaal, S., Atkeson, C. G., & Vijayakumar, S. (2002) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/robot.2000.844072 · Cited by 87
Self-citationArtificial Intelligence Review
Kilani, A., Hamida, A. B., & Hamam, H. (2018) · Advances in computer and electrical engineering book series · doi:10.4018/978-1-5225-7368-5.ch003 · Cited by 72
Supervised Dimension Reduction of Intrinsically Low-Dimensional Data
Vlassis, N. A., Motomura, Y., & Kröse, B. (2002) · Neural Computation · doi:10.1162/089976602753284491 · Cited by 48
Local Dimensionality Reduction
Schaal, S., Vijayakumar, S., & Atkeson, C. G. (1997) · Edinburgh Research Explorer (University of Edinburgh) · Cited by 35
Self-citationRKHS-based functional analysis for exact incremental learning
Vijayakumar, S., & Ogawa, H. (1999) · Neurocomputing · doi:10.1016/s0925-2312(99)00112-5 · Cited by 32
Self-citationLocal Adaptive Subspace Regression
Vijayakumar, S., & Schaal, S. (1998) · Neural Processing Letters · doi:10.1023/a:1009696221209 · Cited by 19
Self-citationLocally Weighted Interpolating Growing Neural Gas
Flentge, F. (2006) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2006.879771 · Cited by 16
Cited by
Robot Programming by Demonstration
Billard, A., Calinon, S., Dillmann, R., & Schaal, S. (2008) · Springer Handbook of Robotics · doi:10.1007/978-3-540-30301-5_60 · Cited by 1,002
Self-citationQuo vadis artificial intelligence?
Jiang, Y., Li, X., Luo, H., Yin, S., & Kaynak, O. (2022) · Discover Artificial Intelligence · doi:10.1007/s44163-022-00022-8 · Cited by 544
Model learning for robot control: A survey
Nguyen-Tuong, D., & Peters, J. (2011) · Cognitive Processing · doi:10.1007/s10339-011-0404-1 · Cited by 503
Learning and Reproduction of Gestures by Imitation
Calinon, S., D'halluin, F., Sauser, E., Caldwell, D., & Billard, A. (2010) · IEEE Robotics & Automation Magazine · doi:10.1109/mra.2010.936947 · Cited by 463
Active learning of inverse models with intrinsically motivated goal exploration in robots
Baranes, A., & Oudeyer, P.-Y. (2013) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2012.05.008 · Cited by 455
Metadata
- Authors
- Vijayakumar, S., D'Souza, A., & Schaal, S.
- Venue
- Neural Computation
- Publisher
- MIT Press - Journals
- Published
- 2005-12-01
- Volume
- 17
- Issue
- 12
- Pages
- 2602-2634
- ISSN
- 0899-7667 (ISSN-L)
- 1530-888X
- OpenAlex
- W1976606095
- Cited by
- 527
- References
- 59
- FWCI
- 22.67
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.