The Collinearity Problem in Linear Regression. The Partial Least Squares (PLS) Approach to Generalized Inverses
The DOI matches its Crossref record; 42 of the 42 references it cites were found in a registry or an index.
Abstract
The use of partial least squares (PLS) for handling collinearities among the independent variables X in multiple regression is discussed. Consecutive estimates $({\text{rank }}1,2,\cdots )$ are obtained using the residuals from previous rank as a new dependent variable y. The PLS method is equivalent to the conjugate gradient method used in Numerical Analysis for related problems. To estimate the “optimal” rank, cross validation is used. Jackknife estimates of the standard errors are thereby obtained with no extra computation. The PLS method is compared with ridge regression and principal components regression on a chemical example of modelling the relation between the measured biological activity and variables describing the chemical structure of a set of substituted phenethylamines.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Cross validatory choice and assessment of statistical predictions
Stone, M. (1974) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1974.tb00994.x · Cited by 10,881
Ridge regression: Biased estimation for nonorthogonal problems
Hoerl, A. E., & Kennard, R. W. (1970) · Technometrics · doi:10.1080/00401706.1970.10488634 · Cited by 9,648
Methods of conjugate gradients for solving linear systems
Hestenes, M. R., & Stiefel, E. (1952) · Journal of Research of the National Bureau of Standards · doi:10.6028/jres.049.044 · Cited by 8,132
Hahn, G. J., Draper, N., & Smith, H. (1982) · Technometrics · doi:10.2307/1267833 · Cited by 5,773
LSQR: An Algorithm for Sparse Linear Equations and Sparse Least Squares
Paige, C. C., & Saunders, M. A. (1982) · ACM Transactions on Mathematical Software · doi:10.1145/355984.355989 · Cited by 4,387
Generalized Cross-Validation as a Method for Choosing a Good Ridge Parameter
Golub, G. H., Heath, M., & Wahba, G. (1979) · Technometrics · doi:10.1080/00401706.1979.10489751 · Cited by 3,934
Craven, P., & Wahba, G. (1978) · Numerische Mathematik · doi:10.1007/bf01404567 · Cited by 3,388
Applied Regression Analysis (2nd ed).
de Carvalho, J. F., Draper, N. R., & Smith, H. (1981) · Journal of the American Statistical Association · doi:10.2307/2287608 · Cited by 3,044
A predictive approach to the random effect model
GEISSER, S. (1974) · Biometrika · doi:10.1093/biomet/61.1.101 · Cited by 2,961
Cross-Validatory Estimation of the Number of Components in Factor and Principal Components Models
Wold, S. (1978) · Technometrics · doi:10.1080/00401706.1978.10489693 · Cited by 2,601
Self-citationGeneralized Inverses, Ridge Regression, Biased Linear Estimation, and Nonlinear Estimation
Marquardt, D. W. (1970) · Technometrics · doi:10.1080/00401706.1970.10488699 · Cited by 2,276
Calculating the Singular Values and Pseudo-Inverse of a Matrix
Golub, G., & Kahan, W. (1965) · Journal of the Society for Industrial and Applied Mathematics Series B Numerical Analysis · doi:10.1137/0702016 · Cited by 1,786
A Biometrics Invited Paper. The Analysis and Selection of Variables in Linear Regression
Hocking, R. R. (1976) · Biometrics · doi:10.2307/2529336 · Cited by 1,765
An Analysis of the Total Least Squares Problem
Golub, G. H., & van Loan, C. F. (1980) · SIAM Journal on Numerical Analysis · doi:10.1137/0717073 · Cited by 1,753
MILLER, R. G. (1974) · Biometrika · doi:10.1093/biomet/61.1.1 · Cited by 1,467
Soft modelling: The Basic Design and Some Extensions
Wold, H. (1982) · Medical Entomology and Zoology · Cited by 1,314
Self-citationAromatic substituent constants for structure-activity correlations
Hansch, C., Leo, A., Unger, S. H., Kim, K. H., Nikaitani, D., & Lien, E. J. (1973) · Journal of Medicinal Chemistry · doi:10.1021/jm00269a003 · Cited by 1,262
Pattern recognition by means of disjoint principal components models
Wold, S. (1976) · Pattern Recognition · doi:10.1016/0031-3203(76)90014-5 · Cited by 1,164
Self-citationRidge Regression: Biased Estimation for Nonorthogonal Problems
Hoerl, A. E., & Kennard, R. W. (2000) · Technometrics · doi:10.1080/00401706.2000.10485983 · Cited by 808
Practical Approximate Solutions to Linear Operator Equations When the Data are Noisy
Wahba, G. (1977) · SIAM Journal on Numerical Analysis · doi:10.1137/0714044 · Cited by 798
Mean Square Error of Prediction as a Criterion for Selecting Variables
Allen, D. M. (1971) · Technometrics · doi:10.1080/00401706.1971.10488811 · Cited by 691
SIMCA: A Method for Analyzing Chemical Data in Terms of Similarity and Analogy
WOLD, S., & SJÖSTRÖM, M. (1977) · ACS Symposium Series · doi:10.1021/bk-1977-0052.ch012 · Cited by 593
Self-citationSoft Modelling by Latent Variables: The Non-Linear Iterative Partial Least Squares (NIPALS) Approach
Wold, H. (1975) · Journal of Applied Probability · doi:10.1017/s0021900200047604 · Cited by 504
Self-citationA completely automatic french curve: fitting spline functions by cross validation
Wahba, G., & Wold, S. (1975) · Communications in Statistics · doi:10.1080/03610927508827223 · Cited by 366
Self-citationMean Square Error of Prediction as a Criterion for Selecting Variables
Allen, D. M. (1971) · Technometrics · doi:10.2307/1267161 · Cited by 362
BURGEN, A. S. V. (1971) · Nature · doi:10.1038/232587a0 · Cited by 328
Selection of Variables for Fitting Equations to Data
Gorman, J. W., & Toman, R. J. (1966) · Technometrics · doi:10.1080/00401706.1966.10490322 · Cited by 284
Ridge Regression and James-Stein Estimation: Review and Comments
Draper, N. R., & Van Nostrand, R. C. (1979) · Technometrics · doi:10.1080/00401706.1979.10489815 · Cited by 227
Factorial Designs, the |X′X| Criterion, and Some Related Matters
Box, M. J., & Draper, N. R. (1971) · Technometrics · doi:10.1080/00401706.1971.10488845 · Cited by 221
On Biased Estimation in Linear Models
Mayer, L. S., & Willke, T. A. (1973) · Technometrics · doi:10.1080/00401706.1973.10489076 · Cited by 220
A Critique of Some Ridge Regression Methods
Smith, G., & Campbell, F. (1980) · Journal of the American Statistical Association · doi:10.1080/01621459.1980.10477428 · Cited by 212
Four levels of pattern recognition
Albano, C., Dunn, W., Edlund, U., Johansson, E., Nordén, B., Sjöström, M., & Wold, S. (1978) · Analytica Chimica Acta · doi:10.1016/s0003-2670(01)83107-x · Cited by 172
Self-citationKonnert, J. H. (1976) · Acta Crystallographica Section A · doi:10.1107/s0567739476001289 · Cited by 159
A Practical Examination of Some Numerical Methods for Linear Discrete Ill-Posed Problems
Varah, J. M. (1979) · SIAM Review · doi:10.1137/1021007 · Cited by 152
Mukherjee, C., Caron, M. G., Mullikin, D., & Lefkowitz, R. J. (1976) · Molecular Pharmacology · doi:10.1016/s0026-895x(25)11140-1 · Cited by 104
Structure-activity study of .beta.-adrenergic agents using the SIMCA method of pattern recognition
Dunn, W. J., Wold, S., & Martin, Y. C. (1978) · Journal of Medicinal Chemistry · doi:10.1021/jm00207a015 · Cited by 59
Self-citationAn Empirical Study of Jackknife-Constructed Confidence Regions in Nonlinear Regression
Duncan, G. T. (1978) · Technometrics · doi:10.1080/00401706.1978.10489636 · Cited by 51
Dunn, W. J., & Wold, S. (1980) · Bioorganic Chemistry · doi:10.1016/0045-2068(80)90010-3 · Cited by 50
Self-citationRelations Between Ridge Regression and Eigenanalysis of the Augmented Correlation Matrix
Hawkins, D. M. (1975) · Technometrics · doi:10.1080/00401706.1975.10489375 · Cited by 28
Geodesy: Dealing with an Enormous Computer Task
Kolata, G. B. (1978) · Science · doi:10.1126/science.200.4340.421 · Cited by 20
A Bidiagonalization Algorithm for Sparse Linear Equations and Least-Squares Problems.
Paige, C. C., & Saunders, M. A. (1978) · Defense Technical Information Center (DTIC) · Cited by 18
On the Transition from Pattern Cognition to Model Building
Wold, H. (1977) · Lecture notes in economics and mathematical systems · doi:10.1007/978-3-642-45494-3_43 · Cited by 16
Self-citation
Research works printed but not linked: 5
- Hansch C., 1973, J. Med. Chem., 16, 120
- 10.2307/1267205 · doi:10.2307/1267205 · Paper page
- Miller Rupert G., 1974, Biometrika, 61, 1
- Wold Herman, 1975, Perspectives in probability and statistics (papers in honour of M. S. Bartlett on the occasion of his 65th birthday), 117
- Wold Herman, 1982, Systems Under Indirect Observation, I-II
Cited by
Wold, S., Esbensen, K., & Geladi, P. (1987) · Chemometrics and Intelligent Laboratory Systems · doi:10.1016/0169-7439(87)80084-9 · Cited by 11,606
Self-citationPLS-regression: a basic tool of chemometrics
Wold, S., Sjöström, M., & Eriksson, L. (2001) · Chemometrics and Intelligent Laboratory Systems · doi:10.1016/s0169-7439(01)00155-1 · Cited by 10,084
Self-citationPartial least-squares regression: a tutorial
Geladi, P., & Kowalski, B. R. (1986) · Analytica Chimica Acta · doi:10.1016/0003-2670(86)80028-9 · Cited by 6,992
Ljung, L. (1999) · Wiley Encyclopedia of Electrical and Electronics Engineering · doi:10.1002/047134608x.w1046 · Cited by 3,778
A review of process fault detection and diagnosis
Venkatasubramanian, V., Rengaswamy, R., Yin, K., & Kavuri, S. N. (2003) · Computers & Chemical Engineering · doi:10.1016/s0098-1354(02)00160-6 · Cited by 2,723
Metadata
- Authors
- Wold, S., Ruhe, A., Wold, H., & Dunn, III
- Venue
- SIAM Journal on Scientific and Statistical Computing
- Published
- 1984-09
- Volume
- 5
- Issue
- 3
- Pages
- 735-743
- ISSN
- 0196-5204 (ISSN-L)
- 2168-3417
- OpenAlex
- W2166446427
- Cited by
- 2,529
- References
- 41
- Citations by year
- 2026: 65 · 2025: 93 · 2024: 78 · 2023: 91 · 2022: 115 · 2021: 133 · 2020: 142 · 2019: 170 · 2018: 108 · 2017: 107 · 2016: 124 · 2015: 102 · 2014: 104 · 2013: 88 · 2012: 88
- FWCI
- 10.44
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- SIAM Journal on Scientific and Statistical Computing (publishedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Statistical Methods and Models
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Applications
- Physical Sciences › Chemistry › Analytical Chemistry › Spectroscopy and Chemometric Analyses
- Keywords
- multicollinearity, partial least squares, generalized inverses, multiple linear regression, rank estimation, cross-validation, conjugate gradient method, ridge regression, principal component regression, structure-activity relationship, phenethylamines
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.