Ridge regression: Biased estimation for nonorthogonal problems
Abstract
In multiple regression it is shown that parameter estimates based on minimum residual sum of squares have a high probability of being unsatisfactory, if not incorrect, if the prediction vectors are not orthogonal. Proposed is an estimation procedure based on adding small positive quantities to the diagonal of X′X. Introduced is the ridge trace, a method for showing in two dimensions the effects of nonorthogonality. It is then shown how to augment X′X to obtain biased estimates with smaller mean square error.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
A linear support vector machine is used for classifying gender, and ridge regression for predicting age and each personality factor.
“We use a linear support vector machine (SVM) [92] for classifying the binary variable of gender, and ridge regression [93] for predicting age and each factor of personality.”
From Personality, Gender, and Age in the Language of Social Media: The Open-Vocabulary Approach (2013) · cites this paper as [93]
Cites
American Institute of Chemical Engineers 25693 (1988) · Applied Catalysis · doi:10.1016/0166-9834(88)80018-6 · Cited by 2,542 · see citing works
Applied Statistical Decision Theory.
Roberts, H. V., Raiffa, H., & Schlaifer, R. (1962) · Journal of the American Statistical Association · doi:10.2307/2282452 · Cited by 1,461 · see citing works
Mathematical Methods for Digital Computers
Taussky, O., Ralston, A., & Wilf, H. S. (1961) · Econometrica · doi:10.2307/1909651 · Cited by 632 · see citing works
Two Case Studies in the Application of Principal Component Analysis
Jeffers, J. N. R. (1967) · Journal of the Royal Statistical Society Series C (Applied Statistics) · doi:10.2307/2985919 · Cited by 455 · see citing works
Confidence Sets for the Mean of a Multivariate Normal Distribution
Stein, C. (1962) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.1111/j.2517-6161.1962.tb00458.x · Cited by 250 · see citing works
Riley, J. D. (1955) · Mathematics of Computation · doi:10.1090/s0025-5718-1955-0074915-1 · Cited by 98 · see citing works
Selection of the Best Subset in Regression Analysis
Hocking, R. R., & Leslie, R. N. (1967) · Technometrics · doi:10.2307/1266192 · Cited by 77 · see citing works
Selection of Variables for Fitting Equations to Data
Gorman, J. W., & Toman, R. J. (1966) · Technometrics · doi:10.2307/1266260 · Cited by 76 · see citing works
Factor Analysis and Regression
Scott, J. T. (1966) · Econometrica · doi:10.2307/1909769 · Cited by 70 · see citing works
Barnard, G. A. (1963) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.1111/j.2517-6161.1963.tb00490.x · Cited by 37 · see citing works
Using the Observations to Estimate the Prior Distribution
Clutton-Brock, M. (1965) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.1111/j.2517-6161.1965.tb00582.x · Cited by 4 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Wold, S., Ruhe, A., Wold, H., & Dunn, III (1984) · SIAM Journal on Scientific and Statistical Computing · doi:10.1137/0905052 · Cited by 2,529 · see citing works
Personality, Gender, and Age in the Language of Social Media: The Open-Vocabulary Approach
Schwartz, H. A., Eichstaedt, J. C., Kern, M. L., Dziurzynski, L., Ramones, S. M., Agrawal, M., Shah, A., Kosinski, M., Stillwell, D., Seligman, M. E. P., et al. (2013) · PLoS ONE · doi:10.1371/journal.pone.0073791 · Cited by 1,700 · see citing works
Validation of Regression Models: Methods and Examples
Snee, R. D. (1977) · Technometrics · doi:10.1080/00401706.1977.10489581 · Cited by 1,283 · see citing works
Machine Learning Methods That Economists Should Know About
Athey, S., & Imbens, G. W. (2019) · Annual Review of Economics · doi:10.1146/annurev-economics-080217-053433 · Cited by 1,163 · see citing works
Climatic patterns and the distribution of C4 grasses in North America
Teeri, J. A., & Stowe, L. G. (1976) · Oecologia · doi:10.1007/bf00351210 · Cited by 749 · see citing works
Characterizing a psychiatric symptom dimension related to deficits in goal-directed control
Gillan, C. M., Kosinski, M., Whelan, R., Phelps, E. A., & Daw, N. D. (2016) · eLife · doi:10.7554/elife.11305 · Cited by 586 · see citing works
Predictive value of statistical models.
Van Houwelingen, J. C., & Le Cessie, S. (1990) · Statistics in Medicine · doi:10.1002/sim.4780091109 · Cited by 621 · see citing works
Multicollinearity: How common factors cause Type 1 errors in multivariate regression
Kalnins, A. (2018) · Strategic Management Journal · doi:10.1002/smj.2783 · Cited by 526 · see citing works
Kessler, R. C., Warner, C. H., Ivany, C., Petukhova, M. V., Rose, S., Bromet, E. J., Brown, M., Cai, T., Colpe, L. J., Cox, K. L., et al. (2015) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2014.1754 · Cited by 447 · see citing works
Association of body mass index with health care expenditures in the united states by age and sex
Ward, Z. J., Bleich, S. N., Long, M. W., & Gortmaker, S. L. (2021) · PLOS ONE · doi:10.1371/journal.pone.0247307 · Cited by 428 · see citing works
Ogutu, J. O., Schulz-Streeck, T., & Piepho, H.-P. (2012) · BMC Proceedings · doi:10.1186/1753-6561-6-s2-s10 · Cited by 440 · see citing works
DE LOS CAMPOS, G., GIANOLA, D., ROSA, G. J. M., WEIGEL, K. A., & CROSSA, J. (2010) · Genetics Research · doi:10.1017/s0016672310000285 · Cited by 361 · see citing works
Attention modulates spatial priority maps in the human occipital, parietal and frontal cortices
Sprague, T. C., & Serences, J. T. (2013) · Nature Neuroscience · doi:10.1038/nn.3574 · Cited by 266 · see citing works
Calibration drift in regression and machine learning models for acute kidney injury
Davis, S. E., Lasko, T. A., Chen, G., Siew, E. D., & Matheny, M. E. (2017) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocx030 · Cited by 312 · see citing works
Machine learning for renewable energy materials
Gu, G. H., Noh, J., Kim, I., & Jung, Y. (2019) · Journal of Materials Chemistry A · doi:10.1039/c9ta02356a · Cited by 311 · see citing works
Energy-based descriptors to rapidly predict hydrogen storage in metal–organic frameworks
Bucior, B. J., Bobbitt, N. S., Islamoglu, T., Goswami, S., Gopalan, A., Yildirim, T., Farha, O. K., Bagheri, N., & Snurr, R. Q. (2019) · Molecular Systems Design & Engineering · doi:10.1039/c8me00050f · Cited by 308 · see citing works
SMIL: Multimodal Learning with Severely Missing Modality
Ma, M., Ren, J., Zhao, L., Tulyakov, S., Wu, C., & Peng, X. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i3.16330 · Cited by 332 · see citing works
Tutorial in biostatistics: data‐driven subgroup identification and analysis in clinical trials
Lipkovich, I., Dmitrienko, A., & B., R. (2017) · Statistics in Medicine · doi:10.1002/sim.7064 · Cited by 291 · see citing works
Shung, D. L., Au, B., Taylor, R. A., Tay, J. K., Laursen, S. B., Stanley, A. J., Dalton, H. R., Ngu, J., Schultz, M., & Laine, L. (2020) · Gastroenterology · doi:10.1053/j.gastro.2019.09.009 · Cited by 227 · see citing works
Automated classification of primary progressive aphasia subtypes from narrative speech transcripts
Fraser, K. C., Meltzer, J. A., Graham, N. L., Leonard, C., Hirst, G., Black, S. E., & Rochon, E. (2014) · Cortex · doi:10.1016/j.cortex.2012.12.006 · Cited by 229 · see citing works
Interpretation of Quantitative Structure-Activity Relationship Models: Past, Present, and Future
Polishchuk, P. (2017) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.7b00274 · Cited by 236 · see citing works
Lassopack: model selection and prediction with regularized regression in Stata
Ahrens, A., Hansen, C. B., & Schaffer, M. E. (2020) · The Stata Journal Promoting communications on statistics and Stata · doi:10.1177/1536867x20909697 · Cited by 212 · see citing works
When Optimism Hurts: Inflated Predictions in Psychiatric Neuroimaging
Whelan, R., & Garavan, H. (2014) · Biological Psychiatry · doi:10.1016/j.biopsych.2013.05.014 · Cited by 209 · see citing works
Kessler, R. C., Hwang, I., Hoffmire, C. A., McCarthy, J. F., Petukhova, M. V., Rosellini, A. J., Sampson, N. A., Schneider, A. L., Bradley, P. A., Katz, I. R., et al. (2017) · International Journal of Methods in Psychiatric Research · doi:10.1002/mpr.1575 · Cited by 209 · see citing works
Herriott, C., & Spear, A. D. (2020) · Computational Materials Science · doi:10.1016/j.commatsci.2020.109599 · Cited by 187 · see citing works
Suryanarayana, G., Lago, J., Geysen, D., Aleksiejuk, P., & Johansson, C. (2018) · Energy · doi:10.1016/j.energy.2018.05.111 · Cited by 155 · see citing works
Factors associated with healthy aging in Latin American populations
Santamaria-Garcia, H., Sainz-Ballesteros, A., Hernandez, H., Moguilner, S., Maito, M., Ochoa-Rosales, C., Corley, M., Valcour, V., Miranda, J. J., Lawlor, B., et al. (2023) · Nature Medicine · doi:10.1038/s41591-023-02495-1 · Cited by 141 · see citing works
Riley, R. D., Snell, K. I. E., Martin, G. P., Whittle, R., Archer, L., Sperrin, M., & Collins, G. S. (2021) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2020.12.005 · Cited by 142 · see citing works
Accommodating Linkage Disequilibrium in Genetic-Association Analyses via Ridge Regression
Malo, N., Libiger, O., & Schork, N. J. (2008) · The American Journal of Human Genetics · doi:10.1016/j.ajhg.2007.10.012 · Cited by 133 · see citing works
Van Calster, B., van Smeden, M., De Cock, B., & Steyerberg, E. W. (2020) · Statistical Methods in Medical Research · doi:10.1177/0962280220921415 · Cited by 120 · see citing works
Prediction of arrival times of freight traffic on US railroads using support vector regression
Barbour, W., Martinez Mori, J. C., Kuppa, S., & Work, D. B. (2018) · Transportation Research Part C: Emerging Technologies · doi:10.1016/j.trc.2018.05.019 · Cited by 116 · see citing works
Yu, B. (2013) · Bernoulli · doi:10.3150/13-bejsp14 · Cited by 113 · see citing works
Supervised enhancer prediction with epigenetic pattern recognition and targeted validation
Sethi, A., Gu, M., Gumusgoz, E., Chan, L., Yan, K.-K., Rozowsky, J., Barozzi, I., Afzal, V., Akiyama, J. A., Plajzer-Frick, I., et al. (2020) · Nature Methods · doi:10.1038/s41592-020-0907-8 · Cited by 109 · see citing works
Integrating Probability and Nonprobability Samples for Survey Inference
Wiśniowski, A., Sakshaug, J. W., Perez Ruiz, D. A., & Blom, A. G. (2020) · Journal of Survey Statistics and Methodology · doi:10.1093/jssam/smz051 · Cited by 102 · see citing works
Lee, P. H., & Wong, F. K. Y. (2015) · Sports Medicine · doi:10.1007/s40279-015-0322-y · Cited by 102 · see citing works
Satisficing: Integrating Two Traditions
Artinger, F. M., Gigerenzer, G., & Jacobs, P. (2022) · Journal of Economic Literature · doi:10.1257/jel.20201396 · Cited by 90 · see citing works
López Pineda, A., Ye, Y., Visweswaran, S., Cooper, G. F., Wagner, M. M., & Tsui, F. (. (2015) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2015.08.019 · Cited by 88 · see citing works
Behravan, H., Hartikainen, J. M., Tengström, M., Pylkäs, K., Winqvist, R., Kosma, V., & Mannermaa, A. (2018) · Scientific Reports · doi:10.1038/s41598-018-31573-5 · Cited by 75 · see citing works
On asymptotic properties of hyperparameter estimators for kernel-based regularization methods
Mu, B., Chen, T., & Ljung, L. (2018) · Automatica · doi:10.1016/j.automatica.2018.04.035 · Cited by 80 · see citing works
Estimating residential energy consumption in metropolitan areas: A microsimulation approach
Zhang, W., Robinson, C., Guhathakurta, S., Garikapati, V. M., Dilkina, B., Brown, M. A., & Pendyala, R. M. (2018) · Energy · doi:10.1016/j.energy.2018.04.161 · Cited by 68 · see citing works
Harle, S. M. (2024) · Asian Journal of Civil Engineering · doi:10.1007/s42107-023-00760-9 · Cited by 65 · see citing works
Machine learning in materials design: Algorithm and application*
Song, Z., Chen, X., Meng, F., Cheng, G., Wang, C., Sun, Z., & Yin, W.-J. (2020) · Chinese Physics B · doi:10.1088/1674-1056/abc0e3 · Cited by 63 · see citing works
Machine learning for glass science and engineering: A review
Liu, H., Fu, Z., Yang, K., Xu, X., & Bauchy, M. (2019) · Journal of Non-Crystalline Solids: X · doi:10.1016/j.nocx.2019.100036 · Cited by 53 · see citing works
Guo, W., & Lin, S. (2009) · Genetic Epidemiology · doi:10.1002/gepi.20382 · Cited by 51 · see citing works
Barfeld, S. J., East, P., Zuber, V., & Mills, I. G. (2014) · BMC Medical Genomics · doi:10.1186/s12920-014-0074-9 · Cited by 47 · see citing works
CausPref: Causal Preference Learning for Out-of-Distribution Recommendation
He, Y., Wang, Z., Cui, P., Zou, H., Zhang, Y., Cui, Q., & Jiang, Y. (2022) · Proceedings of the ACM Web Conference 2022 · doi:10.1145/3485447.3511969 · Cited by 50 · see citing works
Uncovering the genetic blueprint of the <i>C. elegans</i> nervous system
Kovács, I. A., Barabási, D. L., & Barabási, A.-L. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2009093117 · Cited by 42 · see citing works
Sharma, D., Gotlieb, N., Farkouh, M. E., Patel, K., Xu, W., & Bhat, M. (2022) · Journal of the American Heart Association · doi:10.1161/jaha.121.022576 · Cited by 36 · see citing works
Takarangi, M. K. T., Smith, R. A., Strange, D., & Flowe, H. D. (2017) · Clinical Psychological Science · doi:10.1177/2167702616649348 · Cited by 29 · see citing works
A Machine Learning Tutorial for Operational Meteorology. Part I: Traditional Machine Learning
Chase, R. J., Harrison, D. R., Burke, A., Lackmann, G. M., & McGovern, A. (2022) · Weather and Forecasting · doi:10.1175/waf-d-22-0070.1 · Cited by 25 · see citing works
MacDonald, M. E., Williams, R. J., Rajashekar, D., Stafford, R. B., Hanganu, A., Sun, H., Berman, A. J. L., McCreary, C. R., Frayne, R., Forkert, N. D., et al. (2020) · Neurobiology of Aging · doi:10.1016/j.neurobiolaging.2020.06.019 · Cited by 21 · see citing works
Liao, P.-C., Chen, M.-S., Jhou, M.-J., Chen, T.-C., Yang, C.-T., & Lu, C.-J. (2022) · Diagnostics · doi:10.3390/diagnostics12081965 · Cited by 21 · see citing works
Predictive and prescriptive analytics toward passenger-centric ground delay programs
Jacquillat, A. (2022) · Transportation Science · doi:10.1287/trsc.2021.1081 · Cited by 18 · see citing works
DeepRisk: a deep learning approach for genome-wide assessment of common disease risk
Peng, J., Bao, Z., Li, J., Han, R., Wang, Y., Han, L., Peng, J., Wang, T., Hao, J., Wei, Z., et al. (2024) · Fundamental Research · doi:10.1016/j.fmre.2024.02.015 · Cited by 15 · see citing works
Zhu, Y., Tan, T. L., & Cheang, W. K. (2017) · Chemometrics and Intelligent Laboratory Systems · doi:10.1016/j.chemolab.2017.09.019 · Cited by 13 · see citing works
A Fair Individualized Polysocial Risk Score for Identifying Increased Social Risk in Type 2 Diabetes
Huang, Y., Guo, J., Donahoo, W. T., Lee, Y. A., Fan, Z., Lu, Y., Chen, W.-H., Tang, H., Bilello, L., Saguil, A. A., et al. (2024) · Nature Communications · doi:10.1038/s41467-024-52960-9 · Cited by 14 · see citing works
Bayesian Genomic-Enabled Prediction as an Inverse Problem
Cuevas, J., Pérez-Elizalde, S., Soberanis, V., Pérez-Rodríguez, P., Gianola, D., & Crossa, J. (2014) · G3 Genes|Genomes|Genetics · doi:10.1534/g3.114.013094 · Cited by 10 · see citing works
Linear and Non-Linear Regression Methods in Epidemiology and Biostatistics
Vittinghoff, E., McCulloch, C. E., Glidden, D. V., & Shiboski, S. C. (2011) · Essential Statistical Methods for Medical Statistics · doi:10.1016/b978-0-444-53737-9.50006-2 · Cited by 6 · see citing works
Siddiqui, T., Zioupos, P., Kelly, K., England, L., Baldini, E., Viner, M., & Márquez-Grant, N. (2026) · Forensic Science International · doi:10.1016/j.forsciint.2026.113078 · Cited by 1 · see citing works
- Authors
- Hoerl, A. E., & Kennard, R. W.
- Affiliations
- DuPont (United States) (US)
- Venue
- Technometrics
- Publisher
- Informa UK Limited
- Published
- 1970-02
- Volume
- 12
- Issue
- 1
- Pages
- 55-67
- ISSN
- 0040-1706 (ISSN-L)
- 1537-2723
- OpenAlex
- W4234698323
- Cited by
- 9,648
- References
- 11
- Citations by year
- 2026: 1338 · 2025: 968 · 2024: 720 · 2023: 738 · 2022: 728 · 2021: 772 · 2020: 647 · 2019: 553 · 2018: 484 · 2017: 293 · 2016: 283 · 2015: 269 · 2014: 250 · 2013: 238 · 2012: 218
- FWCI
- 8.87
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Technometrics (publishedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Statistical Methods and Models
- Physical Sciences › Mathematics › Applied Mathematics › Statistical and numerical algorithms
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Applications
- Keywords
- ridge regression, biased estimation, multicollinearity, mean squared error, parameter estimation, multiple linear regression
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review