CoChat
Sign Up
Cited by 9,998

Collinearity: a review of methods to deal with it and a simulation study evaluating their performance

Dormann, Carsten F., Elith, Jane, Bacher, Sven, Buchmann, Carsten, Carl, Gudrun, Carré, Gabriel, Marquéz, Jaime R. García, Gruber, Bernd, Lafourcade, Bruno, Leitão, Pedro J., Münkemüller, Tamara, McClean, Colin, Osborne, Patrick E., Reineking, Björn, Schröder, Boris, Skidmore, Andrew K., Zurell, Damaris, Lautenbach, Sven·2013·Ecography
DOI 10.1111/j.1600-0587.2012.07348.x
Open this paper in CoChat — free account

Abstract

Collinearity refers to the non independence of predictor variables, usually in a regression‐type analysis. It is a common feature of any descriptive ecological data set and can be a problem for parameter estimation because it inflates the variance of regression parameters and hence potentially leads to the wrong identification of relevant predictors in a statistical model. Collinearity is a severe problem when a model is trained on data from one region or time, and predicted to another with a different or unknown structure of collinearity. To demonstrate the reach of the problem of collinearity in ecology, we show how relationships among predictors differ between biomes, change over spatial scales and through time. Across disciplines, different approaches to addressing collinearity problems have been developed, ranging from clustering of predictors, threshold‐based pre‐selection, through latent variable methods, to shrinkage and regularisation. Using simulated data with five predictor‐response relationships of increasing complexity and eight levels of collinearity we compared ways to address collinearity with standard multiple regression and machine‐learning approaches. We assessed the performance of each approach by testing its impact on prediction to new data. In the extreme, we tested whether the methods were able to identify the true underlying relationship in a training dataset with strong collinearity by evaluating its performance on a test dataset without any collinearity. We found that methods specifically designed for collinearity, such as latent variable methods and tree based models, did not outperform the traditional GLM and threshold‐based pre‐selection. Our results highlight the value of GLM in combination with penalised methods (particularly ridge) and threshold‐based pre‐selection when omitted variables are considered in the final interpretation. However, all approaches tested yielded degraded predictions under change in collinearity structure and the ‘folk lore’‐thresholds of correlation coefficients between predictor variables of |r| >0.7 was an appropriate indicator for when collinearity begins to severely distort model estimation and subsequent prediction. The use of ecological understanding of the system in pre‐analysis variable selection and the choice of the least sensitive statistical approaches reduce the problems of collinearity, but cannot ultimately solve them.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1111/j.1600-0587.2012.07348.x

Cites

8 of 8 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 80

  1. Abdi H, 2003, Encyclopedia of social sciences research methods, 792
  2. Aichison J, 2003, The statistical analysis of compositional data
  3. Alin A, 2010, Multicollinearity, WIREs Computational Statistics, 10.1002/wics.84 · DOI
  4. Araújo M. B., 2006, How does climate change affect biodiversity?, Science, 313, 1396, 10.1126/science.1131758 · DOI
  5. Aucott L. S., 2000, Regression methods for high dimensional multicollinear data, Comm. Stat. Simul. Comput., 29, 1021, 10.1080/03610910008813652 · DOI
  6. Austin M. P, 1980, Searching for a model for use in vegetation analysis, Vegetatio, 42, 11, 10.1007/BF00048865 · DOI
  7. Austin M. P, 2002, Spatial prediction of species distribution: an interface between ecological theory and statistical modelling, Ecol. Model., 157, 101, 10.1016/S0304-3800(02)00205-3 · DOI
  8. Battin J., 2006, Cross‐scale correlations and the design and analysis of avian habitat selection studies, Condor, 108, 59, 10.1093/condor/108.1.59 · DOI
  9. Belsley D. A, 1991, Conditioning diagnostics: collinearity and weak data regression
  10. Bondell H. D., 2007, Simultaneous regression shrinkage, variable selection, and supervised clustering of predictors with OSCAR, Biometrics, 64, 115, 10.1111/j.1541-0420.2007.00843.x · DOI
  11. Booth G. D., 1994, Identifying proxy sets in multiple linear regression: an aid to better coefficient interpretation, US Dept of Agriculture, Forest Service
  12. Bortz J, 1993, Statistik für Sozialwissenschaftler
  13. Brauner N., 1998, Role of range and precision of the independent variable in regression of data, Am. Inst. Chem. Eng. J., 44, 603, 10.1002/aic.690440311 · DOI
  14. Buja A., 1989, Linear smoothers and additive models, Ann. Stat., 17, 453
  15. Cook R. D., 1991, Discussion of Li (1991), J. Am. Stat. Assoc., 86, 328
  16. De Veaux R. D., 1994, Selecting models from data: AI and statistics IV, 293
  17. Deconinck E., 2008, Boosted regression trees, multivariate adaptive regression splines and their two‐step combinations with multiple linear regression or partial least squares to predict blood‐brain barrier passage: a case study, Anal. Chim. Acta, 609, 13, 10.1016/j.aca.2007.12.033 · DOI
  18. Ding C., 2004, K‐means clustering via principal component analysis, Proc. Int. Conf. Machine Learn., 225
  19. Dobson A. J, 2002, An introduction to generalized linear models
  20. Douglass D. H., 2003, Test for harmful collinearity among predictor variables used in modeling global temperature, Clim. Res., 24, 15, 10.3354/cr024015 · DOI
  21. Echambadi R., 2006, Encouraging best practice in quantitative management research: an incomplete list of opportunities, J. Manage. Studies, 43, 1803, 10.1111/j.1467-6486.2006.00660.x · DOI
  22. Elith J., 2006, Novel methods improve prediction of species’ distributions from occurrence data, Ecography, 29, 129, 10.1111/j.2006.0906-7590.04596.x · DOI · Paper page
  23. Fan R.‐E, 2005, Working set selection using second order information for training SVM, J. Machine Learn. Res., 6, 1889
  24. Faraway J. J, 2005, Linear models with R
  25. Fox J., 1992, Generalized collinearity diagnostics, J. Am. Stat. Assoc., 87, 178, 10.1080/01621459.1992.10475190 · DOI
  26. Fraley C., 1998, How many clusters? Which clustering method? Answers via model‐based cluster analysis, Comput. J., 41, 578, 10.1093/comjnl/41.8.578 · DOI
  27. Freckleton R. P, 2002, On the misuse of residuals in ecology: regression of residuals vs. multiple regression, J. Anim. Ecol., 71, 542, 10.1046/j.1365-2656.2002.00618.x · DOI
  28. Friedman J., 2010, Regularization paths for generalized linear models via coordinate descent, J. Stat. Softw., 33, 1, 10.18637/jss.v033.i01 · DOI · Paper page
  29. Friedman J. H, 1991, Multivariate adaptive regression splines, Annu. Stat., 19, 1
  30. Friedman J. H., 2000, Additive logistic regression: a statistical view of boosting, Ann. Stat., 28, 337, 10.1214/aos/1016218223 · DOI · Paper page
  31. Gelman A., 2007, Data analysis using regression and multilevel/hierarchical models
  32. Grace J. B, 2006, Structural equation modeling and natural systems, 10.1017/CBO9780511617799 · DOI · Paper page
  33. Graham M. H, 2003, Confronting multicollinearity in ecological multiple regression, Ecology, 84, 2809, 10.1890/02-3114 · DOI
  34. Guerard J., 1989, The handbook of financial modeling: the financial executive’s reference guide to accounting, finance, and investment models
  35. Gunst R. F., 1980, Regression analysis and its application: a data‐oriented approach
  36. Gunst R. F., 1976, A comparison of least squares and latent root regression estimators, Technometrics, 18, 75, 10.2307/1267919 · DOI
  37. Hair J. F. Jr, 1995, Multivariate data analysis
  38. Hamilton D, 1987, Sometimes R2 > r2yx1 + r2yx2. Correlated variables are not always redundant, Am. Stat., 41, 129
  39. Harrell F. E. Jr, 2001, Regression modeling strategies – with applications to linear models, logistic regression, and survival analysis, 10.1007/978-1-4757-3462-1 · DOI
  40. Hawkins D. M, 1973, On the investigation of alternative regression by principal components analysis, Appl. Stat., 22, 275, 10.2307/2346776 · DOI
  41. Hawkins D. M., 1982, The Cholesky factorization of the inverse correlation or covariance matrix in multiple regression, Technometrics, 24, 191, 10.1080/00401706.1982.10487758 · DOI
  42. HilleRisLambers J., 2006, Hierarchical modelling for the environmental sciences, 59
  43. Hoerl A. E., 1970, Ridge regression: biased estimation for non‐orthogonal problems, Technometrics, 12, 55, 10.1080/00401706.1970.10488634 · DOI · Paper page
  44. Jain A. K., 1999, Data clustering: a review, ACM Comput. Surv., 31, 264, 10.1145/331499.331504 · DOI
  45. Johnston J, 1984, Econometric methods
  46. Joliffe I. T, 2002, Principal component analysis
  47. Kearney M., 2008, Mechanistic niche modelling: combining physiological and spatial data to predict species’ ranges, Ecol. Lett., 12, 334, 10.1111/j.1461-0248.2008.01277.x · DOI
  48. Kiers H. A. L., 2007, A comparison of various methods for multivariate regression with highly collinear variables, Stat. Methods Appl., 16, 193, 10.1007/s10260-006-0025-5 · DOI
  49. Kohonen T, 2001, Self‐organizing maps, 10.1007/978-3-642-56927-2 · DOI
  50. Lebart L., 1995, Statistique exploratoire multidimensionelle
  51. Li K. C, 1991, Sliced inverse regression for dimension reduction (with discussion), J. Am. Stat. Assoc., 86, 316, 10.1080/01621459.1991.10475035 · DOI
  52. Li K. C, 1992, On principal Hessian directions for data visualization and dimension reduction: another application of Stein's lemma, J. Am. Stat. Assoc., 87, 1025, 10.1080/01621459.1992.10476258 · DOI
  53. Mela C. F., 2002, The impact of collinearity on regression analysis: the asymmetric effect of negative and positive correlations, Appl. Econ., 34, 667, 10.1080/00036840110058482 · DOI
  54. Meloun M., 2002, Crucial problems in regression modelling and their solutions, Analyst, 127, 433, 10.1039/b110779h · DOI
  55. Mikolajczyk R. T., 2008, Evaluation of logistic regression reporting in current obstetrics and gynecology literature, Obstetrics Gynecol., 111, 413, 10.1097/AOG.0b013e318160f38e · DOI
  56. Morlini I, 2006, On multicollinearity and concurvity in some nonlinear multivariate models, Stat. Methods Appl., 15, 3, 10.1007/s10260-006-0005-9 · DOI
  57. Murray C. J. L., 2006, Eight Americas: investigating mortality disparity across races, counties, and race‐counties in the United States, PLoS Med., 3, e260, 10.1371/journal.pmed.0030260 · DOI · Paper page
  58. Murray K., 2009, Methods to quantify variable importance: implications for the analysis of noisy ecological data, Ecology, 90, 348, 10.1890/07-1929.1 · DOI
  59. Murwira A., 2005, The response of elephants to the spatial heterogeneity of vegetation in a southern African agricultural landscape, Landscape Ecol., 20, 217, 10.1007/s10980-004-3159-6 · DOI
  60. Ohlemüller R., 2008, The coincidence of climatic and species rarity: high risk to small‐range species from climate change, Biol. Lett., 4, 568, 10.1098/rsbl.2008.0097 · DOI
  61. Rawlings J. O., 1998, Applied regression analysis: a research tool, 10.1007/b98890 · DOI
  62. Reineking B., 2006, Constrain to perform: regularization of habitat models, Ecol. Model., 193, 675, 10.1016/j.ecolmodel.2005.10.003 · DOI
  63. Schmidt K. S., 2004, Mapping coastal vegetation using an expert system and hyperspectral imagery, Photogr. Eng. Remote Sens., 70, 703, 10.14358/PERS.70.6.703 · DOI
  64. Shana Y., 2006, Machine learning of poorly predictable ecological data, Ecol. Model., 195, 129, 10.1016/j.ecolmodel.2005.11.015 · DOI
  65. Smith A., 2009, Confronting collinearity: comparing methods for disentangling the effects of habitat loss and fragmentation, Landscape Ecol., 24, 1271, 10.1007/s10980-009-9383-3 · DOI
  66. Stewart G. W, 1987, Collinearity and least squares regression, Stat. Sci., 2, 68
  67. Suzuki N., 2008, Developing landscape habitat models for rare amphibians with small geographic ranges: a case study of Siskiyou Mountains salamanders in the western USA, Biodivers. Conserv., 17, 2197, 10.1007/s10531-007-9281-4 · DOI
  68. Synes N. W., 2011, Choice of predictor variables as a source of uncertainty in continental‐scale species distribution modelling under climate change, Global Ecol. Biogeogr., 20, 904, 10.1111/j.1466-8238.2010.00635.x · DOI
  69. Tabachnick B., 1989, Using multivariate statistics
  70. Thuiller W, 2004, Patterns and uncertainties of species’ range shifts under climate change, Global Change Biol., 10, 2020, 10.1111/j.1365-2486.2004.00859.x · DOI
  71. Vigneau E., 1996, Application of latent root regression for calibration in near‐infrared spectroscopy, Chemometr. Intell. Lab. Syst., 35, 231, 10.1016/S0169-7439(96)00051-2 · DOI
  72. Vigneau E., 1997, Principal component regression, ridge regression and ridge principal component regression in spectroscopy calibration, J. Chemometr., 11, 239, 10.1002/(SICI)1099-128X(199705)11:3<239::AID-CEM470>3.0.CO;2-A · DOI
  73. Vigneau E., 2002, A new method of regression on latent variables. Application to spectral data, Chemomet. Intell. Lab. Syst., 63, 7, 10.1016/S0169-7439(02)00032-1 · DOI
  74. Webster J. T., 1974, Latent root regression analysis, Technometrics, 16, 513, 10.1080/00401706.1974.10489232 · DOI
  75. Weisberg S, 2008, dr: methods for dimension reduction for regression, R package ver. 3.0.3
  76. Wen X., 2007, Optimal sufficient dimension reduction in regressions with categorical predictors, J. Stat. Infer. Plan., 137, 1961, 10.1016/j.jspi.2006.05.008 · DOI
  77. Wheeler D. C, 2007, Diagnostic tools and a remedial method for collinearity in geographically weighted regression, Environ. Plan. A, 39, 2464, 10.1068/a38325 · DOI
  78. Zha H., 2001, Spectral relaxation for K‐means clustering, Neural Inform. Process. Syst., 14, 1057
  79. Zou H., 2005, Regularization and variable selection via the elastic net, J. R. Stat. Soc. B, 67, 301, 10.1111/j.1467-9868.2005.00503.x · DOI · Paper page
  80. Zuur A. F., 2009, A protocol for data exploration to avoid common statistical problems, Methods Ecol. Evol., 1, 3, 10.1111/j.2041-210X.2009.00001.x · DOI · Paper page

Cited by

Sorted by citations

Authors
Dormann, C. F., Elith, J., Bacher, S., Buchmann, C., Carl, G., Carré, G., Marquéz, J. R. G., Gruber, B., Lafourcade, B., Leitão, P. J., Münkemüller, T., McClean, C., Osborne, P. E., Reineking, B., Schröder, B., Skidmore, A. K., Zurell, D., & Lautenbach, S.
Venue
Ecography
Publisher
Wiley
Published
2012-05-18
Volume
36
Issue
1
Pages
27-46
ISSN
  • 0906-7590
  • 1600-0587
DOI
10.1111/j.1600-0587.2012.07348.x
Cited by
9,998
References
8
Language
en
Licence
onlinelibrary.wiley.com/termsAndConditions#vor (vor)

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review