Methodological Urban Legends: The Misuse of Statistical Control Variables
The DOI matches its Crossref record; 41 of the 41 references it cites were found in a registry or an index.
Abstract
The automatic or blind inclusion of control variables in multiple regression and other analyses, intended to purify observed relationships among variables of interest, is widespread and can be considered an example of practice based on a methodological urban legend. Inclusion of such variables in most cases implicitly assumes that the control variables are somehow either contaminating the measurement of the variables of interest or affecting the underlying constructs, thus distorting observed relationships among them. There are, however, a number of alternative mechanisms that would produce the same statistical results, thus throwing into question whether inclusion of control variables has led to more or less accurate interpretation of results. The authors propose that researchers should be explicit rather than implicit regarding the role of control variables and match hypotheses precisely to both the choice of variables and the choice of analyses. The authors further propose that researchers avoid testing models in which demographic variables serve as proxies for variables that are of real theoretical interest in their data.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Thus, no correction for these variables was applied [68].”
From Adaptive working memory training does not produce transfer effects in cognition and neuroimaging (2022) · cites this paper as [68]
In the citing paper
“there is converging evidence that control variables should be used with caution (Spector and Brannick, 2011; Bernerth and Aguinis, 2016).”
From Race and Gender Differences in Undergraduate Research Mentoring Structures and Research Outcomes (2017) · cites this paper as Spector and Brannick, 2011
In the citing paper
“For model parsimony, and in accordance with guidelines from Spector and Brannick (2011) regarding the misuse of demographic variables as covariates, the final models did not include these demographic control variables.”
From Time Spent on Social Media and Associations with Mental Health in Young Adults: Examining TikTok, Twitter, Instagram, Facebook, Youtube, Snapchat, and Reddit (2025) · cites this paper as Spector and Brannick (2011)
In the citing paper
“Longitudinal designs provide stronger evidence, but are prone to residual confounding [[36], [37], [38]] and other forms of bias that limit causal inference [[39], [40], [41], [42], [43]].”
From Improving sleep quality leads to better mental health: A meta-analysis of randomised controlled trials (2021) · cites this paper as [37]
Citations
Cites
Convergent and discriminant validation by the multitrait-multimethod matrix.
Campbell, D. T., & Fiske, D. W. (1959) · Psychological Bulletin · doi:10.1037/h0046016 · Cited by 17,116
(2002) · Cited by 13,211
Statistical Methods for Research Workers
(1945) · American Journal of Clinical Pathology · doi:10.1093/ajcp/15.11.552b · Cited by 10,880
Equivalence of the Mediation, Confounding and Suppression Effect
MacKinnon, D. P., Krull, J. L., & Lockwood, C. M. (2000) · Prevention Science · doi:10.1023/a:1026595011371 · Cited by 4,188
(1982) · Cited by 3,737
Health complaints, stress, and distress: Exploring the central role of negative affectivity.
Watson, D., & Pennebaker, J. W. (1989) · Psychological Review · doi:10.1037/0033-295x.96.2.234 · Cited by 3,030
Becker, T. E. (2005) · Organizational Research Methods · doi:10.1177/1094428105278021 · Cited by 2,219
Identifiability and exchangeability for direct and indirect effects
Robins, J. M., & Greenland, S. (1992) · Epidemiology · doi:10.1097/00001648-199203000-00013 · Cited by 1,793
Statistical Methods for Research Workers
(1926) · Journal of Comparative Pathology and Therapeutics · doi:10.1016/s0368-1742(26)80018-9 · Cited by 1,283
Work Process and Quality of Care in Early Childhood Education: The Role of Job Crafting
Leana, C., Appelbaum, E., & Shevchuk, I. (2009) · Academy of Management Journal · doi:10.5465/amj.2009.47084651 · Cited by 726
Spector, P. E., Zapf, D., Chen, P. Y., & Frese, M. (2000) · Journal of Organizational Behavior · doi:10.1002/(sici)1099-1379(200002)21:1<79::aid-job964>3.0.co;2-g · Cited by 581 · cited 1×
“A comparison of results including and excluding neuroticism from stressor–strain relationships has been used to address this issue, showing that in some cases most of the observed stressor–strain relationships could be accounted for by NA, whereas in others NA could be ruled out (Spector et al., 2000).”
ResultSelf-citationHill, E. J. (2005) · Journal of Family Issues · doi:10.1177/0192513x05277542 · Cited by 572
Kennedy, M. T., & Fiss, P. C. (2009) · Academy of Management Journal · doi:10.5465/amj.2009.44633062 · Cited by 474
Bosses' Perceptions of Family-Work Conflict and Women's Promotability: Glass Ceiling Effects
Hoobler, J. M., Wayne, S. J., & Lemmon, G. (2009) · Academy of Management Journal · doi:10.5465/amj.2009.44633700 · Cited by 460
SPECTOR, P. E., COOPER, C. L., POELMANS, S., ALLEN, T. D., O'DRISCOLL, M., SANCHEZ, J. I., SIU, O. L., DEWE, P., HART, P., & LU, L. (2004) · Personnel Psychology · doi:10.1111/j.1744-6570.2004.tb02486.x · Cited by 455
Self-citationSpector, P. E., Dwyer, D. J., & Jex, S. M. (1988) · Journal of Applied Psychology · doi:10.1037/0021-9010.73.1.11 · Cited by 419
Self-citationPromarket Reforms and Firm profitability in Developing Countries
Cuervo-Cazurra, A., & Dau, L. A. (2009) · Academy of Management Journal · doi:10.5465/amj.2009.47085192 · Cited by 418
Williams, L. J., & Anderson, S. E. (1994) · Journal of Applied Psychology · doi:10.1037//0021-9010.79.3.323 · Cited by 386
Van Dyne, L., Vandewalle, D., Kostova, T., Latham, M. E., & Cummings, L. L. (2000) · Journal of Organizational Behavior · doi:10.1002/(sici)1099-1379(200002)21:1<3::aid-job47>3.0.co;2-6 · Cited by 342
Self-citationShaw, J. D., Dineen, B. R., Fang, R., & Vellella, R. F. (2009) · Academy of Management Journal · doi:10.5465/amj.2009.44635525 · Cited by 329 · cited 1×
“where NA remained a viable alternative explanation, the mechanisms that produced the reduction in observed relationships were not clear, and in fact Spector et al. (2000) provided evidence for several mechanisms that refuted the contamination hypothesis.”
ResultCausal Inference, Path Analysis, and Recursive Structural Equations Models
Holland, P. W. (1988) · Sociological Methodology · doi:10.2307/271055 · Cited by 297
Pil, F. K., & Leana, C. (2009) · Academy of Management Journal · doi:10.5465/amj.2009.47084647 · Cited by 280
Beyond Negative Affectivity: Measuring Stress and Satisfaction in the Workplace
Watson, D., Pennebaker, J. W., & Folger, R. (1987) · Journal of Organizational Behavior Management · doi:10.1300/j075v08n02_09 · Cited by 273
Nuisance variables and the ex post facto design
Meehl, P. E. (1970) · University of Minnesota Digital Conservancy (University of Minnesota) · Cited by 241
High school yearbooks: A reply to Schwarz.
Meehl, P. E. (1971) · Journal of Abnormal Psychology · doi:10.1037/h0030750 · Cited by 215 · cited 2×
“We differ from others who have discussed this issue (e.g., Breaugh, 2008; Meehl, 1971) in focusing on the distinction between factors that affect the measurement of a construct versus the underling construct itself.”
BackgroundWhen Does Scientist Recruitment Affect Technological Repositioning?
Tzabbar, D. (2009) · Academy of Management Journal · doi:10.5465/amj.2009.44632853 · Cited by 207
Lowin, A., & Craig, J. R. (1968) · Organizational Behavior and Human Performance · doi:10.1016/0030-5073(68)90020-2 · Cited by 199
Public-Private Institutions as Catalysts of Upgrading in Emerging Market Societies
McDermott, G. A., Corredoira, R. A., & Kruse, G. (2009) · Academy of Management Journal · doi:10.5465/amj.2009.47084929 · Cited by 197
Madsen, P. M. (2009) · Academy of Management Journal · doi:10.5465/amj.2009.47085173 · Cited by 175 · cited 1×
“Two used the term ‘‘impact’’ (Leana, Appelbaum, & Shevchuk, 2009; Madsen, 2009).”
He, J., & Wang, H. C. (2009) · Academy of Management Journal · doi:10.5465/amj.2009.44633414 · Cited by 166
INTERPERSONAL AFFECT AND PERFORMANCE APPRAISAL: A FIELD STUDY
Varma, A., DeNisi, A. S., & Peters, L. H. (1996) · Personnel Psychology · doi:10.1111/j.1744-6570.1996.tb01803.x · Cited by 152
Analyses of Theories and Methods of Physics and Psychology
Radner, M., & Winokur, S. (1972) · Cited by 137
The unmeasured variables problem in path analysis.
James, L. R. (1980) · Journal of Applied Psychology · doi:10.1037/0021-9010.65.4.415 · Cited by 130 · cited 1×
“An important consideration in the design of studies intended to test for possible causal connections is the unmeasured variables problem (James, 1980).”
BackgroundMauro, R. (1990) · Psychological Bulletin · doi:10.1037/0033-2909.108.2.314 · Cited by 130
An investigation of age-related factors in the age-job-satisfaction relationship.
White, A. T., & Spector, P. E. (1987) · Psychology and Aging · doi:10.1037//0882-7974.2.3.261 · Cited by 114
Self-citationThe Infrastructure of Collective Action and Policy Content Diffusion in the Organic Food Industry
Lee, B. H. (2009) · Academy of Management Journal · doi:10.5465/amj.2009.47084925 · Cited by 104 · cited 1×
“Four noted that the control variables might ‘‘influence’’ other variables (Hoobler, Wayne, & Lemmon, 2009; Lee, 2009; Marquis & Huang, 2009; Tzabbar, 2009), for example ‘‘I controlled for a host of variables that might influence policy dynamics’’ (Lee, 2009, p.”
BackgroundBidwell, M. J., & Briscoe, F. (2009) · Academy of Management Journal · doi:10.5465/amj.2009.47084649 · Cited by 99
Breaugh, J. A. (2008) · Human Resource Management Review · doi:10.1016/j.hrmr.2008.03.001 · Cited by 94
The Contingent Nature of Public Policy and the Growth of U.S. Commercial Banking
Marquis, C., & Huang, Z. (2009) · Academy of Management Journal · doi:10.5465/amj.2009.47084828 · Cited by 45
On the inadequacy of the partial and multiple correlation technique.
Burks, B. S. (1926) · Journal of Educational Psychology · doi:10.1037/h0070156 · Cited by 25 · cited 1×
“Such cautions are far from new and can be found as far back as the 1920s (Burks, 1926) suggesting that intemperate use of statistical controls has been common practice for generations.”
BackgroundOn the inadequacy of the partial and multiple correlation technique.
Burks, B. S. (1926) · Journal of Educational Psychology · doi:10.1037/h0075654 · Cited by 10
Research works printed but not linked: 5
- Atinc, G., Paper presented at the Southern Management Association
- Meade, A.W., Behrend, T.S. & Lance, C.E. ( 2009). Dr. StrangeLOVE, or: How I learned to stop worrying and love omitted variables. In C. E. Lance & R. J. Vandenberg (Eds.), Statistical and methodological myths and urban legends: Doctrine, verity and fable in the organizational and social sciences (pp. 89-106). New York, NY: Routledge/Taylor & Francis Group.
- Nunnally, J.C., 1994, Psychometric theory
- 10.1037/0882-7974.2.3.261 · doi:10.1037/0882-7974.2.3.261
- 10.1037/0021-9010.79.3.323 · doi:10.1037/0021-9010.79.3.323 · Paper page
Cited by
A Critical Review and Best‐Practice Recommendations for Control Variable Usage
Bernerth, J. B., & Aguinis, H. (2016) · Personnel Psychology · doi:10.1111/peps.12103 · Cited by 1,731
“As a result, there is a common albeit often unsubstantiated belief that the inclusion of control variables purifies results and uncovers “true” relationships (Atinc et al., 2012, p. 59; Carlson & Wu, 2012; Spector & Brannick, 2011).”
BackgroundResultInfluentialDo Not Cross Me: Optimizing the Use of Cross-Sectional Designs
Spector, P. E. (2019) · Journal of Business and Psychology · doi:10.1007/s10869-018-09613-8 · Cited by 1,366
“…correlations in metaanalyses do not uniformly find larger correlations from cross-sectional designs (e.g., Nixon, Mazzola, Bauer, Krueger, & Spector, 2011; Pindek & Spector, 2016), and even when cross-sectional correlations are larger, it is not necessarily due to common method variance.”
BackgroundSelf-citationPsychological Safety: A Meta‐Analytic Review and Extension
Frazier, M. L., Fainshmidt, S., Klinger, R. L., Pezeshkan, A., & Vracheva, V. (2017) · Personnel Psychology · doi:10.1111/peps.12183 · Cited by 1,231
Embracing Causal Complexity: The Emergence of a Neo-Configurational Perspective
Misangyi, V. F., Greckhamer, T., Furnari, S., Fiss, P. C., Crilly, D., & Aguilera, R. (2017) · Journal of Management · doi:10.1177/0149206316679252 · Cited by 1,099
Scott, A. J., Webb, T. L., Martyn-St James, M., Rowse, G., & Weich, S. (2021) · Sleep Medicine Reviews · doi:10.1016/j.smrv.2021.101556 · Cited by 1,094
Metadata
- Authors
- Spector, P. E., & Brannick, M. T.
- Affiliations
- University of South Florida (US)
- Department of Psychology, University of South Florida, Tampa, USA
- Corresponding author
- Spector, Paul E.
- ORCID
- Spector, Paul E.
- Venue
- Organizational Research Methods
- Publisher
- SAGE Publications
- Published
- 2010-06-02
- Volume
- 14
- Issue
- 2
- Pages
- 287-305
- ISSN
- 1094-4281 (ISSN-L)
- 1552-7425
- OpenAlex
- W2001810891
- Cited by
- 1,146
- References
- 41
- Citations by year
- 2026: 111 · 2025: 89 · 2024: 64 · 2023: 103 · 2022: 101 · 2021: 89 · 2020: 89 · 2019: 80 · 2018: 81 · 2017: 70 · 2016: 72 · 2015: 55 · 2014: 58 · 2013: 40 · 2012: 26
- FWCI
- 21.24
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
- Locations
- Organizational Research Methods (publishedVersion)
- Digital Commons - University of South Florida (University of South Florida) (submittedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Applications
- Social Sciences › Business, Management and Accounting › Strategy and Management › Business Strategies and Management Research
- Keywords
- control variables, multiple linear regression, confounding, feature selection, statistical control, measurement contamination
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.