The DOI matches its Crossref record; 88 of the 88 references it cites were found in a registry or an index.
Common Methods Bias: Does Common Methods Variance Really Bias Results?
Abstract
Methods variance and its effects are at the center of a debate in organizational science. Most of the debate, however, is focused on the prevalence of common methods variance and ignores common methods bias, or the divergence between observed and true relationships among constructs. This article assesses the level of common methods bias in all multitrait-multimethod correlation matrices published over a 12-year period in a set of six social science journals using a combination of structural equation modeling and meta-analysis. The results indicate that only 46% of the variation in measures is attributable to the constructs, that 32% of the observed variation in measures is attributable to common methods variance, and that common methods variance results in a 26% bias in the observed relationships among constructs. This level of bias is cause for concern but does not invalidate many research findings.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“To help mitigate concerns about effects of transient sources of common method bias, we employed a lagged longitudinal design (Study 2; Doty and Glick, 1998;Podsakoff et al., 2003;Spector, 2006).”
From Work hard or play hard: the effect of leisure crafting on opportunity recognition and venture performance (2023) · cites this paper as Doty and Glick, 1998
Citations
Cites
Intraclass correlations: Uses in assessing rater reliability.
Shrout, P. E., & Fleiss, J. L. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.2.420 · Cited by 24,532
Significance tests and goodness of fit in the analysis of covariance structures.
Bentler, P. M., & Bonett, D. G. (1980) · Psychological Bulletin · doi:10.1037/0033-2909.88.3.588 · Cited by 19,933
Convergent and discriminant validation by the multitrait-multimethod matrix.
Campbell, D. T., & Fiske, D. W. (1959) · Psychological Bulletin · doi:10.1037/h0046016 · Cited by 17,116
The Handbook of Social Psychology
Gravatt, A. E., Lindzey, G. E., & Aronson, F. (1969) · The Family Coordinator · doi:10.2307/582242 · Cited by 12,518
Construct validity in psychological tests.
Cronbach, L. J., & Meehl, P. E. (1955) · Psychological Bulletin · doi:10.1037/h0040957 · Cited by 11,195
Wasserman, S. S., Hedges, L. V., & Olkin, I. (1988) · Journal of Educational Statistics · doi:10.2307/1164953 · Cited by 10,377
Motivation through the design of work: test of a theory
Hackman, J. R., & Oldham, G. R. (1976) · Organizational Behavior and Human Performance · doi:10.1016/0030-5073(76)90016-7 · Cited by 8,962
LISREL 7 : user's reference guide
Jöreskog, K. G., & Sörbom, D. (1989) · Medical Entomology and Zoology · Cited by 6,783
A Social Information Processing Approach to Job Attitudes and Task Design
Salancik, G. R., & Pfeffer, J. (1978) · Administrative Science Quarterly · doi:10.2307/2392563 · Cited by 4,372
Statistical Methods for Meta-Analysis
(1985) · Elsevier eBooks · doi:10.1016/c2009-0-03396-0 · Cited by 2,564
Measurement of human service staff satisfaction: Development of the Job Satisfaction Survey
Spector, P. E. (1985) · American Journal of Community Psychology · doi:10.1007/bf00929796 · Cited by 2,188
Representing and Testing Organizational Theories: A Holistic Construal
Bagozzi, R. P., & Phillips, L. W. (1982) · Administrative Science Quarterly · doi:10.2307/2392322 · Cited by 2,064
Lack of method variance in self-reported affect and perceptions at work: reality or artifact?
Williams, L. J., Cote, J. A., & Buckley, M. R. (1989) · Journal of Applied Psychology · doi:10.1037/0021-9010.74.3.462 · Cited by 1,616
Structural Equation Modeling with LISREL: Essentials and Advances.
Land, K. C., & Hayduk, L. A. (1990) · Social Forces · doi:10.2307/2579660 · Cited by 1,313
Phillips, L. W. (1981) · Journal of Marketing Research · doi:10.1177/002224378101800401 · Cited by 1,276
Crampton, S. M., & Wagner, J. A. (1994) · Journal of Applied Psychology · doi:10.1037/0021-9010.79.1.67 · Cited by 1,224
Feldman, J. M., Hunter, J. E., Schmidt, F. L., & Jackson, G. B. (1984) · Academy of Management Review · doi:10.2307/258247 · Cited by 1,203
Expectations and Norms in Models of Consumer Satisfaction
Cadotte, E. R., Woodruff, R. B., & Jenkins, R. L. (1987) · Journal of Marketing Research · doi:10.2307/3151641 · Cited by 1,169
Hierarchically Nested Covariance Structure Models for Multitrait-Multimethod Data
Widaman, K. F. (1985) · Applied Psychological Measurement · doi:10.1177/014662168500900101 · Cited by 1,037
Medsker, G. J., Williams, L. J., & Holahan, P. J. (1994) · Journal of Management · doi:10.1016/0149-2063(94)90022-1 · Cited by 971
Spector, P. E. (1987) · Journal of Applied Psychology · doi:10.1037/0021-9010.72.3.438 · Cited by 956
Role of social desirability in personality testing for personnel selection: The red herring.
Öneş, D. S., Viswesvaran, C., & Reiss, A. D. (1996) · Journal of Applied Psychology · doi:10.1037/0021-9010.81.6.660 · Cited by 924
An Examination of Need-Satisfaction Models of Job Attitudes
Salancik, G. R., & Pfeffer, J. (1977) · Administrative Science Quarterly · doi:10.2307/2392182 · Cited by 849
The Measurement of Job Characteristics
Sims, H. P., Szilagyi, A. D., & Keller, R. T. (1976) · Academy of Management Journal · doi:10.2307/255772 · Cited by 805
Meta-Analysis: Cumulating Research Findings across Studies
Hunter, J. E., Schmidt, F. L., & Jackson, G. B. (1986) · Educational Researcher · doi:10.2307/1175263 · Cited by 792
Estimating Trait, Method, and Error Variance: Generalizing across 70 Construct Validation Studies
Cote, J. A., & Buckley, M. R. (1987) · Journal of Marketing Research · doi:10.2307/3151642 · Cited by 750
Should negative affectivity remain an unmeasured variable in the study of job stress?
Brief, A. P., Burke, M. J., George, J. M., Robinson, B. S., & Webster, J. (1988) · Journal of Applied Psychology · doi:10.1037/0021-9010.73.2.193 · Cited by 748
THEORY OF GENERALIZABILITY: A LIBERALIZATION OF RELIABILITY THEORY†
Cronbach, L. J., Rajaratnam, N., & Gleser, G. C. (1963) · British Journal of Statistical Psychology · doi:10.1111/j.2044-8317.1963.tb00206.x · Cited by 737
Response Effects in Surveys: A Review and Synthesis.
King, B. F., Sudman, S., & Bradburn, N. M. (1976) · Journal of the American Statistical Association · doi:10.2307/2285784 · Cited by 644
The Reliability and Validity of Key Informant Data from Dyadic Relationships in Marketing Channels
John, G., & Reve, T. (1982) · Journal of Marketing Research · doi:10.1177/002224378201900412 · Cited by 632
International Review of Industrial and Organizational Psychology 2008
Hodgkinson, G. P., & Ford, J. K. (2008) · Wiley eBooks · doi:10.1002/9780470773277 · Cited by 613
Bagozzi, R. P., & Yi, Y. (1990) · Journal of Applied Psychology · doi:10.1037/0021-9010.75.5.547 · Cited by 612
Seidler, J. (1974) · American Sociological Review · doi:10.2307/2094155 · Cited by 603
Medsker, G. J., Williams, L. J., & Holahan, P. J. (1994) · Journal of Management · doi:10.1177/014920639402000207 · Cited by 581
Eden, D., & Leviatan, U. (1975) · Journal of Applied Psychology · doi:10.1037/0021-9010.60.6.736 · Cited by 535
Anderson, J. C. (1987) · Management Science · doi:10.1287/mnsc.33.4.525 · Cited by 453
Divergence between Archival and Perceptual Measures of the Environment: Causes and Consequences
Boyd, B., Dess, G. G., & Rasheed, A. M. A. (1993) · Academy of Management Review · doi:10.5465/amr.1993.3997514 · Cited by 434
Spector, P. E., Dwyer, D. J., & Jex, S. M. (1988) · Journal of Applied Psychology · doi:10.1037/0021-9010.73.1.11 · Cited by 419
Negative affectivity as the underlying cause of correlations between stressors and strains.
Chen, P. Y., & Spector, P. E. (1991) · Journal of Applied Psychology · doi:10.1037/0021-9010.76.3.398 · Cited by 400
Williams, L. J., & Anderson, S. E. (1994) · Journal of Applied Psychology · doi:10.1037/0021-9010.79.3.323 · Cited by 386
The job characteristics approach to task design: A critical review.
Roberts, K. H., & Glick, W. H. (1981) · Journal of Applied Psychology · doi:10.1037/0021-9010.66.2.193 · Cited by 351
Self-citationSpector, P. E., & Jex, S. M. (1991) · Journal of Applied Psychology · doi:10.1037/0021-9010.76.1.46 · Cited by 316
Method bias: The importance of theory and measurement
Schmitt, N. (1994) · Journal of Organizational Behavior · doi:10.1002/job.4030150504 · Cited by 314
Methodology Review: Analysis of Multitrait-Multimethod Matrices
Schmitt, N., & Stults, D. M. (1986) · Applied Psychological Measurement · doi:10.1177/014662168601000101 · Cited by 311
The multitrait-multirater approach to measuring managerial job performance.
Lawler, E. E. (1967) · Journal of Applied Psychology · doi:10.1037/h0025095 · Cited by 304
An Evaluation of the Validity of Correlational Research Conducted in Organizations
Mitchell, T. R. (1985) · Academy of Management Review · doi:10.5465/amr.1985.4277939 · Cited by 279
The Reliability and Validity of Key Informant Data from Dyadic Relationships in Marketing Channels
John, G., & Reve, T. (1982) · Journal of Marketing Research · doi:10.2307/3151724 · Cited by 272
The Nature and Effects of Method Variance in Organizational Research
Spector, P. E., & Brannick, M. Τ. (1995) · Digital Commons - University of South Florida (University of South Florida) · Cited by 261
Rater bias in performance ratings: Superior, self-, and peer ratings.
Holzbach, R. L. (1978) · Journal of Applied Psychology · doi:10.1037/0021-9010.63.5.579 · Cited by 249
Time urgency: The construct and its measurement.
Landy, F. J., Rastegary, H., Thayer, J. F., & Colvin, C. A. (1991) · Journal of Applied Psychology · doi:10.1037/0021-9010.76.5.644 · Cited by 227
Glick, W. H., Jenkins, G. D., & Gupta, N. (1986) · Academy of Management Journal · doi:10.5465/256218 · Cited by 218
Self-citationPath analysis: Psychological examples.
Werts, C. E., & Linn, R. L. (1970) · Psychological Bulletin · doi:10.1037/h0029778 · Cited by 187
Generalizability theory: 1973–1980
Shavelson, R. J., & Webb, N. M. (1981) · British Journal of Mathematical and Statistical Psychology · doi:10.1111/j.2044-8317.1981.tb00625.x · Cited by 165
Measures of Organizational Structure: A Methodological Note
Pennings, J. (1973) · American Journal of Sociology · doi:10.1086/225590 · Cited by 163
Validation of the Index of Organizational Reaction with the JDI, the MSQ, and Faces Scales.
Dunham, R. B., SMITH, F. J., & Blackburn, R. S. (1977) · Academy of Management Journal · doi:10.2307/255415 · Cited by 154
Gillet, B., & Schwab, D. P. (1975) · Journal of Applied Psychology · doi:10.1037/h0076751 · Cited by 146
The construct validity of burnout
Meier, S. T. (1984) · Journal of Occupational Psychology · doi:10.1111/j.2044-8325.1984.tb00163.x · Cited by 141
Structural Equation Modeling with LISREL: Essentials and Advances
Wardrop, J. L., Hayduk, L. A., & Bollen, K. A. (1993) · Journal of Educational Statistics · doi:10.2307/1165186 · Cited by 127
Miller, C. C., Glick, W. H., Wang, Y.-D., & Huber, G. P. (1991) · Academy of Management Journal · doi:10.2307/256447 · Cited by 122
Self-citationJohnson, S. M., Smith, P. C., & Tucker, S. M. (1982) · Journal of Applied Psychology · doi:10.1037/0021-9010.67.4.500 · Cited by 109
An exercise design approach to understanding assessment center dimension and exercise constructs.
Schneider, J. R., & Schmitt, N. (1992) · Journal of Applied Psychology · doi:10.1037/0021-9010.77.1.32 · Cited by 108
Pierce, J. L., & Dunham, R. B. (1978) · Academy of Management Journal · doi:10.2307/255668 · Cited by 106
A comparison of the behaviorally anchored rating and mixed standard scale formats.
Dickinson, T. L., & Zellinger, P. M. (1980) · Journal of Applied Psychology · doi:10.1037/0021-9010.65.2.147 · Cited by 105
Alternative Questionnaire Formats for Country Image Studies
Jaffe, E. D., & Nebenzahl, I. D. (1984) · Journal of Marketing Research · doi:10.2307/3151472 · Cited by 102
Rentz, J. O. (1987) · Journal of Marketing Research · doi:10.1177/002224378702400102 · Cited by 101
Becker, T. E., & Vance, R. J. (1993) · Journal of Management · doi:10.1016/0149-2063(93)90010-k · Cited by 87
Work, nonwork, and withdrawal.
Youngblood, S. A. (1984) · Journal of Applied Psychology · doi:10.1037/0021-9010.69.1.106 · Cited by 83
Institutional Versus Questionnaire Measures of Organizational Structure.
Sathe, V. (1978) · Academy of Management Journal · doi:10.2307/255756 · Cited by 78
Estimation Problems in the Block-Diagonal Model of the Multitrait-Multimethod Matrix
Brannick, M. Τ., & Spector, P. E. (1990) · Applied Psychological Measurement · doi:10.1177/014662169001400401 · Cited by 74
Construct validity of multiple job performance measures using confirmatory factor analysis.
Vance, R. J., MacCallum, R. C., Coovert, M. D., & Hedge, J. W. (1988) · Journal of Applied Psychology · doi:10.1037/0021-9010.73.1.74 · Cited by 74
INFLUENCE OF QUALITY OF WORK LIFE ON COMPANY AND UNION COMMITMENT.
Fields, M., & Thacker, J. W. (1992) · Academy of Management Journal · doi:10.2307/256382 · Cited by 74
Validation of an S-O-R Model for Situation, Enduring, and Response Components of Involvement
Arora, R. (1982) · Journal of Marketing Research · doi:10.2307/3151723 · Cited by 74
The job characteristics model in Hong Kong.
Birnbaum, P. H., Farh, J., & Wong, G. Y. (1986) · Journal of Applied Psychology · doi:10.1037/0021-9010.71.4.598 · Cited by 68
Productivity in organizations: new perspectives from industrial and organizational psychology
(1989) · Choice Reviews Online · doi:10.5860/choice.26-4561 · Cited by 68
Huber, G. P., Miller, C. C., & Glick, W. H. (1990) · Organization Science · doi:10.1287/orsc.1.1.11 · Cited by 55
Self-citationLeader-group interactions: A longitudinal field investigation.
Greene, C. N., & Schriesheim, C. A. (1980) · Journal of Applied Psychology · doi:10.1037/0021-9010.65.1.50 · Cited by 53
Becker, T. E., & Vance, R. J. (1993) · Journal of Management · doi:10.1177/014920639301900309 · Cited by 52
Rentz, J. O. (1987) · Journal of Marketing Research · doi:10.2307/3151750 · Cited by 47
Gupta, N., & Beehr, T. A. (1982) · Journal of Vocational Behavior · doi:10.1016/0001-8791(82)90058-6 · Cited by 46
Becker, T. E. (1994) · Journal of Management · doi:10.1016/0149-2063(94)90006-x · Cited by 46
Wolfe, J., & Jackson, C. N. (1987) · Journal of Management · doi:10.1177/014920638701300110 · Cited by 39
Salancik, G. R. (1984) · Journal of Management · doi:10.1177/014920638401000209 · Cited by 34
Becker, T. E., & Cote, J. A. (1994) · Journal of Management · doi:10.1177/014920639402000306 · Cited by 28
Measuring member satisfaction with the Church.
Silverman, W. H., Pargäment, K. I., Johnson, S., Echemendía, R. J., & Snyder, S. E. (1983) · Journal of Applied Psychology · doi:10.1037/0021-9010.68.4.664 · Cited by 19
Soutar, G. N., & WEAVER, J. R. (1982) · Journal of Occupational Psychology · doi:10.1111/j.2044-8325.1982.tb00075.x · Cited by 12
THE CONVERGENT AND DISCRIMINANT VALIDITIES BETWEEN THE IOR AND THE JDI: ENGLISH AND SPANISH FORMS.
McCabe, D. J., Dalessio, A., Briga, J., & Sasaki, J. (1980) · Academy of Management Journal · doi:10.2307/255565 · Cited by 12
Brass, D. J. (1979) · Illinois Digital Environment for Access to Learning and Scholarship (University of Illinois at Urbana-Champaign) · Cited by 3
Attribution processes in the use of performance feedback in an organization
Fichman, M. (1982) · Deep Blue (University of Michigan) · Cited by 1
Research works printed but not linked: 7
- 10.1177/002224378702400307 · DOI
- Campbell, D. T., 1982, New directions for methodology of social and behavioral science: Forms of validity in research, 93
- Doty, D. H., Annual Meetings of the Society of Industrial and Organizational Psychology
- Doty, D. H., National Meetings of the Academy of Management
- Fiske, D. W., 1982, New directions for methodology of social and behavioral science: Forms of validity in research, 77
- 10.2307/256218 · DOI
- Hackman, J. R., 1980, Work redesign
Cited by
Sorted by citations
Sources of method bias in social science research and recommendations on how to control it
Podsakoff, P. M., MacKenzie, S. B., & Podsakoff, N. P. (2012) · Annual Review of Psychology · doi:10.1146/annurev-psych-120710-100452 · Cited by 15,744
“In addition, they note that complex, abstract questions among the systematic error variance components, thereby increasing the bias in the observed relationships between constructs” (Doty & Glick 1998, p. 381). accurate responses is item ambiguity.”
InfluentialThe job demands-resources model of burnout.
Demerouti, E., Bakker, A. B., Nachreiner, F., & Schaufeli, W. B. (2001) · Journal of Applied Psychology · doi:10.1037/0021-9010.86.3.499 · Cited by 13,643
“The second source of data and the second level of analysis were added to avoid method artifacts caused by using only one single method or to find relations (within the individual) that can be predicted by merely applying principles of cognitive consistency (Doty & Click, 1998).”
Network Structure and Knowledge Transfer: The Effects of Cohesion and Range
Reagans, R., & McEvily, B. (2003) · Administrative Science Quarterly · doi:10.2307/3556658 · Cited by 3,592
Method Variance in Organizational Research: Truth or Urban Legend?
Spector, P. E. (2006) · Organizational Research Methods · doi:10.1177/1094428105284955 · Cited by 3,445
“Doty and Glick (1988) discussed the nature of methods, providing a taxonomy that classified measures along a facet of measurement technique versus data source.”
BackgroundMethodSteel, P. (2007) · Psychological bulletin · doi:10.1037/0033-2909.133.1.65 · Cited by 3,417
Metadata
- Authors
- Doty, D. H., & Glick, W. H.
- Affiliations
- Syracuse University (US)
- Arizona State University (US)
- ORCID
- Doty, D. Harold
- Venue
- Organizational Research Methods
- Publisher
- SAGE Publications
- Published
- 1998-10
- Volume
- 1
- Issue
- 4
- Pages
- 374-406
- ISSN
- 1094-4281 (ISSN-L)
- 1552-7425
- OpenAlex
- W2168082050
- Cited by
- 1,380
- References
- 84
- Citations by year
- 2026: 37 · 2025: 39 · 2024: 51 · 2023: 52 · 2022: 51 · 2021: 64 · 2020: 62 · 2019: 58 · 2018: 55 · 2017: 67 · 2016: 63 · 2015: 64 · 2014: 95 · 2013: 89 · 2012: 92
- FWCI
- 9.20
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Organizational Research Methods (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Social Sciences › Business, Management and Accounting › Strategy and Management › Business Strategies and Management Research
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Keywords
- measurement bias, construct relationships, measurement error, organizational analysis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review