Surveying for “artifacts”: The susceptibility of the OCB–performance evaluation relationship to common rater, item, and measurement context effects.
The DOI matches its Crossref record; 12 of the 12 references it cites were found in a registry or an index.
Abstract
Despite the increased attention paid to biases attributable to common method variance (CMV) over the past 50 years, researchers have only recently begun to systematically examine the effect of specific sources of CMV in previously published empirical studies. Our study contributes to this research by examining the extent to which common rater, item, and measurement context characteristics bias the relationships between organizational citizenship behaviors and performance evaluations using a mixed-effects analytic technique. Results from 173 correlations reported in 81 empirical studies (N = 31,146) indicate that even after controlling for study-level factors, common rater and anchor point number similarity substantially biased the focal correlations. Indeed, these sources of CMV (a) led to estimates that were between 60% and 96% larger when comparing measures obtained from a common rater, versus different raters; (b) led to 39% larger estimates when a common source rated the scales using the same number, versus a different number, of anchor points; and (c) when taken together with other study-level predictors, accounted for over half of the between-study variance in the focal correlations. We discuss the implications for researchers and practitioners and provide recommendations for future research.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Podsakoff, P. M., MacKenzie, S. B., Lee, J.-Y., & Podsakoff, N. P. (2003) · Journal of Applied Psychology · doi:10.1037/0021-9010.88.5.879 · Cited by 82,194
Self-citationSources of method bias in social science research and recommendations on how to control it
Podsakoff, P. M., MacKenzie, S. B., & Podsakoff, N. P. (2012) · Annual Review of Psychology · doi:10.1146/annurev-psych-120710-100452 · Cited by 15,744
Self-citationOrganizational Citizenship Behavior: It's Construct Clean-Up Time
Organ, D. W. (1997) · Human Performance · doi:10.1207/s15327043hup1002_2 · Cited by 2,807
Podsakoff, N. P., Whiting, S. W., Podsakoff, P. M., & Blume, B. D. (2009) · Journal of Applied Psychology · doi:10.1037/a0013079 · Cited by 2,356
Self-citationRichardson, H. A., Simmering, M. J., & Sturman, M. C. (2009) · Organizational Research Methods · doi:10.1177/1094428109332834 · Cited by 1,721
Transfer of Training: A Meta-Analytic Review
Blume, B. D., Ford, J. K., Baldwin, T. T., & Huang, J. L. (2010) · Journal of Management · doi:10.1177/0149206309352880 · Cited by 1,477
The Forgotten Ones? The Validity of Consideration and Initiating Structure in Leadership Research.
Judge, T. A., Piccolo, R. F., & Ilies, R. (2004) · Journal of Applied Psychology · doi:10.1037/0021-9010.89.1.36 · Cited by 1,265
Effects of message repetition and position on cognitive response, recall, and persuasion.
Cacioppo, J. T., & Petty, R. E. (1979) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.37.1.97 · Cited by 813
MacKenzie, S. B., Podsakoff, P. M., & Fetter, R. (1991) · Organizational Behavior and Human Decision Processes · doi:10.1016/0749-5978(91)90037-t · Cited by 807
Sharma, R., Yetton, P., & Crawford, J. (2009) · MIS Quarterly · doi:10.2307/20650305 · Cited by 496
Cited by
Common method bias: It’ s bad, it’ s complex, it’ s widespread, and it’ s not easy to fix
Podsakoff, P. M., Podsakoff, N. P., Williams, L. J., Huang, C., & Yang, J. (2024) · Annual Review of Organizational Psychology and Organizational Behavior · doi:10.1146/annurev-orgpsych-110721-040030 · Cited by 1,899
Self-citationGabriel, A. S., Podsakoff, N. P., Beal, D. J., Scott, B. A., Sonnentag, S., Trougakos, J. P., & Butts, M. M. (2019) · Organizational Research Methods · doi:10.1177/1094428118802626 · Cited by 755
Self-citationRaineri, N., & Paillé, P. (2016) · Journal of Business Ethics · doi:10.1007/s10551-015-2548-x · Cited by 432
Podsakoff, N. P., Podsakoff, P. M., MacKenzie, S. B., Maynes, T. D., & Spoelma, T. M. (2014) · Journal of Organizational Behavior · doi:10.1002/job.1911 · Cited by 425
Self-citationCommon Method Variance in Advertising Research: When to Be Concerned and How to Control for It
Malhotra, N. K., Schaller, T. K., & Patil, A. (2017) · Journal of Advertising · doi:10.1080/00913367.2016.1252287 · Cited by 312
Johnco, C. J., Magson, N. R., Fardouly, J., Oar, E. L., Forbes, M. K., Richardson, C., & Rapee, R. M. (2021) · Depression and Anxiety · doi:10.1002/da.23197 · Cited by 86
Metadata
- Authors
- Podsakoff, N. P., Whiting, S. W., Welsh, D. T., & Mai, K. M.
- Venue
- Journal of Applied Psychology
- Published
- 2013
- Volume
- 98
- Issue
- 5
- Pages
- 863-874
- ISSN
- 0021-9010 (ISSN-L)
- 1939-1854
- OpenAlex
- W1977700973
- Cited by
- 130
- References
- 12
- FWCI
- 5.89
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.