Substantive and operational issues of response bias across levels of analysis: An example of climate-satisfaction relationships.
Abstract
Two studies tested whether method variance is present at multiple levels of analysis and whether methodological procedures can minimize its impact. In Study 1, 8,052 employees from 71 hotels completed measures of climate, work environment characteristics, and satisfaction. A comparison of correlations at the individual level, cross-level, cross-level split, aggregate level, and aggregate-split level of analysis revealed that response bias was present across multiple levels. Results suggest that samples should be split in half when cross-level and aggregate correlations are computed to ameliorate response bias problems that arise from individual-level method variance. In Study 2, results indicated that the temporal spacing of measures of climate and satisfaction influenced response bias. Implications and recommendations for future research are discussed.
Cites (54)
- Handbook of Industrial and Organizational Psychology.Hammer, Tove Helland, Anderson, John, Gurdon, Michael, Jick, Todd, Petit, Andre, Stern, Robert, Schwartz, Howard, Vardi, Yoav, Dunnette, Marvin D. · 1977 · Administrative Science Quarterly
- Review of Psychometric theory (2nd ed.).SITGREAVES, ROSEDITH · 1979 · Contemporary Psychology: A Journal of Reviews
- Research in Organizational BehaviorHammer, Eliot R., Cummings, L. L., Staw, Barry M. · 1984 · The Academy of Management Review
- Multilevel Theory, Research, and Methods in Organizations: Foundations, Extensions, and New DirectionsKatherine J. Klein, Steve W. J. Kozlowski · 2000
- Estimating within-group interrater reliability with and without response biasJames, Lawrence R., Demaree, Robert G., Wolf, Gerrit · 1984 · Journal of Applied Psychology
- Ecological Correlations and the Behavior of IndividualsRobinson, W. S. · 1950 · American Sociological Review
- Overall job satisfaction: How good are single-item measures?John P. Wanous, Arnon E. Reichers, Michael J. Hudy · 1997 · Journal of Applied Psychology
- Perceived organizational support and employee diligence, commitment, and innovation.Robert Eisenberger, Peter M. Fasolo, Valerie LaMastro · 1990 · Journal of Applied Psychology
- International Review of Industrial and Organizational Psychology: Contents of Previous Volumes2003 · International Review of Industrial and Organizational Psychology 2004
- Aggregation bias in estimates of perceptual agreement.James, Lawrence R. · 1982 · Journal of Applied Psychology
Cited by (41)
- Sources of method bias in social science research and recommendations on how to control itPodsakoff, Philip M., MacKenzie, Scott B., Podsakoff, Nathan P. · 2012 · Annual Review of Psychology
- Common methods variance detection in business researchFuller, Christie M., Simmering, Marcia J., Atinc, Guclu, Atinc, Yasemin, Babin, Barry J. · 2016 · Journal of Business Research
- Differences Between Tight and Loose Cultures: A 33-Nation StudyGelfand, Michele J., Raver, Jana L., Nishii, Lisa, Leslie, Lisa M., Lun, Janetta, Lim, Beng Chong, Duan, Lili, Almaliach, Assaf, Ang, Soon, Arnadottir, Jakobina et al. · 2011 · Science
- Leadership Behavior and Employee Voice: Is the Door Really Open?Detert, James R., Burris, Ethan R. · 2007 · Academy of Management Journal
- Common method bias: It’ s bad, it’ s complex, it’ s widespread, and it’ s not easy to fixPodsakoff, Philip M., Podsakoff, Nathan P., Williams, Larry J., Huang, Chengquan, Yang, Junhui · 2024 · Annual Review of Organizational Psychology and Organizational Behavior
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Substantive and operational issues of response bias across levels of analysis: An example of climate-satisfaction relationships. https://app.cochat.ai/papers/10.1037//0021-9010.87.2.355
- The paper
- Ostroff, C. L., Kinicki, A. J., & Clark, M. A. (2002). Substantive and operational issues of response bias across levels of analysis: An example of climate-satisfaction relationships. Journal of Applied Psychology. https://doi.org/10.1037//0021-9010.87.2.355
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.