Cited by 515Reliability and Agreement in MeasurementPsychometric Methodologies and TestingDelphi Technique in Research

The DOI matches its Crossref record; 32 of the 33 references it cites were found in a registry or an index.

A disagreement about within-group agreement: disentangling issues of consistency versus consensus

Kozlowski, Steve W., Hattrup, Keith·1992·Journal of Applied Psychology

Open this paper in CoChat — free account

Abstract

James, Demaree, and Wolf (1984) developed an indes, r WG , for assessing within-group agreement appropriate when only a single target is rated. The assessment of interperceiver agreement in such situations is of particular relevance to the composition model for climate. Schmidt and Hunter (1989) have criticized the conceptual foundation of r WG because it is not consistent with the classical model of reliability. They proposed an alternative approach, the use of the rating standard deviation (SD X ), the standard error of the rating mean (SE M ), and the associated confidence intervals for SE M to index interrater agreement. This comment argues that the critique of r WG dit not clearly distinguish the concepts of interrater consensus (i.e., agreement) and interrater consistency (i.e., reliability)

Integrity signals

Registry

Crossref · 10.1037/0021-9010.77.2.161

Cites

32 of 32 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

A Coefficient of Agreement for Nominal Scales
Cohen, J. (1960) · Educational and Psychological Measurement · doi:10.1177/001316446002000104 · Cited by 42,398
Intraclass correlations: Uses in assessing rater reliability.
Shrout, P. E., & Fleiss, J. L. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.2.420 · Cited by 24,532
The Social Psychology of Organizations.
Hackman, J. R., Katz, D., & Kahn, R. L. (1979) · Administrative Science Quarterly · doi:10.2307/2989929 · Cited by 12,243
Weighted kappa: nominal scale agreement provision for scaled disagreement or partial credit
Cohen, J. (1968) · Psychological Bulletin · doi:10.1037/h0026256 · Cited by 8,736
Estimating within-group interrater reliability with and without response bias
James, L. R., Demaree, R. G., & Wolf, G. (1984) · Journal of Applied Psychology · doi:10.1037/0021-9010.69.1.85 · Cited by 4,774
TRANSFER OF TRAINING: A REVIEW AND DIRECTIONS FOR FUTURE RESEARCH
Baldwin, T. T., & Ford, J. K. (1988) · Personnel Psychology · doi:10.1111/j.1744-6570.1988.tb00632.x · Cited by 3,662
International Review of Industrial and Organizational Psychology
Cooper, C. L., & Robertson, I. T. (2003) · Wiley eBooks · doi:10.1002/0470013311.oth1 · Cited by 2,787
Aggregation bias in estimates of perceptual agreement.
James, L. R. (1982) · Journal of Applied Psychology · doi:10.1037/0021-9010.67.2.219 · Cited by 2,139
Personality, affect, and behavior in groups
George, J. M. (1990) · Journal of Applied Psychology · doi:10.1037/0021-9010.75.2.107 · Cited by 1,372
Employee and customer perceptions of service in banks: Replication and extension.
Schneider, B., & Bowen, D. E. (1985) · Journal of Applied Psychology · doi:10.1037/0021-9010.70.3.423 · Cited by 1,344

Flagged first, then by citations

Show all 33 references

1 of the 33 references couldn’t be matched to a known paper.

Sorted by citations

PERSON‐ORGANIZATION FIT: AN INTEGRATIVE REVIEW OF ITS CONCEPTUALIZATIONS, MEASUREMENT, AND IMPLICATIONS
Kristof‐Brown, A. L. (1996) · Personnel Psychology · doi:10.1111/j.1744-6570.1996.tb01790.x · Cited by 5,243
Answers to 20 questions about interrater reliability and interrater agreement
LeBreton, J. M., & Senter, J. L. (2008) · Organizational Research Methods · doi:10.1177/1094428106296642 · Cited by 3,608
The Work Design Questionnaire (WDQ): Developing and validating a comprehensive measure for assessing job design and the nature of work.
Morgeson, F. P., & Humphrey, S. E. (2006) · Journal of Applied Psychology · doi:10.1037/0021-9010.91.6.1321 · Cited by 2,994
Functional relations among constructs in the same content domain at different levels of analysis: A typology of composition models.
Chan, D. K. (1998) · Journal of Applied Psychology · doi:10.1037/0021-9010.83.2.234 · Cited by 2,981
Untitled work
(2000) · Cited by 2,531
Self-citation
HUMAN RESOURCE MANAGEMENT, MANUFACTURING STRATEGY, AND FIRM PERFORMANCE.
Youndt, M. A., Snell, S. A., J. W. DEAN, Jr., & Lepak, D. P. (1996) · Academy of Management Journal · doi:10.2307/256714 · Cited by 2,274
Knowledge Management Enablers, Processes, and Organizational Performance: An Integrative View and Empirical Examination
Lee, H., & Choi, B. (2003) · Journal of Management Information Systems · doi:10.1080/07421222.2003.11045756 · Cited by 2,269
rwg: an assessment of within-group interrater agreement
James, L. R., Demaree, R. G., & Wolf, G. (1993) · Journal of Applied Psychology · doi:10.1037/0021-9010.78.2.306 · Cited by 2,221
RELATIONS BETWEEN WORK GROUP CHARACTERISTICS AND EFFECTIVENESS: IMPLICATIONS FOR DESIGNING EFFECTIVE WORK GROUPS
Campion, M. A., Medsker, G. J., & Higgs, A. C. (1993) · Personnel Psychology · doi:10.1111/j.1744-6570.1993.tb01571.x · Cited by 2,189
HOW MUCH DO HIGH‐PERFORMANCE WORK PRACTICES MATTER? A META‐ANALYSIS OF THEIR EFFECTS ON ORGANIZATIONAL PERFORMANCE
Combs, J. G., Liu, Y., Hall, A. T., & Ketchen, D. J. (2006) · Personnel Psychology · doi:10.1111/j.1744-6570.2006.00045.x · Cited by 2,208
Consensus development methods, and their use in clinical guideline development.
Murphy, M., Black, Lamping, McKee, M., Sanderson, Askham, & Marteau (1998) · Health Technology Assessment · doi:10.3310/hta2030 · Cited by 2,076
The Sources of Four Commonly Reported Cutoff Criteria: What Did They Really Say?
Lance, C. E., Butts, M. M., & Michels, L. C. (2006) · Organizational Research Methods · doi:10.1177/1094428105284919 · Cited by 2,053
Social Structure of “Coopetition” Within a Multiunit Organization: Coordination, Competition, and Intraorganizational Knowledge Sharing
Tsai, W. (2002) · Organization Science · doi:10.1287/orsc.13.2.179.536 · Cited by 1,960
Defining and developing measures of lean production
Shah, R., & Ward, P. T. (2007) · Journal of Operations Management · doi:10.1016/j.jom.2007.01.019 · Cited by 2,399
Levels issues in theory development, data collection and analysis
Klein, K. J., Dansereau, F., & Hall, R. J. (1994) · The Academy of Management Review · doi:10.2307/258703 · Cited by 1,607
An Overview of the Logic and Rationale of Hierarchical Linear Models
Hofmann, D. (1997) · Journal of Management · doi:10.1177/014920639702300602 · Cited by 1,681
From Micro to Meso: Critical Steps in Conceptualizing and Conducting Multilevel Research
Klein, K. J., & Kozlowski, S. W. J. (2000) · Organizational Research Methods · doi:10.1177/109442810033001 · Cited by 1,473
Self-citation
EMPLOYEE ATTRIBUTIONS OF THE “WHY” OF HR PRACTICES: THEIR EFFECTS ON EMPLOYEE ATTITUDES AND BEHAVIORS, AND CUSTOMER SATISFACTION
NISHII, L. H., LEPAK, D. P., & SCHNEIDER, B. (2008) · Personnel Psychology · doi:10.1111/j.1744-6570.2008.00121.x · Cited by 1,410
Examining the Human Resource Architecture: The Relationships Among Human Capital, Employment, and Human Resource Configurations
Lepak, D. P., & Snell, S. A. (2002) · Journal of Management · doi:10.1177/014920630202800403 · Cited by 1,370
Vertical versus shared leadership as predictors of the effectiveness of change management teams: An examination of aversive, directive, transactional, transformational, and empowering leader behaviors.
Pearce, C. L., & Jr. Sims Henry, P. (2002) · Group Dynamics Theory Research and Practice · doi:10.1037/1089-2699.6.2.172 · Cited by 1,403
The Structure and Function of Collective Constructs: Implications for Multilevel Research and Theory Development
Morgeson, F. P., & Hofmann, D. A. (1999) · Academy of Management Review · doi:10.5465/amr.1999.1893935 · Cited by 1,224
Measurement reliability and agreement in psychiatry
Shrout, P. E. (1998) · Statistical Methods in Medical Research · doi:10.1177/096228029800700306 · Cited by 1,211
Innovation in top management teams.
West, M. A., & Anderson, N. R. (1996) · Journal of Applied Psychology · doi:10.1037/0021-9010.81.6.680 · Cited by 1,148
On the validity of subjective measures of company performance
WALL, T. D., MICHIE, J., PATTERSON, M., WOOD, S. J., SHEEHAN, M., CLEGG, C. W., & WEST, M. (2004) · Personnel Psychology · doi:10.1111/j.1744-6570.2004.tb02485.x · Cited by 1,121
Individual consideration viewed at multiple levels of analysis: A multi-level framework for examining the diffusion of transformational leadership
Avolio, B. J., & Bass, B. M. (1995) · The Leadership Quarterly · doi:10.1016/1048-9843(95)90035-7 · Cited by 1,050
Relationships between psychological climate perceptions and work outcomes: a meta‐analytic review
Parker, C. P., Baltes, B. B., Young, S. A., Huff, J. W., Altmann, R. A., Lacost, H. A., & Roberts, J. E. (2003) · Journal of Organizational Behavior · doi:10.1002/job.198 · Cited by 1,079
Do they see eye to eye? Management and employee perspectives of high-performance work systems and influence processes on service quality.
Liao, H., Toya, K., Lepak, D., & Hong, Y. (2009) · Journal of Applied Psychology · doi:10.1037/a0013504 · Cited by 1,030
Top management team diversity, group process, and strategic consensus
Knight, D., Pearce, C. L., Smith, K. G., Olian, J. D., Sims, H. P., Smith, K. A., & Flood, P. C. (1999) · Strategic Management Journal · doi:10.1002/(sici)1097-0266(199905)20:5 · Cited by 918
MONKEY SEE, MONKEY DO: THE INFLUENCE OF WORK GROUPS ON THE ANTISOCIAL BEHAVIOR OF EMPLOYEES.
Robinson, S. L., & O’Leary-Kelly, A. M. (1998) · Academy of Management Journal · doi:10.2307/256963 · Cited by 872
Climate strength: a new direction for climate research.
Schneider, B., Salvaggio, A. N., & Ferrer, M. S. (2002) · Journal of Applied Psychology · doi:10.1037/0021-9010.87.2.220 · Cited by 775
How Organizational Complaint Handling Drives Customer Loyalty: An Analysis of the Mechanistic and the Organic Approach
Homburg, C., & Fürst, A. (2005) · Journal of Marketing · doi:10.1509/jmkg.69.3.95.66367 · Cited by 804
RELATIONS BETWEEN WORK TEAM CHARACTERISTICS AND EFFECTIVENESS: A REPLICATION AND EXTENSION
Campion, M. A., Papper, E. M., & Medsker, G. J. (1996) · Personnel Psychology · doi:10.1111/j.1744-6570.1996.tb01806.x · Cited by 754
Expatriate managers and the psychological contract.
Guzzo, R. A., Noonan, K. A., & Elron, E. (1994) · Journal of Applied Psychology · doi:10.1037/0021-9010.79.4.617 · Cited by 723
MEASUREMENT ERROR IN RESEARCH ON HUMAN RESOURCES and FIRM PERFORMANCE: HOW MUCH ERROR IS THERE AND HOW DOES IT INFLUENCE EFFECT SIZE ESTIMATES?
Gerhart, B. A., Wright, P. M., MAHAN, G. C. M., & Snell, S. A. (2000) · Personnel Psychology · doi:10.1111/j.1744-6570.2000.tb02418.x · Cited by 679
CORRELATES OF CHARISMATIC LEADER BEHAVIOR IN MILITARY UNITS: SUBORDINATES' ATTITUDES, UNIT CHARACTERISTICS, AND SUPERIORS' APPRAISALS OF LEADER PERFORMANCE.
Shamir, B., Zakay, E., Breinin, E., & Popper, M. (1998) · Academy of Management Journal · doi:10.2307/257080 · Cited by 674
Handbook of Industrial, Work & Organizational Psychology
Anderson, N. R., Öneş, D. S., Sinangil, H. K., & Viswesvaran, C. (2001) · Sage eBooks · Cited by 632
A Multiple-Goal, Multilevel Model of Feedback Effects on the Regulation of Individual and Team Performance.
DeShon, R. P., Kozlowski, S. W. J., Schmidt, A. M., Milner, K. R., & Wiechmann, D. (2004) · Journal of Applied Psychology · doi:10.1037/0021-9010.89.6.1035 · Cited by 639
Self-citation
Transforming service employees and climate: A multilevel, multisource examination of transformational leadership in building long-term service relationships.
Liao, H., & Chuang, A. (2007) · Journal of Applied Psychology · doi:10.1037/0021-9010.92.4.1006 · Cited by 585
Embedding Ethical Leadership within and across Organization Levels
Schaubroeck, J. M., Hannah, S. T., Avolio, B. J., Kozlowski, S. W. J., Lord, R. G., Treviño, L. K., Dimotakis, N., & Peng, A. C. (2012) · Academy of Management Journal · doi:10.5465/amj.2011.0064 · Cited by 577
Self-citation
An empirical investigation of KM styles and their effect on corporate performance
Choi, B., & Lee, H. (2003) · Information & Management · doi:10.1016/s0378-7206(02)00060-5 · Cited by 642
The Impact of Justice Climate and Justice Orientation on Work Outcomes: A Cross-Level Multifoci Framework.
Liao, H., & Rupp, D. E. (2005) · Journal of Applied Psychology · doi:10.1037/0021-9010.90.2.242 · Cited by 563
Climate quality and climate consensus as mediators of the relationship between organizational antecedents and outcomes.
Lindell, M. K., & Brandt, C. J. (2000) · Journal of Applied Psychology · doi:10.1037/0021-9010.85.3.331 · Cited by 570
A Meta-Analysis of Cohesion and Performance
Gully, S. M., Devine, D. J., & Whitney, D. J. (1995) · Small Group Research · doi:10.1177/1046496495264003 · Cited by 527
Organization structure as a moderator of the relationship between procedural justice, interactional justice, perceived organizational support, and supervisory trust.
Ambrose, M. L., & Schminke, M. J. (2003) · Journal of Applied Psychology · doi:10.1037/0021-9010.88.2.295 · Cited by 550
Team effectiveness: Beyond skills and cognitive ability.
Neuman, G. A., & Wright, J. A. (1999) · Journal of Applied Psychology · doi:10.1037/0021-9010.84.3.376 · Cited by 506
Knowledge management strategy and its link to knowledge creation process
Choi, B., & Lee, H. (2002) · Expert Systems with Applications · doi:10.1016/s0957-4174(02)00038-6 · Cited by 466
On Average Deviation Indices for Estimating Interrater Agreement
Burke, M. J., Finkelstein, L. M., & Dusig, M. S. (1999) · Organizational Research Methods · doi:10.1177/109442819921004 · Cited by 462
The effect of leader moral development on ethical climate and employee attitudes
Schminke, M. J., Ambrose, M. L., & Neubaum, D. O. (2005) · Organizational Behavior and Human Decision Processes · doi:10.1016/j.obhdp.2005.03.006 · Cited by 455
RETRACTED: Psychological processes linking authentic leadership to follower behaviors
Walumbwa, F. O., Wang, P., Wang, H., Schaubroeck, J. M., & Avolio, B. J. (2010) · The Leadership Quarterly · doi:10.1016/j.leaqua.2010.07.015 · Cited by 458
Retracted
Estimating Interrater Agreement with the Average Deviation Index: A User’s Guide
Burke, M. J., & Dunlap, W. P. (2002) · Organizational Research Methods · doi:10.1177/1094428102005002002 · Cited by 448
Transformational leadership and innovative work behavior
Afsar, B., & Umrani, W. A. (2019) · European Journal of Innovation Management · doi:10.1108/ejim-12-2018-0257 · Cited by 511
Collectivistic orientation in teams: an individual and group-level analysis
Eby, L. T. D. T., & Dobbins, G. H. (1997) · Journal of Organizational Behavior · doi:10.1002/(sici)1099-1379(199705)18:3 · Cited by 441
Untitled work
(2006) · eCommons (Cornell University) · Cited by 437
Organizational climate and culture: Reflections on the history of the constructs in the Journal of Applied Psychology.
Schneider, B., González‐Romá, V., Ostroff, C. L., & West, M. A. (2017) · Journal of Applied Psychology · doi:10.1037/apl0000090 · Cited by 541
The External Leadership of Self-Managing Teams: Intervening in the Context of Novel and Disruptive Events.
Morgeson, F. P. (2005) · Journal of Applied Psychology · doi:10.1037/0021-9010.90.3.497 · Cited by 447
Advancing Multilevel Research Design
Kozlowski, S. W. J., Chao, G. T., Grand, J. A., Braun, M. T., & Kuljanin, G. (2013) · Organizational Research Methods · doi:10.1177/1094428113493119 · Cited by 456
Self-citation
Interrater Reliability and Agreement
Tinsley, H. E. A., & Weiss, D. J. (2000) · Elsevier eBooks · doi:10.1016/b978-012691360-6/50005-7 · Cited by 411
SATISFACTION, CITIZENSHIP BEHAVIORS, AND PERFORMANCE IN WORK UNITS: A META-ANALYSIS OF COLLECTIVE CONSTRUCT RELATIONS
Whitman, D. S., ROOY, D. L. V., & Viswesvaran, C. (2010) · Personnel Psychology · doi:10.1111/j.1744-6570.2009.01162.x · Cited by 382
INDIVIDUAL AND SITUATIONAL INFLUENCES ON THE DEVELOPMENT OF SELF‐EFFICACY: IMPLICATIONS FOR TRAINING EFFECTIVENESS
Mathieu, J. E., Martineau, J., & Tannenbaum, S. I. (1993) · Personnel Psychology · doi:10.1111/j.1744-6570.1993.tb00870.x · Cited by 369
THE INFLUENCE OF PROPORTIONAL AND PERCEPTUAL CONFLICT COMPOSITION ON TEAM PERFORMANCE
Jehn, K. A., & Chatman, J. A. (2000) · International Journal of Conflict Management · doi:10.1108/eb022835 · Cited by 363
ATTITUDES AND EFFECTIVENESS: EXAMINING RELATIONS AT AN ORGANIZATIONAL LEVEL
Ryan, A. M., Schmit, M. J., & JOHNSON, R. (1996) · Personnel Psychology · doi:10.1111/j.1744-6570.1996.tb02452.x · Cited by 363
Within-group agreement: On the use (and misuse) of r and r in leadership research and some best practice guidelines
Biemann, T., Cole, M. S., & Voelpel, S. C. (2011) · The Leadership Quarterly · doi:10.1016/j.leaqua.2011.11.006 · Cited by 364
A trickle-down model of organizational justice: Relating employees' and customers' perceptions of and reactions to fairness.
Masterson, S. S. (2001) · Journal of Applied Psychology · doi:10.1037/0021-9010.86.4.594 · Cited by 361
Cohesiveness and organizational citizenship behavior: A multilevel analysis using work groups and individuals
Kidwell, R. E., Mossholder, K. W., & Bennett, N. (1997) · Journal of Management · doi:10.1177/014920639702300605 · Cited by 357
A social identity perspective on leadership and employee creativity
Hirst, G., van Dick, R., & van Knippenberg, D. (2009) · Journal of Organizational Behavior · doi:10.1002/job.600 · Cited by 356
The effect of organizational structure on perceptions of procedural fairness.
Schminke, M. J., Ambrose, M. L., & Cropanzano, R. (2000) · Journal of Applied Psychology · doi:10.1037/0021-9010.85.2.294 · Cited by 349
Affective mechanisms linking dysfunctional behavior to performance in work teams: A moderated mediation study.
Cole, M. S., Walter, F. H., & Bruch, H. (2008) · Journal of Applied Psychology · doi:10.1037/0021-9010.93.5.945 · Cited by 353
Measuring Normative Beliefs and Shared Behavioral Expectations in Organizations: The Reliability and Validity of the Organizational Culture Inventory
Cooke, R. A., & Szumal, J. L. (1993) · Psychological Reports · doi:10.2466/pr0.1993.72.3c.1299 · Cited by 336
Individual Moral Development and Ethical Climate: The Influence of Person–Organization Fit on Job Attitudes
Ambrose, M. L., Arnaud, A., & Schminke, M. J. (2007) · Journal of Business Ethics · doi:10.1007/s10551-007-9352-1 · Cited by 335
MAXIMIZING THE BENEFITS OF TASK CONFLICT: THE ROLE OF CONFLICT MANAGEMENT
DeChurch, L. A., & Marks, M. A. (2001) · International Journal of Conflict Management · doi:10.1108/eb022847 · Cited by 331
Group cohesion: From "field of forces" to multidimensional construct.
Dion, K. L. (2000) · Group Dynamics Theory Research and Practice · doi:10.1037/1089-2699.4.1.7 · Cited by 319
A FRAMEWORK FOR CONDUCTING MULTI-LEVEL CONSTRUCT VALIDATION
Chen, G., Mathieu, J. E., & Bliese, P. D. (2004) · Research in multi-level issues · doi:10.1016/s1475-9144(04)03013-9 · Cited by 317
The Dynamics of Emergence: Cognition and Cohesion in Work Teams
Kozlowski, S. W. J., & Chao, G. T. (2012) · Managerial and Decision Economics · doi:10.1002/mde.2552 · Cited by 338
Self-citation
Responsible leadership and employee's proenvironmental behavior: The role of organizational commitment, green shared vision, and internal environmental locus of control
Afsar, B., Maqsoom, A., Shahjehan, A., Afridi, S. A., Nawaz, A., & Fazliani, H. (2019) · Corporate Social Responsibility and Environmental Management · doi:10.1002/csr.1806 · Cited by 311
Perceived team and player efficacy in hockey.
Feltz, D. L., & Lirgg, C. (1998) · Journal of Applied Psychology · doi:10.1037/0021-9010.83.4.557 · Cited by 302
Interrater Agreement Reconsidered: An Alternative to the rwg Indices
Brown, R. D., & Hauenstein, N. M. A. (2005) · Organizational Research Methods · doi:10.1177/1094428105275376 · Cited by 299
Work Outcomes and Job Design for Contract Versus Permanent Information Systems Professionals On Software Development Teams1, 2
Ang, S., & Slaughter, S. A. (2001) · MIS Quarterly · doi:10.2307/3250920 · Cited by 296
The Etiology of the Multilevel Paradigm in Management Research
Mathieu, J. E., & Chen, G. (2011) · Journal of Management · doi:10.1177/0149206310364663 · Cited by 314
Predicting employee aggression against coworkers, subordinates and supervisors: the roles of person behaviors and perceived workplace factors
Greenberg, L., & Barling, J. I. (1999) · Journal of Organizational Behavior · doi:10.1002/(sici)1099-1379(199911)20:6 · Cited by 308
AN EXAMINATION OF THE RELATIONSHIP BETWEEN WORK GROUP CHARACTERISTICS AND PERFORMANCE: ONCE MORE INTO THE BREECH
HYATT, D. E., & Ruddy, T. M. (1997) · Personnel Psychology · doi:10.1111/j.1744-6570.1997.tb00703.x · Cited by 291
Organizational efforts to affirm sexual diversity: A cross-level examination.
Button, S. B. (2001) · Journal of Applied Psychology · doi:10.1037/0021-9010.86.1.17 · Cited by 281
Advancing research on team process dynamics
Kozlowski, S. W. J. (2015) · Organizational Psychology Review · doi:10.1177/2041386614533586 · Cited by 305
Self-citation
Team structure, team climate and the quality of care in primary care: an observational study
Bower, P. (2003) · Quality and Safety in Health Care · doi:10.1136/qhc.12.4.273 · Cited by 277
What Drives Key Informant Accuracy?
Homburg, C., Klarmann, M., Reimann, M., & Schilke, O. (2012) · Journal of Marketing Research · doi:10.1509/jmr.09.0174 · Cited by 276
The impact of financial and nonfinancial incentives on business-unit outcomes over time.
Peterson, S. J., & Luthans, F. (2006) · Journal of Applied Psychology · doi:10.1037/0021-9010.91.1.156 · Cited by 269
Variability Within Organizations: Implications for Strategic Human Resource ManagementPreprint
Nishii, L. H., & Wright, P. M. (2007) · eCommons (Cornell University) · Cited by 267
Shared Authentic Leadership and New Venture Performance
Hmieleski, K. M., Cole, M. S., & Baron, R. A. (2011) · Journal of Management · doi:10.1177/0149206311415419 · Cited by 283
Shared leadership: Toward a multi-level theory of leadership
Pearce, C. L., & Sims, H. P. (2004) · Advances in interdisciplinary studies of work teams · doi:10.1016/s1572-0977(00)07008-4 · Cited by 274
MEASUREMENT ERROR IN RESEARCH ON HUMAN RESOURCES AND FIRM PERFORMANCE: ADDITIONAL DATA AND SUGGESTIONS FOR FUTURE RESEARCH
Wright, P. M., Gardner, T. M., Moynihan, L. M., Park, H. J., Gerhart, B. A., & Delery, J. E. (2001) · Personnel Psychology · doi:10.1111/j.1744-6570.2001.tb00235.x · Cited by 260
A FIVE‐YEAR STUDY OF UPWARD FEEDBACK: WHAT MANAGERS DO WITH THEIR RESULTS MATTERS
Walker, A. G., & Smither, J. W. (1999) · Personnel Psychology · doi:10.1111/j.1744-6570.1999.tb00166.x · Cited by 257
The Restriction of Variance Hypothesis and Interrater Reliability and Agreement: Are Ratings from Multiple Sources Really Dissimilar?
LeBreton, J. M., Burgess, J. R. D., Kaiser, R. B., Atchley, E. K., & James, L. R. (2003) · Organizational Research Methods · doi:10.1177/1094428102239427 · Cited by 272
A Revised Index of Interrater Agreement for Multi-Item Ratings of a Single Target
Lindell, M. K., Brandt, C. J., & Whitney, D. J. (1999) · Applied Psychological Measurement · doi:10.1177/01466219922031257 · Cited by 255
Data analytic methods for the analysis of multilevel questions
Castro, S. L. (2002) · The Leadership Quarterly · doi:10.1016/s1048-9843(01)00105-9 · Cited by 250
Integrating Knowledge Activities for Team Innovation: Effects of Transformational Leadership
Jiang, Y., & Chen, C. C. (2016) · Journal of Management · doi:10.1177/0149206316628641 · Cited by 286
A multidimensional approach to the group cohesion-group performance relationship
Chang, A., & Bordia, P. (2001) · Small Group Research · doi:10.1177/104649640103200401 · Cited by 243
The differential effect of team members' trust on team performance: The mediation role of team cohesion
Mach, M., Dolan, S. L., & Tzafrir, S. S. (2009) · Journal of Occupational and Organizational Psychology · doi:10.1348/096317909x473903 · Cited by 265
Looking inside and out: The impact of employee and community demographic composition on organizational diversity climate.
Pugh, S. D., Dietz, J., Brief, A. P., & Wiley, J. W. (2008) · Journal of Applied Psychology · doi:10.1037/a0012696 · Cited by 243
Organization structure and fairness perceptions: The moderating effects of organizational level
Schminke, M. J., Cropanzano, R., & Rupp, D. E. (2002) · Organizational Behavior and Human Decision Processes · doi:10.1016/s0749-5978(02)00034-1 · Cited by 242
A HIERARCHICAL CONSTRUCT OF SELF‐MANAGEMENT LEADERSHIP AND ITS RELATIONSHIP TO QUALITY OF WORK LIFE AND PERCEIVED WORK GROUP EFFECTIVENESS
Cohen, S. G., Chang, L., & Ledford, G. E. (1997) · Personnel Psychology · doi:10.1111/j.1744-6570.1997.tb00909.x · Cited by 230
SERVICE CLIMATE EFFECTS ON CUSTOMER ATTITUDES: AN EXAMINATION OF BOUNDARY CONDITIONS.
Dietz, J., Pugh, S. D., & Wiley, J. W. (2004) · Academy of Management Journal · doi:10.2307/20159561 · Cited by 224

Authors: Kozlowski, S. W., & Hattrup, K.
Affiliations: Michigan State University (US)
ORCID: Kozlowski, Steve W.
Venue: Journal of Applied Psychology
Publisher: American Psychological Association (APA)
Published: 1992-04
Volume: 77
Issue: 2
Pages: 161-167
ISSN: 0021-9010 (ISSN-L)
1939-1854
DOI: 10.1037/0021-9010.77.2.161
OpenAlex: W2069294908
Cited by: 514
References: 33
Citations by year: 2026: 8 · 2025: 6 · 2024: 9 · 2023: 12 · 2022: 5 · 2021: 23 · 2020: 11 · 2019: 15 · 2018: 16 · 2017: 15 · 2016: 19 · 2015: 27 · 2014: 24 · 2013: 17 · 2012: 26
FWCI: 12.78
Type: journal article
Language: en
Open access: closed
Locations: Journal of Applied Psychology (publishedVersion)
Topics: Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement
Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
Social Sciences › Sociology and Political Science › Delphi Technique in Research
Keywords: interobserver agreement, confidence intervals, classical test theory, reliability statistics

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

A disagreement about within-group agreement: disentangling issues of consistency versus consensus

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks