Cited by 515Reliability and Agreement in MeasurementPsychometric Methodologies and TestingDelphi Technique in Research

The DOI matches its Crossref record; 32 of the 33 references it cites were found in a registry or an index.

A disagreement about within-group agreement: disentangling issues of consistency versus consensus

Kozlowski, Steve W., Hattrup, Keith·1992·Journal of Applied Psychology

Open this paper in CoChat — free account

Abstract

James, Demaree, and Wolf (1984) developed an indes, r WG , for assessing within-group agreement appropriate when only a single target is rated. The assessment of interperceiver agreement in such situations is of particular relevance to the composition model for climate. Schmidt and Hunter (1989) have criticized the conceptual foundation of r WG because it is not consistent with the classical model of reliability. They proposed an alternative approach, the use of the rating standard deviation (SD X ), the standard error of the rating mean (SE M ), and the associated confidence intervals for SE M to index interrater agreement. This comment argues that the critique of r WG dit not clearly distinguish the concepts of interrater consensus (i.e., agreement) and interrater consistency (i.e., reliability)

Integrity signals

Registry

Crossref · 10.1037/0021-9010.77.2.161

Cites

32 of 32 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

A Coefficient of Agreement for Nominal Scales
Cohen, J. (1960) · Educational and Psychological Measurement · doi:10.1177/001316446002000104 · Cited by 42,398
Intraclass correlations: Uses in assessing rater reliability.
Shrout, P. E., & Fleiss, J. L. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.2.420 · Cited by 24,532
The Social Psychology of Organizations.
Hackman, J. R., Katz, D., & Kahn, R. L. (1979) · Administrative Science Quarterly · doi:10.2307/2989929 · Cited by 12,243
Weighted kappa: nominal scale agreement provision for scaled disagreement or partial credit
Cohen, J. (1968) · Psychological Bulletin · doi:10.1037/h0026256 · Cited by 8,736
Estimating within-group interrater reliability with and without response bias
James, L. R., Demaree, R. G., & Wolf, G. (1984) · Journal of Applied Psychology · doi:10.1037/0021-9010.69.1.85 · Cited by 4,774
TRANSFER OF TRAINING: A REVIEW AND DIRECTIONS FOR FUTURE RESEARCH
Baldwin, T. T., & Ford, J. K. (1988) · Personnel Psychology · doi:10.1111/j.1744-6570.1988.tb00632.x · Cited by 3,662
International Review of Industrial and Organizational Psychology
Cooper, C. L., & Robertson, I. T. (2003) · Wiley eBooks · doi:10.1002/0470013311.oth1 · Cited by 2,787
Aggregation bias in estimates of perceptual agreement.
James, L. R. (1982) · Journal of Applied Psychology · doi:10.1037/0021-9010.67.2.219 · Cited by 2,139
Personality, affect, and behavior in groups
George, J. M. (1990) · Journal of Applied Psychology · doi:10.1037/0021-9010.75.2.107 · Cited by 1,372
Employee and customer perceptions of service in banks: Replication and extension.
Schneider, B., & Bowen, D. E. (1985) · Journal of Applied Psychology · doi:10.1037/0021-9010.70.3.423 · Cited by 1,344

Flagged first, then by citations

Show all 33 references

1 of the 33 references couldn’t be matched to a known paper.

Sorted by citations

Items 101–200 of 515

MEASUREMENT ERROR IN RESEARCH ON THE HUMAN RESOURCES AND FIRM PERFORMANCE RELATIONSHIP: FURTHER EVIDENCE AND ANALYSIS
Gerhart, B. A., Wright, P. M., & McMahan, G. C. (2000) · Personnel Psychology · doi:10.1111/j.1744-6570.2000.tb02420.x · Cited by 218
I will speak up if my voice is socially desirable: A moderated mediating process of promotive versus prohibitive voice.
Wei, X., Zhang, Z.-X., & Chen, X.-P. (2015) · Journal of Applied Psychology · doi:10.1037/a0039046 · Cited by 218
Social and cognitive sources of potential inaccuracy in job analysis.
Morgeson, F. P., & Campion, M. A. (1997) · Journal of Applied Psychology · doi:10.1037//0021-9010.82.5.627 · Cited by 218
Introducing Team Mindfulness and Considering its Safeguard Role Against Conflict Transformation and Social Undermining
Yu, L., & Zellmer-Bruhn, M. E. (2017) · Academy of Management Journal · doi:10.5465/amj.2016.0094 · Cited by 225
Reliability and agreement of student ratings of the classroom environment: A reanalysis of TIMSS data
Lüdtke, O., Trautwein, U., Kunter, M., & Baumert, J. (2007) · Learning Environments Research · doi:10.1007/s10984-006-9014-8 · Cited by 211
MULTIPLE RATERS IN SURVEY‐BASED OPERATIONS MANAGEMENT RESEARCH: A REVIEW AND TUTORIAL
Boyer, K. K., & Verma, R. (2000) · Production and Operations Management · doi:10.1111/j.1937-5956.2000.tb00329.x · Cited by 208
Investigating Linear and Interactive Effects of Shared Mental Models on Safety and Efficiency in a Field Setting.
Smith‐Jentsch, K. A., Mathieu, J. E., & Kraiger, K. (2005) · Journal of Applied Psychology · doi:10.1037/0021-9010.90.3.523 · Cited by 205
New to the game and questioning the rules: The experiences and beliefs of founders who start imitative versus innovative firms
Cliff, J. E., Jennings, P. D., & Greenwood, R. (2005) · Journal of Business Venturing · doi:10.1016/j.jbusvent.2005.02.010 · Cited by 205
To reflect or not to reflect: Prior team performance as a boundary condition of the effects of reflexivity on learning and final team performance
Schippers, M. C., Homan, A. C., & van Knippenberg, D. (2012) · Journal of Organizational Behavior · doi:10.1002/job.1784 · Cited by 199
RETRACTED: Impact of spiritual leadership on unit performance
Fry, L. W., Hannah, S. T., Noel, M., & Walumbwa, F. O. (2011) · The Leadership Quarterly · doi:10.1016/j.leaqua.2011.02.002 · Cited by 219
Retracted
An examination of the use of high‐investment human resource systems for core and support employees
Lepak, D. P., Taylor, M. S., Tekleab, A. G., Marrone, J. A., & Cohen, D. J. (2007) · Human Resource Management · doi:10.1002/hrm.20158 · Cited by 192
Overcoming roadblocks to effectiveness: Incorporating management of performance barriers into models of work group effectiveness.
Tesluk, P. E., & Mathieu, J. E. (1999) · Journal of Applied Psychology · doi:10.1037/0021-9010.84.2.200 · Cited by 189
Statistical properties of the rWG(J) index of agreement.
Cohen, A., Doveh, E., & Eick, U. (2001) · Psychological Methods · doi:10.1037/1082-989x.6.3.297 · Cited by 167
Creative Climate: A Leadership Lever for Innovation
Isaksen, S. G., & AKKERMANS, H. J. (2011) · The Journal of Creative Behavior · doi:10.1002/j.2162-6057.2011.tb01425.x · Cited by 165
The Role of Character Strengths for Task Performance, Job Dedication, Interpersonal Facilitation, and Organizational Support
Harzer, C., & Ruch, W. (2014) · Human Performance · doi:10.1080/08959285.2014.913592 · Cited by 158
Energy at work: A measurement validation and linkage to unit effectiveness
Cole, M. S., Bruch, H., & Vogel, B. (2011) · Journal of Organizational Behavior · doi:10.1002/job.759 · Cited by 169
Unit-Level OCB and Unit Effectiveness: Examining the Incremental Effect of Helping Behavior
Ehrhart, M. G., Bliese, P. D., & Thomas, J. L. (2006) · Human Performance · doi:10.1207/s15327043hup1902_4 · Cited by 156
The missing link? Investigating organizational identity strength and transformational leadership climate as mechanisms that connect CEO charisma with firm performance
Boehm, S. A., Dwertmann, D. J. G., Bruch, H., & Shamir, B. (2014) · The Leadership Quarterly · doi:10.1016/j.leaqua.2014.07.012 · Cited by 189
Relationship between collective efficacy and team cohesion: Conceptual and measurement issues.
Paskevich, D. M., Brawley, L. R., Dorsch, K. D., & Widmeyer, W. N. (1999) · Group Dynamics Theory Research and Practice · doi:10.1037/1089-2699.3.3.210 · Cited by 153
STRATEGIC COMPENSATION FOR INTEGRATED MANUFACTURING: THE MODERATING EFFECTS OF JOBS AND ORGANIZATIONAL INERTIA.
Snell, S. A., & J. W. DEAN, Jr. (1994) · Academy of Management Journal · doi:10.2307/256668 · Cited by 154
A MULTILEVEL FRAMEWORK FOR UNDERSTANDING POLICE CULTURE: THE ROLE OF THE WORKGROUP
Ingram, J. R., Paoline, E. A., & Terrill, W. (2013) · Criminology · doi:10.1111/1745-9125.12009 · Cited by 149
The benefits of tourism for rural community development
Liu, Y.-L., Chiang, J.-T., & Ko, P.-F. (2023) · Humanities and Social Sciences Communications · doi:10.1057/s41599-023-01610-4 · Cited by 149
Top Management Team Diversity
Homberg, F., & Bui, H. (2013) · Group & Organization Management · doi:10.1177/1059601113493925 · Cited by 168
ASSESSING THE INCREMENTAL VALIDITY OF TEAM CONSENSUS RATINGS OVER AGGREGATION OF INDIVIDUAL‐LEVEL DATA IN PREDICTING TEAM EFFECTIVENESS
Kirkman, B. L., Tesluk, P. E., & Rosen, B. (2001) · Personnel Psychology · doi:10.1111/j.1744-6570.2001.tb00226.x · Cited by 147
GENERALIZATION OF EMPLOYEE INVOLVEMENT TRAINING TO THE JOB SETTING: INDIVIDUAL AND SITUATIONAL EFFECTS
Tesluk, P. E., Farr, J. L., Mathieu, J. E., & Vance, R. J. (1995) · Personnel Psychology · doi:10.1111/j.1744-6570.1995.tb01773.x · Cited by 147
How do leader–member exchange quality and differentiation affect performance in teams? An integrated multilevel dual process model.
Li, A. N., & Liao, H. (2014) · Journal of Applied Psychology · doi:10.1037/a0037233 · Cited by 146
The Effect of Strategic Alliance Knowledge Complementarity on New Product Innovativeness in China
Fang, E. (2010) · Organization Science · doi:10.1287/orsc.1090.0512 · Cited by 145
The Impact of Service Characteristics on the Switching Costs–Customer Loyalty Link
Blut, M., Beatty, S. E., Evanschitzky, H., & Brock, C. (2014) · Journal of Retailing · doi:10.1016/j.jretai.2014.04.003 · Cited by 160
Differentiating transformational and non-transformational leaders on the basis of neurological imaging
Balthazard, P. A., Waldman, D. A., Thatcher, R. W., & Hannah, S. T. (2011) · The Leadership Quarterly · doi:10.1016/j.leaqua.2011.08.002 · Cited by 140
Transformational leadership climate: Performance linkages, mechanisms, and boundary conditions at the organizational level
Menges, J. I., Walter, F., Vogel, B., & Bruch, H. (2011) · The Leadership Quarterly · doi:10.1016/j.leaqua.2011.07.010 · Cited by 155
Does the Measure of Dispersion Matter in Multilevel Research? A Comparison of the Relative Performance of Dispersion Indexes
Roberson, Q. M., Sturman, M. C., & Simons, T. L. (2007) · Organizational Research Methods · doi:10.1177/1094428106294746 · Cited by 140
Enhancing Team Mental Model Measurement with Performance Appraisal Practices
Webber, S. S., Chen, G., Payne, S. C., Marsh, S. M., & Zaccaro, S. J. (2000) · Organizational Research Methods · doi:10.1177/109442810034001 · Cited by 146
When is female leadership an advantage? Coordination requirements, team cohesion, and team interaction norms
Post, C. A. (2015) · Journal of Organizational Behavior · doi:10.1002/job.2031 · Cited by 163
Multilevel Investigation of Adaptive Performance
Han, T. Y., & Williams, K. J. (2008) · Group & Organization Management · doi:10.1177/1059601108326799 · Cited by 136
Structural impacts on the occurrence and effectiveness of transformational leadership: An empirical study at the organizational level of analysis
Walter, F. H., & Bruch, H. (2010) · The Leadership Quarterly · doi:10.1016/j.leaqua.2010.07.006 · Cited by 135
Work Design
Morgeson, F. P., & Campion, M. A. (2003) · Wiley eBooks · doi:10.1002/0471264385.wei1217 · Cited by 137
Multiple Informant Methodology: A Critical Review and Recommendations
Wagner, S. M., Rau, C., & Lindemann, E. (2010) · Sociological Methods & Research · doi:10.1177/0049124110366231 · Cited by 143
Assessing interrater agreement on the job relevance of a test: A comparison of CVI, T, rWG(J)}, and r*WG(J)} indexes.
Lindell, M. K., & Brandt, C. J. (1999) · Journal of Applied Psychology · doi:10.1037/0021-9010.84.4.640 · Cited by 131
Labor-Management Cooperation: Antecedents and Impact on Organizational Performance
Deery, S. J., & Iverson, R. D. (2005) · Industrial and Labor Relations Review · doi:10.1177/001979390505800404 · Cited by 129
Intrinsic or extrinsic motivations for hospitality employees’ creativity: The moderating role of organization-level regulatory focus
Chang, J., & Teng, C.-C. C. (2016) · International Journal of Hospitality Management · doi:10.1016/j.ijhm.2016.10.003 · Cited by 127
Empathic and nonempathic interaction in chronic pain couples.
Cano, A., Barterian, J. A., & Heller, J. B. (2008) · The Clinical Journal of Pain · doi:10.1097/ajp.0b013e31816753d8 · Cited by 125
Mistakes and How to Avoid Mistakes in Using Intercoder Reliability Indices
Feng, G. C. (2014) · Methodology · doi:10.1027/1614-2241/a000086 · Cited by 132
The Impact of Group Interaction Styles on Problem-Solving Effectiveness
Cooke, R. A., & Szumal, J. L. (1994) · The Journal of Applied Behavioral Science · doi:10.1177/0021886394304005 · Cited by 122
Investigating associations between school climate and bullying in secondary schools: Multilevel contextual effects modeling
Konishi, C., Miyazaki, Y., Hymel, S. C., & Waterhouse, T. (2017) · School Psychology International · doi:10.1177/0143034316688730 · Cited by 120
Recent Issues Regarding rWG, rWG, rWG(J), and rWG(J)
LeBreton, J. M., James, L. R., & Lindell, M. K. (2004) · Organizational Research Methods · doi:10.1177/1094428104272181 · Cited by 118
A mediation model linking dispatcher leadership and work ownership with safety climate as predictors of truck driver safety performance
Zohar, D. M., Huang, Y.-H., Lee, J., & Robertson, M. M. (2013) · Accident Analysis & Prevention · doi:10.1016/j.aap.2013.09.005 · Cited by 123
When do emotionally exhausted employees speak up? Exploring the potential curvilinear relationship between emotional exhaustion and voice
Qin, X., DiRenzo, M. S., Xu, M., & Duan, Y. (2014) · Journal of Organizational Behavior · doi:10.1002/job.1948 · Cited by 120
An Integrated Framework for Measuring Product Development Performance in High Technology Industries
Mallick, D. N., & Schroeder, R. G. (2005) · Production and Operations Management · doi:10.1111/j.1937-5956.2005.tb00015.x · Cited by 117
Outperforming whom? A multilevel study of performance-prove goal orientation, performance, and the moderating role of shared team identification.
Dietz, B., van Knippenberg, D., Hirst, G., & Restubog, S. L. D. (2015) · Journal of Applied Psychology · doi:10.1037/a0038888 · Cited by 117
From social context and resilience to performance through job satisfaction: A multilevel study over time
Meneghel, I., Боргогни, Л., Miraglia, M., Salanova, M., & Martínez, I. M. (2016) · Human Relations · doi:10.1177/0018726716631808 · Cited by 136
The Effects of Absence Culture on Individual Absence
Martocchio, J. J. (1994) · Human Relations · doi:10.1177/001872679404700301 · Cited by 116
How does responsible leadership affect employees’ voluntary workplace green behaviors? A multilevel dual process model of voluntary workplace green behaviors
Zhang, J., Ul‐Durar, S., Akhtar, M. N., Zhang, Y., & Lu, L. (2021) · Journal of Environmental Management · doi:10.1016/j.jenvman.2021.113205 · Cited by 119
School Climate as Group Evaluation and Group Consensus: Student and Parent Perceptions of the Elementary School Environment
Griffith, J. E. (2000) · The Elementary School Journal · doi:10.1086/499658 · Cited by 122
The School Leadership/School Climate Relation: Identification of School Configurations Associated with Change in Principals
Griffith, J. E. (1999) · Educational Administration Quarterly · doi:10.1177/00131619921968545 · Cited by 114
How Collective Engagement Creates Competitive Advantage for Organizations: A Business‐Level Model of Shared Vision, Competitive Intensity, and Service Performance
Eldor, L. (2019) · Journal of Management Studies · doi:10.1111/joms.12438 · Cited by 132
VALIDATING THE COMPETING VALUES MODEL AS A REPRESENTATION OF ORGANIZATIONAL CULTURE THROUGH INTER‐INSTITUTIONAL COMPARISONS
Kwan, P., & Walker, A. D. (2004) · Organizational Analysis · doi:10.1108/eb028984 · Cited by 113
Levels of analysis issues in group psychology: Using efficacy as an example of a multilevel model.
Moritz, S. E., & Watson, C. B. (1998) · Group Dynamics Theory Research and Practice · doi:10.1037/1089-2699.2.4.285 · Cited by 110
Expertise, extraversion and group interaction styles as performance indicators in virtual teams
Balthazard, P. A., Potter, R. E., & Warren, J. (2004) · ACM SIGMIS Database the DATABASE for Advances in Information Systems · doi:10.1145/968464.968469 · Cited by 111
Leadership and Social Identification in Military Units: Direct and Indirect Relationships1
Shamir, B., Zakay, E., Brainin, E., & Popper, M. (2000) · Journal of Applied Social Psychology · doi:10.1111/j.1559-1816.2000.tb02499.x · Cited by 111
Virtual team interaction styles: assessment and effects
Potter, R. E., & Balthazard, P. A. (2002) · International Journal of Human-Computer Studies · doi:10.1006/ijhc.2002.1001 · Cited by 109
IRT Models for Expert-Coded Panel Data
Marquardt, K. L., & Pemstein, D. (2017) · Political Analysis · doi:10.1017/pan.2018.28 · Cited by 108
The impact of leadership and quality climate on hospital performance
Shipton, H. J., Armstrong, C., West, M. A., & Dawson, J. (2008) · International Journal for Quality in Health Care · doi:10.1093/intqhc/mzn037 · Cited by 107
Service employees give as they get: Internal service as a moderator of the service climate–service outcomes link.
Ehrhart, K. H., Witt, L. A., Schneider, B., & Perry, S. J. (2010) · Journal of Applied Psychology · doi:10.1037/a0022071 · Cited by 106
No Safety in Numbers
Houghton, S. M., Simon, M., Aquino, K. F., & Goldberg, C. (2000) · Group & Organization Management · doi:10.1177/1059601100254002 · Cited by 106
The role of within‐group agreement in understanding transformational leadership
Feinberg, B., Ostroff, C. L., & Burke, W. W. (2005) · Journal of Occupational and Organizational Psychology · doi:10.1348/096317905x26156 · Cited by 105
A Two-Phase Longitudinal Model of a Turnover Event: Disruption, Recovery Rates, and Moderators of Collective Performance
Hale, D., Ployhart, R. E., & Shepherd, W. J. (2015) · Academy of Management Journal · doi:10.5465/amj.2013.0546 · Cited by 104
An Overview of Interrater Agreement on Likert Scales for Researchers and Practitioners
O’Neill, T. (2017) · Frontiers in Psychology · doi:10.3389/fpsyg.2017.00777 · Cited by 127
Dominating Interpersonal Behavior and Perceived Victimization in Groups: Evidence for a Curvilinear Relationship
Aquino, K. F., & Byron, K. L. (2002) · Journal of Management · doi:10.1177/014920630202800105 · Cited by 104
Rationale and construct validity evidence for a measure of perceived organizational family support (POFS): Because purported practices may not reflect reality
Jahn, E. W., Thompson, C. A., & Kopelman, R. E. (2003) · Community Work & Family · doi:10.1080/13668800302545 · Cited by 104
Team collaboration in virtual worlds
Kahai, S. S., Carroll, E., & Jestice, R. J. (2007) · ACM SIGMIS Database the DATABASE for Advances in Information Systems · doi:10.1145/1314234.1314246 · Cited by 105
Data Aggregation in Multilevel Analysis: A Review of Conceptual and Statistical Issues
Dixon, M. A., & Cunningham, G. B. (2006) · Measurement in Physical Education and Exercise Science · doi:10.1207/s15327841mpee1002_2 · Cited by 96
Novelty and new firm performance: The case of employment systems in knowledge-intensive service organizations
Jennings, P. D., Jennings, P. D., & Greenwood, R. (2008) · Journal of Business Venturing · doi:10.1016/j.jbusvent.2008.03.003 · Cited by 96
Selecting sustainable teams for PPP projects
Kumaraswamy, M. M., & Anvuur, A. M. (2007) · Building and Environment · doi:10.1016/j.buildenv.2007.02.001 · Cited by 87
Interdependence and Organizational Citizenship Behavior: Exploring the Mediating Effect of Group Cohesion in Multilevel Analysis
Chen, C.-H. V., Tang, Y.-Y., & Wang, S.-J. (2009) · The Journal of Psychology · doi:10.1080/00223980903218273 · Cited by 87
Measuring Interrater Agreement for Ratings of a Single Target
Lindell, M. K., & Brandt, C. J. (1997) · Applied Psychological Measurement · doi:10.1177/01466216970213006 · Cited by 86
Psychomotor abilities via touch‐panel testing: Measurement innovations, construct, and criterion validity
Ackerman, P. L., & Cianciolo, A. T. (1999) · Human Performance · doi:10.1207/s15327043hup1203 · Cited by 85
Is it “trustworthy”? A multiple-levels-of-analysis reexamination of an ohio state leadership study, with implications for future research
Schriesheim, C. A., Cogliser, C. C., & Neider, L. L. (1995) · The Leadership Quarterly · doi:10.1016/1048-9843(95)90031-4 · Cited by 85
Relating Cohesion to Stress, Strain, Disintegration, and Performance: An Organizing Framework
Griffith, J. E., & Vaitkus, M. A. (1999) · Military Psychology · doi:10.1207/s15327876mp1101_3 · Cited by 84
Ethics, Values, and Organizational Justice: Individuals, Organizations, and Beyond
Schminke, M. J., Arnaud, A., & Taylor, R. M. (2014) · Journal of Business Ethics · doi:10.1007/s10551-014-2251-3 · Cited by 94
Contemporary perspectives on the study of psychological climate: A commentary
Burke, M. J., Borucki, C. C., & Kaufman, J. D. (2002) · European Journal of Work and Organizational Psychology · doi:10.1080/13594320244000210 · Cited by 83
Testing extrinsic and intrinsic motivation as explanatory variables for the safety climate–safety performance relationship among long-haul truck drivers
Zohar, D. M., Huang, Y.-H., Lee, J., & Robertson, M. M. (2015) · Transportation Research Part F Traffic Psychology and Behaviour · doi:10.1016/j.trf.2015.01.014 · Cited by 89
Modeling Heterogeneity in the Satisfaction, Loyalty Intention, and Shareholder Value Linkage: A Cross-Industry Analysis at the Customer and Firm Levels
Larivière, B., Keiningham, T. L., Aksoy, L., Yalçın, A., Morgeson, F. V., & Mithas, S. (2015) · Journal of Marketing Research · doi:10.1509/jmr.12.0143 · Cited by 82
Exploitation and Exploration Climates’ Influence on Performance and Creativity: Diminishing Returns as Function of Self-Efficacy
Hirst, G., van Knippenberg, D., Zhou, Q., Zhu, C. J., & Tsai, P. C.-F. (2015) · Journal of Management · doi:10.1177/0149206315596814 · Cited by 86
Disaster Experience and Hospital Information Systems: An Examination of Perceived Information Assurance, Risk, Resilience, and HIS Usefulness1
Park, I., Sharman, R., & Rao, H. (2015) · MIS Quarterly · doi:10.25300/misq/2015/39.2.03 · Cited by 85
Centralization of member voice in teams: Its effects on expertise utilization and team performance.
Sherf, E. N., Sinha, R., Tangirala, S., & Awasty, N. (2018) · Journal of Applied Psychology · doi:10.1037/apl0000305 · Cited by 80
From Self‐Efficacy to Collective Efficacy in Sport: Transitional Methodological Issues
Myers, N. D., & Feltz, D. L. (2007) · Wiley eBooks · doi:10.1002/9781118270011.ch36 · Cited by 79
Human capital: the link between leadership and organizational learning
Pasamar, S., Díaz-Fernández, M., & de la Rosa-Navarro, M. D. (2019) · European Journal of Management and Business Economics · doi:10.1108/ejmbe-08-2017-0003 · Cited by 69
The effects of collaboration on performance: a multilevel validation in project teams
Franç, N. A., Chiocchio, O., Grenier, S., Taoheed, T. O., Neill, N. A., Savaria, K., & Willms, J. D. (2012) · International Journal of Project Organisation and Management · doi:10.1504/ijpom.2012.045362 · Cited by 96
The Role of Individual Memory and Attention Processes During Electronic Brainstorming1
Potter, R. E., & Balthazard, P. A. (2004) · MIS Quarterly · doi:10.2307/25148657 · Cited by 77
Work analysis: From technique to theory.
Morgeson, F. P., & Dierdorff, E. C. (2010) · American Psychological Association eBooks · doi:10.1037/12170-001 · Cited by 75
Knowledge transfer in age‐diverse coworker dyads in China and Germany: How and when do age‐inclusive human resource practices have an effect?
Burmeister, A., HEIJDEN, B. I. J. M. V. D., Yang, J., & Deller, J. (2018) · Human Resource Management Journal · doi:10.1111/1748-8583.12207 · Cited by 83
External Web content and its influence on organizational performance
Meroño‐Cerdán, A. L., & Soto‐Acosta, P. (2007) · European Journal of Information Systems · doi:10.1057/palgrave.ejis.3000656 · Cited by 71
Some Common Indices of Group Diversity: Upper Boundaries
Solanas, A., Selvam, R., Navarro, J., & Leiva, D. (2012) · Psychological Reports · doi:10.2466/01.09.21.pr0.111.6.777-796 · Cited by 67
Testing Agreement for Multi-Item Scales With the Indices r WG(J) and AD M(J)
Cohen, A., Doveh, E., & Nahum-Shani, I. B. (2007) · Organizational Research Methods · doi:10.1177/1094428107300365 · Cited by 69
Analyse von Lernumwelten
Lüdtke, O., Trautwein, U., Kunter, M., & Baumert, J. (2006) · Zeitschrift für Pädagogische Psychologie · doi:10.1024/1010-0652.20.12.85 · Cited by 70
A multilevel examination of affective job insecurity climate on safety outcomes.
Jiang, L., & Probst, T. M. (2015) · Journal of Occupational Health Psychology · doi:10.1037/ocp0000014 · Cited by 75
Unobtrusive measurement of psychological constructs in organizational research
Hill, A., White, M. A., & Wallace, J. C. (2013) · Organizational Psychology Review · doi:10.1177/2041386613505613 · Cited by 69
The Need for Dual Openness to Change: A Longitudinal Study Evaluating the Impact of Employees’ Openness to Organizational Change Content and Process on Intervention Outcomes
Augustsson, H., Richter, A., Hasson, H., & Schwarz, U. V. T. (2017) · The Journal of Applied Behavioral Science · doi:10.1177/0021886317691930 · Cited by 70
Charismatic leadership and follower effects
Conger, J. A., KANUNGO, R. N., & Menon, S. T. (2000) · Journal of Organizational Behavior · doi:10.1002/1099-1379(200011)21:7 · Cited by 718
Team change climate: A group-level analysis of the relationships among change information and change participation, role stressors, and well-being
Rafferty, A. E., & Jimmieson, N. L. (2009) · European Journal of Work and Organizational Psychology · doi:10.1080/13594320903007869 · Cited by 66

Authors: Kozlowski, S. W., & Hattrup, K.
Affiliations: Michigan State University (US)
ORCID: Kozlowski, Steve W.
Venue: Journal of Applied Psychology
Publisher: American Psychological Association (APA)
Published: 1992-04
Volume: 77
Issue: 2
Pages: 161-167
ISSN: 0021-9010 (ISSN-L)
1939-1854
DOI: 10.1037/0021-9010.77.2.161
OpenAlex: W2069294908
Cited by: 514
References: 33
Citations by year: 2026: 8 · 2025: 6 · 2024: 9 · 2023: 12 · 2022: 5 · 2021: 23 · 2020: 11 · 2019: 15 · 2018: 16 · 2017: 15 · 2016: 19 · 2015: 27 · 2014: 24 · 2013: 17 · 2012: 26
FWCI: 12.78
Type: journal article
Language: en
Open access: closed
Locations: Journal of Applied Psychology (publishedVersion)
Topics: Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement
Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
Social Sciences › Sociology and Political Science › Delphi Technique in Research
Keywords: interobserver agreement, confidence intervals, classical test theory, reliability statistics

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

A disagreement about within-group agreement: disentangling issues of consistency versus consensus

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks