Cited by 2,221Reliability and Agreement in MeasurementPsychometric Methodologies and TestingTeam Dynamics and Performance

The DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index.

rwg: an assessment of within-group interrater agreement

James, Lawrence R., Demaree, Robert G., Wolf, Gerrit·1993·Journal of Applied Psychology

DOI 10.1037/0021-9010.78.2.306

Open this paper in CoChat — free account

Abstract

Schmidt and Hunter (1989) critiqued the within-group interrater reliability statistic (r wg ) described by James, Demaree, and Wolf (1984). Kozlowski and Hattrup (1992) responded to the Schmidt and Hunter critique and argued that r wg is a suitable index of interrater agreement. This article focuses on the interpretation of r wg as a measure of agreement among judges' ratings of a single target. A new derivation of r wg is given that underscores this interpretation

Integrity signals

Registry

Crossref · 10.1037/0021-9010.78.2.306

Cites

21 of 21 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Statistical principles in experimental design.
Winer, B. J. (1962) · McGraw-Hill Book Company · doi:10.1037/11774-000 · Cited by 27,137
Intraclass correlations: Uses in assessing rater reliability.
Shrout, P. E., & Fleiss, J. L. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.2.420 · Cited by 24,532
Applications of Item Response Theory To Practical Testing Problems
Lord, F. M. (2012) · Routledge eBooks · doi:10.4324/9780203056615 · Cited by 5,009
Estimating within-group interrater reliability with and without response bias
James, L. R., Demaree, R. G., & Wolf, G. (1984) · Journal of Applied Psychology · doi:10.1037/0021-9010.69.1.85 · Cited by 4,774
Self-citation
The Dependability of Behavioral Measurements: Theory of Generalizability for Scores and Profiles
Merrifield, P. R., Cronbach, L. J., Gleser, G. C., Nanda, H., & Rajaratnam, N. (1974) · American Educational Research Journal · doi:10.2307/1162145 · Cited by 1,819
Employee and customer perceptions of service in banks: Replication and extension.
Schneider, B., & Bowen, D. E. (1985) · Journal of Applied Psychology · doi:10.1037/0021-9010.70.3.423 · Cited by 1,344
Superiors' evaluations and subordinates' perceptions of transformational and transactional leadership.
Hater, J. J., & Bass, B. M. (1988) · Journal of Applied Psychology · doi:10.1037/0021-9010.73.4.695 · Cited by 1,136
Response Sets and Test Validity
Cronbach, L. J. (1946) · Educational and Psychological Measurement · doi:10.1177/001316444600600405 · Cited by 898
Interrater reliability and agreement of subjective judgments.
Tinsley, H. E. A., & Weiss, D. J. (1975) · Journal of Counseling Psychology · doi:10.1037/h0076640 · Cited by 883
On various intraclass correlation reliability coefficients.
Bartko, J. J. (1976) · Psychological Bulletin · doi:10.1037/0033-2909.83.5.762 · Cited by 830

Show all 21 references

Sorted by citations

Items 901–1,000 of 1,001

Adaptive, generative, and transformative learning in project teams
Sessa, V. I., London, M., Pingor, C., Gullu, B., & Patel, J. (2011) · Team Performance Management · doi:10.1108/13527591111143691 · Cited by 44
Making “sense” of ethical decision making
Zeni, T. A., Buckley, M. R., Mumford, M. D., & Griffith, J. A. (2016) · The Leadership Quarterly · doi:10.1016/j.leaqua.2016.09.002 · Cited by 53
The influence of the structure of social networks on academic spin-offs’ entrepreneurial orientation
Diánez‐González, J. P., & Camelo‐Ordaz, C. (2017) · Industrial Marketing Management · doi:10.1016/j.indmarman.2017.12.009 · Cited by 46
Variations in coaching style and their impact on subordinates' work outcomes
Hui, R. T., & Sue‐Chan, C. (2018) · Journal of Organizational Behavior · doi:10.1002/job.2263 · Cited by 43
Reaping the Rewards of Functional Diversity in Healthcare Teams: Why Team Processes Improve Performance
Johnson, A. M., Nguyen, H., Groth, M., & White, L. (2018) · Group & Organization Management · doi:10.1177/1059601118769192 · Cited by 47
“We first followed the procedure developed by James, Demaree, and Wolf (1993) to determine the agreement between team members.”
Method
Team entrepreneurial competence: multilevel effects on individual cognitive strategies
Santos, S. C., Morris, M. H., Caetano, A., Costa, S. F., & Neumeyer, X. (2019) · International Journal of Entrepreneurial Behaviour & Research · doi:10.1108/ijebr-03-2018-0126 · Cited by 42
“These included the index of within-group interrater agreement – Rwg( j) ( James et al., 1993), designed for multiple-item scales, using the 0.”
Method
Climate as a group level phenomenon: Theoretical assumptions and methodological considerations
Luria, G. (2019) · Journal of Organizational Behavior · doi:10.1002/job.2417 · Cited by 42
Managing the trade-off between autonomy and task interdependence in creative teams: The role of organizational-level cultural control
Grabner, I., Klein, A., & Speckbacher, G. (2022) · Accounting Organizations and Society · doi:10.1016/j.aos.2022.101347 · Cited by 41
Differentiation strategy, high-performance human resource practices, and firm performance: moderation by employee commitment
Sun, L., & Pan, W. (2011) · The International Journal of Human Resource Management · doi:10.1080/09585192.2011.560860 · Cited by 41
The Workgroup Emotional Climate Scale
Liu, X., Härtel, C. E. J., & Sun, J. (2014) · Group & Organization Management · doi:10.1177/1059601114554453 · Cited by 47
The Team Descriptive Index (TDI): A Multidimensional Scaling Approach for Team Description
Lee, S. M., Koopman, J., Hollenbeck, J. R., Wang, L. C., & Lanaj, K. (2015) · Academy of Management Discoveries · doi:10.5465/amd.2013.0001 · Cited by 46
Firm-level diversity management competencies: development and initial validation of a measure
Carstens, J., & de Kock, F. D. (2017) · The International Journal of Human Resource Management · doi:10.1080/09585192.2015.1128460 · Cited by 36
Early efforts to develop absorptive capacity and their performance implications: differences among corporate and independent ventures
Larrañeta, B., González, J. L. G., & Aguilar, R. M. (2016) · The Journal of Technology Transfer · doi:10.1007/s10961-016-9488-1 · Cited by 41
Aligning organizational priorities with ARC to improve youth mental health service outcomes.
Glisson, C. A., Williams, N. J., Hemmelgarn, A. L., Proctor, E. K., & Green, P. (2016) · Journal of Consulting and Clinical Psychology · doi:10.1037/ccp0000107 · Cited by 40
The individual and organizational level effects of TQM practices on job satisfaction
Prajogo, D., & Cooper, B. (2017) · International Journal of Manpower · doi:10.1108/ijm-12-2014-0240 · Cited by 42
Top Management Team Tenure Diversity and Performance: The Moderating Role of Behavioral Integration
Yi, Y., Ndofor, H. A., He, X., & Wei, Z. (2017) · IEEE Transactions on Engineering Management · doi:10.1109/tem.2017.2737663 · Cited by 41
Examining the Indirect Effects of Perceived Organizational Support for Teamwork Training on Acute Health Care Team Productivity and Innovation: The Role of Shared Objectives
Lyubovnikova, J., West, T. H., Dawson, J., & West, M. A. (2018) · Group & Organization Management · doi:10.1177/1059601118769742 · Cited by 43
Contextualizing Senior Executive Advice Seeking: The Role of Decision Process Comprehensiveness and Empowerment Climate
Alexiev, A. S., Volberda, H., Jansen, J. J. P., & van den Bosch, F. (2019) · Organization Studies · doi:10.1177/0170840619830128 · Cited by 48
“We calculated an interrater agreement score (r wg ) for each study variable (James, Demaree, & Wolf, 1993).”
Applying the Delphi method to determine best practices for outsourcing logistics in disaster relief
Gossler, T., Sigala, I. F., Wakolbinger, T., & Buber, R. (2019) · Journal of Humanitarian Logistics and Supply Chain Management · doi:10.1108/jhlscm-06-2018-0044 · Cited by 41
Team Decision Making: The Dynamic Effects of Team Decision Style Composition and Performance via Decision Strategy
Zhu, X., Wolfson, M. A., Dalal, D. K., & Mathieu, J. E. (2020) · Journal of Management · doi:10.1177/0149206320916232 · Cited by 47
Shaping positive and negative ties to improve team effectiveness: The roles of leader humility and team helping norms
Chiu, C., Balkundi, P., Owens, B. P., & Tesluk, P. E. (2020) · Human Relations · doi:10.1177/0018726720968135 · Cited by 46
“We calculated the value of the within-group agreement index (rwg, James et al., 1993) and intraclass correlation coefficients (ICC1 and ICC2; Bliese, 2000) to justify construct aggregation.”
Method
The effects of shared leadership on team performance
Han, J., Yoon, J., Choi, W., & Hong, G. (2021) · Leadership & Organization Development Journal · doi:10.1108/lodj-01-2020-0023 · Cited by 41
A diary study on shared leadership, team work engagement, and goal attainment
Klasmeier, K. N., & Rowold, J. (2021) · Journal of Occupational and Organizational Psychology · doi:10.1111/joop.12371 · Cited by 51
Bringing Employees Closer: The Effect of Proximity on Communication When Teams Function under Time Pressure
Chong, D., van Eerde, W., Rutte, C. G., & Chai, K. (2012) · Journal of Product Innovation Management · doi:10.1111/j.1540-5885.2011.00890.x · Cited by 41
Das Teamklima-Inventar (TKI) für Innovation in Gruppen
Brodbeck, F. C., & Maier, G. W. (2001) · Zeitschrift für Arbeits- und Organisationspsychologie A&O · doi:10.1026//0932-4089.45.2.59 · Cited by 41
Exploring the Processes Underlying Within-Group Homogeneity
Mason, C. (2006) · Small Group Research · doi:10.1177/1046496406288972 · Cited by 41
Embracing Paradox: TMT paradoxical processes as a steppingstone between TMT reflexivity and organizational ambidexterity
van Neerijnen, P., Tempelaar, M. P., & van de Vrande, V. (2021) · Organization Studies · doi:10.1177/01708406211058640 · Cited by 40
“Values above 0.7 indicate a ‘good’ fit (James et al., 1993).”
BackgroundInfluential
Can virtuality be protective of team trust? Conflict and effectiveness in hybrid teams
Alves, M. P., Dimas, I. D., Lourenço, P. R., Rebelo, T., Peñarroja, V., & Gamero, N. (2022) · Behaviour and Information Technology · doi:10.1080/0144929x.2022.2046163 · Cited by 50
“To ensure that the aggregation was appropriate in our sample, we assessed the degree of intrateam consensus by calculating the interrater reliability index rWG (James, Demaree, and Wolf 1993) and ICC(1) and ICC(2) (Bliese 2000).”
Method
A multilevel model of the effects of equal opportunity climate on job satisfaction in the military.
Walsh, B. M., Matthews, R. A., Tuller, M. D., Parks, K. M., & McDonald, D. P. (2010) · Journal of Occupational Health Psychology · doi:10.1037/a0018756 · Cited by 47
Work-Health and Safety-Risk Perceptions of Construction-Industry Stakeholders Using Photograph-Based Q Methodology
Zhang, R. P., Lingard, H. C., Blismas, N. G., Wakefield, R. R., & Kleiner, B. M. (2014) · Journal of Construction Engineering and Management · doi:10.1061/(asce)co.1943-7862.0000954 · Cited by 40
Are we paid to be creative? The effect of compensation gap on creativity in an expatriate context
Hon, A. H. Y., & Lu, L. (2014) · Journal of World Business · doi:10.1016/j.jwb.2014.03.002 · Cited by 43
Opposites attract or attack? The moderating role of diversity climate in the team diversity–interpersonal aggression relationship.
Drach‐Zahavy, A., & Trogan, R. (2013) · Journal of Occupational Health Psychology · doi:10.1037/a0033989 · Cited by 41
How Can Leaders Foster Team Learning? Effects of Leader-Assigned Mastery and Performance Goals and Psychological Safety
Ashauer, S., & Macan, T. M. (2013) · The Journal of Psychology · doi:10.1080/00223980.2012.719940 · Cited by 46
Product radicalness and firm performance in B2B marketing: A moderated mediation model
Lavado, A. C., Gopalakrishnan, S., & Zhang, H. (2019) · Industrial Marketing Management · doi:10.1016/j.indmarman.2019.08.013 · Cited by 40
From Nuisance to Novel Research Questions: Using Multilevel Models to Predict Heterogeneous Variances
Lester, H. F., Cullen, K. L., & Walters, R. W. (2019) · Organizational Research Methods · doi:10.1177/1094428119887434 · Cited by 39
Adverse Events and Burnout
Vogus, T. J., Ramanujam, R., Novikov, Z., Venkataramani, V., & Tangirala, S. (2020) · Medical Care · doi:10.1097/mlr.0000000000001341 · Cited by 48
On the destructiveness of laissez-faire versus abusive supervision: a comparative, multilevel investigation of destructive forms of leadership
Klasmeier, K. N., Schleu, J. E., Millhoff, C., Poethke, U., & Bormann, K. C. (2021) · European Journal of Work and Organizational Psychology · doi:10.1080/1359432x.2021.1968375 · Cited by 50
“Given the hierarchical nature of our data, we calculated intraclass correlation coefficients (Bliese, 2000) and average rwg(j) to quantify the degree of within-team agreement (James et al., 1993).”
Method
Promoting group potency in project teams: The importance of group identification
Lee, C., Farh, J., & Chen, Z. (2011) · Journal of Organizational Behavior · doi:10.1002/job.741 · Cited by 40
A PRACTITIONER‐BASED ANALYSIS OF INTERVIEWER JOB EXPERTISE AND SCALE FORMAT AS CONTEXTUAL FACTORS IN SITUATIONAL INTERVIEWS
Maurer, S. D. (2002) · Personnel Psychology · doi:10.1111/j.1744-6570.2002.tb00112.x · Cited by 40
Leadership, Managerial Performance and 360‐Degree Feedback
Carless, S. A., Mann, L., & Wearing, A. J. (1998) · Applied Psychology · doi:10.1111/j.1464-0597.1998.tb00039.x · Cited by 40
Moderating effects on the compensation gap between locals and expatriates in China: A multi-level analysis
Leung, K. Y., Wang, Z., & Hon, A. H. Y. (2011) · Journal of International Management · doi:10.1016/j.intman.2010.12.002 · Cited by 39
Procedural fairness and cooperation in public‐private partnerships in China
Zhang, Z., & Jia, M. (2010) · Journal of Managerial Psychology · doi:10.1108/02683941011048409 · Cited by 39
“79 for the second) and then averaged these two scores from two boundary spanners into a single index, which was further weighted by James et al. 's (1993)”
Background
The Meaning and Measure of Organizational Learning Mechanisms in Secondary Schools
Schechter, C., & Atarchi, L. (2013) · Educational Administration Quarterly · doi:10.1177/0013161x13508772 · Cited by 39
Contextual Bias in Verbal Credibility Assessment: Criteria‐Based Content Analysis, Reality Monitoring and Scientific Content Analysis
Bogaard, G., Meijer, E. H., Vrij, A., Broers, N. J., & Merckelbach, H. L. G. J. (2013) · Applied Cognitive Psychology · doi:10.1002/acp.2959 · Cited by 41
Differences between follower and dyadic measures of LMX as mediators of emotional intelligence and employee performance, well-being, and turnover intention
Clarke, N. M. P., & Mahadi, N. B. (2016) · European Journal of Work and Organizational Psychology · doi:10.1080/1359432x.2016.1263185 · Cited by 48
Advancing theory development: exploring the leadership–climate relationship as a mechanism of the implementation of cultural competence
Guerrero, E. G., Fenwick, K. M., & Kong, Y. (2017) · Implementation Science · doi:10.1186/s13012-017-0666-9 · Cited by 44
“Second, we conducted a within-group consistency analysis using rWG to test whether members of each of the 112 treatment programs agreed with one another in their responses to the leadership scales [73].”
Method
An Empirical Examination of a Model of Social Climate in Elementary Schools
Griffith, J. E. (1995) · Basic and Applied Social Psychology · doi:10.1080/01973533.1995.9646134 · Cited by 39
Examination of the Discriminant, Convergent, and Criterion‐Related Validity of Self‐Ratings on the Emotional Competence Inventory
Byrne, J. C., Dominick, P. G., Smither, J. W., & Reilly, R. R. (2007) · International Journal of Selection and Assessment · doi:10.1111/j.1468-2389.2007.00393.x · Cited by 39
Group initiative and self-organizational activities in industrial work groups
Brav, A., Andersson, K., & Lantz, A. (2009) · European Journal of Work and Organizational Psychology · doi:10.1080/13594320801960482 · Cited by 39
“We used James, Demaree, and Wolf's ( 1984 , 1993 ) within-group agreement index r wg(j) to measure reliability.”
(How) do we learn from errors? A prospective study of the link between the ward's learning practices and medication administration errors.
Drach-Zahavy, A., Somech, A., Admi, H., Peterfreund, I., Peker, H., & Priente, O. (2014) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2013.06.010 · Cited by 40
The more we get together: Coordinating network externality product introduction in the RFID industry
Dew, N., & Read, S. (2007) · Technovation · doi:10.1016/j.technovation.2006.12.005 · Cited by 39
Interdependence Issues in Analyzing Negotiation Data
Turel, O. (2008) · Group Decision and Negotiation · doi:10.1007/s10726-008-9118-x · Cited by 39
Effects of transactional leadership, psychological empowerment and empowerment climate on creative performance of subordinates: A cross-level study
Wei, F., Yuan, X., & Di, Y. (2010) · Frontiers of Business Research in China · doi:10.1007/s11782-010-0002-6 · Cited by 39
“The median Rwg value for empowerment climate (0.97) was above the recommended criterion of 0.70 ( James, Demaree and Wolf, 1993; Klein & Kozlowski, 2000).”
MethodInfluential
Shared leadership and group interaction styles in problem-solving virtual teams
Balthazard, P. A., Waldman, D. A., Howell, J. M., & ATWATER, L. E. (2004) · Hawaii International Conference on System Sciences (HICSS) · doi:10.1109/hicss.2004.1265159 · Cited by 41
Top Management Team Advice Seeking and Exploratory Innovation: The Moderating Role of TMT HeterogeneityPreprint
Alexiev, A. S., Jansen, J. J. P., Bosch, F. A. J. V. D., & Volberda, H. (2008) · SSRN Electronic Journal · doi:10.2139/ssrn.1083987 · Cited by 39
Adding fuel to the fire
Khan, M. S., Breitenecker, R. J., & Schwarz, E. J. (2015) · Management Decision · doi:10.1108/md-02-2014-0066 · Cited by 39
The antecedents of collective creative efficacy for information system development teams
Cheng, H.-H., & Yang, H. (2014) · Journal of Engineering and Technology Management · doi:10.1016/j.jengtecman.2013.12.001 · Cited by 38
Linking Firms, Employees, and Customers
Wong, I. A. (2015) · Cornell Hospitality Quarterly · doi:10.1177/1938965515578715 · Cited by 38
Antecedents and Consequences of Creativity in Teams: When and How Leader Humility Promotes Performance via Team Creativity
Wang, X., Li, H., & Yin, H. (2019) · The Journal of Creative Behavior · doi:10.1002/jocb.410 · Cited by 47
CEO TRANSFORMATIONAL LEADERSHIP AND SME INNOVATION: THE MEDIATING ROLE OF SOCIAL CAPITAL AND EMPLOYEE PARTICIPATION
Cortes, A. F., & Herrmann, P. (2019) · International Journal of Innovation Management · doi:10.1142/s1363919620500243 · Cited by 38
The trickle-down effect of OCB in schools: the link between leader OCB and team OCB
Somech, A., & Ohayon, B.-E. (2019) · Journal of Educational Administration · doi:10.1108/jea-03-2019-0056 · Cited by 38
Predicting Sustainable Employability in Swedish Healthcare: The Complexity of Social Job Resources
Roczniewska, M. A., Richter, A., Hasson, H., & Schwarz, U. V. T. (2020) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph17041200 · Cited by 44
The Effect of Psychosocial Safety Climate on Engagement and Psychological Distress: A Multilevel Study on the Healthcare Sector
Platania, S., Morando, M., Caruso, A., & Scuderi, V. (2022) · Safety · doi:10.3390/safety8030062 · Cited by 39
“Homogeneity of climate perceptions was assessed with rwg [110,111], intraclass correlation [ICC (1) Anova Random Effect 1-way], and reliability of the mean [ICC (2) 2-way Anova Fixed effect], [112,113].”
Method
Development and Validation of the Controller Acceptance Rating Scale (CARS): Results of Empirical Research
Lee, K. K., Kerns, K., & Bone, R. S. (2001) · Cited by 38
Positive affective tone and team performance: The moderating role of collective emotional skills
Collins, A. L., Jordan, P. J., Lawrence, S. A., & Troth, A. C. (2015) · Cognition & Emotion · doi:10.1080/02699931.2015.1043857 · Cited by 43
The proximal‐virtual team continuum: A study of performance
Workman, M. (2007) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.20545 · Cited by 39
The effects from technology-mediated interaction and openness in virtual team performance measures
Workman, M. (2006) · Behaviour and Information Technology · doi:10.1080/01449290500402809 · Cited by 38
Bringing group-level personality to the electronic realm: A comparison of face-to-face and virtual contexts.
MacDonnell, R., O’Neill, T., Kline, T. J. B., & Hambley, L. A. (2009) · The Psychologist-Manager Journal · doi:10.1080/10887150802371773 · Cited by 38
“Using the r wg statistic ( James, Demaree, & Wolf, 1984 ; 1993 ), scores for the 62 teams ranged from 0.”
Method
DO SITUATIONAL VARIABLES ACT AS SUBSTANTIVE CAUSES OF RELATIONSHIPS BETWEEN INDIVIDUAL DIFFERENCE VARIABLES? TWO LARGE‐SCALE TESTS OF “COMMON CAUSE” MODELS
Burke, M. J., Rupinski, M. T., Dunlap, W. P., & Davison, H. K. (1996) · Personnel Psychology · doi:10.1111/j.1744-6570.1996.tb01585.x · Cited by 38
Good and bad simultaneously?
Bai, Y., Harms, P. D., Han, G. H., & Cheng, W. (2015) · International Journal of Conflict Management · doi:10.1108/ijcma-09-2014-0070 · Cited by 37
How the organizational ethical climate accounts for employee voice behavior: a multilevel analysis
Wang, H., & Yen, Y.-F. Y. (2021) · Asia Pacific Journal of Human Resources · doi:10.1111/1744-7941.12315 · Cited by 37
Unpacking the relationships between conflicts and team innovation
He, Y., Ding, X., & Yang, K. (2014) · Management Decision · doi:10.1108/md-03-2014-0127 · Cited by 37
Testing an empathy model of guest‐directed citizenship and counterproductive behaviours in the hospitality industry: Findings from three hotels
Ho, V. T., & Gupta, N. (2011) · Journal of Occupational and Organizational Psychology · doi:10.1111/j.2044-8325.2011.02046.x · Cited by 37
Investigating the Relation Between Team Learning and the Team Situation Model
van der Haar, S., Segers, M. S. R., Jehn, K. A., & den Bossche, P. V. (2014) · Small Group Research · doi:10.1177/1046496414558840 · Cited by 40
Performer, Rater, Occasion, and Sequence as Sources of Variability in Music Performance Assessment
Bergee, M. J. (2007) · Journal of Research in Music Education · doi:10.1177/0022429408317515 · Cited by 37
Seeing you seeing me: Stereotypes and the stigma magnification effect.
Mikolon, S., Kreiner, G., & Wieseke, J. (2016) · Journal of Applied Psychology · doi:10.1037/apl0000060 · Cited by 32
How, Why, and When High-Involvement Work Systems Are Related to OCB: A Multilevel Examination of the Mediating Role of POS and of the Moderating Role of Organizational Structures
Tremblay, M. (2017) · Group & Organization Management · doi:10.1177/1059601117736684 · Cited by 39
Creativity through Connectedness: The Role of Closeness and Perspective Taking in Group Creativity
Oztop, P., Katsikopoulos, K. V., & Gummerum, M. (2018) · Creativity Research Journal · doi:10.1080/10400419.2018.1488347 · Cited by 37
“Median of perspective taking scores’ r wg values was .87 and it was above the generally accepted cut-off of r wg = .70 , using a uniform expected distribution (James et al., 1993).”
Ethical Leadership and Follower Moral Actions: Investigating an Emotional Linkage
Zhang, Y., Fangfang, Z., & Mao, J. (2018) · Frontiers in Psychology · doi:10.3389/fpsyg.2018.01881 · Cited by 44
Antecedents and consequences of collective psychological ownership: The validation of a conceptual model
Giordano, A. P., Patient, D., Passos, A., & Sguera, F. (2019) · Journal of Organizational Behavior · doi:10.1002/job.2418 · Cited by 43
Corporate Social Responsibility and Firms’ Financial Performance: A New Insight
Mahmood, F., Qadeer, F., Sattar, U., Ariza‐Montes, A., Saleem, M., & Aman, J. (2020) · Sustainability · doi:10.3390/su12104211 · Cited by 37
Does employee participation matter? An empirical study on the effects of participation on well-being and organizational performance
Andres, U. U., Garmendia, A., & Elorza, U. (2020) · Central European Journal of Operations Research · doi:10.1007/s10100-020-00704-7 · Cited by 50
Controlling for Quality: Climate, Leadership, and Behavior
Luria, G. (2008) · Quality Management Journal · doi:10.1080/10686967.2008.11918054 · Cited by 37
“Homogeneity of climate perceptions was assessed with rwg (James, Demaree, and Wolf 1993), intra-class correlation (ICC1), and reliability of mean (ICC2)( James 1982; Shrout and Fleiss 1979).”
BackgroundMethod
Corporate and social performance links in the gaming industry
Vong, F., & Wong, I. A. (2013) · Journal of Business Research · doi:10.1016/j.jbusres.2012.12.014 · Cited by 37
Work-safety tension, perceived risk, and worker injuries: A meso-mediational model
McGonagle, A. K., & Kath, L. M. (2010) · Journal of Safety Research · doi:10.1016/j.jsr.2010.09.002 · Cited by 37
Analyzing the relationship between distributed leadership and instructional quality in Taiwan: The mediating roles of teacher autonomy and teacher innovation
Hsieh, C., Song, Y., & Li, H. (2024) · Educational Management Administration & Leadership · doi:10.1177/17411432241231421 · Cited by 36
“Assessing the validity of this aggregation, we calculated rWG(j) as a measure of the within-group interrater agreement among school teachers regarding the school-level constructs (James et al., 1993).”
Influential
Antecedents of relationship conflict in top management teams
Camelo‐Ordaz, C., Cruz, J. G., & Ginel, E. S. (2014) · International Journal of Conflict Management · doi:10.1108/ijcma-06-2012-0054 · Cited by 41
Impact of component supplier branding on profitability
Worm, S., & Srivastava, R. K. (2014) · International Journal of Research in Marketing · doi:10.1016/j.ijresmar.2014.05.005 · Cited by 36
Leader and Peer Ethical Behavior Influences on Job Embeddedness
Ferreira, A. I. (2017) · Journal of Leadership & Organizational Studies · doi:10.1177/1548051817702095 · Cited by 36
Process standardization and error reduction: A revisit from a choice approach
Nissinboim, N., & Naveh, E. (2017) · Safety Science · doi:10.1016/j.ssci.2017.11.015 · Cited by 36
“In order to support the aggregation of the variables from the individual level to the average department level, a within-department agreement must be established using the rwg agreement index (James et al., 1993).”
Method
Positioning through B2B carrier signals: Understanding how service quality is communicated via websites
Williams, Z., Lueg, J. E., Hancock, T. D., & Goffnett, S. P. (2018) · Industrial Marketing Management · doi:10.1016/j.indmarman.2018.09.007 · Cited by 36
How Newcomers’ Work-Related Use of Enterprise Social Media Affects Their Thriving at Work—The Swift Guanxi Perspective
Sun, Y., Zhu, M., & Zhang, Z. (2019) · Sustainability · doi:10.3390/su11102794 · Cited by 37
“7, which suggested that the within-group agreement in our sample were sufficient to aggregate [73,74].”
Background
The effect of work-leisure conflict on front-line employees’ work engagement: A cross-level study from the emotional perspective
Wang, F., & Shi, W. (2020) · Asia Pacific Journal of Management · doi:10.1007/s10490-020-09722-0 · Cited by 41
Familiness, social capital and market orientation in the family firm
Martín‐Santana, J. D., Suárez, M. K. C., & Déniz, M. D. L. C. D. (2020) · European Journal of Marketing · doi:10.1108/ejm-04-2018-0274 · Cited by 36
Ethical leadership, perceived leader–member ethical communication and organizational citizenship behavior: development and validation of a multilevel model
Bakar, H. A., & Connaughton, S. L. (2022) · Leadership & Organization Development Journal · doi:10.1108/lodj-07-2021-0356 · Cited by 36
THE EFFECT OF PERSONAL VALUES, ORGANIZATIONAL CULTURE, AND PERSON- ORGANIZATION FIT ON INDIVIDUAL OUTCOMES IN THE RESTAURANT INDUSTRY
Tepeci, M. (2001) · Cited by 36
Crossover specificity of team‐level work‐family conflict to individual‐level work‐family conflict
van Emmerik, I. H., & Peeters, M. C. W. (2009) · Journal of Managerial Psychology · doi:10.1108/02683940910939331 · Cited by 38
The effect of training in verbal self-guidance on performance effectiveness in a MBA program.
Brown, T. C., & Latham, G. P. (2006) · Canadian Journal of Behavioural Science / Revue canadienne des sciences du comportement · doi:10.1037/h0087266 · Cited by 23
Team emotional intelligence in working contexts: development and validation of the team-trait meta mood scale (T-TMMS)
Aritzeta, A., Mindeguia, R., Soroa, G., Balluerka, N., Gorostiaga, A., Elorza, U., & Aliri, J. (2020) · Frontiers in Psychology · doi:10.3389/fpsyg.2020.00893 · Cited by 21
Emotional labor among team members: do employees follow emotional display norms for teams, not for customers?
Hong, D., & Kim, M. (2023) · Frontiers in Psychology · doi:10.3389/fpsyg.2023.1265581 · Cited by 7
“In fact, before aggregating the display norms ratings, we examined the average r wg (0.74) (Barcikowski, 1981; James et al., 1993).”
BackgroundInfluential

Authors: James, L. R., Demaree, R. G., & Wolf, G.
Affiliations: University of Tennessee at Knoxville (US)
Venue: Journal of Applied Psychology
Publisher: American Psychological Association (APA)
Published: 1993-04
Volume: 78
Issue: 2
Pages: 306-309
ISSN: 0021-9010 (ISSN-L)
1939-1854
DOI: 10.1037/0021-9010.78.2.306
OpenAlex: W2022950201
Cited by: 2,221
References: 21
Citations by year: 2026: 54 · 2025: 88 · 2024: 76 · 2023: 85 · 2022: 88 · 2021: 97 · 2020: 95 · 2019: 98 · 2018: 108 · 2017: 82 · 2016: 86 · 2015: 100 · 2014: 111 · 2013: 110 · 2012: 96
FWCI: 12.64
Type: journal article
Language: en
Open access: closed
Locations: Journal of Applied Psychology (publishedVersion)
Topics: Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement
Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
Social Sciences › Psychology › Social Psychology › Team Dynamics and Performance
Keywords: inter-rater reliability, interobserver agreement

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

rwg: an assessment of within-group interrater agreement

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks