Cited by 2,221Reliability and Agreement in MeasurementPsychometric Methodologies and TestingTeam Dynamics and Performance

The DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index.

rwg: an assessment of within-group interrater agreement

James, Lawrence R., Demaree, Robert G., Wolf, Gerrit·1993·Journal of Applied Psychology

DOI 10.1037/0021-9010.78.2.306

Open this paper in CoChat — free account

Abstract

Schmidt and Hunter (1989) critiqued the within-group interrater reliability statistic (r wg ) described by James, Demaree, and Wolf (1984). Kozlowski and Hattrup (1992) responded to the Schmidt and Hunter critique and argued that r wg is a suitable index of interrater agreement. This article focuses on the interpretation of r wg as a measure of agreement among judges' ratings of a single target. A new derivation of r wg is given that underscores this interpretation

Integrity signals

Registry

Crossref · 10.1037/0021-9010.78.2.306

Cites

21 of 21 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Statistical principles in experimental design.
Winer, B. J. (1962) · McGraw-Hill Book Company · doi:10.1037/11774-000 · Cited by 27,137
Intraclass correlations: Uses in assessing rater reliability.
Shrout, P. E., & Fleiss, J. L. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.2.420 · Cited by 24,532
Applications of Item Response Theory To Practical Testing Problems
Lord, F. M. (2012) · Routledge eBooks · doi:10.4324/9780203056615 · Cited by 5,009
Estimating within-group interrater reliability with and without response bias
James, L. R., Demaree, R. G., & Wolf, G. (1984) · Journal of Applied Psychology · doi:10.1037/0021-9010.69.1.85 · Cited by 4,774
Self-citation
The Dependability of Behavioral Measurements: Theory of Generalizability for Scores and Profiles
Merrifield, P. R., Cronbach, L. J., Gleser, G. C., Nanda, H., & Rajaratnam, N. (1974) · American Educational Research Journal · doi:10.2307/1162145 · Cited by 1,819
Employee and customer perceptions of service in banks: Replication and extension.
Schneider, B., & Bowen, D. E. (1985) · Journal of Applied Psychology · doi:10.1037/0021-9010.70.3.423 · Cited by 1,344
Superiors' evaluations and subordinates' perceptions of transformational and transactional leadership.
Hater, J. J., & Bass, B. M. (1988) · Journal of Applied Psychology · doi:10.1037/0021-9010.73.4.695 · Cited by 1,136
Response Sets and Test Validity
Cronbach, L. J. (1946) · Educational and Psychological Measurement · doi:10.1177/001316444600600405 · Cited by 898
Interrater reliability and agreement of subjective judgments.
Tinsley, H. E. A., & Weiss, D. J. (1975) · Journal of Counseling Psychology · doi:10.1037/h0076640 · Cited by 883
On various intraclass correlation reliability coefficients.
Bartko, J. J. (1976) · Psychological Bulletin · doi:10.1037/0033-2909.83.5.762 · Cited by 830

Flagged first, then by citations

Show all 21 references

Sorted by citations

Items 701–800 of 1,001

Maintaining Employees’ Commitment to Organizational Change
Shin, J., Seo, M., Shapiro, D. L., & Taylor, M. S. (2015) · The Journal of Applied Behavioral Science · doi:10.1177/0021886315603123 · Cited by 65
“The measures of agreement among work-unit members’ ratings produced a median rwg(j) exceeding .70 (James et al., 1993), ICC(1) of .16, and ICC(2) of .42 (F = 1.66, p < .001; Bliese, 2000), which provided support for aggregation of transformational leader behavior ratings.”
BackgroundInfluential
Ethical Leadership and Multidimensional Organizational Citizenship Behaviors
Yang, C., Ding, C. G., & Lo, K. W. (2015) · Group & Organization Management · doi:10.1177/1059601115594973 · Cited by 67
“12 (Bliese & Halverson, 1998; James et al., 1993) and thus, sufficient for group-aggregated measures.”
Background
How Does Ethical Leadership Trickle Down? Test of an Integrative Dual-Process Model
Wang, Z., Xu, H., & Liu, Y. (2016) · Journal of Business Ethics · doi:10.1007/s10551-016-3361-x · Cited by 67
“91 respectively and were above the suggested cutoff point (Bliese 2000; James et al. 1993), supporting the”
Background
Slacking Off in Comfort: A Dual-Pathway Model for Psychological Safety Climate
Deng, H., Leung, K. Y., Lam, C. K., & Huang, X. (2017) · Journal of Management · doi:10.1177/0149206317693083 · Cited by 79
Is there agreement between worker self and supervisor assessment of worker safety performance? An examination in the construction industry
Xia, N., Griffin, M. A., Wang, X., Liu, X., & Wang, D. (2018) · Journal of Safety Research · doi:10.1016/j.jsr.2018.03.001 · Cited by 65
Directing idea generation using brainstorming with specific novelty goals
Litchfield, R. C., Fan, J., & Brown, V. R. (2011) · Motivation and Emotion · doi:10.1007/s11031-011-9203-3 · Cited by 62
“To justify aggregating ratings, we computed the average rwg for novelty (.78), creativity (.79), effectiveness (.77), and practicality (.71) of the rated ideas ( James et al. 1984, 1993 ).”
Method
Outcomes of Job Insecurity Climate: The Role of Climate Strength
Sora, B., Cuyper, N. D., Caballer, A., Silla, J. M. P., & Witte, H. D. (2012) · Applied Psychology · doi:10.1111/j.1464-0597.2012.00485.x · Cited by 66
Team mental models, relationship conflict and effectiveness over time
Santos, C. M., & Passos, A. (2013) · Team Performance Management · doi:10.1108/tpm-01-2013-0003 · Cited by 75
“To justify aggregation, we computed RwgðjÞ ( James et al., 1993), designed for multiple-item scales, and intraclass correlations coefficients (ICC) (Bliese, 2000).”
Method
Bridging the implementation gaps in the knowledge management system for enhancing corporate performance
LIN, C., & Tseng, S. (2005) · Expert Systems with Applications · doi:10.1016/j.eswa.2005.01.015 · Cited by 64
“(2003) Which industries have been developing knowledge that could pose a threat to you? Can employees screen out the useful KM for the firm from the external environment? How effectiveness and efficiency employees can communicate the knowledge obtained from the external environment with their managers? Do you know abo…”
BackgroundInfluential
Organizational citizenship behavior in work groups: A team cultural perspective
Lai, J. Y. M., Lam, L. W., & Lam, S. S. K. (2012) · Journal of Organizational Behavior · doi:10.1002/job.1840 · Cited by 67
“To assess the within-group agreement, we computed the within-group inter-rater reliability (rwg) by using the method developed by James, Demaree, and Wolf (1984, 1993).”
Method
Linking Employee Confidence to Performance: A Study of Self-Managing Service Teams
de Jong, A. (2006) · Journal of the Academy of Marketing Science · doi:10.1177/0092070306287126 · Cited by 64
A multilevel study of transformational leadership, dual organizational change and innovative behavior in groups
Feng, C., Huang, X., & Zhang, L. (2016) · Journal of Organizational Change Management · doi:10.1108/jocm-01-2016-0005 · Cited by 64
Perceived supervisor support and turnover intention: moderating effect of authentic leadership
Arıcı, H. E. (2018) · Leadership & Organization Development Journal · doi:10.1108/lodj-07-2018-0248 · Cited by 64
Workplace favouritism, psychological contract violation and turnover intention: Moderating roles of authentic leadership and job insecurity climate
Araslı, H., Arıcı, H. E., & Arıcı, N. Ç. (2019) · German Journal of Human Resource Management Zeitschrift für Personalforschung · doi:10.1177/2397002219839896 · Cited by 64
Group Diversity, Occupational Commitment, and Occupational Turnover Intentions among NCAA Division IA Football Coaching Staffs
Cunningham, G. B., & Sagas, M. (2004) · Journal of Sport Management · doi:10.1123/jsm.18.3.236 · Cited by 64
How an organization's ethical climate contributes to customer satisfaction and financial performance
Moon, H. K., & Choi, B. K. (2013) · European Journal of Innovation Management · doi:10.1108/ejim-03-2013-0020 · Cited by 68
Schools as team-based organizations: A structure-process-outcomes approach.
Somech, A., & Drach‐Zahavy, A. (2007) · Group Dynamics Theory Research and Practice · doi:10.1037/1089-2699.11.4.305 · Cited by 66
A Multilevel Investigation of the Influences of Employees' Resistance to Empowerment
Maynard, M. T., Mathieu, J. E., Marsh, W. M., & Ruddy, T. M. (2007) · Human Performance · doi:10.1080/08959280701332885 · Cited by 65
Effects of Organizational Support on Potency in Work Teams
Kennedy, F. A., Loughry, M. L., Klammer, T., & Beyerlein, M. (2008) · Small Group Research · doi:10.1177/1046496408326744 · Cited by 61
Problem-Solving Team Behaviors
Taggar, S., & Brown, T. C. (2001) · Small Group Research · doi:10.1177/104649640103200602 · Cited by 62
The Importance of Transformational Leadership Behaviors in Team Mental Model Similarity, Team Efficacy, and Intra-Team Conflict
Ayoko, O. B., & Chua, E. L. (2014) · Group & Organization Management · doi:10.1177/1059601114550080 · Cited by 68
The relationship between organizational democracy and socio-moral climate: Exploring effects of the ethical context in organizations
Verdorfer, A. P., Weber, W., Unterrainer, C., & Seyr, S. (2012) · Economic and Industrial Democracy · doi:10.1177/0143831x12450054 · Cited by 63
“Yet to test whether the respondents across the 10 companies agreed on their perceptions of SMC, the rwg score as an indicator of interrater agreement was calculated (James et al., 1993).”
Background
Organizations’ Use of Temporary Employment and a Climate of Job Insecurity among Belgian and Spanish Permanent Workers
Cuyper, N. D., Sora, B., Witte, H. D., Caballer, A., & Silla, J. M. P. (2009) · Economic and Industrial Democracy · doi:10.1177/0143831x09336808 · Cited by 62
Knowledge Management Enablers, Processes, and Organizational Performance: An Integration and Empirical Examination
Lee, H., & Choi, B. (2000) · Cited by 62
Are Your Data Nonindependent? A Practical Guide to Evaluating Nonindependence and Within-Group Agreement
Grawitch, M. J., & Munz, D. C. (2004) · Understanding Statistics · doi:10.1207/s15328031us0304_2 · Cited by 62
“The second source of nonindependence, as defined by Kenny and Judd (1986), results from a compositional effect.”
BackgroundMethod
Mindfulness-based stress reduction
Giluk, T. L. (2010) · doi:10.17077/etd.4ax4yjs5 · Cited by 67
Entrepreneurial team locus of control: diversity and trust
Khan, M. S., Breitenecker, R. J., & Schwarz, E. J. (2014) · Management Decision · doi:10.1108/md-06-2013-0349 · Cited by 60
Moderators of shared leadership: work function and team autonomy
Fausing, M. S., Jeppesen, H. J., Jønsson, T., Lewandowski, J., & Bligh, M. C. (2013) · Team Performance Management · doi:10.1108/tpm-11-2012-0038 · Cited by 72
The role of person-environment fit in the job satisfaction and tenure intentions of African American employees.
Lyons, H. Z., & O’Brien, K. M. (2006) · Journal of Counseling Psychology · doi:10.1037/0022-0167.53.4.387 · Cited by 61
Psychosocial safety climate, workplace violence and self‐rated health: A multi‐level study among hospital nurses
Pien, L., Cheng, Y., & Cheng, W. (2018) · Journal of Nursing Management · doi:10.1111/jonm.12715 · Cited by 61
Perceived Combat Readiness as Collective Efficacy: Individual- and Group-Level Analysis
Shamir, B., Brainin, E., Zakay, E., & Popper, M. (2000) · Military Psychology · doi:10.1207/s15327876mp1202_2 · Cited by 61
Perceived entitlement to pain-related support and pain catastrophizing: associations with perceived and observed support.
Cano, A., Leong, L., Heller, J. B., & Lutz, J. R. (2009) · Pain · doi:10.1016/j.pain.2009.09.023 · Cited by 60
“Furthermore, these other measures do not allow for restriction of range, which can happen when a coder rates validation of the spouse similarly across the two discussions or a group of raters agrees on spouse invalidation ratings across the two interactions [15,16].”
Background
Critical Success Factors in SOA Implementation: An Exploratory Study
Lee, J. H., Shim, H.-J., & Kim, K. K. (2010) · Information Systems Management · doi:10.1080/10580531003685188 · Cited by 60
“This study checked inter-rater reliability to determine the degree of homogeneity or consensus in the ratings awarded by raters ( James, Demaree, & Wolf, 1984 , 1993 ).”
Background
How self-gratification and social values shape revisit intention and customer loyalty of Airbnb customers
Tajeddini, K., Gamage, T. C., Hameed, W. U., Qumsieh-Mussalam, G., Chaijani, M. H., Rasoolimanesh, S. M., & Kallmuenzer, A. (2022) · International Journal of Hospitality Management · doi:10.1016/j.ijhm.2021.103093 · Cited by 60
Interpersonal aggression and team effectiveness: The mediating role of team goal commitment
Aubé, C., & Rousseau, V. (2010) · Journal of Occupational and Organizational Psychology · doi:10.1348/096317910x492568 · Cited by 70
Applied process knowledge and market performance: the moderating effect of environmental uncertainty
Claycomb, C., Dröge, C., & Germain, R. N. (2001) · Journal of Knowledge Management · doi:10.1108/13673270110401239 · Cited by 60
Team Conflict Mediates the Effects of Organizational Politics on Employee Performance: A Cross-Level Analysis in China
Bai, Y., Han, G. H., & Harms, P. D. (2015) · Journal of Business Ethics · doi:10.1007/s10551-015-2604-6 · Cited by 66
“70 (James et al. 1993).”
Method
Virtual team culture and the amplification of team boundary permeability on performance
Workman, M. (2005) · Human Resource Development Quarterly · doi:10.1002/hrdq.1149 · Cited by 60
Evolution in the strategic manufacturing planning process of organizations
Papke‐Shields, K. E., Malhotra, M. K., & Grover, V. (2006) · Journal of Operations Management · doi:10.1016/j.jom.2005.11.012 · Cited by 60
Predicting International Joint Venture Interaction Frequency in U.S.–Chinese Ventures
Lin, X. H., & Germain, R. N. (1999) · Journal of International Marketing · doi:10.1177/1069031x9900700202 · Cited by 60
The Dark Triad and Organizational Citizenship Behaviors: the Moderating Role of High Involvement Management Climate
Webster, B. D., & Smith, M. B. (2018) · Journal of Business and Psychology · doi:10.1007/s10869-018-9562-9 · Cited by 64
Unlocking and closing the gender gap in creative performance: A multilevel model
Hora, S., Lemoine, G. J., Xu, N., & Shalley, C. E. (2021) · Journal of Organizational Behavior · doi:10.1002/job.2500 · Cited by 77
Cross-level effect of resonant leadership on remote engagement: A moderated mediation analysis in the unprecedented COVID-19 crisis
Gaan, N., Malik, S., & Dagar, V. (2023) · European Management Journal · doi:10.1016/j.emj.2023.01.004 · Cited by 66
How firms leverage corporate environmental strategy to nurture green behavior: Role of multi‐level environmentally responsible leadership
Faraz, N. A., Ahmed, F., & Xiong, Z. (2023) · Corporate Social Responsibility and Environmental Management · doi:10.1002/csr.2565 · Cited by 66
Implications of internal organization structure for firm boundaries
Weigelt, C., & Miller, D. J. (2013) · Strategic Management Journal · doi:10.1002/smj.2074 · Cited by 62
The Effects Of Justice And Cooperation On Team Effectiveness
Sinclair, A. L. (2003) · Small Group Research · doi:10.1177/1046496402239578 · Cited by 59
The Impact of Client Variability on Nurses' Occupational Strain and Injury: Cross-Level Moderation by Safety Climate
Chowdhury, S. K., & Endres, M. L. (2010) · Academy of Management Journal · doi:10.5465/amj.2010.48037720 · Cited by 62
To Guide or to Divide: The Dual-Side Effects of Transformational Leadership on Team Innovation
Jiang, W., Gu, Q., & Wang, G. G. (2014) · Journal of Business and Psychology · doi:10.1007/s10869-014-9395-0 · Cited by 61
The Importance of Ethical Leadership in Employees’ Value Congruence and Turnover
Tang, G., Cai, Z., Liu, Z., Zhu, H., Yang, X., & Li, J. (2014) · Cornell Hospitality Quarterly · doi:10.1177/1938965514563159 · Cited by 59
“These rwg(j) values were all above the conventionally acceptable rwg value of .70 (James, Demaree, and Wolf 1993).”
Background
Shared temporal cognitions as substitute for temporal leadership: An analysis of their effects on temporal conflict and team performance
Santos, C. M., Passos, A., Uitdewilligen, S., & Nübold, A. (2016) · The Leadership Quarterly · doi:10.1016/j.leaqua.2015.12.002 · Cited by 59
Who's the Boss at the Top? A Micro‐Level Analysis of Director Expertise, Status and Conformity Within Boards
Veltrop, D. B., Molleman, E., Hooghiemstra, R. B. H., & van Ees, H. (2017) · Journal of Management Studies · doi:10.1111/joms.12276 · Cited by 60
Talking about COVID-19 is positively associated with team cultural tightness: Implications for team deviance and creativity.
Qin, X., Yam, K. C., Chen, C., Li, W., & Dong, X. (2021) · Journal of Applied Psychology · doi:10.1037/apl0000918 · Cited by 65
Building collaborative structures for teachers’ autonomy and self-efficacy: the mediating role of participative management and learning culture
Lu, J., Jiang, X., Yu, H., & Li, D. (2014) · School Effectiveness and School Improvement · doi:10.1080/09243453.2014.888086 · Cited by 69
“70 (James et al., 1993), but the other two demonstrated strong agreement between participants in each school for participative management and learning culture.”
Background
Micro-foundations of organizational design and sustainability: The mediating role of learning ambidexterity
Al‐Atwi, A. A., Amankwah‐Amoah, J., & Khan, Z. (2019) · International Business Review · doi:10.1016/j.ibusrev.2019.101656 · Cited by 62
A temporal common ground for learning: The moderating effect of shared mental models on the relation between team learning behaviours and performance improvement
Santos, C. M., Uitdewilligen, S., & Passos, A. (2015) · European Journal of Work and Organizational Psychology · doi:10.1080/1359432x.2015.1049158 · Cited by 67
Safety perception referents of permanent and temporary employees: Safety climate boundaries in the industrial workplace
Luria, G., & Yagil, D. (2009) · Accident Analysis & Prevention · doi:10.1016/j.aap.2009.02.016 · Cited by 58
Interorganizational teams as boundary spanners: The role of team diversity, boundedness, and extrateam links
Drach‐Zahavy, A. (2010) · European Journal of Work and Organizational Psychology · doi:10.1080/13594320903115936 · Cited by 59
“I tested for the homogeneity of responses at the unit level by calculating the r wg coefficients (James, Demaree, & Wolf, 1984 ) for each of the four boundary activities for each of the 49 participating interorganizational teams.”
Method
Organizational ambidexterity and professional firm performance: the moderating role of organizational capital
Fu, N., Flood, P. C., & Morris, T. (2016) · Journal of Professions and Organization · doi:10.1093/jpo/jov010 · Cited by 60
When Leaders Stifle Innovation in Work Teams: The Role of Abusive Supervision
Rousseau, V., & Aubé, C. (2016) · Journal of Business Ethics · doi:10.1007/s10551-016-3258-8 · Cited by 65
The role of knowledge sharing and LMX to enhance employee creativity in theme park work team
Liao, S.-H., Chen, C.-C., & Hu, D.-C. (2018) · International Journal of Contemporary Hospitality Management · doi:10.1108/ijchm-09-2016-0522 · Cited by 58
A longitudinal study of work group innovation: The importance of transformational leadership and morale
Wilson‐Evered, E., Härtel, C. E. J., & Neale, M. (2004) · Advances in health care management · doi:10.1016/s1474-8231(01)02032-8 · Cited by 58
A tale of two factions: Exploring the relationship between factional faultlines and conflict management in pension fund boards
Veltrop, D. B., Hermes, C. L. M., Postma, T. J. B. M., & de Haan, J. (2012) · University of Groningen research database (University of Groningen / Centre for Information Technology) · Cited by 58
Personality and Charismatic Leadership in Context: The Moderating Role of Situational Stress
Oreg, S., & Berson, Y. (2013) · Personnel Psychology · doi:10.1111/peps.12073 · Cited by 57
Top management Team's cognitive diversity and the Firm's ambidextrous innovation capability: The mediating role of ambivalent interpretation
Kanchanabha, B., & Badir, Y. F. (2021) · Technology in Society · doi:10.1016/j.techsoc.2020.101499 · Cited by 57
A contingent approach to the role of human capital and competencies on firm strategy
Díaz-Fernández, M., Cabrales, Á. L., & Cabrera, R. V. (2014) · BRQ Business Research Quarterly · doi:10.1016/j.brq.2014.01.002 · Cited by 57
Exploring Team Support: The Role of Team's Design, Values, and Leader's Support.
Drach‐Zahavy, A. (2004) · Group Dynamics Theory Research and Practice · doi:10.1037/1089-2699.8.4.235 · Cited by 57
The effects of stand-up and sit-down meeting formats on meeting outcomes.
Bluedorn, A. C., Turban, D. B., & Love, M. S. (1999) · Journal of Applied Psychology · doi:10.1037/0021-9010.84.2.277 · Cited by 57
Nurturing employee market knowledge absorptive capacity through unified internal communication and integrated information technology
Castillo, D. J., & Pérez, M. S. (2013) · Information & Management · doi:10.1016/j.im.2013.01.001 · Cited by 57
Management commitment to safety vs. employee perceived safety training and association with future injury
Huang, Y.-H., Verma, S. K., Chang, W.-R., Courtney, T. K., Lombardi, D. A., Brennan, M. J., & Perry, M. J. (2012) · Accident Analysis & Prevention · doi:10.1016/j.aap.2011.12.001 · Cited by 57
The role of relational resources in the knowledge management capability and innovation of professional service firms
Fu, N. (2014) · Human Relations · doi:10.1177/0018726714543479 · Cited by 57
“Inter-rater agreement was assessed using rWG (James et al., 1984, 1993) for each variable (see Table 1).”
Method
Intolerable Cruelty: A Multilevel Examination of the Impact of Toxic Leadership on U.S. Military Units and Service Members
Gallus, J. A., Walsh, B. M., van Driel, M., Gouge, M. C., & Antolic, E. (2013) · Military Psychology · doi:10.1037/mil0000022 · Cited by 57
R&D project team climate and team performance in Korea: A multidimensional approach
Kim, Y., & Lee, B.-H. (1995) · R and D Management · doi:10.1111/j.1467-9310.1995.tb00910.x · Cited by 57
Variance vs average: supply chain lead‐time as a predictor of financial performance
Christensen, W. J., Germain, R. N., & Birou, L. M. (2007) · Supply Chain Management An International Journal · doi:10.1108/13598540710776926 · Cited by 57
“) was generated for the five constructs in the model ( James et al. , 1993”
Method
From Metrics to Meaning: Culture Change and Quality of Acute Hospital Care for Older People
Patterson, M. G., Nolan, M., Rick, J., Brown, J., & Adams, R. J. (2010) · Cited by 56
Experimentation in the face of ambiguity: How mindful leaders develop emotional capabilities for change in teams
Harvey, J., & Kudesia, R. S. (2023) · Journal of Organizational Behavior · doi:10.1002/job.2693 · Cited by 60
The proficiency trap: how to balance enriched job designs and the team's need for support
Drach‐Zahavy, A. (2004) · Journal of Organizational Behavior · doi:10.1002/job.284 · Cited by 56
Firm culture and performance: intensity's effects and limits
Murphy, P. J., Cooke, R. A., & Lopez, Y. P. (2013) · Management Decision · doi:10.1108/00251741311309715 · Cited by 57
“We executed the rwg procedure to assess agreement among members to warrant aggregation to unit level scores (James et al., 1993).”
Method
Direct and Multiplicative Effects of Ethical Dispositions and Ethical Climates on Personal Justice Norms: A Virtue Ethics Perspective
Lau, V. P., & Wong, Y. Y. (2009) · Journal of Business Ethics · doi:10.1007/s10551-009-0042-z · Cited by 52
Organizational culture and climate profiles: relationships with fidelity to three evidence-based practices for autism in elementary schools
Williams, N. J., Frank, H. E., Frederick, L., Beidas, R. S., Mandell, D. S., Aarons, G. A., Green, P., & Locke, J. J. (2019) · Implementation Science · doi:10.1186/s13012-019-0863-9 · Cited by 61
“, proficiency, rigidity, resistance, engagement, functionality, stress) was provided by the rwg(j) statistic [51, 52] where values > .”
Classroom observation and mathematics education research
Bostic, J. D., Lesseig, K., Sherman, M. F., & Boston, M. D. (2019) · Journal of Mathematics Teacher Education · doi:10.1007/s10857-019-09445-0 · Cited by 56
Tell us about your leadership style: A structured interview approach for assessing leadership behavior constructs
Heimann, A. L., Ingold, P. V., & Kleinmann, M. (2019) · The Leadership Quarterly · doi:10.1016/j.leaqua.2019.101364 · Cited by 70
The social perception of emotional abilities: Expanding what we know about observer ratings of emotional intelligence.
Elfenbein, H. A., Barsade, S. G., & Eisenkraft, N. (2015) · Emotion · doi:10.1037/a0038436 · Cited by 65
Group Task Satisfaction
Mason, C., & Griffin, M. A. (2005) · Group & Organization Management · doi:10.1177/1059601104269522 · Cited by 55
Shared Leadership and Team Effectiveness: An Investigation of Whether and When in Engineering Design Teams
Wu, Q., & Cormican, K. T. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2020.569198 · Cited by 55
Does integrity matter for CSR practice in organizations? The mediating role of transformational leadership
Veríssimo, J. M. C., & de Lacerda, T. C. (2014) · Business Ethics A European Review · doi:10.1111/beer.12065 · Cited by 58
Antecedents and consequences of environmental stewardship in boundary-spanning B2B teams
de Ruyter, K., de Jong, A., & Wetzels, M. G. M. (2009) · Journal of the Academy of Marketing Science · doi:10.1007/s11747-009-0138-0 · Cited by 56
“These findings demonstrate that individual ratings within groups are highly consistent (James et al. 1993).”
Team Decision Making in Virtual and Face-to-Face Environments
O’Neill, T. A., Hancock, S. E., Zivkov, K., Larson, N. L., & Law, S. J. (2016) · Group Decision and Negotiation · doi:10.1007/s10726-015-9465-3 · Cited by 54
Collectivism-oriented human resource management on team creativity: effects of interpersonal harmony and human resource management strength
Chen, S.-L., Fan, Y., Zhang, G.-L., & Zhang, Y. (2019) · International journal of human resources management · doi:10.1080/09585192.2019.1640765 · Cited by 48
Diversified Knowledge, R&D Team Centrality and Radical Creativity
Tang, C., & Ye, L. (2015) · Creativity and Innovation Management · doi:10.1111/caim.12110 · Cited by 54
How Should Customers Be Integrated for Effective Interorganizational NPD Teams? An Input–Process–Output Perspective
Stock, R. (2014) · Journal of Product Innovation Management · doi:10.1111/jpim.12112 · Cited by 55
Properties of group safety climate in construction: the development and evaluation of a typology
Lingard, H. C., Cooke, T., & Blismas, N. G. (2010) · Construction Management and Economics · doi:10.1080/01446193.2010.501807 · Cited by 54
Private equity and entrepreneurial management in management buy-outs
Bruining, H., Verwaal, E., & Wright, M. (2011) · Small Business Economics · doi:10.1007/s11187-011-9386-8 · Cited by 54
Is training leaders in functional leadership a useful tool for improving the performance of leadership functions and team effectiveness?
Santos, J. P., Caetano, A., & Tavares, S. M. D. O. E. M. (2015) · The Leadership Quarterly · doi:10.1016/j.leaqua.2015.02.010 · Cited by 74
Gender diversity in leadership succession: Preparing for the future
Virick, M., & Greer, C. R. (2012) · Human Resource Management · doi:10.1002/hrm.21487 · Cited by 56
Informational Faultlines, Integrative Capability, and Team Creativity
Qu, X., & Liu, X. (2017) · Group & Organization Management · doi:10.1177/1059601117716008 · Cited by 57
When Team Identity Helps Innovation and When It Hurts: Team Identity and Its Relationship to Team and Cross‐Team Innovative Behavior
Litchfield, R. C., Karakitapoğlu‐Aygün, Z., Gumusluoğlu, L., Carter, M. R., & Hirst, G. (2017) · Journal of Product Innovation Management · doi:10.1111/jpim.12410 · Cited by 54
Talking Safety: Health and Safety Communication and Safety Climate in Subcontracted Construction Workgroups
Lingard, H. C., Pirzadeh, P., & Oswald, D. (2019) · Journal of Construction Engineering and Management · doi:10.1061/(asce)co.1943-7862.0001651 · Cited by 54
Perceived organisational climate, knowledge transfer and innovation in China-based research and development companies
Zhang, Y., & Begley, T. M. (2011) · The International Journal of Human Resource Management · doi:10.1080/09585192.2011.538967 · Cited by 54
“To ensure appropriate comparisons of data, we assessed within-group interrater agreement ( r wg ) for the variables from the two subsamples (James, Demaree and Wolf 1993 ).”
Method
The relationship among leadership, innovation and knowledge sharing: A guidance for analysis
Elrehail, H. (2018) · Data in Brief · doi:10.1016/j.dib.2018.04.138 · Cited by 54
“Following [17] recommendations, intra-class correlations analysis (ICC), with the aid of two-way mixed and absolute agreement definitions, were used to assess the level of agreement betweengroups.”
Method
You raise me up and I reciprocate: Linking empowering leadership to organizational citizenship behavior and unethical pro‐organizational behavior
Wang, H., Zhang, Y., Li, P., & Henry, S. E. (2022) · Applied Psychology · doi:10.1111/apps.12398 · Cited by 53

Authors: James, L. R., Demaree, R. G., & Wolf, G.
Affiliations: University of Tennessee at Knoxville (US)
Venue: Journal of Applied Psychology
Publisher: American Psychological Association (APA)
Published: 1993-04
Volume: 78
Issue: 2
Pages: 306-309
ISSN: 0021-9010 (ISSN-L)
1939-1854
DOI: 10.1037/0021-9010.78.2.306
OpenAlex: W2022950201
Cited by: 2,221
References: 21
Citations by year: 2026: 54 · 2025: 88 · 2024: 76 · 2023: 85 · 2022: 88 · 2021: 97 · 2020: 95 · 2019: 98 · 2018: 108 · 2017: 82 · 2016: 86 · 2015: 100 · 2014: 111 · 2013: 110 · 2012: 96
FWCI: 12.64
Type: journal article
Language: en
Open access: closed
Locations: Journal of Applied Psychology (publishedVersion)
Topics: Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement
Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
Social Sciences › Psychology › Social Psychology › Team Dynamics and Performance
Keywords: inter-rater reliability, interobserver agreement

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

rwg: an assessment of within-group interrater agreement

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks