The DOI matches its Crossref record; 68 of the 68 references it cites were found in a registry or an index.
Within-group agreement: On the use (and misuse) of r and r in leadership research and some best practice guidelines
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Intraclass correlations: Uses in assessing rater reliability.
Shrout, P. E., & Fleiss, J. L. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.2.420 · Cited by 24,532
Statistical Theories of Mental Test Scores
Rozeboom, W. W., Lord, F. M., Novick, M. R., & Birnbaum, A. (1969) · American Educational Research Journal · doi:10.2307/1162101 · Cited by 8,022
Forming inferences about some intraclass correlation coefficients.
McGraw, K. O., & Wong, S. P. (1996) · Psychological Methods · doi:10.1037/1082-989x.1.1.30 · Cited by 6,903
Klein, K. J., & Kozlowski, S. W. J. (2000) · Cited by 4,914
Estimating within-group interrater reliability with and without response bias
James, L. R., Demaree, R. G., & Wolf, G. (1984) · Journal of Applied Psychology · doi:10.1037/0021-9010.69.1.85 · Cited by 4,774
Consequences of abusive supervision.
Tepper, B. J. (2000) · Academy of Management Journal · doi:10.2307/1556375 · Cited by 3,810 · cited 1×
“For example, assume researchers are interested in aggregating subordinates' ratings of managers' abusive supervisory behavior by means of the measure developed by Tepper (2000).”
BackgroundAnswers to 20 questions about interrater reliability and interrater agreement
LeBreton, J. M., & Senter, J. L. (2008) · Organizational Research Methods · doi:10.1177/1094428106296642 · Cited by 3,608 · cited 14×
“By computing IRA and IRR statistics and comparing these values to cut-points (a priori) based on previously published research, multilevel leadership researchers can better determine if their data “lack reliability, agreement, neither, or both” (LeBreton & Senter, 2008, p. 840).”
BackgroundMethodInfluential(2000) · Cited by 3,573
Predicting unit performance by assessing transformational and transactional leadership.
Bass, B. M., Avolio, B. J., Jung, D. I., & Berson, Y. (2003) · Journal of Applied Psychology · doi:10.1037/0021-9010.88.2.207 · Cited by 3,026
Chan, D. K. (1998) · Journal of Applied Psychology · doi:10.1037//0021-9010.83.2.234 · Cited by 2,972
rwg: an assessment of within-group interrater agreement
James, L. R., Demaree, R. G., & Wolf, G. (1993) · Journal of Applied Psychology · doi:10.1037/0021-9010.78.2.306 · Cited by 2,221 · cited 3×
“…a common tendency among raters is to select a socially desirable response option rather than one reflective of their true beliefs, the subsequent appearance of high interrater agreement is indicative of a response bias and is not reflective of true agreement among raters (James et al., 1984, 1993).”
BackgroundThe Sources of Four Commonly Reported Cutoff Criteria: What Did They Really Say?
Lance, C. E., Butts, M. M., & Michels, L. C. (2006) · Organizational Research Methods · doi:10.1177/1094428105284919 · Cited by 2,053
Avolio, B. J., Zhu, W., Koh, W., & Bhatia, P. (2004) · Journal of Organizational Behavior · doi:10.1002/job.283 · Cited by 1,669
Personality, affect, and behavior in groups
George, J. M. (1990) · Journal of Applied Psychology · doi:10.1037/0021-9010.75.2.107 · Cited by 1,372
SELF-CONCORDANCE AT WORK: TOWARD UNDERSTANDING THE MOTIVATIONAL EFFECTS OF TRANSFORMATIONAL LEADERS.
Bono, J. E., & Judge, T. A. (2003) · Academy of Management Journal · doi:10.2307/30040649 · Cited by 1,331
Mezulis, A. H., Abramson, L. Y., Hyde, J. S., & Hankin, B. L. (2004) · Psychological Bulletin · doi:10.1037/0033-2909.130.5.711 · Cited by 1,266
Interrater reliability and agreement of subjective judgments.
Tinsley, H. E., & Weiss, D. J. (1975) · Journal of Counseling Psychology · doi:10.1037/h0076640 · Cited by 852
On various intraclass correlation reliability coefficients.
Bartko, J. J. (1976) · Psychological Bulletin · doi:10.1037//0033-2909.83.5.762 · Cited by 825
Growth Modeling Using Random Coefficient Models: Model Building, Testing, and Illustrations
Bliese, P. D., & Ployhart, R. E. (2002) · Organizational Research Methods · doi:10.1177/109442802237116 · Cited by 786
Herold, D. M., Fedor, D. B., Caldwell, S., & Liu, Y. (2008) · Journal of Applied Psychology · doi:10.1037/0021-9010.93.2.346 · Cited by 720
Transformational leadership and team innovation: Integrating team climate principles.
Eisenbeiss, S. A., van Knippenberg, D., & Boerner, S. (2008) · Journal of Applied Psychology · doi:10.1037/a0012716 · Cited by 712
Shamir, B., Zakay, E., Breinin, E., & Popper, M. (1998) · Academy of Management Journal · doi:10.2307/257080 · Cited by 669
Group Size, ICC Values, and Group-Level Correlations: A Simulation
Bliese, P. D. (1998) · Organizational Research Methods · doi:10.1177/109442819814001 · Cited by 636 · cited 7×
“Whereas LeBreton and Senter (2008) have suggested that an ICC(1)=.05 represents a small to medium effect (p. 838), Bliese (1998) has simulated conditions where only 1% of the variance is attributed to group membership (ICC(1)=.01) and, still, strong group-level relationships were detected that were…”
BackgroundMethodInfluentialLiao, H., & Chuang, A. (2007) · Journal of Applied Psychology · doi:10.1037/0021-9010.92.4.1006 · Cited by 583
Rubin, R. S., Munz, D. C., & Bommer, W. H. (2005) · Academy of Management Journal · doi:10.5465/amj.2005.18803926 · Cited by 576
Liao, H., & Rupp, D. E. (2005) · Journal of Applied Psychology · doi:10.1037/0021-9010.90.2.242 · Cited by 563
Workplace emotions: The role of supervision and leadership.
Bono, J. E., Foldes, H. J., Vinson, G., & Muros, J. P. (2007) · Journal of Applied Psychology · doi:10.1037/0021-9010.92.5.1357 · Cited by 559
Jung, D. (., Wu, A., & Chow, C. W. (2008) · The Leadership Quarterly · doi:10.1016/j.leaqua.2008.07.007 · Cited by 556
Klein, K. J., Conn, A. B., Smith, D. B., & Sorra, J. S. (2001) · Journal of Applied Psychology · doi:10.1037//0021-9010.86.1.3 · Cited by 516
A disagreement about within-group agreement: disentangling issues of consistency versus consensus
Kozlowski, S. W., & Hattrup, K. (1992) · Journal of Applied Psychology · doi:10.1037/0021-9010.77.2.161 · Cited by 514
Leadership and levels of analysis: A state-of-the-science review
Yammarino, F. J., Dionne, S. D., Uk Chun, J., & Dansereau, F. (2005) · The Leadership Quarterly · doi:10.1016/j.leaqua.2005.09.002 · Cited by 501
WALUMBWA, F. O., AVOLIO, B. J., & ZHU, W. (2008) · Personnel Psychology · doi:10.1111/j.1744-6570.2008.00131.x · Cited by 482
Bliese, P. D., & Hanges, P. J. (2004) · Organizational Research Methods · doi:10.1177/1094428104268542 · Cited by 412
Hooper, D. T., & Martin, R. (2008) · The Leadership Quarterly · doi:10.1016/j.leaqua.2007.12.002 · Cited by 338
AN EXPLORATION OF CLIMATES FOR TECHNICAL UPDATING AND PERFORMANCE
KOZLOWSKI, S. W. J., & HULTS, B. M. (1987) · Personnel Psychology · doi:10.1111/j.1744-6570.1987.tb00614.x · Cited by 316
A FRAMEWORK FOR CONDUCTING MULTI-LEVEL CONSTRUCT VALIDATION
Chen, G., Mathieu, J. E., & Bliese, P. D. (2004) · Research in Multi Level Issues · doi:10.1016/s1475-9144(04)03013-9 · Cited by 310
Interrater Agreement Reconsidered: An Alternative to the rwg Indices
Brown, R. D., & Hauenstein, N. M. A. (2005) · Organizational Research Methods · doi:10.1177/1094428105275376 · Cited by 297
Berson, Y., & Avolio, B. J. (2004) · The Leadership Quarterly · doi:10.1016/j.leaqua.2004.07.003 · Cited by 291
Rowold, J., & Heinitz, K. (2007) · The Leadership Quarterly · doi:10.1016/j.leaqua.2007.01.003 · Cited by 277
Lebreton, J. M., Burgess, J. R. D., Kaiser, R. B., Atchley, E. K., & James, L. R. (2003) · Organizational Research Methods · doi:10.1177/1094428102239427 · Cited by 254
Data analytic methods for the analysis of multilevel questions
Castro, S. L. (2002) · The Leadership Quarterly · doi:10.1016/s1048-9843(01)00105-9 · Cited by 250
Leadership, Collective Personality, and Performance.
Hofmann, D. A., & Jones, L. M. (2005) · Journal of Applied Psychology · doi:10.1037/0021-9010.90.3.509 · Cited by 234
Dunlap, W. P., Burke, M. J., & Smith-Crowe, K. (2003) · Journal of Applied Psychology · doi:10.1037/0021-9010.88.2.356 · Cited by 219
Setting the stage for effective leadership: Antecedents of transformational leadership behavior
Bommer, W. H., Rubin, R. S., & Baldwin, T. T. (2004) · The Leadership Quarterly · doi:10.1016/j.leaqua.2004.02.012 · Cited by 184
Benchmarking multilevel methods in leadership
Bliese, P. D., Halverson, R. R., & Schriesheim, C. A. (2002) · The Leadership Quarterly · doi:10.1016/s1048-9843(01)00101-1 · Cited by 175
Shin, S. J., Morgeson, F. P., & Campion, M. A. (2006) · Journal of International Business Studies · doi:10.1057/palgrave.jibs.8400247 · Cited by 170
Statistical properties of the rWG(J) index of agreement.
Cohen, A., Doveh, E., & Eick, U. (2001) · Psychological Methods · doi:10.1037/1082-989x.6.3.297 · Cited by 166
Climate strength – How leaders form consensus
Luria, G. (2008) · The Leadership Quarterly · doi:10.1016/j.leaqua.2007.12.004 · Cited by 164
Bliese, P. D., & Halverson, R. R. (1996) · Journal of Applied Social Psychology · doi:10.1111/j.1559-1816.1996.tb02291.x · Cited by 159
Multi-level nature of and multi-level approaches to leadership
Yammarino, F. J., & Dansereau, F. (2008) · The Leadership Quarterly · doi:10.1016/j.leaqua.2008.01.001 · Cited by 145
Roberson, Q. M., Sturman, M. C., & Simons, T. L. (2007) · Organizational Research Methods · doi:10.1177/1094428106294746 · Cited by 145
Cole, M. S., & Bedeian, A. G. (2007) · The Leadership Quarterly · doi:10.1016/j.leaqua.2007.07.002 · Cited by 127
Self-citationInterrater reliability coefficients cannot be computed when only one stimulus is rated.
Schmidt, F. L., & Hunter, J. E. (1989) · Journal of Applied Psychology · doi:10.1037/0021-9010.74.2.368 · Cited by 126
Dyads in Organizational Research: Conceptual Issues and Multilevel Analyses
Gooty, J., & Yammarino, F. J. (2010) · Organizational Research Methods · doi:10.1177/1094428109358271 · Cited by 124
Kearney, E. (2008) · Journal of Occupational and Organizational Psychology · doi:10.1348/096317907x256717 · Cited by 123
Recent Issues Regarding rWG, rWG, rWG(J), and rWG(J)
LeBreton, J. M., James, L. R., & Lindell, M. K. (2004) · Organizational Research Methods · doi:10.1177/1094428104272181 · Cited by 118
Schriesheim, C. A., Cogliser, C. C., & Neider, L. L. (1995) · The Leadership Quarterly · doi:10.1016/1048-9843(95)90031-4 · Cited by 99
Dickson, M. W., Resick, C. J., & Hanges, P. J. (2006) · The Leadership Quarterly · doi:10.1016/j.leaqua.2006.07.005 · Cited by 96
Cole, M. S., Bedeian, A. G., & Bruch, H. (2011) · The Leadership Quarterly · doi:10.1016/j.leaqua.2011.02.012 · Cited by 87
Self-citationTesting Agreement for Multi-Item Scales With the Indices r WG(J) and AD M(J)
Cohen, A., Doveh, E., & Nahum-Shani, I. B. (2007) · Organizational Research Methods · doi:10.1177/1094428107300365 · Cited by 80
Avoiding the “Fallacy of the Wrong Level”: A Within and between Analysis (WABA) Approach
Dansereau, F., Cho, J., & Yammarino, F. J. (2006) · Group & Organization Management · doi:10.1177/1059601106291131 · Cited by 70
A LONGITUDINAL EXAMINATION OF CONCOMITANT CHANGES IN TEAM LEADERSHIP AND CUSTOMER SATISFACTION
Walker, A. G., Smither, J. W., & Waldman, D. A. (2008) · Personnel Psychology · doi:10.1111/j.1744-6570.2008.00122.x · Cited by 68
On the Use of Likert-Type Scales in Multilevel Data: Influence on Aggregate Variables
Beal, D. J., & Dawson, J. (2007) · Organizational Research Methods · doi:10.1177/1094428106295492 · Cited by 56
Is more discussion about levels of analysis really necessary?When is such discussion sufficient?
Dansereau, F., & Yammarino, F. J. (2006) · The Leadership Quarterly · doi:10.1016/j.leaqua.2006.07.002 · Cited by 38
Statistical properties of the r-sub(WG(J) ) index of agreement.
Cohen, A., Doveh, E., & Eick, U. (2001) · Psychological Methods · doi:10.1037//1082-989x.6.3.297 · Cited by 32
Assessing Within-Group Agreement
Lüdtke, O., & Robitzsch, A. (2008) · Organizational Research Methods · doi:10.1177/1094428108317406 · Cited by 29
Testing for Between-Group Differences in Within-Group Interrater Agreement
Pasisz, D. J., & Hurtz, G. M. (2008) · Organizational Research Methods · doi:10.1177/1094428108319128 · Cited by 17
Estimation of Quantiles for the Sampling Distribution of the rWG within Group Agreement Index
Charnes, J. M., & Schriesheim, C. A. (1995) · Educational and Psychological Measurement · doi:10.1177/0013164495055003007 · Cited by 12
Cited by
Hu, J., Erdogan, B., Jiang, K., Bauer, T. N., & Liu, S. (2018) · Journal of Applied Psychology · doi:10.1037/apl0000277 · Cited by 432
The Role of Inclusive Leadership in Supporting an Inclusive Climate in Diverse Public Sector Teams
Ashikali, T., Groeneveld, S., & Kuipers, B. (2021) · Review of Public Personnel Administration · doi:10.1177/0734371x19899722 · Cited by 326
Assessing Teamwork Skills for Assurance of Learning Using CATME Team Tools
Loughry, M. L., Ohland, M. W., & Woehr, D. J. (2014) · Journal of Marketing Education · doi:10.1177/0273475313499023 · Cited by 271
Shared Authentic Leadership and New Venture Performance
Hmieleski, K. M., Cole, M. S., & Baron, R. A. (2012) · Journal of Management · doi:10.1177/0149206311415419 · Cited by 265
Self-citationFarahnak, L. R., Ehrhart, M. G., Torres, E. M., & Aarons, G. A. (2020) · Journal of Leadership & Organizational Studies · doi:10.1177/1548051818824529 · Cited by 261
Metadata
- Authors
- Biemann, T., Cole, M. S., & Voelpel, S.
- Venue
- The Leadership Quarterly
- Publisher
- Elsevier BV
- Published
- 2012-02
- Volume
- 23
- Issue
- 1
- Pages
- 66-80
- ISSN
- 1048-9843 (ISSN-L)
- OpenAlex
- W2081730522
- Cited by
- 361
- References
- 68
- FWCI
- 10.13
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.