Cited by 515Reliability and Agreement in MeasurementPsychometric Methodologies and TestingDelphi Technique in Research

The DOI matches its Crossref record; 32 of the 33 references it cites were found in a registry or an index.

A disagreement about within-group agreement: disentangling issues of consistency versus consensus

Kozlowski, Steve W., Hattrup, Keith·1992·Journal of Applied Psychology

Open this paper in CoChat — free account

Abstract

James, Demaree, and Wolf (1984) developed an indes, r WG , for assessing within-group agreement appropriate when only a single target is rated. The assessment of interperceiver agreement in such situations is of particular relevance to the composition model for climate. Schmidt and Hunter (1989) have criticized the conceptual foundation of r WG because it is not consistent with the classical model of reliability. They proposed an alternative approach, the use of the rating standard deviation (SD X ), the standard error of the rating mean (SE M ), and the associated confidence intervals for SE M to index interrater agreement. This comment argues that the critique of r WG dit not clearly distinguish the concepts of interrater consensus (i.e., agreement) and interrater consistency (i.e., reliability)

Integrity signals

Registry

Crossref · 10.1037/0021-9010.77.2.161

Cites

32 of 32 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

A Coefficient of Agreement for Nominal Scales
Cohen, J. (1960) · Educational and Psychological Measurement · doi:10.1177/001316446002000104 · Cited by 42,398
Intraclass correlations: Uses in assessing rater reliability.
Shrout, P. E., & Fleiss, J. L. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.2.420 · Cited by 24,532
The Social Psychology of Organizations.
Hackman, J. R., Katz, D., & Kahn, R. L. (1979) · Administrative Science Quarterly · doi:10.2307/2989929 · Cited by 12,243
Weighted kappa: nominal scale agreement provision for scaled disagreement or partial credit
Cohen, J. (1968) · Psychological Bulletin · doi:10.1037/h0026256 · Cited by 8,736
Estimating within-group interrater reliability with and without response bias
James, L. R., Demaree, R. G., & Wolf, G. (1984) · Journal of Applied Psychology · doi:10.1037/0021-9010.69.1.85 · Cited by 4,774
TRANSFER OF TRAINING: A REVIEW AND DIRECTIONS FOR FUTURE RESEARCH
Baldwin, T. T., & Ford, J. K. (1988) · Personnel Psychology · doi:10.1111/j.1744-6570.1988.tb00632.x · Cited by 3,662
International Review of Industrial and Organizational Psychology
Cooper, C. L., & Robertson, I. T. (2003) · Wiley eBooks · doi:10.1002/0470013311.oth1 · Cited by 2,787
Aggregation bias in estimates of perceptual agreement.
James, L. R. (1982) · Journal of Applied Psychology · doi:10.1037/0021-9010.67.2.219 · Cited by 2,139
Personality, affect, and behavior in groups
George, J. M. (1990) · Journal of Applied Psychology · doi:10.1037/0021-9010.75.2.107 · Cited by 1,372
Employee and customer perceptions of service in banks: Replication and extension.
Schneider, B., & Bowen, D. E. (1985) · Journal of Applied Psychology · doi:10.1037/0021-9010.70.3.423 · Cited by 1,344

Flagged first, then by citations

Show all 33 references

1 of the 33 references couldn’t be matched to a known paper.

Sorted by citations

Items 301–400 of 515

Facing Climate Change: What Drives Internal Migration Decisions in the Karst Rocky Regions of Southwest China
Shi, G., LYU, Q., Shangguan, Z., & Jiang, T. (2019) · Sustainability · doi:10.3390/su11072142 · Cited by 19
Team Social Media Usage and Team Creativity: The Role of Team Knowledge Sharing and Team-Member Exchange
Wang, H., Xiao, Y.-T., Su, X., & Li, X.-Q. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.755208 · Cited by 22
Non-Suicidal Self-Injury in Women Offenders: Motivations, Emotions, and Precipitating Events
Power, J., Brown, S. L., & Usher, A. M. (2013) · International Journal of Forensic Mental Health · doi:10.1080/14999013.2013.832442 · Cited by 19
The importance of a homogeneous transformational leadership climate for organizational performance
de Jong, S. B., & Bruch, H. (2011) · UEA Digital Repository (University of East Anglia) · Cited by 19
Newcomer Socialization research: The Importance and Application of Multilevel Theory and Communication
Manata, B., Miller, V. D., DeAngelis, B. N., & Paik, J. E. (2016) · Annals of the International Communication Association · doi:10.1080/23808985.2015.11735264 · Cited by 17
Thriving Organizational Sustainability through Innovation: Incivility Climate and Teamwork
Yang, J. (2016) · Sustainability · doi:10.3390/su8090860 · Cited by 24
Impact of active constructive leadership and passive corrective leadership on collective organizational commitment
Wei, F., Lee, J., & Kwan, H. K. (2016) · Leadership & Organization Development Journal · doi:10.1108/lodj-08-2014-0150 · Cited by 17
Organizational culture, leadership and performance in context : trust and rationality in organizations
Koene, B. A. S. (1996) · doi:10.26481/dis.19960913bk · Cited by 17
Empirical Test Guidelines for Content Validity: Wash, Rinse, and Repeat until Clean
Schmitz, K., & Storey, V. C. (2020) · Communications of the Association for Information Systems · doi:10.17705/1cais.04736 · Cited by 16
Claims about the reliability of student evaluations of instruction: The ecological fallacy rides again
Morley, D. D. (2012) · Studies In Educational Evaluation · doi:10.1016/j.stueduc.2012.01.001 · Cited by 18
Collaboration in Virtual Worlds
Nah, F. F., Schiller, S. Z., Mennecke, B. E., Siau, K. L., Eschenbrenner, B., & Sattayanuwat, P. (2017) · Journal of Database Management · doi:10.4018/jdm.2017100104 · Cited by 16
Employment relationships as drivers of firm flexibility and learning
Cabrales, Á. L., Cabrera, R. V., & González, J. L. G. (2011) · Personnel Review · doi:10.1108/00483481111154478 · Cited by 16
Nonresponse in Employee Attitude Surveys: A Group-Level Analysis
Fauth, T., Hattrup, K., Mueller, K., & Roberts, B. (2012) · Journal of Business and Psychology · doi:10.1007/s10869-012-9260-y · Cited by 21
Self-citation
Building a better mouse trap: Item characteristics associated with rating discrepancies in 360-degree feedback.
Kaiser, R. B., & Craig, S. B. (2005) · Consulting psychology journal · doi:10.1037/1065-9293.57.4.235 · Cited by 16
The contribution of consensus within staff and client groups as well as concordance between staff and clients to treatment engagement
Melnick, G., Wexler, H. K., Chaple, M., & Banks, S. M. (2006) · Journal of Substance Abuse Treatment · doi:10.1016/j.jsat.2006.05.002 · Cited by 16
Relational demography and officer occupational attitudes: The influence of workgroup context
Ingram, J. R., & Terrill, W. (2014) · Journal of Criminal Justice · doi:10.1016/j.jcrimjus.2014.04.001 · Cited by 15
Differential Effects of Survey-Guided Feedback
Born, D. H., & Mathieu, J. E. (1996) · Group & Organization Management · doi:10.1177/1059601196214002 · Cited by 15
The role of work group context and information seeking in newcomer socialization: An interactionist perspective
Zou, W., Qing, T., & Liu, J. (2015) · Journal of Management & Organization · doi:10.1017/jmo.2014.73 · Cited by 15
Do Raters Agree More on Observable Items?
Roch, S. G., Paquin, A. R., & Littlejohn, T. W. (2009) · Human Performance · doi:10.1080/08959280903248344 · Cited by 15
Leadership in a Crisis: A Social Network Perspective on Leader Brokerage Strategy, Intra-Organizational Communication Patterns, and Business Recovery
Li, N., Zheng, X., Ni, D., Kirkman, B. L., Zhang, M., Xu, M., & Liu, C. (2024) · Journal of Management · doi:10.1177/01492063241237227 · Cited by 20
BARS and Those Mysterious, Missing Middle Anchors
Hauenstein, N. M. A., Brown, R. D., & Sinclair, A. L. (2010) · Journal of Business and Psychology · doi:10.1007/s10869-010-9180-7 · Cited by 14
A multi-level study on employee voice: Evidence from a chain of retail stores
Liang, J., & Tang, J. (2010) · Frontiers of Business Research in China · doi:10.1007/s11782-010-0110-3 · Cited by 15
A field examination of the moderating role of group trust in group efficacy formation
Lee, D., Stajković, A. D., & Sergent, K. (2016) · Journal of Occupational and Organizational Psychology · doi:10.1111/joop.12161 · Cited by 16
The Nature of Organizational Psychology
Kozlowski, S. W. J. (2012) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780199928309.013.0001 · Cited by 15
Self-citation
The Contextualization and Multilevel Issues in Research of Organizational Psychology
ZHANG, Z.-X. (2010) · Acta Psychologica Sinica · doi:10.3724/sp.j.1041.2010.00010 · Cited by 14
Work Design
Morgeson, F. P., Garza, A. S., & Campion, M. A. (2012) · Wiley eBooks · doi:10.1002/9781118133880.hop212020 · Cited by 15
Promoting shared leadership in Lean Six Sigma project teams: toward a three-way interaction model
Wu, Q., Zhou, Q., & Cormican, K. T. (2023) · International Journal of Lean Six Sigma · doi:10.1108/ijlss-03-2023-0048 · Cited by 16
The power of many: analyzing the impact of shared leadership on project teams
Zhou, Q., Wu, Q., Sun, Y., & Cormican, K. T. (2024) · International Journal of Managing Projects in Business · doi:10.1108/ijmpb-05-2024-0132 · Cited by 20
Skill underutilization and collective turnover in a professional service firm
Mitchell, M. E., & Zatzick, C. D. (2015) · Journal of Management Development · doi:10.1108/jmd-09-2013-0112 · Cited by 13
School Process and Teacher Job Satisfaction at Alternative Schools: A Multilevel Study Using SASS 2007–08 Data
Xia, J., Izumi, M., & Gao, X. (2015) · Leadership and Policy in Schools · doi:10.1080/15700763.2014.997935 · Cited by 13
High-investment HR values and firm performance among local firms and U.S. MNCs’ subsidiaries in South Asia: a comparative study
Colakoglu, S. S., Allen, M., Miah, M. K., & Bird, A. W. (2015) · The International Journal of Human Resource Management · doi:10.1080/09585192.2015.1075573 · Cited by 13
Analyzing Consensus to Personality Judgments: A Variance Components Approach
Shrout, P. E. (1993) · Journal of Personality · doi:10.1111/j.1467-6494.1993.tb00790.x · Cited by 13
Implicit Morality Theories: Employees’ Beliefs About the Malleability of Moral Character Shape Their Workplace Behaviors
Feng, Z., Keng, F. T., Hu, L., & Savani, K. (2022) · Journal of Business Ethics · doi:10.1007/s10551-022-05113-1 · Cited by 17
Middle power plays: How and when Mach middle managers use downward abuse and upward guanxi to gain and maintain power
Keng, F. T., Feng, Z., Yam, K. C., Chen, X.-P., & Hu, L. (2024) · Journal of Organizational Behavior · doi:10.1002/job.2794 · Cited by 13
Teamgeist im Orchester: Das Zusammenspiel zwischen Flow und Führung
Boerner, S., & von Streit, C. F. (2006) · Musicae Scientiae · doi:10.1177/102986490601000205 · Cited by 12
Measuring interprofessional education and collaborative practice competencies: a content validity study of the Jefferson Teamwork Observation Guide®
Sicks, S., Umland, E. M., Koch, A. J., Hass, R. W., & Tenpa, J. (2021) · Journal of Interprofessional Care · doi:10.1080/13561820.2021.1972946 · Cited by 12
An across-organization analysis of the implementation of advanced manufacturing technologies
Hattrup, K., & Kozlowski, S. W. J. (1993) · The Journal of High Technology Management Research · doi:10.1016/1047-8310(93)90003-x · Cited by 12
Self-citation
Service climate in New Zealand English language centres
Walker, J. (2007) · Journal of Educational Administration · doi:10.1108/09578230710747839 · Cited by 12
Absence Culture
Iverson, R. D., Buttigieg, D. M., & Maguire, C. (2004) · Relations industrielles · doi:10.7202/007496ar · Cited by 12
Intragroup Functional Diversity and Intergroup Relations in American and Chinese Workgroups
Zhang, Y., & Tsui, A. S. (2013) · Journal of Cross-Cultural Psychology · doi:10.1177/0022022112471897 · Cited by 12
A Multi-Level Examination of Organizational Context on Adult Probation Officer Attitudes Toward Evidence-Based Practice
Viglione, J. (2016) · International Journal of Offender Therapy and Comparative Criminology · doi:10.1177/0306624x16675630 · Cited by 14
Estimation of Quantiles for the Sampling Distribution of the rWG within Group Agreement Index
Charnes, J. M., & Schriesheim, C. A. (1995) · Educational and Psychological Measurement · doi:10.1177/0013164495055003007 · Cited by 12
High-wearable EEG-based transducer for engagement detection in pediatric rehabilitation
Apicella, A., Arpaïa, P., Giugliano, S., Mastrati, G., & Moccaldi, N. (2021) · Brain-Computer Interfaces · doi:10.1080/2326263x.2021.2015149 · Cited by 11
Leader trait affective presence and safety behaviors: The role of work engagement
Chen, Y., Qiang, W., & Li, F. (2023) · Safety Science · doi:10.1016/j.ssci.2023.106339 · Cited by 15
Team Collaboration in Virtual Worlds: The Role of Task Complexity
Nah, F. F., Schiller, S. Z., Mennecke, B. E., Siau, K. L., & Sattayanuwat, P. (2011) · Journal of the Association for Information Systems · Cited by 11
A review of agreement measure as a subset of association measure between raters
Adejumo, A. O., Heumann, C., & Toutenburg, H. (2004) · Econstor (Econstor) · doi:10.5282/ubm/epub.1755 · Cited by 11
A Multilevel, Emergent Journey to Unpack Team Process Dynamics
Kozlowski, S. W. J. (2024) · Small Group Research · doi:10.1177/10464964241281347 · Cited by 15
Self-citation
Organizational Climate and Service Climate in Tourism and Hospitality: A Review
Manning, M. L., Shacklock, A. H., Bell, N., & Manning, R. L. (2012) · Griffith Research Online · Cited by 10
A new approach to analyzing the Achilles' heel of multisource feedback programs: Can we really trust ratings of leaders at the group level of analysis?
Markham, S. E., Smith, J. W., Markham, I. S., & Braekkan, K. F. (2014) · The Leadership Quarterly · doi:10.1016/j.leaqua.2014.10.003 · Cited by 10
Taking the Good with the Bad–—Customer Type as a Segmentation Criterion and Differential Influencer of Sales Performance
Shannahan, R. J., Shannahan, K. L. J., Bush, A. J., & Moncrief, W. C. (2016) · The Journal of Marketing Theory and Practice · doi:10.1080/10696679.2016.1170535 · Cited by 11
The relationship between team explanatory style and team success
Carron, A. V., Shapcott, K. M., & Martin, L. J. (2013) · International Journal of Sport and Exercise Psychology · doi:10.1080/1612197x.2014.853898 · Cited by 10
Unfolding the phenomenon of interrater agreement: a multicomponent approach for in-depth examination was proposed
Slaug, B., Schilling, O. K., Helle, T., Iwarsson, S., Carlsson, G., & Brandt, Å. (2012) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2012.02.016 · Cited by 10
Work Outcomes and Job Design for Contract Versus Permanent Information Systems Professionals on Software Development Teams
Ang, S., & Slaughter, S. A. (2007) · Information Systems Outsourcing · doi:10.1007/978-3-540-34877-1_15 · Cited by 11
Client consensus on beliefs about abstinence: Effects on substance abuse treatment outcomes
Melnick, G., Wexler, H. K., & Cleland, C. M. (2008) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2007.08.014 · Cited by 10
Understanding digital information production in virtual communities from the perspective of Actor-Network Theory
Yuan, B. (2023) · Journal of Hospitality and Tourism Management · doi:10.1016/j.jhtm.2023.08.003 · Cited by 9
A Multiple Criteria Decision Analysis Framework for Dispersed Group Decision-Making Contexts
Carneiro, J., Martinho, D., Alves, P., Conceição, L., Marreiros, G., & Nováis, P. (2020) · Applied Sciences · doi:10.3390/app10134614 · Cited by 9
Agreement and consensus in a Q-mode research design: an empirical comparison of measures, and an application
Dijkstra, L., & van Eijnatten, F. M. (2009) · Quality & Quantity · doi:10.1007/s11135-009-9249-4 · Cited by 9
Evaluating Implementation of the Transparency and Openness Promotion Guidelines: Reliability of Instruments to Assess Journal Policies, Procedures, and Practices
Kianersi, S., Grant, S., Naaman, K., Henschel, B., Mellor, D. T., Apte, S., Deyoe, J. E., Eze, P., Huo, C., Lavender, B. L., et al. (2023) · Advances in Methods and Practices in Psychological Science · doi:10.1177/25152459221149735 · Cited by 11
Ein ambidextres Führungsklima – Erfolgsfaktor in der neuen Arbeitswelt
Szlang, J. J. K., & Bruch, H. (2020) · Gruppe Interaktion Organisation Zeitschrift für Angewandte Organisationspsychologie (GIO) · doi:10.1007/s11612-020-00511-3 · Cited by 8
Mediation of scenario planning on the reflection-performance relationship in new product development teams
Açıkgöz, A., Latham, G. P., & Açikgöz, F. (2020) · Journal of Business and Industrial Marketing · doi:10.1108/jbim-07-2019-0321 · Cited by 8
A mindfulness perspective on the link between abusive supervision climate and team conflict
Qu, Y. E., Todorova, G., Dasborough, M. T., & Shi, Y. (2023) · International Journal of Conflict Management · doi:10.1108/ijcma-03-2023-0035 · Cited by 9
The Writer in the Reader
Dwyer, B., & Larson, L. C. (2013) · Advances in educational technologies and instructional design book series · doi:10.4018/978-1-4666-4341-3.ch012 · Cited by 8
Something Fishy: Exploring Change, Job Engagement and Work Environment in the Norwegian Directorate of Fisheries
Lofquist, E. A., Isaksen, S. G., & Dahl, T. J. (2018) · Journal of Change Management · doi:10.1080/14697017.2018.1466823 · Cited by 8
Does organizational citizenship behavior add value to human interaction with e-services?
Liao, C. H. (2015) · Online Information Review · doi:10.1108/oir-01-2015-0005 · Cited by 8
Citizenship in Context: Investigating the Effects of Work Group Climate on Organizational Citizenship Perceptions and Behavior.
Dekas, K. H. (2010) · Deep Blue (University of Michigan) · Cited by 8
Non-suicidal self-injury in federally sentenced women : prevalence, nature, motivations, and pathways
Power, J. (2011) · doi:10.22215/etd/2011-09499 · Cited by 8
How and when supervisor bottom‐line mentality affects employees' voluntary workplace green behaviors: A goal‐shielding perspective
Zhang, J.-W., Akhtar, M., Zhang, Y.-J., Tang, J., & Yang, Q. (2024) · Corporate Social Responsibility and Environmental Management · doi:10.1002/csr.2877 · Cited by 8
Influence of Digital Communication Configuration in Virtual Teams: A Faultline Perspective
Wong, S. I., Zhang, L., Černe, M., & Moe, N. B. (2024) · Journal of Management Information Systems · doi:10.1080/07421222.2024.2415776 · Cited by 7
CEO Humility and Its Relationship with Middle Manager Behaviors and Performance: Examining the CEO-Middle Manager Interface
Ou, Y. (2011) · Cited by 7
Comparison ofe-rater® Automated Essay Scoring Model Calibration Methods Based on Distributional Targets
Zhang, M., Williamson, D. M., Breyer, F. J., & Trapani, C. S. (2012) · International Journal of Testing · doi:10.1080/15305058.2011.645973 · Cited by 7
A Discriminant Validity Study of Aggregate-Level Constructs and Measures of Local Union Formalization, Centralization, and Innovation
Mellor, S., & Mathieu, J. E. (1999) · The Journal of Psychology · doi:10.1080/00223989909599772 · Cited by 7
Opportunities and Challenges in Healthcare Information Systems Research: Caring for Patients with Chronic Conditions
Zhang, M., Gable, G. G., & Tate, M. (2019) · Communications of the Association for Information Systems · doi:10.17705/1cais.04501 · Cited by 7
A multilevel model of abusive supervision climate
Khan, S. N., Kiazad, K., Sendjaya, S., & Cooper, B. K. (2021) · Personnel Review · doi:10.1108/pr-02-2021-0109 · Cited by 7
Rater Agreement across Multiple Data Collection Media
Kacmar, K. M., & Hochwarter, W. A. (1996) · The Journal of Social Psychology · doi:10.1080/00224545.1996.9714028 · Cited by 7
A multi-level model examining the effects of unit-level culture on abusive supervision
Siegel, J. L. (2011) · UA Campus Repository (The University of Arizona) · Cited by 7
Humane Orientation : A cross-cultural study in 26 countries
Schlösser, O., & Justus Liebig University Giessen (2006) · University writing server of the Justus Liebig University Giessen (Justus-Liebig-Universität Gießen) · doi:10.22029/jlupub-7819 · Cited by 7
The a d coefficient as a descriptive measure of the within‐group agreement of ratings
Kreuzpointner, L., Simon, P., & Theis, F. J. (2009) · British Journal of Mathematical and Statistical Psychology · doi:10.1348/000711009x465647 · Cited by 6
The (Mis)Use ofSDas Age Diversity Measure: Introducing the MeanSD
Meulenaere, K. D., Biemann, T., & Boone, C. (2021) · Work Aging and Retirement · doi:10.1093/workar/waab042 · Cited by 6
Perceived Mutual Understanding (PMU)
Burtscher, M. J., & Oostlander, J. (2016) · European Journal of Psychological Assessment · doi:10.1027/1015-5759/a000360 · Cited by 6
Arbeitseinstellungen, Wohlbefinden und Leistung
Schmidt, K., Neubach, B., & Heuer, H. (2007) · Zeitschrift für Arbeits- und Organisationspsychologie A&O · doi:10.1026/0932-4089.51.1.16 · Cited by 6
Gruppenstimmung (group mood) als Erfolgsbedingung transformationaler Führung
Boerner, S., & von Streit, C. F. (2006) · Zeitschrift für Arbeits- und Organisationspsychologie A&O · doi:10.1026/0932-4089.50.1.3 · Cited by 6
Millennials sex differences on Snapchat perceived privacy
Rauzzino, A., & Correa, J. C. (2017) · Suma Psicológica · doi:10.1016/j.sumpsi.2017.08.002 · Cited by 6
Innovation through individual-focused transformational leadership in China: mediation by leader identification and cross-level moderation by team-focused leadership and innovation climate
Wang, X., Park, H., & Kim, H. J. (2025) · Frontiers in Psychology · doi:10.3389/fpsyg.2025.1615515 · Cited by 7
Swiftly judging whom to bring on board: How person perception (accurate or not) influences selection of prospective team members
Derfler-Rozin, R. R., Isaakyan, S., & Park, H. (2022) · Organizational Behavior and Human Decision Processes · doi:10.1016/j.obhdp.2022.104206 · Cited by 8
Die Bedeutung der LehrerInnengerechtigkeit: Klimaerleben oder persönliches Erleben?
Peter, F., & Dalbert, C. (2013) · Springer eBooks · doi:10.1007/978-3-531-93128-9_2 · Cited by 5
[Retracted] Coding ATC Incident Data Using HFACS: Intercoder Consensus
Wang, L., Wang, Y., Yang, X., Cheng, K., Yang, H., Zhu, B., Fan, C., & Ji, X. (2011) · Journal of Quality and Reliability Engineering · doi:10.1155/2011/379129 · Cited by 5
Retracted
Cross-level effects of buyer-supplier collaboration and competition on individual gains and individual skill utilization
Backmann, J., Hoegl, M., & Parboteeah, K. P. (2016) · Journal of Engineering and Technology Management · doi:10.1016/j.jengtecman.2016.06.001 · Cited by 5
Data aggregation adequacy testing in multilevel research: A critical literature review and preliminary solutions to key issues
Haiteng, Z. H. U. (2020) · Advances in Psychological Science · doi:10.3724/sp.j.1042.2020.01392 · Cited by 5
An Analysis of the Relationships Between the Perceived Organizational Climate and Professional Burnout in Libraries and Computing Centers in West Virginia Public Higher Education Institutions
Miller, A. R. (2003) · Cited by 5
Multilevel influences on work attitudes: Organisational and individual predictors of pay satisfaction
Griffin, M. A. (1997) · Australian Psychologist · doi:10.1080/00050069708257380 · Cited by 5
Handling Observations with Low Interrater Agreement Values
Liu, G., Amini, M. M., Babakus, E., & Stafford, M. R. (2011) · Journal of Analytical Sciences Methods and Instrumentation · doi:10.4236/jasmi.2011.12002 · Cited by 5
The Manager as Change Agent : Communication Channels, Timing of Information, and Attitude Change
Costa, L. A., de Matos, J. A., & Cunha, M. P. E. (2003) · International Studies of Management and Organization · doi:10.1080/00208825.2003.11043689 · Cited by 5
Perspectiva sistémica y multinivel del análisis organizacional: una aproximación conceptual y metodológica
Aritzeta, A., Ayestarán, S., & Balluerka, N. (2003) · International Journal of Social Psychology Revista de Psicología Social · doi:10.1174/021347403322470846 · Cited by 5
Justification on Knowledge Management Strategies: A New Perspective on Knowledge Creating Process.
Choi, B., & Lee, H. (2001) · Journal of the Association for Information Systems · Cited by 5
What Makes Experts Reliable?Preprint
Marquardt, K. L., Pemstein, D., Seim, B., & Wang, Y.-T. (2018) · SSRN Electronic Journal · doi:10.2139/ssrn.3190946 · Cited by 5
The Dual Effect of Transformational Leadership on Individual- and Team-Level Creativity
Akinlade, E. (2014) · Figshare · Cited by 5
Improving the utility of multisource feedback for medical consultants in a tertiary hospital: a study of the psychometric properties of a survey tool
Corbett, H. M., Pearson, K., Karimi, L., & Lim, W. K. (2018) · Australian Health Review · doi:10.1071/ah17219 · Cited by 4
The Construct of Job Insecurity at Multiple Levels: Implications for Its Conceptualization and Theory Development
Sora, B., Höge, T., Caballer, A., & Silla, J. M. P. (2023) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph20043052 · Cited by 12
Multilevel influences of team social media usage on creativity: the moderating effect of knowledge leadership
Wang, H., Li, X., & Xie, M. (2023) · Behaviour and Information Technology · doi:10.1080/0144929x.2023.2242522 · Cited by 4
Impact of spiritual leadership on unit performance [RETRACTED]
Fry, L. W., Hannah, S. T., Noel, M., & Walumbwa, F. O. (2011) · Lincoln (University of Nebraska) · Cited by 4

Authors: Kozlowski, S. W., & Hattrup, K.
Affiliations: Michigan State University (US)
ORCID: Kozlowski, Steve W.
Venue: Journal of Applied Psychology
Publisher: American Psychological Association (APA)
Published: 1992-04
Volume: 77
Issue: 2
Pages: 161-167
ISSN: 0021-9010 (ISSN-L)
1939-1854
DOI: 10.1037/0021-9010.77.2.161
OpenAlex: W2069294908
Cited by: 514
References: 33
Citations by year: 2026: 8 · 2025: 6 · 2024: 9 · 2023: 12 · 2022: 5 · 2021: 23 · 2020: 11 · 2019: 15 · 2018: 16 · 2017: 15 · 2016: 19 · 2015: 27 · 2014: 24 · 2013: 17 · 2012: 26
FWCI: 12.78
Type: journal article
Language: en
Open access: closed
Locations: Journal of Applied Psychology (publishedVersion)
Topics: Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement
Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
Social Sciences › Sociology and Political Science › Delphi Technique in Research
Keywords: interobserver agreement, confidence intervals, classical test theory, reliability statistics

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

A disagreement about within-group agreement: disentangling issues of consistency versus consensus

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks