Integration and generalization of kappas for multiple raters
The DOI matches its Crossref record; 3 of the 3 references it cites were found in a registry or an index.
Abstract
J. A. Cohen's kappa (1960) for measuring agreement between 2 raters, using a nominal scale, has been extended for use with multiple raters by R. J. Light (1971) and J. L. Fleiss (1971). In the present article, these indices are analyzed and reformulated in terms of agreement statistics based on all
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“with interrater reliability measured using Cohen’s Kappa (κ = 0.63–0.82), indicating moderate to substantial agreement (Conger, 1980).”
From Rethinking pre-training: cognitive load implications for learners with varying prior knowledge (2025) · cites this paper as Conger, 1980
In the citing paper
“Furthermore, (25) provides an alternative proof that Conger's and Fleiss' quadratically weighted kappas converge to each other as the number of raters, 𝑅, tends to infinity (Conger, 1980;Gwet, 2014).”
From Quadratically Weighted Agreement Coefficients: Interpretations and Connections (2026) · cites this paper as Conger, 1980
In the citing paper
“The average pairwise Pearson, Spearman, Kendall's τ b , ϕ, and MCC over the R 2 judge pairs are all equal, and coincide with κ F when all judges share the same marginal MET rate.”
From Agreement Metrics for LLM-as-Judge Evaluation: What to Report and Why (2026) · cites this paper as Conger (1980)
In the citing paper
“A generalization of Cohen’s kappa (i.e., Light’s Kappa [59]) was calculated to assess agreement between judges across their allocation of all items to domains.”
From Discriminant content validity of a theoretical domains framework questionnaire for use in implementation research (2014) · cites this paper as [59]
Citations
Cites
A Coefficient of Agreement for Nominal Scales
Cohen, J. (1960) · Educational and Psychological Measurement · doi:10.1177/001316446002000104 · Cited by 42,398
Measuring nominal scale agreement among many raters.
Fleiss, J. L. (1971) · Psychological Bulletin · doi:10.1037/h0031619 · Cited by 8,733
Measures of response agreement for qualitative data: Some generalizations and alternatives.
Light, R. J. (1971) · Psychological Bulletin · doi:10.1037/h0031643 · Cited by 747
Cited by
Items 401–500 of 555
Peer review: Explicit criteria and training can help
Delcomyn, F. (1991) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00065778 · Cited by 3
Wright, K. C., Harnden, P., Moss, S., Berney, D. M., & Melia, J. (2011) · Journal of Clinical Pathology · doi:10.1136/jcp.2010.086330 · Cited by 3
RRep: A composite index to assess and test rater precision
Vanacore, A., & Pellegrino, M. S. (2018) · Quality and Reliability Engineering International · doi:10.1002/qre.2334 · Cited by 4
Horak, H., Jermakovs, K., & Haamer, R. E. (2022) · IEEE Access · doi:10.1109/access.2022.3160474 · Cited by 3
LİSE FİZİK DERS KİTAPLARININ ETKİNLİĞİNİN İNCELENMESİ: ÖĞRETMEN ADAYLARININ DEĞERLENDİRMESİ
Balaban, Z., & Ogan‐Bekiroglu, F. (2023) · Erciyes Akademi · doi:10.48070/erciyesakademi.1211874 · Cited by 3
Some indices of the reliability of peer review
Rosenthal, R. (1991) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00065948 · Cited by 3
Should the blinded lead the blinded?
Lock, S. (1991) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00065894 · Cited by 3
Chairman's action: The importance of executive decisions in peer review
Tyrer, P. J. (1991) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00065997 · Cited by 4
What to do about peer review: Is the cure worse than the disease?
Zentall, T. R. (1991) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00066012 · Cited by 3
Justice, efficiency and epistemology in the peer review of scientific manuscripts
Mahoney, M. J. (1991) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00065900 · Cited by 3
The Upper Attachment of the Uncinate Process and Anterior Ethmoidal Artery
Yaprak, F., Çoban, İ., Özer, M. A., Eraslan, C., & Gövsa, F. (2023) · Annals of Otology Rhinology & Laryngology · doi:10.1177/00034894231191311 · Cited by 2
“9 Variations in the attachment of the UP may affect the AEA, which enters the nasal cavity through the orbital medial wall (lamina papyracea).”
BackgroundDivergent Mental Models as a Trigger of Team Adaptation
Müller, R., & Antoni, C. H. (2023) · Zeitschrift für Arbeits- und Organisationspsychologie A&O · doi:10.1026/0932-4089/a000421 · Cited by 2
Sauger, C., Quinquis, J., Kellner, K., Berthelin, C., Lepoittevin, M., Elie, N., & Dubroca, L. (2026) · Animals · doi:10.3390/ani16030519 · Cited by 2
“The results from both readings of the calibration exercise led to the quantification of a reading disagreement percentage between the three agents for each slide through the use of two inter-rater reliability indexes: the percentage agreement between readers [43], and Fleiss’s kappa [44,45].”
A Study of Chance-Corrected Agreement Coefficients for the Measurement of Multi-Rater Consistency
Xie, Z., Gadepalli, C., & Cheetham, B. M. G. (2018) · International Journal of Simulation Systems Science & Technology · doi:10.5013/ijssst.a.19.02.10 · Cited by 2
“It was originally proposed [5] for categorical rating by two raters, and was generalised by Hubert [6] and Conger [7] to a multi-rater version that may be expressed [2] as follows:”
MethodBerry, K. J., Johnston, J. E., & Mielke, P. W. (2018) · Springer eBooks · doi:10.1007/978-3-319-98926-6_5 · Cited by 2
How do persons with apraxia of speech deal with morphological stress in Spanish? A preliminary study
Baqué, L. (2019) · Clinical Linguistics & Phonetics · doi:10.1080/02699206.2019.1622155 · Cited by 2
“agreement by calculating Conger’s exact kappa (Conger, 1980; Fleiss, 1971) using the ‘irr’ package (Gamer, Lemon, Fellows, & Singh, 2015) in R (version 3.”
MethodAnalysis of the Weighted Kappa and Its Maximum with Markov Moves
Rapallo, F. (2022) · Psychometrika · doi:10.1007/s11336-022-09844-y · Cited by 2
“Also notice that the Conger’s κC can be defined in the general case of g-wise agreement, as in the original paper Conger (1980).”
InfluentialIORapp: An R tool for Inter-Observer Reliability Assessment of Time and Motion data
Guidi, S., Tanzini, M., & Westbrook, J. I. (2021) · European Conference on Cognitive Ergonomics · doi:10.1145/3452853.3452890 · Cited by 2
van Oest, R. D. (2022) · Psychometrika · doi:10.1007/s11336-022-09881-7 · Cited by 2
“…raters together (instead of rater pairs) but is equivalent to a weighted version of Conger’s kappa, where expected weighted agreement∑C c=1 ∑C c̃=1 wc,c̃ pcqc̃ becomes the corresponding average across all rater pairs, with rater-specific category proportions pc and qc̃ (Conger 1980; Warrens 2012b).”
BackgroundMethodComputed Tomography Based Evaluation of the Anterior Group of the Paranasal Sinuses
Yaprak, F., Çoban, İ., Sarıoğlu, O., Özer, M. A., & Gövsa, F. (2023) · European Journal of Therapeutics · doi:10.58600/eurjther.20232903-1618.z · Cited by 2
Analysis of Effect of Answering Reflection Prompts in a Computer Organization Class
Resch, C. L., Stapleton, P., Rheault, B., Wu, A., & Gardner‐McCune, C. (2024) · ASEE Annual Conference & Exposition · doi:10.18260/1-2--40428 · Cited by 2
Concordance of performance metrics among US trauma centers caring for injured children
Sathya, C., Burd, R. S., Nance, M. L., Karanicolas, P. J., Wales, P. W., Scales, D. C., Xiong, W. G., & Nathens, A. B. (2015) · The Journal of Trauma: Injury, Infection, and Critical Care · doi:10.1097/ta.0000000000000678 · Cited by 2
“Furthermore, a heat map was generated(21) to visually depict the consistency of center-level performance across quality metrics, andweightedJ statisticswere used to test for agreement.(22,23) This work was approved by the Research Ethics Board at Sunnybrook Health Sciences Center, Toronto, Ontario, Canada.”
MethodPeer review is not enough: Editors must work with librarians to ensure access to research
Fuller, S. (1991) · Behavioral and Brain Sciences · doi:10.1017/s0140525x0006581x · Cited by 2
Unreliable peer review: Causes and cures of human misery
Colman, A. M. (1991) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00065742 · Cited by 3
Correlations in search of a theory: Interpreting the predictive validity of security of attachment
Feinman, S. (1984) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00026571 · Cited by 2
The process of peer review: Unanswered questions
Nelson, L. D. (1991) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00065924 · Cited by 2
Ogasawara, H. (2020) · Journal of Classification · doi:10.1007/s00357-020-09366-1 · Cited by 2
Statistical evaluation of binary measurement systems
Erdmann, T. P. (2012) · UvA-DARE (University of Amsterdam) · Cited by 3
Security of infantile attachment: The person–situation debate revisited
Mills, C. J., & Eiserer, L. A. (1984) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00026650 · Cited by 2
Stranger in a strange situation: Comments by a comparative psychologist
Denenberg, V. H. (1984) · Behavioral and Brain Sciences · doi:10.1017/s0140525x0002656x · Cited by 2
Varma, A. O. (1986) · Journal Of Clinical Periodontology · doi:10.1111/j.1600-051x.1986.tb01481.x · Cited by 2
Measurement Systems Analysis, Attribute
Borror, C. M. (2007) · Wiley eBooks · doi:10.1002/9780470061572.eqr302 · Cited by 2
Standardizing diagnostic assessment of Autism Spectrum Disorder : assets and challenges
Zander, E. (2015) · Figshare · Cited by 3
Evaluating scholarly works: How many reviewers? How much anonymity?
Cone, J. D. (1991) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00065754 · Cited by 2
Consensus and the reliability of peer-review evaluations
Cole, S. (1991) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00065730 · Cited by 4
Freedman, D. G. (1984) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00026583 · Cited by 2
The Alignment of Knowledge Strategies
Denford, J. S. (2009) · QSpace (Queen's University Library) · Cited by 4
Diabetic retinopathy screening and treatment
Nissen, T. P. H. (2023) · VBN Forskningsportal (Aalborg Universitet) · doi:10.54337/aau588589522 · Cited by 2
Veytia, D., Airoldi, L., Claudet, J., Cooley, S. R., Magnan, A., Barclay, V. M., Neill, S. P., Sumaila, U. R., Thébaud, O., Voolstra, C. R., et al. (2024) · Research Square · doi:10.21203/rs.3.pex-2324/v1 · Cited by 2
Post-Ureteroscopic Lesion Scale: limitaciones en su aplicabilidad clínica
Polo, R., Nebot, À. C., Caballero-Romeu, J. P., Caballero, P., Galán-Llopis, J.-A., Soria, F., de la Cruz, J. E., & Tuells, J. (2023) · Actas Urológicas Españolas · doi:10.1016/j.acuro.2023.08.004 · Cited by 1
Jordan, A.-K., Esterbauer, E., & Schumacher, K. (2024) · Musiktherapeutische Umschau · doi:10.13109/muum.2024.45.3.273 · Cited by 1
Bayesian Gower agreement for categorical data
Hughes, J. (2025) · Scientific Reports · doi:10.1038/s41598-025-90873-9 · Cited by 1
Naito, Y., Kawahara, A., MAKINO, R., Tokumitsu, T., Nagayama, D., Sadashima, E., Nakamura, K., Nishizaki, R., Nagatomo, T., Fujino, Y., et al. (2025) · Acta Cytologica · doi:10.1159/000544997 · Cited by 1
“The exact kappa, a modi fi cation of Fleiss ’ kappa, was used to measure interobserver agreement for categorical data among two or more raters [15].”
MRI major and ancillary features of LI-RADS for diagnosis of pediatric malignant liver tumors
Choi, S. J., Shin, J. I., Kim, P. H., Park, H., Kim, S., Jung, A. Y., Cho, Y. A., Lee, J. S., Song, I. H., Gang, S., et al. (2025) · European Radiology · doi:10.1007/s00330-025-11775-y · Cited by 1
“Inter-reader agreement regarding each MRI LI-RADS feature and category was assessed by calculating Fleiss ’ Kappa coef fi cient to measure the strength of agreement between multiple readers [18, 19].”
Nelson, K. P., & Zhou, T. (2025) · Statistics in Medicine · doi:10.1002/sim.70141 · Cited by 1
“…the intra-class correlation coefficient defined by Bloch and Kraemer [17], and modified Cohen’s kappa-like statistics by Light [18] and Conger [19]; however, in practice, these measures are vulnerable to the same issuesasCohen’skappaandsimilarlyareunabletoaccommodate patient characteristics…”
OSCE Dyadic Subscales: Validity and Reliability Testing with Nursing and Social Work Practitioners
Tyler, T. R., Franklin, A. E., & Gilbert, G. E. (2025) · Studies in Clinical Social Work Transforming Practice Education and Research · doi:10.1080/28376811.2025.2605024 · Cited by 1
Seiters, J. C., Molitor, V., Morandi, A., Zieschang, T., Dichter, M., Haastert, B., Erdmann, M., Hoffmann, F., & Palm, R. (2025) · BMJ Open · doi:10.1136/bmjopen-2025-112357 · Cited by 1
Artificially intelligent opinion polling
Cerina, R., & Duch, R. M. (2026) · Royal Society Open Science · doi:10.1098/rsos.251150 · Cited by 2
Li, C., Cui, J., Liang, X., Zhang, L., Guo, X., Xiang, Y., & Liu, A. (2026) · Frontiers in Pharmacology · doi:10.3389/fphar.2026.1808669 · Cited by 1
“Fleiss’ kappa ( κ ) was calculated for each primary category, with interpretation based on Landis and Koch’s criteria: ≤ 0.20 (poor), 0.21–0.40 (fair), 0.41–0.60 (moderate), 0.61–0.80 (substantial), and 0.81–1.00 (almost perfect) (Conger, 1980).”
InfluentialGladhill, G., Schwei, R. J., Griffin, M., Glinert, R., Haleem, A., Hess, J., Keenan, T. D., McBride, J. A., Redwood, R., & Pulia, M. S. (2026) · Journal of Emergency Medicine · doi:10.1016/j.jemermed.2026.03.023 · Cited by 1
Kriukova, K. K., Li, L., Staba, R. J., Boswell, M., Asifriyaz, T., Thomas, R., Gugger, J. J., Shamas, M., Diaz-Arrastia, R. R., Blanco, M. B., et al. (2026) · Epilepsia · doi:10.1002/epi.70235 · Cited by 1
Deman, F., Rasic, D., Broeckx, G., Siozopoulou, V., Lænkholm, A., Salgado, R., Torres‐Saavedra, P. A., Ács, B., & Dendooven, A. (2026) · Laboratory Investigation · doi:10.1016/j.labinv.2026.106150 · Cited by 1
Déformation thoracique dite « en boîte d’allumettes » : reproductibilité inter-examinateurs
Nisand, M. (2011) · Kinésithérapie la Revue · doi:10.1016/s1779-0123(11)75155-5 · Cited by 1
Caveats on the use of evolutionary concepts
Klopfer, P. H. (1984) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00026625 · Cited by 1
Infantile attachment: The forest and the trees
Kovach, J. K., & Kovach, M. E. (1984) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00026637 · Cited by 1
On the Equivalence of Multirater Kappas Based on 2-Agreement and 3-Agreement with Binary Scores
Warrens, M. J. (2012) · ISRN Probability and Statistics · doi:10.5402/2012/656390 · Cited by 1
Maguire, C. (2015) · ScholarWorks (Central Washington University) · Cited by 1
Hastings, J., Batchelor, C. R., & Okada, M. (2013) · Cited by 1
Bayesian testing of agreement criteria under order constraints
Ganjali, M., Moradzadeh, N., & Baghfalaki, T. (2016) · Journal of the Korean Statistical Society · doi:10.1016/j.jkss.2016.06.004 · Cited by 1
A comparison of multi-way similarity coefficients for binary sequences
Warrens, M. J. (2013) · Leiden Repository (Leiden University) · Cited by 1
Haselbach, L. M., Hougham, R. J., & Langfitt, Q. (2015) · Cited by 1
Berry, K. J., Johnston, J. E., & Mielke, P. W. (2018) · Springer eBooks · doi:10.1007/978-3-319-98926-6_4 · Cited by 1
Measurement of Rater Consistency by Chance-Corrected Agreement Coefficients
Xie, Z., Gadepalli, C., & Cheetham, B. M. G. (2018) · UKSim International Conference on Computer Modelling and Simulation (UKSim) · doi:10.1109/uksim.2018.00015 · Cited by 2
“The generalisation to more than two raters is due to Conger [7] and Hubert [6].”
BackgroundMethodRausch, J. L., & Pascoe, A. (2021) · Carroll Collected (John Carroll University) · Cited by 4
Leske, S., Weir, B., Adam, G., & Kõlves, K. (2023) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph21010052 · Cited by 1
“We compared inter-rater agreement with several measures, including percent agreement, Brennan and Prediger [35], Cohen [36,37]/Conger’s [38] Kappa, Scott [39]/Fleiss’ Pi [40], Gwet’s AC [41,42], and Krippendorff’s Alpha [43–45].”
MethodFeatures of Сohen’s kappa coefficient as a measure of expert opinion agreement
Ybytayeva, G., Mamyrbayev, O. J., Khairova, N., Mukhsina, K., & Жумажанов, Б. (2023) · Bulletin of the National Engineering Academy of the Republic of Kazakhstan · doi:10.47533/2023.1606-146x.26 · Cited by 1
Wagemann, O., Brendel, M., Franzmeier, N., Nübling, G., Gnörich, J. S., Zaganjori, M., Prix, C., Stockbauer, A., Wlasich, E., Loosli, S. V., et al. (2025) · Frontiers in Neuroscience · doi:10.3389/fnins.2024.1505999 · Cited by 1
Girianelli, V. R., Tomazelli, J. G., Nogueira, M. C., Corrêa, C. S. L., de Souza, E. O., Gabrielli, L., Aquino, E. M. L., Guerra, M. R., Stavola, B. L. D., dos‐Santos‐Silva, I., et al. (2019) · Revista Brasileira de Epidemiologia · doi:10.1590/1980-549720190045 · Cited by 1
Negation and Speculation Target Identification
Zou, B., Zhou, G., & Zhu, Q. (2014) · Communications in computer and information science · doi:10.1007/978-3-662-45924-9_4 · Cited by 1
Popping, R. (2005) · Wiley eBooks · doi:10.1002/0471667196.ess1793.pub2 · Cited by 1
Examining Shared Mental Models of Air Traffic Controllers: What do they entail and what is shared?
Justen, P. C., Pal, J., Zijlstra, F., van Doorn, R. R. A., de Waard, D., Axelsson, A., Berglund, M., Peters, B., & Weikert, C. (2010) · Research Publications (Maastricht University) · Cited by 1
The evolution of ethological attachment theory
Hay, D. F. (1984) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00026613 · Cited by 2
How to think about the evolution of behavioral development
Ghiselin, M. T. (1984) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00026595 · Cited by 1
On inferring evolutionary adaptation
Rajecki, D. W. (1984) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00026674 · Cited by 1
Wasserman, G. S. (1991) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00066000 · Cited by 1
Novel methods of benthic habitat assessment in designated waterbodies around Ireland
O’Carroll, J. P. J. (2017) · Arrow@dit (Dublin Institute of Technology) · doi:10.13025/17377 · Cited by 1
Shared knowledge in complex teams: an investigation of the shared mental model construct
Sander, P. C. (2016) · Research Publications (Maastricht University) · doi:10.26481/dis.20160331ps · Cited by 3
Popping, R. (2004) · Wiley eBooks · doi:10.1002/0471667196.ess1793 · Cited by 1
Phenotyping of chronic lung allograft dysfunction in the ScanCLAD study
Magnusson, J. M., Raivio, P. M., Leuckfeld, I., Svahn, J. C. E., Holmberg, E. C., Lund, T. K., & Dellgren, G. (2026) · ERJ Open Research · doi:10.1183/23120541.00084-2026 · Cited by 1
Quadratically Weighted Agreement Coefficients: Interpretations and Connections
van Oest, R., & Moss, J. (2026) · Psychometrika · doi:10.1017/psy.2026.10085
MATRIX KAPPA: A PROPOSAL FOR A CARD SORT STATISTIC FOR IS SURVEY INSTRUMENT DEVELOPMENT
Denford, J. S. (2010) · International Conference on Information Systems
Characteristics of Valid Biographical Items
Stricker, L. J., Alderton, D. L., & Rock, D. A. (2011) · Military Psychology · doi:10.1037/h0094761
A matrix approach to data base exploration: Analysis of classifier results
Goin, J. E., & Fritz, S. L. (1983) · Pattern Recognition · doi:10.1016/0031-3203(83)90028-6
Reification and “statification” in attachment theory and research
Masters, J. C. (1984) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00026649
Studying the security of infant-adult attachment: A reprise
Lamb, M. E., Gardner, W. P., Charnov, E. L., Thompson, R. A., & Estes, D. (1984) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00026698
Bonding behaviours, behavioural binds, and biological bases
Salzen, E. A. (1984) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00026686
Brown, R. T. (1984) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00026534
Rausch, J. L. (2015) · The Qualitative Report · doi:10.46743/2160-3715/2012.1816
Les orientations de la recherche en éducation dans les constituantes de l’Université du Québec
Papillon, S., Rousseau, R., Tremblay, Y., & Potvin, P. (2009) · Revue des sciences de l éducation · doi:10.7202/900551ar
“La concordance des jugements portés par des juges ou le degré d'accord entre plusieurs juges a fait l'objet de nombreux écrits, dont les principaux sont ceux de Cohen (1960), Maxwell et Pilliner (1968), Fleiss (1971), Light (1971), Fleiss et Cohen (1973), Conger (1980) et Rae (1984).”
BackgroundMandeville, P. B. (2011) · Dialnet (Universidad de la Rioja)
Supporting Elementary Education In-Service Teachers' Proficiency in Planning STEM-Centric Lessons
Bowers, S. W. (2015) · VTechWorks (Virginia Tech)
Does group discussion contribute reliability of complex judgments?
Cohen, P. R. (1991) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00065729
Now that we know how low the reliability is, what shall we do?
Salzinger, K. (1991) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00065961
Is there an alternative to peer review?
Greene, R. (1991) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00065845
Replication, reliability and peer review: A case study
Gorman, M. E. (1991) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00065833
Different rates of agreement on acceptance and rejection: A statistical artifact?
Demorest, M. E. (1991) · Behavioral and Brain Sciences · doi:10.1017/s0140525x0006578x
Permutation P-values for Inter-rater Agreement Measures
Um, Y. (2015) · Journal of the Korea Society of Computer and Information · doi:10.9708/jksci.2015.20.12.169
Principles for the Validation of a Composite Score
Falissard, B. (2011) · Chapman & Hall/CRC eBooks · doi:10.1201/b11190-10
Design and analysis of biomedical studies
Hansen, M. K. (2015)
Metadata
- Authors
- Conger, A. J.
- Affiliations
- Purdue University West Lafayette (US)
- Corresponding author
- Conger, Anthony J.
- Venue
- Psychological Bulletin
- Published
- 1980-09
- Volume
- 88
- Issue
- 2
- Pages
- 322-328
- ISSN
- 0033-2909 (ISSN-L)
- 1939-1455
- OpenAlex
- W1978217317
- Cited by
- 545
- References
- 3
- Citations by year
- 2026: 20 · 2025: 24 · 2024: 35 · 2023: 24 · 2022: 23 · 2021: 23 · 2020: 25 · 2019: 25 · 2018: 22 · 2017: 16 · 2016: 18 · 2015: 27 · 2014: 29 · 2013: 15 · 2012: 15
- FWCI
- 7.19
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Psychological Bulletin (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Multidisciplinary Science and Engineering Research
- Keywords
- Cohen's kappa, Fleiss' kappa, interobserver agreement, multiple raters, nominal scale
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.