The DOI matches its Crossref record; 10 of the 10 references it cites were found in a registry or an index.
K-Alpha Calculator—Krippendorff’s Alpha Calculator: A user-friendly tool for computing Krippendorff’s Alpha inter-rater reliability coefficient
Open accessAbstract
Krippendorff's Alpha is a measure for assessing inter-rater reliability, facilitating triangulated evaluations among multiple raters. Indeed, Krippendorff's Alpha is key in validating the dependability of human assessments, thereby reinforcing the robustness of human-based choices in contexts where interpretive variability could otherwise undermine the research outcomes. Despite its versatility across various data types, the procedure for computing this coefficient might limit its applicability for researchers unfamiliar with specialised statistical software. Addressing this issue, this paper introduces the "K-Alpha Calculator"-Krippendorff's Alpha Calculator-a freely accessible, user-friendly, web application available at https://www.k-alpha.org designed for easy computation of Krippendorff's Alpha. By offering a web interface without any software dependency, the K-Alpha Calculator seeks to promote the broader integration of such a reliability metric in research processes. We envisage that the tool's contribution could support researchers in enhancing methodological rigour, fostering conclusions are rooted in triangulated interpretations and assessments. The K-Alpha Calculator is both a computational tool and an educational resource, encouraging more researchers to include reliability measures in their studies.•Development of the K-Alpha Calculator.The study centres on the introduction of a web-based application: the K-Alpha Calculator. The tool is developed to simplify the calculation of Krippendorff's Alpha without any required software dependency.•User-friendly interface for diverse data types.The K-Alpha Calculator is characterised by its accessibility and user-friendliness, designed to simplify the procedures for computing the Krippendorff's Alpha. Its utility extends to various data types, which aligns with the intrinsic versatility of this coefficient.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Inter-coder reliability was 0.81 based on Krippendorff’s Alpha [45,46], indicating a good level of agreement among coders.”
From Help-Seeking Situations Related to Visual Interactions on Mobile Platforms and Recommended Designs for Blind and Visually Impaired Users (2024) · cites this paper as [46]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A Coefficient of Agreement for Nominal Scales
Cohen, J. (1960) · Educational and Psychological Measurement · doi:10.1177/001316446002000104 · Cited by 42,398
Content Analysis: An Introduction to Its Methodology
Krippendorff, K. (2019) · doi:10.4135/9781071878781 · Cited by 15,448
Measuring nominal scale agreement among many raters.
Fleiss, J. L. (1971) · Psychological Bulletin · doi:10.1037/h0031619 · Cited by 8,733
Answering the Call for a Standard Reliability Measure for Coding Data
Hayes, A. F., & Krippendorff, K. (2007) · Communication Methods and Measures · doi:10.1080/19312450709336664 · Cited by 4,225
Intercoder Reliability in Qualitative Research: Debates and Practical Guidelines
O’Connor, C., & Joffe, H. (2020) · International Journal of Qualitative Methods · doi:10.1177/1609406919899220 · Cited by 3,871
Answers to 20 questions about interrater reliability and interrater agreement
LeBreton, J. M., & Senter, J. L. (2008) · Organizational Research Methods · doi:10.1177/1094428106296642 · Cited by 3,608
Zapf, A., Castell, S., Morawietz, L., & Karch, A. (2016) · BMC Medical Research Methodology · doi:10.1186/s12874-016-0200-9 · Cited by 377
Zhang, Y. E., Liu, G., Zhang, L., Xu, S., & Cheung, M. W. ‐. (2020) · Journal of Management · doi:10.1177/0149206320917195 · Cited by 212
Multiple‐Principal Demands and CEO Compliance in Emerging Market State‐Owned Enterprises
Apriliyanti, I. D., Dieleman, M. H., & Randøy, T. (2023) · Journal of Management Studies · doi:10.1111/joms.12977 · Cited by 36
KANOM: Stata module to estimate Krippendorff's alpha for nominal variables
Mitnik, P. A. (2016) · RePEc: Research Papers in Economics · Cited by 1
Flagged first, then by citations
Research works printed but not linked: 2
- Freelon, 2010, ReCal: intercoder reliability calculation as a web service, Int. J. Internet Sci., 5, 20
- Freelon, 2013, ReCal OIR: ordinal, interval, and ratio intercoder reliability as a web service, Int. J. Internet Sci., 8, 10
Cited by
Sorted by citations
Items 101–200 of 291
Change in social entrepreneurship: the imprinting perspective
Aktaş, Ü. N. (2025) · Social enterprise journal · doi:10.1108/sej-09-2024-0150 · Cited by 1
Shared Conceptual Ground in CSCL: Evidence from a Global Survey
Simon, S., Borchers, C., Ataş, A. H., Tajik, E., Çelik, B., Čarapina, M., Liu, Y., shahrokhian, B., Sankaranarayanan, S., Balzan, F., et al. (2025) · Computer-supported collaborative learning/The Computer-Supported Collaborative Learning Conference · doi:10.22318/cscl2025.987431 · Cited by 1
Probability of freshwater fish dispersal into and out of urban stormwater management ponds
Illyes, E., Boston, C. M., Turner, N. A., & Mandrak, N. E. (2025) · Biological Invasions · doi:10.1007/s10530-025-03639-9 · Cited by 2
“Krippendorff’s alpha coefficient is used to evaluate the agreement among the study participants and it ranges from 1 representing 100% agreement, through 0, showing complete absence of agreement, to negative numbers indicating systematic disagreement (Hayes and Krippendorff 2007; Marzi et al. 2024).”
Hwang, K., Kim, M., Kim, M. A., Park, C., Park, Y., Lee, C., Lim, J., & Oh, H. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.129548 · Cited by 1
Talay, Ö. (2025) · Health Communication · doi:10.1080/10410236.2025.2558135 · Cited by 1
“This result, calculated using the analytical tool recommended by Marzi et al. (2024), indicates acceptable reliability for exploratory research contexts and confirms the consistency of our coding procedures.”
Chachignon, P., Barbenchon, E. L., & Dany, L. (2025) · Current Psychology · doi:10.1007/s12144-025-08429-z · Cited by 1
La mission et la RSE : exploration dans le champ des sociétés à mission
Stimec, A. (2025) · Revue de l’organisation responsable · doi:10.54695/ror.203.0031 · Cited by 1
Applications of Cobots in manufacturing: A bibliometric analysis
Kumar, P., Gupta, S., Virmani, N., & Jagtap, S. (2025) · International Journal of Innovation Studies · doi:10.1016/j.ijis.2025.10.002 · Cited by 1
Phenotyping Functional Gait Disorder: An Exploratory Analysis of Existing Frameworks
Issak, S., Kanaan, R., Fini, N. A., Nielsen, G., & Williams, G. P. (2025) · Movement Disorders Clinical Practice · doi:10.1002/mdc3.70437 · Cited by 1
Asranna, A., Chatterjee, A., Duble, S. N., Mundlamuri, R. C., Kenchaiah, R., Mariyappa, N., Bhargava, G. K., Viswanathan, L. G., Sangeeth, T. A., James, N., et al. (2025) · Epileptic Disorders · doi:10.1002/epd2.70132 · Cited by 1
Framework for interdisciplinary research between engineering and behavioral sciences
Mora, V. F. P., & Oleas, J. C. M. (2025) · Discover Education · doi:10.1007/s44217-025-00916-6 · Cited by 1
“A measure of reliability that facilitates triangulated evaluations among judges and reinforces the robustness of decisions based on expert judgment through a statistical method [48, 63].”
InfluentialStrategic leadership in a non-WEIRD context: An integrative review of strategic leaders in China
Cao, L., & Özgen, S. (2025) · The Leadership Quarterly · doi:10.1016/j.leaqua.2025.101926 · Cited by 1
Kim, S. K., Kim, G. M., Cha, S., & Jung, M. (2025) · Journal of Korean Academy of Psychiatric and Mental Health Nursing · doi:10.12934/jkpmhn.2025.34.s1.23 · Cited by 2
Semantic prosody, categorisation and inter-rater reliability
Russnes, M. (2025) · Research Methods in Applied Linguistics · doi:10.1016/j.rmal.2025.100264 · Cited by 1
Large Language Models for API Classification: An Explorative Study
Morais, G., Lemelin, E., Adda, M., & Bork, D. (2025) · International Conference on Evaluation and Assessment in Software Engineering (EASE) · doi:10.1145/3756681.3756997 · Cited by 1
Fuentes, J. C. P., Cano-Ortíz, A., Ramírez, L. R., & Carmona, E. C. (2025) · Education Sciences · doi:10.3390/educsci16010056 · Cited by 1
“During the main coding phase, κ values ranged between ~0.72 and 0.85, with α coefficients within a similar range, indicating substantial to near-perfect reliability (Marzi et al., 2024).”
InfluentialSoodan, V., Chakraborty, S., Tailor, R. K., & Sharma, D. (2026) · F1000Research · doi:10.12688/f1000research.176435.1 · Cited by 1
“64 Before the rating exercise, the authors underwent a training session that included familiarizing them with the rating scheme, practising rating exercises, and discussion to clarify any ambiguities.”
Anticipating Paradoxical Tensions in the Technological Design of Smart Working Environments
Dornelles, J. D. A., & Frank, A. G. (2026) · IEEE Transactions on Engineering Management · doi:10.1109/tem.2026.3655547 · Cited by 1
“Analysis via the K-Alpha Calculator [37] yielded a Krippendorff’s Alpha of 0.881 (95% CI: [0.686, 1.000]) for tensions and 0.846 (95% CI: [0.626, 1.000]) for categories, indicating good reliability, despite some disagreement [34].”
InfluentialSzucs‐Farkas, Z., Perrin, L. G., M’Hamdi, B., Zaytoun, H., Christe, A., & Huber, A. T. (2026) · Investigative Radiology · doi:10.1097/rli.0000000000001266 · Cited by 1
Soodan, V., Chakraborty, S., Tailor, R. K., & Sharma, D. (2026) · F1000Research · doi:10.12688/f1000research.176435.2 · Cited by 1
Skating, Loss, and Solidarity: Examining Media Framing of Post-Tragedy Figure Skating News Coverage
Sadri, S. R., Sams, A. L., Cain, J. A., & Deng, F. (2026) · Communication & Sport · doi:10.1177/21674795261430906 · Cited by 1
Psychometric Evaluation of the Nonmainstream Pornography Use Extent Scale
Way, B. M., Grubbs, J. B., Jennings, T. L., & Kraus, S. W. (2026) · Sexual Health & Compulsivity · doi:10.1080/26929953.2026.2636889 · Cited by 1
“Using Krippendorff’s alpha (ordinal; Marzi et al., 2024), the NPUES demonstrated low reliability ( α k = .015; 95% CI [.0001, .022]).”
Recovering Revisions of Pull Requests with Altered History
Anonymous, Randrianaina, G. A., Xiao, T., Kamei, Y., & McIntosh, S. (2026) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3832783.3837417 · Cited by 1
“A build is still triggered by the rebase action, but the PR has no changes to label.”
Carrasco, F. G., Álvarez‐Aguado, I., & Córdova, V. V. (2026) · Siglo Cero: Revista Española sobre Discapacidad Intelectual · doi:10.14201/scero.32498 · Cited by 1
Khaririyatun, N., Hermanto, B., Purnomo, M., Chan, A., & Saptana, S. (2026) · Cogent Business & Management · doi:10.1080/23311975.2026.2652107 · Cited by 1
Lima, P. A. B., Domingues, A. M., de Medeiros, J. F., Furlan, M., & Groot, J. I. M. D. (2026) · Circular Economy · doi:10.55845/joce-2026-41316 · Cited by 1
Cuomo, M., Tortora, D., Loia, F., Zoccoli, P., & Celsi, L. R. (2026) · Journal of Knowledge Management · doi:10.1108/jkm-12-2025-1847
Murnan, A., Mahon, J. D., England, G., Pate, S., Thompson, S., Stewart, C., Doone, J., Daniels, L., Brown, J. L., Manchak, S. M., et al. (2026) · Journal of Human Trafficking · doi:10.1080/23322705.2026.2668373 · Cited by 1
Wang, B., Kabir, D., Clark, C. R., Choi, K. W., & Smoller, J. W. (2026) · JMIR Medical Informatics · doi:10.2196/89534 · Cited by 4
Cooperative interaction: A taxonomy of human-GenAI interaction patterns in tourism
Zhao, Y., Gao, Y., Wang, Y., Zhang, H., & Zheng, B. (2026) · Journal of Hospitality and Tourism Management · doi:10.1016/j.jhtm.2026.101472 · Cited by 1
Paisana, M., Couraceiro, P., Vasconcelos, A., & Cardoso, G. (2026) · Communication & Society · doi:10.15581/003.39.1.036 · Cited by 1
Semi-automatic pipeline for constructing HTR corpora from Ukrainian-language historical documents
Івасечко, А. В., & Lipianina-Honcharenko, K. (2026) · doi:10.55056/ceur-ws.org/vol-4159/paper12.pdf · Cited by 1
Sundaram, G., Venktesh, B., V, & Sundharakumar, K. B. (2026) · Information Processing & Management · doi:10.1016/j.ipm.2026.105031 · Cited by 1
Classification of videolaryngoscopy: a systematic review
Grün, C., Dankert, A., Wünsch, V. A., Engels, Y., Ewert, A., Sieg, H.-H., Plümer, L., Zöllner, C., & Petzoldt, M. (2026) · Anaesthesia · doi:10.1111/anae.70358 · Cited by 1
Tuomi, E., Lindström, R., Aro, T. I., Kykyri, V., & Laitila, A. A. (2026) · Nordisk välfärdsforskning | Nordic Welfare Research · doi:10.18261/nwr.11.3.5 · Cited by 1
Jensen, A. L., Bruun, K. S. G., Heimann, S., Langhorn, R., & Krogh, A. K. H. (2024) · The Veterinary Journal · doi:10.1016/j.tvjl.2024.106294 · Cited by 1
Astorga, G. H., Liang, X., & Ronen, G. M. (2025) · Epilepsy & Behavior · doi:10.1016/j.yebeh.2025.110517 · Cited by 1
Fleming‐Milici, F., Gershman, H., Agresta, H. O., McCann, M., & Harris, J. L. (2025) · Journal of the Academy of Nutrition and Dietetics · doi:10.1016/j.jand.2025.05.010 · Cited by 1
Feminist fan engagement and collective agency: A case study of Barbie in China
Yuan, X., Yang, R., & Zhang, Z. (2025) · International Journal of Cultural Studies · doi:10.1177/13678779251358881 · Cited by 1
“Inter-coder reliability was assessed using the K-Alpha Calculator for Krippendorff ’ s α , yielding a strong agreement score ( α = 0.847), indicating high reliability (Marzi et al., 2024).”
Influentialİlkokul Dördüncü Sınıf Öğrencilerinin Eleştirel Okuma ve Eleştirel Yazma İlişkisinin İncelenmesi
Okur, T., & Ateş, S. (2025) · Ana Dili Eğitimi Dergisi · doi:10.16916/aded.1658104 · Cited by 1
Ghadesi, A., Lamothe, M., & Li, H. (2024) · Empirical Software Engineering · doi:10.1007/s10664-024-10499-9
Competence building in software technologies: the case of the German automotive industry
Burr, W. (2024) · Technology Analysis and Strategic Management · doi:10.1080/09537325.2024.2392138
“As only one rater conducted the analysis, calculating numerical indices of inter-rater reliability, such as Krippenforff´s alpha, was not possible (Marzi, Balzano, and Marchiori 2024).”
Gleaton, E. C., & Catrambone, R. (2024) · Proceedings of the International Symposium on Human Factors and Ergonomics in Health Care · doi:10.1177/2327857924131037
“The inter-rater reliability was assessed and results indicated that there was high agreement between raters (α = 0.9; Krippendorff, 2004; Marzi et al., 2024).”
InfluentialBypassing the Nuances of Portuguese Covert Hate Speech Through Contextual Analysis
Ramos, G., Batista, F., Ribeiro, R., Fialho, P., Moro, S., Fonseca, A. J. F., Guerra, R., Carvalho, P., Marques, C. M. V. A., & Silva, C. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73503-5_20
Analisis Representasi dan Stereotip Gender dalam Film Animasi Anak
Rabani, H. D., Wibowo, K. A., & Rahmawan, D. (2024) · Jurnal Komunikasi Global · doi:10.24815/jkg.v13i2.39594
Millians, M. N., Kable, J. A., Coles, C. D., & Mattson, S. N. (2025) · Psychology in the Schools · doi:10.1002/pits.23428
“The examiners ’ scorings for each subtest were entered into the web ‐ based statistical program, K ‐ Alpha Calculator for analysis (Marzi, Balzano, and Marchiori 2024).”
Shared Conceptual Ground in CSCL: Evidence from a Global SurveyPreprint
Simon, S., Borchers, C., Ataş, A. H., Tajik, E., Çelik, B., Čarapina, M., Liu, Y., shahrokhian, B., Sankaranarayanan, S., Balzan, F., et al. (2025) · doi:10.35542/osf.io/pxrdf_v1
Business model innovation in a research ecosystem: the case of rare diseases
Sánchez, M. D. C., Sarmiento-Suárez, R., Hooft, R. W. W., Moreau, C., Biétrix, F., Jervelund, C., Mejer, M., Kreiner, K., Haddad, T., Santos, L. O. B. D. S., et al. (2025) · International journal of organizational analysis · doi:10.1108/ijoa-06-2024-4588
Diseño gráfico y creatividad en la identidad urbana
Díez, L. G., & Bengoa, M. T. (2025) · Street Art & Urban Creativity · doi:10.62161/sauc.v11.5710
Bruns, C. J., Huffman, D. R., Neff, P. D., Timm, K. M. F., & Roop, H. (2025) · Environmental Communication · doi:10.1080/17524032.2025.2512373
“Krippendorff’s Alpha, a reliability coefficient developed to assess agreement among coders, was used to statistically assess the reliability of the inter-coder agreement (Marzi et al., 2024).”
Incorporating Participant Perspectives to Refine Citizen Science Evaluation FrameworksPreprint
Janssen, J., Bogert, J. M., Gold, M. M., & de Vries, S. (2025) · doi:10.31219/osf.io/2x97q_v2
Evaluating Participant Outcomes in the Weather Monitoring Citizen Science Project “Delft Meet Regen”
Janssen, J., Bogert, J. M., Gold, M. M., & Vries, S. D. (2025) · Citizen Science Theory and Practice · doi:10.5334/cstp.757
Separated/Divorced Individuals’ Experiences with the Legal System in India: A Qualitative Inquiry
Maliakkal, D. J., & George, T. S. (2025) · Family Transitions · doi:10.1080/28375300.2025.2524666
GeneticPrism: Multifaceted Visualization of Citation-Based Scholarly Research Evolution
Sun, Y., Liu, Z., Luo, Y., Xia, L., & Shi, L. (2025) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2025.3589485
“To assess the consistency of topic modeling, we compared the highest-similaritytopicsforeachpaperacrossmultipleiterations usinganadjustedversionofKrippendorff’sAlpha[38].”
Can a Rule-Based Expert System Diagnose Nasal Obstruction from Nasoendoscopy Videos?
Navaratnam, A. V., Phoommanee, N., Acharya, V., Pendolino, A. L., Leung, T. S., & Andrews, P. J. (2025) · Facial Plastic Surgery · doi:10.1055/a-2656-6413
Rolbina, M., Caughey, S., & Rezaeian, M. (2025) · Studies in Higher Education · doi:10.1080/03075079.2025.2541227
“Krippendorff’s Alpha was employed to assess the inter-rater reliability of the rating scheme, indicating the extent of agreement among rates beyond chance (Marzi, Balzano, and Marchiori 2024).”
Davis, C. P., Villar, E. L. D., Pandey, S., Papadopoulou, E., Ngow, Z., & Wenner, C. A. (2025) · Frontiers in Education · doi:10.3389/feduc.2025.1624916
Malaeb, R., Maalouf, E., Mouallem, A., & Sabra, J. (2025) · ASEE Annual Conference & Exposition · doi:10.18260/1-2--56054
Wu, T., Chen, K., Kang, Y., Hoang, K. D., Fang, H., & Chen, C. (2025) · Journal of Evaluation in Clinical Practice · doi:10.1111/jep.70265
Salwa, M., Chowdhury, S. M., Rois, R., Uddin, M. K., Akhter, S., Nahar, K., Mullick, A. R., Mannan, M., Kundu, G. K., Fatema, K., et al. (2025) · BMJ Open · doi:10.1136/bmjopen-2025-105688
Flores, E., Peñafiel, R., & Molano, P. G. M. (2025) · Seminars in Medical Writing and Education · doi:10.56294/mw2025817
Rahimi, A. R., & Ghorbani, B. D. (2025) · Asian-Pacific Journal of Second and Foreign Language Education · doi:10.1186/s40862-025-00353-w · Cited by 5
“A Krippen-dorff’s alpha analysis was then conducted with 1000 bootstrap samples, and an alpha of 0.84 was obtained, exceeding the cutoff point of 0.80 (Marzi et al., 2024) which suggest a high degree of expert consensus on the final scale version as shown in Table 2.”
InfluentialPraat, D., & Schmaltz, G. (2025) · Ecology and Evolution · doi:10.1002/ece3.72379
Jacobson, L. A., Northrup, R. A., Carey, L. B., & Ruble, K. (2025) · Journal of the International Neuropsychological Society · doi:10.1017/s135561772510129x
“…raters (a neuropsy-chologist, special educator, and research assistant) separately for identification of neurocognitive domains falling below the identified point and for identification of corresponding supports from school support plans, using Krippendorf ’ s alpha (Marzi et al., 2024).”
Pink suits and big fins: gender-layered constraints in scuba diving
Bideci, C., & Bideci, M. (2025) · Tourism Geographies · doi:10.1080/14616688.2025.2580391
“…was applied to assess inter-rater reliability that enhances confidence in the researcher ′ s interpretative process and the robustness of the outcomes (Marzi et al., 2024), yielding a near-perfect agreement score of 0.913 ( p < 0.001). consistent with feminist geographic research, we acknowledge…”
InfluentialEvaluating Generative AI and Human Performance in Question-Answer Validation Tasks
Opaiţ, E.-E., Duca, A., & Olariu, M.-E. (2025) · Procedia Computer Science · doi:10.1016/j.procs.2025.09.522
Betschinske, D., & Peters, S. (2025) · IEEE International Automated Vehicle Validation Conference (IAVVC) · doi:10.1109/iavvc61942.2025.11219634
“To assess the reliability of the human reference ratings, the inter-rater agreement was quantified using Krippendorff’s Alpha [31, 32].”
Yuan, P., Lyu, F., Zhang, Y., Yang, C., Gao, Z., Wu, Z. W., Dong, J., Lyu, T., Zhao, W., Coatrieux, J.-L., et al. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.132022
Representing Public Opinion: Examining Ideological Extremity and Incivility in News Exemplifications
Koo, G. H., & Farquharson, K.-R. (2025) · Journalism Practice · doi:10.1080/17512786.2025.2586056
Mantaring, E. J., Wong, S. Y. S., McInerney, M., Thorpe, V., & Stewart, A. (2025) · Developmental Medicine & Child Neurology · doi:10.1111/dmcn.70110
“19 The results were analysed using the web-based statistical package Krippendorff's Alpha Calculator 20 (www. k-alpha. org/ ).”
Multimodal robotic storytelling integrating sound effects and background music
Steinhaeusser, S. C., Maier, S., & Lugrin, B. (2025) · Computers in Human Behavior Artificial Humans · doi:10.1016/j.chbah.2025.100248
Khan, A., Madihie, A., Haider, M., Haider, S., Khan, S. U., & Bahadar, A. (2025) · Journal of Disability Research · doi:10.57197/jdr-2025-0752
Potencial ChatGPT pri razvoju Slovarja sopomenk sodobne slovenščine
Holdt, Š. A., Gapsa, M., Gantar, P., & Kosem, I. (2025) · Contributions to Contemporary History · doi:10.51663/pnz.65.3.08
How social responses to online hate messages affect hatefulness
Yang, Y., Lange, R., Zhang, Y., & Walther, J. B. (2025) · New Media & Society · doi:10.1177/14614448251396951
Exploring layoff survivor experiences: a systematic literature review adopting the TCCM framework
George, T., & Maheshwari, B. (2026) · Global Knowledge, Memory and Communication · doi:10.1108/gkmc-11-2024-0752
Kalok, A. H. M., Atan, I. K., Sharip, S., & Shah, S. A. (2026) · Frontiers in Public Health · doi:10.3389/fpubh.2025.1714125
“Krippendorff’s Cu-α was calculated to determine intercoder reliability, and a value >0.8 indicates a satisfactory level of agreement between coders (22).”
InfluentialBroqvist, H. (2026) · Policy Studies Journal · doi:10.1111/psj.70093
“Following previous NPF studies (Smith-Walter 2018; Shanahan, Jones, et al. Shanahan, Jones, and McBeth 2018), intercoder reliability was assessed using both percentage agreement and Krippendorff's alpha (Krippendorff 2004; Marzi et al. 2024).”
Tian, P., & Yang, M. (2026) · SAGE Open · doi:10.1177/21582440251413957
“To quantify the consistency of coding outcomes across coders, intercoder reliability was evaluated using the K-Alpha Calculator (Marzi et al., 2024), a user-friendly tool specifically designed for computing Krippendorff’s Alpha, one of the most robust metrics for assessing inter-rater reliability,…”
Sánchez-Marañón, I., & Rodríguez-Virgili, J. (2026) · Doxa Comunicación Revista interdisciplinar de estudios de comunicación y ciencias sociales · doi:10.31921/doxacom.n42a2942
von Hessling, A., Sieber, C., Blatow, M., Berner, C., Lehnick, D., & Kellner‐Weldon, F. (2026) · Tomography · doi:10.3390/tomography12020013
“Agreement categories were defined as follows: α = 1— perfect agreement , α ≥ 0.80— satisfactory level of agreement , 0.67 ≤ α 0.79— moderate agreement , and α 0.67— poor agreement [17,18].”
InfluentialNews Framing of Assisted Death Through Argument Structures in Portugal and the United Kingdom
Costa, B. (2026) · Journalism and Media · doi:10.3390/journalmedia7010016
“The Krippendorff alpha was employed to assess the inter-coder reliability of the coding scheme (Marzi et al., 2024).”
Reza, M. N. H., hossain, S., Mamun, A. A., Gunasekaran, A., Malarvizhi, C. A., & Munikrishnan, U. T. (2026) · IEEE Transactions on Engineering Management · doi:10.1109/tem.2026.3657965
“To assess the consistency of their decisions, Krippendorff’s alpha [34] was calculated, yielding a value of 0.81, which indicates substantial agreement and high inter-rater reliability.”
InfluentialWho’s to Blame? Rethinking the Brittleness of Automated Web GUI Testing from a Pragmatic Perspective
Zhang, H., Yao, K., Ding, Z., Liao, L., & Shang, W. (2025) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase63991.2025.00251
“We then quantify and interpret the level of agreement achieved [57].”
Proposition of an Ontology Supporting Context Modeling in Operational Scenario Development
Disdier, A., Masson, D. H., Janković, M., & Boy, G. A. (2026) · Journal of Air Transportation · doi:10.2514/1.d0538
Esendemir, Y., & Kerman, K. (2026) · Journal of Veterinary Behavior · doi:10.1016/j.jveb.2026.02.001
Andrej Babiš as a Populist Leader: Media Framing of the Czech Presidential Elections 2023
Křovák, J., Konrádová, M., & Shavit, A. (2026) · Civilia. · doi:10.5507/civ.2025.016
Skoutelis, V. C., Moutsiou, R., Spanou, M., Dimitriadis, Z., Kalamvoki, E., Zouvelou, V., & Dinopoulos, A. (2026) · Children · doi:10.3390/children13030368
“Krippendorff’s Alpha values were interpreted using established thresholds: ≥ 0.80 satisfactory, 0.67–0.79 tentative, and 0.67 poor agreement [22].”
InfluentialAvram, E. M. G. (2026) · Journal of Chemical Education · doi:10.1021/acs.jchemed.5c00443
Frank, J. A., Gan, E. X., Hooper, W. B., Ott, I. M., & Iwasaki, A. (2026) · Scientific Reports · doi:10.1038/s41598-026-41410-9
Distinguishing abdominal wall denervation injury from normal anatomy via cross section imaging
Carvalho, A., Bennett, W. C., Woo, K. P., Tocci, N. X., Schmidt, E. M., Tastaldi, L. G., Miller, B. T., Beffa, L., Petro, C. C., Krpata, D. M., et al. (2026) · Hernia · doi:10.1007/s10029-026-03609-7
Yoo, T.-A., Yang, H.-S., Kim, M., & Chung, J. (2026) · Interactive Learning Environments · doi:10.1080/10494820.2026.2646275 · Cited by 1
Шурма, С. Г. (2026) · Frontiers in Communication · doi:10.3389/fcomm.2026.1652959
Development, Validation, and Reliability of a Teacher-Reported Speech and Language Disorders Scale
Kavakcı, M., Saatçioğlu, Ö., Uysal, H. T., Barmak, E., & ÇILDIR, B. (2026) · Language Speech and Hearing Services in Schools · doi:10.1044/2025_lshss-25-00094
Moktan, A., Banerjee, S., & Barua, A. (2026) · Research Square · doi:10.21203/rs.3.rs-9097430/v1
Rethman, K., Mansfield, C. J., Stephens, J. A., Lahman, C. C., Wayman, K. J., Quatman‐Yates, C. C., & Briggs, M. S. (2026) · Physiotherapy Research International · doi:10.1002/pri.70203
“Values of α ≥ 0.80 are considered acceptable agreement, values of α ≥ 0.67 to ≤ 0.79 are considered moderate agreement, and values α < 0.67 are considered poor agreement (Marzi et al. 2024).”
InfluentialDecoding of Typhoon Emergency Policy in Coastal Cities: Policy Text Mining From China
Zhongyou, W. (2026) · Risk Analysis · doi:10.1111/risa.70241
“After the Krippendorff’s alpha coefficient test, it was found that the coding reliability was 0.728, which met the standard of Krippendorff’s alpha ≥ 0.7 (Krippendorff 2018 ; Marzi et al. 2024 ; Zapf et al. 2016 ).”
InfluentialHalck, M. L., Skovlund, C. R., Jensen, K. S., Adji, A. V., Albert, M., Rasmussen, M. H., & Bjørnvad, C. R. (2026) · Veterinary Record · doi:10.1002/vetr.70550
Metzger, F. M., & Kumar, D. (2026) · Transportation Research Interdisciplinary Perspectives · doi:10.1016/j.trip.2026.101977
Yang, C.-L., Li, J., Chang, X., Li, J., Yatani, K., & Lee, Y. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790331 · Cited by 1
Metadata
- Authors
- Marzi, G., Balzano, M., & Marchiori, D.
- Affiliations
- IMT School for Advanced Studies Lucca (IT)
- Ca' Foscari University of Venice (IT)
- Corresponding author
- Marzi, Giacomo
- Venue
- MethodsX
- Publisher
- Elsevier BV
- Published
- 2024-06
- Volume
- 12
- Pages
- 102545
- ISSN
- 2215-0161 (ISSN-L)
- PMC
- PMC11636850
- OpenAlex
- W4390636334
- Cited by
- 292
- References
- 10
- Citations by year
- 2026: 143 · 2025: 113 · 2024: 36
- FWCI
- 80.91
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- MethodsX (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- ArTS Archivio della ricerca di Trieste (University of Trieste https://www.units.it/) (submittedVersion, cc-by)
- ARCA (Università Ca' Foscari Venezia) (submittedVersion, other-oa)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Multidisciplinary Science and Engineering Research
- Keywords
- Krippendorff's alpha, inter-rater reliability, web-based calculator, reliability analysis software
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
