Cited by 803Reliability and Agreement in MeasurementQualitative Research Methods and ApplicationsComputational and Text Analysis Methods

The DOI matches its Crossref record; 91 of the 92 references it cites were found in a registry or an index.

Guidelines for Establishing Reliability When Coding Narrative Data

Syed, Moin, Nelson, Sarah C.·2015·Emerging Adulthood

DOI 10.1177/2167696815587648

Published version of a preprint

Open this paper in CoChat — free account

Abstract

The use of quantitative, qualitative, and mixed methods approaches has been foundational to research on emerging adulthood, yet there remain many unresolved methodological issues pertaining to how to handle qualitative data. The purpose of this article is to review best practices for coding and establishing reliability when working with narrative data. In doing so, we highlight how establishing reliability must be seen as an evolving process, rather than simply a focus on the end product. The review is divided into three broad sections. In the first section, we discuss relatively more quantitatively focused methods of coding and establishing reliability, whereas in the second section we discuss relatively more qualitatively focused methods. In the final section, we provide recommendations for researchers interested in coding narrative and other types of open-ended data. This article is intended to serve as an essential resource for researchers working on a variety of topics related to emerging adulthood and beyond.

Integrity signals

Registry

Crossref · 10.1177/2167696815587648

Cites

91 of 91 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Using thematic analysis in psychology
Braun, V., & Clarke, V. (2006) · Qualitative Research in Psychology · doi:10.1191/1478088706qp063oa · Cited by 194,414
A Coefficient of Agreement for Nominal Scales
Cohen, J. (1960) · Educational and Psychological Measurement · doi:10.1177/001316446002000104 · Cited by 42,398
Intraclass correlations: Uses in assessing rater reliability.
Shrout, P. E., & Fleiss, J. L. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.2.420 · Cited by 24,532
Content Analysis: An Introduction to its Methodology.
Shelley, M., & Krippendorff, K. (1984) · Journal of the American Statistical Association · doi:10.2307/2288384 · Cited by 24,346
Untitled work
(2009) · Cited by 21,330
Mixed Methods Research: A Research Paradigm Whose Time Has Come
Johnson, R. B., & Onwuegbuzie, A. J. (2004) · Educational Researcher · doi:10.3102/0013189x033007014 · Cited by 11,170 · cited 1×
“Although qualitative analytic methods are not homogenous in approach, subjectivity is generally valued over consistency in illuminating constructs (Johnson & Onwuegbuzie, 2004).”
Background
Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology.
Cicchetti, D. V. (1994) · Psychological Assessment · doi:10.1037/1040-3590.6.4.284 · Cited by 9,349
Social Research Methods: Qualitative and Quantitative Approaches
Djamba, Y. K., & Neuman, W. L. (2002) · Teaching Sociology · doi:10.2307/3211488 · Cited by 8,853
Untitled work
(2011) · Cited by 7,157
False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567
Development and validation of ego-identity status.
Marcia, J. E. (1966) · Journal of Personality and Social Psychology · doi:10.1037/h0023281 · Cited by 5,172 · cited 1×
“For example, the theory-driven approach was used by McLean et al. (2014) to develop narrative coding for domain-specific identity development originally specified by Marcia (1966, 1993) and other identity status theorists.”
Method
Verification strategies for establishing reliability and validity in qualitative research
Morse, J. M., Barrett, M., Mayan, M., Olson, K., & Spiers, J. (2002) · International Journal of Qualitative Methods · doi:10.1177/160940690200100202 · Cited by 4,978
Social Research Methods: Qualitative and Quantitative Approaches
Bernard, H. R. (2012) · Cited by 3,524
Qualitative Data Analysis for Health Services Research: Developing Taxonomy, Themes, and Theory
Bradley, E. H., Curry, L. A., & Devers, K. J. (2007) · Health Services Research · doi:10.1111/j.1475-6773.2006.00684.x · Cited by 3,437
Bulletin of the Menninger Clinic
(1989) · Psychiatric Bulletin · doi:10.1192/pb.13.12.698-a · Cited by 3,380
The Equivalence of Weighted Kappa and the Intraclass Correlation Coefficient as Measures of Reliability
Fleiss, J. L., & Cohen, J. (1973) · Educational and Psychological Measurement · doi:10.1177/001316447303300309 · Cited by 3,354
High agreement but low Kappa: I. the problems of two paradoxes
Feinstein, A. R., & Cicchetti, D. V. (1990) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(90)90158-l · Cited by 3,034
Coding In-depth Semistructured Interviews: Problems of Unitization and Intercoder Reliability and Agreement
Campbell, J. L., Quincy, C., Osserman, J., & Pedersen, O. K. (2013) · Sociological Methods & Research · doi:10.1177/0049124113500475 · Cited by 2,974
A Guide to Conducting Consensual Qualitative Research
Hill, C. E., Thompson, B. J., & Williams, E. N. (1997) · The Counseling Psychologist · doi:10.1177/0011000097254001 · Cited by 2,884
Consensual qualitative research: An update.
Hill, C. E., Knox, S., Thompson, B. J., Williams, E. N., Hess, S. A., & Ladany, N. (2005) · Journal of Counseling Psychology · doi:10.1037/0022-0167.52.2.196 · Cited by 2,841
Two Decades of Developments in Qualitative Inquiry: A Personal, Experiential Perspective
Patton, M. Q. (2002) · Qualitative Social Work · doi:10.1177/1473325002001003636 · Cited by 2,564 · cited 1×
“Thus, rigor is as much a product of the researcher and research context, bias, and subjectivity as it is the data (Guba & Lincoln, 1986; Patton, 2002).”
Narrative Research: Reading, Analysis, and Interpretation
Lieblich, A., Tuval‐Mashiach, R., & Zilber, T. (1998) · Medical Entomology and Zoology · Cited by 2,251
Validity in qualitative research
Whittemore, R., Chase, S. K., & Mandle, C. L. (2001) · Qualitative Health Research · doi:10.1177/104973201129119299 · Cited by 2,144
Realities and Relationships: Soundings in Social Construction.
Kotarba, J. A., & Gergen, K. J. (1996) · Contemporary Sociology A Journal of Reviews · doi:10.2307/2077591 · Cited by 1,922
The Social Scientific Study of Leadership: Quo Vadis?
House, R. J., & Aditya, R. N. (1997) · Journal of Management · doi:10.1177/014920639702300306 · Cited by 1,918
Reliability of Content Analysis: The Case of Nominal Scale Coding
Scott, W. A. (1955) · Public Opinion Quarterly · doi:10.1086/266577 · Cited by 1,853
Quantifying Qualitative Analyses of Verbal Data: A Practical Guide
Michelene, T. H. (1997) · Journal of the Learning Sciences · doi:10.1207/s15327809jls0603_1 · Cited by 1,765
Coefficient Kappa: Some Uses, Misuses, and Alternatives
Brennan, R. L., & Prediger, D. J. (1981) · Educational and Psychological Measurement · doi:10.1177/001316448104100307 · Cited by 1,676
Narrative identity
McAdams, D. P., & McLean, K. C. (2013) · Current Directions in Psychological Science · doi:10.1177/0963721413475622 · Cited by 1,497
Journal of Research on Adolescence
(2010) · Journal of Research on Adolescence · doi:10.1111/(issn)1532-7795 · Cited by 1,274
Editorial: The New Era of Mixed Methods
Tashakkori, A., & Creswell, J. W. (2007) · Journal of Mixed Methods Research · doi:10.1177/2345678906293042 · Cited by 1,204
The Handbook of Research Synthesis
Hanka, R. (1994) · BMJ · doi:10.1136/bmj.309.6952.488a · Cited by 1,065
Reliability in Coding Open-Ended Data: Lessons Learned from HIV Behavioral Research
Hruschka, D. J., Schwartz, D., St.John, D. C., Picone-Decaro, E., Jenkins, R. A., & Carey, J. W. (2004) · Field Methods · doi:10.1177/1525822x04266540 · Cited by 829
The Oxford Handbook of Identity Development
McLean, K. C., & Syed, M. (2015) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780199936564.001.0001 · Cited by 827
Self-citation
Ego Identity: A Handbook for Psychosocial Research
Matteson, D. R. (1993) · Medical Entomology and Zoology · Cited by 615
Life's little (and big) lessons: Identity statuses and meaning-making in the turning point narratives of emerging adults.
McLean, K. C., & Pratt, M. W. (2006) · Developmental Psychology · doi:10.1037/0012-1649.42.4.714 · Cited by 610
Living into the story: Agency and coherence in a longitudinal study of narrative identity development and mental health over the course of psychotherapy
Adler, J. M. (2012) · Journal of Personality and Social Psychology · doi:10.1037/a0025289 · Cited by 588 · cited 2×
“…& McAdams, 2011), written surveys (Bauer & McAdams, 2010), computer-based surveys (Syed & Azmitia, 2008), transcribed conversations (Morgan & Korobov, 2012), ethnographic methods (Hammack, 2010), and in applied settings such as therapy (Adler, 2012) and alcoholics anonymous (Dunlop & Tracy, 2013).”
Method
Developing a Life Story: Constructing Relations between Self and Experience in Autobiographical Narratives
Pasupathi, M., Mansour, E., & Brubaker, J. R. (2007) · Human Development · doi:10.1159/000100939 · Cited by 531
The Ego Identity Status Approach to Ego Identity
Marcia, J. E. (1993) · Ego Identity · doi:10.1007/978-1-4613-8330-7_1 · Cited by 503
Ego Identity
Marcia, J. E., Waterman, A. S., Matteson, D. R., Archer, S. L., & Orlofsky, J. L. (1993) · Springer eBooks · doi:10.1007/978-1-4613-8330-7 · Cited by 470
Agreement and Information in the Reliability of Coding
Krippendorff, K. (2011) · Communication Methods and Measures · doi:10.1080/19312458.2011.568376 · Cited by 451
EVALUATING CODING DECISIONS
Vevea, J. L., Zelinsky, N. A. M., & Orwin, R. G. (2019) · Russell Sage Foundation eBooks · doi:10.7758/9781610448864.13 · Cited by 417
Why are you telling me that? A conceptual model of the social function of autobiographical memory
Alea, N. L., & Bluck, S. (2003) · Memory · doi:10.1080/741938207 · Cited by 401
Continuity and change in the life story: a longitudinal study of autobiographical memories in emerging adulthood.
McAdams, D., Bauer, J. J., Sakaeda, A. R., Anyidoho, N. A., Machado, M., Magrino-Failla, K., White, K. W., & Pals, J. L. (2006) · Journal of Personality · doi:10.1111/j.1467-6494.2006.00412.x · Cited by 367
Traits and Stories: Links Between Dispositional and Narrative Features of Personality
McAdams, D. P., Anyidoho, N. A., Brown, C., Huang, Y. T., Kaplan, B., & Machado, M. A. (2004) · Journal of Personality · doi:10.1111/j.0022-3506.2004.00279.x · Cited by 355 · cited 2×
“First, it obviously works only for coding categories that are continuous in nature (e.g., most forms of autobiographical reasoning; Lilgendahl & McAdams, 2011; McAdams et al., 2004).”
Background
Balanced Incomplete Block Designs for Inter-Rater Reliability Studies
Fleiss, J. L. (1981) · Applied Psychological Measurement · doi:10.1177/014662168100500115 · Cited by 313
A narrative approach to ethnic identity in emerging adulthood: Bringing life to the identity status model.
Syed, M., & Azmitia, M. (2008) · Developmental Psychology · doi:10.1037/0012-1649.44.4.1012 · Cited by 303
Self-citation
Talk, Small Stories, and Adolescent Identities
Bamberg, M. (2004) · Human Development · doi:10.1159/000081039 · Cited by 294
Untitled work
(2010) · Cited by 263
Constructing Stories of Self-Growth: How Individual Differences in Patterns of Autobiographical Reasoning Relate to Well-being in Midlife
Lilgendahl, J., & McAdams, D. (2011) · Journal of Personality · doi:10.1111/j.1467-6494.2010.00688.x · Cited by 257
Communications Through Limited Response Questioning
Bennett, E., Alpert, R., & Goldstein, A. C. (1954) · Public Opinion Quarterly · doi:10.1086/266520 · Cited by 254
Confusing Claims for Data: A Critique of Common Practices for Presenting Qualitative Research on Learning
Hammer, D., & Berland, L. K. (2013) · Journal of the Learning Sciences · doi:10.1080/10508406.2013.802652 · Cited by 246
Another look at interrater agreement.
Zwick, R. (1988) · Psychological Bulletin · doi:10.1037/0033-2909.103.3.374 · Cited by 229
Identity Configurations: A New Perspective on Identity Formation in Contemporary Society
Schachter, E. P. (2003) · Journal of Personality · doi:10.1111/j.0022-3506.2004.00260.x · Cited by 214
Relations Between Narrative Coherence, Identity, and Psychological Well-being in Emerging Adulthood
Waters, T. E. A., & Fivush, R. (2014) · Journal of Personality · doi:10.1111/jopy.12120 · Cited by 202
Inattentive Listening Undermines Self‐Verification in Personal Storytelling
Pasupathi, M., & Rich, B. (2005) · Journal of Personality · doi:10.1111/j.1467-6494.2005.00338.x · Cited by 202
Eudaimonic growth: Narrative growth goals predict increases in ego development and subjective well-being 3 years later.
Bauer, J. J., & McAdams, D. P. (2010) · Developmental Psychology · doi:10.1037/a0019654 · Cited by 192
Interrater agreement for journal manuscript reviews.
Whitehurst, G. J. “. (1984) · American Psychologist · doi:10.1037/0003-066x.39.1.22 · Cited by 189
The Present of the Past: Dialogues With Memory Over Time
Josselson, R. (2009) · Journal of Personality · doi:10.1111/j.1467-6494.2009.00560.x · Cited by 188
Configurations of Identity Among Sexual Minority Youth: Context, Desire, and Narrative
Hammack, P. L., Thompson, E. M., & Pilecki, A. (2008) · Journal of Youth and Adolescence · doi:10.1007/s10964-008-9342-3 · Cited by 173
The content and processes of autobiographical reasoning in narrative identity
McLean, K. C., & Fournier, M. A. (2007) · Journal of Research in Personality · doi:10.1016/j.jrp.2007.08.003 · Cited by 167
On telling the whole story: Facts and interpretations in autobiographical memory narratives from childhood through midadolescence.
Pasupathi, M., & Wainryb, C. (2010) · Developmental Psychology · doi:10.1037/a0018897 · Cited by 166
Sobering stories: Narratives of self-redemption predict behavioral change and improved health among recovering alcoholics
Dunlop, W. L., & Tracy, J. L. (2013) · Journal of Personality and Social Psychology · doi:10.1037/a0031185 · Cited by 166
Context and Identity Formation
Schachter, E. P. (2005) · Journal of Adolescent Research · doi:10.1177/0743558405275172 · Cited by 158
Identity, Conflict, and Coexistence
Hammack, P. L. (2006) · Journal of Adolescent Research · doi:10.1177/0743558406289745 · Cited by 157
Positioning a ‘mature’ self in interactive practices: How adolescent males negotiate ‘physical attraction’ in group talk
Korobov, N., & Bamberg, M. (2004) · British Journal of Developmental Psychology · doi:10.1348/0261510042378281 · Cited by 153
The distinguishing characteristics of narrative identity in adults with features of Borderline Personality Disorder: An empirical investigation
Adler, J. M., Chin, E. D., Kolisetty, A., & Oltmanns, T. (2012) · Journal of Personality Disorders · doi:10.1521/pedi.2012.26.4.498 · Cited by 146
Feminist social psychologies : international perspectives
Wilkinson, S. (1996) · Cited by 145
Narrating personality change.
Lodi‐Smith, J., Geise, A. C., Roberts, B. W., & Robins, R. W. (2009) · Journal of Personality and Social Psychology · doi:10.1037/a0014611 · Cited by 136
The Role of Domain Content in Understanding Identity Development Processes
McLean, K. C., Syed, M., Yoder, A., & Greenhoot, A. F. (2014) · Journal of Research on Adolescence · doi:10.1111/jora.12169 · Cited by 127
Self-citation
Interpretive Psychology: Stories, Circles, and Understanding Lived Experience
Tappan, M. B. (1997) · Journal of Social Issues · doi:10.1111/0022-4537.00041 · Cited by 111
Delta: A new measure of agreement between two raters
Andrés, A. M., & Femia, P. (2004) · British Journal of Mathematical and Statistical Psychology · doi:10.1348/000711004849268 · Cited by 110
To be or not to be: An exploration of ethnic identity development in context
Way, N., Santos, C. E., Niwa, E. Y., & Kim-Gervey, C. G. (2008) · New Directions for Child and Adolescent Development · doi:10.1002/cd.216 · Cited by 85
Might the Survey Be the Intervention? Participating in Ethnicity-Related Research as a Consciousness-Raising Experience
Syed, M., Juan, M. J. D., & Juang, L. P. (2011) · Identity · doi:10.1080/15283488.2011.613581 · Cited by 83
Self-citation
When is meaning making unhealthy for the self? The roles of neuroticism, implicit theories, and memory telling in trauma and transgression memories
Lilgendahl, J. P., McLean, K. C., & Mansfield, C. D. (2012) · Memory · doi:10.1080/09658211.2012.706615 · Cited by 69
Narratives in and About Families: An Examination of Coding Schemes and a Guide for Family Researchers.
Fiese, B. H., & Spagnola, M. (2005) · Journal of Family Psychology · doi:10.1037/0893-3200.19.1.51 · Cited by 68
Intimacy and Distancing
Korobov, N., & Thorne, A. (2006) · Journal of Adolescent Research · doi:10.1177/0743558405284035 · Cited by 63
Interpersonal identity formation in conversations with close friends about dating relationships
Morgan, E. M., & Korobov, N. (2011) · Journal of Adolescence · doi:10.1016/j.adolescence.2011.09.005 · Cited by 60
Stability and change in early memories over 22 years: themes, variations, and cadenzas.
Josselson, R. (2000) · PubMed · Cited by 56
Detecting differential rater functioning over time (DRIFT) using a Rasch multi-faceted rating scale model.
Wolfe, E. W., Moulder, B., & Myford, C. M. (2001) · PubMed · Cited by 55
Narrative and the Politics of Identity
Hammack, P. L. (2010) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780195394467.001.0001 · Cited by 43
Between Trauma and Redemption
Benish‐Weisman, M. (2009) · Journal of Cross-Cultural Psychology · doi:10.1177/0022022109346956 · Cited by 38
How late-adolescent friends share stories about relationships: The importance of mitigating the seriousness of romantic problems
Korobov, N., & Thorne, A. (2007) · Journal of Social and Personal Relationships · doi:10.1177/0265407507084193 · Cited by 31
Memorable everyday events in college: Narratives of the intersection of ethnicity and academia.
Syed, M. (2010) · Journal of Diversity in Higher Education · doi:10.1037/a0018503 · Cited by 23
Self-citation
The Vitality Criterion
Arnett, J. J. (2005) · Journal of Adolescent Research · doi:10.1177/0743558404271251 · Cited by 23
The Field of Identity Development Needs an Identity
McLean, K. C., & Syed, M. (2014) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780199936564.013.023 · Cited by 23
Self-citation
‘He's got no game’: young men's stories about failed romantic and sexual experiences
Korobov, N. (2009) · Journal of Gender Studies · doi:10.1080/09589230902812406 · Cited by 18
Emerging Adulthood —The Journal
van Dulmen, M. H. M. (2013) · Emerging Adulthood · doi:10.1177/2167696812469356 · Cited by 16
Is the American Psychological Association ready to accept a division for qualitative inquiry?
Lyons, A. W. (2009) · The Humanistic Psychologist · doi:10.1080/08873260802394566 · Cited by 13
American Educational Research Association
(1986) · Educational Researcher · doi:10.3102/0013189x015009019 · Cited by 10
Social Support Mediates the Relation between Attachment and Responses to Potentially Traumatic Events
Shallcross, S. L. (2013) · University of Minnesota Digital Conservancy (University of Minnesota) · Cited by 2

Research works printed but not linked: 8

Bakeman R., 1986, Observing behavior: An introduction to sequential analysis
Cook T. D., 1979, Quasi-experimentation: Design and analysis for field setting
Guba E. G., 1986, Evaluation Studies Review Annual, 11, 70
Lincoln Y., 2000, Journal of Management, 23, 409
Main M., 1994, Adult attachment classification system
10.1111/j.1540-4560.1997.tb02453.x · DOI
Waters T. E., 2014, Journal of Personality
Wolfe E. W., 2000, Journal of Applied Measurement, 2, 256

Sorted by citations

Thinking about the Coding Process in Qualitative Data Analysis
Elliott, V. (2018) · The Qualitative Report · doi:10.46743/2160-3715/2018.3560 · Cited by 746
““Reliability is generally defined as the consistency of a measure, or the degree to which scores approximate each other across multiple assessments of an instrument or multiple ratings of the same event” (Syed & Nelson, 2015).”
Research Methods for Studying Narrative Identity: A Primer
Adler, J. M., Dunlop, W. L., Fivush, R., Lilgendahl, J. P., Lodi-Smith, J., McAdams, D. P., McLean, K. C., Pasupathi, M., & Syed, M. (2017) · Social Psychological and Personality Science · doi:10.1177/1948550617698202 · Cited by 359
Self-citation
The empirical structure of narrative identity: The initial Big Three
McLean, K. C., Syed, M., Pasupathi, M., Adler, J. M., Dunlop, W. L., Drustrup, D., Fivush, R., Graci, M. E., Lilgendahl, J. P., Lodi-Smith, J., et al. (2020) · Journal of Personality and Social Psychology · doi:10.1037/pspp0000247 · Cited by 358
Self-citation
Beyond a coefficient: an interactive process for achieving inter-rater consistency in qualitative coding
Hemmler, V. L., Kenney, A. W., Langley, S. D., Callahan, C. M., Gubbins, E. J., & Holder, S. (2020) · Qualitative Research · doi:10.1177/1468794120976072 · Cited by 241
Identity development in cultural context: The role of deviating from master narratives
McLean, K. C., Lilgendahl, J. P., Fordham, C., Alpert, E., Marsden, E., Szymanowski, K., & McAdams, D. P. (2017) · Journal of Personality · doi:10.1111/jopy.12341 · Cited by 237

See all 803

Authors: Syed, M., & Nelson, S. C.
Affiliations: University of Minnesota (US)
Corresponding author: Syed, Moin
ORCID: Syed, Moin
Nelson, Sarah C.
Venue: Emerging Adulthood
Publisher: SAGE Publications
Published: 2015-05-26
Volume: 3
Issue: 6
Pages: 375-387
ISSN: 2167-6968 (ISSN-L)
2167-6984
DOI: 10.1177/2167696815587648
OpenAlex: W2232317411
Cited by: 801
References: 92
Citations by year: 2026: 63 · 2025: 83 · 2024: 106 · 2023: 89 · 2022: 117 · 2021: 126 · 2020: 96 · 2019: 62 · 2018: 28 · 2017: 23 · 2016: 7 · 2015: 1
FWCI: 17.94
Type: journal article
Language: en
Open access: closed
Licence: journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
Locations: Emerging Adulthood (publishedVersion)
OSF Preprints (OSF Preprints) (submittedVersion)
Topics: Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement
Social Sciences › Education › Qualitative Research Methods and Applications
Social Sciences › General Social Sciences › Computational and Text Analysis Methods
Keywords: narrative data, coding reliability, qualitative data analysis, coding practices, mixed methods

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Guidelines for Establishing Reliability When Coding Narrative Data

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks