Cited by 803Reliability and Agreement in MeasurementQualitative Research Methods and ApplicationsComputational and Text Analysis Methods

Guidelines for establishing reliability when coding narrative data

Syed, Moin, Nelson, Sarah C.·2015·Emerging Adulthood

Published version of a preprint

Abstract

The use of quantitative, qualitative, and mixed methods approaches has been foundational to research on emerging adulthood, yet there remain many unresolved methodological issues pertaining to how to handle qualitative data. The purpose of this article is to review best practices for coding and establishing reliability when working with narrative data. In doing so, we highlight how establishing reliability must be seen as an evolving process, rather than simply a focus on the end product. The review is divided into three broad sections. In the first section, we discuss relatively more quantitatively focused methods of coding and establishing reliability, whereas in the second section we discuss relatively more qualitatively focused methods. In the final section, we provide recommendations for researchers interested in coding narrative and other types of open-ended data. This article is intended to serve as an essential resource for researchers working on a variety of topics related to emerging adulthood and beyond.

Integrity signals

Registry

Crossref · 10.1177/2167696815587648

Cites

91 of 91 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Trained raters allow multiple coders to demonstrate inter-rater reliability, which enhances results' internal and external validity.

    “Chiefly, it allows multiple coders to demonstrate inter-rater reliability, which enhances results’ internal and external validity (Adler et al., 2017; Syed & Nelson, 2015).”

Cites

Flagged first, then by citations

1 of the 92 references couldn’t be matched to a known paper.

Cited by

Sorted by citations

Authors
Syed, M., & Nelson, S. C.
Affiliations
University of Minnesota (US)
Corresponding author
Syed, Moin
Venue
Emerging Adulthood
Published
2015-05-26
Volume
3
Issue
6
Pages
375-387
ISSN
  • 2167-6968 (ISSN-L)
  • 2167-6984
OpenAlex
W2232317411
Cited by
801
References
92
Citations by year
2026: 63 · 2025: 83 · 2024: 106 · 2023: 89 · 2022: 117 · 2021: 126 · 2020: 96 · 2019: 62 · 2018: 28 · 2017: 23 · 2016: 7 · 2015: 1
FWCI
17.94
Type
journal article
Language
en
Open access
closed
Locations
  • Emerging Adulthood (publishedVersion)
  • OSF Preprints (OSF Preprints) (submittedVersion)
Topics
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement
  • Social Sciences › Education › Qualitative Research Methods and Applications
  • Social Sciences › General Social Sciences › Computational and Text Analysis Methods
Keywords
narrative data, coding reliability, qualitative data analysis, coding practices, mixed methods

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review