CoChat

Confidence intervals for population reliability coefficients: Evaluation of methods, recommendations, and software for composite measures.

Kelley, Ken, Pornprasertmanit, Sunthud · 2016 · Psychological Methods

The DOI matches its Crossref record; 80 of the 80 references it cites were found in a registry or an index.

Abstract

A composite score is the sum of a set of components. For example, a total test score can be defined as the sum of the individual items. The reliability of composite scores is of interest in a wide variety of contexts due to their widespread use and applicability to many disciplines. The psychometric literature has devoted considerable time to discussing how to best estimate the population reliability value. However, all point estimates of a reliability coefficient fail to convey the uncertainty associated with the estimate as it estimates the population value. Correspondingly, a confidence interval is recommended to convey the uncertainty with which the population value of the reliability coefficient has been estimated. However, many confidence interval methods for bracketing the population reliability coefficient exist and it is not clear which method is most appropriate in general or in a variety of specific circumstances. We evaluate these confidence interval methods for 4 reliability coefficients (coefficient alpha, coefficient omega, hierarchical omega, and categorical omega) under a variety of conditions with 3 large-scale Monte Carlo simulation studies. Our findings lead us to generally recommend bootstrap confidence intervals for hierarchical omega for continuous items and categorical omega for categorical items. All of the methods we discuss are implemented in the freely available R language and environment via the MBESS package.

Overview

Integrity signals

Registry

Crossref · 10.1037/a0040086

Cites

80 of 80 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Show all 80 references

Cited by

See all 280

Metadata

Authors
Kelley, K., & Pornprasertmanit, S.
Affiliations
  • University of Notre Dame (US)
  • Texas Tech University (US)
Venue
Psychological Methods
Published
2016
Volume
21
Issue
1
Pages
69-92
ISSN
  • 1082-989X (ISSN-L)
  • 1939-1463
OpenAlex
W2295825737
Cited by
259
References
80
Citations by year
2026: 20 · 2025: 17 · 2024: 27 · 2023: 31 · 2022: 35 · 2021: 38 · 2020: 29 · 2019: 22 · 2018: 22 · 2017: 13 · 2016: 4
FWCI
22.28
Type
journal article
Language
en
Open access
closed
Locations
  • Psychological Methods (publishedVersion)
  • PubMed (publishedVersion)
Funders
Notre Dame Deloitte Center for Ethical Leadership
Topics
  • Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement
  • Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
Keywords
confidence intervals, composite score, Cronbach's alpha, bootstrap confidence intervals, reliability estimation, Monte Carlo simulation, R software
MeSH terms
Humans, Psychometrics, Reproducibility of Results, Confidence Intervals

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.