The Reliability of Difference Scores When Errors Are Correlated
The DOI matches its Crossref record.
Abstract
The usual formulas for the reliability of differences between two test scores X and Y are based on the assumption that the error scores E X and E Y are uncorrelated. In modern developments of test score theory, such as that of Lord and Novick, a true score is defined as the expected value of an individual's observed score. This definition implies that true scores on any test are uncorrelated with error scores on any test, but it does not imply that error scores on distinct tests X and Y are uncorrelated. A zero correlation between the errors can be obtained only by introducing an additional assumption of "experimental independence" that does not follow from the other axioms in the model. This assumption restricts severely the class of random variables to which the usual formulas for reliability of differences will apply. The present paper investigated the reliability of difference scores in more general cases where it is not assumed that error scores on distinct tests are uncorrelated. The formulas derived are relatively simple, and they reduce to the usual ones of the classical model when E X and E Y are uncorrelated.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cited by
A growth curve approach to the measurement of change.
Rogosa, D., Brandt, D., & Zimowski, M. (1982) · Psychological Bulletin · doi:10.1037//0033-2909.92.3.726 · Cited by 1,081
Questions and Answers in the Measurement of Change
Willett, J. B. (1988) · Review of Research in Education · doi:10.2307/1167368 · Cited by 454
DEMONSTRATING THE RELIABILITY THE DIFFERENCE SCORE IN THE MEASUREMENT OF CHANGE
ROGOSA, D. R., & WILLETT, J. B. (1983) · Journal of Educational Measurement · doi:10.1111/j.1745-3984.1983.tb00211.x · Cited by 435
Willett, J. B. (1989) · Educational and Psychological Measurement · doi:10.1177/001316448904900309 · Cited by 278
Obtaining Unbiased Results in Meta-Analysis: The Importance of Correcting for Statistical Artifacts
Wiernik, B. M., & Dahlke, J. A. (2020) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245919885611 · Cited by 135
“Generally, difference scores are much less reliable than averages of single scores (Williams & Zimmerman, 1977).”
Background
Metadata
- Authors
- Williams, R. H., & Zimmerman, D. W.
- Venue
- Educational and Psychological Measurement
- Publisher
- SAGE Publications
- Published
- 1977-10
- Volume
- 37
- Issue
- 3
- Pages
- 679-689
- ISSN
- 0013-1644 (ISSN-L)
- 1552-3888
- OpenAlex
- W1983494042
- Cited by
- 58
- References
- 0
- FWCI
- 2.48
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: The Reliability of Difference Scores When Errors Are Correlated. https://app.cochat.ai/papers/10.1177/001316447703700310
- The paper
- Williams, R. H., & Zimmerman, D. W. (1977). The Reliability of Difference Scores When Errors Are Correlated. Educational and Psychological Measurement. https://doi.org/10.1177/001316447703700310
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.