The DOI matches its Crossref record; 23 of the 23 references it cites were found in a registry or an index.
Examining Rater Errors in the Assessment of Written Composition With a Many‐Faceted Rasch Model
Abstract
This study describes several categories of rater errors (rater severity, halo effect, central tendency, and restriction of range). Criteria are presented for evaluating the quality of ratings based on a many‐faceted Rasch measurement (FACETS) model for analyzing judgments. A random sample of 264 compositions rated by 15 raters and a validity committee from the 1990 administration of the Eighth Grade Writing Test in Georgia is used to illustrate the model. The data suggest that there are significant differences in rater severity. Evidence of a halo effect is found for two raters who appear to be rating the compositions holistically rather than analytically. Approximately 80% of the ratings are in the two middle categories of the rating scale, indicating that the error of central tendency is present. Restriction of range is evident when the unadjusted raw score distribution is examined, although this rater error is less evident when adjusted estimates of writing competence are used
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Statistical Methods for Meta-Analysis
Wasserman, S., Hedges, L. V., & Olkin, I. (1988) · Journal of Educational Statistics · doi:10.2307/1164953 · Cited by 10,326
Probabilistic Models for Some Intelligence and Attainment Tests
(1961) · Information and Control · doi:10.1016/s0019-9958(61)80061-2 · Cited by 6,939
Statistical Methods for Meta-Analysis.
Freeman, P. R., Hedges, L. V., & Olkin, I. (1986) · Biometrics · doi:10.2307/2531069 · Cited by 4,784
A constant error in psychological ratings.
Thorndike, E. L. (1920) · Journal of Applied Psychology · doi:10.1037/h0071663 · Cited by 2,163
Probabilistic Models for some Intelligence and Attainment Tests.
Wolins, L., Wright, B. D., & Rasch, G. (1982) · Journal of the American Statistical Association · doi:10.2307/2287805 · Cited by 1,979
Andrich, D. (1988) · doi:10.4135/9781412985598 · Cited by 1,345
Rating Scale Analysis: Rasch Measurement
Wolins, L., Wright, B. D., & Masters, G. N. (1983) · Journal of the American Statistical Association · doi:10.2307/2288670 · Cited by 1,029
Complex, Performance-Based Assessment: Expectations and Validation Criteria
Linn, R. L., Baker, E. L., & Dunbar, S. B. (1991) · Educational Researcher · doi:10.2307/1176232 · Cited by 926
Best Test Design. Rasch Measurement.
Wright, B. D., & Stone, M. H. (1979) · Cited by 612
Rating the ratings: Assessing the psychometric quality of rating data.
Saal, F. E., Downey, R. G., & Lahey, M. A. (1980) · Psychological Bulletin · doi:10.1037//0033-2909.88.2.413 · Cited by 566
Cited by
Eurasian Journal of Educational Research
(2021) · Eurasian Journal of Educational Research · doi:10.14689/ejer · Cited by 638
A Straightforward Approach to Markov Chain Monte Carlo Methods for Item Response Models
Patz, R. J., & Junker, B. W. (1999) · Journal of Educational and Behavioral Statistics · doi:10.3102/10769986024002146 · Cited by 476
Conceptualizing and measuring short-term changes in L2 writing complexity.
Bulté, B., & Housen, A. (2014) · Journal of Second Language Writing · doi:10.1016/j.jslw.2014.09.005 · Cited by 474
Rater types in writing performance assessments: A classification approach to rater variability
Eckes, T. (2008) · Language Testing · doi:10.1177/0265532207086780 · Cited by 257
Myford, C. M., & Wolfe, E. W. (2009) · Journal of Educational Measurement · doi:10.1111/j.1745-3984.2009.00088.x · Cited by 90
Metadata
- Authors
- Engelhard, G.
- Venue
- Journal of Educational Measurement
- Publisher
- Wiley
- Published
- 1994-06
- Volume
- 31
- Issue
- 2
- Pages
- 93-112
- ISSN
- 0022-0655 (ISSN-L)
- 1745-3984
- OpenAlex
- W2123555614
- Cited by
- 288
- References
- 23
- FWCI
- 8.46
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Examining Rater Errors in the Assessment of Written Composition With a Many‐Faceted Rasch Model. https://app.cochat.ai/papers/10.1111/j.1745-3984.1994.tb00436.x
- The paper
- Engelhard, G. (1994). Examining Rater Errors in the Assessment of Written Composition With a Many‐Faceted Rasch Model. Journal of Educational Measurement. https://doi.org/10.1111/j.1745-3984.1994.tb00436.x
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.