The DOI matches its Crossref record; 1 of the 1 references it cites were found in a registry or an index.
Investigating variability in tasks and rater judgements in a performance test of foreign language speaking
Abstract
Much of the recent debate that has surrounded the development and use of 'performance', or 'communicative' language tests has focused on a supposed trade-off between two sets of desirable qualities: correspondence between test tasks and test performance to nontest language use for content relevance; and reliability of scores derived from test performance. One area that has been of particular concern with performance tests is the potential variability in tasks and rater judgements, and this has been investigated in the language testing literature with two complementary approaches: generalizability the ory and many faceted Rasch modelling. GENOVA, which performs general izability theory analyses, estimates the relative contribution of variation in test tasks and rater judgements to variation in test scores. FACETS, which performs many faceted Rasch modelling, estimates differences in task difficulty and rater severity, and adjusts ability estimates of test takers, taking these differences into account. In this article we first discuss the design and development of a foreign language (Spanish) test battery that was designed for two purposes: first, to place University of California Education Abroad students into programmes at universities abroad that are appropriate for their level of language ability, and secondly to provide diagnostic information that will be useful for designing appropriate teaching and learning pro grammes for prospective education abroad students. The test battery consists of four subtests: reading, listening and note-taking, speaking, and writing. All subtests share a common theme or topic, and are interdependent. We then discuss the results of the GENOVA and FACETS analyses of the speaking subtest, based on a full field trial with a group of University of California undergraduate students who had been selected for participation in the Education Abroad Program. Finally, we discuss the implications of these results for the use of G-theory and many faceted Rasch modelling for the development of performance tests of foreign language ability.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The interplay of evidence and consequences in the validation of performance assessments
Messick, S. (1994) · Educational Researcher · doi:10.3102/0013189x023002013 · Cited by 1,315
Cited by
Chapelle, C. A. (2003) · Language Learning & Language Teaching · doi:10.1075/lllt.7 · Cited by 377
Modern language testing at the turn of the century: assuring that what we count counts
Bachman, L. F. (2000) · Language Testing · doi:10.1177/026553220001700101 · Cited by 309
Self-citationRater types in writing performance assessments: A classification approach to rater variability
Eckes, T. (2008) · Language Testing · doi:10.1177/0265532207086780 · Cited by 257
Application of Generalizability Theory to the Big Five Inventory
Arterberry, B. J., Martens, M. P., Cadigan, J. M., & Rohrer, D. (2014) · Personality and Individual Differences · doi:10.1016/j.paid.2014.05.015 · Cited by 65
Metadata
- Authors
- Bachman, L. F., Lynch, B. K., & Mason, M.
- Venue
- Language Testing
- Publisher
- SAGE Publications
- Published
- 1995-07
- Volume
- 12
- Issue
- 2
- Pages
- 238-257
- ISSN
- 0265-5322 (ISSN-L)
- 1477-0946
- OpenAlex
- W2130631175
- Cited by
- 190
- References
- 1
- FWCI
- 4.75
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Investigating variability in tasks and rater judgements in a performance test of foreign language speaking. https://app.cochat.ai/papers/10.1177/026553229501200206
- The paper
- Bachman, L. F., Lynch, B. K., & Mason, M. (1995). Investigating variability in tasks and rater judgements in a performance test of foreign language speaking. Language Testing. https://doi.org/10.1177/026553229501200206
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.