The DOI matches its Crossref record; 49 of the 49 references it cites were found in a registry or an index.
Lexical Diversity in Writing and Speaking Task Performances
Abstract
In the rating scales of major international language tests, as well as in automated evaluation systems (e.g. e-rater), a positive relationship is often claimed between lexical diversity, holistic quality of written or spoken discourses, and language proficiency of candidates. This article reports a posteriori validation study that analysed a sample of the archived data of an international language test to examine empirically to what extent such relationships exist. It is also noted that previous studies on lexical diversity in the field of applied linguistics have focused exclusively on either written or spoken discourses, no study to date has compared lexical diversity of spoken and written discourses produced by the same participants. Therefore, the second aim of this article is to understand the differences in lexical diversity between writing and speaking task performances, and to what extent the topics of the writing prompts may affect lexical diversity of written discourses. Using D as a measure of lexical diversity (Malvern and Richards 1997, 2002; Malvern et al.2004), it was found that D had a statistically significant and positive correlation with the overall quality ratings of both writing and speaking performances as well as the candidates’ general language proficiency. Nevertheless, the significant relationships were not borne out across the subgroups of the sample in terms of gender, first language background, purpose of taking the test and topics of the writing prompts. The different writing topics also had significant effects on lexical diversity—especially the topics that candidates were highly familiar with—even after controlling for writing ability and overall language proficiency. The lexical diversity of candidates’ writing and speaking performances were approximately at the same level; further, D was found to be a better predictor of speaking than writing performance. The implications of these findings are discussed with specific reference to the use of lexical diversity measures to inform language test validation and the development of lexical diversity parameters in automated evaluation systems.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Tabachnick, B. G., & Fidell, L. S. (1983) · Cited by 79,866
Applied Multivariate Statistics for the Social Sciences
Stevens, J. P. (2012) · Routledge eBooks · doi:10.4324/9780203843130 · Cited by 10,648
How many subjects does it take to do a regression analysis
Green, S. B. (1991) · Multivariate Behavioral Research · doi:10.1207/s15327906mbr2603_7 · Cited by 3,916 · cited 1×
“Tabachnick and Fidell (2003, p. 117), quoting Green (1991) , suggest a more conservative formula for calculating sample size requirements (n ≥ 50 + 8m for testing multiple correlation, and n ≥ 104 + m for testing individual predictors, where m = number of predictors).”
BackgroundThe CHILDES project: tools for analyzing talk
MacWhinney, B. (1992) · Child Language Teaching and Therapy · doi:10.1177/026565909200800211 · Cited by 3,396
(2018) · The SAGE Encyclopedia of Educational Research, Measurement, and Evaluation · doi:10.4135/9781506326139.n223 · Cited by 1,384
Vocabulary size and use: Lexical richness in l2 written production
LAUFER, B., & NATION, P. (1995) · Applied Linguistics · doi:10.1093/applin/16.3.307 · Cited by 1,316
Chomsky, N. A. D. T. (1993) · Medical Entomology and Zoology · Cited by 632
The relationship of lexical proficiency to the quality of ESL compositions
Engber, C. A. (1995) · Journal of Second Language Writing · doi:10.1016/1060-3743(95)90004-7 · Cited by 491
The Statistical Study of Literary Vocabulary.
Rider, P. R., & Yule, G. U. (1944) · Journal of the American Statistical Association · doi:10.2307/2280636 · Cited by 438
How Variable May a Constant be? Measures of Lexical Richness in Perspective
Tweedie, F. J., & Baayen, R. H. (1998) · Computers and the Humanities · doi:10.1023/a:1001749303137 · Cited by 423
vocd: A theoretical and empirical evaluation
McCarthy, P. M., & Jarvis, S. (2007) · Language Testing · doi:10.1177/0265532207080767 · Cited by 341
Assessment criteria in a large-scale writing test: what do they really mean to the raters?
Lumley, T. (2002) · Language Testing · doi:10.1191/0265532202lt230oa · Cited by 321
Type/Token Ratios: what do they really tell us?
Richards, B. (1987) · Journal of Child Language · doi:10.1017/s0305000900012885 · Cited by 292
Lexical Richness in the Spontaneous Speech of Bilinguals
Daller, H. (2003) · Applied Linguistics · doi:10.1093/applin/24.2.197 · Cited by 284
Short texts, best-fitting curves and new measures of lexical diversity
Jarvis, S. (2002) · Language Testing · doi:10.1191/0265532202lt220oa · Cited by 281
Malvern, D., & Richards, B. (2002) · Language Testing · doi:10.1191/0265532202lt221oa · Cited by 266
Measuring vocabulary diversity using dedicated software
McKee, G. (2000) · Literary and Linguistic Computing · doi:10.1093/llc/15.3.323 · Cited by 236
Coming to grips with lexical richness in spontaneous speech data
Vermeer, A. (2000) · Language Testing · doi:10.1177/026553220001700103 · Cited by 231
Developmental Trends in Lexical Diversity
Duran, P. (2004) · Applied Linguistics · doi:10.1093/applin/25.2.220 · Cited by 190
RELATIONSHIP BETWEEN LEXICAL COMPETENCE AND LANGUAGE PROFICIENCY: Variable Sensitivity
Zareva, A., Schwanenflugel, P., & Nikolova, Y. (2005) · Studies in Second Language Acquisition · doi:10.1017/s0272263105050254 · Cited by 175
THREE LANGUAGE VARIABLES IN COMMUNICATION RESEARCH: INTENSITY, IMMEDIACY, AND DIVERSITY
BRADAC, J. J., BOWERS, J. W., & COURTRIGHT, J. A. (1979) · Human Communication Research · doi:10.1111/j.1468-2958.1979.tb00639.x · Cited by 163
Another Perspective on Vocabulary Richness
Hoover, D. L. (2003) · Computers and the Humanities · doi:10.1023/a:1022673822140 · Cited by 128
Linnarud, M. (1986) · C.W.K. Gleerup eBooks · Cited by 127
The Development of L2 Lexis in the Expression of the Advanced Learner
Laufer, B. (1991) · Modern Language Journal · doi:10.1111/j.1540-4781.1991.tb05380.x · Cited by 127
Lexical Frequency Profiles: A Monte Carlo Analysis
Meara, P. M. (2005) · Applied Linguistics · doi:10.1093/applin/amh037 · Cited by 123
Klee, T. M., Stokes, S. F., Wong, A. M.-Y., Fletcher, P. J., & Gavin, W. J. (2004) · Journal of Speech Language and Hearing Research · doi:10.1044/1092-4388(2004/104 · Cited by 122
Lexical Diversity in the Spontaneous Speech of Children With Specific Language Impairment
Owen, A. J., & Leonard, L. B. (2002) · Journal of Speech Language and Hearing Research · doi:10.1044/1092-4388(2002/075 · Cited by 103
ESSAY TEST SCORING: INTERACTION OF RELEVANT VARIABLES
Chase, C. I. (1986) · Journal of Educational Measurement · doi:10.1111/j.1745-3984.1986.tb00232.x · Cited by 101
P-Lex: A Simple and Effective Way of Describing the lexical Characteristics of Short L2 Tests.
Meara, P. M., & Bell, H. (2001) · Cited by 99
BEYOND ESSAY LENGTH: EVALUATING E‐RATER®'S PERFORMANCE ON TOEFL® ESSAYS
Chodorow, M. S., & Burstein, J. C. (2004) · ETS Research Report Series · doi:10.1002/j.2333-8504.2004.tb01931.x · Cited by 99
Lexical characteristics of near‐native second‐language learners of Swedish
Hyltenstam, K. (1988) · Journal of Multilingual and Multicultural Development · doi:10.1080/01434632.1988.9994320 · Cited by 97
Measuring lexical diversity in children who stutter: application of vocd
Wagovich, S. A., & Ratner, N. B. (2002) · Journal of Fluency Disorders · doi:10.1016/s0094-730x(02)00162-6 · Cited by 90
Lexical Frequency Profiles: From Monte Carlo to the Real World
Laufer, B. (2005) · Applied Linguistics · doi:10.1093/applin/ami029 · Cited by 85
Lexical density in candidate output on direct and semi-direct versions of an oral proficiency test
O’Loughlin, K. (1995) · Language Testing · doi:10.1177/026553229501200205 · Cited by 82
Herdan, G. (1961) · Language · doi:10.2307/411255 · Cited by 80
The test-takers' choice: an investigation of the effect of topic on language-test performance
Jennings, M., Fox, J. D., Graves, B., & Shohamy, E. (1999) · Language Testing · doi:10.1177/026553229901600402 · Cited by 76
Influences of Handwriting Quality on Teacher Evaluation of Written Work
Markham, L. R. (1976) · American Educational Research Journal · doi:10.3102/00028312013004277 · Cited by 75
Measures of lexical diversity in aphasia
Wright, H. H., Wagovich, S. A., & Newhoff, M. (2003) · Aphasiology · doi:10.1080/02687030344000166 · Cited by 59
EFFECTS OF ACHIEVEMENT EXPECTATIONS AND HANDWRITING QUALITY ON SCORING ESSAYS
HUGHES, D. C., Keeling, B., & Tuck, B. F. (1983) · Journal of Educational Measurement · doi:10.1111/j.1745-3984.1983.tb00190.x · Cited by 52
THE INFLUENCE OF HANDWRITING ON ASSESSMENT
Briggs, D. (1970) · Educational Research · doi:10.1080/0013188700130107 · Cited by 52
A pilot study on gender differences in conversational speech on lexical in richness measures
Singh, S. (2001) · Literary and Linguistic Computing · doi:10.1093/llc/16.3.251 · Cited by 47
CONTRAST EFFECTS IN EVALUATING ESSAYS
Daly, J. A., & Dickson‐Markman, F. (1982) · Journal of Educational Measurement · doi:10.1111/j.1745-3984.1982.tb00136.x · Cited by 41
Hales, L. W., & Tokar, E. (1975) · Journal of Educational Measurement · doi:10.1111/j.1745-3984.1975.tb01014.x · Cited by 40
The lexical richness of L2 written productions and the validity of vocabulary tests
Arnaud, P. J. L. (1984) · HAL · Cited by 34
The Validity of the Handwriting Scale From the Test of Written Language
Graham, S., Boyer-Shick, K., & Tippets, E. (1989) · The Journal of Educational Research · doi:10.1080/00220671.1989.10885886 · Cited by 34
What is in a teacher's mind? Teacher ratings of EFL essays and different aspects of lexical richness
Daller, H., & Phelan, D. M. (2007) · Cambridge University Press eBooks · doi:10.1017/cbo9780511667268.016 · Cited by 31
THE IMPACT OF ACHIEVEMENT EXPECTATIONS AND HANDWRITING QUALITY ON SCORING ESSAY TESTS
Chase, C. I. (1979) · Journal of Educational Measurement · doi:10.1111/j.1745-3984.1979.tb00084.x · Cited by 23
Validity and threats to the validity of vocabulary measurement
Richards, B., & Malvern, D. D. (2007) · Cambridge University Press eBooks · doi:10.1017/cbo9780511667268.007 · Cited by 20
Comparative Analysis of Text Data in Successful Face-to-Face and Electronic Negotiations
Sokolova, M., Shah, M., & Śzpakowicz, S. (2006) · Group Decision and Negotiation · doi:10.1007/s10726-006-9024-z · Cited by 17
Cited by
The Relationship of Lexical Richness to the Quality of ESL Learners’ Oral Narratives
LU, X. (2012) · The Modern Language Journal · doi:10.1111/j.1540-4781.2011.01232_1.x · Cited by 526
Guo, L., Crossley, S. A., & McNamara, D. S. (2013) · Assessing Writing · doi:10.1016/j.asw.2013.05.002 · Cited by 278
Lee, U., Han, A., Lee, J., Lee, E., Kim, J., Kim, H., & Lim, C. (2024) · Education and Information Technologies · doi:10.1007/s10639-023-12150-4 · Cited by 107
Dynamic Development in Speaking Versus Writing in Identical Twins
Chan, H., Verspoor, M., & Vahtrick, L. (2015) · Language Learning · doi:10.1111/lang.12107 · Cited by 68
Towards Evaluating Narrative Quality In Student Writing
Somasundaran, S., Flor, M., Chodorow, M., Molloy, H., Gyawali, B., & McCulla, L. (2018) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00007 · Cited by 29
Metadata
- Authors
- Yu, G.
- Venue
- Applied Linguistics
- Publisher
- Oxford University Press (OUP)
- Published
- 2009-06-04
- Volume
- 31
- Issue
- 2
- Pages
- 236-259
- ISSN
- 0142-6001 (ISSN-L)
- 1477-450X
- OpenAlex
- W1981131584
- Cited by
- 324
- References
- 49
- FWCI
- 8.07
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.