The DOI matches its Crossref record; 46 of the 46 references it cites were found in a registry or an index.
vocd: A theoretical and empirical evaluation
Abstract
A reliable index of lexical diversity (LD) has remained stubbornly elusive for over 60 years. Meanwhile, researchers in fields as varied as stylistics, neuropathology, language acquisition, and even forensics continue to use flawed LD indices — often ignorant that their results are questionable and in some cases potentially dangerous. Recently, an LD measurement instrument known as vocd has become the virtual tool of the LD trade. In this paper, we report both theoretical and empirical evidence that calls into question the rationale for vocd and also indicates that its reliability is not optimal. Although our evidence shows that vocd's output (D) is a relatively robust indicator of the aggregate probabilities of word occurrences in a text, we show that these probabilities — and thus also D — are affected by text length. Malvern, Richards, Chipere and Durán (2004) acknowledge that D (as calculated by vocd's default method) can be affected by text length, but claim that the effects are not significant for the ranges of text lengths with which they are concerned. In this paper, we explain why D is affected by text length, and demonstrate with an extensive empirical analysis that the effects of text length are significant over certain ranges, which we identify.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A Mathematical Theory of Communication
Shannon, C. E. (1948) · Bell System Technical Journal · doi:10.1002/j.1538-7305.1948.tb01338.x · Cited by 88,486
Photogrammetric Engineering and Remote Sensing
Nicol, J., King, B., Quattrochi, D., Dowman, I. J., Ehlers, M., & Xiaoli, D. (2007) · UCL Discovery (University College London) · Cited by 9,325
(1967) · Medical Entomology and Zoology · Cited by 7,532
Journal of The American Statistical Association
Fienberg, S. E. (2005) · Encyclopedia of Statistical Sciences · doi:10.1002/0471667196.ess1317 · Cited by 6,125
Variation across Speech and Writing
Biber, D. (1988) · doi:10.1017/cbo9780511621024 · Cited by 5,227
Proceedings of the 29th Annual Conference of the Cognitive Science Society
Rohde, H., Kehler, A., & Elman, J. L. (2007) · Cognitive Science · Cited by 1,788
Read, J. (2000) · doi:10.1017/cbo9780511732942 · Cited by 1,305
Certain Language Skills in Children: Their Development and Interrelationships
TEMPLIN, M. C. (1957) · University of Minnesota Press eBooks · doi:10.5749/j.ctttv2st · Cited by 899
Information retrieval, computational and theoretical aspects
Heaps, H. S. (1978) · CERN Document Server (European Organization for Nuclear Research) · Cited by 811
Lexical Diversity and Language Development
Malvern, D., Richards, B., Chipere, N., & Durán, P. (2004) · Palgrave Macmillan UK eBooks · doi:10.1057/9780230511804 · Cited by 494
A Corpus of English Conversation
Svartvik, J., & Quirk, R. (1980) · Medical Entomology and Zoology · Cited by 464
Statistical Techniques for the Study of Language and Language Behaviour
Rietveld, T., & Hout, R. V. (1993) · Mouton eBooks · doi:10.1515/9783110871609 · Cited by 441
The Statistical Study of Literary Vocabulary.
Rider, P. R., & Yule, G. U. (1944) · Journal of the American Statistical Association · doi:10.2307/2280636 · Cited by 438
How Variable May a Constant be? Measures of Lexical Richness in Perspective
Tweedie, F. J., & Baayen, R. H. (1998) · Computers and the Humanities · doi:10.1023/a:1001749303137 · Cited by 423
Johansson, S., Leech, G., & Goodluck, H. (1978) · Cited by 344
Lexical Richness in the Spontaneous Speech of Bilinguals
Daller, H. (2003) · Applied Linguistics · doi:10.1093/applin/24.2.197 · Cited by 284
Lexical Diversity and Language Development : Quantification and Assessment
Malvern, D. D., Richards, B., Chipere, N., & Durán, P. (2004) · Medical Entomology and Zoology · Cited by 283
Short texts, best-fitting curves and new measures of lexical diversity
Jarvis, S. (2002) · Language Testing · doi:10.1191/0265532202lt220oa · Cited by 281
Self-citationOn a Distribution Law for Word Frequencies
Sichel, H. S. (1975) · Journal of the American Statistical Association · doi:10.1080/01621459.1975.10482469 · Cited by 237
Measuring vocabulary diversity using dedicated software
McKee, G. (2000) · Literary and Linguistic Computing · doi:10.1093/llc/15.3.323 · Cited by 236
Bucks, R. S., Singh, S., Cuerden, J. M., & Wilcock, G. K. (2000) · Aphasiology · doi:10.1080/026870300401603 · Cited by 233
Developmental Trends in Lexical Diversity
Duran, P. (2004) · Applied Linguistics · doi:10.1093/applin/25.2.220 · Cited by 190
Fodor, J. A., & Carroll, J. B. (1965) · The Modern Language Journal · doi:10.2307/322944 · Cited by 164
P-Lex: A Simple and Effective Way of Describing the lexical Characteristics of Short L2 Tests.
Meara, P. M., & Bell, H. (2001) · Cited by 153
Owen, A. J., & Leonard, L. B. (2002) · Journal of Speech Language and Hearing Research · doi:10.1044/1092-4388(2002/075 · Cited by 136
IV. A statistical and comparative analysis of individual written language samples.
Chotlos, J. W. (1944) · The Psychological Monographs · doi:10.1037/h0093511 · Cited by 130
Another Perspective on Vocabulary Richness
Hoover, D. L. (2003) · Computers and the Humanities · doi:10.1023/a:1022673822140 · Cited by 128
POSNER, R. R., & Herdan, G. (1966) · Journal of the Royal Statistical Society Series A (General) · doi:10.2307/2343922 · Cited by 127
Interviewing techniques and the assessment of statement credibility
Colwell, K., Hiscock, C. K., & Memon, A. (2002) · Applied Cognitive Psychology · doi:10.1002/acp.788 · Cited by 117
Variation in Language and Cohesion across Written and Spoken Registers
Louwerse, M. M., McCarthy, P. M., McNamara, D. S., & Graesser, A. C. (2004) · Proceedings of the Annual Meeting of the Cognitive Science Society · Cited by 105
Self-citationSample size and type-token ratios for oral language of preschool children.
Hess, C. W., Sefton, K. M., & Landry, R. G. (1986) · Journal of Speech and Hearing Research · doi:10.1044/jshr.2901.129 · Cited by 96
The Statistical Study of Literary Vocabulary.
I., L., & Yule, G. (1944) · Journal Of The Royal Statistical Society · doi:10.2307/2981280 · Cited by 82
Herdan, G. (1961) · Language · doi:10.2307/411255 · Cited by 80
Parental Perceptions of Children's Communicative Development at Stuttering Onset
Ratner, N. B., & Silverman, S. W. (2000) · Journal of Speech Language and Hearing Research · doi:10.1044/jslhr.4305.1252 · Cited by 77
Measures of lexical diversity in aphasia
Wright, H. H., Wagovich, S. A., & Newhoff, M. (2003) · Aphasiology · doi:10.1080/02687030344000166 · Cited by 76
Thordardottir, E. T., & Weismer, S. E. (2001) · International Journal of Language & Communication Disorders · doi:10.1080/13682820118239 · Cited by 72
Patterns of parental vocabulary selection in speech to very young children
Ratner, N. B. (1988) · Journal of Child Language · doi:10.1017/s0305000900012514 · Cited by 67
A pilot study on gender differences in conversational speech on lexical in richness measures
Singh, S. (2001) · Literary and Linguistic Computing · doi:10.1093/llc/16.3.251 · Cited by 58
Learning Styles and Composition
Carrell, P. L., & Monroe, L. B. (1993) · Modern Language Journal · doi:10.1111/j.1540-4781.1993.tb01958.x · Cited by 54
Beginning to Communicate After Cochlear Implantation
Ertmer, D. J., Strong, L. M., & Sadagopan, N. (2003) · Journal of Speech Language and Hearing Research · doi:10.1044/1092-4388(2003/026 · Cited by 49
An accurate computation of the hypergeometric distribution function
Wu, T. (1993) · ACM Transactions on Mathematical Software · doi:10.1145/151271.151274 · Cited by 47
Lexical verb diversity in children with Down syndrome
Grela, B. G. (2002) · Clinical Linguistics & Phonetics · doi:10.1080/02699200210131987 · Cited by 42
Reciprocal scaffolding: A context for communication treatment in aphasia
Avent, J. R., & Hula, S. N. A. (2003) · Aphasiology · doi:10.1080/02687030244000743 · Cited by 41
The lexical richness of L2 written productions and the validity of vocabulary tests
Arnaud, P. J. L. (1984) · HAL · Cited by 34
A stylometric analysis of conversational speech of aphasic patients
Holmes, D., & Singh, S. (1996) · Literary and Linguistic Computing · doi:10.1093/llc/11.3.133 · Cited by 29
Socioeconomic and sociolinguistic predictors of children’s L2 and L1 writing quality
Ransdell, S., & Wengelin, Å. (2003) · NSUWorks (Nova Southeastern University) · Cited by 14
Cited by
Towards an Organic Approach to Investigating CAF in Instructed SLA: The Case of Complexity
Norris, J. M., & Ortega, L. (2009) · Applied Linguistics · doi:10.1093/applin/amp044 · Cited by 1,250
McCarthy, P. M., & Jarvis, S. (2010) · Behavior Research Methods · doi:10.3758/brm.42.2.381 · Cited by 905
Self-citationAutomated Evaluation of Text and Discourse with Coh-Metrix
McNamara, D. S., Graesser, A. C., McCarthy, P. M., & Cai, Z. (2014) · Cambridge University Press eBooks · doi:10.1017/cbo9780511894664 · Cited by 892
Self-citationCoh-Metrix: Providing Multilevel Analyses of Text Characteristics
Graesser, A. C., McNamara, D. S., & Kulikowich, J. M. (2011) · Educational Researcher · doi:10.3102/0013189x11413260 · Cited by 619
Linguistic Features of Writing Quality
McNamara, D. S., Crossley, S. A., & McCarthy, P. M. (2010) · Written Communication · doi:10.1177/0741088309351547 · Cited by 570
Self-citation
Metadata
- Authors
- McCarthy, P. M., & Jarvis, S.
- Venue
- Language Testing
- Publisher
- SAGE Publications
- Published
- 2007-10
- Volume
- 24
- Issue
- 4
- Pages
- 459-488
- ISSN
- 0265-5322 (ISSN-L)
- 1477-0946
- OpenAlex
- W2102229948
- Cited by
- 341
- References
- 46
- FWCI
- 7.88
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.