CoChat
Sign Up
Cited by 909Computational and Text Analysis MethodsLinguistic research and analysisAuthorship Attribution and Profiling

MTLD, vocd-D, and HD-D: A validation study of sophisticated approaches to lexical diversity assessment

McCarthy, Philip M., Jarvis, Scott·2010·Behavior Research Methods
DOI 10.3758/brm.42.2.381
Open this paper in CoChat — free account

Abstract

The main purpose of this study was to examine the validity of the approach to lexical diversity assessment known as the measure of textual lexical diversity (MTLD). The index for this approach is calculated as the mean length of word strings that maintain a criterion level of lexical variation. To validate the MTLD approach, we compared it against the performances of the primary competing indices in the field, which include vocd-D, TTR, Maas, Yule's K, and an HD-D index derived directly from the hypergeometric distribution function. The comparisons involved assessments of convergent validity, divergent validity, internal validity, and incremental validity. The results of our assessments of these indices across two separate corpora suggest three major findings. First, MTLD performs well with respect to all four types of validity and is, in fact, the only index not found to vary as a function of text length. Second, HD-D is a viable alternative to the vocd-D standard. And third, three of the indices--MTLD, vocd-D (or HD-D), and Maas--appear to capture unique lexical information. We conclude by advising researchers to consider using MTLD, vocd-D (or HD-D), and Maas in their studies, rather than any single index, noting that lexical diversity can be assessed in many ways and each approach may be informative as to the construct under investigation.

Integrity signals

Registry

Crossref · 10.3758/brm.42.2.381

Cites

46 of 46 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • MTLD is a well-established measure of lexical diversity that is less sensitive to text length differences than TTR.

    “We identify vocabulary richness by using a well-established measure of textual, lexical diversity (MTLD)46 which is often used in the field of automated essay grading25,45,47. It takes into account the number of unique words but unlike the best-known measure of lexical diversity, the type-token ratio (TTR), it is not as sensitive to the difference in the length of the texts.”

    From A large-scale comparison of human-written versus ChatGPT-generated essays (2023) · cites this paper as [46]

  • Moon et al. found that in a corpus of college admissions essays, each additional human-written essay introduced more novel ideas than each additional GPT-4-generated essay.

    “Particularly relevant to the present study is Moon et al. 42 , who found that in a corpus of college admissions essays, each additional humanwritten essay introduced more novel ideas than each additional GPT-4-generated essay.”

    From Style, sentiment, and quality of undergraduate writing in the AI era: A cross-sectional and longitudinal analysis of 4,820 authentic empirical reports (2025) · cites this paper as [42]

Cites

Flagged first, then by citations

4 of the 50 references couldn’t be matched to a known paper.

Cited by

Sorted by citations

Related

Authors
McCarthy, P. M., & Jarvis, S.
Affiliations
  • University of Memphis (US)
  • Ohio University (US)
Corresponding author
McCarthy, Philip M.
ORCID
Venue
Behavior Research Methods
Publisher
Springer Science and Business Media LLC
Published
2010-05
Volume
42
Issue
2
Pages
381-392
ISSN
  • 1554-351X (ISSN-L)
  • 1554-3528
DOI
10.3758/brm.42.2.381
OpenAlex
W2087388117
Semantic Scholar
70085f28eb99e5bbba6f5abf7ea964f17eba26ea
Cited by
905
References
50
Citations by year
2026: 147 · 2025: 103 · 2024: 85 · 2023: 82 · 2022: 92 · 2021: 70 · 2020: 71 · 2019: 53 · 2018: 24 · 2017: 35 · 2016: 38 · 2015: 26 · 2014: 27 · 2013: 17 · 2012: 23
FWCI
128.67
Type
journal article
Language
en
Open access
closed
Licence
Locations
  • Behavior Research Methods (publishedVersion)
  • PubMed (publishedVersion)
Topics
  • Social Sciences › General Social Sciences › Computational and Text Analysis Methods
  • Social Sciences › Arts and Humanities › Language and Linguistics › Linguistic research and analysis
  • Physical Sciences › Computer Science › Artificial Intelligence › Authorship Attribution and Profiling
Keywords
lexical diversity, hypergeometric distribution, convergent validity, discriminant validity, incremental validity
MeSH terms
Humans, Language Tests, Psychology, Experimental, Reproducibility of Results

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review