The DOI matches its Crossref record; 55 of the 55 references it cites were found in a registry or an index.
Automatically Assessing Lexical Sophistication: Indices, Tools, Findings, and Application
Abstract
This study explores the construct of lexical sophistication and its applications for measuring second language lexical and speaking proficiency. In doing so, the study introduces the Tool for the Automatic Analysis of LE xical Sophistication (TAALES), which calculates text scores for 135 classic and newly developed lexical indices related to word frequency, range, bigram and trigram frequency, academic language, and psycholinguistic word information. TAALES is freely available; runs on Windows, Mac, and Linux operating systems; and has a simple graphic user interface that allows for batch processing of .txt files. The tool is fast, reliable, and outputs results to a comma‐separated value file that can be accessed using spreadsheet software. The study examines the ability of TAALES indices to explain the variance in human judgments of lexical proficiency and speaking proficiency for second language (L2) learners. Overall, these indices were able to explain 47.5% of the variance in holistic scores of lexical proficiency and 48.7% of the variance in holistic scores of speaking proficiency. This study has important implications for second language acquisition, for assessing L2 learners' productive skills (writing and speaking), and for L2 pedagogy. Limitations and future directions are also discussed.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Witten, I. H. (2008) · Cited by 9,143
(1967) · Medical Entomology and Zoology · Cited by 7,532
Learning Vocabulary in Another Language
Nation, I. S. P. (2001) · Cambridge University Press · doi:10.1017/cbo9781139524759 · Cited by 6,608
Applied Multiple Regression/Correlation Analysis for the Behavioral Sciences.
Fong, G. T., Cohen, J., & Cohen, P. (1985) · Journal of the American Statistical Association · doi:10.2307/2287930 · Cited by 4,475
Brysbaert, M., & New, B. (2009) · Behavior Research Methods · doi:10.3758/brm.41.4.977 · Cited by 2,821
Coxhead, A. (2000) · TESOL Quarterly · doi:10.2307/3587951 · Cited by 2,780
The teacher's word book of 30,000 words.
Witty, P. (1945) · Journal of Consulting Psychology · doi:10.1037/h0050354 · Cited by 2,661
The Teacher's Word Book of 30, 000 Words
Thorndike, E. L., & Lorge, I. D. (2018) · Cited by 2,656
The MRC Psycholinguistic Database
Coltheart, M. (1981) · The Quarterly Journal of Experimental Psychology Section A · doi:10.1080/14640748108400805 · Cited by 2,437
Ellis, N. C. (2002) · Studies in Second Language Acquisition · doi:10.1017/s0272263102002024 · Cited by 2,230 · cited 1×
“First, current theories of language development (e.g., Ellis, 2002) suggest that as L2 learners develop, they will use less frequent lexical items.”
BackgroundConcreteness ratings for 40 thousand generally known English word lemmas
Brysbaert, M., Warriner, A. B., & Kuperman, V. (2014) · Behavior Research Methods · doi:10.3758/s13428-013-0403-5 · Cited by 1,821
Teaching and learning vocabulary
Nation, P. (1994) · System · doi:10.1016/0346-251x(94)90065-5 · Cited by 1,733
If you look at …: Lexical Bundles in University Teaching and Textbooks
Biber, D. (2004) · Applied Linguistics · doi:10.1093/applin/25.3.371 · Cited by 1,539
Coh-Metrix: Analysis of text on cohesion and language
Graesser, A. C., McNamara, D. S., Louwerse, M. M., & Cai, Z. (2004) · Behavior Research Methods, Instruments, & Computers · doi:10.3758/bf03195564 · Cited by 1,530
Vocabulary size and use: Lexical richness in l2 written production
LAUFER, B., & NATION, P. (1995) · Applied Linguistics · doi:10.1093/applin/16.3.307 · Cited by 1,316
Age-of-acquisition ratings for 30,000 English words
Kuperman, V., Stadthagen-Gonzalez, H., & Brysbaert, M. (2012) · Behavior Research Methods · doi:10.3758/s13428-012-0210-4 · Cited by 1,217
Developing and exploring the behaviour of two new versions of the Vocabulary Levels Test
Schmitt, N., Schmitt, D., & Clapham, C. (2001) · Language Testing · doi:10.1177/026553220101800103 · Cited by 1,211
Insights into Second Language Reading: A Cross-Linguistic Approach
Koda, K. (2005) · doi:10.1017/cbo9781139524841 · Cited by 1,007
McCarthy, P. M., & Jarvis, S. (2010) · Behavior Research Methods · doi:10.3758/brm.42.2.381 · Cited by 905
Automated Evaluation of Text and Discourse with Coh-Metrix
McNamara, D. S., Graesser, A. C., McCarthy, P. M., & Cai, Z. (2014) · Cambridge University Press eBooks · doi:10.1017/cbo9780511894664 · Cited by 892
An Academic Formulas List: New Methods in Phraseology Research
Simpson-Vlach, R., & Ellis, N. C. (2010) · Applied Linguistics · doi:10.1093/applin/amp058 · Cited by 863
Handbook of Semantic Word Norms
Underwood, B. J., Toglia, M. P., & Battig, W. F. (1979) · The American Journal of Psychology · doi:10.2307/1421579 · Cited by 669
AUTOMATED ESSAY SCORING WITH E‐RATER® V.2.0
Attali, Y., & Burstein, J. (2004) · ETS Research Report Series · doi:10.1002/j.2333-8504.2004.tb01972.x · Cited by 622
Linguistic Features of Writing Quality
McNamara, D. S., Crossley, S. A., & McCarthy, P. M. (2010) · Written Communication · doi:10.1177/0741088309351547 · Cited by 570
Self-citationDispersions and adjusted frequencies in corpora
Gries, S. T. (2008) · International Journal of Corpus Linguistics · doi:10.1075/ijcl.13.4.02gri · Cited by 536
A Corpus of English Conversation
Svartvik, J., & Quirk, R. (1980) · Medical Entomology and Zoology · Cited by 464
Predicting second language writing proficiency: the roles of cohesion and linguistic sophistication
Crossley, S. A., & McNamara, D. S. (2012) · Journal of Research in Reading · doi:10.1111/j.1467-9817.2010.01449.x · Cited by 331
Self-citationDisciplinary Discourses, Michigan Classics Ed.: Social Interactions in Academic Writing
Hyland, K. (2004) · doi:10.3998/mpub.6719 · Cited by 328
Assessing Text Readability Using Cognitively Based Indices
CROSSLEY, S. A., GREENFIELD, J., & McNAMARA, D. S. (2008) · TESOL Quarterly · doi:10.1002/j.1545-7249.2008.tb00142.x · Cited by 303
Self-citationPerformance and Competence in Second Language Acquisition
Brown, G., Malmkjær, K., & Williams, J. N. (1996) · Cited by 259
Computing the vocabulary demands of L2 reading
Cobb, T. (2007) · Language Learning & Technology · doi:10.64152/10125/44117 · Cited by 238
Lexical and Syntactic Features of ESL Writing by Students at Different Levels of L2 Proficiency
Ferris, D. R. (1994) · TESOL Quarterly · doi:10.2307/3587446 · Cited by 231
Predicting lexical proficiency in language learner texts using computational indices
Crossley, S. A., Salsbury, T., McNamara, D. S., & Jarvis, S. (2011) · Language Testing · doi:10.1177/0265532210378031 · Cited by 198
Self-citationThe Development of Writing Proficiency as a Function of Grade Level: A Linguistic Analysis
Crossley, S. A., Weston, J. L., McLain Sullivan, S. T., & McNamara, D. S. (2011) · Written Communication · doi:10.1177/0741088311410188 · Cited by 190
Self-citationThe Development of Polysemy and Frequency Use in English Second Language Speakers
Crossley, S., Salsbury, T., & McNamara, D. (2010) · Language Learning · doi:10.1111/j.1467-9922.2010.00568.x · Cited by 189
Self-citationThe Lexical Profile of Second Language Writing: Does It Change Over Time?
Laufer, B. (1994) · RELC Journal · doi:10.1177/003368829402500202 · Cited by 172
Developing and exploring the behaviour of two new versions of the Vocabulary Levels Test
Schmitt, N., Schmitt, D., & Clapham, C. (2001) · Language Testing · doi:10.1191/026553201668475857 · Cited by 167
The Academic Word List 10 Years On: Research and Teaching Implications
COXHEAD, A. (2011) · TESOL Quarterly · doi:10.5054/tq.2011.254528 · Cited by 163
Morris, L., & Cobb, T. (2003) · System · doi:10.1016/j.system.2003.05.001 · Cited by 149
A frequency count of 190,000 words in theLondon-Lund Corpus of English Conversation
Brown, G. D. A. (1984) · Behavior Research Methods, Instruments, & Computers · doi:10.3758/bf03200836 · Cited by 131
Lexical Frequency Profiles: A Monte Carlo Analysis
Meara, P. M. (2005) · Applied Linguistics · doi:10.1093/applin/amh037 · Cited by 123
Hand-Book of Semantic Word Norms
Keppel, G., Toglia, M. P., Battig, W. F., Barrow, K., Cartwright, D. S., Posnansky, C. J., Pellegrino, J. W., Moore, T. J., & Camilli, G. (1979) · Modern Language Journal · doi:10.2307/324239 · Cited by 120
Coh-Metrix: An Automated Tool for Theoretical and Applied Natural Language Processing
McNamara, D. S., & Graesser, A. C. (2012) · IGI Global eBooks · doi:10.4018/978-1-60960-741-8.ch011 · Cited by 111
Crossley, S. A., Cobb, T., & McNamara, D. S. (2013) · System · doi:10.1016/j.system.2013.08.002 · Cited by 110
Self-citationValidation in Language Assessment
(2013) · Routledge eBooks · doi:10.4324/9780203053768 · Cited by 101
Psycholinguistic word information in second language oral discourse
Salsbury, T., Crossley, S. A., & McNamara, D. S. (2011) · Second language Research · doi:10.1177/0267658310395851 · Cited by 97
Self-citationApplications of Text Analysis Tools for Spoken Response Grading
Crossley, S. A., & McNamara, D. S. (2013) · Language learning & technology · doi:10.64152/10125/44329 · Cited by 89
Self-citationWhat Is Lexical Proficiency? Some Answers From Computational Models of Speech Data
Crossley, S. A., Salsbury, T., McNamara, D. S., & Jarvis, S. (2011) · TESOL Quarterly · doi:10.5054/tq.2010.244019 · Cited by 78
Self-citationApplied Natural Language Processing: Identification, Investigation and Resolution
Hassel, M., & Dalianis, H. (2011) · IGI Global eBooks · doi:10.4018/978-1-60960-741-8 · Cited by 77
Crossley, S. A., Subtirelu, N. C., & Salsbury, T. (2013) · Studies in Second Language Acquisition · doi:10.1017/s0272263113000375 · Cited by 75
Self-citationApproaching Language Transfer through Text Classification
(2012) · Multilingual Matters eBooks · doi:10.21832/9781847696991 · Cited by 63
What automated analyses of corpora can tell us about students’ writing skills
Deane, P., & Quinlan, T. (2010) · Journal of Writing Research · doi:10.17239/jowr-2010.02.02.4 · Cited by 43
Crossley, S. A., Clevinger, A. M., & Kim, Y. (2014) · Language Assessment Quarterly · doi:10.1080/15434303.2014.926905 · Cited by 42
Self-citationJarvis, S., & Crossley, S. A. (2012) · Cited by 41
Self-citationNative Language Identification: A Key N-gram Category Approach
Kyle, K. D., Crossley, S. A., Dai, J., & McNamara, D. S. (2013) · Cited by 7
Self-citation
Cited by
Research Methods in Applied Linguistics
(2021) · Research methods in applied linguistics · doi:10.1075/rmal · Cited by 2,793
Vocabulary in Language Teaching
Schmitt, N., & Schmitt, D. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108569057 · Cited by 878
Applications of deep learning in stock market prediction: Recent progress
Jiang, W. (2021) · Expert Systems with Applications · doi:10.1016/j.eswa.2021.115537 · Cited by 714
Exploring the potential of using an AI language model for automated essay scoring
Mizumoto, A., & Eguchi, M. (2023) · Research Methods in Applied Linguistics · doi:10.1016/j.rmal.2023.100050 · Cited by 647
The tool for the automatic analysis of lexical sophistication (TAALES): version 2.0
Kyle, K., Crossley, S., & Berger, C. (2018) · Behavior Research Methods · doi:10.3758/s13428-017-0924-4 · Cited by 335
Self-citation
Metadata
- Authors
- Kyle, K., & Crossley, S. A.
- Venue
- TESOL Quarterly
- Publisher
- Wiley
- Published
- 2014-09-25
- Volume
- 49
- Issue
- 4
- Pages
- 757-786
- ISSN
- 0039-8322 (ISSN-L)
- 1545-7249
- OpenAlex
- W2034849674
- Cited by
- 557
- References
- 55
- FWCI
- 45.19
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.