CoChat
Sign Up
Cited by 83Second Language Acquisition and LearningEFL/ESL Teaching and LearningLanguage Development and Disorders

MEASURING LONGITUDINAL WRITING DEVELOPMENT USING INDICES OF SYNTACTIC COMPLEXITY AND SOPHISTICATION

Open access
Kyle, Kristopher, Crossley, Scott, Verspoor, Marjolijn·2021·Studies in Second Language Acquisition
DOI 10.1017/s0272263120000546
Open this paper in CoChat — free accountOpen PDF

Abstract

Measures of syntactic complexity such as mean length of T-unit have been common measures of language proficiency in studies of second language acquisition. Despite the ubiquity and usefulness of such structure-based measures, they could be complemented with measures based on usage-based theories, which focus on the development of not just syntactic forms but also form-meaning pairs, called constructions (Ellis, 2002). Recent cross-sectional research (Kyle & Crossley, 2017) has indicated that indices related to usage-based characteristics of verb argument construction (VAC) use may be better indicators of writing proficiency than structure-based indices of syntactic complexity. However, because cross-sectional studies can only show general trends across proficiency benchmarks, it is important to test these findings in individuals over time (Lowie & Verspoor, 2019). Thus, this study investigates the developmental trajectories of second language learners of English across two academic years with regard to syntactic complexity and VAC sophistication.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1017/s0272263120000546

Cites

4 of 4 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • EFL writers tend to employ a narrower vocabulary range, favor high-frequency items, and produce lower scores on established lexical diversity measures compared to more proficient or native speakers.

    “EFL writers tend to employ a narrower vocabulary range, favor high-frequency items, and produce lower scores on established lexical diversity measures compared to more proficient or native speakers”

    From ARE AI DETECTORS FAIR TO EFL LEARNERS? A LIBRARY RESEARCH ON DETECTION BIAS AND ACADEMIC INTEGRITY (2026) · cites this paper as Kyle et al., 2021

    The VAC SOPHISTICATION section says more proficient L2 users tend to use a higher proportion of low-frequency words, supporting the claim about word frequency. It does not establish that EFL writers have a narrower vocabulary range or lower scores on lexical-diversity measures than more proficient or native speakers. The paper's longitudinal findings concern syntactic complexity and verb-argument-construction sophistication, not those lexical-diversity comparisons. (LLM, conf=0.87)

    Experimental — accuracy not yet measured

  • One empirically observable property of interlanguage writing is its constrained lexical and syntactic profile.

    “One of the empirically observable properties of interlanguage writing is its constrained lexical and syntactic profile.”

    From ARE AI DETECTORS FAIR TO EFL LEARNERS? A LIBRARY RESEARCH ON DETECTION BIAS AND ACADEMIC INTEGRITY (2026) · cites this paper as Kyle et al., 2021

    The Conclusion (page unavailable) reports observable lexical and syntactic differences as learners developed: they used more dependent clauses and less frequent main verbs over time. This supports the idea that learner writing has a measurable lexical and syntactic profile, but the paper does not establish that interlanguage writing generally has a constrained profile across proficiency levels. (LLM, conf=0.78)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 41

  1. Bestgen, Y., & Granger, S. (2014). Quantifying the development of phraseological competence in L2 English writing: An automated approach. Journal of Second Language Writing, 26, 28-41. https://doi.org/10.1016/ j.jslw.2014.09.004. · DOI
  2. Biber, D., Gray, B., & Poonpon, K. (2011). Should we use characteristics of conversation to measure grammatical complexity in L2 writing development? TESOL Quarterly, 45, 5-35.
  3. Biber, D., Gray, B., & Staples, S. (2014). Predicting patterns of grammatical complexity across language exam task types and proficiency levels. Applied Linguistics, 37, 639-668. https://doi.org/10.1093/applin/amu059. · DOI · Paper page
  4. Bulté, B., & Housen, A. (2014). Conceptualizing and measuring short-term changes in L2 writing complexity. Journal of Second Language Writing, 26, 42-65. https://doi.org/10.1016/j.jslw.2014.09.005. · DOI · Paper page
  5. Bulté, B., & Housen, A. (2018). Syntactic complexity in L2 writing: Individual pathways and emerging group trends. International Journal of Applied Linguistics, 28, 147-164. https://doi.org/10.1111/ijal.12196. · DOI
  6. Crossley, S. A., Kyle, K., & Salsbury, T. (2016). A usage-based investigation of L2 lexical acquisition: The role of input and output. The Modern Language Journal, 100, 702-715.
  7. Crossley, S. A., & McNamara, D. S. (2014). Does writing development equal writing quality? A computational investigation of syntactic complexity in L2 learners. Journal of Second Language Writing, 26, 66-79. https:// doi.org/10.1016/j.jslw.2014.09.006. · DOI
  8. Crossley, S. A., Salsbury, T., & McNamara, D. (2010). The development of polysemy and frequency use in English second language speakers. Language Learning, 60, 573-605. https://doi.org/10.1111/j.1467- 9922.2010.00568.x. · DOI
  9. Crymes, R. (1971). The relation of study about language to language performance: With special reference to nominalization. TESOL Quarterly, 5, 217-230. Measuring Longitudinal Writing Development Using Indices 809 use, available at https://www.cambridge.org/core/terms. https://doi.org/10.1017/S0272263120000546 Downloaded from https://www.cambridge.org/core. University of Groningen, on 20 Oct 2021 at 09:57:26, subject to the Cambridge Core terms of
  10. Davies, M. (2009). The 385+ million word Corpus of Contemporary American English (1990-2008+): Design, architecture, and linguistic insights. International Journal of Corpus Linguistics, 14, 159-190. https:// doi.org/10.1075/ijcl.14.2.02dav. · DOI
  11. de Bot, K., Lowie, V., Thorne, S., & Verspoor, M. (2013). Dynamic systems theory as a comprehensive theory of second language development. In M. P. G. Mayo, M. J. G. Mangado, & M. M. Adrián (Eds.), Contemporary approaches to second language acquisition (pp. 199-220).
  12. Durrant, P., & Schmitt, N. (2009). To what extent do native and non-native writers make use of collocations? IRAL-International Review of Applied Linguistics in Language Teaching, 47, 157-177.
  13. Ellis, N. C., & Ferreira-Junior, F. (2009). Constructions and their acquisition: Islands and the distinctiveness of their occupancy. Annual Review of Cognitive Linguistics, 7, 188-221. https://doi.org/10.1075/arcl.7.08ell. · DOI
  14. Eskildsen, S. W. (2009). Constructing another language-Usage-based linguistics in second language acqui- sition. Applied Linguistics, 30, 335-357. https://doi.org/10.1093/applin/amn037. · DOI
  15. Fisher, R. A. (1934). Two new properties of mathematical likelihood. Proceedings of the Royal Society of London. Series A, Containing Papers of a Mathematical and Physical Character, 144, 285-307.
  16. Garner, J., Crossley, S., & Kyle, K. (2019). N-gram measures and L2 writing proficiency. System, 80, 176-187. https://doi.org/10.1016/j.system.2018.12.001. · DOI
  17. Goldberg, A. E. (2006). Constructions at work: The nature of generalization in language. Oxford University Press.
  18. Gries, S. T., Hampe, B., & Schönefeld, D. (2005). Converging evidence: Bringing together experimental and corpus data on the association of verbs and constructions. Cognitive Linguistics, 16, 635-676. https://doi.org/ 10.1515/cogl.2005.16.4.635. · DOI
  19. Grömping, U. (2006). Relative importance for linear regression in R: The package relaimpo. Journal of Statistical Software, 17, 1-27.
  20. Knoch, U., Rouhshad, A., & Storch, N. (2014). Does the writing of undergraduate ESL students develop after one year of study in an English-medium university? Assessing Writing, 21, 1-17. https://doi.org/10.1016/ j.asw.2014.01.001. · DOI
  21. Kyle, K., & Crossley, S. A. (2015). Automatically assessing lexical sophistication: Indices, tools, findings, and application. TESOL Quarterly, 49, 757-786. https://doi.org/10.1002/tesq.194. · DOI
  22. Larsen-Freeman, D. (1978). An ESL index of development. TESOL Quarterly, 12, 439-448. https://doi.org/ 10.2307/3586142. · DOI
  23. Larsen-Freeman, D., & Cameron, L. (2008). Research methodology on language development from a complex systems perspective. The Modern Language Journal, 92, 200-213.
  24. Laufer, B., & Nation, I. S. P. (1995). Vocabulary size and use: Lexical richness in L2 written production. Applied Linguistics, 16, 307-322. https://doi.org/10.1093/applin/16.3.307. · DOI · Paper page
  25. Lu, X. (2010). Automatic analysis of syntactic complexity in second language writing. International Journal of Corpus Linguistics, 15, 474-496. https://doi.org/10.1075/ijcl.15.4.02lu Lu, X. (2011). A corpus-based evaluation of syntactic complexity measures as indices of college-level ESL writers' language development. TESOL Quarterly, 45, 36-62. · DOI
  26. Murakami, A., & Alexopoulou, T. (2016). Longitudinal L2 development of the English article in individual learners. In D. Papafragou, D. Grodner, D. Mirman, & J. Trueswell (Eds.), Proceedings of the 38th Annual Meeting of the Cognitive Science Society (pp. 1050-1055). Cognitive Science Society.
  27. Ninio, A. (1999). Pathbreaking verbs in syntactic development and the question of prototypical transitivity. Journal of Child Language, 26, 619-653.
  28. Norris, J. M., & Ortega, L. (2009). Towards an organic approach to investigating CAF in instructed SLA: The case of complexity. Applied Linguistics, 30. https://doi.org/10.1093/applin/amp044. · DOI
  29. O'Donnell, M. B., & Ellis, N. (2010). Towards an inventory of English verb argument constructions (pp. 9-16). http://dl.acm.org/citation.cfm?id=1866732.1866734.
  30. Penris, W., & Verspoor, M. (2017). Academic writing development: A complex, dynamic process. In S. Pfenniger & Navracsics (Eds.), Future Research Directions for Applied Linguistics (Vol. 109, pp. 215-242). Multilingual Matters Ltd.
  31. Römer, U., O'Donnell, M. B., & Ellis, N. C. (2015). Using COBUILD grammar patterns for a large-scale analysis of verb-argument constructions. N. Groom, M. Charles, & J. Suganthi (Eds.), (Vol. 73, p. 43). John Benjamins Publishing Company.
  32. Römer, U., Roberson, A., O'Donnell, M. B., & Ellis, N. C. (2014). Linking learner corpus and experimental data in studying second language learners' knowledge of verb-argument constructions. ICAME Journal, 38, 115-135.
  33. Thornhill, D. E. (1969). A quantitative analysis of the development of syntactical fluency of four young adult Spanish speakers learning English [Unpublished doctoral dissertation]. The Florida State University.
  34. Verspoor, M. (2017). Complex Dynamic Systems Theory and L2 pedagogy. In L. Ortega & Z. Han (Eds.), Complexity Theory and Language Development: In celebration of Diane Larsen-Freeman (Vol. 48, pp. 143-162).
  35. Verspoor, M., & Behrens, H. (2011). Dynamic systems theory and a usage-based approach to second language development. A dynamic approach to second language development: Methods and techniques (pp. 25-38).
  36. Verspoor, M., de Bot, K., & Xu, X. (2015). The effects of English bilingual education in the Netherlands. Journal of Immersion and Content-Based Language Education, 3, 4-27.
  37. Verspoor, M., Schmid, M. S., & Xu, X. (2012). A dynamic usage based perspective on L2 writing. Journal of Second Language Writing, 21, 239-263. https://doi.org/10.1016/j.jslw.2012.03.007. · DOI
  38. Verspoor, M., & Smiskova, H. (2012). Foreign language writing development from a dynamic usage-based perspective. In R. M. Manchón (Ed.), L2 writing development: Multiple perspectives (pp. 17-46).
  39. Vyatkina, N. (2012). The development of second language writing complexity in groups and individuals: A longitudinal learner corpus study. The Modern Language Journal, 96, 576-598.
  40. Wolfe-Quintero, K., Inagaki, S., & Kim, H.-Y. (1998). Second language development in writing: Measures of fluency, accuracy & complexity. University of Hawaii Press.
  41. Yates, F. (1934). Contingency tables involving small numbers and the χ2 test. Supplement to the Journal of the Royal Statistical Society, 1, 217-235. https://doi.org/10.2307/2983604. · DOI

Cited by

Sorted by citations

Authors
Kyle, K., Crossley, S., & Verspoor, M.
Affiliations
  • University of Oregon (US)
  • Georgia State University (US)
  • University of Groningen (NL)
Corresponding author
  • Kyle, Kristopher
  • Crossley, Scott
  • Verspoor, Marjolijn
ORCID
Venue
Studies in Second Language Acquisition
Publisher
Cambridge University Press (CUP)
Published
2020-10-27
Volume
43
Issue
4
Pages
781-812
ISSN
  • 0272-2631 (ISSN-L)
  • 1470-1545
DOI
10.1017/s0272263120000546
OpenAlex
W3097478885
Cited by
83
References
4
Citations by year
2026: 18 · 2025: 15 · 2024: 22 · 2023: 16 · 2022: 11 · 2020: 1
FWCI
10.57
Type
journal article
Language
en
Open access
Licence
www.cambridge.org/core/terms (unspecified)
Locations
  • Studies in Second Language Acquisition (publishedVersion)
  • University of Groningen research database (University of Groningen / Centre for Information Technology) (submittedVersion, other-oa)
Topics
  • Social Sciences › Psychology › Developmental and Educational Psychology › Second Language Acquisition and Learning
  • Social Sciences › Arts and Humanities › Language and Linguistics › EFL/ESL Teaching and Learning
  • Social Sciences › Psychology › Developmental and Educational Psychology › Language Development and Disorders
Keywords
syntactic complexity, construction grammar, usage-based theory, second language writing, writing skills, developmental trajectories, longitudinal development, L2 English learners

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review