Predicting patterns of grammatical complexity across language exam task types and proficiency levels

Abstract

In the present article, we explore the extent to which previous research on register variation can be used to predict spoken/written task-type variation as well as differences across score levels in the context of a major standardized language exam (TOEFL iBT). Specifically, we carry out two sets of linguistic analyses based on a large corpus of TOEFL iBT responses: one investigating the use of 23 grammatical complexity features, and the second based on co-occurrence patterns among linguistic features, using Multi-Dimensional (MD) analysis. The first set of analyses confirms the predictions from research on register variation: there are systematic linguistic differences among spoken versus written and independent versus integrated task types. However, hypothesized developmental progressions in the use of these grammatical complexity features were generally not confirmed by score-level differences. In contrast, the MD analysis yielded more robust predictors of both task types and score-level differences, indicating that linguistic descriptions are more reliable and informative when they are based on dimensions of co-occurring lexico-grammatical features. In conclusion, we discuss the application of such dimensions as holistic complexity measures in language development/testing research.