The DOI matches its Crossref record; 71 of the 71 references it cites were found in a registry or an index.
Feedforwarding diagnostic language assessment: Artificial intelligence- (AI-) driven weakness identification and contextualised feedback for second language speaking
Open accessAbstract
The current study proposes a new approach to weakness identification in diagnostic language assessment (DLA) for speaking skills. We also propose to design actionable and contextualised diagnostic feedback through the systematic integration of feedback and remedial learning activities. Focusing on lexical use in second language speaking, the current study developed and validated our DLA programme in terms of actual learning gains, using an experimental design. A total of 59 beginner-to-intermediate-level Japanese learners of English were randomly assigned to control or experimental groups. While both groups engaged in task repetition with a conversational artificial intelligence (AI) agent on six occasions, only the experimental group received the diagnostic feedback on lexical use including the paraphrased utterances of their original utterance. The results showed that the control group (task repetition only) demonstrated significant improvement during the task repetition sessions but failed to transfer and retain the learning gains. In contrast, despite the lack of practice effects, the experimental group (task repetition with diagnostic feedback) outperformed the control group at the posttest with a near-medium effect size. A qualitative investigation into learners’ perceptions further confirmed that the proposed contextualised diagnostic feedback succeeded in heightening their awareness of weaknesses.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
R: A Language and Environment for Statistical Computing
R Core Team (2025) · doi:10.32614/r.manuals · Cited by 352,181
Fitting Linear Mixed-Effects Models Using lme4
Bates, D., Mächler, M., Bolker, B., & Walker, S. (2015) · Journal of Statistical Software · doi:10.18637/jss.v067.i01 · Cited by 86,835
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
A Unified Approach to Interpreting Model Predictions
Lundberg, S., & Lee, S.-I. (2017) · arXiv · doi:10.48550/arxiv.1705.07874 · Cited by 23,707
17. A Value for n-Person Games
Shapley, L. S. (1953) · Contributions to the Theory of Games (AM-28), Volume II · doi:10.1515/9781400881970-018 · Cited by 6,950
How Big Is “Big”? Interpreting Effect Sizes in L2 Research
Plonsky, L., & Oswald, F. L. (2014) · Language Learning · doi:10.1111/lang.12079 · Cited by 1,960
(2014) · Cited by 1,210
Memory and Metamemory Considerations in the Training of Human Beings
Bjork, R. A. (1994) · Metacognition · doi:10.7551/mitpress/4561.003.0011 · Cited by 1,084
Bachman, L. F., & Palmer, A. S. (2010) · Cited by 1,008
Cognitive Bases of Second Language Fluency
Segalowitz, N. (2010) · doi:10.4324/9780203851357 · Cited by 881
Explainable AI (XAI): A systematic meta-survey of current challenges and future opportunities
Saeed, W., & Omlin, C. (2023) · Knowledge-Based Systems · doi:10.1016/j.knosys.2023.110273 · Cited by 783
Engagement in language learning: a systematic review of 20 years of research methods and definitions
Hiver, P., Al-Hoorie, A. H., Vitta, J. P., & Wu, J. (2024) · Language Teaching Research · doi:10.1177/13621688211001289 · Cited by 677 · cited 1×
“Following the literature of learner engagement (Hiver et al., 2021), we analysed their speech data in terms of the number of words produced and the mean length of turns in words as a proxy for behavioural engagement.”
Spacing Effects in Learning: A Temporal Ridgeline of Optimal Retention
Cepeda, N. J., Vul, E., Rohrer, D., Wixted, J. T., & Pashler, H. (2008) · Psychological Science · doi:10.1111/j.1467-9280.2008.02209.x · Cited by 631
Diagnosing Foreign Language Proficiency : The Interface between Learning and Assessment
(2005) · Bloomsbury Academic eBooks · doi:10.5040/9781474212151 · Cited by 570
Dimensions of L2 Performance and Proficiency: Complexity, Accuracy and Fluency in SLA
(2012) · Language Learning & Language Teaching · doi:10.1075/lllt.32 · Cited by 452
Speech Production and Second Language Acquisition
Kormos, J. (2014) · Routledge eBooks · doi:10.4324/9780203763964 · Cited by 438
Speech production and second language acquisition
Sunderman, G. (2009) · International Journal of Bilingual Education and Bilingualism · doi:10.1080/13670050802149382 · Cited by 349
Selvi, A. F. (2019) · The Routledge Handbook of Research Methods in Applied Linguistics · doi:10.4324/9780367824471-37 · Cited by 293
TASK REPETITION AND SECOND LANGUAGE SPEECH PROCESSING
Lambert, C., Kormos, J., & Minn, D. (2017) · Studies in Second Language Acquisition · doi:10.1017/s0272263116000085 · Cited by 281 · cited 1×
“ISLA research has suggested that task repetition can facilitate the consolidation of linguistic knowledge (Kakitani & Kormos, 2024; Lambert et al., 2017; Y. Suzuki & Hanzawa, 2022).”
FACETS OF SPEAKING PROFICIENCY
de Jong, N. H., Steinel, M. P., Florijn, A. F., Schoonen, R., & Hulstijn, J. H. (2012) · Studies in Second Language Acquisition · doi:10.1017/s0272263111000489 · Cited by 265
Sato, M., & Lyster, R. (2012) · Studies in Second Language Acquisition · doi:10.1017/s0272263112000356 · Cited by 260
Common European Framework of Reference for Languages
Little, D. (2020) · The TESOL Encyclopedia of English Language Teaching · doi:10.1002/9781118784235.eelt0114.pub2 · Cited by 255
de Jong, N., & Perfetti, C. A. (2011) · Language Learning · doi:10.1111/j.1467-9922.2010.00620.x · Cited by 250 · cited 1×
“Exact task repetition generally yields greater learning gains (N. de Jong & Perfetti, 2011; Y. Kim & Tracy-Ventura, 2013).”
THE PSYCHOLOGY OF FOREIGN LANGUAGE VOCABULARY ACQUISITION: IMPLICATIONS FOR CALL
Ellis, N. C. (1995) · Computer Assisted Language Learning · doi:10.1080/0958822940080202 · Cited by 222
Kim, Y., & Tracy-Ventura, N. (2013) · System · doi:10.1016/j.system.2013.08.005 · Cited by 199 · cited 1×
“Exact task repetition generally yields greater learning gains (N. de Jong & Perfetti, 2011; Y. Kim & Tracy-Ventura, 2013).”
Harding, L., Alderson, J. C., & Brunfaut, T. (2015) · Language Testing · doi:10.1177/0265532214564505 · Cited by 194
Alderson, J. C., Brunfaut, T., & Harding, L. (2014) · Applied Linguistics · doi:10.1093/applin/amt046 · Cited by 171
The Effects of Complexity, Accuracy, and Fluency on Communicative Adequacy in Oral Task Performance
Revesz, A., Ekiert, M., & Torgersen, E. N. (2014) · Applied Linguistics · doi:10.1093/applin/amu069 · Cited by 161
Increasing Retention Without Increasing Study Time
Rohrer, D., & Pashler, H. (2007) · Current Directions in Psychological Science · doi:10.1111/j.1467-8721.2007.00500.x · Cited by 153
Automated Scoring of Nonnative Speech Using the SpeechRaterSMv. 5.0 Engine
Chen, L., Zechner, K., Yoon, S., Evanini, K., Wang, X., Loukina, A., Tao, J., Davis, L., Lee, C., Ma, M., et al. (2018) · ETS Research Report Series · doi:10.1002/ets2.12198 · Cited by 152
The Cambridge Handbook of Cognition and Education
Dunlosky, J. T., & Rawson, K. A. (2019) · Cambridge University Press eBooks · doi:10.1017/9781108235631 · Cited by 150
Saito, K. (2021) · TESOL Quarterly · doi:10.1002/tesq.3027 · Cited by 142
Oral Fluency: The Neglected Component in the Communicative Language Classroom
Rossiter, M. J., Derwing, T. M., Manimtim, L. G., & Thomson, R. I. (2010) · Canadian Modern Language Review/ La Revue canadienne des langues vivantes · doi:10.3138/cmlr.66.4.583 · Cited by 134
Xie, Q., & Andrews, S. (2012) · Language Testing · doi:10.1177/0265532212442634 · Cited by 134
Suzuki, S., & Kormos, J. (2022) · Studies in Second Language Acquisition · doi:10.1017/s0272263121000899 · Cited by 128
Self-citationExperimental Research Methods In Language Learning
Phakiti, A. (2018) · Cited by 126
The Relationship Between Utterance and Perceived Fluency: A Meta‐Analysis of Correlational Studies
Suzuki, S., Kormos, J., & Uchihara, T. (2021) · Modern Language Journal · doi:10.1111/modl.12706 · Cited by 126
Self-citationOral proficiency training in Dutch L2: The contribution of ASR-based corrective feedback
Catia Cucchiarini, A., Neri, A., & Strik, H. (2009) · Speech Communication · doi:10.1016/j.specom.2009.03.003 · Cited by 116
A1–B2 vocabulary: insights and issues arising from the English Profile Wordlists project
Capel, A. (2010) · English Profile Journal · doi:10.1017/s2041536210000048 · Cited by 116
Diagnosing diagnostic language assessment
Lee, Y. (2015) · Language Testing · doi:10.1177/0265532214565387 · Cited by 111
Spoken grammar practice and feedback in an ASR-based CALL system
de Vries, B. P., Cucchiarini, C., Bodnar, S., Strik, H., & van Hout, R. W. N. M. (2014) · Computer Assisted Language Learning · doi:10.1080/09588221.2014.889713 · Cited by 106
Completing the English Vocabulary Profile: C1 and C2 vocabulary
Capel, A. (2012) · English Profile Journal · doi:10.1017/s2041536212000013 · Cited by 100
Human versus Computer Partner in the Paired Oral Discussion Test
Ockey, G. J., & Chukharev‐Hudilainen, E. (2020) · Applied Linguistics · doi:10.1093/applin/amaa067 · Cited by 88
Jang, E. E., Dunlop, M., Park, G., & van der Boom, E. H. (2015) · Language Testing · doi:10.1177/0265532215570924 · Cited by 77
Yokoi, S., Takahashi, R., Akama, R., Suzuki, J., & Inui, K. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.236 · Cited by 76
Understanding Learner Strengths and Weaknesses: Assessing Performance on an Integrated Writing Task
Sawaki, Y., Quinlan, T., & Lee, Y. W. (2013) · Language Assessment Quarterly · doi:10.1080/15434303.2011.633305 · Cited by 74
Revising the revised format of the ACTFL Oral Proficiency Interview
Salaberry, M. R. (2000) · Language Testing · doi:10.1177/026553220001700301 · Cited by 71
SUZUKI, Y., NAKATA, T., & DEKEYSER, R. (2019) · The Modern Language Journal · doi:10.1111/modl.12585 · Cited by 71
Diagnosing EAP writing ability using the Reduced Reparameterized Unified Model
Kim, Y. (2011) · Language Testing · doi:10.1177/0265532211400860 · Cited by 70
MASSED TASK REPETITION IS A DOUBLE-EDGED SWORD FOR FLUENCY DEVELOPMENT: AN EFL CLASSROOM STUDY
Suzuki, Y., & Hanzawa, K. (2022) · Studies in Second Language Acquisition · doi:10.1017/s0272263121000358 · Cited by 59 · cited 5×
“To capture the learning gains comprehensively, the current study followed recent L2 speaking task repetition research (Kakitani & Kormos, 2024; Y. Suzuki & Hanzawa, 2022), assessing the improvement (a) during the practice sessions, (b) after the sessions (i.e., retention) and (c) to a new task…”
InfluentialEnhancing the Quality of Student Learning Using Distributed Practice
Wiseheart, M., Küpper-Tetzel, Weston, T., Kim, A. S. N., Kapler, I. V., & Foot‐Seymour, V. (2019) · Cambridge University Press eBooks · doi:10.1017/9781108235631.023 · Cited by 53
Focus on form through task repetition in TBLT
van de Guchte, M., Braaksma, M., Rijlaarsdam, G., & Bimmel, P. (2016) · Language Teaching Research · doi:10.1177/1362168815609616 · Cited by 50 · cited 1×
“…activities and techniques, such as corrective feedback and consciousness-raising tasks, can be combined to direct learners’ attention to certain linguistic forms so that multiple aspects of speaking performance can be enhanced simultaneously (Tran & Saito, 2021; van de Guchte et al., 2016).”
Saito, K., Suzuki, S., Oyama, T., & Akiyama, Y. (2019) · Second language Research · doi:10.1177/0267658319884981 · Cited by 48
Self-citationAssessing L2 English speaking using automated scoring technology: examining automarker reliability
Xu, J., Jones, E., Laxton, V., & Galaczi, E. D. (2021) · Assessment in Education Principles Policy and Practice · doi:10.1080/0969594x.2021.1979467 · Cited by 37
Kormos, J., & Wilby, J. (2019) · Springer eBooks · doi:10.1007/978-3-030-28380-3_13 · Cited by 33
Effects of the 4/3/2 activity revisited: Extending Boers (2014) and Thai & Boers (2016)
Tran, M. T. C., & Saito, K. (2021) · Language Teaching Research · doi:10.1177/1362168821994136 · Cited by 28
The effects of distributed practice on second language fluency development
Kakitani, J., & Kormos, J. (2024) · Studies in Second Language Acquisition · doi:10.1017/s0272263124000251 · Cited by 28
Alderson, J. C., & Huhta, A. (2011) · EUROSLA Yearbook · doi:10.1075/eurosla.11.04ald · Cited by 26
Gu, L., Davis, L., Tao, J., & Zechner, K. (2020) · Assessment in Education Principles Policy and Practice · doi:10.1080/0969594x.2020.1735995 · Cited by 26
Taguchi, N., Hirschi, K., & Kang, O. (2021) · Studies in Second Language Acquisition · doi:10.1017/s0272263121000486 · Cited by 26
Ramanarayanan, V., Lange, P. L., Evanini, K., Molloy, H. R., & Suendermann‐Oeft, D. (2017) · Interspeech (USB) · doi:10.21437/interspeech.2017-1213 · Cited by 25
Saeki, M., Takatsu, H., Kurata, F., Suzuki, S., Eguchi, M., Matsuura, R., Takizawa, K., Yoshikawa, S., & Matsuyama, Y. (2024) · Annual Meeting of the Special Interest Group on Discourse and Dialogue · doi:10.18653/v1/2024.sigdial-1.34 · Cited by 23
Self-citationIsbell, D. R. (2021) · Language Assessment Quarterly · doi:10.1080/15434303.2021.1874382 · Cited by 22
Xie, Q., & Lei, Y. (2021) · Language Assessment Quarterly · doi:10.1080/15434303.2021.1903470 · Cited by 20
Opening the Black Box: Exploring Automated Speaking Evaluation
Khabbazbashi, N., Xu, J., & Galaczi, E. D. (2021) · Springer eBooks · doi:10.1007/978-981-33-4232-3_25 · Cited by 19
Strategy use in a spoken dialog system–delivered paired discussion task: A stimulated recall study
Göktürk, N., & Chukharev‐Hudilainen, E. (2023) · Language Testing · doi:10.1177/02655322231152620 · Cited by 15
Xi, X., Zechner, K., & Bejar, I. I. (2006) · Cited by 12
Gnowsis: Multimodal multitask learning for oral proficiency assessments
Takatsu, H., Suzuki, S., Eguchi, M., Matsuura, R., Saeki, M., & Matsuyama, Y. (2025) · Computer Speech & Language · doi:10.1016/j.csl.2025.101860 · Cited by 11
Self-citationToward Automated Content Feedback Generation for Non-native Spontaneous Speech
Yoon, S., Hsieh, C., Zechner, K., Mulholland, M., Wang, Y., & Madnani, N. (2019) · Workshop on Innovative Use of NLP for Building Educational Applications · doi:10.18653/v1/w19-4432 · Cited by 5
Etinger, I. C., & Black, A. W. (2019) · Workshop on Noisy User-generated Text (W-NUT) · doi:10.18653/v1/d19-5502 · Cited by 5
Cumulative Testing for L2 Vocabulary Learning: The Impact of Retrieval Practice and Proficiency
Maie, R., Oikawa, G., Chen, Y., & Uchihara, T. (2025) · TESOL Quarterly · doi:10.1002/tesq.3391 · Cited by 5
Research works printed but not linked: 3
- OpenAI Achiam J. Adler S. Agarwal S. Ahmad L. Akkaya I. Aleman F. L. Almeida D. Altenschmidt J. Altman S. Anadkat S. Avila R. Babuschkin I. Balaji S. Balcom V. Baltescu P. Bao H. Bavarian M. Belgum J. . . . Zoph B. (2023). GPT-4 technical report. http://arxiv.org/abs/2303.08774 · arXiv
- Takatsu H., Suzuki S., Matsuura R., Koyama M., Matsuyama Y. (forthcoming). A personalized diagnostic feedback system to enhance conversational vocabulary competence of second language learners.
- Vaswani A. Shazeer N. Parmar N. Uszkoreit J. Jones L. Gomez A. N. Kaiser L. Polosukhin I. (2017). Attention is all you need. http://arxiv.org/abs/1706.03762 · arXiv
Cited by
Jang, E. E., & Sawaki, Y. (2025) · Language Testing · doi:10.1177/02655322251348685 · Cited by 31
Quan, Y. (2025) · Learning and Motivation · doi:10.1016/j.lmot.2025.102192 · Cited by 27
Harding, L. (2025) · Language Testing · doi:10.1177/02655322251350717 · Cited by 7
Gablasová, D., Harding, L., Březina, V., Hazelhurst, E. T., O’Sullivan, B., & Spiby, R. (2026) · Language Testing · doi:10.1177/02655322261453702 · Cited by 6
Meng, F., & Zou, B. (2026) · Journal of China Computer-Assisted Language Learning · doi:10.1515/jccall-2025-0033 · Cited by 4
Metadata
- Authors
- Suzuki, S., Takatsu, H., Matsuura, R., Koyama, M., Saeki, M., & Matsuyama, Y.
- Affiliations
- Waseda University (JP)
- Corresponding author
- Suzuki, Shungo
- Venue
- Language Testing
- Publisher
- SAGE Publications
- Published
- 2025-07-28
- Volume
- 42
- Issue
- 4
- Pages
- 476-507
- ISSN
- 0265-5322 (ISSN-L)
- 1477-0946
- OpenAlex
- W4412694316
- Cited by
- 6
- References
- 71
- Citations by year
- 2026: 3 · 2025: 3
- FWCI
- 21.61
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by-nc
- creativecommons.org/licenses/by-nc/4.0/ (vor)
- journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
- Locations
- Language Testing (publishedVersion, cc-by-nc)
- Funders
- New Energy and Industrial Technology Development Organization (JPNP20006)
- Topics
- Social Sciences › Arts and Humanities › Language and Linguistics › EFL/ESL Teaching and Learning
- Social Sciences › Education › Student Assessment and Feedback
- Social Sciences › Psychology › Developmental and Educational Psychology › Second Language Acquisition and Learning
- Keywords
- second language speaking, conversational AI agents, Japanese learners of English, task repetition, transfer of learning, learner awareness
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.