Machine Learning–Driven Language Assessment
Open accessThe DOI matches its Crossref record; 76 of the 77 references it cites were found in a registry or an index.
Abstract
We describe a method for rapidly creating language proficiency assessments, and provide experimental evidence that such tests can be valid, reliable, and secure. Our approach is the first to use machine learning and natural language processing to induce proficiency scales based on a given standard, and then use linguistic models to estimate item difficulty directly for computer-adaptive testing. This alleviates the need for expensive pilot testing with human subjects. We used these methods to develop an online proficiency exam called the Duolingo English Test, and demonstrate that its scores align significantly with other high-stakes English assessments. Furthermore, our approach produces test scores that are highly reliable, while generating item banks large enough to satisfy security requirements.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
An introduction to ROC analysis
Fawcett, T. (2006) · Pattern Recognition Letters · doi:10.1016/j.patrec.2005.10.010 · Cited by 21,879
(1991) · Cited by 15,554
The standards for educational and psychological testing.
Eignor, D. R. (2013) · APA handbook of testing and assessment in psychology, Vol. 1: Test theory and testing and assessment in industrial and organizational psychology. · doi:10.1037/14047-013 · Cited by 6,988
Probabilistic Models for Some Intelligence and Attainment Tests
(1961) · Information and Control · doi:10.1016/s0019-9958(61)80061-2 · Cited by 6,939
Common European Framework of Reference for Languages: learning, teaching, assessment
Verhelst, N. D., Avermaet, P. V., Takala, S., Figueras, N., & North, B. (2009) · Ghent University Academic Bibliography (Ghent University) · Cited by 6,120
Applications of Item Response Theory To Practical Testing Problems
Lord, F. M. (2012) · Routledge eBooks · doi:10.4324/9780203056615 · Cited by 5,007
A Rasch model for partial credit scoring
Masters, G. N. (1982) · Psychometrika · doi:10.1007/bf02296272 · Cited by 3,782
A Tutorial on the Cross-Entropy Method
de Boer, P.-T., Kroese, D. P., Mannor, S., & Rubinstein, R. Y. (2005) · Annals of Operations Research · doi:10.1007/s10479-005-5724-z · Cited by 3,024
A Rating Formulation for Ordered Response Categories
Andrich, D. (1978) · Psychometrika · doi:10.1007/bf02293814 · Cited by 2,970
Generating Sequences With Recurrent Neural Networks
Graves, A. (2013) · arXiv · doi:10.48550/arxiv.1308.0850 · Cited by 2,894
Research works printed but not linked: 21
- Brown J. D., 1989, JALT Journal, 11, 46
- Cambridge English. 2012. Preliminary wordlist.
- Cau S., 2015, Global Times
- Curto P., 2015, Proceedings of the International Conference on Computer Supported Education, 36
- Dudley R., 2016, Reuters Investigates
- Freedle R., 1999, Language Testing, 16, 2
- Graves A., 2014, arXiv
- Heafield K., 2013, Proceedings of the Association for Computational Linguistics (ACL), 690
- Lin W. Y., 2008, Journal of Pan-Pacific Association of Applied Linguistics, 12, 61
- Linacre J. M., 2014, Rasch Measurement Transactions, 27, 1441
- Loukina A., 2016, Proceedings of the International Conference on Computational Linguistics (COLING), 3245
- McNamara D. S., 2011, Annual Meeting of the American Educational Research Association (AERA)
- Mostow J., 2012, Proceedings of the Workshop on Building Educational Applications Using NLP, 136
- Napoles C., 2010, Proceedings of the Workshop on Computational Linguistics and Writing: Writing Processes and Authoring Aids, 42
- Perez-Beltrachini L., 2012, Proceedings of the Workshop on Building Educational Applications Using NLP, 147
- Reichert M., 2010, The C-Test: Contributions from Current Research, 205
- Sculley D., 2010, Proceedings of the Conference on Knowledge Discovery and Data Mining (KDD), 979
- Susanti Y., 2016, Proceedings of the International Conference on Computer Supported Education, 267
- Vajjala S., 2016, Proceedings of the Workshop on Computational Linguistics for Linguistic Complexity, 38
- Vajjala S., 2018, Proceedings of the Workshop on Innovative Use of NLP for Building Educational Applications, 147
- Volodina E., 2016, Proceedings of EUROCALL, 456
Cited by
Wang, X., Liu, Q., Pang, H., Tan, S. C., Lei, J., Wallace, M. P., & Li, L. (2023) · Computers & Education · doi:10.1016/j.compedu.2022.104703 · Cited by 299
Shanmugasundaram, M., & Tamilarasu, A. (2023) · Frontiers in Cognition · doi:10.3389/fcogn.2023.1203077 · Cited by 291
“Such applications can also be used for language proficiency tests (Settles et al., 2020).”
BackgroundTransforming Assessment: The Impacts and Implications of Large Language Models and Generative AI
Hao, J., von Davier, A. A., Yaneva, V., Lottridge, S., von Davier, M., & Harris, D. J. (2024) · Educational Measurement: Issues and Practice · doi:10.1111/emip.12602 · Cited by 93
The interactive reading task: Transformer-based automatic item generation
Attali, Y., Runge, A., LaFlair, G. T., Yancey, K. P., Goodwin, S., Park, Y., & von Davier, A. A. (2022) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2022.903077 · Cited by 127
Test Review: Current options in at-home language proficiency tests for making high-stakes decisions
Isbell, D. R., & Kremmel, B. (2020) · Language Testing · doi:10.1177/0265532220943483 · Cited by 82
Automated reading passage generation with OpenAI's large language model
Bezirhan, U., & von Davier, M. (2023) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2023.100161 · Cited by 66
Lee, P., Fyffe, S., Son, M., Jia, Z., & Yao, Z. (2022) · Journal of Business and Psychology · doi:10.1007/s10869-022-09864-6 · Cited by 75
Text-based Question Difficulty Prediction: A Systematic Review of Automatic Approaches
AlKhuzaey, S., Grasso, F., Payne, T. R., & Tamma, V. A. M. (2023) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-023-00362-1 · Cited by 78
Duolingo English Test, Revised Version July 2019
Wagner, E. (2020) · Language Assessment Quarterly · doi:10.1080/15434303.2020.1771343 · Cited by 48
Investigating the affordances of OpenAI's large language model in developing listening assessments
Aryadoust, V., Zakaria, A., & Jia, Y. (2024) · Computers and Education Artificial Intelligence · doi:10.1016/j.caeai.2024.100204 · Cited by 92
Hajahmadi, S., Clementi, L., Jiménez‐López, M. D., & Marfia, G. (2024) · IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW) · doi:10.1109/vrw62533.2024.00067 · Cited by 43
A Survey on Recent Approaches to Question Difficulty Estimation from Text
Benedetto, L., Cremonesi, P., Caines, A., Buttery, P. J., Cappelli, A., Giussani, A., & Turrin, R. (2022) · ACM Computing Surveys · doi:10.1145/3556538 · Cited by 79
A Theoretical Assessment Ecosystem for a Digital-First Assessment - The Duolingo English Test
Burstein, J. C., LaFlair, G. T., Kunnan, A. J., & von Davier, A. A. (2022) · doi:10.46999/kiqf4328 · Cited by 47
BOBCAT: Bilevel Optimization-Based Computerized Adaptive Testing
Ghosh, A., & Lan, A. (2021) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2021/332 · Cited by 36
Question Generation for Adaptive Education
Srivastava, M., & Goodman, N. D. (2021) · doi:10.18653/v1/2021.acl-short.88 · Cited by 58
Artificial Intelligence in Computerized Adaptive Testing
Mujtaba, D. F., & Mahapatra, N. R. (2020) · 2020 International Conference on Computational Science and Computational Intelligence (CSCI) · doi:10.1109/csci51800.2020.00116 · Cited by 24
Jump-Starting Item Parameters for Adaptive Language Tests
McCarthy, A. D., Yancey, K. P., LaFlair, G. T., Egbert, J., Liao, M., & Settles, B. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.67 · Cited by 56
Self-citationDigital-First Learning and Assessment Systems for the 21st Century
Langenfeld, T. E., Burstein, J. C., & von Davier, A. A. (2022) · Frontiers in Education · doi:10.3389/feduc.2022.857604 · Cited by 40
Improving evidence-based assessment of players using serious games
Alonso‐Fernández, C., Freire, M., Martínez‐Ortiz, I., & Fernández‐Manjón, B. (2021) · Telematics and Informatics · doi:10.1016/j.tele.2021.101583 · Cited by 28
Gorgun, G., & Bulut, O. (2024) · Education and Information Technologies · doi:10.1007/s10639-024-12771-3 · Cited by 29
Duolingo English Test Responsible AI Standards
Burstein, J. C. (2023) · doi:10.46999/vcae5025 · Cited by 36
A review of Artificial Intelligence in Foreign Language Learning
Gkountara, D. N., & Prasad, R. (2022) · International Symposium on Wireless Personal Multimedia Communications (WPMC) · doi:10.1109/wpmc55625.2022.10014767 · Cited by 22
Štěpánek, L., Dlouhá, J., & Martinková, P. (2023) · Mathematics · doi:10.3390/math11194104 · Cited by 26
Field-Testing Multiple-Choice Questions With AI Examinees: English Grammar Items
Maeda, H. (2024) · Educational and Psychological Measurement · doi:10.1177/00131644241281053 · Cited by 28
Using Artificial Intelligence (AI) to create language exam questions: A case study
Kic-Drgas, J., & Kılıçkaya, F. (2024) · XLinguae · doi:10.18355/xl.2024.17.01.02 · Cited by 21
SIAKAD machine learning for correcting errors in speaking Arabic
Ritonga, M., Zulmuqim, Z., Bambang, B., Kurniawan, R., & Pahri, P. (2022) · World Journal on Educational Technology Current Issues · doi:10.18844/wjet.v14i3.7214 · Cited by 15
Chen, L., Chang, K., Yang, S.-C., & Chen, S.-C. (2023) · Applied Sciences · doi:10.3390/app13031699 · Cited by 16
Msayer, M. E., Aoula, E.-S., & Bouihi, B. (2024) · International Conference on Intelligent Systems and Computer Vision (ISCV) · doi:10.1109/iscv60512.2024.10620092 · Cited by 12
Chen, T., & Sun, S. (2025) · PLoS ONE · doi:10.1371/journal.pone.0320811 · Cited by 14
Improving Robot-Assisted Virtual Teaching Using Transformers, GANs, and Computer Vision
Li, X., Chen, Y., Peng, Y., & Ghadi, Y. Y. (2024) · Journal of Organizational and End User Computing · doi:10.4018/joeuc.336481 · Cited by 13
ALEN App: Argumentative Writing Support To Foster English Language Learning
Wambsganss, T., Caines, A., & Buttery, P. (2022) · Workshop on Innovative Use of NLP for Building Educational Applications · doi:10.18653/v1/2022.bea-1.18 · Cited by 8
Learning to Paraphrase Sentences to Different Complexity Levels
Chi, A., Chen, L., Chang, Y., Lee, S., & Chang, J. S. (2023) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00606 · Cited by 27
Application of Grammar Error Detection Method for English Composition Based on Machine Learning
He, H., & Maia, D. (2022) · Security and Communication Networks · doi:10.1155/2022/7957365 · Cited by 9
RetractedConstructing Open Cloze Tests Using Generation and Discrimination Capabilities of Transformers
Felice, M., Taslimipoor, S., & Buttery, P. J. (2022) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2022.findings-acl.100 · Cited by 7
The Duolingo English Test: Psychometric Considerations
Maris, G. (2020) · doi:10.46999/mfkw9830 · Cited by 7
Otero, N., Druga, S., & Lan, A. (2025) · Discover Education · doi:10.1007/s44217-025-00742-w · Cited by 24
Yao, D. (2023) · PLoS ONE · doi:10.1371/journal.pone.0291629 · Cited by 6
Multi-objective optimization of item selection in computerized adaptive testing
Mujtaba, D. F., & Mahapatra, N. R. (2021) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/3449639.3459334 · Cited by 11
Alternative Methods for Item Parameter Estimation: From CTT to IRT
Guo, H., Lu, R., Johnson, M., & McCaffrey, D. (2022) · ETS Research Report Series · doi:10.1002/ets2.12355 · Cited by 6
Inventory Optimization Model Design with Machine Learning Approach in Feed Mill Company
Nasution, A. A., Matondang, N., & Ishak, A. (2022) · Jurnal Sistem Teknik Industri · doi:10.32734/jsti.v24i2.8637 · Cited by 6
Automated Scoring of Speaking and Writing: Starting to Hit its Stride
Jones, D. M., Cheng, L., & Tweedie, M. G. (2023) · Canadian Journal of Learning and Technology · doi:10.21432/cjlt28241 · Cited by 7
Zelikman, E., Ma, W. A., Tran, J. E., Yang, D., Yeatman, J. D., & Haber, N. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.135 · Cited by 30
Stronger Transformers for Neural Multi-Hop Question GenerationPreprint
Sachan, D. S., Wu, L., Sachan, M., & Hamilton, W. L. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.11374 · Cited by 16
Xie, Q. (2024) · Assessing Writing · doi:10.1016/j.asw.2024.100846 · Cited by 5
Can Large Language Models Recognize and Respond to Student Misconceptions?
Smart, F., Bos, N. D., & Bos, J. T. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-60609-0_21 · Cited by 5
Generating language assessment content free from representational harms
Choi, I., & Zu, J. (2025) · Language Testing · doi:10.1177/02655322251349560 · Cited by 5
A Supervised Learning Approach to Estimating IRT Models in Small SamplesPreprint
Belov, D. I., Lüdtke, O., & Ulitzsch, E. (2024) · doi:10.31234/osf.io/w3cyq · Cited by 5
Ma, X., & Zhang, H. S. (2024) · Frontiers in Education · doi:10.3389/feduc.2023.1275518 · Cited by 4
Belzak, W. C. M., & Lockwood, J. R. (2024) · Applied Psychological Measurement · doi:10.1177/01466216241284585 · Cited by 4
El impacto de la Inteligencia Artificial en la Enseñanza de ELE a estudiantes con dislexia
Clementi, L., & Jiménez‐López, M. D. (2024) · Doblele Revista de lengua y literatura · doi:10.5565/rev/doblele.151 · Cited by 11
Ulitzsch, E., Belov, D. I., Lüdtke, O., & Robitzsch, A. (2025) · Journal of Educational Measurement · doi:10.1111/jedm.12426 · Cited by 11
Using Knowledge Graphs to Improve Question Difficulty Estimation from Text
Gherardi, E., Benedetto, L., Matera, M., & Buttery, P. J. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-64299-9_24 · Cited by 4
Semi-Supervised Joint Estimation of Word and Document Readability
Fujinuma, Y., & Hagiwara, M. (2021) · Workshop on Graph Based Methods for Natural Language Processing · doi:10.18653/v1/2021.textgraphs-1.16 · Cited by 4
Self-citationMaintaining and monitoring quality of a continuously administered digital assessment
Liao, M., Attali, Y., Lockwood, J. R., III, & von Davier, A. A. (2022) · Frontiers in Education · doi:10.3389/feduc.2022.857496 · Cited by 3
Controlled Language Generation for Language Learning Items
Stowe, K., Ghosh, D., & Zhao, M. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-industry.30 · Cited by 16
Improving Test Validity and Accessibility with Digital-First Assessments
Care, N. M., & Maddox, B. (2022) · doi:10.46999/rnqv8188 · Cited by 4
Douglas, D. (2025) · Wiley eBooks · doi:10.1002/9781119985068.ch10 · Cited by 7
Patient reported outcome measures: from the classics to AI
Harrison, C. J., & Trickett, R. W. (2025) · Journal of Hand Surgery (European Volume) · doi:10.1177/17531934251327291 · Cited by 2
Vorobel, O., & SMITH, B. K. (2020) · CALICO Journal · doi:10.1558/cj.42243 · Cited by 2
Difficulty-Controlled Question Generation in Adaptive Education for Few-Shot Learning
Wang, Y., & Li, L. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-46677-9_40 · Cited by 4
Comparison on Applications and Impact of AI in English Grammar Learning
Selvi, V. T., & Vaishnavi, B. (2024) · Recent Research Reviews Journal · doi:10.36548/rrrj.2024.2.004 · Cited by 5
Quality Assurance in Digital-First Assessments
Liao, M., Attali, Y., von Davier, A. A., & Lockwood, J. R., III (2022) · Springer proceedings in mathematics & statistics · doi:10.1007/978-3-031-04572-1_20 · Cited by 2
Semi-Supervised Joint Estimation of Word and Document Readability
Fujinuma, Y., & Hagiwara, M. (2021) · Workshop on Graph Based Methods for Natural Language Processing · doi:10.18653/v1/11.textgraphs-1.16 · Cited by 4
Self-citationComputational Psychometrics for Digital-First Assessments
LaFlair, G. T., Yancey, K. P., Settles, B., & von Davier, A. A. (2023) · Routledge eBooks · doi:10.4324/9781003278658-9 · Cited by 4
Self-citationCEFR-based Contextual Lexical Complexity Classifier in English and French
Aleksandrova, D., & Pouliot, V. (2023) · Workshop on Innovative Use of NLP for Building Educational Applications · doi:10.18653/v1/2023.bea-1.43 · Cited by 3
Explanatory Cognitive Diagnosis Models Incorporating Item Features
Liao, M., Jiao, H., & He, Q. (2024) · Journal of Intelligence · doi:10.3390/jintelligence12030032 · Cited by 2
Clementi, L., & López, M. D. J. (2025) · Journal of Spanish Language Teaching · doi:10.1080/23247797.2025.2534203 · Cited by 3
Cai, H., Yan, X., Chuang, P.-L., Pan, Y., & Huo, M. (2025) · Applied Linguistics · doi:10.1093/applin/amaf079 · Cited by 1
Optimizing inter-rater reliability in foreign language constructed-response assessments
Elizondo-González, J. F. (2025) · Research in Pedagogy · doi:10.5937/istrped2502471f · Cited by 1
Beyond the Algorithm: A Critical Synthesis for Human-Centered AI in K-12 STEM Education
Lodhi, S. S., & Roehrig, G. (2026) · AI and Ethics · doi:10.1007/s43681-026-00994-8 · Cited by 8
Dunn, K., Skidmore, L., & Rogers, T. (2026) · Frontiers in Education · doi:10.3389/feduc.2026.1740237 · Cited by 1
Navigating the Current “New World” of Teaching with Technology: A Glimpse into Our Teachers’ Minds
Cooney, M., & Sjöberg, J. (2023) · Lecture notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering · doi:10.1007/978-3-031-31392-9_11 · Cited by 2
Is This Hard for You? Personalized Human Difficulty Estimation for Skin Lesion Diagnosis
Kampen, P. J. T., Christensen, A. N., & Hannemose, M. R. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72390-2_17 · Cited by 1
Enhancing Spoken English Assessment Using Convolutional Recurrent Neural Networks
Islam, M. S., Sulochana, A., Sugadev, T., Hameed, N. S., Vijayakumar, S., & Faizal, M. K. M. (2025) · International Conference on Advances in Electrical, Computing, Communication and Sustainable Technologies (ICAECT) · doi:10.1109/icaect63952.2025.10958913 · Cited by 1
An Item Response Theory Framework for Persuasion
Kornilova, A., Eidelman, V., & Douglass, D. (2022) · Findings of the Association for Computational Linguistics: NAACL · doi:10.18653/v1/2022.findings-naacl.7 · Cited by 7
4 ways that AI can help students
O’Byrne, W. I. (2023) · doi:10.64628/aai.hdncjr4tu · Cited by 1
BOBCAT: Bilevel Optimization-Based Computerized Adaptive TestingPreprint
Ghosh, A., & Lan, A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.07386 · Cited by 15
Ehara, Y. (2021) · Workshop on Noisy User-generated Text (W-NUT) · doi:10.18653/v1/2021.wnut-1.50
On the relevance and learner dependence of co-text complexity for exercise difficulty
Heck, T., & Meurers, D. (2023) · Linköping electronic conference proceedings · doi:10.3384/ecp197009
Jyothi, N. M., & Madhusudhanan, S. (2023) · AIP conference proceedings · doi:10.1063/5.0132328
Assessment of Test Validity in the Context of the Duolingo English Test
Chen, C. (2024) · Open Journal of Modern Linguistics · doi:10.4236/ojml.2024.141001
Trustworthy AI Meets Educational Assessment: Challenges and Opportunities
Li, S. (2025) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v39i27.35089 · Cited by 6
Hameed, N. S., Vijayakumar, S., Rajaraman, K., Kumar, V. V., Tamilarasan, P., & Faizal, M. K. M. (2025) · International Conference on Advances in Electrical, Computing, Communication and Sustainable Technologies (ICAECT) · doi:10.1109/icaect63952.2025.10958834
Chen, T., & Sun, S. (2024) · International Conference on Information Education and Artificial Intelligence (ICIEAI) · doi:10.1145/3724504.3724580
Opportunities of natural language processing for comparative judgment assessment of essays
Vrindt, M. D., Tack, A., Noortgate, W. V. D., Lesterhuis, M., & Bouwer, R. (2025) · Computers and Education Artificial Intelligence · doi:10.1016/j.caeai.2025.100414 · Cited by 2
Classification of Tahfidz Certification Success Among Students Using Machine Learning Algorithms
Pamoengkas, M. A., & Utami, E. (2025) · International Conference on Computer Sciences, Engineering, and Technology Innovation (ICoCSETI) · doi:10.1109/icocseti63724.2025.11019153
Generating Diverse Difficulty Examples in Embedding Vector Spaces via Inverse Embedding
Ehara, Y. (2025) · Communications in computer and information science · doi:10.1007/978-3-031-99267-4_9
Self-improvement for Computerized Adaptive Testing
Rudolph, Y., Neubauer, K., & Brefeld, U. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-05981-9_5 · Cited by 1
User experiences in fishing tourism
Sánchez, F. T., Martín-Fuentes, E., & Perdomo-Verdecia, V. (2025) · Tourism Recreation Research · doi:10.1080/02508281.2025.2569872
von Davier, A. A., Runge, A., Park, Y., Attali, Y., Church, J., & LaFlair, G. T. (2024) · Emerald Group Publishing Limited eBooks · doi:10.1108/979-8-88730-606-320251002
Fully Data-Driven Computerized Adaptive Testing
Ghosh, A., Feng, W., Sireci, S. G., & Lan, A. (2024) · Emerald Group Publishing Limited eBooks · doi:10.1108/979-8-88730-606-320251009
Computer Programming for Applied Linguistics
Collentine, J. (2025) · Wiley eBooks · doi:10.1002/9781405198431.wbeal20522
A Span Vector Model to Classify Pragmatic Patterns of Adverbial Collocates across Domains
Ma, T. (2025) · International Conference on Artificial Intelligence, Digital Media Technology and Social Computing (ICAIDS) · doi:10.1109/icaids67687.2025.00034
Generative AI tools in designing MCQs for English language examinations: Insights from Lecturers
Vu, T. K. C., Nguyen, T. T. A., & Vo, D. V. C. (2025) · ICTE Conference Proceedings · doi:10.54855/979-8-9870112-9-4_1
Machine learning in personalizing learning experiences
Mardiana, H. (2026) · AIP conference proceedings · doi:10.1063/5.0296186
Application of Machine Learning for Real-Time Feedback on Spoken English in Language Learning Apps
Gopal, G. L., Gopal, G. L., Silvest, R. S. R., Vasudeva, S. P., Vyawahare, S., & Selvakumarasamy, K. (2025) · International Conference on Artificial Intelligence and Quantum Computation-Based Sensor Application (ICAIQSA) · doi:10.1109/icaiqsa67794.2025.11440581
Reimagining Education With Artificial Intelligence
Duran, V., & Çelık, F. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/407417
Kamal, G. (2026) · Frontiers in Education · doi:10.3389/feduc.2026.1876290 · Cited by 1
Literature Review: Analysing the Factors Affecting College Students’ Digital Language Tests
Zhang, J., & Razak, R. A. (2026) · Africa Education Review · doi:10.1080/18146627.2026.2659587
Metadata
- Authors
- Settles, B., T. LaFlair, G., & Hagiwara, M.
- Affiliations
- Duolingo (United States) (US)
- Duolingo, Pittsburgh, PA USA. geoff@duolingo.com
- Octanove Labs, Seattle, WA USA. masato@octanove.com
- Venue
- Transactions of the Association for Computational Linguistics
- Publisher
- MIT Press
- Published
- 2020-12
- Volume
- 8
- Pages
- 247-263
- ISSN
- 2307-387X (ISSN-L)
- OpenAlex
- W3019082520
- Cited by
- 106
- References
- 77
- Citations by year
- 2026: 12 · 2025: 21 · 2024: 22 · 2023: 17 · 2022: 18 · 2021: 10 · 2020: 6
- FWCI
- 5.01
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- cc-by
- Locations
- Transactions of the Association for Computational Linguistics (publishedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
- Keywords
- language proficiency assessment, computerized adaptive testing, item difficulty estimation, test security, item bank, Duolingo English Test, English proficiency, test reliability
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.