The effects of adaptive learning in a massive open online course on learners' skill development
Abstract
We report an experimental implementation of adaptive learning functionality in a self-paced Microsoft MOOC (massive open online course) on edX. In a personalized adaptive system, the learner's progress toward clearly defined goals is continually assessed, the assessment occurs when a student is ready to demonstrate competency, and supporting materials are tailored to the needs of each learner. Despite the promise of adaptive personalized learning, there is a lack of evidence-based instructional design, transparency in many of the models and algorithms used to provide adaptive technology or a framework for rapid experimentation with different models. ALOSI (Adaptive Learning Open Source Initiative) provides open source adaptive learning technology and a common framework to measure learning gains and learner behavior. This study explored the effects of two different strategies for adaptive learning and assessment: Learners were randomly assigned to three groups. In the first adaptive group ALOSI prioritized a strategy of remediation - serving learners items on topics with the least evidence of mastery; in the second adaptive group ALOSI prioritized a strategy of continuity - that is learners would be more likely served items on similar topic in a sequence until mastery is demonstrated. The control group followed the pathways of the course as set out by the instructional designer, with no adaptive algorithms. We found that the implemented adaptivity in assessment, with emphasis on remediation is associated with a substantial increase in learning gains, while producing no big effect on the drop-out. Further research is needed to confirm these findings and explore additional possible effects and implications to course design.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Cognitive graphs can support personalized learning paths as learners progress, ensuring content relevance and promoting cognitive processing and mastery of problem-solving skills.
“Cognitive graphs can further support PL paths as learners progress, ensuring that the delivery of content remains relevant to the learner's needs, promoting better cognitive processing and mastery of problem-solving skills [176,189,203].”
From Leveraging AI in E-Learning: Personalized Learning and Adaptive Assessment through Cognitive Neuropsychology—A Systematic Analysis (2024) · cites this paper as [189]
Cites
Knowledge tracing: Modeling the acquisition of procedural knowledge
Corbett, A. T., & Anderson, J. R. (1995) · User Modelling and User-Adapted Interaction · doi:10.1007/bf01099821 · Cited by 2,324 · see citing works
Intelligent Tutoring Goes To School in the Big City
Koedinger, K. R., Anderson, J. R., Hadley, W. H., & Mark, M. A. (2018) · doi:10.1184/r1/6470153 · Cited by 1,105 · see citing works
Proceedings of the 7th International Conference on Educational Data Mining.
Stamper, J. C., Pardos, Z. A., Mavrikis, M., & McLaren, B. M. (2014) · Educational Data Mining · Cited by 667 · see citing works
KT-IDEM: introducing item difficulty to the knowledge tracing model
Pardos, Z. A., & Heffernan, N. T. (2011) · International Conference on User Modeling, Adaptation, and Personalization · doi:10.5555/2021855.2021877 · Cited by 126 · see citing works
Does Help Help? Introducing the Bayesian Evaluation and Assessment Methodology
Beck, J. E., Chang, K.-M. K., Mostow, J., & Corbett, A. T. (2008) · Lecture notes in computer science · doi:10.1007/978-3-540-69132-7_42 · Cited by 125 · see citing works
Experimental Evaluation of Automatic Hint Generation for a Logic Tutor
Stamper, J. C., Eagle, M. J., Barnes, T. M., & Croy, M. J. (2013) · International Journal of Artificial Intelligence in Education · doi:10.3233/jai-130029 · Cited by 117 · see citing works
Enabling Real-Time Adaptivity in MOOCs with a Personalized Next-Step Recommendation Framework
Pardos, Z. A., Tang, S., Davis, D., & Le, C. V. (2017) · ACM Conference on Learning @ Scale (L@S) · doi:10.1145/3051457.3051471 · Cited by 79 · see citing works
Pardos, Z. A., & Heffernan, N. T. (2010) · Cited by 65 · see citing works
Hawkins, W. J., Heffernan, N. T., & Baker, R. S. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-07221-0_18 · Cited by 49 · see citing works
Designing Adaptive Assessments in MOOCs
Rosen, Y., Rushkin, I., Ang, A., Federicks, C., Tingley, D. H., & Blink, M. J. (2017) · ACM Conference on Learning @ Scale (L@S) · doi:10.1145/3051457.3053993 · Cited by 17 · see citing works
Self-citationAdaptive Assessment Experiment in a HarvardX MOOC.
Rushkin, I., Rosen, Y., Ang, A., Fredericks, C., Tingley, D. H., Blink, M. J., & Lopez, G. (2017) · Educational Data Mining · Cited by 10 · see citing works
Self-citationA Data Driven Approach to the Discovery of Better Cognitive Models.
Koedinger, K. R., & Stamper, J. C. (2010) · Cited by 8 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
du Plooy, E., Casteleijn, D., & Franzsen, D. (2024) · Heliyon · doi:10.1016/j.heliyon.2024.e39630 · Cited by 494 · see citing works
Halkiopoulos, C., & Gkintoni, E. (2024) · Electronics · doi:10.3390/electronics13183762 · Cited by 328 · see citing works
“AL systems that adjust the complexity of numeric problems based on the performance of each learner ensure that each gets an appropriate challenge [189].”
Raj, N. S., & Renumol, V. G. (2022) · Journal of Computers in Education · doi:10.1007/s40692-021-00199-4 · Cited by 253 · see citing works
Artificial intelligence-enabled adaptive learning platforms: A review
Tan, L. Y., Hu, S., Yeo, D. J., & Cheong, K. H. (2025) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2025.100429 · Cited by 207 · see citing works
Fairer but Not Fair Enough On the Equitability of Knowledge Tracing
Doroudi, S., & Brunskill, E. (2019) · International Conference on Learning Analytics and Knowledge (LAK) · doi:10.1145/3303772.3303838 · Cited by 60 · see citing works
Pardos, Z. A., Tang, M., Anastasopoulos, I., Sheel, S. K., & Zhang, E. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581574 · Cited by 62 · see citing works
Personalization of learning using adaptive technologies and augmented reality
Marienko, M., Носенко, Ю. Г., & Shyshkina, M. P. (2020) · doi:10.31812/123456789/4418 · Cited by 55 · see citing works
Navigating the Personalization Pathway
Tariq, M. U. (2024) · Advances in educational technologies and instructional design book series · doi:10.4018/979-8-3693-3641-0.ch012 · Cited by 48 · see citing works
Кazimova, D., Tazhigulova, G. O., Shraimanova, G. S., Zatyneyko, A., & Sharzadin, A. M. (2025) · International Journal of Engineering Pedagogy (iJEP) · doi:10.3991/ijep.v15i1.50773 · Cited by 32 · see citing works
Adaptive learning: a cluster-based literature review (2011-2022)
Фадєєва, Л. О. (2023) · Educational Technology Quarterly · doi:10.55056/etq.613 · Cited by 27 · see citing works
Towards Sustainable e-Learning Systems Using an Adaptive Learning Approach
Smaili, E. M., Sraidi, S., Azzouzi, S., & Charaf, M. E. H. (2021) · Advances in Science, Technology & Innovation/Advances in science, technology & innovation · doi:10.1007/978-3-030-53440-0_38 · Cited by 15 · see citing works
Ewais, A., Awad, M., & Hadia, K. (2020) · Smart innovation, systems and technologies · doi:10.1007/978-981-15-1918-5_1 · Cited by 13 · see citing works
Remote Co-teaching in Rural Classroom: Current Practices, Impacts, and Challenges
Guo, S., Sun, T., Gong, J., Lu, Z., Zhang, L., & Wang, Q. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3501924 · Cited by 12 · see citing works
An Adaptive Learning Approach for Better Retention of Learners in MOOCs
Smaili, E. M., Sraidi, S., Azzouzi, S., & Charaf, M. E. H. (2020) · International Conference on Networking, Information Systems & Security (NISS) · doi:10.1145/3386723.3387845 · Cited by 11 · see citing works
Choice-based Personalization in MOOCs: Impact on Activity and Perceived Value
Ram, I., Harris, S., & Roll, I. (2023) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-023-00334-5 · Cited by 10 · see citing works
Behavioural Cloning of Teachers for Automatic Homework Selection
Moore, R., Caines, A., Rice, A. C., & Buttery, P. J. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-23204-7_28 · Cited by 10 · see citing works
KNEWTON ANALYTICAL REVIEW AS A PLATFORM FOR PERSONALIZATION OF LEARNING CONTENT
Носенко, Ю. Г. (2020) · Information Technologies in Education · doi:10.14308//ite000727 · Cited by 6 · see citing works
Personalization of learning using adaptive technologies and augmented reality
Marienko, M., Nosenko, Y. H., & Shyshkina, M. P. (2020) · International Workshop on Augmented Reality in Education (AREdu) · doi:10.55056/ceur-ws.org/vol-2731/paper20.pdf · Cited by 6 · see citing works
Wang, M., & Guicang, L. (2022) · International Journal of e-Collaboration · doi:10.4018/ijec.301260 · Cited by 6 · see citing works
Fromm, Y. M., Rasch, J., & Ifenthaler, D. (2025) · Smart Learning Environments · doi:10.1186/s40561-025-00407-z · Cited by 8 · see citing works
Adaptive Learning Algorithms and Platforms: A Concise Overview
Mezin, H., Kharrou, S. Y., & Lahcen, A. A. (2022) · Lecture notes in networks and systems · doi:10.1007/978-3-030-91738-8_1 · Cited by 5 · see citing works
Латыпова, В. А. (2019) · Modelirovanie, optimizaciâ i informacionnye tehnologii. · doi:10.26102/2310-6018/2019.26.3.015 · Cited by 5 · see citing works
The Role of Adaptive Learning Technologies and Conditional Learning
Thompson, K., Farr, A. C., Saunders, T., & Winter, G. (2023) · University development and administration · doi:10.1007/978-981-19-9438-8_26-1 · Cited by 4 · see citing works
Hasanov, A., Laine, T. H., Kim, J., & Chung, T. (2024) · Electronics · doi:10.3390/electronics13132498 · Cited by 5 · see citing works
HeliCoach: An Adaptive Orientation and Mobility Training System in a Drone-based 3D Audio Space
Gong, J., Ding, Q., Zhang, Y., Zhang, L., Xu, P., Liu, M., Xu, Y., & Wang, Q. (2020) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3334480.3382880 · Cited by 4 · see citing works
Sengupta, S., Parab, S., Shetty, S., Nile, V., & Sambhaji, A. (2025) · Asian Conference on Innovation in Technology (ASIANCON) · doi:10.1109/asiancon66527.2025.11280793 · Cited by 3 · see citing works
The Role of Adaptive Learning Technologies and Conditional Learning
Thompson, K., Farr, A. C., Saunders, T., & Winter, G. (2023) · University development and administration · doi:10.1007/978-981-99-4170-4_26 · Cited by 3 · see citing works
Identifying the Essentials of Corporate MOOCs’ Application to the Leading Organizations
Hafshejani, F. F., Zeinalipour, H., Ebner, M., Fini, A. A. S., Vajargah, K. F., & Jafari, E. (2023) · International Journal of Emerging Technologies in Learning (iJET) · doi:10.3991/ijet.v18i03.33937 · Cited by 2 · see citing works
Rosen, Y., Stoeffler, K., Yudelson, M. V., & Simmering, V. R. (2020) · ACM Conference on Learning @ Scale (L@S) · doi:10.1145/3386527.3405946 · Cited by 2 · see citing works
Self-citationALPY PLUS - Adaptive Model Oriented to Pathway Planning in Virtual Learning System
Núñez, Y. H., Parra, C., & Carrillo-Ramos, Á. (2022) · Communications in computer and information science · doi:10.1007/978-3-031-19951-6_6 · Cited by 1 · see citing works
An Ontology-based Recommender System for Identifying Learners’ Confusion in MOOCs
Tazi, K., Azzouzi, S., & Charaf, M. E. H. (2023) · IEEE Congress on Information Science and Technology (CiSt) · doi:10.1109/cist56084.2023.10409939 · Cited by 1 · see citing works
A MIXED METHODS STUDY OF ENGINEERING UNDERGRADUATES' ADAPTIVE LEARNING EXPERIENCES
Kwak, S. J. (2021) · IADIS International Conferences Mobile Learning and Educational Technologies (ML and ICEduTech) · doi:10.33965/ml_icedutech2021_202102l017 · Cited by 1 · see citing works
ОРГАНІЗАЦІЙНО-ТЕХНІЧНІ І ПРАВОВІ АСПЕКТИ ФОРМУВАННЯ КОМПЕТЕНТНОСТЕЙ ВІЙСЬКОВИХ ФАХІВЦІВ
Shchavinsky, Y., Левчук, О., Левчук, В., & Сирський, О. (2023) · Військова освіта · doi:10.33099/2617-1783/2022-46/311-324 · Cited by 1 · see citing works
An Adaptive Content Skipping Methodology based on User Behavioral Modeling
Tu, Y., Tenorio, E., & Brinton, C. G. (2020) · Conference on Information Sciences and Systems · doi:10.1109/ciss48834.2020.1570629135
Current State of Learning Analytics
Sun, W., Niels Pinkwart, C., & Li, T. (2021) · Advances in educational technologies and instructional design book series · doi:10.4018/978-1-7998-7103-3.ch001
Використання сервісів адаптивних хмаро орієнтованих систем у діяльності вчителя
Barladym, V. М., Берідзе, К. С., Бруяка, А. В., Горбаченко, С. В., Коваленко, В. В., Носенко, Ю. Г., Marienko, M., Семеріков, С. О., & Шишкіна, М. П. (2020) · Педагогічна думка eBooks · doi:10.31812/123456789/4349
An Optimized Adaptive Learning Approach Based on Cuckoo Search Algorithm
Smaili, E. M., Azzouzi, S., & Charaf, M. E. H. (2022) · International Conference on Optimization and Applications (ICOA) · doi:10.1109/icoa55659.2022.9934280
- Authors
- Rosen, Y., Rushkin, I., Rubin, R., Munson, L., Ang, A., Weber, G., Lopez, G., & Tingley, D.
- Affiliations
- Harvard University (US)
- Independent
- Microsoft (United States) (US)
- Venue
- Proceedings of the Fifth Annual ACM Conference on Learning at Scale
- Publisher
- ACM
- Published
- 2018-06-26
- Pages
- 1-8
- OpenAlex
- W2808942252
- Cited by
- 38
- References
- 12
- Citations by year
- 2025: 4 · 2024: 4 · 2023: 8 · 2022: 5 · 2021: 4 · 2020: 10 · 2019: 3
- FWCI
- 1.73
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM Conference on Learning @ Scale (L@S) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
- Physical Sciences › Computer Science › Computer Science Applications › Online Learning and Analytics
- Social Sciences › Education › Online and Blended Learning
- Keywords
- adaptive learning, massive open online courses, personalized learning, remediation, continuity, adaptive testing, skill development, learning gains, dropout, competency-based assessment, instructional design
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
