Including Learning Analytics in the Loop of Self-Paced Online Course Learning Design
Open accessAbstract
Online education is growing because of its benefits and advantages that students enjoy. Educational technologies (e.g., learning analytics, student modelling, and intelligent tutoring systems) bring great potential to online education. Many online courses, particularly in self-paced online learning (SPOL), face some inherent barriers such as learning awareness and academic intervention. These barriers can affect the academic performance of online learners. Recently, learning analytics has been shown to have great potential in removing these barriers. However, it is challenging to achieve the full potential of learning analytics with the traditional online course learning design model. Thus, focusing on SPOL, this study proposes that learning analytics should be included in the course learning design loop to ensure data collection and pedagogical connection. We propose a novel learning design-analytics model in which course learning design and learning analytics can support each other to increase learning success. Based on the proposed model, a set of online course design strategies are recommended for online educators who wish to use learning analytics to mitigate the learning barriers in SPOL. These strategies and technologies are inspired by Jim Greer’s work on student modelling. By following these recommended design strategies, a computer science course is used as an example to show our initial practices of including learning analytics in the course learning design loop. Finally, future work on how to develop and evaluate learning analytics enabled learning systems is outlined.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The work by Yan and Lin [95] improves absence of self-awareness of learning.
“improves absence of self-awareness of learning”
From Artificial Intelligence for Assessment and Feedback to Enhance Student Success in Higher Education (2022) · cites this paper as [95]
The work by Yan and Lin [95] scales up personalized and adaptive learning.
“Scales up personalized and adaptive learning”
From Artificial Intelligence for Assessment and Feedback to Enhance Student Success in Higher Education (2022) · cites this paper as [95]
By implementing formative assessment, intime and immediate feedback to struggling students they detect anomalies in the learning contents of the course.
“us, by implementing formative assessment, intime and immediate feedback to struggling students they detect anomalies in the learning contents of the course.”
From Artificial Intelligence for Assessment and Feedback to Enhance Student Success in Higher Education (2022) · cites this paper as [95]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
(2001) · Cited by 13,930 · see citing works
Zimmerman, B. J. (2000) · Handbook of Self-Regulation · doi:10.1016/b978-012109890-2/50031-7 · Cited by 6,069 · see citing works
The development of student feedback literacy: Enabling uptake of feedback
Carless, D., & Boud, D. (2018) · Assessment & Evaluation in Higher Education · doi:10.1080/02602938.2018.1463354 · Cited by 2,428 · see citing works
Knowledge tracing: Modeling the acquisition of procedural knowledge
Corbett, A. T., & Anderson, J. R. (1995) · User Modelling and User-Adapted Interaction · doi:10.1007/bf01099821 · Cited by 2,324 · see citing works
Item Response Theory: Principles and Applications
Hambleton, R. K., & Swaminathan, H. (1984) · Medical Entomology and Zoology · Cited by 1,779 · see citing works
Penetrating the Fog: Analytics in Learning and Education.
Siemens, G., & Long, P. (2011) · Adelaide Research & Scholarship (AR&S) (University of Adelaide) · Cited by 1,752 · see citing works
Conducting Educational Design Research
McKenney, S. E., & Reeves, T. C. (2018) · Routledge eBooks · doi:10.4324/9781315105642 · Cited by 1,537 · see citing works
Educational Data Mining and Learning Analytics
Baker, R. S., & Inventado, P. S. (2014) · Springer eBooks · doi:10.1007/978-1-4614-3305-7_4 · Cited by 1,333 · see citing works
Formative Assessment: Revisiting the Territory
Sadler, D. R. (1998) · Assessment in Education: Principles, Policy & Practice · doi:10.1080/0969595980050104 · Cited by 1,107 · see citing works
Science Education International
International Council of Associations for Science Education (2019) · Science education international · doi:10.33828/sei · Cited by 1,070 · see citing works
Let’s not forget: Learning analytics are about learning
Gašević, D., Dawson, S., & Siemens, G. (2015) · TechTrends · doi:10.1007/s11528-014-0822-x · Cited by 1,049 · see citing works
Multi-Armed Bandit Allocation Indices.
Bather, J. A., & Gittins, J. C. (1990) · Journal of the Royal Statistical Society Series A (Statistics in Society) · doi:10.2307/2982809 · Cited by 1,043 · see citing works
Deep Knowledge TracingPreprint
Piech, C., Spencer, J., Huang, J., Ganguli, S., Sahami, M., Guibas, L. J., & Sohl‐Dickstein, J. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.05908 · Cited by 875 · see citing works
(2009) · Cited by 781 · see citing works
A review of online course dropout research: implications for practice and future research
Lee, Y., & Choi, J. (2011) · Educational Technology Research and Development · doi:10.1007/s11423-010-9177-y · Cited by 747 · see citing works
A reference model for learning analytics
Chatti, M. A., Dyckhoff, A. L., Schroeder, U., & Thüs, H. (2012) · International Journal of Technology Enhanced Learning · doi:10.1504/ijtel.2012.051815 · Cited by 702 · see citing works
Assessment in Education: Principles, Policy & Practice
Wiliam, D. (2008) · Cited by 696 · see citing works
Berry, D. A., & Fristedt, B. E. (1985) · Springer eBooks · doi:10.1007/978-94-015-3711-7 · Cited by 645 · see citing works
Individualized Bayesian Knowledge Tracing Models
Yudelson, M. V., Koedinger, K. R., & Gordon, G. J. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-39112-5_18 · Cited by 502 · see citing works
Lockyer, L., Heathcote, E., & Dawson, S. (2013) · American Behavioral Scientist · doi:10.1177/0002764213479367 · Cited by 473 · see citing works
Tempelaar, D. T., Rienties, B., & Giesbers, B. (2015) · Computers in Human Behavior · doi:10.1016/j.chb.2014.05.038 · Cited by 449 · see citing works
Engaging Students In STEM Education
Kennedy, T. J., & O’Dell, M. (2014) · Science education international · Cited by 440 · see citing works
Mangaroska, K., & Giannakos, M. N. (2018) · IEEE Transactions on Learning Technologies · doi:10.1109/tlt.2018.2868673 · Cited by 395 · see citing works
Learning analytics to unveil learning strategies in a flipped classroom
Jovanović, J., Gašević, D., Dawson, S., Pardo, A., & Mirriahi, N. (2017) · The Internet and Higher Education · doi:10.1016/j.iheduc.2017.02.001 · Cited by 376 · see citing works
Introduction to knowledge spaces: How to build, test, and search them.
Falmagne, J., Koppen, M., Villano, M. A., Doignon, J., & Johannesen, L. J. (1990) · Psychological Review · doi:10.1037/0033-295x.97.2.201 · Cited by 294 · see citing works
From Evidence to Action: A Seamless Process in Formative Assessment?
Heritage, M., Kim, J.-O., Vendlinski, T., & Herman, J. L. (2009) · Educational Measurement Issues and Practice · doi:10.1111/j.1745-3992.2009.00151.x · Cited by 263 · see citing works
Learning Analytics in Higher Education—A Literature Review
Leitner, P., Khalil, M., & Ebner, M. (2017) · Studies in systems, decision and control · doi:10.1007/978-3-319-52977-6_1 · Cited by 263 · see citing works
Bayesian knowledge tracing, logistic models, and beyond: an overview of learner modeling techniques
Pelánek, R. (2017) · User Modeling and User-Adapted Interaction · doi:10.1007/s11257-017-9193-2 · Cited by 254 · see citing works
Awareness Is Not Enough: Pitfalls of Learning Analytics Dashboards in the Educational Practice
Jivet, I., Scheffel, M., Drachsler, H., & Specht, M. M. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-66610-5_7 · Cited by 222 · see citing works
Learning dispositions and transferable competencies
Shum, S. B., & Crick, R. D. (2012) · International Conference on Learning Analytics and Knowledge (LAK) · doi:10.1145/2330601.2330629 · Cited by 202 · see citing works
Shaffer, C. A., Cooper, M. L., Alon, A. J. D., Akbar, M., Stewart, M., Ponce, S., & Edwards, S. H. (2010) · ACM Transactions on Computing Education · doi:10.1145/1821996.1821997 · Cited by 198 · see citing works
Learning analytics and AI: Politics, pedagogy and practices
Shum, S. B., & Luckin, R. (2019) · British Journal of Educational Technology · doi:10.1111/bjet.12880 · Cited by 171 · see citing works
Multi-Armed Bandit Allocation Indices
Jones, P. W., & Gittins, J. C. (1989) · Journal of the Operational Research Society · doi:10.2307/2582928 · Cited by 165 · see citing works
Catalytic assessment: understanding how MCQs and EVS can foster deep learning
DRAPER, S. W. (2009) · British Journal of Educational Technology · doi:10.1111/j.1467-8535.2008.00920.x · Cited by 159 · see citing works
A review of learning analytics intervention in higher education (2011–2018)
Wong, B. T. M., & Li, K. C. (2019) · Journal of Computers in Education · doi:10.1007/s40692-019-00143-7 · Cited by 156 · see citing works
Open learner models and learning analytics dashboards
Bodily, R., Kay, J., Aleven, V., Jivet, I., Davis, D., Xhakaj, F., & Verbert, K. (2018) · International Conference on Learning Analytics and Knowledge (LAK) · doi:10.1145/3170358.3170409 · Cited by 154 · see citing works
Cognitively Diagnostic Assessments and the Cognitive Diagnosis Model Framework
TORRE, J. D. L., & Minchen, N. (2014) · Psicologí a Educativa · doi:10.1016/j.pse.2014.11.001 · Cited by 132 · see citing works
Piecing the learning analytics puzzle: a consolidated model of a field of research and practice
Gašević, D., Kovanović, V., & Joksimovíc, S. (2017) · Learning Research and Practice · doi:10.1080/23735082.2017.1286142 · Cited by 132 · see citing works
Assessment of learning with multiple-choice questions
Brady, A. (2005) · Nurse Education in Practice · doi:10.1016/j.nepr.2004.12.005 · Cited by 123 · see citing works
Learning analytics: challenges and limitations
Wilson, A. N., Watson, C., Thompson, T. L., Drew, V., & Doyle, S. (2017) · Teaching in Higher Education · doi:10.1080/13562517.2017.1332026 · Cited by 116 · see citing works
Stone, C. (2017) · Cited by 112 · see citing works
Learner modelling: systematic review of the literature from the last 5 years
Abyaa, A., Idrissi, M. K., & Bennani, S. (2019) · Educational technology research and development · doi:10.1007/s11423-018-09644-1 · Cited by 108 · see citing works
The impact of assessment methods on the learning of nursing students
Leung, S. F., Mok, E., & Wong, D. (2008) · Nurse Education Today · doi:10.1016/j.nedt.2007.11.004 · Cited by 104 · see citing works
Learning Analytics: Principles and Constraints
Khalil, M., & Ebner, M. (2015) · doi:10.13140/rg.2.1.1733.2083 · Cited by 103 · see citing works
Karavirta, V., & Shaffer, C. A. (2013) · ACM Conference on Innovation and Technology in Computer Science Education (ITiCSE) · doi:10.1145/2462476.2462487 · Cited by 83 · see citing works
Formative assessment and learning analytics
Tempelaar, D. T., Heck, A., Cuypers, H., van der Kooij, H., & van de Vrie, E. (2013) · International Conference on Learning Analytics and Knowledge (LAK) · doi:10.1145/2460296.2460337 · Cited by 75 · see citing works
Big Opportunities and Big Concerns of Big Data in Education
Wang, Y. (2016) · TechTrends · doi:10.1007/s11528-016-0072-1 · Cited by 74 · see citing works
Alfy, S. E., Gómez, J. M., & Dani, A. (2018) · Information Discovery and Delivery · doi:10.1108/idd-06-2018-0018 · Cited by 63 · see citing works
Preparing for a Renaissance in Assessment
Hill, P. W., & Barber, M. (2014) · Cited by 59 · see citing works
A Review of Multimodal Learning Analytics Architectures
Shankar, S. K., Prieto, L. P., Rodríguez‐Triana, M. J., & Ruiz‐Calleja, A. (2018) · IEEE International Conference on Advanced Learning Technologies (ICALT) · doi:10.1109/icalt.2018.00057 · Cited by 55 · see citing works
Granularity-Based Reasoning and Belief Revision in Student Models
McCalla, G. I., & Greer, J. E. (1994) · Springer eBooks · doi:10.1007/978-3-662-03037-0_2 · Cited by 51 · see citing works
A Review of Recent Advances in Adaptive Assessment
Vie, J.-J., Popineau, F., Bruillard, É., & Bourda, Y. (2017) · Studies in systems, decision and control · doi:10.1007/978-3-319-52977-6_4 · Cited by 50 · see citing works
Baldwin, D., & Brady, A. F. (2010) · ACM Transactions on Computing Education · doi:10.1145/1731041.1731042 · Cited by 41 · see citing works
Learning Analytics: From Theory to Practice – Data Support for Learning and Teaching
Greller, W., Ebner, M., & Schön, M. (2014) · Communications in computer and information science · doi:10.1007/978-3-319-08657-6_8 · Cited by 38 · see citing works
Aye, S. S., & Marzo, R. R. (2020) · World Journal on Educational Technology Current Issues · doi:10.18844/wjet.v12i3.4992 · Cited by 36 · see citing works
Measurement and its Uses in Learning Analytics
Bergner, Y. (2017) · Society for Learning Analytics Research (SoLAR) eBooks · doi:10.18608/hla17.003 · Cited by 34 · see citing works
Explaining predictive models to learning specialists using personas
Brooks, C., & Greer, J. (2014) · Proceedings of the Fourth International Conference on Learning Analytics And Knowledge · doi:10.1145/2567574.2567612 · Cited by 32 · see citing works
Learning Analytics and the Imperative for Theory-Driven Research
Rogers, T., Dawson, S., & Gaevi, D. (2016) · SAGE Publications Ltd eBooks · doi:10.4135/9781473955011.n12 · Cited by 28 · see citing works
Venant, R., Vidal, P., & Broisin, J. (2016) · IEEE International Conference on Advanced Learning Technologies (ICALT) · doi:10.1109/icalt.2016.60 · Cited by 11 · see citing works
Using Design Patterns to Support E-Learning Design
Frizell, S. S., & Hübscher, R. (2009) · IGI Global eBooks · doi:10.4018/978-1-59904-861-1.ch006 · Cited by 8 · see citing works
Bassen, J., Howley, I. K., Fast, E., Mitchell, J. C., & Thille, C. (2018) · ACM Conference on Learning @ Scale (L@S) · doi:10.1145/3231644.3231669 · Cited by 6 · see citing works
Tateo, L. (2019) · Mind Culture and Activity · doi:10.1080/10749039.2019.1686028 · Cited by 6 · see citing works
Using Design Patterns to Support E-Learning Design
Frizell, S. S., & Hübscher, R. (2011) · IGI Global eBooks · doi:10.4018/978-1-60960-503-2.ch111 · Cited by 4 · see citing works
Flagged first, then by citations
1 of the 64 references couldn’t be matched to a known paper.
Research works printed but not linked: 4
- Bassen, J., Howley, I., Fast, E., Mitchell, J., & Thille, C. (2018, June). OARS: Exploring instructor analytics for online learning. In Proceedings of the Fifth Annual ACM Conference on Learning at Scale (pp. 1-10).
- Karavirta, V., & Shaffer, C. A. (2013, July). JSAV: The JavaScript algorithm visualization library. In Proceedings of the 18th ACM conference on Innovation and technology in computer science education (pp. 159-164).
- Lin, F. (2020). Adaptive quiz generation using Thompson sampling, Third Workshop Eliciting Adaptive Sequences for Learning (WASL 2020), co-located with AIED 2020.
- Rezaie, M., & Golshan, M. (2015). Computer adaptive test (CAT): Advantages and limitations. International Journal of Educational Investigations, 2(5), 128-137.
Cited by
Sorted by citations
Artificial Intelligence for Assessment and Feedback to Enhance Student Success in Higher Education
Hooda, M., Rana, C., Dahiya, O., Rizwan, A., & Hossain, M. S. (2022) · Mathematical Problems in Engineering · doi:10.1155/2022/5215722 · Cited by 444 · see citing works
Effects and acceptance of precision education in an AI-supported smart learning environment
Hu, Y.-H. (2022) · Education and Information Technologies · doi:10.1007/s10639-021-10664-3 · Cited by 68 · see citing works
Ulfatin, N., Putra, A. B. N. R., Heong, Y. M., Zahro, A., & Rahmawati, A. D. (2022) · International Journal of Interactive Mobile Technologies (iJIM) · doi:10.3991/ijim.v16i04.28993 · Cited by 14 · see citing works
The effects of visualisation literacy and data storytelling dashboards on teachers’ cognitive load
Liu, Y., Pozdniakov, S., & Martínez‐Maldonado, R. (2024) · Australasian Journal of Educational Technology · doi:10.14742/ajet.8988 · Cited by 13 · see citing works
Enhancing Collaborative and Self-Paced Learning in Traditional and Distance Education Settings
Ebbini, E. (2023) · Springer eBooks · doi:10.1007/978-3-031-33568-6_12 · Cited by 11 · see citing works
Gavhane, J. M., & Pagare, R. (2024) · Education + Training · doi:10.1108/et-04-2023-0117 · Cited by 11 · see citing works
Creativity in Learning Analytics: A Systematic Literature Review
Mirzaei, S., Nikmehr, H., Liu, S., & Marmolejo‐Ramos, F. (2025) · Journal of Intelligence · doi:10.3390/jintelligence13120153 · Cited by 5 · see citing works
An Intelligent Tutoring System to Support Code Maintainability Skill Development
Luburić, N., Dorić, L., Сливка, Ј., Vidaković, D., Grujić, K.-G., Kovačević, A., & Prokić, S. (2025) · IEEE Transactions on Learning Technologies · doi:10.1109/tlt.2025.3545641 · Cited by 5 · see citing works
Norman, M. K., Radomski, T. R., Mayowski, C. A., Zimmerman-Cooney, M., Crevasse, I., & Rubio, D. M. (2024) · Journal of Clinical and Translational Science · doi:10.1017/cts.2024.601 · Cited by 3 · see citing works
An AI-Learner Shared Control Model Design for Adaptive Practicing
Yan, H., Lin, F. O., & Kinshuk, K. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-63028-6_21 · Cited by 4 · see citing works
Self-citationConstructing Intelligent Learning Dashboard for Online Learners
Farahmand, A., Dewan, M. A. A., & Lin, F. O. (2021) · IEEE Intl Conf on Dependable, Autonomic and Secure Computing, Intl Conf on Pervasive Intelligence and Computing, Intl Conf on Cloud and Big Data Computing, Intl Conf on Cyber Science and Technology Congress (DASC/PiCom/CBDCom/CyberSciTech) · doi:10.1109/dasc-picom-cbdcom-cyberscitech52372.2021.00141 · Cited by 2 · see citing works
Self-citationDarmawan, D., Risda, D., Jenuri, Suwarma, D. M., Rostika, D., & Komariah, K. (2024) · Revista de Gestão Social e Ambiental · doi:10.24857/rgsa.v18n6-147 · Cited by 2 · see citing works
Exploring the Impacts of Implementing Block Mode of Teaching in Higher Education
Trinh, N. T. T., Ghapanchi, A. H., & Purarjomandlangrudi, A. (2024) · International Journal of Emerging Technologies in Learning (iJET) · doi:10.3991/ijet.v19i06.49229 · Cited by 2 · see citing works
(2024) · European Journal of Contemporary Education · doi:10.13187/ejced.2024.1.54 · Cited by 1 · see citing works
Self-directed professional development: An algorithmic and activity-based model for teachers
Sagimbayeva, A., Issabayeva, D., Shekerbekova, S. T., Oshanova, N. T., Smagulova, L., & Tulbassova, B. (2024) · Journal of Infrastructure Policy and Development · doi:10.24294/jipd9246 · Cited by 1 · see citing works
Al-Abbass, A., Loriya, G., Stidd, J., Trask, T., & Lytle, N. (2025) · IEEE Digital Education and MOOCS Conference (DEMOcon) · doi:10.1109/democon65705.2025.11282657 · Cited by 1 · see citing works
(2024) · International Journal of Advanced Technology and Engineering Exploration · doi:10.19101/ijatee.2023.10102002 · Cited by 1 · see citing works
Online and Self-Paced Learning Using AI
(2024) · Advances in higher education and professional development book series · doi:10.4018/979-8-3693-4074-5.ch009 · Cited by 1 · see citing works
Where Is the Teacher in Data Analytics in Education?
Nguyen, N. B. C., & Karunaratne, T. (2024) · International Journal of Emerging Technologies in Learning (iJET) · doi:10.3991/ijet.v19i06.50165
Imran, M., Haque, A. U., & Kozlovski, E. (2024) · International Symposium on Computer Science and Educational Technology (ISCSET) · doi:10.1109/iscset58624.2024.10807946
Liu, Y., & Wang, Y. (2026) · Discover Artificial Intelligence · doi:10.1007/s44163-026-00876-2
Deep reinforcement learning-based optimization design for college data structures course content
Che, D., & Zhou, J. (2026) · Discover Applied Sciences · doi:10.1007/s42452-026-08714-x
Early-Stage Robustness of Knowledge Tracing Models Under Constrained Interaction Budgets
Nurfaizah, Isnanto, R. R., & Tahyudin, I. (2026) · International Conference on Current Research in Artificial Intelligence and Data Science (ICCRAIDS) · doi:10.1109/iccraids67816.2026.11519755
Jony, A. I., Navarra, P. L., & Serradell-López, E. (2026) · Emerald Group Publishing Limited eBooks · doi:10.1108/978-1-83662-878-120261009
- Authors
- Yan, H., Lin, F., & Kinshuk
- Affiliations
- University of Eastern Finland (FI)
- Athabasca University (CA)
- Corresponding author
- Yan, Hongxin
- Venue
- International Journal of Artificial Intelligence in Education
- Publisher
- Elsevier BV
- Published
- 2021-12
- Volume
- 31
- Issue
- 4
- Pages
- 878-895
- ISSN
- 1560-4292 (ISSN-L)
- OpenAlex
- W3112128060
- Cited by
- 25
- References
- 64
- Citations by year
- 2026: 4 · 2025: 3 · 2024: 12 · 2023: 2 · 2022: 2 · 2021: 2
- FWCI
- 1.37
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- International Journal of Artificial Intelligence in Education (publishedVersion, cc-by-nc-nd)
- UEF eRepo (University of Eastern Finland) (submittedVersion, cc-by)
- Funders
- University of Eastern Finland (UEF) including Kuopio University Hospital
- Topics
- Physical Sciences › Computer Science › Computer Science Applications › Online Learning and Analytics
- Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
- Social Sciences › Education › Online and Blended Learning
- Keywords
- learning analytics, self-paced online learning, barriers to learning, student modeling, intelligent tutoring systems, learning awareness, academic intervention, computer science education
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
