CoChat
Sign Up
Cited by 382Online Learning and AnalyticsIntelligent Tutoring Systems and Adaptive LearningOnline and Blended Learning

Student Engagement Predictions in an e-Learning System and Their Impact on Student Course Assessment Scores

Open access
Hussain, Mushtaq, Zhu, Wenhao, Zhang, Wu, Abidi, Syed Muhammad Raza·2018·Computational Intelligence and Neuroscience
DOI 10.1155/2018/6347186PMC PMC6189675
Open this paper in CoChat — free accountOpen PDF

Abstract

Several challenges are associated with e-learning systems, the most significant of which is the lack of student motivation in various course activities and for various course materials. In this study, we used machine learning (ML) algorithms to identify low-engagement students in a social science course at the Open University (OU) to assess the effect of engagement on student performance. The input variables of the study included highest education level , final results , score on the assessment , and the number of clicks on virtual learning environment (VLE) activities, which included dataplus , forumng , glossary , oucollaborate , oucontent , resources , subpages , homepage , and URL during the first course assessment. The output variable was the student level of engagement in the various activities. To predict low-engagement students, we applied several ML algorithms to the dataset. Using these algorithms, trained models were first obtained; then, the accuracy and kappa values of the models were compared. The results demonstrated that the J48, decision tree, JRIP, and gradient-boosted classifiers exhibited better performance in terms of the accuracy, kappa value, and recall compared to the other tested models. Based on these findings, we developed a dashboard to facilitate instructor at the OU. These models can easily be incorporated into VLE systems to help instructors evaluate student engagement during VLE courses with regard to different activities and materials and to provide additional interventions for students in advance of their final exam. Furthermore, this study examined the relationship between student engagement and the course assessment score.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1155/2018/6347186

Cites

3 of 3 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Hussain et al. (2018) used machine learning algorithms to evaluate student engagement in a social science course at the Open University, including final results, assessment scores and the number of clicks that students make in the VLE, which can alert instructors to the need for intervention.

    “Hussain et al. (2018) used machine learning algorithms to evaluate student engagement in a social science course at the Open University, including final results, assessment scores and the number of clicks that students make in the VLE, which can alert instructors to the need for intervention”

    From Systematic review of research on artificial intelligence applications in higher education – where are the educators? (2019) · cites this paper as Hussain et al. (2018)

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 79

  1. [1]Shishehchi S. Banihashem S. Y. Zin N. A. M. Noah S. A. M. Review of personalized recommendation techniques for learners in e-learning systems Proceedings of 2011 International Conference on Semantic Technology and Information Retrieval June 2011 Putrajaya, Malaysia 277 281
  2. [3]Ye C. Biswas G. Early prediction of student dropout and performance in MOOCs using higher granularity temporal information Journal of Learning Analytics 2014 1 3 169 172 10.18608/jla.2014.13.14 · DOI
  3. [4]Mutahi J. Kinai A. Bore N. Diriye A. Weldemariam K. Studying engagement and performance with learning technology in an African classroom Proceedings of Seventh International Learning Analytics & Knowledge Conference March 2017 Vancouver, BC, Canada 148 152
  4. [5]Simpson O. 22% - can we do better?’ The CWP Retention Literature Review Report to the Centre for Widening Participation, Open University, 2010
  5. [6]Kuzilek J. Hlosta M. Herrmannova D. Zdrahal Z. Wolff A. OU analyse: analysing at-risk students at the Open University Proceedings of First International Workshop on Visual Aspects of Learning Analytics and Knowledge Conference (LAK 2015) March 2015 Poughkeepsie, NY, USA 1 16
  6. [7]Jung Y. Lee J. Learning engagement and persistence in massive open online courses (MOOCS) Computers & Education 2018 122 9 22 10.1016/j.compedu.2018.02.013 2-s2.0-85044101623 · DOI
  7. [8]Kushwaha R. C. Singhal A. Chaurasia P. K. Study of students’ performance in learning management system International Journal of Contemporary Research in Computer Science and Technology (IJCRCST) 2015 1 6 213 217
  8. [9]Holmes N. Engaging with assessment: increasing student engagement through continuous assessment Active Learning in Higher Education 2018 19 1 23 34 10.1177/1469787417723230 2-s2.0-85042282695 · DOI · Paper page
  9. [10]Blin F. Munro M. Why hasn’t technology disrupted academics’ teaching practices? Understanding resistance to change through the lens of activity theory Computers & Education 2008 50 2 475 490 10.1016/j.compedu.2007.09.017 2-s2.0-38149083494 · DOI
  10. [11]Herrmannova D. Hlosta M. Kuzilek J. Zdrahal Z. Evaluating weekly predictions of at-risk students at the open university: results and issues Proceedings of EDEN 2015 Annual Conference Expanding Learning Scenarios: Opening Out the Educational Landscape July 2015 Spain Barcelona 9 12
  11. [12]Mogus A. M. Djurdjevic I. Suvak N. The impact of student activity in a virtual learning environment on their final mark Active Learning in Higher Education 2012 13 3 177 189 10.1177/1469787412452985 2-s2.0-84869075693 · DOI
  12. [13]Hlosta M. Zdrahal Z. Zendulka J. Ouroboros: early identification of at-risk students without models based on legacy data Proceedings of Seventh International Learning Analytics & Knowledge Conference March 2017 Vancouver, British Columbia, Canada 6 15
  13. [15]Staikopoulos A. O’Keeffe I. Yousuf B. Enhancing student engagement through personalized motivations Proceedings of IEEE 15th International Conference on Advanced Learning Technologies July 2015 Hualien, Taiwan 340 344
  14. [16]Wang Y. Baker R. Content or platform: why do students complete MOOCs? Journal of Online Learning and Teaching 2015 11 1 17 30
  15. [18]Beer C. Online student engagement: new measures for new methods Masters Dissertation, CQUniversity, Rockhampton, QLD, Australia, 2010
  16. [19]Douglas I. Alemanne N. D. Measuring student participation and effort Proceedings of International Conference on Cognition and Exploratory Learning in Digital Age December 2007 Algarve, Portugal 299 302
  17. [20]Bulger M. E. Mayer R. E. Almeroth K. C. Blau S. D. Measuring learner engagement in computer-equipped college classrooms Journal of Educational Multimedia and Hypermedia 2008 17 2 129 143
  18. [22]Corrigan O. Smeaton A. F. Glynn M. Smyth S. Using educational analytics to improve test performance Proceedings of EC-TEL 2015 10th European Conference on Technology Enhanced Learning September 2015 Toledo, Spain 15 18
  19. [23]Jayaprakash S. M. Moody E. W. Lauría E. J. M. Regan J. R. Baron J. D. Early alert of academically at-risk students: an open source analytics initiative Journal of Learning Analytics 2014 1 1 6 47 10.18608/jla.2014.11.3 · DOI
  20. [24]Holland J. H. Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control and Artificial Intelligence 1992 Cambridge, MA, USA MIT Press
  21. [25]Greller W. Ebner M. Schön M. Kalz M. Ras E. Learning analytics: from theory to practice–data support for learning and teaching Computer Assisted Assessment. Research into E-Assessment 2014 Berlin, Germany Springer 79 87
  22. [26]Leitner P. Khalil M. Ebner M. Peña-Ayala A. Learning analytics in higher education—a literature review Learning Analytics: Fundaments, Applications, and Trends: A View of the Current State of the Art to Enhance E-Learning 2017 Berlin, Germany Springer 1 23
  23. [27]Wolff A. Zdrahal Z. Herrmannova D. Knoth P. Peña-Ayala A. Predicting student performance from combined data sources Educational Data Mining. Studies in Computational Intelligence 2013 Berlin, Germany Springer 175 202
  24. [28]Kotsiantis S. Pierrakeas C. Pintelas P. Predicting students’ performance in distance learning using machine learning techniques Applied Artificial Intelligence 2004 18 5 411 426 10.1080/08839510490442058 2-s2.0-2942552288 · DOI
  25. [29]Arnold K. E. Pistilli M. D. Course signals at Purdue: using learning analytics to increase student success Proceedings of 2nd International Conference on Learning Analytics and Knowledge April 2012 Vancouver, British Columbia, Canada 267 270
  26. [30]Kai S. Andres J. M. L. Paquette L. Predicting student retention from behaviour in an online orientation course Proceedings of 10th International Conference on Educational Data Mining 2016 250 255
  27. [32]Krause K. L. Coates H. Students’ engagement in first-year university Assessment & Evaluation in Higher Education 2008 33 5 493 505 10.1080/02602930701698892 2-s2.0-77950361434 · DOI · Paper page
  28. [33]Zhuang M. Demartini G. Toms E. G. Understanding engagement through search behaviour Proceedings of 2017 ACM on Conference on Information and Knowledge Management November 2017 Singapore 1957 1966
  29. [34]Hlosta M. Herrmannova D. Vachova L. Modelling student online behaviour in a virtual learning environment Proceedings of Seventh International Learning Analytics & Knowledge Conference on (LAK’17) March 2014 Indianapolis, IN, USA
  30. [35]Mitchell T. Buchanan B. De Jong G. Dietterich T. Rosenbloom P. Waibel A. Machine learning Annual Review of Computer Science 1989 4 1 417 433 10.1146/annurev.cs.04.060190.002221 · DOI
  31. [36]Wolff A. Zdrahal Z. Herrmannova D. Kuzilek J. Hlosta M. Developing predictive models for early detection of at-risk students on distance learning modules Proceedings of 4th International Learning Analytics & Knowledge Conference on (LAK’14) March 2014 Indianapolis, IN, USA
  32. [37]Guo P. J. Kim J. Rubin R. How video production affects student engagement: an empirical study of MOOC videos Proceedings of First ACM Conference on Learning @ Scale March 2014 Atlanta, GA, USA 41 50
  33. [38]Bonafini F. C. Chae C. Park E. Jablokow K. W. How much does student engagement with videos and forums in a MOOC affect their achievement? Online Learning 2017 21 4 223 240 10.24059/olj.v21i4.1270 2-s2.0-85036605046 · DOI
  34. [39]Ramesh A. Goldwasser D. Huang B. Daum H. Getoor L. Learning latent engagement patterns of students in online courses Proceedings of Twenty-Eighth AAAI Conference on Artificial Intelligence July 2014 Québec, Canada AAAI Press 1272 1278
  35. [40]Ramesh A. Goldwasser D. Huang B. Daumé H. III Getoor L. Modeling learner engagement in MOOCs using probabilistic soft logic Proceedings of NIPS Workshop on Data Driven Education December 2013 Lake Tahoe, NV, USA p. 62
  36. [41]Manwaring K. C. Larsen R. Graham C. R. Henrie C. R. Halverson L. R. Investigating student engagement in blended learning settings using experience sampling and structural equation modelling Internet and Higher Education 2017 35 21 33 10.1016/j.iheduc.2017.06.002 2-s2.0-85025676847 · DOI
  37. [42]Aguiar E. Ambrose G. A. Chawla N. V. Goodrich V. Brockman J. Engagement vs performance: using electronic portfolios to predict first semester engineering student persistence Journal of Learning Analytics 2014 1 3 7 33 10.18608/jla.2014.13.3 · DOI
  38. [43]Thomas C. Jayagopi D. B. Predicting student engagement in classrooms using facial behavioral cues Proceedings of 1st ACM SIGCHI International Workshop on Multimodal Interaction for Education November 2017 Glasgow, UK 33 40
  39. [45]Bosch N. Detecting student engagement: human versus machine Proceedings of 24th International Conference on User Modelling, Adaptation and Personalization July 2016 Halifax, Nova Scotia, Canada 317 320
  40. [46]Kizilcec R. F. Piech C. Schneider E. Deconstructing disengagement: analyzing learner subpopulations in massive open online courses Proceedings of 3rd International Conference on Learning Analytics and Knowledge April 2013 Leuven, Belgium 170 179
  41. [47]Milligan C. Littlejohn A. Margaryan A. Patterns of engagement in connectivist MOOCs Journal of Online Learning and Teaching 2013 9 2 149 159
  42. [48]Ding L. Er E. Orey M. An exploratory study of student engagement in gamified online discussions Computers & Education 2018 120 213 226 10.1016/j.compedu.2018.02.007 2-s2.0-85042215952 · DOI
  43. [49]Wells M. Wollenschlaeger A. Lefevre D. Magoulas G. D. Poulovassilis A. Analysing engagement in an online management programme and implications for course design Proceedings of Sixth International Conference on Learning Analytics & Knowledge April 2016 Edinburgh, UK 236 240
  44. [50]Pardo A. Han F. Ellis R. A. Exploring the relation between self-regulation, online activities, and academic performance: a case study Proceedings of Sixth International Conference on Learning Analytics & Knowledge April 2016 Edinburgh, UK 422 429
  45. [51]Bahati B. Uno F. Tedre M. Can student engagement in online courses predict performance on online knowledge surveys? International Journal of Learning, Teaching and Educational Research 2017 16 3 73 87
  46. [52]Hamid S. S. A. Admodisastro N. Manshor N. Kamaruddin A. Ghani A. A. A. Ghazali R. Deris M. Nawi N. Abawajy J. Dyslexia adaptive learning model: student engagement prediction using machine learning approach Advances in Intelligent Systems and Computing 2018 Berlin, Germany Springer 372 384
  47. [53]Bote-Lorenzo M. L. Gomez-Sanchez E. Predicting the decrease of engagement indicators in a MOOC Proceedings of Seventh International Conference on Learning Analytics and Knowledge March 2017 Vancouver, British Columbia, Canada 143 147
  48. [54]Marks H. M. Student engagement in instructional activity: patterns in the elementary middle and high school years American Educational Research Journal 2000 37 1 153 472 10.2307/1163475 · DOI
  49. [55]Kuh G. D. Assessing what really matters to student learning inside the national survey of student engagement Change: The Magazine of Higher Learning 2001 33 3 10 17 10.1080/00091380109601795 · DOI · Paper page
  50. [56]Rodgers T. Student engagement in the e-learning process and the impact on their grade International Journal of Cyber Society and Education 2008 1 3 143 156
  51. [57]Kabra R. Bichkar R. Performance prediction of engineering students using decision trees International Journal of Computer Applications 2011 36 11 8 12
  52. [58]Sharma A. K. Sahni S. A comparative study of classification algorithms for spam email data analysis International Journal on Computer Science and Engineering 2011 3 5 1890 1895
  53. [59]Asif R. Merceron A. Pathan M. K. Predicting student academic performance at degree level: a case study International Journal of Intelligent Systems and Applications 2014 7 1 49 61 10.5815/ijisa.2015.01.05 · DOI
  54. [60]Wolff A. Zdrahal Z. Improving retention by identifying and supporting at-risk students Educause Review Online, 2012
  55. [61]Rajput A. Aharwal R. P. Dubey M. Saxena S. Raghuvanshi M. J48 and JRIP rules for e-governance data International Journal of Computer Science and Security (IJCSS) 2011 5 2 201 207
  56. [63]Lakshmi T. M. Martin A. Begum R. M. Venkatesan V. P. An analysis on performance of decision tree algorithms using student’s qualitative data International Journal of Modern Education and Computer Science 2013 5 5 18 27 10.5815/ijmecs.2013.05.03 · DOI
  57. [64]Márquez-Vera C. Cano A. Romero C. Noaman A. Y. M. Mousa Fardoun H. Ventura S. Early dropout prediction using data mining: a case study with high school students Expert Systems 2015 33 1 107 124 10.1111/exsy.12135 2-s2.0-84958177306 · DOI
  58. [65]Cohen W. W. Fast effective rule induction Proceedings of Twelfth International Conference on Machine Learning July 1995 Tahoe City, CA, USA 115 123
  59. [66]O’Mahony M. P. Cunningham P. Smyth B. An assessment of machine learning techniques for review recommendation Proceedings of Irish Conference on Artificial Intelligence and Cognitive Science August 2009 Dublin, Ireland Springer 241 250
  60. [67]Mousa H. Maghari A. School student’s performance prediction using data mining classification International Journal of Advance Research in Computer and Communication Technology 2017 6 8 136 141
  61. [68]Kuzilek J. Hlosta M. Zdrahal Z. Open university learning analytics dataset Scientific Data 2017 4 170171 10.1038/sdata.2017.171 2-s2.0-85036530553 PMC5704676 29182599 · DOI
  62. [69]Parsons S. A. Nuland L. R. Parsons A. W. The ABCs of student engagement Phi Delta Kappan 2014 95 8 23 27 10.1177/003172171409500806 2-s2.0-84898942236 · DOI
  63. [70]Siemens G. Connectivism: a learning theory for the digital age International Journal of Instructional Technology and Distance Learning 2004 2 1 3 10
  64. [72]Vrasidas C. McIsaac M. S. Factors influencing interaction in an online course American Journal of Distance Education 1999 13 3 22 36 10.1080/08923649909527033 · DOI
  65. [73]Albuquerque R. M. D. Bezerra A. A. Souza D. A. D. Using neural networks to predict the future performance of students Proceedings of International Symposium on Computers in Education (SIIE) November 2015 Setubal, Portugal 109 113
  66. [74]Cawley G. C. Talbot N. L. C. Gene selection in cancer classification using sparse logistic regression with Bayesian regularization Bioinformatics 2006 22 19 2348 2355 10.1093/bioinformatics/btl386 2-s2.0-33750012146 16844704 · DOI
  67. [75]Rovira S. Puertas E. Igual L. Data-driven system to predict academic grades and dropout PLoS One 2017 12 2 e0171207 10.1371/journal.pone.0171207 2-s2.0-85012115729 PMC5308611 28196078 · DOI
  68. [76]Hussain M. Zhu W. Zhang W. Abidi S. M. R. Ali S. Using machine learning to predict student difficulties from learning session data Artificial Intelligence Review 2018 1 27 10.1007/s10462-018-9620-8 2-s2.0-85041857282 · DOI
  69. [77]Freitas A. A. Data Mining and Knowledge Discovery with Evolutionary Algorithms 2002 Berlin, Germany Springer
  70. [78]Villagrá-Arnedo C. Gallego-Duraìn F. J. Compan˜-Rosique P. Llorens-Largo F. Molina-Carmona R. Predicting academic performance from behavioural and learning data International Journal of Design & Nature and Ecodynamics 2016 11 3 239 249 10.2495/dne-v11-n3-239-249 2-s2.0-84991487907 · DOI
  71. [80]Zacharis N. Z. A multivariate approach to predicting student outcomes in web-enabled blended learning courses Internet and Higher Education 2015 27 44 53 10.1016/j.iheduc.2015.05.002 2-s2.0-84929643228 · DOI
  72. [81]Luan J. Zhao C.-M. Practicing data mining for enrolment management and beyond New Directions for Institutional Research 2006 2006 131 117 122 10.1002/ir.191 · DOI
  73. [82]Kovacic Z. Early prediction of student success: mining students’ enrolment data Proceedings of Informing Science and IT Education Conference (InSITE) June 2010 Cassino, Italy Open Polytechnic 647 665
  74. [83]Wolff A. Zdrahal Z. Nikolov A. Pantucek M. Improving retention: predicting at-risk students by analysing clicking behaviour in a virtual learning environment Proceedings of Third International conference on Learning Analytics and Knowledge April 2013 Leuven, Belgium 145 149
  75. [84]Jung I. Choi S. Lim C. Leem J. Effects of different types of interaction on learning achievement, satisfaction and participation in web-based instruction Innovations in Education and Teaching International 2002 39 2 153 162 10.1080/14703290252934603 2-s2.0-0036587866 · DOI
  76. [85]Roberts J. Styron R. Student satisfaction and persistence: factors vital to student retention Research in Higher Education Journal 2010 6 3 1 18
  77. [87]Romero C. López M.-I. Luna J.-M. Ventura S. Predicting students’ final performance from participation in on-line discussion forums Computers & Education 2013 68 458 472 10.1016/j.compedu.2013.06.009 2-s2.0-84880300450 · DOI
  78. [88]Sathyadevan S. Nair R. R. Comparative analysis of decision tree algorithms: ID3, C4.5 and random forest Proceedings of International Conference on Computational Intelligence in Data Mining December 2015 New Delhi, India
  79. [89]Skinner E. A. Pitzer J. R. Christenson S. L. Reschly A. L. Wylie C. Developmental dynamics of student engagement, coping, and everyday resilience Handbook of Research on Student Engagement 2012 Berlin, Germany Springer 21 44

Cited by

Sorted by citations

Authors
Hussain, M., Zhu, W., Zhang, W., & Abidi, S. M. R.
Affiliations
Shanghai University (CN)
Corresponding author
Zhu, Wenhao
ORCID
Venue
Computational Intelligence and Neuroscience
Publisher
Wiley
Published
2018-10-02
Volume
2018
Pages
1-21
ISSN
  • 1687-5265 (ISSN-L)
  • 1687-5273
DOI
10.1155/2018/6347186
PMC
PMC6189675
OpenAlex
W2895615678
Cited by
382
References
3
Citations by year
2026: 32 · 2025: 47 · 2024: 53 · 2023: 55 · 2022: 81 · 2021: 71 · 2020: 33 · 2019: 9 · 2016: 1
FWCI
35.73
Type
journal article
Language
en
Open access
Licence
Locations
  • Computational Intelligence and Neuroscience (publishedVersion, cc-by)
  • Europe PMC (PubMed Central) (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
Funders
  • National Natural Science Foundation of China (61572434, 91630206, 2017YFB0701501)
  • National Key R&D Program of China (61572434, 91630206, 2017YFB0701501)
Topics
  • Physical Sciences › Computer Science › Computer Science Applications › Online Learning and Analytics
  • Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
  • Social Sciences › Education › Online and Blended Learning
Keywords
student engagement, online learning, virtual learning environments, student performance prediction, social science courses, Open University, machine learning classifiers, J48 decision tree, student motivation
MeSH terms
Machine Learning, Stakeholder Participation, Academic Success, Computer-Assisted Instruction, Humans, Social Sciences, Students, Universities, Internet

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review