The DOI matches its Crossref record; 105 of the 105 references it cites were found in a registry or an index.
What predicts student satisfaction with MOOCs: A gradient boosting trees supervised machine learning and sentiment analysis approach
Abstract
This study defines MOOC success as the extent of student satisfaction with the course. Having more satisfied MOOC students can extend the reach of an institution to more people, build the brand name of the institution, and even help the institution use MOOCs as a source of revenue. Traditionally, student completion rate is frequently used to define MOOC success, which however, is often inaccurate because many students have no intention of finishing a MOOC. Informed by Moore's theory of transactional distance, this study adopted supervised machine learning algorithm, sentiment analysis and hierarchical linear modelling to analyze the course features of 249 randomly sampled MOOCs and 6393 students' perceptions of these MOOCs. The results showed that course instructor, content, assessment, and schedule play significant roles in explaining student satisfaction, while course structure, major, duration, video, interaction, perceived course workload and perceived difficulty play no significant roles. This study adds to the extant literature by examining specific learner-level and course-level factors that can predict MOOC learner satisfaction and estimating their relative effects. Implications for MOOC instructors and practitioners are also provided.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Cortes, C., & Vapnik, V. (1995) · Machine Learning · doi:10.1007/bf00994018 · Cited by 59,421
Greedy function approximation: A gradient boosting machine
Friedman, J. H. (2001) · The Annals of Statistics · doi:10.1214/aos/1013203451 · Cited by 30,731
Hierarchical Linear Models: Applications and Data Analysis Methods.
Longford, N. T., Bryk, A. S., & Raudenbush, S. W. (1993) · Contemporary Sociology · doi:10.2307/2075823 · Cited by 18,724
Hierarchical Linear Models: Applications and Data Analysis Methods
Davidian, M. (2003) · Journal of the American Statistical Association · doi:10.1198/jasa.2003.s288 · Cited by 13,723
A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection
Kohavi, R. (1995) · Zenodo · doi:10.5281/zenodo.19712698 · Cited by 10,909
(2012) · Cited by 9,062
(2012) · Cited by 8,009
Multilevel analysis techniques and applications
Hox, J. J., Moerbeek, M., & van de Schoot, R. (2002) · Cited by 6,735
What Matters in College? Four Critical Years Revisited.
Feldman, K. A., & Astin, A. W. (1994) · The Journal of Higher Education · doi:10.2307/2943781 · Cited by 5,745
An Introduction to Kernel and Nearest-Neighbor Nonparametric Regression
Altman, N. S. (1992) · The American Statistician · doi:10.1080/00031305.1992.10475879 · Cited by 5,708
Sentiment Analysis and Opinion Mining
Liu, B. (2012) · Synthesis Lectures on Human Language Technologies · doi:10.1007/978-3-031-02145-9 · Cited by 3,281
Sentiment Analysis and Opinion Mining
Zhang, L., & Liu, B. (2017) · Encyclopedia of Machine Learning and Data Mining · doi:10.1007/978-1-4899-7687-1_907 · Cited by 3,260
The Adult Learner: The Definitive Classic in Adult Education and Human Resource Development
Knowles, M. S., Holton, E. F., Robinson, P. A., & Caraccioli, C. (2005) · Civil War Book Review · Cited by 2,860
The Regression Analysis of Binary Sequences
Cox, D. R. (1958) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1958.tb00292.x · Cited by 2,813
Editorial: Three types of interaction
Moore, M. G. (1989) · American Journal of Distance Education · doi:10.1080/08923648909526659 · Cited by 2,471
Engagement and disaffection in the classroom: Part of a larger motivational dynamic?
Skinner, E., Furrer, C., Marchand, G., & Kindermann, T. (2008) · Journal of Educational Psychology · doi:10.1037/a0012840 · Cited by 2,037
How video production affects student engagement: an empirical study of MOOC videos
Guo, P. J., Kim, J., & Rubin, R. (2014) · Proceedings of the first ACM conference on Learning @ scale conference · doi:10.1145/2556325.2566239 · Cited by 1,850
Selecting and interpreting measures of thematic classification accuracy
Stehman, S. V. (1997) · Remote Sensing of Environment · doi:10.1016/s0034-4257(97)00083-7 · Cited by 1,729
Suthaharan, S. (2016) · Integrated Series in Information Systems · doi:10.1007/978-1-4899-7641-3_9 · Cited by 1,579
Encyclopedia of Language and Linguistics
Kehrein, W. (2006) · Elsevier eBooks · Cited by 1,200
Kuo, Y.-C., Walker, A. E., Schroder, K. E. E., & Belland, B. R. (2014) · The Internet and Higher Education · doi:10.1016/j.iheduc.2013.10.001 · Cited by 1,082
The Theory of Transactional Distance
Moore, M. G. (2015) · Handbook of Distance Education · doi:10.4324/9780203803738.ch5 · Cited by 1,070
Multilevel Analysis: Techniques and Applications
Robinson, T. J. (2003) · Journal of the American Statistical Association · doi:10.1198/jasa.2003.s281 · Cited by 1,039
A study of student satisfaction in a blended e-learning system environment
Wu, J.-H., Tennyson, R. D., & Hsia, T.-L. (2010) · Computers & Education · doi:10.1016/j.compedu.2009.12.012 · Cited by 1,001
Deconstructing disengagement: analyzing learner subpopulations in massive open online courses
Kizilcec, R. F., Piech, C., & Schneider, E. (2013) · Proceedings of the Third International Conference on Learning Analytics and Knowledge · doi:10.1145/2460296.2460330 · Cited by 984
Eom, S. B., Wen, H. J., & Ashill, N. (2006) · Decision Sciences Journal of Innovative Education · doi:10.1111/j.1540-4609.2006.00114.x · Cited by 876
Initial trends in enrolment and completion of massive open online courses
Jordan, K. (2014) · The International Review of Research in Open and Distributed Learning · doi:10.19173/irrodl.v15i1.1651 · Cited by 862
Getting the Mix Right Again: An Updated and Theoretical Rationale for Interaction
Anderson, T. (2003) · The International Review of Research in Open and Distributed Learning · doi:10.19173/irrodl.v4i2.149 · Cited by 810
Richardson, J. C., Maeda, Y., Lv, J., & Caskurlu, S. (2017) · Computers in Human Behavior · doi:10.1016/j.chb.2017.02.001 · Cited by 787
Virtual Classroom Characteristics and Student Satisfaction with Internet-Based MBA Courses
Arbaugh, J. B. (2000) · Journal of Management Education · doi:10.1177/105256290002400104 · Cited by 742
Paechter, M., Maier, B., & Macher, D. (2010) · Computers & Education · doi:10.1016/j.compedu.2009.08.005 · Cited by 731
Exploring the factors affecting MOOC retention: A survey study
Hone, K. S., & El Said, G. R. (2016) · Computers & Education · doi:10.1016/j.compedu.2016.03.016 · Cited by 720
Key Factors for Determining Student Satisfaction in Online Courses
Bolliger, D. U. (2004) · International journal on e-learning · Cited by 719
Understanding the MOOCs continuance: The role of openness and reputation
Alraimi, K. M., Zo, H., & Ciganek, A. P. (2015) · Computers & Education · doi:10.1016/j.compedu.2014.08.006 · Cited by 657
Measuring student satisfaction at a UK university
Douglas, J., Douglas, A., & Barnes, B. (2006) · Quality Assurance in Education · doi:10.1108/09684880610678568 · Cited by 616
The Adult Learner: The Definitive Classic in Adult Education and Human Resource Development
(2012) · Industrial and Commercial Training · doi:10.1108/00197851211268045 · Cited by 561
The student-as-consumer approach in higher education and its effects on academic performance
Bunce, L., Baird, A., & Jones, S. E. (2017) · Studies in Higher Education · doi:10.1080/03075079.2015.1127908 · Cited by 537
Massive open online course completion rates revisited: Assessment, length and attrition
Jordan, K. (2015) · The International Review of Research in Open and Distributed Learning · doi:10.19173/irrodl.v16i3.2112 · Cited by 508
Promoting engagement in online courses: What strategies can we learn from three highly rated MOOCS
Hew, K. F. (2016) · British Journal of Educational Technology · doi:10.1111/bjet.12235 · Cited by 463
Self-citationHow Video Production Affects Student Engagement:An Empirical Study of MOOC Videos
McVerry, G. (2019) · Cited by 431
Joo, Y. J., So, H.-J., & Kim, N. H. (2018) · Computers & Education · doi:10.1016/j.compedu.2018.01.003 · Cited by 420
Joo, Y. J., Lim, K. Y., & Kim, E. K. (2011) · Computers & Education · doi:10.1016/j.compedu.2011.02.008 · Cited by 397
The influence of internet customer reviews on the online sales and prices in hotel industry
Öğüt, H., & Onur Taş, B. K. (2012) · The Service Industries Journal · doi:10.1080/02642069.2010.529436 · Cited by 378
Editorial: Distance education theory
Moore, M. G. (1991) · American Journal of Distance Education · doi:10.1080/08923649109526758 · Cited by 367
“Predicting MOOC dropout over weeks using machine learning methods,”
Kloft, M., Stiehler, F., Zheng, Z., & Pinkwart, N. (2014) · Proceedings of the EMNLP 2014 Workshop on Analysis of Large Scale Social Interaction in MOOCs · doi:10.3115/v1/w14-4111 · Cited by 325
Artino, A. R. (2008) · Journal of Computer Assisted Learning · doi:10.1111/j.1365-2729.2007.00258.x · Cited by 310
Sentiment Analysis in MOOC Discussion Forums: What does it tell us?
Wen, M., Yang, D., & Rosé, C. P. (2014) · Educational Data Mining · Cited by 288
What Satisfies Students? Mining Student-Opinion Data with Regression and Decision Tree Analysis
Thomas, E. H., & Galambos, N. (2004) · Research in Higher Education · doi:10.1023/b:rihe.0000019589.79439.6e · Cited by 280
Maki, R. H., Maki, W. S., Patterson, M., & Whittaker, P. D. (2000) · Behavior Research Methods, Instruments, & Computers · doi:10.3758/bf03207788 · Cited by 279
Progress and new directions for teaching and learning in MOOCs
Deng, R., Benckendorff, P., & Gannaway, D. (2019) · Computers & Education · doi:10.1016/j.compedu.2018.10.019 · Cited by 273
Ginns, P., Prosser, M., & Barrie, S. (2007) · Studies in Higher Education · doi:10.1080/03075070701573773 · Cited by 268
A systematic review of research methods and topics of the empirical MOOC literature (2014–2016)
Zhu, M., Sari, A., & Lee, M. M. (2018) · The Internet and Higher Education · doi:10.1016/j.iheduc.2018.01.002 · Cited by 243
Predictors of Retention and Achievement in a Massive Open Online Course
Greene, J. A., Oswald, C. A., & Pomerantz, J. (2015) · American Educational Research Journal · doi:10.3102/0002831215584621 · Cited by 222
Visualizing patterns of student engagement and performance in MOOCs
Coffrin, C., Corrin, L., de Barba, P., & Kennedy, G. (2014) · Proceedings of the Fourth International Conference on Learning Analytics And Knowledge · doi:10.1145/2567574.2567586 · Cited by 216
Falloon, G. (2011) · Journal of Research on Technology in Education · doi:10.1080/15391523.2011.10782569 · Cited by 215
Democratizing higher education: Exploring MOOC use among those who cannot afford a formal education
Dillahunt, T. R., Wang, B., & Teasley, S. D. (2014) · The International Review of Research in Open and Distributed Learning · doi:10.19173/irrodl.v15i5.1841 · Cited by 208
Bridging the Transactional Distance Gap in Online Learning Environments
Stein, D. S., Wanstreet, C. E., Calvin, J., Overtoom, C., & Wheaton, J. E. (2005) · American Journal of Distance Education · doi:10.1207/s15389286ajde1902_4 · Cited by 208
A Review of Missing Data Handling Methods in Education Research
Cheema, J. R. (2014) · Review of Educational Research · doi:10.3102/0034654314532697 · Cited by 205
Follow the successful crowd: raising MOOC completion rates through social comparison at scale
Davis, D., Jivet, I., Kizilcec, R. F., Chen, G., Hauff, C., & Houben, G. (2017) · International Conference on Learning Analytics and Knowledge (LAK) · doi:10.1145/3027385.3027411 · Cited by 201
MOOC Pedagogy: Gleaning Good Practice from Existing MOOCs
Bali, M. (2014) · Cited by 198
Building a theoretical framework of web‐based instruction in the context of distance education
Jung, I. (2001) · British Journal of Educational Technology · doi:10.1111/1467-8535.00222 · Cited by 196
Refining success and dropout in massive open online courses based on the intention–behavior gap
Henderikx, M. A., Kreijns, K., & Kalz, M. (2017) · Distance Education · doi:10.1080/01587919.2017.1369006 · Cited by 191
Luu, K., & Freeman, J. G. (2011) · Computers & Education · doi:10.1016/j.compedu.2010.11.008 · Cited by 189
Should your MOOC forum use a reputation system?
Coetzee, D., Fox, A., Hearst, M. A., & Hartmann, B. (2014) · Proceedings of the 17th ACM conference on Computer supported cooperative work & social computing · doi:10.1145/2531602.2531657 · Cited by 179
Virtual Communities and the Assessment of Online Marketing Education
Peltier, J. W., Drago, W., & Schibrowsky, J. A. (2003) · Journal of Marketing Education · doi:10.1177/0273475303257762 · Cited by 174
Attrition in MOOC: Lessons Learned from Drop-Out Students
Gütl, C., Rizzardini, R. H., Chang, V., & Morales, M. (2014) · Communications in Computer and Information Science · doi:10.1007/978-3-319-10671-7_4 · Cited by 174
Rhode, J. (2009) · The International Review of Research in Open and Distributed Learning · doi:10.19173/irrodl.v10i1.603 · Cited by 172
Factors influencing peer learning and performance in MOOC asynchronous online discussion forum
Chiu, T. K. F., & Hew, T. K. F. (2018) · Australasian Journal of Educational Technology · doi:10.14742/ajet.3240 · Cited by 171
Exploring learner to content interaction as a success factor in online courses
Zimmerman, T. D. (2012) · The International Review of Research in Open and Distributed Learning · doi:10.19173/irrodl.v13i4.1302 · Cited by 169
Lee, J. (2014) · The International Review of Research in Open and Distributed Learning · doi:10.19173/irrodl.v15i1.1638 · Cited by 168
Student evaluation of courses: what predicts satisfaction?
Denson, N., Loveday, T., & Dalton, H. M. (2010) · Higher Education Research & Development · doi:10.1080/07294360903394466 · Cited by 165
Li, K. (2019) · Computers & Education · doi:10.1016/j.compedu.2019.01.003 · Cited by 163
Xiong, Y., Li, H., Kornhaber, M. L., Suen, H. K., Pursel, B. K., & Goins, D. D. (2015) · Global education review · Cited by 159
Yükseltürk, E., & Yıldırım, Z. (2008) · OpenMETU (Middle East Technical University) · Cited by 157
Predicting Student Retention in Massive Open Online Courses using Hidden Markov Models
Balakrishnan, G. (2013) · UC Berkeley · Cited by 156
Theoretical Principles of Distance Education
Keegan, D. J. (2005) · Routledge eBooks · doi:10.4324/9780203983065 · Cited by 156
Systematic Review of Discussion Forums in Massive Open Online Courses (MOOCs)
Almatrafi, O., & Johri, A. (2018) · IEEE Transactions on Learning Technologies · doi:10.1109/tlt.2018.2859304 · Cited by 126
Li, Q., & Baker, R. B. (2018) · Computers & Education · doi:10.1016/j.compedu.2018.08.005 · Cited by 124
Adult MOOC Learners as Self-Directed: Perceptions of Motivation, Success, and Completion
Loizzo, J., Ertmer, P. A., Watson, W. R., & Watson, S. L. (2017) · Online Learning · doi:10.24059/olj.v21i2.889 · Cited by 119
Reconsidering Moore's Transactional Distance Theory
Giossos, Y. P., Koutsouba, M. I., Lıonarakıs, A., & Skavantzos, K. (2009) · European Journal of Open Distance and E-Learning · Cited by 111
Hew, K. F., Qiao, C., & Tang, Y. (2018) · The International Review of Research in Open and Distributed Learning · doi:10.19173/irrodl.v19i3.3596 · Cited by 109
Self-citationAssessing learners’ satisfaction in collaborative online courses through a big data approach
Elia, G., Solazzo, G., Lorenzo, G., & Passıante, G. (2018) · Computers in Human Behavior · doi:10.1016/j.chb.2018.04.033 · Cited by 108
Said, G. R. E. (2016) · Journal of Educational Computing Research · doi:10.1177/0735633116681302 · Cited by 107
An empirical investigation of the antecedents of learner-centered outcome measures in MOOCs
Rabin, E., Kalman, Y. M., & Kalz, M. (2019) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-019-0144-3 · Cited by 106
Predictors of Student Satisfaction with University Psychology Courses: A Review
Green, H. J., Hood, M., & Neumann, D. L. (2015) · Psychology Learning & Teaching · doi:10.1177/1475725715590959 · Cited by 105
Learner Satisfaction with Massive Open Online Courses
Gameel, B. G. (2017) · American Journal of Distance Education · doi:10.1080/08923647.2017.1300462 · Cited by 103
Brooker, A., Corrin, L., Barba, P. D., Lodge, J. M., & Kennedy, G. E. (2017) · Australasian Journal of Educational Technology · doi:10.14742/ajet.3237 · Cited by 102
Why do Institutions Offer MOOCs?
Hollands, F. M., & Tirthali, D. (2014) · Online Learning · doi:10.24059/olj.v18i3.464 · Cited by 101
Interaction Online: A Reevaluation
Battalio, J. T. (2007) · ScholarWorks (Boise State University) · Cited by 82
Does instructor's image size in video lectures affect learning outcomes?
Pi, Z., Hong, J. K., & Yang, J. (2017) · Journal of Computer Assisted Learning · doi:10.1111/jcal.12183 · Cited by 70
Hew, K. F. (2018) · Teachers College Record The Voice of Scholarship in Education · doi:10.1177/016146811812000107 · Cited by 65
Self-citationEngaging with Massive Online CoursesPreprint
Anderson, A., Huttenlocher, D. P., Kleinberg, J. M., & Leskovec, J. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1403.3100 · Cited by 61
Watson, S. L., Watson, W. R., Janakiraman, S., & Richardson, J. C. (2017) · The International Review of Research in Open and Distributed Learning · doi:10.19173/irrodl.v18i2.2663 · Cited by 47
Interaction and effectiveness of corporate e-learning programmes
Rodríguez, B. C. P., & Armellini, A. (2013) · Human Resource Development International · doi:10.1080/13678868.2013.803753 · Cited by 45
MOOC Quality: the need for new measures
Hood, N., & Littlejohn, A. H. (2016) · Journal of Learning for Development · doi:10.56059/jl4d.v3i3.165 · Cited by 45
Kember, D. R., & Ginns, P. (2012) · Cited by 40
Sutherland, D., Warwick, P., Anderson, J. R., & Learmonth, M. (2018) · Organizational Behavior Teaching Review · doi:10.1177/1052562918787849 · Cited by 38
Owusu‐Agyeman, Y., & Larbi-Siaw, O. (2018) · Cogent Education · doi:10.1080/2331186x.2018.1456780 · Cited by 37
Liu, S., Peng, X., Cheng, H. N. H., Liu, Z., Sun, J., & Yang, C. (2018) · Journal of Educational Computing Research · doi:10.1177/0735633118757181 · Cited by 35
Zhang, Q., Bonafini, F. C., Lockee, B. B., Jablokow, K., & Hu, X. (2019) · The International Review of Research in Open and Distributed Learning · doi:10.19173/irrodl.v20i2.3730 · Cited by 33
Self-citationEngaged Learning in MOOCs: A Study Using the UK Engagement Survey.
Wintrup, J., Wakefield, K., & Davis, H. (2015) · ePrints Soton (University of Southampton) · Cited by 23
Learning Technology for Education in Cloud. MOOC and Big Data
Uden, L., Sinclair, J. E., Tao, Y., & Liberona, D. (2014) · Communications in computer and information science · doi:10.1007/978-3-319-10671-7 · Cited by 22
Espinoza, Ó., & McGinn, N. (2018) · International Journal of Educational Research · doi:10.1016/j.ijer.2018.05.009 · Cited by 21
Jaesamchung, & 이지현 (2012) · Cited by 6
Cited by
Ouyang, F., Zheng, L., & Jiao, P. (2022) · Education and Information Technologies · doi:10.1007/s10639-022-10925-9 · Cited by 831
Bond, M., Khosravi, H., De Laat, M., Bergdahl, N., Negrea, V., Oxley, E., Pham, P., Chong, S. W., & Siemens, G. (2024) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-023-00436-z · Cited by 852
“Ouyang et al. (2020) highlighted a paper in their review (Hew et al., 2020), which analysed the course features of 249 randomly sampled MOOCs, and 6,393 students’ perceptions were examined to understand what factors predicted student satisfaction.”
BackgroundArtificial intelligence innovation in education: A twenty-year data-driven historical analysis
Guan, C., Mou, J., & Jiang, Z. (2020) · International Journal of Innovation Studies · doi:10.1016/j.ijis.2020.09.001 · Cited by 665
Sentiment Analysis of Students’ Feedback with NLP and Deep Learning: A Systematic Mapping Study
Kastrati, Z., Dalipi, F., Imran, A. S., Pireva Nuci, K., & Wani, M. A. (2021) · Applied Sciences · doi:10.3390/app11093986 · Cited by 256
She, L., Ma, L., Jan, A., Sharif Nia, H., & Rahmatpour, P. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.743936 · Cited by 170
Metadata
- Authors
- Hew, K. F., Hu, X., Qiao, C., & Tang, Y.
- Venue
- Computers & Education
- Publisher
- Elsevier BV
- Published
- 2020-02
- Volume
- 145
- Pages
- 103724
- ISSN
- 0360-1315 (ISSN-L)
- OpenAlex
- W2977285514
- Cited by
- 366
- References
- 105
- FWCI
- 44.11
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.