MOOC Dropout Prediction: Lessons Learned from Making Pipelines Interpretable
Open accessThe DOI matches its Crossref record; 4 of the 4 references it cites were found in a registry or an index.
Abstract
Dropout prediction in MOOCs is a well-researched problem where we classify which students are likely to persist or drop out of a course. Most research into creating models which can predict outcomes is based on student engagement data. Why these students might be dropping out has only been studied through retroactive exit surveys. This helps identify an important extension area to dropout prediction- how can we interpret dropout predictions at the student and model level? We demonstrate how existing MOOC dropout prediction pipelines can be made interpretable, all while having predictive performance close to existing techniques. We explore each stage of the pipeline as design components in the context of interpretability. Our end result is a layer which longitudinally interprets both predictions and entire classification models of MOOC dropout to provide researchers with in-depth insights of why a student is likely to dropout.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
How video production affects student engagement: an empirical study of MOOC videos
Guo, P. J., Kim, J., & Rubin, R. (2014) · Proceedings of the first ACM conference on Learning @ scale conference · doi:10.1145/2556325.2566239 · Cited by 1,850 · cited 1×
“Student engagement can be notoriously difficult to quantify [10] as a result, various proxy measures are used in literature.”
BackgroundModel-Agnostic Interpretability of Machine Learning
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · arXiv · doi:10.48550/arxiv.1606.05386 · Cited by 699 · cited 1×
“In-terpretable classification techniques identified in [22] include linear models [13], decision trees [26], falling rule lists [30].”
Method“Predicting MOOC dropout over weeks using machine learning methods,”
Kloft, M., Stiehler, F., Zheng, Z., & Pinkwart, N. (2014) · Proceedings of the EMNLP 2014 Workshop on Analysis of Large Scale Social Interaction in MOOCs · doi:10.3115/v1/w14-4111 · Cited by 325
How to Explain Individual Classification Decisions
(2010) · Journal of Machine Learning Research · doi:10.5555/1756006.1859912 · Cited by 54
Research works printed but not linked: 22
- N. V. Chawla, K. W. Bowyer, L. O. Hall, and W. P. Kegelmeyer. Smote: synthetic minority over-sampling technique. Journal of artificial intelligence research, 16:321--357, 2002.
- N. V. Chawla, N. Japkowicz, and A. Kotcz. Editorial: special issue on learning from imbalanced data sets. ACM Sigkdd Explorations Newsletter, 6(1):1--6, 2004. · DOI · Paper page
- M. W. Craven and J. W. Shavlik. Extracting tree-structured representations of trained networks. Advances in neural information processing systems, pages 24--30, 1996.
- J. Dillon, N. Bosch, M. Chetlur, N. Wanigasekara, G. A. Ambrose, B. Sengupta, and S. K. D'Mello. Student emotion, co-occurrence, and dropout in a MOOC context. The 9th International Conference on Educational Data Mining, pages 353--357, 2016.
- M. Fei and D.-Y. Yeung. Temporal models for predicting student dropout in massive open online courses. In 2015 IEEE International Conference on Data Mining Workshop (ICDMW), pages 256--263. IEEE, 2015. · DOI · Paper page
- A. A. Freitas. Comprehensible classification models: a position paper. ACM SIGKDD explorations newsletter, 15(1):1--10, 2014.
- C. Gütl, R. H. Rizzardini, V. Chang, and M. Morales. Attrition in mooc: Lessons learned from drop-out students. In International Workshop on Learning Technology for Education in Cloud, pages 37--48. Springer, 2014. · DOI · Paper page
- J. He, J. Bailey, B. I. Rubinstein, and R. Zhang. Identifying at-risk students in massive open online courses. In AAAI, pages 1749--1755, Austin Texas, USA, 2015. AAAI Press. · DOI · Paper page
- G. Kennedy, C. Coffrin, P. de Barba, and L. Corrin. Predicting success: how learners' prior knowledge, skills and activities predict mooc performance. In Proceedings of the Fifth International Conference on Learning Analytics And Knowledge, pages 136--140, New York, NY, USA, 2015. ACM, ACM New York, NY, USA 2015. · DOI · Paper page
- H. Khalil and M. Ebner. Moocs completion rates and possible methods to improve retention - a literature review. In J. Viteli and M. Leikomaa, editors, Proceedings of EdMedia: World Conference on Educational Media and Technology 2014, pages 1305--1313, Tampere, Finland, June 2014. Association for the Advancement of Computing in Education (AACE).
- H. Lakkaraju, S. H. Bach, and J. Leskovec. Interpretable decision sets: A joint framework for description and prediction. In Proceedings of the 22Nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD '16, pages 1675--1684, New York, NY, USA, 2016. ACM. · DOI · Paper page
- C. Márquez-Vera, A. Cano, C. Romero, A. Y. M. Noaman, H. Mousa Fardoun, and S. Ventura. Early dropout prediction using data mining: a case study with high school students. Expert Systems, 33(1):107--124, 2016. · DOI · Paper page
- D. F. Onah, J. Sinclair, and R. Boyatt. Dropout rates of massive open online courses: behavioural patterns. EDULEARN14 Proceedings, pages 5825--5834, 2014.
- M. T. Ribeiro, S. Singh, and C. Guestrin. "why should i trust you?": Explaining the predictions of any classifier. arXiv preprint arXiv:1602.04938, abs/1602.04938, 2016. · arXiv
- T. Sinha, P. Jermann, N. Li, and P. Dillenbourg. Your click decides your fate: Inferring information processing and attrition behavior from mooc video clickstream interactions. arXiv preprint arXiv:1407.7131, 2014. · arXiv
- R. M. Stein and G. Allione. Mass attrition: An analysis of drop out from a principles of microeconomics mooc. Social Science Research Network, pages 1--19, 2014. · doi:10.2139/ssrn.2505028
- J. K. Tang, H. Xie, and T.-L. Wong. A big data framework for early identification of dropout students in mooc. In International Conference on Technology in Education, pages 127--132. Springer, 2015. · DOI
- C. Taylor, K. Veeramachaneni, and U.-M. O'Reilly. Likely to stop? predicting stopout in massive open online courses. arXiv preprint arXiv:1408.3382, 2014. · arXiv
- J. Whitehill, J. J. Williams, G. Lopez, C. A. Coleman, and J. Reich. Beyond prediction: First steps toward automatic intervention in mooc student stopout. Social Science Research Network, 2015. · DOI · Paper page
- W. Xing, X. Chen, J. Stein, and M. Marcinkowski. Temporal predication of dropouts in moocs: Reaching the low hanging fruit through stacking generalization. Computers in Human Behavior, 58:119--129, 2016. · DOI · Paper page
- K. Xu, J. Ba, R. Kiros, K. Cho, A. Courville, R. Salakhutdinov, R. S. Zemel, and Y. Bengio. Show, attend and tell: Neural image caption generation with visual attention. arXiv preprint arXiv:1502.03044, 2(3):5, 2015. · arXiv
- D. Yang, T. Sinha, D. Adamson, and C. P. Rose. Turn on, Tune in, Drop out: Anticipating student dropouts in Massive Open Online Courses. In Proceedings of the 2013 NIPS Data-driven education workshop, volume 11, page 14, Lake Tahoe, Nevada, USA, 2013. · DOI
Cited by
A Systematic Literature Review of Student’ Performance Prediction Using Machine Learning Techniques
Albreiki, B., Zaki, N., & Alashwal, H. (2021) · Education Sciences · doi:10.3390/educsci11090552 · Cited by 427
Interpretable dropout prediction: towards XAI-based personalized intervention
Nagy, M., & Molontay, R. (2024) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-023-00331-8 · Cited by 179
“Nagrecha et al (2017) predicted MOOC dropout and used LIME to make the individual predictions of black-box machine learning models (Random Forest and Gradient Boosting Trees) interpretable, moreover, they compared it to intrinsically interpretable models (Logistic regression and Decision Tree).”
Coussement, K., Phan, M., De Caigny, A., Benoit, D. F., & Raes, A. (2020) · Decision Support Systems · doi:10.1016/j.dss.2020.113325 · Cited by 125
Springer, A., & Whittaker, S. (2019) · Proceedings of the 24th International Conference on Intelligent User Interfaces · doi:10.1145/3301275.3302322 · Cited by 115
“Many such attempts at transparency are not tested with users or simulate user studies [5,57,60].”
BackgroundProgressive Disclosure: When, Why, and How Do Users Want Algorithmic Transparency Information?
Springer, A., & Whittaker, S. (2020) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3374218 · Cited by 84
Metadata
- Authors
- Nagrecha, S., Dillon, J. Z., & Chawla, N. V.
- Venue
- Proceedings of the 26th International Conference on World Wide Web Companion - WWW '17 Companion
- Publisher
- ACM Press
- Published
- 2017
- Pages
- 351-359
- OpenAlex
- W2612657834
- Cited by
- 87
- References
- 4
- FWCI
- 9.37
- Type
- conference paper
- Language
- en
- Open access
- gold
- Funders
- National Science Foundation (IIS-1447795)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: MOOC Dropout Prediction: Lessons Learned from Making Pipelines Interpretable. https://app.cochat.ai/papers/10.1145/3041021.3054162
- The paper
- Nagrecha, S., Dillon, J. Z., & Chawla, N. V. (2017). MOOC Dropout Prediction: Lessons Learned from Making Pipelines Interpretable. Proceedings of the 26th International Conference on World Wide Web Companion - WWW '17 Companion. https://doi.org/10.1145/3041021.3054162
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.