Learning analytics on structured and unstructured heterogeneous data sources: Perspectives from procrastination, help-seeking, and machine-learning defined cognitive engagement
The DOI matches its Crossref record; 22 of the 22 references it cites were found in a registry or an index.
Abstract
Abstract Statistics is one of the most challenging courses for university students. The personal learning environment (PLE) pedagogical design was introduced to assist students’ Statistics learning. With the PLE pedagogy, this study examined learners’ demographic backgrounds, motivational measures (i.e., help-seeking and academic procrastination due to the use of Information and Communication Technologies, ICT), and ill-structured data (i.e., Facebook posts and comments) to understand what student demographic information, how they feel, and what they do can impact their statistics learning performance. Seventy-eight people joined Facebook groups to form statistics learning communities. Using weakly supervised machine learning (ML), we categorized students’ Facebook messages into statistics-relevant and statistics-irrelevant. Results of the learning analytics on multimodal sources of student data showed that help-seeking positively predicted statistics achievement. In contrast, academic procrastination with ICT negatively predicted statistics achievement, controlling for students' demographics information, including age, gender, prior knowledge, and Internet/social media use. Moreover, the ensemble ML classified messages constructed by taking the sum of relevance coding (0 or 1) across three selected ML algorithms was highly aligned with the human coded message in terms of the degree of relevance to statistics. The ensemble ML classified messages were conceptualized as students’ cognitive engagement in statistics learning due to their high consistency with the human-labeled relevance coding and were positively associated with statistics achievement with a large effect size. The study contributed to developing an integrated learning analytics framework with the PLE pedagogical design encompassing learner backgrounds and unstructured learner artifacts.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Intraclass correlations: Uses in assessing rater reliability.
Shrout, P. E., & Fleiss, J. L. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.2.420 · Cited by 24,532
Becoming a Self-Regulated Learner: An Overview
Zimmerman, B. J. (2002) · Theory Into Practice · doi:10.1207/s15430421tip4102_2 · Cited by 10,197
Nine Ways to Reduce Cognitive Load in Multimedia Learning
Mayer, R. E., & Moreno, R. (2003) · Educational Psychologist · doi:10.1207/s15326985ep3801_6 · Cited by 4,207
Reliability and Predictive Validity of the Motivated Strategies for Learning Questionnaire (Mslq)
Pintrich, P. R., Smith, D. A. F., Garcia, T., & Mckeachie, W. J. (1993) · Educational and Psychological Measurement · doi:10.1177/0013164493053003024 · Cited by 3,646
A brief introduction to weakly supervised learning
Zhou, Z.-H. (2018) · National Science Review · doi:10.1093/nsr/nwx106 · Cited by 1,899
Comparing online and blended learner's self-regulated learning strategies and academic performance
Broadbent, J. (2017) · The Internet and Higher Education · doi:10.1016/j.iheduc.2017.01.004 · Cited by 679
Challenges and Future Directions of Big Data and Artificial Intelligence in Education
Luan, H., Geczy, P., Lai, H., Gobert, J., Yang, S. J. H., Ogata, H., Baltes, J., Guerra, R., Li, P., & Tsai, C.-C. (2020) · Frontiers in Psychology · doi:10.3389/fpsyg.2020.580820 · Cited by 667
Help Seeking and Help Design in Interactive Learning Environments
Aleven, V., Stahl, E., Schworm, S., Fischer, F., & Wallace, R. (2003) · Review of Educational Research · doi:10.3102/00346543073003277 · Cited by 609
Academic procrastination and statistics anxiety
Onwuegbuzie, A. J. (2004) · Assessment & Evaluation in Higher Education · doi:10.1080/0260293042000160384 · Cited by 538
Affective, cognitive, and behavioral differences between high and low procrastinators.
Rothblum, E. D., Solomon, L. J., & Murakami, J. (1986) · Journal of Counseling Psychology · doi:10.1037/0022-0167.33.4.387 · Cited by 521
Worked-out examples: Instructional explanations support learning by self-explanations
Renkl, A. (2002) · Learning and Instruction · doi:10.1016/s0959-4752(01)00030-5 · Cited by 452
Gender differences in mediated communication: women connect more than do men
Kimbrough, A. M., Guadagno, R. E., Muscanell, N. L., & Dill, J. (2013) · Computers in Human Behavior · doi:10.1016/j.chb.2012.12.005 · Cited by 367
Gender divide and acceptance of collaborative Web 2.0 applications for learning in higher education
Huang, W.-H. D., Hood, D. W., & Yoo, S. J. (2013) · The Internet and Higher Education · doi:10.1016/j.iheduc.2012.02.001 · Cited by 287
Meier, A., Reinecke, L., & Meltzer, C. E. (2016) · Computers in Human Behavior · doi:10.1016/j.chb.2016.06.011 · Cited by 277
Applying the SOBC paradigm to explain how social media overload affects academic performance
Whelan, E., Islam, A. K. M. N., & Brooks, S. (2020) · Computers & Education · doi:10.1016/j.compedu.2019.103692 · Cited by 236
Engagement as an inherent aspect of the learning process
Boekaerts, M. (2016) · Learning and Instruction · doi:10.1016/j.learninstruc.2016.02.001 · Cited by 217
Rozgonjuk, D., Kattago, M., & Täht, K. (2018) · Computers in Human Behavior · doi:10.1016/j.chb.2018.08.003 · Cited by 176
Understanding and facilitating self‐regulated help seeking
Karabenick, S. A., & Dembo, M. H. (2011) · New Directions for Teaching and Learning · doi:10.1002/tl.442 · Cited by 154
Wu, J.-Y., & Kwok, O.-M. (2012) · Structural Equation Modeling: A Multidisciplinary Journal · doi:10.1080/10705511.2012.634703 · Cited by 148
Self-citationSnorkel DryBell: A Case Study in Deploying Weak Supervision at Industrial Scale
Bach, S. H., Rodriguez, D., Liu, Y., Luo, C., Shao, H., Xia, C., Sen, S., Ratner, A., Hancock, B., Alborzi, H., et al. (2019) · Proceedings of the 2019 International Conference on Management of Data · doi:10.1145/3299869.3314036 · Cited by 118
Wu, J.-Y. (2017) · Computers & Education · doi:10.1016/j.compedu.2016.10.010 · Cited by 104
Self-citationUniversity students' Motivated Attention and use of regulation strategies on social media
Wu, J.-Y. (2015) · Computers & Education · doi:10.1016/j.compedu.2015.08.016 · Cited by 76
Self-citation
Research works printed but not linked: 66
- Almatrafi, 2018, Needle in a haystack: Identifying learner posts that require urgent response in MOOC discussion forums, Computers & Education, 118, 1, 10.1016/j.compedu.2017.11.002 · DOI
- Bradley, 1997, The use of the area under the ROC curve in the evaluation of machine learning algorithms, Pattern Recognition, 30, 1145, 10.1016/S0031-3203(96)00142-2 · DOI · Paper page
- Cheng, 2013, The role of Internet-specific epistemic beliefs and self-regulation in high school students’ online academic help seeking: A strual equation modeling analysis, Journal of Educational Computing Research, 48, 469, 10.2190/EC.48.4.d · DOI
- Chew, 2014, Statistics anxiety update: Refining the construct and recommendations for a new research agenda, Perspectives on Psychological Science, 9, 196, 10.1177/1745691613518077 · DOI · Paper page
- Chew, 2018, An examination of the internal consistency and structure of the Statistical Anxiety Rating Scale (STARS), PloS One, 13, 10.1371/journal.pone.0194195 · DOI
- Chiesi, 2010, Cognitive and non-cognitive factors related to students' statistics achievement, Statistics Education Research Journal, 9, 10.52041/serj.v9i1.385 · DOI · Paper page
- Chinchor, 1992, MUC-4 evaluation metrics, Proceedings of the 4th Conference on Message Understanding, 22
- Cruise, 1985, Development and validation of an instrument to measure statistical anxiety, American Statistical Association Proceedings of the Section on Statistical Education, 4, 92
- Dabbagh, 2012, Personal learning environments, social media, and self-regulated learning: A natural formula for connecting formal and informal learning, The Internet and Higher Education, 15, 3, 10.1016/j.iheduc.2011.06.002 · DOI · Paper page
- De Boom, 2016, Representation learning for very short texts using weighted word embedding aggregation, Pattern Recognition Letters, 80, 150, 10.1016/j.patrec.2016.06.012 · DOI · Paper page
- Dietterich, 2000, Ensemble methods in machine learning, Multiple Classifier Systems, 1–15, 10.1007/3-540-45014-9_1 · DOI · Paper page
- Ezen-Can, 2015, Unsupervised modeling for understanding mooc discussion forums: A learning analytics approach, Proceedings of the Fifth International Conference on Learning Analytics and Knowledge, 146, 10.1145/2723576.2723589 · DOI
- Feng, 2019, The internet and Facebook usage on academic distraction of college students, Computers & Education, 134, 41, 10.1016/j.compedu.2019.02.005 · DOI · Paper page
- Gao, 2013, Designing asynchronous online discussion environments: Recent progress and possible future directions, British Journal of Educational Technology, 44, 469, 10.1111/j.1467-8535.2012.01330.x · DOI · Paper page
- Guyon, 2003, An introduction to variable and feature selection, Journal of Machine Learning Research, 3, 1157
- Hingmire, 2014, Topic labeled text classification: A weakly supervised approach, Proceedings of the 37th International ACM SIGIR Conference on Research & Development in Information Retrieval, 385
- Hofseth, 2018, Getting rigorous with scientific rigor, Carcinogenesis, 39, 21, 10.1093/carcin/bgx085 · DOI · Paper page
- Karabenick, 2011, Classroom and technology-supported help seeking: The need for converging research paradigms, Learning and Instruction, 21, 290, 10.1016/j.learninstruc.2010.07.007 · DOI
- Kohavi, 1995, A study of cross-validation and bootstrap for accuracy estimation and model selection, Vol. 14, 1137
- Kotsiantis, 2007, Supervised machine learning: A review of classification techniques, Informatica, 31, 249
- Kuder, 1937, The theory of the estimation of test reliability, Psychometrika, 2, 151, 10.1007/BF02288391 · DOI · Paper page
- Lalayants, 2012, Overcoming graduate students' negative perceptions of statistics, Journal of Teaching in Social Work, 32, 356, 10.1080/08841233.2012.705259 · DOI
- Lee, 2015, Facilitating critical thinking using the C-QRAC collaboration script: Enhancing science reading literacy in a computer-supported collaborative learning environment, Computers & Education, 88, 182, 10.1016/j.compedu.2015.05.004 · DOI
- Lee, 2018, Scripting to enhance university students' critical thinking in flipped learning: Implications of the delayed effect on science reading literacy, Interactive Learning Environments, 26, 569, 10.1080/10494820.2017.1372483 · DOI
- Lee, 2014, A latent profile analysis of self-control and self-esteem and the grouping effect on adolescent quality of life across two consecutive years, Social Indicators Research, 117, 523, 10.1007/s11205-013-0360-5 · DOI
- Lin, 2009, Discovering genres of online discussion threads via text mining, Computers & Education, 52, 481, 10.1016/j.compedu.2008.10.005 · DOI
- Lin, 2016, Reading subtitles and taking Enotes while learning scientific materials in a multimedia environment: Cognitive load perspectives on EFL students, Educational Technology & Society, 19, 47
- Lu, 2018, Applying learning analytics for the early prediction of students' academic performance in blended learning, Journal of Educational Technology & Society, 21, 220
- Macher, 2012, Statistics anxiety, trait anxiety, learning behavior, and academic performance, European Journal of Psychology of Education, 27, 483, 10.1007/s10212-011-0090-5 · DOI · Paper page
- Mangaroska, 2018, Learning analytics for learning design: A systematic literature review of analytics-driven design to enhance learning, IEEE Transactions on Learning Technologies, 12, 516, 10.1109/TLT.2018.2868673 · DOI · Paper page
- McGrath, 2015, Reducing anxiety and increasing self-efficacy within an advanced graduate psychology statistics course, Canadian Journal for the Scholarship of Teaching and Learning, 6, 5, 10.5206/cjsotl-rcacea.2015.1.5 · DOI
- Moghavvemi, 2018, Incremental impact of time on students' use of E-learning via Facebook, British Journal of Educational Technology, 49, 560, 10.1111/bjet.12545 · DOI
- Newman, 2008, The motivational role of adaptive help seeking in self-regulated learning, Motivation and Self-Regulated Learning: Theory, Research, and Applications, 315
- Ogan-Bekiroglu, 2012, Examination of the relationship between engagement in scientific argumentation and conceptual knowledge, International Journal of Science and Mathematics Education, 10, 1415, 10.1007/s10763-012-9346-z · DOI
- Onwuegbuzie, 2003, Modeling statistics achievement among graduate students, Educational and Psychological Measurement, 63, 1020, 10.1177/0013164402250989 · DOI
- Onwuegbuzie, 1997, The components of statistics anxiety: A phenomenological study, Focus on Learning Problems in Mathematics, 19, 11
- Onwuegbuzie, 2003, Statistics Anxiety: Nature, etiology, antecedents, effects, and treatments–a comprehensive review of the literature, Teaching in Higher Education, 8, 195, 10.1080/1356251032000052447 · DOI · Paper page
- Pan, 2010, A survey on transfer learning, IEEE Transactions on Knowledge and Data Engineering, 22, 1345, 10.1109/TKDE.2009.191 · DOI · Paper page
- Raftery, 1995, Bayesian model selection in social research, Sociological Methodology, 25, 111, 10.2307/271063 · DOI · Paper page
- Ratner, 2020, Snorkel: Rapid training data creation with weak supervision, The VLDB Journal, 29, 709, 10.1007/s00778-019-00552-1 · DOI · Paper page
- Reeves, 2015, A comparison of technologically mediated and face-to-face help-seeking sources, British Journal of Educational Psychology, 85, 570, 10.1111/bjep.12088 · DOI
- Romero, 2013, Predicting students' final performance from participation in on-line discussion forums, Computers & Education, 68, 458, 10.1016/j.compedu.2013.06.009 · DOI · Paper page
- Romero, 2020, Educational data mining and learning analytics: An updated survey, WIREs Data Mining and Knowledge Discovery, 10, e1355, 10.1002/widm.1355 · DOI · Paper page
- Rupley, 2010, Building prior knowledge and vocabulary in science in the intermediate grades: Creating hooks for learning, Literacy Research and Instruction, 49, 99, 10.1080/19388070902780472 · DOI
- Salmerón, 2017, Scanning and deep processing of information in hypertext: An eye tracking and cued retrospective think-aloud study, Journal of Computer Assisted Learning, 33, 222, 10.1111/jcal.12152 · DOI · Paper page
- Schworm, 2012, e-learning in universities: Supporting help-seeking processes by instructional prompts, British Journal of Educational Technology, 43, 272, 10.1111/j.1467-8535.2011.01176.x · DOI · Paper page
- Sebastiani, 2002, Machine learning in automated text categorization, ACM Computing Surveys, 34, 1, 10.1145/505282.505283 · DOI · Paper page
- Shannon, 1948, A mathematical theory of communication, Bell System Technical Journal, 27, 379, 10.1002/j.1538-7305.1948.tb01338.x · DOI · Paper page
- Slootmaeckers, 2014, Too afraid to learn: Attitudes towards statistics as a barrier to learning statistics and to acquiring quantitative skills, Politics, 34, 191, 10.1111/1467-9256.12042 · doi:10.1111/1467-9256.12042
- Smith, 2016, “A real double-edged sword:” Undergraduate perceptions of social media in their learning, Computers & Education, 103, 44, 10.1016/j.compedu.2016.09.009 · DOI
- Sokolova, 2018, Big Text advantages and challenges: Classification perspective, International Journal of Data Science and Analytics, 5, 1, 10.1007/s41060-017-0087-5 · DOI
- Sokolova, 2006, Beyond accuracy, F-score and roc: A family of discriminant measures for performance evaluation, Australasian Joint Conference on Artificial Intelligence, 1015
- Song, 2016, Interplay of prior knowledge, self-regulation and motivation in complex multimedia learning environments, Journal of Computer Assisted Learning, 32, 31, 10.1111/jcal.12117 · DOI
- Sproesser, 2016, Fostering self-concept and interest for statistics through specific learning environments, Statistics Education Research Journal, 15, 10.52041/serj.v15i1.256 · DOI
- Veletsianos, 2016, Scholars in an increasingly open and digital world: How do education professors and students use Twitter?, The Internet and Higher Education, 30, 1, 10.1016/j.iheduc.2016.02.002 · DOI · Paper page
- Vieira, 2018, Visual learning analytics of educational data: A systematic literature review and research agenda, Computers & Education, 122, 119, 10.1016/j.compedu.2018.03.018 · DOI · Paper page
- Williams, 2010, Statistics anxiety and instructor immediacy, Journal of Statistics Education, 18, 10.1080/10691898.2010.11889495 · DOI
- Wood, 1999, Help seeking, learning and contingent tutoring, Computers & Education, 33, 153, 10.1016/S0360-1315(99)00030-5 · DOI · Paper page
- Wu, 2020, The predictive validities of individual working-memory capacity profiles and note-taking strategies on online search performance, Journal of Computer Assisted Learning, 36, 876, 10.1111/jcal.12441 · DOI
- Wu, 2019, Who is better adapted in learning online within the personal learning environment? Relating gender differences in cognitive attention networks to digital distraction, Computers & Education, 128, 312, 10.1016/j.compedu.2018.08.016 · DOI
- Wu, 2014, Using design-based latent growth curve modeling with cluster-level predictor to address dependency, The Journal of Experimental Education, 82, 431, 10.1080/00220973.2013.876226 · DOI
- Wu, 2017, A solution to modeling multilevel confirmatory factor analysis with data obtained from complex survey sampling to avoid conflated parameter estimates, Frontiers in Psychology, 8, 10.3389/fpsyg.2017.01464 · DOI
- Wu, 2021, Learning Analytics 2.0 for Precision Education: An integrative theoretical framework of the human and machine symbiotic learning, Educational Technology & Society, 24
- Wu, 2020, Using supervised machine learning on large-scale online forums to classify course-related Facebook messages in predicting learning achievement within the personal learning environment, Interactive Learning Environments, 28, 65, 10.1080/10494820.2018.1515085 · DOI · Paper page
- Wu, 2018, Using time pressure and note-taking to prevent digital distraction behavior and enhance online search performance: Perspectives from the load theory of attention and cognitive control, Computers in Human Behavior, 88, 244, 10.1016/j.chb.2018.07.008 · DOI
- Zeidner, 1991, Statistics and mathematics anxiety in social science students: Some interesting parallels, British Journal of Educational Psychology, 61, 319, 10.1111/j.2044-8279.1991.tb00989.x · DOI · Paper page
Cited by
Liao, C.-H., & Wu, J.-Y. (2022) · Computers & Education · doi:10.1016/j.compedu.2022.104599 · Cited by 89
Self-citationYang, T.-C., Hsu, Y.-C., & Wu, J.-Y. (2025) · Interactive Learning Environments · doi:10.1080/10494820.2025.2450659 · Cited by 80
“Analyzing these digital footprints or traces and their psychological responses within a multi-modal or heterogeneous Learning Analytics framework (Liao & Wu, 2023; Rahman et al., 2021; Wu, 2021) may reveal ChatGPT ’ s role in programming learning.”
Self-citationThe Influence of Active and Passive Procrastination on Academic Performance: A Meta-Analysis
Kooren, N. S., Van Nooijen, C., & Paas, F. (2024) · Education Sciences · doi:10.3390/educsci14030323 · Cited by 33
Metadata
- Authors
- Wu, J.-Y.
- Venue
- Computers & Education
- Publisher
- Elsevier BV
- Published
- 2021-04
- Volume
- 163
- Pages
- 104066
- ISSN
- 0360-1315 (ISSN-L)
- OpenAlex
- W3097838782
- Cited by
- 51
- References
- 22
- FWCI
- 6.60
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.elsevier.com/tdm/userlicense/1.0/ (tdm)
- www.elsevier.com/legal/tdmrep-license (tdm)
- Funders
- Ministry of Science and Technology, Taiwan
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.