CoChat

Learning analytics on structured and unstructured heterogeneous data sources: Perspectives from procrastination, help-seeking, and machine-learning defined cognitive engagement

Wu, Jiun-Yu · 2021 · Computers & Education
Cited by 51

The DOI matches its Crossref record; 22 of the 22 references it cites were found in a registry or an index.

Abstract

Abstract Statistics is one of the most challenging courses for university students. The personal learning environment (PLE) pedagogical design was introduced to assist students’ Statistics learning. With the PLE pedagogy, this study examined learners’ demographic backgrounds, motivational measures (i.e., help-seeking and academic procrastination due to the use of Information and Communication Technologies, ICT), and ill-structured data (i.e., Facebook posts and comments) to understand what student demographic information, how they feel, and what they do can impact their statistics learning performance. Seventy-eight people joined Facebook groups to form statistics learning communities. Using weakly supervised machine learning (ML), we categorized students’ Facebook messages into statistics-relevant and statistics-irrelevant. Results of the learning analytics on multimodal sources of student data showed that help-seeking positively predicted statistics achievement. In contrast, academic procrastination with ICT negatively predicted statistics achievement, controlling for students' demographics information, including age, gender, prior knowledge, and Internet/social media use. Moreover, the ensemble ML classified messages constructed by taking the sum of relevance coding (0 or 1) across three selected ML algorithms was highly aligned with the human coded message in terms of the degree of relevance to statistics. The ensemble ML classified messages were conceptualized as students’ cognitive engagement in statistics learning due to their high consistency with the human-labeled relevance coding and were positively associated with statistics achievement with a large effect size. The study contributed to developing an integrated learning analytics framework with the PLE pedagogical design encompassing learner backgrounds and unstructured learner artifacts.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1016/j.compedu.2020.104066

Cites

22 of 22 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Research works printed but not linked: 66

  1. Almatrafi, 2018, Needle in a haystack: Identifying learner posts that require urgent response in MOOC discussion forums, Computers & Education, 118, 1, 10.1016/j.compedu.2017.11.002 · DOI
  2. Bradley, 1997, The use of the area under the ROC curve in the evaluation of machine learning algorithms, Pattern Recognition, 30, 1145, 10.1016/S0031-3203(96)00142-2 · DOI · Paper page
  3. Cheng, 2013, The role of Internet-specific epistemic beliefs and self-regulation in high school students’ online academic help seeking: A strual equation modeling analysis, Journal of Educational Computing Research, 48, 469, 10.2190/EC.48.4.d · DOI
  4. Chew, 2014, Statistics anxiety update: Refining the construct and recommendations for a new research agenda, Perspectives on Psychological Science, 9, 196, 10.1177/1745691613518077 · DOI · Paper page
  5. Chew, 2018, An examination of the internal consistency and structure of the Statistical Anxiety Rating Scale (STARS), PloS One, 13, 10.1371/journal.pone.0194195 · DOI
  6. Chiesi, 2010, Cognitive and non-cognitive factors related to students' statistics achievement, Statistics Education Research Journal, 9, 10.52041/serj.v9i1.385 · DOI · Paper page
  7. Chinchor, 1992, MUC-4 evaluation metrics, Proceedings of the 4th Conference on Message Understanding, 22
  8. Cruise, 1985, Development and validation of an instrument to measure statistical anxiety, American Statistical Association Proceedings of the Section on Statistical Education, 4, 92
  9. Dabbagh, 2012, Personal learning environments, social media, and self-regulated learning: A natural formula for connecting formal and informal learning, The Internet and Higher Education, 15, 3, 10.1016/j.iheduc.2011.06.002 · DOI · Paper page
  10. De Boom, 2016, Representation learning for very short texts using weighted word embedding aggregation, Pattern Recognition Letters, 80, 150, 10.1016/j.patrec.2016.06.012 · DOI · Paper page
  11. Dietterich, 2000, Ensemble methods in machine learning, Multiple Classifier Systems, 1–15, 10.1007/3-540-45014-9_1 · DOI · Paper page
  12. Ezen-Can, 2015, Unsupervised modeling for understanding mooc discussion forums: A learning analytics approach, Proceedings of the Fifth International Conference on Learning Analytics and Knowledge, 146, 10.1145/2723576.2723589 · DOI
  13. Feng, 2019, The internet and Facebook usage on academic distraction of college students, Computers & Education, 134, 41, 10.1016/j.compedu.2019.02.005 · DOI · Paper page
  14. Gao, 2013, Designing asynchronous online discussion environments: Recent progress and possible future directions, British Journal of Educational Technology, 44, 469, 10.1111/j.1467-8535.2012.01330.x · DOI · Paper page
  15. Guyon, 2003, An introduction to variable and feature selection, Journal of Machine Learning Research, 3, 1157
  16. Hingmire, 2014, Topic labeled text classification: A weakly supervised approach, Proceedings of the 37th International ACM SIGIR Conference on Research & Development in Information Retrieval, 385
  17. Hofseth, 2018, Getting rigorous with scientific rigor, Carcinogenesis, 39, 21, 10.1093/carcin/bgx085 · DOI · Paper page
  18. Karabenick, 2011, Classroom and technology-supported help seeking: The need for converging research paradigms, Learning and Instruction, 21, 290, 10.1016/j.learninstruc.2010.07.007 · DOI
  19. Kohavi, 1995, A study of cross-validation and bootstrap for accuracy estimation and model selection, Vol. 14, 1137
  20. Kotsiantis, 2007, Supervised machine learning: A review of classification techniques, Informatica, 31, 249
  21. Kuder, 1937, The theory of the estimation of test reliability, Psychometrika, 2, 151, 10.1007/BF02288391 · DOI · Paper page
  22. Lalayants, 2012, Overcoming graduate students' negative perceptions of statistics, Journal of Teaching in Social Work, 32, 356, 10.1080/08841233.2012.705259 · DOI
  23. Lee, 2015, Facilitating critical thinking using the C-QRAC collaboration script: Enhancing science reading literacy in a computer-supported collaborative learning environment, Computers & Education, 88, 182, 10.1016/j.compedu.2015.05.004 · DOI
  24. Lee, 2018, Scripting to enhance university students' critical thinking in flipped learning: Implications of the delayed effect on science reading literacy, Interactive Learning Environments, 26, 569, 10.1080/10494820.2017.1372483 · DOI
  25. Lee, 2014, A latent profile analysis of self-control and self-esteem and the grouping effect on adolescent quality of life across two consecutive years, Social Indicators Research, 117, 523, 10.1007/s11205-013-0360-5 · DOI
  26. Lin, 2009, Discovering genres of online discussion threads via text mining, Computers & Education, 52, 481, 10.1016/j.compedu.2008.10.005 · DOI
  27. Lin, 2016, Reading subtitles and taking Enotes while learning scientific materials in a multimedia environment: Cognitive load perspectives on EFL students, Educational Technology & Society, 19, 47
  28. Lu, 2018, Applying learning analytics for the early prediction of students' academic performance in blended learning, Journal of Educational Technology & Society, 21, 220
  29. Macher, 2012, Statistics anxiety, trait anxiety, learning behavior, and academic performance, European Journal of Psychology of Education, 27, 483, 10.1007/s10212-011-0090-5 · DOI · Paper page
  30. Mangaroska, 2018, Learning analytics for learning design: A systematic literature review of analytics-driven design to enhance learning, IEEE Transactions on Learning Technologies, 12, 516, 10.1109/TLT.2018.2868673 · DOI · Paper page
  31. McGrath, 2015, Reducing anxiety and increasing self-efficacy within an advanced graduate psychology statistics course, Canadian Journal for the Scholarship of Teaching and Learning, 6, 5, 10.5206/cjsotl-rcacea.2015.1.5 · DOI
  32. Moghavvemi, 2018, Incremental impact of time on students' use of E-learning via Facebook, British Journal of Educational Technology, 49, 560, 10.1111/bjet.12545 · DOI
  33. Newman, 2008, The motivational role of adaptive help seeking in self-regulated learning, Motivation and Self-Regulated Learning: Theory, Research, and Applications, 315
  34. Ogan-Bekiroglu, 2012, Examination of the relationship between engagement in scientific argumentation and conceptual knowledge, International Journal of Science and Mathematics Education, 10, 1415, 10.1007/s10763-012-9346-z · DOI
  35. Onwuegbuzie, 2003, Modeling statistics achievement among graduate students, Educational and Psychological Measurement, 63, 1020, 10.1177/0013164402250989 · DOI
  36. Onwuegbuzie, 1997, The components of statistics anxiety: A phenomenological study, Focus on Learning Problems in Mathematics, 19, 11
  37. Onwuegbuzie, 2003, Statistics Anxiety: Nature, etiology, antecedents, effects, and treatments–a comprehensive review of the literature, Teaching in Higher Education, 8, 195, 10.1080/1356251032000052447 · DOI · Paper page
  38. Pan, 2010, A survey on transfer learning, IEEE Transactions on Knowledge and Data Engineering, 22, 1345, 10.1109/TKDE.2009.191 · DOI · Paper page
  39. Raftery, 1995, Bayesian model selection in social research, Sociological Methodology, 25, 111, 10.2307/271063 · DOI · Paper page
  40. Ratner, 2020, Snorkel: Rapid training data creation with weak supervision, The VLDB Journal, 29, 709, 10.1007/s00778-019-00552-1 · DOI · Paper page
  41. Reeves, 2015, A comparison of technologically mediated and face-to-face help-seeking sources, British Journal of Educational Psychology, 85, 570, 10.1111/bjep.12088 · DOI
  42. Romero, 2013, Predicting students' final performance from participation in on-line discussion forums, Computers & Education, 68, 458, 10.1016/j.compedu.2013.06.009 · DOI · Paper page
  43. Romero, 2020, Educational data mining and learning analytics: An updated survey, WIREs Data Mining and Knowledge Discovery, 10, e1355, 10.1002/widm.1355 · DOI · Paper page
  44. Rupley, 2010, Building prior knowledge and vocabulary in science in the intermediate grades: Creating hooks for learning, Literacy Research and Instruction, 49, 99, 10.1080/19388070902780472 · DOI
  45. Salmerón, 2017, Scanning and deep processing of information in hypertext: An eye tracking and cued retrospective think-aloud study, Journal of Computer Assisted Learning, 33, 222, 10.1111/jcal.12152 · DOI · Paper page
  46. Schworm, 2012, e-learning in universities: Supporting help-seeking processes by instructional prompts, British Journal of Educational Technology, 43, 272, 10.1111/j.1467-8535.2011.01176.x · DOI · Paper page
  47. Sebastiani, 2002, Machine learning in automated text categorization, ACM Computing Surveys, 34, 1, 10.1145/505282.505283 · DOI · Paper page
  48. Shannon, 1948, A mathematical theory of communication, Bell System Technical Journal, 27, 379, 10.1002/j.1538-7305.1948.tb01338.x · DOI · Paper page
  49. Slootmaeckers, 2014, Too afraid to learn: Attitudes towards statistics as a barrier to learning statistics and to acquiring quantitative skills, Politics, 34, 191, 10.1111/1467-9256.12042 · doi:10.1111/1467-9256.12042
  50. Smith, 2016, “A real double-edged sword:” Undergraduate perceptions of social media in their learning, Computers & Education, 103, 44, 10.1016/j.compedu.2016.09.009 · DOI
  51. Sokolova, 2018, Big Text advantages and challenges: Classification perspective, International Journal of Data Science and Analytics, 5, 1, 10.1007/s41060-017-0087-5 · DOI
  52. Sokolova, 2006, Beyond accuracy, F-score and roc: A family of discriminant measures for performance evaluation, Australasian Joint Conference on Artificial Intelligence, 1015
  53. Song, 2016, Interplay of prior knowledge, self-regulation and motivation in complex multimedia learning environments, Journal of Computer Assisted Learning, 32, 31, 10.1111/jcal.12117 · DOI
  54. Sproesser, 2016, Fostering self-concept and interest for statistics through specific learning environments, Statistics Education Research Journal, 15, 10.52041/serj.v15i1.256 · DOI
  55. Veletsianos, 2016, Scholars in an increasingly open and digital world: How do education professors and students use Twitter?, The Internet and Higher Education, 30, 1, 10.1016/j.iheduc.2016.02.002 · DOI · Paper page
  56. Vieira, 2018, Visual learning analytics of educational data: A systematic literature review and research agenda, Computers & Education, 122, 119, 10.1016/j.compedu.2018.03.018 · DOI · Paper page
  57. Williams, 2010, Statistics anxiety and instructor immediacy, Journal of Statistics Education, 18, 10.1080/10691898.2010.11889495 · DOI
  58. Wood, 1999, Help seeking, learning and contingent tutoring, Computers & Education, 33, 153, 10.1016/S0360-1315(99)00030-5 · DOI · Paper page
  59. Wu, 2020, The predictive validities of individual working-memory capacity profiles and note-taking strategies on online search performance, Journal of Computer Assisted Learning, 36, 876, 10.1111/jcal.12441 · DOI
  60. Wu, 2019, Who is better adapted in learning online within the personal learning environment? Relating gender differences in cognitive attention networks to digital distraction, Computers & Education, 128, 312, 10.1016/j.compedu.2018.08.016 · DOI
  61. Wu, 2014, Using design-based latent growth curve modeling with cluster-level predictor to address dependency, The Journal of Experimental Education, 82, 431, 10.1080/00220973.2013.876226 · DOI
  62. Wu, 2017, A solution to modeling multilevel confirmatory factor analysis with data obtained from complex survey sampling to avoid conflated parameter estimates, Frontiers in Psychology, 8, 10.3389/fpsyg.2017.01464 · DOI
  63. Wu, 2021, Learning Analytics 2.0 for Precision Education: An integrative theoretical framework of the human and machine symbiotic learning, Educational Technology & Society, 24
  64. Wu, 2020, Using supervised machine learning on large-scale online forums to classify course-related Facebook messages in predicting learning achievement within the personal learning environment, Interactive Learning Environments, 28, 65, 10.1080/10494820.2018.1515085 · DOI · Paper page
  65. Wu, 2018, Using time pressure and note-taking to prevent digital distraction behavior and enhance online search performance: Perspectives from the load theory of attention and cognitive control, Computers in Human Behavior, 88, 244, 10.1016/j.chb.2018.07.008 · DOI
  66. Zeidner, 1991, Statistics and mathematics anxiety in social science students: Some interesting parallels, British Journal of Educational Psychology, 61, 319, 10.1111/j.2044-8279.1991.tb00989.x · DOI · Paper page

Cited by

See all 51

Metadata

Authors
Wu, J.-Y.
Venue
Computers & Education
Publisher
Elsevier BV
Published
2021-04
Volume
163
Pages
104066
ISSN
0360-1315 (ISSN-L)
OpenAlex
W3097838782
Cited by
51
References
22
FWCI
6.60
Type
journal article
Language
en
Open access
closed
Licence
  • www.elsevier.com/tdm/userlicense/1.0/ (tdm)
  • www.elsevier.com/legal/tdmrep-license (tdm)
Funders
Ministry of Science and Technology, Taiwan

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.