Using supervised machine learning on large-scale online forums to classify course-related Facebook messages in predicting learning achievement within the personal learning environment
The DOI matches its Crossref record; 10 of the 10 references it cites were found in a registry or an index.
Abstract
This paper demonstrated the use of the supervised Machine Learning (ML) for text classification to predict students’ final course grades in a hybrid Advanced Statistics course and exhibited the potential of using ML classified messages to identify students at risk of course failure. We built three classification models with training data of 76,936 posts from two large online forums and applied the models to classify messages into statistics-related and non-statistics-related posts in a private Facebook group. Three ML algorithms were compared in terms of classification effectiveness and congruency with human coding. Students with more messages endorsed by two or more ML algorithms as statistics-related had higher final course grades. Students who failed the course also had significantly fewer messages endorsed by all three ML algorithms than those who passed. Results suggest that ML can be used for identifying students in need of support within the personal learning environment and for quality control of the large-scale educational data.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538
Intraclass correlations: Uses in assessing rater reliability.
Shrout, P. E., & Fleiss, J. L. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.2.420 · Cited by 24,532
Machine learning: Trends, perspectives, and prospects
Jordan, M. I., & Mitchell, T. M. (2015) · Science · doi:10.1126/science.aaa8415 · Cited by 9,955 · cited 1×
“Nevertheless, recent studies documented negative associations between social media use and academic achievement (see Liu, Kirschner, & Karpinski, 2017 for a review), leading to caveats about using social media for learning (Kirschner, 2015; Lee & Wu, 2012, 2013; Wu, 2014).”
BackgroundDesign-based research: Putting a stake in the ground
Barab, S., & Squire, K. (2004) · Journal of the Learning Sciences · doi:10.1207/s15327809jls1301_1 · Cited by 2,642 · cited 1×
“Results of the ML classi fi ed messages can also be used for redesigning and revising the course to provide just-in-time interventions for students at risk of course failure in the design-based research (Barab & Squire, 2004).”
MethodEducational Data Mining: A review of the state of the art
Romero, C., & Ventura, S. (2010) · IEEE Transactions on Systems Man and Cybernetics Part C (Applications and Reviews) · doi:10.1109/tsmcc.2010.2053532 · Cited by 2,251
Akyol, Z., & Garrison, D. R. (2011) · British Journal of Educational Technology · doi:10.1111/j.1467-8535.2009.01029.x · Cited by 588 · cited 1×
“For the second, using online discussion transcript analysis, Akyol and Garrison (2011) applied a cognitive coding scheme to categorize posts into four di ff erent themes: triggering events, exploration, integration, and resolution.”
MethodLiu, D., Kirschner, P. A., & Karpinski, A. C. (2017) · Computers in Human Behavior · doi:10.1016/j.chb.2017.08.039 · Cited by 139 · cited 1×
“Nevertheless, recent studies documented negative associations between social media use and academic achievement (see Liu, Kirschner, & Karpinski, 2017 for a review), leading to caveats about using social media for learning (Kirschner, 2015; Lee & Wu, 2012, 2013; Wu, 2014).”
BackgroundWu, J.-Y. (2017) · Computers & Education · doi:10.1016/j.compedu.2016.10.010 · Cited by 104
Self-citationLee, Y.-H., & Wu, J.-Y. (2013) · Computers & Education · doi:10.1016/j.compedu.2013.03.001 · Cited by 102
Self-citationUniversity students' Motivated Attention and use of regulation strategies on social media
Wu, J.-Y. (2015) · Computers & Education · doi:10.1016/j.compedu.2015.08.016 · Cited by 76 · cited 1×
“In contrast, though online discussion premised on social media (e.g. Facebook, Wu, 2015, 2017) can accommodate students ’ speci fi c learning needs, it requires professional techniques such as the use of Facebook API and web crawling techniques to obtain structured and unstructured user data.”
BackgroundSelf-citation
Research works printed but not linked: 37
- 10.1016/j.compedu.2017.11.002 · DOI
- Anderson L. W., 2001, A taxonomy for learning, teaching and assessing: A revision of Bloom’s taxonomy of educational objectives
- 10.1080/10494820.2013.788035 · DOI
- 10.1080/1475939X.2017.1365753 · DOI
- Burges C. J., 1998, Data mining and knowledge discovery, 2, 121
- 10.5334/dsj-2015-002 · DOI · Paper page
- 10.1016/j.compedu.2016.02.006 · DOI · Paper page
- 10.1016/j.compedu.2010.07.024 · DOI · Paper page
- 10.1111/j.1467-8535.2005.00542.x · DOI · Paper page
- 10.1002/1097-0142(20010415)91:8+<1615::AID-CNCR1175>3.0.CO;2-L · DOI · Paper page
- 10.1016/j.patrec.2016.06.012 · DOI · Paper page
- 10.1061/(ASCE)0887-3801(2001)15:3(208) · doi:10.1061/(asce)0887-3801(2001)15:3(208
- 10.1162/153244303322753616 · DOI · Paper page
- Hair J. F., 2010, Multivariate data analysis
- 10.1109/101.8118 · DOI · Paper page
- 10.1109/ACCESS.2015.2490723 · DOI
- 10.1007/BFb0026683 · DOI · Paper page
- 10.1016/j.chb.2015.03.011 · DOI
- 10.1016/j.compedu.2015.05.004 · DOI
- 10.1080/10494820.2017.1372483 · DOI
- 10.1016/j.lindif.2012.01.007 · DOI
- 10.1016/j.compedu.2008.10.005 · DOI
- 10.1080/15391523.2006.10782479 · DOI
- 10.1016/j.compedu.2013.06.009 · DOI · Paper page
- 10.1145/505282.505283 · DOI · Paper page
- 10.1002/j.1538-7305.1948.tb01338.x · DOI · Paper page
- 10.1007/s41060-017-0087-5 · DOI
- 10.1108/eb026526 · DOI · Paper page
- 10.1007/978-1-4615-3618-5 · DOI · Paper page
- Tan P.-N., 2018, Introduction to data mining
- 10.1007/s10639-013-9279-3 · DOI
- 10.1145/3170358.3170403 · DOI
- 10.1145/2883851.2883916 · DOI · Paper page
- 10.1111/jcal.2014.30.issue-3 · doi:10.1111/jcal.2014.30.issue-3
- 10.1016/j.compedu.2018.08.016 · DOI
- 10.1016/j.chb.2018.07.008 · DOI
- 10.1016/j.knosys.2008.01.001 · DOI
Cited by
A Review of Artificial Intelligence (AI) in Education from 2010 to 2020
Zhai, X., Chu, X., Chai, C. S., Jong, M. S. Y., Istenic, A., Spector, M., Liu, J.-B., Yuan, J., & Li, Y. (2021) · Complexity · doi:10.1155/2021/8812542 · Cited by 1,364
Artificial intelligence innovation in education: A twenty-year data-driven historical analysis
Guan, C., Mou, J., & Jiang, Z. (2020) · International Journal of Innovation Studies · doi:10.1016/j.ijis.2020.09.001 · Cited by 665
Liao, C.-H., & Wu, J.-Y. (2022) · Computers & Education · doi:10.1016/j.compedu.2022.104599 · Cited by 89
Self-citationAugmenting assessment with AI coding of online student discourse: A question of reliability
Misiejuk, K., Kaliisa, R., & Scianna, J. (2024) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2024.100216 · Cited by 33
Metadata
- Authors
- Wu, J.-Y., Hsiao, Y.-C., & Nian, M.-W.
- Venue
- Interactive Learning Environments
- Publisher
- Informa UK Limited
- Published
- 2018-09-03
- Volume
- 28
- Issue
- 1
- Pages
- 65-80
- ISSN
- 1049-4820 (ISSN-L)
- 1744-5191
- OpenAlex
- W2890637277
- Cited by
- 88
- References
- 10
- FWCI
- 12.06
- Type
- journal article
- Language
- en
- Open access
- closed
- Funders
- Ministry of Science and Technology (104-2628-H-009-001-SS3,105-2511-S-009-009-MY3)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Using supervised machine learning on large-scale online forums to classify course-related Facebook messages in predicting learning achievement within the personal learning environment. https://app.cochat.ai/papers/10.1080/10494820.2018.1515085
- The paper
- Wu, J.-Y., Hsiao, Y.-C., & Nian, M.-W. (2020). Using supervised machine learning on large-scale online forums to classify course-related Facebook messages in predicting learning achievement within the personal learning environment. Interactive Learning Environments. https://doi.org/10.1080/10494820.2018.1515085
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.