Instruction, Student Engagement, and Learning Outcomes: A Case Study Using Anonymous Social Media in a Face-to-Face Classroom
Open accessThe DOI matches its Crossref record; 69 of the 87 references it cites were found in a registry or an index.
Abstract
With the wide availability of mobile devices and the growing interest in social media, numerous applications have emerged to support student engagement in the classroom. There is conflicting evidence, however, on whether the engagement benefits of such applications outweigh their potential cost as a source of disaffection. To investigate these issues, this article presents a case study on the usage of a social media app (SpeakUp) during a semester-long face-to-face university course, and its relations with the context and the learning outcomes. In this mixed-methods study, we gathered data from multiple sources (video recordings of the lessons, SpeakUp logs and messages, student questionnaires, and course assessments) in order to extract self-reported and observable behavioral and emotional indicators. Our findings reveal that simple measures of behavioral engagement were insufficient to predict academic performance. Nevertheless, our models significantly improved using relatively simple and unobtrusive indicators of both behavioral and emotional engagement and disaffection. This article also indicates that the mere introduction of social media in educational settings does not guarantee a positive impact on learning. To promote an effective use of social media in the classroom (raising engagement while avoiding disaffection), teachers need to design the learning activities aligning the use of social media with the learning goals.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
The Art of Case Study Research
Johnson, K. E., & Stake, R. E. (1996) · The Modern Language Journal · doi:10.2307/329758 · Cited by 15,826
School Engagement: Potential of the Concept, State of the Evidence
Fredricks, J. A., Blumenfeld, P. C., & Paris, A. H. (2004) · Review of Educational Research · doi:10.3102/00346543074001059 · Cited by 12,581 · cited 6×
“6) Engagement and disaffection are multidimensional constructs that require thorough analysis [15], [19], [28], [29]: This study extracts indicators related to behavioral and emotional dimensions of engagement and disaffection that go beyond single behavior variables (see Fig.”
BackgroundResultInfluentialMixed Methods Research: A Research Paradigm Whose Time Has Come
Johnson, R. B., & Onwuegbuzie, A. J. (2004) · Educational Researcher · doi:10.3102/0013189x033007014 · Cited by 11,170 · cited 1×
“A. Informants, Data Gathering, and Data Sources Aligned with the trends in technology-enhanced learning research [61] and the case-study methodology [21], we used mixed methods [62], [63] in order to look at our questions from different perspectives.”
MethodCriteria for assessing the trustworthiness of naturalistic inquiries
Guba, E. G. (1981) · ECTJ · doi:10.1007/bf02766777 · Cited by 5,527 · cited 1×
“Then, the results from these analyses were triangulated [64] to increase the trustworthiness of our findings.”
MethodStudent Involvement: A Developmental Theory for Higher Education
Astin, A. W. (2014) · Routledge eBooks · doi:10.4324/9781315051888-15 · Cited by 5,265
Social media? Get serious! Understanding the functional building blocks of social media
Kietzmann, J. H., Hermkens, K., McCarthy, I. P., & Silvestre, B. S. (2011) · Business Horizons · doi:10.1016/j.bushor.2011.01.005 · Cited by 4,970
Engagement and disaffection in the classroom: Part of a larger motivational dynamic?
Skinner, E., Furrer, C., Marchand, G., & Kindermann, T. (2008) · Journal of Educational Psychology · doi:10.1037/a0012840 · Cited by 2,037
Student engagement literature review
Trowler, V. (2010) · Cited by 1,852
Kreijns, K., Kirschner, P. A., & Jochems, W. (2003) · Computers in Human Behavior · doi:10.1016/s0747-5632(02)00057-2 · Cited by 1,607
Sung, Y.-T., Chang, K.-E., & Liu, T.-C. (2016) · Computers & Education · doi:10.1016/j.compedu.2015.11.008 · Cited by 1,574 · cited 1×
“On the other hand, a recent meta-analysis on the use of mobile devices in the classroom nuances these claims, and shows moderate positive learning effects [39].”
BackgroundWhat Student Affairs Professionals Need to Know About Student Engagement
Kuh, G. D. (2009) · Journal of College Student Development · doi:10.1353/csd.0.0099 · Cited by 1,298
IMPLEMENTING THE SEVEN PRINCIPLES: Technology as Lever
Ehrmann, S. C. (2004) · Cited by 1,024
Measuring student engagement in technology-mediated learning: A review
Henrie, C. R., Halverson, L. R., & Graham, C. R. (2015) · Computers & Education · doi:10.1016/j.compedu.2015.09.005 · Cited by 1,010 · cited 10×
“Tooperationalizethe different elements of the theoretical model proposed by Skinner et al. [19] (namely, context, action, and outcomes), multiple indicators proposed in the technology-mediated learning literature [24] have been used (see Fig.”
BackgroundMethodInfluentialThe role of social media in higher education classes (real and virtual) – A literature review
Tess, P. A. (2013) · Computers in Human Behavior · doi:10.1016/j.chb.2012.12.032 · Cited by 984
Toward an Understanding of Definitions and Measures of School Engagement and Related Terms
Jimerson, S. R., Campos, E., & Greif, J. L. (2003) · The California School Psychologist · doi:10.1007/bf03340893 · Cited by 811 · cited 2×
“As part of our analysis, we explore whether simple indicators of behavioral engagement predict reliably learning outcomes, or rather we need more complex constructs that distinguish behavioral and emotional engagement and disaffection [19], [20].”
BackgroundEngaging online learners: The impact of Web-based learning technology on college student engagement
Chen, P.-S. D., Lambert, A. D., & Guidry, K. R. (2010) · Computers & Education · doi:10.1016/j.compedu.2009.11.008 · Cited by 785 · cited 1×
“While several studies show significant correlations between the use of educational technology and student engagement [26], [42], [43], many of these studies take place in contexts where the use of technology can almost exclusively be on-tasks (e.g., a university’s learning management system or a…”
BackgroundSocial media and education: reconceptualizing the boundaries of formal and informal learning
Greenhow, C., & Lewin, C. (2016) · Learning, Media and Technology · doi:10.1080/17439884.2015.1064954 · Cited by 777
Shum, S. B., & Ferguson, R. (2012) · OPUS - Open Publications of UTS Scholars (University of Technology Sydney) · Cited by 728
New Frontiers: Regulating Learning in CSCL
Järvelä, S., & Hadwin, A. F. (2013) · Educational Psychologist · doi:10.1080/00461520.2012.748006 · Cited by 722 · cited 1×
“9) Look at individual and social forms [58], [59]: As Section V-D shows, the social aspect is considered both as a contextual factor (in relation to the individual and collaborative activities) and as an analysis dimension (looking both at the individual and at the classroom level).”
BackgroundLaptop multitasking hinders classroom learning for both users and nearby peers
Sana, F., Weston, T., & Cepeda, N. J. (2013) · Computers & Education · doi:10.1016/j.compedu.2012.10.003 · Cited by 707 · cited 2×
“Certain studies suggest that laptop multitasking hinders learning for both users and nearby peers [13].”
BackgroundAlternative Perspectives on the Student Experience: Alienation and engagement
Mann, S. J. (2001) · Studies in Higher Education · doi:10.1080/03075070020030689 · Cited by 673
Classroom Response Systems: A Review of the Literature
Fies, C., & Marshall, J. (2006) · Journal of Science Education and Technology · doi:10.1007/s10956-006-0360-1 · Cited by 525
Social learning analytics: five approaches
Ferguson, R., & Buckingham Shum, S. (2012) · Proceedings of the 2nd International Conference on Learning Analytics and Knowledge · doi:10.1145/2330601.2330616 · Cited by 523
A model of online and general campus‐based student engagement
Coates, H. (2007) · Assessment & Evaluation in Higher Education · doi:10.1080/02602930600801878 · Cited by 501
Trees, A. R., & Jackson, M. H. (2007) · Learning, Media and Technology · doi:10.1080/17439880601141179 · Cited by 356
Alternative Approaches to Assessing Student Engagement Rates
Chapman, E. (2020) · UWA Profiles and Research Repository (UWA) · doi:10.7275/3e6e-8353 · Cited by 338
Benefits of Electronic Audience Response Systems on Student Participation, Learning, and Emotion
Stowell, J. R., & Nelson, J. M. (2007) · Teaching of Psychology · doi:10.1080/00986280701700391 · Cited by 336
Mao, J. (2014) · Computers in Human Behavior · doi:10.1016/j.chb.2014.01.002 · Cited by 320
The Merits of Mixing Methods in Evaluation
Greene, J. C., Benjamin, L., & Goodyear, L. (2001) · Evaluation · doi:10.1177/13563890122209504 · Cited by 255
Rambe, P., & Bere, A. (2013) · British Journal of Educational Technology · doi:10.1111/bjet.12057 · Cited by 243
Collaborative group engagement in a computer-supported inquiry learning environment
Sinha, S., Rogat, T. K., Adams-Wiggins, K. R., & Hmelo-Silver, C. E. (2015) · International Journal of Computer-Supported Collaborative Learning · doi:10.1007/s11412-015-9218-y · Cited by 226
Pardo, A., Han, F., & Ellis, R. A. (2017) · IEEE Transactions on Learning Technologies · doi:10.1109/tlt.2016.2639508 · Cited by 216
Kuznekoff, J. H., Munz, S., & Titsworth, S. (2015) · Communication Education · doi:10.1080/03634523.2015.1038727 · Cited by 196
Beer, C., Clark, K., & Jones, D. (2010) · ASCILITE Publications · doi:10.14742/apubs.2010.2044 · Cited by 169
Engagement and Disaffection as Central to Processes of Motivational Resilience and Development
Skinner, E. A. (2016) · doi:10.4324/9781315773384.ch8 · Cited by 145
Dyson, B., Vickers, K., Turtle, J., Cowan, S., & Tassone, A. (2015) · Higher Education · doi:10.1007/s10734-014-9776-3 · Cited by 134
Escuela Superior Politécnica del Litoral, E., & Ochoa, X. (2017) · Handbook of Learning Analytics · doi:10.18608/hla17.011 · Cited by 129
Mobile phones in the classroom: if you can't beat them, join them
Scornavacca, E., Huff, S., & Marshall, S. (2009) · Communications of the ACM · doi:10.1145/1498765.1498803 · Cited by 128
Prieto, L. P., Sharma, K., Kidzinski, Ł., Rodríguez‐Triana, M. J., & Dillenbourg, P. (2018) · Journal of Computer Assisted Learning · doi:10.1111/jcal.12232 · Cited by 119
Self-citationPromoting Interaction in Large Classes with Computer-Mediated Feedback
Anderson, R., Anderson, R., VanDeGrift, T., Wolfman, S., & Yasuhara, K. (2003) · Springer eBooks · doi:10.1007/978-94-017-0195-2_16 · Cited by 114
Engaging Students through Social Media: Evidence-Based Practices for Use in Student Affairs
Reynol Junco, A., Bass, J., & Eaton, P. W. (2014) · Cited by 106
Analytics for Student Engagement
Vytasek, J., Patzak, A., & Winne, P. H. (2019) · Intelligent systems reference library · doi:10.1007/978-3-030-13743-4_3 · Cited by 88
Towards Actionable Learning Analytics Using Dispositions
Tempelaar, D. T., Rienties, B., & Nguyen, Q. (2017) · IEEE Transactions on Learning Technologies · doi:10.1109/tlt.2017.2662679 · Cited by 82
Hollander, E. L., Saltmarsh, J., & Zlotkowski, E. (2002) · Outreach scholarship · doi:10.1007/978-1-4615-0885-4_3 · Cited by 82
Blood, E., & Neel, R. S. (2008) · The Journal of Technology and Teacher Education · Cited by 55
Mittelmeier, J., Rienties, B., Tempelaar, D., Hillaire, G., & Whitelock, D. (2018) · Computers & Education · doi:10.1016/j.compedu.2017.11.003 · Cited by 47
Erickson, J., & Siau, K. L. (2003) · Communications of the ACM · doi:10.1145/903893.903928 · Cited by 44
How we used two social media tools to enhance aspects of active learning during lectures
George, D. R., Dreibelbis, T. D., & Aumiller, B. B. (2013) · Medical Teacher · doi:10.3109/0142159x.2013.818631 · Cited by 42
Instructional Uses of Instant Messaging (IM) During Classroom Lectures
Kinzie, M. B., Whitaker, S. D., & Hofer, M. J. (2005) · Cited by 41
Hotseat: Opening the Backchannel in Large Lectures
(2010) · Cited by 39
Daele, T. V., Frijns, C., & Lievens, J. (2016) · British Journal of Educational Technology · doi:10.1111/bjet.12500 · Cited by 36
Digital Backchannel: Promoting Students' Engagement in EFL Large Class
Harunasari, S. Y., & Halim, N. (2019) · International Journal of Emerging Technologies in Learning (iJET) · doi:10.3991/ijet.v14i07.9128 · Cited by 36
(2008) · doi:10.1145/1358628.1358756 · Cited by 36
Baron, D., Bestbier, A., Case, J., & Collier-Reed, B. I. (2015) · Computers & Education · doi:10.1016/j.compedu.2015.11.007 · Cited by 33
Blending Digital and Face-to-Face Interaction Using a Co-Located Social Media App in Class
Govaerts, S., Holzer, A., Kocher, B., Vozniuk, A., Garbinato, B., & Gillet, D. (2018) · IEEE Transactions on Learning Technologies · doi:10.1109/tlt.2018.2856804 · Cited by 28
Self-citationEncouraging Initiative in the Classroom with Anonymous Feedback
Bergstrom, T., Harris, A., & Karahalios, K. G. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-23774-4_49 · Cited by 28
SpeakUp – A Mobile App Facilitating Audience Interaction
Holzer, A., Govaerts, S., Ondruš, J., Vozniuk, A., Rigaud, D., Garbinato, B., & Gillet, D. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-41175-5_2 · Cited by 22
Self-citationKappers, W., & Cutler, S. L. (2020) · ASEE Annual Conference & Exposition · doi:10.18260/1-2--22921 · Cited by 21
Learning in the Wild: Coding Reddit for Learning and Practice
Kumar, P., Gruzd, A. A., Haythornthwaite, C., Gilbert, S., Valle, M. E. D., & Paulin, D. (2018) · Hawaii International Conference on System Sciences (HICSS) · doi:10.24251/hicss.2018.244 · Cited by 16
Holzer, A., Govaerts, S., Vozniuk, A., Kocher, B., & Gillet, D. (2014) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/2559206.2581211 · Cited by 15
Self-citationExamining the Effects of Social Media in Co-located Classrooms: A Case Study Based on SpeakUp
Rodríguez‐Triana, M. J., Holzer, A., Prieto, L. P., & Gillet, D. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-45153-4_19 · Cited by 14
Self-citationRetelny, D., Birnholtz, J. P., & Hancock, J. T. (2012) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/2141512.2141578 · Cited by 10
(2016) · Cited by 7
Self-citationHandsUp: An In-Class Question Posing Tool
Jackson, H., Nayyar, A., Denny, P. C., Luxton-Reilly, A., & Tempero, E. D. (2018) · International Conference on Learning and Teaching in Computing and Engineering (LaTICE) · doi:10.1109/latice.2018.00-11 · Cited by 3
Research works printed but not linked: 5
- creswell, 2003, Advanced mixed methods research designs, Handbook of Mixed Methods in Social and Behavioral Research, 209
- ha, 2018, Using mobile-based slido for effective management of a university english reading class, Multimedia Assisted Language Learning, 21, 37
- bergstrom, 0, Encouraging initiative in the classroom with anonymous feedback, Proc Int Conf Human Computer Interaction, 6946, 627
- harry, 0, Backchan.nl: Integrating backchannels with physical space, Proc Conf Human Factors Comput Syst, 2751
- yardi, 0, The role of the backchannel in collaborative learning environments, Proc 7th Int Conf Learning Sci, 852
Cited by
Yu, Z., Yu, L., Xu, Q., Xu, W., & Wu, P. (2022) · Technology, Pedagogy and Education · doi:10.1080/1475939x.2022.2045215 · Cited by 164
Al-Rahmi, A. M., Shamsuddin, A., Wahab, E., Al-Rahmi, W. M., Alyoussef, I. Y., & Crawford, J. (2022) · Frontiers in Public Health · doi:10.3389/fpubh.2022.1003007 · Cited by 102
Adeshola, I., & Agoyi, M. (2022) · Interactive Learning Environments · doi:10.1080/10494820.2022.2029493 · Cited by 54
Anzar, S. M., Subheesh, N. P., Panthakkan, A., Malayil, S., & Al-Ahmad, H. (2021) · IEEE Access · doi:10.1109/access.2021.3092260 · Cited by 25
Huang, L.-Z., Wang, K., Li, S., & Guo, J. (2023) · Frontiers in Psychology · doi:10.3389/fpsyg.2022.1098585 · Cited by 27
Metadata
- Authors
- Rodriguez-Triana, M. J., Prieto, L. P., Holzer, A., & Gillet, D.
- Affiliations
- Tallinn University (EE)
- University of Neuchâtel (CH)
- École Polytechnique Fédérale de Lausanne (CH)
- Venue
- IEEE Transactions on Learning Technologies
- Published
- 2020-10-01
- Volume
- 13
- Issue
- 4
- Pages
- 718-733
- ISSN
- 1939-1382 (ISSN-L)
- 2372-0050
- OpenAlex
- W3028326094
- Cited by
- 33
- References
- 87
- Citations by year
- 2026: 1 · 2025: 3 · 2024: 9 · 2023: 11 · 2022: 6 · 2021: 3
- FWCI
- 4.70
- Type
- journal article
- Language
- en
- Open access
- green
- infoscience.epfl.ch
- Licence
- Locations
- IEEE Transactions on Learning Technologies (publishedVersion)
- Infoscience (Ecole Polytechnique Fédérale de Lausanne) (submittedVersion, cc-by-nc-nd)
- Funders
- European Commission (731685, 669074)
- Topics
- Social Sciences › Education › Educational Technology in Learning
- Physical Sciences › Computer Science › Computer Science Applications › Online Learning and Analytics
- Social Sciences › Education › Online and Blended Learning
- Keywords
- social media in education, student engagement, learning outcomes, disaffection, behavioral engagement, emotional engagement, face-to-face classroom, student performance prediction, mobile learning, learning activity design
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.