CoChat
Sign Up
Cited by 24Visual and Cognitive Learning ProcessesEvaluation of Teaching PracticesOnline and Blended Learning

The DOI matches its Crossref record; 1 of the 1 references it cites were found in a registry or an index.

Continuous Evaluation of Video Lectures from Real-Time Difficulty Self-Report

Srivastava, Namrata, Velloso, Eduardo, Lodge, Jason M., Erfani, Sarah, Bailey, James·2019·Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems
DOI 10.1145/3290605.3300816
Open this paper in CoChat — free account

Abstract

With the increased reach and impact of video lectures, it is crucial to understand how they are experienced. Whereas previous studies typically present questionnaires at the end of the lecture, they fail to capture students' experience in enough granularity. In this paper we propose recording the lecture difficulty in real-time with a physical slider, enabling continuous and fine-grained analysis of the learning experience. We evaluated our approach in a study with 100 participants viewing two variants of two short lectures. We demonstrate that our approach helps us paint a more complete picture of the learning experience. Our analysis has design implications for instructors, providing them with a method that helps them compare their expectations with students' beliefs about the lectures and to better understand the specific effects of different instructional design decisions.

Overview

Integrity signals

Registry

Crossref · 10.1145/3290605.3300816

Cites

1 of 1 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Pupils dilate with increasing task difficulty across tasks like reading, problem-solving, or visual search activities.

    “Previous research has shown that pupils dilate with increasing task difficulty [47] across tasks like reading, problem-solving, or visual search activities [208], where the cognitive demand can be estimated in real-time using machine learning [131].”

    From A Survey on Measuring Cognitive Workload in Human-Computer Interaction (2023) · cites this paper as [208]

  • The fixation rate of an area of interest describes the number of times an object was looked at, which can be interpreted as a repeated interest in an area and relevant for the current cognitive activity.

    “The fixation rate of an area of interest describes the number of times an object was looked at -this can be interpreted as a repeated interest in an area and relevant for the current cognitive activity [208].”

    From A Survey on Measuring Cognitive Workload in Human-Computer Interaction (2023) · cites this paper as [208]

Citations

Cites

Flagged first, then by citations

Research works printed but not linked: 37

  1. 10.1145/2566486.2568042 · DOI
  2. Amaël Arguel Jason Lodge Mariya Pachman and Paula de Barba. 2016. Confidence drives exploration strategies in interactive simulations. ASCILITE. · DOI
  3. S Adams Becker Michele Cummins Adam Davis Alex Freeman C Glesinger Hall and Vanish Ananthanarayanan. 2017. NMC horizon report: 2017 higher education edition. Technical Report. The New Media Consortium.
  4. 10.1371/journal.pone.0148037 · DOI
  5. Lori Breslow, David E Pritchard, Jennifer DeBoer, Glenda S Stump, Andrew D Ho, and Daniel T Seaton. 2013. Studying learning in the worldwide classroom research into edX's first MOOC. Research & Practice in Assessment 8 (2013), 13--25.
  6. 10.1023/B:TRUC.0000021812.96911.c5 · DOI · Paper page
  7. 10.1145/2858036.2858498 · DOI · Paper page
  8. 10.1207/s15389286ajde1904_3 · DOI
  9. Konstantinos Chorianopoulos. 2013. Collective intelligence within web video. Human-centric Computing and Information Sciences 3, 1 (15 Jun 2013), 10.
  10. 10.1145/2470654.2470766 · DOI
  11. 10.1109/FIE.2014.7044485 · DOI
  12. 10.1109/FIE.2014.7044485 · DOI
  13. Päivi Karppinen. 2005. Meaningful learning with digital and online videos: Theoretical perspectives. AACE Journal 13, 3 (2005), 233--250.
  14. 10.1145/2556325.2566237 · DOI
  15. 10.1037/edu0000013 · DOI
  16. 10.1145/2556288.2557207 · DOI
  17. KH Vincent Lau, Pue Farooque, Gary Leydon, Michael L Schwartz, R Mark Sadler, and Jeremy J Moeller. 2018. Using learning analytics to evaluate a video-based lecture series. Medical teacher 40, 1 (2018), 91--98.
  18. 10.1109/ACII.2009.5349539 · DOI
  19. Nan Li, Lukasz Kidzinski, Patrick Jermann, and Pierre Dillenbourg. 2015. How Do In-video Interactions Reflect Perceived Video Difficulty? Proceedings Papers (2015), 112.
  20. Shih-Yin Lin, John M Aiken, Daniel T Seaton, Scott S Douglas, Edwin F Greco, Brian D Thoms, and Michael F Schatz. 2016. Exploring University Students' Engagement with Online Video Lectures in a Blended Introductory Mechanics Course. arXiv preprint arXiv:1603.03348 (2016). · arXiv
  21. 10.19173/irrodl.v14i3.1455 · DOI
  22. Jason Lodge, Jared Cooney Horvath, Alex Horton, Gregor Kennedy, Sven Venema, and Shane Dawson. 2017. Designing videos for learning: Separating the good from the bad and the ugly. (01 2017).
  23. Jason Lodge and Gregor Kennedy. 2015. Prior knowledge confidence and understanding in interactive tutorials and simulations. ASCILITE. · DOI
  24. 10.3389/feduc.2018.00049 · DOI
  25. Richard E Mayer and Roxana Moreno. 2003. Nine ways to reduce cognitive load in multimedia learning. Educational psychologist 38, 1 (2003), 43--52.
  26. 10.1080/13596748.2010.526802 · DOI
  27. 10.1145/2000119.2000142 · DOI
  28. Fred Paas, Juhani E Tuovinen, Huib Tabbers, and Pascal WM Van Gerven. 2003. Cognitive load measurement as a means to advance cognitive load theory. Educational psychologist 38, 1 (2003), 63--71.
  29. Fred GWC Paas and Jeroen JG Van Merriënboer. 1993. The efficiency of instructional conditions: An approach to combine mental effort and performance measures. Human factors 35, 4 (1993), 737--743.
  30. 10.1109/TLT.2012.15 · DOI
  31. 10.5555/2062780.2062814 · DOI
  32. 10.1145/1101149.1101244 · DOI
  33. 10.1145/3173574.3173893 · DOI
  34. Nicholas C Soderstrom, Carole L Yue, and Elizabeth Ligon Bjork. 2016. 11 Metamemory and Education. The Oxford Handbook of Metamemory (2016), 197.
  35. 10.1016/j.compedu.2010.10.028 · DOI
  36. 10.1007/s10639-013-9266-8 · DOI
  37. 10.1145/2876034.2876051 · DOI

Cited by

Sorted by citations

  • A Survey on Measuring Cognitive Workload in Human-Computer Interaction

    Kosch, T., Karolus, J., Zagermann, J., Reiterer, H., Schmidt, A., & Woźniak, P. W. (2023) · ACM Computing Surveys · doi:10.1145/3582272 · Cited by 304

    “Research showed various links between respiratory activities and an increased cognitive workload: Here, respiration belts or thermistor-based flow sensors underneath the participant’s nose were used to measure the respiration rate, its variance, and depth [106].”

Metadata

Authors
Srivastava, N., Velloso, E., Lodge, J. M., Erfani, S., & Bailey, J.
Affiliations
  • The University of Melbourne (AU)
  • The University of Queensland (AU)
ORCID
Venue
Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems
Publisher
ACM
Published
2019-05-02
Pages
1-12
DOI
10.1145/3290605.3300816
OpenAlex
W2941171376
Cited by
24
References
1
Citations by year
2026: 4 · 2025: 2 · 2024: 3 · 2023: 3 · 2022: 7 · 2021: 2 · 2020: 2 · 2019: 1
FWCI
3.78
Type
conference paper
Language
en
Open access
closed
Licence
www.acm.org/publications/policies/copyright_policy#Background (vor)
Locations
CHI Conference on Human Factors in Computing Systems (publishedVersion)
Funders
Australian Research Council (DE180100315)
Topics
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Visual and Cognitive Learning Processes
  • Social Sciences › Education › Evaluation of Teaching Practices
  • Social Sciences › Education › Online and Blended Learning
Keywords
video lectures, continuous assessment, learning experience, student perceptions, instructional design, instructor feedback

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review