CoChat
Sign Up
Cited by 306Online Learning and AnalyticsIntelligent Tutoring Systems and Adaptive LearningData Visualization and Analytics

Review of Research on Student-Facing Learning Analytics Dashboards and Educational Recommender Systems

Open access
Bodily, Robert, Verbert, Katrien·2017·IEEE Transactions on Learning Technologies
DOI 10.1109/tlt.2017.2740172
Open this paper in CoChat — free accountOpen PDF

Abstract

This article is a comprehensive literature review of student-facing learning analytics reporting systems that track learning analytics data and report it directly to students. This literature review builds on four previously conducted literature reviews in similar domains. Out of the 945 articles retrieved from databases and journals, 93 articles were included in the analysis. Articles were coded based on the following five categories: functionality, data sources, design analysis, student perceptions, and measured effects. Based on this review, we need research on learning analytics reporting systems that targets the design and development process of reporting systems, not only the final products. This design and development process includes needs analyses, visual design analyses, information selection justifications, and student perception surveys. In addition, experiments to determine the effect of these systems on student behavior, achievement, and skills are needed to add to the small existing body of evidence. Furthermore, experimental studies should include usability tests and methodologies to examine student use of these systems, as these factors may affect experimental findings. Finally, observational study methods, such as propensity score matching, should be used to increase student access to these systems but still rigorously measure experimental effects.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/tlt.2017.2740172

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Reliance on static questionnaires has been increasingly questioned, as self-reported learning styles may not accurately reflect learners' actual study behaviors in digital environments.

    “However, the reliance on static questionnaires has been increasingly questioned, as selfreported learning styles may not accurately reflect learners' actual study behaviors in digital environments.”

    From AI for All: Adaptive, Accessible, and Inclusive Learning Experiences in the Age of Intelligent LMSs (2026) · cites this paper as [11]

    The Introduction (page unavailable) discusses collecting learner interaction data, and §4.4 discusses students’ perceptions of reporting systems. Neither addresses whether static learning-style questionnaires accurately reflect study behavior. Searches for “learning styles,” “static questionnaires,” and self-reports versus observed behavior found no passage making that comparison. (LLM, conf=0.74)

    Experimental — accuracy not yet measured

  • Discrepancies between self-reported and behavior-derived learning styles were explicitly identified, highlighting the fluid and context-dependent nature of learner preferences.

    “Importantly, discrepancies between self-reported and behavior-derived learning styles were explicitly identified, highlighting the fluid and context-dependent nature of learner preferences [8][9][10][11].”

    From AI for All: Adaptive, Accessible, and Inclusive Learning Experiences in the Age of Intelligent LMSs (2026) · cites this paper as [11]

    Section 4.4 discusses students’ perceptions of learning-analytics systems, and Section 6 discusses tracked behavior, but neither identifies discrepancies between self-reported and behavior-derived learning styles. Searches for learning styles, self-reports, and learner preferences found no passage addressing the claimed discrepancy or the fluid, context-dependent nature of preferences. (LLM, conf=0.72)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

The citation lists haven’t been fetched yet

References 54 · Cited by 306

Research works printed but not linked: 45

  1. K.E. Arnold et al., "Course Signals at Purdue: Using Learning Analytics to Increase Student Success," Proc. Second Int'l Conf. Learning Analytics and Knowledge (LAK12), 2012, pp. 2-5.
  2. S.S. Beheshitha et al., "The Role of Achievement Goal Orientations When Studying Effect of Learning Analytics Visualizations," Proc. Sixth Int'l Conf. Learning Analytics and Knowledge (LAK16), April 2016; http://doi.org/10.1145/2883851.2883904. · DOI
  3. R. Burke, "Hybrid recommender systems: Survey and experi- ments," User modeling and user-adapted interaction, vol. 12, no. 4, 2002, pp. 331-370.
  4. J.P. Campbell and D.G. Oblinger, "Academic analytics," Edu- cause Review, vol. 42, no. 4, July/August 2007, pp. 40-57.
  5. G.D. Chen, C.K. Chang, and C.Y. Wang, "Ubiquitous learning website: Scaffold learners by mobile devices with information- aware techniques," Computers and Education, vol. 50 no. 1, 2008; http://doi.org/10.1016/j.compedu.2006.03.004. · DOI
  6. D. Clow, "The learning analytics cycle: Closing the loop effectively." Proc. Second Int'l Conf. Learning Analytics and Knowledge (LAK12), 2012, pp. 134-138.
  7. T. Denley, "How predictive analytics and choice achitecture can improve student success," Research & Practice in Assessment, vol. 9, no. 2, 2014, pp. 61-69.
  8. B. Dodge, J. Whitmer, and J.P. Frazee, "Improving undergraduate student achievement in large blended courses through data-driven interventions," Proc. Fifth Int'l Conf. Learning Analytics and Knowledge (LAK15), 2015; http://doi.org/10.1145/2723576.2723657. · DOI
  9. H. Drachsler et al., "Panorama of Recommender Systems to Support Learning," Recommender Systems Handbook, 2015; http://doi.org/10.1007/978-1-4899-7637-6. · DOI
  10. J. Feild, "Improving Student Performance Using Nudge Analytics," Proc. Eighth Int'l Conf. Educational Data Mining (EDM15), 2015, pp. 464-467.
  11. R. Ferguson, M. Sharkey, and N. Mirriahi, "The NTU student dashboard: Implementing a whole-iInstitution learning analytics platform to improve student engagement," Proc. Sixth Int'l Conf. Learning Analytics And Knowledge (LAK16), 2016, pp. 14-15.
  12. S. Few, Information Dashboard Design: Displaying data for at-a- glance monitoring, Analytics Press, 2013.
  13. J. Grann and D. Bushway, "Competency Map: Visualizing Student Learning to Promote Student Success," Proc. Fourth Int'l Conf. Learning Analytics and Knowledge (LAK14), 2014; http://doi.org/10.1145/2567574.2567622. · DOI
  14. W. Greller and H. Drachsler, "Translating Learning into Num- bers: A Generic Framework for Learning Analytics," Educational Technology & Society, vol. 15, no. 3, 2012, pp. 42-57.
  15. J.R. Hartley and D.H. Sleeman, "Towards more intelligent teach- ing systems," International Journal of Man-Machine Studies, vol. 5, no. 2, 1973, pp. 215-236.
  16. O. Holanda et al., "Educational Resources Recommendation System Based on Agents and Semantic Web for Helping Students in a Virtual Learning Environment," International Journal of Web Based Communities, vol. 8, no. 3, 2012; http://doi.org/10.1207/s15327752jpa8502. · DOI
  17. M.H. Hsu, "A personalized English learning recommender system for ESL students," Expert Systems with Applications, vol. 34, no. 1, 2008; http://doi.org/10.1016/j.eswa.2006.10.004. · DOI
  18. Y.M. Huang et al., "A Markov-based recommendation model for exploring the transfer of learning on the Web," Educational Technology and Society, vol. 12, no. 2, 2009; http://doi.org/10.1207/S15328015TLM1504. · DOI
  19. L. Iandoli et al., "Socially augmented argumentation tools: Rationale, design and evaluation of a debate dashboard," International Journal of Human Computer Studies, vol. 72, no. 3, 2014; http://doi.org/10.1016/j.ijhcs.2013.08.006. · DOI · Paper page
  20. J. Janssen et al., "Self-organising navigational support in lifelong learning: How predecessors can lead the way," Computers and Education, vol. 49, no. 3, 2007; http://doi.org/10.1016/j.compedu.2005.11.022. · DOI
  21. M. Ji et al., "DDART, a dynamic dashboard for collection, analysis and visualization of activity and reporting traces," Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), LNCS 8719, 2014; http://doi.org/10.1007/978-3-319-11200-8_39. · DOI
  22. A. Kerly, R. Ellis, and S. Bull, "CALMsystem: A Conversational Agent for Learner Modelling," Knowledge-Based Systems, vol. 21, no. 3, 2008; http://doi.org/10.1016/j.knosys.2007.11.015. · DOI
  23. J. Kim, I.-H. Jo, and Y. Park, "Effect of learning analytics dashboard: analyzing the relations among dashboard utilization, satisfaction, and learning achievement," Asia Pacific Education Review, vol. 17, no. 1, 2016, pp. 13-24.
  24. K. Kuosa et al., "Interactive Visualization Tools to Improve Learning and Teaching in Online Learning Environments," International Journal of Distance Education Technologies (IJDET), vol. 14, no. 1, 2016; http://doi.org/10.4018/IJDET.2016010101. · DOI
  25. Y.J. Lee, "VisSearch: A collaborative Web searching environment," Computers and Education, vol. 44, no. 4, 2005; http://doi.org/10.1016/j.compedu.2004.04.009. · DOI
  26. K. Muldner et al., "Exploring the impact of a learning dashboard on student affect," Artificial Intelligence in Education Seventeenth Int'l Conf. (AIED15), 2015; http://doi.org/10.1007/978-3-319- 19773-9. · DOI
  27. J. Nakahara et al., "ITree: Does the mobile phone encourage learners to be more involved in collaborative learning?" Proc. Int'l Conf. Computer Supported Collaborative Learning (CSCL2005), 2005, pp. 470-478.
  28. M. Olmos and L. Corrin, "Learning analytics: A case study of the process of design visualizations," Journal of Asynchronous Learning Networks, vol. 16, no. 3, 2012, pp. 39-49.
  29. C. Ott et al., "Illustrating performance indicators and course characteristics to support students' self-regulated learning in CS1," Computer Science Education, vol. 25, no. 2, 2015; http://doi.org/10.1080/08993408.2015.1033129. · DOI
  30. Y. Park and I. Jo, "Development of the Learning Analytics Dashboard to Support Students' Learning Performance," Journal of Universal Computer Science, vol. 21, no. 1, 2015, pp. 110-133.
  31. J.L. Santos et al., "Goal-oriented visualizations of activity tracking: A case study with engineering students," Proc. Second Int'l Conf. Learning Analytics and Knowledge (LAK12), 2012; http://doi.org/10.1145/2330601.2330639. · DOI
  32. J.L. Santos, K. Verbert, and E. Duval, (2012). "Empowering students to reflect on their activity with StepUp!: Two case studies with engineering students," CEUR Workshop Proceedings, vol. 931, 2012, pp. 73-86.
  33. J.L. Santos et al., "Addressing Learner Issues with StepUp!: An Evaluation," Proc. Third Int'l Conf. Learning Analytics and Knowledge (LAK13), 2013; http://doi.org/10.1145/2460296.2460301. · DOI
  34. O.C. Santos, J.G. Boticario, and D. Perez-Marin, "Extending web- based educational systems with personalised support through User Centred Designed recommendations along the e-learning life cycle," Science of Computer Programming, vol. 88, 2014; http://doi.org/10.1016/j.scico.2013.12.004. · DOI
  35. C. Saul and H.D. Wuttke, "Turning learners into effective better learners: The use of the askMe! System for learning analytics," CEUR Workshop Proceedings, vol. 1181, 2014, pp. 57-60.
  36. B. Schwendimann et al., "Perceiving learning at a glance: A systematic literature review of learning dashboard research," IEEE Transactions on Learning Technologies, 2016; doi: 10.1109/TLT.2016.2599522. · DOI
  37. G. Siemens, 1st International Conference on Learning Analytics and Knowledge 2011, July 22, 2010, retrieved March 30, 2016 from https://tekri.athabascau.ca/analytics/.
  38. A. Van Barneveld, K.E. Arnold, and J.P. Campbell, "Analytics in higher education: Establishing a common language," Educause Learning Initiative, vol. 1, 2012, pp. 1-11.
  39. K. Verbert et al., "Learning Analytics Dashboard Applications," American Behavioral Scientist, 2013; http://doi.org/10.1177/0002764213479363. · DOI
  40. K. Verbert et al., "Learning dashboards: An overview and future research opportunities," Personal and Ubiquitous Computing, vol. 18, no. 6, 2014; http://doi.org/10.1007/s00779-013-0751-2. · DOI
  41. B. Vesin et al., "Applying recommender systems and adaptive hypermedia for e-learning personalization," Computing and Informatics, vol. 32, no. 3, 2013, pp. 629-659.
  42. F.-H. Wang, "Content Recommendation Based on Education- Contextualized Browsing Events for Web-based Personalized Learning," Educational Technology & Society, vol. 11, no. 4, 2008, pp. 94-112.
  43. A. Wise, Y. Zhao, and S. Hausknecht, "Learning Analytics for Online Discussions: Embedded and Extracted Approaches," Journal of Learning Analytics, vol. 1, no. 2, 2014, pp. 48-71.
  44. Z. Xu and A. Makos, "Investigating the impact of a notification system on student behaviors in a discourse-intensive hybrid course," Proc Fifth Int'l Conf. Learning Analytics and Knowledge (LAK15), 2015; http://doi.org/10.1145/2723576.2723663. · DOI
  45. Y. Yoo et al., "Educational Dashboards for Smart Learning: Review of Case Studies," Emerging Issues in Smart Learning, 2015; http://doi.org/10.1007/978-3-662-44188-6. · DOI

Authors
Bodily, R., & Verbert, K.
Affiliations
  • Brigham Young University (US)
  • KU Leuven (BE)
ORCID
Venue
IEEE Transactions on Learning Technologies
Publisher
Institute of Electrical and Electronics Engineers (IEEE)
Published
2017-10-01
Volume
10
Issue
4
Pages
405-418
ISSN
1939-1382 (ISSN-L)
DOI
10.1109/tlt.2017.2740172
OpenAlex
W2745945089
Cited by
306
References
54
Citations by year
2026: 17 · 2025: 30 · 2024: 35 · 2023: 50 · 2022: 48 · 2021: 46 · 2020: 37 · 2019: 26 · 2018: 17
FWCI
26.23
Type
journal article
Language
en
Open access
Licence
Locations
  • IEEE Transactions on Learning Technologies (publishedVersion)
  • Lirias (submittedVersion, other-oa)
Funders
  • Research Foundation Flanders (FWO) (G0C9515N)
  • KU Leuven Research Council (STG/14/019)
Topics
  • Physical Sciences › Computer Science › Computer Science Applications › Online Learning and Analytics
  • Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Data Visualization and Analytics
Keywords
learning analytics dashboards, educational recommender systems, student perceptions, needs assessment, visual design, information selection, student behavior, academic performance, usability testing, propensity score matching

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review