The DOI matches its Crossref record; 69 of the 69 references it cites were found in a registry or an index.
License to evaluate: preparing learning analytics dashboards for educational practice
Open accessAbstract
Learning analytics can bridge the gap between learning sciences and data analytics, leveraging the expertise of both fields in exploring the vast amount of data generated in online learning environments. A typical learning analytics intervention is the learning dashboard, a visualisation tool built with the purpose of empowering teachers and learners to make informed decisions about the learning process. Related work has investigated learning dashboards, yet none have explored the theoretical foundation that should inform the design and evaluation of such interventions. In this systematic literature review, we analyse the extent to which theories and models from learning sciences have been integrated into the development of learning dashboards aimed at learners. Our analysis revealed that very few dashboard evaluations take into account the educational concepts that were used as a theoretical foundation for their design. Furthermore, we report findings suggesting that comparison with peers, a common reference frame for contextualising information on learning analytics dashboards, was not perceived positively by all learners. We summarise the insights gathered through our literature review in a set of recommendations for the design and evaluation of learning analytics dashboards for learners.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: the PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & for the PRISMA Group (2009) · BMJ · doi:10.1136/bmj.b2535 · Cited by 83,823
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & The PRISMA Group (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000097 · Cited by 64,042
A theory of social comparison processes
Festinger, L. (1954) · Human Relations · doi:10.1177/001872675400700202 · Cited by 20,926
SUS: A 'Quick and Dirty' Usability Scale
(1996) · Usability Evaluation In Industry · doi:10.1201/9781498710411-35 · Cited by 8,989
Self-Regulated Learning and Academic Achievement: An Overview
Zimmerman, B. J. (1990) · Educational Psychologist · doi:10.1207/s15326985ep2501_2 · Cited by 3,687
(1991) · Cited by 3,259
Multiple goals, multiple pathways: The role of goal orientation in learning and achievement
Pintrich, P. R. (2000) · Journal of Educational Psychology · doi:10.1037/0022-0663.92.3.544 · Cited by 2,254
The revised two‐factor Study Process Questionnaire: R‐SPQ‐2F
Biggs, J., Kember, D., & Leung, D. Y. P. (2001) · British Journal of Educational Psychology · doi:10.1348/000709901158433 · Cited by 2,122 · cited 1×
“…Achievement Goal Orien-tations [21], the Revised Two-Factor Study Process Questionnaire (R-SPQ-2F) to determine students’ approaches to learning [8], the Online Self-Regulated Learning Questionnaire (OSLQ) for measuring self-regulation in online and blended learning environments [6], the…”
BackgroundHandbook of research on educational communications and technology
(2004) · British Journal of Educational Technology · doi:10.1111/j.1467-8535.2004.00409_15.x · Cited by 1,840
Learning analytics: drivers, developments and challenges
Ferguson, R. (2012) · International Journal of Technology Enhanced Learning · doi:10.1504/ijtel.2012.051816 · Cited by 1,354
Let’s not forget: Learning analytics are about learning
Gašević, D., Dawson, S., & Siemens, G. (2015) · TechTrends · doi:10.1007/s11528-014-0822-x · Cited by 1,049
Measuring self-regulation in online and blended learning environments
Barnard, L., Lan, W. Y., To, Y. M., Paton, V. O., & Lai, S.-L. (2009) · The Internet and Higher Education · doi:10.1016/j.iheduc.2008.10.005 · Cited by 913
A 3 × 2 achievement goal model.
Elliot, A. J., Murayama, K., & Pekrun, R. (2011) · Journal of Educational Psychology · doi:10.1037/a0023952 · Cited by 853
(2012) · DSpace (Open University in the Netherlands) · Cited by 754
Learning Analytics Dashboard Applications
Verbert, K., Duval, E., Klerkx, J., Govaerts, S., & Santos, J. L. (2013) · American Behavioral Scientist · doi:10.1177/0002764213479363 · Cited by 632
Perceiving Learning at a Glance: A Systematic Literature Review of Learning Dashboard Research
Schwendimann, B. A., Rodriguez-Triana, M. J., Vozniuk, A., Prieto, L. P., Boroujeni, M. S., Holzer, A., Gillet, D., & Dillenbourg, P. (2017) · IEEE Transactions on Learning Technologies · doi:10.1109/tlt.2016.2599522 · Cited by 545
Learning dashboards: an overview and future research opportunities
Verbert, K., Govaerts, S., Duval, E., Santos, J. L., Van Assche, F., Parra, G., & Klerkx, J. (2013) · Personal and Ubiquitous Computing · doi:10.1007/s00779-013-0751-2 · Cited by 453
Stupid tutoring systems, intelligent humans
Baker, R. S. (2016) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-016-0105-0 · Cited by 425 · cited 1×
“Recently, there has been a shift from mining learning data to support automated interventions towards directly reporting the data, usually in a visual form, thus empowering learners and teachers to better leverage human judgement [5].”
BackgroundMotivated Strategies for Learning Questionnaire
Pintrich, P. R., & De Groot, E. V. (2012) · PsycTESTS Dataset · doi:10.1037/t09161-000 · Cited by 399
Bodily, R., & Verbert, K. (2017) · IEEE Transactions on Learning Technologies · doi:10.1109/tlt.2017.2740172 · Cited by 306
Guest Editorial-Learning and Knowledge Analytics
Siemens, G., & Gašević, D. (2012) · HAL · Cited by 293
Investigating the learning‐theory foundations of game‐based learning: a meta‐analysis
Wu, W., Hsiao, H., Wu, P., Lin, C., & Huang, S. (2012) · Journal of Computer Assisted Learning · doi:10.1111/j.1365-2729.2011.00437.x · Cited by 279
Reflection: Issues Related to Improving Professors’ Teaching and Students’ Learning
McAlpine, L., & Weston, C. (2002) · Teacher Thinking, Beliefs and Knowledge in Higher Education · doi:10.1007/978-94-010-0593-7_4 · Cited by 258
Designing pedagogical interventions to support student use of learning analytics
Wise, A. F. (2014) · Proceedings of the Fourth International Conference on Learning Analytics And Knowledge · doi:10.1145/2567574.2567588 · Cited by 252
Time will tell: The role of mobile learning analytics in self-regulated learning
Tabuenca, B., Kalz, M., Drachsler, H., & Specht, M. (2015) · Computers & Education · doi:10.1016/j.compedu.2015.08.004 · Cited by 242
Self-citationAwareness Is Not Enough: Pitfalls of Learning Analytics Dashboards in the Educational Practice
Jivet, I., Scheffel, M., Drachsler, H., & Specht, M. (2017) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-66610-5_7 · Cited by 211
Self-citationDevelopment of the Learning Analytics Dashboard to Support Students' Learning Performance
Jo, I.-H., & Park, Y. (2020) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.3217/jucs-021-01-0110 · Cited by 203
Learning dispositions and transferable competencies: pedagogy, modelling and learning analytics
Buckingham Shum, S., & Crick, R. D. (2012) · Proceedings of the 2nd International Conference on Learning Analytics and Knowledge · doi:10.1145/2330601.2330629 · Cited by 202
Follow the successful crowd: raising MOOC completion rates through social comparison at scale
Davis, D., Jivet, I., Kizilcec, R. F., Chen, G., Hauff, C., & Houben, G. (2017) · International Conference on Learning Analytics and Knowledge (LAK) · doi:10.1145/3027385.3027411 · Cited by 201
Self-citationTrends and issues in student-facing learning analytics reporting systems research
Bodily, R., & Verbert, K. (2017) · International Conference on Learning Analytics and Knowledge (LAK) · doi:10.1145/3027385.3027403 · Cited by 173
Reflection: issues related to improving professors' teaching and students' learning
McAlpine, L., & Weston, C. (2000) · Instructional Science · doi:10.1023/a:1026583208230 · Cited by 165
Ruipérez‐Valiente, J. A., Muñoz‐Merino, P. J., Leony, D., & Kloos, C. D. (2014) · Computers in Human Behavior · doi:10.1016/j.chb.2014.07.002 · Cited by 163
Tempelaar, D. T., Niculescu, A., Rienties, B., Gijselaers, W., & Giesbers, B. (2011) · The Internet and Higher Education · doi:10.1016/j.iheduc.2011.10.003 · Cited by 162
Factors influencing beliefs for adoption of a learning analytics tool: An empirical study
Ali, L., Asadi, M., Gašević, D., Jovanović, J., & Hatala, M. (2012) · Computers & Education · doi:10.1016/j.compedu.2012.10.023 · Cited by 158
Kim, J., Jo, I., & Park, Y. (2015) · Asia Pacific Education Review · doi:10.1007/s12564-015-9403-8 · Cited by 157
Lonn, S., Aguilar, S. J., & Teasley, S. D. (2015) · Computers in Human Behavior · doi:10.1016/j.chb.2014.07.013 · Cited by 152
Learning Analytics for Online Discussions: Embedded and Extracted Approaches
Wise, A. F., Zhao, Y., & Hausknecht, S. N. (2014) · Journal of Learning Analytics · doi:10.18608/jla.2014.12.4 · Cited by 131
Evaluating Training Programs: Evidence vs. Proof.
Kirkpatrick, D. L. (1977) · Training and development journal · Cited by 114
What if learning analytics were based on learning science?
Marzouk, Z., Raković, M., Liaqat, A., Vytasek, J., Samadi, D., Stewart-Alonso, J., Ram, I., Woloshen, S., Winne, P. H., & Nesbit, J. C. (2016) · Australasian Journal of Educational Technology · doi:10.14742/ajet.3058 · Cited by 111
Learning analytics as a "middle space"
Suthers, D., & Verbert, K. (2013) · International Conference on Learning Analytics and Knowledge (LAK) · doi:10.1145/2460296.2460298 · Cited by 95
Enhancing Learning with Visualization Techniques
Klerkx, J., Verbert, K., & Duval, E. (2013) · Springer eBooks · doi:10.1007/978-1-4614-3185-5_64 · Cited by 93
The role of achievement goal orientations when studying effect of learning analytics visualizations
Beheshitha, S. S., Hatala, M., Gašević, D., & Joksimovíc, S. (2016) · International Conference on Learning Analytics and Knowledge (LAK) · doi:10.1145/2883851.2883904 · Cited by 93
Guerra, J., Hosseini, R., Somyürek, S., & Brusilovsky, P. L. (2016) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/2856767.2856784 · Cited by 92
How do students interpret feedback delivered via dashboards?
Corrin, L., & Barba, P. D. (2015) · International Conference on Learning Analytics and Knowledge (LAK) · doi:10.1145/2723576.2723662 · Cited by 91
Educational Dashboards for Smart Learning: Review of Case Studies
Yoo, Y., Lee, H., Jo, I., & Park, Y. (2014) · Lecture notes in educational technology · doi:10.1007/978-3-662-44188-6_21 · Cited by 80
Susan Bull, A., Ginon, B., Boscolo, C., & Johnson, M. (2016) · International Conference on Learning Analytics and Knowledge (LAK) · doi:10.1145/2883851.2883853 · Cited by 69
Supporting learning by considering emotions: tracking and visualization a case study
Ruiz, S., Charleer, S., Urretavizcaya, M., Klerkx, J., Fernández-Castro, I., & Duval, E. (2016) · International Conference on Learning Analytics and Knowledge (LAK) · doi:10.1145/2883851.2883888 · Cited by 60
Emerging Issues in Smart Learning
Chen, G., Kumar, V., Kinshuk, K., Huang, R., & Kong, S. C. (2014) · Lecture notes in educational technology · doi:10.1007/978-3-662-44188-6 · Cited by 58
Measuring and Visualizing Students’ Behavioral Engagement in Writing Activities
Liu, M., Calvo, R. A., Pardo, A., & Martin, A. J. (2014) · IEEE Transactions on Learning Technologies · doi:10.1109/tlt.2014.2378786 · Cited by 58
Mejía, C., Florián-Gaviria, B., Vatrapu, R., Susan Bull, A., Gómez, S., & Fabregat, R. (2016) · IEEE Transactions on Learning Technologies · doi:10.1109/tlt.2016.2626292 · Cited by 58
Creating Effective Learning Analytics Dashboards: Lessons Learnt
Charleer, S., Klerkx, J., Duval, E., Laet, T. D., & Verbert, K. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-45153-4_4 · Cited by 56
Chou, C., Tseng, S., Chih, W.-C., Chen, Z., Chao, P.-Y., Lai, K.-H. R., Chan, C., Yu, L.-C., & Lin, Y.-L. (2015) · IEEE Transactions on Emerging Topics in Computing · doi:10.1109/tetc.2015.2501805 · Cited by 53
Koh, E., Shibani, A., Tan, J. P.-L., & Hong, H. T. S. (2016) · International Conference on Learning Analytics and Knowledge (LAK) · doi:10.1145/2883851.2883914 · Cited by 50
Mastery Grids: An Open Source Social Educational Progress Visualization
Loboda, T. D., Guerra, J., Hosseini, R., & Brusilovsky, P. L. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-11200-8_18 · Cited by 49
Exploring the Impact of a Learning Dashboard on Student Affect
Müldner, K., Wixon, M., Rai, D., Burleson, W., Woolf, B. P., & Arroyo, I. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-19773-9_31 · Cited by 47
Constructing Interpretative Views of Learners’ Interaction Behavior in an Open Learner Model
Papanikolaou, K. A. (2014) · IEEE Transactions on Learning Technologies · doi:10.1109/tlt.2014.2363663 · Cited by 46
Aljohani, N. R., & Davis, H. (2013) · International Conference on Next Generation Mobile Applications, Services and Technologies (NGMAST) · doi:10.1109/ngmast.2013.54 · Cited by 44
The Diabetes Time Management Questionnaire
Gafarian, C. T., Heiby, E. M., Blair, P., & Singer, F. (1999) · The Science of Diabetes Self-Management and Care · doi:10.1177/014572179902500411 · Cited by 42
Validity and reliability of time management questionnaire
ALAY, S., & Koçak, S. (2014) · DergiPark (Istanbul University) · Cited by 38
Guest editorial: technology supported cognition and exploratory learning
Ifenthaler, D., Isaias, P. T., Kinshuk, K., Sampson, D. G., & Spector, J. M. (2012) · Portuguese National Funding Agency for Science, Research and Technology (RCAAP Project by FCT) · Cited by 37
A visual recommender tool in a collaborative learning experience
Anaya, A. R., Luque, M., & Peinado, M. (2015) · Expert Systems with Applications · doi:10.1016/j.eswa.2015.01.071 · Cited by 37
Scheffel, M., Drachsler, H., Toisoul, C., Ternier, S., & Specht, M. M. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-66610-5_15 · Cited by 37
Self-citationDaley, S. G., Hillaire, G., & Sutherland, L. M. (2014) · British Journal of Educational Technology · doi:10.1111/bjet.12221 · Cited by 34
Tan, J. P.-L., Yang, S., Koh, E., & Jonathan, C. R. (2016) · International Conference on Learning Analytics and Knowledge (LAK) · doi:10.1145/2883851.2883965 · Cited by 31
Evaluating the Student Activity Meter: Two Case Studies
Govaerts, S., Verbert, K., & Duval, E. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-25813-8_20 · Cited by 24
Konert, J., Bohr, C., Bellhäuser, H., et al. (2016) · IEEE International Conference on Advanced Learning Technologies (ICALT) · doi:10.1109/icalt.2016.100 · Cited by 18
Widget, widget as you lead, I am performing well indeed!
Scheffel, M., Drachsler, H., Kreijns, K., de Kraker, J., & Specht, M. M. (2017) · International Conference on Learning Analytics and Knowledge (LAK) · doi:10.1145/3027385.3027428 · Cited by 13
Self-citationVenant, R., Vidal, P., & Broisin, J. (2016) · IEEE International Conference on Advanced Learning Technologies (ICALT) · doi:10.1109/icalt.2016.60 · Cited by 11
Associations between students' approaches to learning and learning analytics visualizations
Hatala, M., Beheshitha, S. S., & Gašević, D. (2016) · HAL · Cited by 8
Cited by
The current landscape of learning analytics in higher education
Viberg, O., Hatakka, M., Bälter, O., & Mavroudi, A. (2018) · Computers in Human Behavior · doi:10.1016/j.chb.2018.07.027 · Cited by 766
Matcha, W., Uzir, N. A., Gasevic, D., & Pardo, A. (2020) · IEEE Transactions on Learning Technologies · doi:10.1109/tlt.2019.2916802 · Cited by 399
Towards hybrid human‐AI learning technologies
Molenaar, I. (2022) · European Journal of Education · doi:10.1111/ejed.12527 · Cited by 348
Human-Centred Learning Analytics
Buckingham Shum, S., Ferguson, R., & Martinez-Maldonado, R. (2019) · Journal of Learning Analytics · doi:10.18608/jla.2019.62.1 · Cited by 258
Learning analytics in European higher education—Trends and barriers
Tsai, Y.-S., Rates, D., Moreno-Marcos, P. M., Muñoz-Merino, P. J., Jivet, I., Scheffel, M., Drachsler, H., Delgado Kloos, C., & Gašević, D. (2020) · Computers & Education · doi:10.1016/j.compedu.2020.103933 · Cited by 192
Self-citation
Metadata
- Authors
- Jivet, I., Scheffel, M., Specht, M., & Drachsler, H.
- Venue
- Proceedings of the 8th International Conference on Learning Analytics and Knowledge
- Publisher
- ACM
- Published
- 2018-03-07
- Pages
- 31-40
- OpenAlex
- W2784078400
- Cited by
- 277
- References
- 69
- FWCI
- 37.89
- Type
- conference paper
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.