Emerging technologies in higher education assessment and feedback practices: A systematic literature review
Open accessAbstract
The use of Emerging Technologies, such as Artificial Intelligence (AI), Learning Analytics (LA) and Extended Reality (XR) applications, in higher education has proliferated in recent times, as these technologies are considered to have a significant impact on the future of postsecondary teaching and learning. We wanted to find out the emerging technologies used in computing education, its evaluation and effectiveness, and limitations and gaps for future research. We carried out a Systematic Literature Review study on the use of Emerging Technologies in higher education computing education to identify the state of the art in the use of these three groups of technologies for assessment and feedback practices. After systematic search and filtering from a search pool of 3038 studies published between 2016 and 2021, we selected 38 articles for detailed meta-analysis. Our findings reveal that 71% of the reviewed studies are journal articles, 50% studies focus on learning analytics, and the majority of the studies employ quantitative approaches. The results from this systematic review suggest that XR technologies have received least attention to date in computing education (amongst the emerging technologies considered for the review) and there is a lack of frameworks for design, evaluation and use of emerging technologies in higher education. The findings of this review will be beneficial for researchers and educators to obtain an in-depth understanding of the main areas of application of emerging technologies in higher education computing education, an inventory of emerging technology tools used for assessment and feedback, effectiveness indicators, and evaluation approaches that have been used. For evidence-based guidance on future assessment and feedback practices using emerging technologies, we also present a brief research agenda, drawing attention to the need to trial more XR, focus on formative assessment and feedback practices, better understand impact of human-centric issues and take more thoughtful consideration of ethics in the use of emerging technologies in computing education.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Participants listed disparities in access to AI tools and digital infrastructure as particularly salient concerns.
“Echoing the findings of Sembey et al. (2024), participants listed disparities in access to AI tools and digital infrastructure as particularly salient concerns.”
From Redesigning assessments for AI-enhanced learning: a framework for educators in the generative AI era (2025) · cites this paper as Sembey et al. (2024)
Cites
Hattie, J., & Timperley, H. (2007) · Review of Educational Research · doi:10.3102/003465430298487 · Cited by 12,905 · see citing works
Zawacki-Richter, O., Marín, V. I., Bond, M., & Gouverneur, F. (2019) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-019-0171-0 · Cited by 7,242 · see citing works
Nicol, D. J., & Macfarlane‐Dick, D. (2006) · Studies in Higher Education · doi:10.1080/03075070600572090 · Cited by 5,846 · see citing works
Shute, V. J. (2008) · Review of Educational Research · doi:10.3102/0034654307313795 · Cited by 4,631 · see citing works
A Taxonomy of Mixed Reality Visual Displays
Milgram, P., & Kishino, F. (1994) · IEICE Transactions on Information and Systems · Cited by 4,614 · see citing works
Radianti, J., Majchrzak, T. A., Fromm, J., & Wohlgenannt, I. (2020) · Computers & Education · doi:10.1016/j.compedu.2019.103778 · Cited by 3,585 · see citing works
A review of the use of virtual reality head-mounted displays in education and training
Jensen, L., & Konradsen, F. (2018) · Education and Information Technologies · doi:10.1007/s10639-017-9676-0 · Cited by 1,638 · see citing works
Merchant, Z., Goetz, E. T., Cifuentes, L., Keeney-Kennicutt, W., & Davis, T. J. (2014) · Computers & Education · doi:10.1016/j.compedu.2013.07.033 · Cited by 1,508 · see citing works
Online formative assessment in higher education: A review of the literature
Gikandi, J., Morrow, D., & Davis, N. E. (2011) · Computers & Education · doi:10.1016/j.compedu.2011.06.004 · Cited by 1,383 · see citing works
Klopfer, E., & Squire, K. (2008) · Educational Technology Research and Development · doi:10.1007/s11423-007-9037-6 · Cited by 989 · see citing works
Ramaprasad, A. (1983) · Behavioral Science · doi:10.1002/bs.3830280103 · Cited by 915 · see citing works
The Metaphysics of Virtual Reality
Heim, M. (1994) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780195092585.001.0001 · Cited by 853 · see citing works
A Critical Review of the Use of Virtual Reality in Construction Engineering Education and Training
Wang, P., Wu, P., Wang, J., Chi, H.-L., & Wang, X. (2018) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph15061204 · Cited by 782 · see citing works
Methodological guidance paper: High-quality meta- analysis in a systematic review
Pigott, T. D., & Polanin, J. R. (2020) · Review of Educational Research · doi:10.3102/0034654319877153 · Cited by 568 · see citing works
2020 EDUCAUSE Horizon Report: Teaching and Learning Edition.
Brown, M., McCormack, M., Reeves, J., Brook, D. C., Grajek, S., Alexander, B., Bali, M., Bulger, S. R., Dark, S., Engelbert, N., et al. (2020) · Bibliothèque et Archives nationales du Québec (Québec government) · Cited by 502 · see citing works
Augmented Reality Learning Experiences: Survey of Prototype Design and Evaluation
Santos, M. E. C., Chen, A., Taketomi, T., Yamamoto, G., Miyazaki, J., & Kato, H. (2014) · IEEE Transactions on Learning Technologies · doi:10.1109/tlt.2013.37 · Cited by 468 · see citing works
Automatic feedback in online learning environments: A systematic literature review
Cavalcanti, A. P., Barbosa, A., Carvalho, R., Freitas, F., Tsai, Y.-S., Gašević, D., & Mello, R. F. (2021) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2021.100027 · Cited by 440 · see citing works
Using learning analytics to scale the provision of personalised feedback
Pardo, A., Jovanovic, J., Dawson, S., Gašević, D., & Mirriahi, N. (2019) · British Journal of Educational Technology · doi:10.1111/bjet.12592 · Cited by 425 · see citing works
Feng, Z., González, V. A., Amor, R., Lovreglio, R., & Cabrera-Guerrero, G. (2018) · Computers & Education · doi:10.1016/j.compedu.2018.09.002 · Cited by 423 · see citing works
Assessment in the age of artificial intelligence
Swiecki, Z., Khosravi, H., Chen, G., Martinez-Maldonado, R., Lodge, J. M., Milligan, S., Selwyn, N., & Gašević, D. (2022) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2022.100075 · Cited by 409 · see citing works
Learning online: what research tells us about whether, when and how
(2014) · Choice Reviews Online · doi:10.5860/choice.185227 · Cited by 344 · see citing works
Emerging Technologies in Distance Education
Veletsianos, G. (2010) · Athabasca University Press eBooks · doi:10.15215/aupress/9781897425763.01 · Cited by 220 · see citing works
Process mining for self-regulated learning assessment in e-learning
Cerezo, R., Bogarín, A., Esteban, M., & Romero, C. (2019) · Journal of Computing in Higher Education · doi:10.1007/s12528-019-09225-y · Cited by 173 · see citing works
Understanding Virtual Reality: Presence, Embodiment, and Professional Practice
Tham, J. C. K., Duin, A. H., Gee, L., Ernst, N., Abdelqader, B., & McGrath, M. (2018) · IEEE Transactions on Professional Communication · doi:10.1109/tpc.2018.2804238 · Cited by 148 · see citing works
Analytics of Learning Strategies
Matcha, W., Gašević, D., Uzir, N. A., Jovanović, J., & Pardo, A. (2019) · International Conference on Learning Analytics and Knowledge (LAK) · doi:10.1145/3303772.3303787 · Cited by 142 · see citing works
Cen, L., Ruta, D., Powell, L., Hirsch, B., & Ng, J. (2016) · International Journal of Computer-Supported Collaborative Learning · doi:10.1007/s11412-016-9234-6 · Cited by 125 · see citing works
Lim, L.-A., Dawson, S., Gašević, D., Joksimović, S., Pardo, A., Fudge, A., & Gentili, S. (2021) · Assessment & Evaluation in Higher Education · doi:10.1080/02602938.2020.1782831 · Cited by 113 · see citing works
Kochmar, E., Vu, D. D., Belfer, R., Gupta, V., Serban, I. V., & Pineau, J. (2022) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-021-00267-x · Cited by 112 · see citing works
A synthesis of systematic review research on emerging learning environments and technologies
Martin, F., Dennen, V. P., & Bonk, C. J. (2020) · Educational Technology Research and Development · doi:10.1007/s11423-020-09812-2 · Cited by 107 · see citing works
Chou, C.-Y., & Zou, N.-B. (2020) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-020-00233-y · Cited by 102 · see citing works
Ask-Elle: an Adaptable Programming Tutor for Haskell Giving Automated Feedback
Gerdes, A., Heeren, B. J., Jeuring, J. T., & van Binsbergen, L. T. (2016) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-015-0080-x · Cited by 96 · see citing works
Using Learning Analytics to Assess Student Learning in Online Courses
Martin, F., & Ndoye, A. (2016) · Journal of University Teaching and Learning Practice · doi:10.53761/1.13.3.7 · Cited by 89 · see citing works
Zheng, L., Niu, J., & Zhong, L. (2021) · British Journal of Educational Technology · doi:10.1111/bjet.13156 · Cited by 78 · see citing works
Tempelaar, D. T. (2019) · Assessment & Evaluation in Higher Education · doi:10.1080/02602938.2019.1677855 · Cited by 76 · see citing works
An Educational System for Learning Search Algorithms and Automatically Assessing Student Performance
Grivokostopoulou, F., Perikos, I., & Hatzilygeroudis, I. (2017) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-016-0116-x · Cited by 71 · see citing works
Predicting learners' effortful behaviour in adaptive assessment using multimodal data
Sharma, K., Papamitsiou, Z. K., Olsen, J. K., & Giannakos, M. N. (2020) · International Conference on Learning Analytics and Knowledge (LAK) · doi:10.1145/3375462.3375498 · Cited by 66 · see citing works
Students’ sense-making of personalised feedback based on learning analytics
Lim, L.-A., Dawson, S., Gašević, D., Joksimović, S., Fudge, A., Pardo, A., & Gentili, S. (2020) · Australasian Journal of Educational Technology · doi:10.14742/ajet.6370 · Cited by 66 · see citing works
Wang, D., & Han, H. (2021) · Journal of Computer Assisted Learning · doi:10.1111/jcal.12502 · Cited by 55 · see citing works
Designing and evaluating tutoring Feedback Strategies for Digital Learning
Narciss, S. (2013) · Dialnet (Universidad de la Rioja) · Cited by 54 · see citing works
Ochôa, X., & Domínguez, F. (2020) · British Journal of Educational Technology · doi:10.1111/bjet.12987 · Cited by 54 · see citing works
Reflecting on reflective writing analytics
Shum, S. B., Sándor, Á., Goldsmith, R., Wang, X., Bass, R. V., & McWilliams, M. (2016) · International Conference on Learning Analytics and Knowledge (LAK) · doi:10.1145/2883851.2883955 · Cited by 53 · see citing works
Doble, C. W., Matayoshi, J., Cosyn, E., Uzun, H., & Karami, A. (2019) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-019-00176-0 · Cited by 48 · see citing works
Perikos, I., Grivokostopoulou, F., & Hatzilygeroudis, I. (2017) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-017-0139-y · Cited by 44 · see citing works
Student appreciation of data-driven feedback: A pilot study on OnTask
Tsai, Y.-S., Mello, R. F., Jovanović, J., & Gašević, D. (2021) · LAK21: 11th International Learning Analytics and Knowledge Conference · doi:10.1145/3448139.3448212 · Cited by 41 · see citing works
Leite, A., & Blanco, S. A. (2020) · ACM Technical Symposium on Computer Science Education (SIGCSE) · doi:10.1145/3328778.3366921 · Cited by 39 · see citing works
Omer, U., Farooq, M. S., & Abid, A. (2020) · Sustainability · doi:10.3390/su12176990 · Cited by 38 · see citing works
Chango, W. G., Cerezo, R., Sánchez‐Santillán, M., Azevedo, R., & Romero, C. (2021) · Journal of Computing in Higher Education · doi:10.1007/s12528-021-09298-8 · Cited by 38 · see citing works
What Learning Analytics-Based Prediction Models Tell us About Feedback Preferences of Students
Nguyen, Q., Rienties, B., Tempelaar, D. T., & Giesbers, B. (2016) · Quarterly review of distance education · doi:10.1108/qrde-12-2016-0003 · Cited by 31 · see citing works
Towards Progress Assessment for Adaptive Hints in Educational Virtual Reality Games
Drey, T., Jansen, P., Fischbach, F., Frommel, J., & Rukzio, E. (2020) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3334480.3382789 · Cited by 29 · see citing works
Hooshyar, D., Ahmad, R. B., Yousefi, M., Fathi, M., Abdollahi, A., Horng, S., & Lim, H. (2016) · Educational Technology Research and Development · doi:10.1007/s11423-016-9433-x · Cited by 23 · see citing works
Azevedo, J. M., Oliveira, E. P., & Beites, P. D. (2019) · The International Journal of Information and Learning Technology · doi:10.1108/ijilt-02-2019-0023 · Cited by 19 · see citing works
A Systematic Review of Formative Assessment Tools in the Blended Learning Environment
Febriani, I., & Abdullah, M. I. (2018) · International Journal of Engineering & Technology · doi:10.14419/ijet.v7i4.11.20684 · Cited by 18 · see citing works
Research on Immersion for Learning Using Virtual Reality, Augmented Reality and Mixed Reality
Chang, C., Chena, C.-L. D., & Chang, W.-K. (2019) · Enfance · doi:10.3917/enf2.193.0413 · Cited by 18 · see citing works
Learning UI Functional Design Principles Through Simulation With Feedback
de la Peña, J. R., Serral, E., & Snoeck, M. (2020) · IEEE Transactions on Learning Technologies · doi:10.1109/tlt.2020.3028596 · Cited by 15 · see citing works
Lee, H. H. (2021) · ACM Technical Symposium on Computer Science Education (SIGCSE) · doi:10.1145/3408877.3432463 · Cited by 14 · see citing works
Peer assessment using soft computing techniques
Pinargote-Ortega, M., Bowen-Mendoza, L. E., Meza, J., & Ventura, S. (2021) · Journal of Computing in Higher Education · doi:10.1007/s12528-021-09296-w · Cited by 12 · see citing works
Transaction-level learning analytics in online authentic assessments
Nyland, R., Davies, R. S., Chapman, J., et al. (2016) · Journal of Computing in Higher Education · doi:10.1007/s12528-016-9122-0 · Cited by 11 · see citing works
Learning analytics for the development of adapted e‐assessment workflow system
Hlaoui, Y. B., Hajjej, F., & Ayed, L. J. B. (2016) · Computer Applications in Engineering Education · doi:10.1002/cae.21770 · Cited by 9 · see citing works
Personalized Student Assessment based on Learning Analytics and Recommender Systems
Gonçalves, A. L., Carlos, L. M., da Silva, J. B., et al. (2018) · International Conference of the Portuguese Society for Engineering Education (CISPEE) · doi:10.1109/cispee.2018.8593493 · Cited by 8 · see citing works
Villamañe, M., Larrañaga, M., Álvarez, A., & Ferrero, B. (2016) · Ninth International Conference on Technological Ecosystems for Enhancing Multiculturality (TEEM'21) · doi:10.1145/3012430.3012541 · Cited by 4 · see citing works
Ribeiro, J., Dias, A., Marques, J. O. D. A., Avidos, L. M. M. A., Araújo, I., Araújo, N. M. F., & Figueiredo, M. (2019) · International Conference on e-Education, e-Business, e-Management and e-Learning · doi:10.1145/3306500.3306515 · Cited by 4 · see citing works
Using learning analytics to assess project management skills on engineering degree courses
Menchaca, I., Guenaga, M., & Solabarrieta, J. (2016) · Ninth International Conference on Technological Ecosystems for Enhancing Multiculturality (TEEM'21) · doi:10.1145/3012430.3012542 · Cited by 3 · see citing works
Taher, M., Ghani, U., & Khan, A. S. (2018) · ASEE Annual Conference & Exposition · doi:10.18260/1-2--27452 · Cited by 1 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Redesigning assessments for AI-enhanced learning: a framework for educators in the generative AI era
Khlaif, Z. N., Alkouk, W. A., Salama, N., & Abu Eideh, B. (2025) · Education Sciences · doi:10.3390/educsci15020174 · Cited by 127 · see citing works
- Authors
- Sembey, R., Hoda, R., & Grundy, J.
- Affiliations
- Monash University (AU)
- Corresponding author
- Sembey, Ruchi
- Venue
- Journal of Systems and Software
- Publisher
- Elsevier BV
- Published
- 2024-05
- Volume
- 211
- Pages
- 111988
- ISSN
- 0164-1212 (ISSN-L)
- OpenAlex
- W4391848882
- Cited by
- 49
- References
- 63
- Citations by year
- 2026: 22 · 2025: 18 · 2024: 9
- FWCI
- 3.68
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Journal of Systems and Software (publishedVersion, cc-by)
- Monash University Research Portal (Monash University) (publishedVersion, other-oa)
- Funders
- Australian Research Council
- Topics
- Physical Sciences › Computer Science › Computer Science Applications › Online Learning and Analytics
- Physical Sciences › Computer Science › Information Systems › Artificial Intelligence in Education
- Social Sciences › Education › Student Assessment and Feedback
- Keywords
- emerging technologies, higher education, assessment, feedback, computing education, artificial intelligence, learning analytics, extended reality, formative assessment, educational technology evaluation, research agenda, ethics of emerging technologies
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
