A systematic review of the role of learning analytics in enhancing feedback practices in higher education

In the paper

“we divided these groups of data into two main category including (a) data used for LA feedback intervention and (b) data used for researching LA feedback intervention”

In the paper

“Study context varied - from medicine to statistics -but studies of courses in the field of computer science were found most common (N = 11, 24%).”

In the paper

“Studies from The Netherlands (N = 5, 11%), Turkey (N = 5, 11%), and The UK (N = 3, 8%) were the next most common.”

In the paper

“Satisfaction | 1 | 2% |”

In the paper

“Recommendation | 10 | 22% |”

In the paper

“As shown in Table 2, most of the included papers have been published since 2019 (N = 37, 80%).”

In the paper

“Most of the studies reported were quantitative (N = 28, 61%), alongside 11 qualitative studies (24%), and 7 mixed methods studies (15%).”

In the paper

“A considerable number of papers reported on studies conducted in Australia of the OnTask platform (N = 10, 22%).”

In the paper

“Reduce procrastination | 1 | 2% |”

In the paper

“Reflective thinking skills | 1 | 2% |”

In the paper

“Academic performance | 17 | 37% |”

In the paper

“Transactional distance | 1 | 2% |”

In the paper

“In the first phase of screening in selected databases, we identified a total of 1318 papers (Web of Science (n = 807), Scopus (n = 200), ERIC (n = 101), IEEE (n = 166), and LAK Conference contributions (n = 53)).”

In the paper

“All 46 studies in fact met the minimum score for the quality of methodology and were therefore included in the final analysis.”

In the paper

“Reviewed publications revealed that LA-based feedback underpinned by survey data can affect student cognitive load (Sun et al., 2019; Zheng et al., 2021), the community of inquiry, reflective thinking skills (Yilmaz, 2020), motivation, and transactional distance (Yilmaz & Yilmaz, 2020), and engagement (Sun et al., 2019; Yilmaz & Yilmaz, 2021, pp. 1 -12).”

What the sources say

• Not checked yetcites Sun et al., 2019 source
• Not checked yetcites Zheng et al., 2021
• Not checked yetcites Yilmaz, 2020

In the paper

“In a few of the studies we reviewed, LA is used as a recommender system to guide learners and the learning process (Sedrakyan et al., 2016), and suggest useful course material (Azcona et al., 2019).”

In the paper

“Furthermore, LA-based feedback can influence students ' reflective thinking skills and transactional distance (Yilmaz, 2020; Yilmaz et al., 2020).”

In the paper

“Assessment data is often used by LA in feedback studies to provide recommendations (Lewis et al., 2021; Tempelaar et al., 2018), assessment (Chaudy & Connolly, 2019; Tempelaar et al., 2018), predictive feedback (Arnold & Pistilli, 2012; Baneres et al., 2019), and personalized feedback (Iraj et al., 2020; Lim, Gentili, et al., 2021).”

What the sources say

• Not checked yetcites Lewis et al., 2021 source
• Not checked yetcites Tempelaar et al., 2018 source
• Not checked yetcites Chaudy & Connolly, 2019

In the paper

“Selected studies suggest that survey data-based LA can foster predictive feedback (Broos et al., 2020), recommendation (Perikos et al., 2017), reflective feedback (Smith, 2020; Tsai et al., 2021), and assessment (Dalla Valle et al., 2018).”

What the sources say

• Not checked yetcites Broos et al., 2020 source
• Not checked yetcites Perikos et al., 2017 source
• Not checked yetcites Smith, 2020 source

In the paper

“This data type is one of the most commonly employed in LA studies (Azcona et al., 2019; Tempelaar et al., 2015).”

What the sources say

• Not checked yetcites Azcona et al., 2019 source
• Not checked yetcites Tempelaar et al., 2015 source

In the paper

“Students with higher self-efficacy more appreciated LA-based feedback and had positive experiences with it (Tsai et al., 2021).”

In the paper

“Reviewed studies indicate that such data can be used to generate different types of feedback such as reflective feedback”

In the paper

“However, using LA-based feedback has made it easier for educators to monitor their students ' learning progress and to provide personalized and real-time feedback (Shibani et al., 2020).”

In the paper

“We found that LA-based feedback can also impact students ' motivation (e.g., Smith, 2020) and engagement (e.g., Iraj et al., 2020).”

In the paper

“If LA-based feedback relies only on warning systems, then it can increase the mental load, while putting warning systems with the encouragement of what to do can lead to higher emotional and cognitive engagement (Sun et al., 2019).”

In the paper

“LA-supported feedback entails real-time reflections on students ' performance during their learning processes and that personalized and reflective feedback can help students to be more aware of their own learning processes in order to regulate their learning (Lim, Gasevic, et al., 2021).”

In the paper

“It is often supplementary data for other types of data that can be mined from LMS or other online platforms and can offer more in-depth information.”

In the paper

“in LA feedback studies, it is often combined with other types of data to provide reflective (Lim, Gentili, et al., 2021) and predictive feedback (Azcona et al., 2019), and formative assessment (Tempelaar et al., 2018).”

What the sources say

• Not checked yetcites Lim, Gentili, et al., 2021
• Not checked yetcites Azcona et al., 2019 source
• Not checked yetcites Tempelaar et al., 2018 source

In the paper

“As our review revealed, unsurprisingly, in all included publications, students were mentioned as the LA-supported feedback stakeholder, followed by educators mentioned in 15 out 46 included studies.”

In the paper

“Information visualization methods seem to be commonly used alongside other LA methods to identify students at risk of failure and to understand student perceptions and interactions.”

In the paper

“while at the descriptive level, LA-based feedback provides reflections on students learning activities and learning progress”

In the paper

“at the predictive level, is to predict students ' retention or dropout by using data mining methods.”

In the paper

“but also assist them with improving their reflective thinking skills (e.g. Yilmaz, 2020)”

What the sources say

• Not checked yetcites Yilmaz, 2020

In the paper

“and also has a reminder system that helps students to be more focused and less procrastinated (e.g. Lim et al., 2020).”

What the sources say

• Not checked yetcites Lim et al., 2020 source

In the paper

“Methods used to support feedback include predictive modelling (Azcona et al., 2019; Tempelaar et al., 2015), classifi­ cation (Baneres et al., 2019), clustering (Lim, Gasevic, et al., 2021; Tempelaar et al., 2018; Tsai et al., 2021), Bayesian network analysis (Choi & Cho, 2020), natural language processing (Perikos et al., 2017), process mining (Sedrakyan et al., 2016), relationship mining (T”

What the sources say

• Not checked yetcites Azcona et al., 2019 source
• Not checked yetcites Tempelaar et al., 2015 source
• Not checked yetcites Baneres et al., 2019 source

In the paper

“while LA feedback for students is mostly focused on their own learning performance to provide personalized feedback and to help them with regulating their learning (e.g., Pardo et al., 2019; Sedrakyan et al., 2020).”

What the sources say

• Not checked yetcites Pardo et al., 2019 source
• Not checked yetcites Sedrakyan et al., 2020 source

In the paper

“LA mostly supported students to be informed about how they are doing, reflect on their own learning process to adapt their learning based on the learning needs.”

In the paper

“While for educators, LA mainly provided insights into tracking students learning activities for timely intervention, assessing students learning progress, and early identifying students ' who are at risk”

In the paper

“It is worth noting that many projects employed multiple methods.”

In the paper

“Additional data types employed - demographic, assessment, and academic history data -are also 'about students ' , confirming that most LA for feedback work is student-focused and seeks to serve students as the primary stakeholders.”

In the paper

“Studies have shown that high-quality feedback contributes to authentic learning by promoting student metacognitive skills, and by providing clear indicators of performance and areas for further improvement (e.g. Callender et al., 2016; Molin et al., 2020).”

What the sources say

• Not checked yetcites Callender et al., 2016 source
• Not checked yetcites Molin et al., 2020 source

In the paper

“Effective feedback represents one of the key features of quality teaching (Ramsden, 2003).”

What the sources say

• Not checked yetcites Ramsden, 2003

In the paper

“Consequently, feedback as a core pedagogical practice has therefore gained more attention (e.g. Er et al., 2021; Misiejuk et al., 2021).”

What the sources say

• Not checked yetcites Er et al., 2021
• Not checked yetcites Misiejuk et al., 2021 source

In the paper

“Van der Schaaf et al. (2017) reported that LA enhanced the quality and efficiency of feedback in professional education.”

What the sources say

• Not checked yetcites Van der Schaaf et al., 2017 source

In the paper

“For example, the recent study by Cavalcanti et al. (2021) provide insights into the role of automated feedback in students ' performance and instructors ' workload.”

What the sources say

• Not checked yetcites Cavalcanti et al., 2021

In the paper

“Various analytics methods (techniques) can be applied at different levels, depending on data type and LA goals (Chatti et al., 2012);”

What the sources say

• Not checked yetcites Chatti et al., 2012 source

In the paper

“Then, Theelen et al. ' s (2019) critical appraisal strategy was applied to review the quality of publications.”

What the sources say

• Not checked yetcites Theelen et al., 2019 source

In the paper

“According to this model, LA consists of four dimensions including (1) what (what types of data does the system capture and analyze), (2) how (how does the system perform analytics), (3) why (for what reasons does the system gather and analyze data), and (4) who (who is served by the analytics) (Chatti et al., 2012).”

What the sources say

• Not checked yetcites Chatti et al., 2012 source

In the paper

“The Kappa results showed 91 percent agreement between the two coders ( κ = 0.84, p < .001), which confirms the high consistency and reliability of the coding.”

In the paper

“The interdisciplinary field of LA has borrowed concepts and ideas from diverse disciplines, including computer science, learning science, statistics, data mining, psychology, and pedagogy (Chatti et al., 2012).”

What the sources say

• Not checked yetcites Chatti et al., 2012 source

In the paper

“and it has been suggested that LA should therefore be an essential component of feedback design in data-rich environments (Er et al., 2021; Pardo, 2018).”

What the sources say

• Not checked yetcites Er et al., 2021
• Not checked yetcites Pardo, 2018 source

In the paper

“Sedrakyan et al. (2020) and Lim et al. ' s (2021) found that the use of LA-based feedback supported self-regulated learning”

What the sources say

• Not checked yetcites Sedrakyan et al., 2020 source
• Not checked yetcites Lim et al., 2021

In the paper

“Finally, LA can serve different stakeholders - students, educators, tutors, mentors, institutions, re­ searchers, and/or system designers (Chatti et al., 2012), and no LA system can serve all stakeholders.”

What the sources say

• Not checked yetcites Chatti et al., 2012 source

In the paper

“This study adopted the PRISMA methodology for the systematic review (Moher et al., 2009) to undertake a transparent process of search strategy development, definition of inclusion criteria, and identification of relevant publications.”

What the sources say

• Not checked yetcites Moher et al., 2009 source

In the paper

“Next, a coding scheme was developed based on the suggested theoretical framework and used to analyze these publications.”

In the paper

“We included only studies published in the English language (because most of the publications in this area of research are written in English). We only included publications from 2011 to 2022, since LA as a new field of study emerged in 2011. Finally, to ensure the originality, validity, and quality of selected publications, we included only peer-reviewed articles published in scholarly journals.”

In the paper

“and no LA system can serve all stakeholders.”

What the sources say

• Not checked yetcites Chatti et al., 2012 source

In the paper

“LA seeks to translate learning data into knowledge (Wong et al., 2019) that can give students insight into their own learning, and enable educators to make appropriate and evidence-based interventions for the improvement of teaching and learning (Wise, 2014).”

What the sources say

• Not checked yetcites Wong et al., 2019
• Not checked yetcites Wise, 2014

In the paper

“A further interesting area of work involves the use of feedback analytics to support student academic writing (Gibson et al., 2017; Knight et al., 2020).”

What the sources say

• Not checked yetcites Gibson et al., 2017 source
• Not checked yetcites Knight et al., 2020 source

In the paper

“it has been criticized that initial LA tools were too reliant on this type of data for generating early-warning systems (Ga ˇ sevi ´ c et al., 2016; Lim, Gentili, et al., 2021).”

What the sources say

• Not checked yetcites Ga ˇ sevi ´ c et al., 2016
• Not checked yetcites Lim, Gentili, et al., 2021

In the paper

“For this reason, data mining methods are applied mainly on survey data.”

In the paper

“These insights help teachers to have an overview of students ' interactions with each other and to identify which student is less engaged to support him/her.”

In the paper

“This could be due to the reason that students with high self-efficacy are more able to execute behaviors that are necessary to produce a specific performance (Bandura et al., 1999).”

What the sources say

• Not checked yetcites Bandura et al., 1999 source

In the paper

“Generally, such methods mainly seek to discover patterns, correlations, and anomalies in large volumes of data (Lee & Siau, 2001).”

What the sources say

• Not checked yetcites Lee & Siau, 2001

In the paper

“Furthermore, LA-based recommendations and personalized feedback typically refer to LA at the prescriptive level.”

In the paper

“this figure illustrates that LA services are provided at different levels of analytics according to Gartner Analytics Ascendancy Model (GAAM) (Eriksson et al., 2020; Maoz, 2013).”

What the sources say

• Not checked yetcites Eriksson et al., 2020 source
• Not checked yetcites Maoz, 2013

In the paper

“LA in predictive analytics level tells us ' what will happen ' , while in the prescriptive analytics level, LA recommends actions and interventions for the future (Turkington et al., 2018).”

What the sources say

• Not checked yetcites Turkington et al., 2018

In the paper

“Based on this explanation, it is expected that SNA methods are intended to be used at descriptive and diagnostic levels.”

In the paper

“Network analysis methods appear to be helpful in understanding how students perceive feedback and how they reflect on their learning process alongside exploring how students interact with each other.”

In the paper

“This implies that predictive feedback is expected to be provided at the predictive level.”

In the paper

“the effective and appropriate use of LA to empower feedback practices in higher education requires a clear understanding of the educators ' and students ' pedagogical needs for LA support, required data to meet these needs, and the proper choice of analytics.”

In the paper

“In our findings, we did not make a distinction between educator feedback and peer feedback and how differences between these two types of feedback may lead to different use of LA in higher education contexts.”

In the paper

“We also did not investigate the effect size of different types of data to see which certain type of data can more support feedback activities in higher education settings.”

In the paper

“Third, it is also important to acknowledge that there is a publication bias which means that research with null findings is often not published (Larrabee S ø nderlund et al., 2019).”

What the sources say

• Not checked yetcites Larrabee S ø nderlund et al., 2019

In the paper

“According to the literature, there is a difference between teacher feedback and peer feedback as teachers are more subject-matter experts with lots of feedback experiences and knowledge, and compared to students they need less support in providing feedback (Gielen et al., 2010; Paulus, 1999).”

What the sources say

• Not checked yetcites Gielen et al., 2010 source
• Not checked yetcites Paulus, 1999 source

In the paper

“This hierarchy of steps incorporated into our conceptual framework is in line with the findings of prior studies (e.g. Banihashem et al., 2022; Chatti et al., 2012; Elias, 2011, pp. 1 -22; Clow, 2012).”

What the sources say

• Not checked yetcites Banihashem et al., 2022 source
• Not checked yetcites Chatti et al., 2012 source
• Not checked yetcites Elias, 2011

In the paper

“Defining the recipient of the LA and the goal of the use of LA would make it clear what kind of data should be captured and analyzed (Chatti et al., 2012).”

What the sources say

• Not checked yetcites Chatti et al., 2012 source

In the paper

“These findings suggest that for educators, LA was mainly used at diagnostic and predictive levels, while for students, LA was employed at descriptive and prescriptive levels.”

In the paper

“However, we did not provide insights into how different categories of data are connected to different LA objectives and methods for feedback purposes in higher education.”

In the paper

“the effective and appropriate use of LA to empower feedback practices in higher education requires a clear understanding of the educators ' and students ' pedagogical needs for LA support, required data to meet these needs, and the proper choice of analytics.”

In the paper

“the effective and appropriate use of LA to empower feedback practices in higher education requires a clear understanding of the educators ' and students ' pedagogical needs for LA support, required data to meet these needs, and the proper choice of analytics.”

In the paper

“These findings imply that LA role in the improvement of feedback practices in higher education have not been tightened to feedback itself, but also has evolved beyond our first anticipation based on our proposed model.”

In the paper

“First, our review was limited to empirical studies for sake of reporting only authentic findings, and we recognize that some noteworthy reviews and conceptual papers may have been missed.”

In the paper

“we acknowledge that we may have missed studies not indexed in our selected databases.”

In the paper

“In this study, we did not provide information on identifying the gaps regarding the feedback features and purposes that have not been addressed by LA in higher education.”

In the paper

“Fourth, while this study expands our understanding of how feedback in higher education is influenced by LA, it does not investigate LA-based feedback studies in any K-12 educational contexts. Our findings may therefore not be generalizable to all modes of educational contexts.”

In the paper

“In addition, our findings do not provide insights into the role of LA in features of feedback, for example how LA can influence affective, cognitive, and constructive features of feedback quality.”

In the paper

“Providing feedback for students causes a more cognitive workload compared to teachers (Noroozi et al., 2022).”

What the sources say

• Not checked yetcites Noroozi et al., 2022

In the paper

“By following these steps, we can expect to see LA is delivered properly to support feedback.”