A Scoping Review on How Generative Artificial Intelligence Transforms Assessment in Higher Education

May combine two claims.

GenAI added three opportunities-perceived unbiased feedback, immediate and diverse feedback, and self-assessment-and a challenge-student academic integrity-to student assessments.

“Overall, GenAI added three opportunities—perceived unbiased feedback, immediate and diverse feedback, and self-assessment—and a challenge—student academic integrity—to student assessments in the following four aspects (see Fig. 3).”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

ChatGPT performs differently on different tests, possibly because it generates output based on different data sources.

“It was because ChatGPT performs differently on different tests, possibly because it generates output based on different data sources.”

Classified as probably not a substantial claim. May combine two claims.

ChatGPT might use different evaluation criteria from the courses or teachers' own, and its lack of specific knowledge about the class and the students could lead to inappropriate feedback.

“For example, Guo and Wang (2023) suggested that ChatGPT might use different evaluation criteria from the courses or teachers’ own, and its lack of specific knowledge about the class and the students could lead to inappropriate feedback.”

• Also stated in: § Method · hedged

  “In addition, some teachers noticed that ChatGPT might use different evaluation criteria from their own, and its lack of specific knowledge about the class and students could lead to inappropriate feedback (Morjaria et al., 2023).”

• cites Guo & Wang, 2023Not checked yet

  Not checked in the free run.

• cites Morjaria et al., 2023Not checked yet

  Not checked in the free run.

May combine two claims.

ChatGPT can help generate ideas and rubrics, as well as learning questions, for students' self-assessment.

“ChatGPT, for example, can help generate ideas and rubrics (Rajabi et al., 2023), as well as learning questions (Cheung et al., 2023; Cross et al., 2023; Kang, 2023; Naidu & Sevnarayan, 2023) for students’ self-assessment.”

• cites Rajabi et al., 2023Not checked yet

  Not checked in the free run.

• cites Cheung et al., 2023Not checked yet

  Not checked in the free run.

• cites Cross et al., 2023Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Most of the reviewed articles were published in 2023.

“Most of the articles were published in 2023.”

• Also stated in: § Method · plain

  “Most of the articles were published in 2023.”

May combine two claims.

The use of GenAI presented significant challenges to academic integrity and raised concerns about ethical behavior (e.g., cheating) in academic settings.

“They revealed that the use of GenAI presented significant challenges to academic integrity and raised concerns about ethical behavior (e.g., cheating) in academic settings.”

Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

While ChatGPT offers significant potential in educational settings, its limitations, impact on academic integrity, and the need for adapted assessment methods and policies are key areas of concern and focus for educators, students, and institutions.

“While ChatGPT offers significant potential in educational settings, its limitations, impact on academic integrity, and the need for adapted assessment methods and policies are key areas of concern and focus for educators, students, and institutions.”

May combine two claims.

The reviewed articles represent 18 different regions and 11 disciplines, with education, medicine, and computer science being the top three disciplines.

“In total, they represent 18 different regions and 11 disciplines, with education, medicine, and computer science being the top three disciplines.”

May combine two claims.

The learning tasks benefited student learning but did not involve critical thinking.

“The articles highlighted that the learning tasks benefited student learning but did not involve critical thinking (Currie et al., 2023; Rajabi et al., 2023).”

• cites Currie et al., 2023Not checked yet

  Not checked in the free run.

• cites Rajabi et al., 2023Not checked yet

  Not checked in the free run.

May combine two claims.

Submissions generated by ChatGPT scored impressively, and its performance typically qualifies for a First-Class grade, the highest academic distinction awarded in UK universities.

“For example, it was found that submissions generated by ChatGPT scored impressively, and its performance typically qualifies for a First-Class grade, the highest academic distinction awarded in UK universities (Yeadon et al., 2023).”

• cites Yeadon et al., 2023Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The screening retrieved 969 articles and selected 32 empirical studies for analysis.

“In the screening, we retrieved 969 articles and selected 32 empirical studies for analysis.”

• Also stated in: § Method · plain

  “Ultimately, 32 articles were retained for this scoping review; see Table 1.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Short-form essays have become an obsolete assessment tool.

“For example, short-form essays have become an obsolete assessment tool (Yeadon et al., 2023).”

• cites Yeadon et al., 2023Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Some teachers already trusted the information that was generated from ChatGPT and used it as a "one-stop shop" for knowledge sourcing.

“Some teachers already trusted the information that was generated from ChatGPT and used it as a “one-stop shop” for knowledge sourcing (Cross et al., 2023).”

• cites Cross et al., 2023Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Teachers generally agreed that ChatGPT should be integrated into teacher teaching and student learning.

“Teachers generally agreed that ChatGPT should be integrated into teacher teaching and student learning (Cross et al., 2023).”

• cites Cross et al., 2023Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Teachers felt that ChatGPT's feedback, often lengthy and complex, included irrelevant comments.

“For example, teachers felt that ChatGPT’s feedback, often lengthy and complex, included irrelevant comments.”

• cites Guo & Wang, 2023Not checked yet

  Not checked in the free run.

May combine two claims.

GenAI technologies could facilitate teacher assessment processes and pose challenges to the diversity and innovation of assessment methods.

“Its technologies could facilitate teacher assessment processes and pose challenges to the diversity and innovation of assessment methods.”

May combine two claims.

About one-third of the 31 reviewed articles recognize that GenAI will be changing assessment methods, with a need for educators to prepare for these changes.

“About one-third of the 31 reviewed articles recognize that GenAI will be changing assessment methods, with a need for educators to prepare for these changes (Alexander et al., 2023; Guo & Wang, 2023; Naidu & Sevnarayan, 2023; Ouh et al., 2023; Yeadon et al., 2023).”

• cites Alexander et al., 2023Not checked yet

  Not checked in the free run.

• cites Guo & Wang, 2023Not checked yet

  Not checked in the free run.

• cites Naidu & Sevnarayan, 2023Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Nikolic et al. (2023) suggested an alternative assessment method: video-conferencing for evaluating student assignments through questioning and answering.

“For instance, Nikolic et al. (2023) suggested an alternative assessment method: video-conferencing for evaluating student assignments through questioning and answering.”

• cites Nikolic et al., 2023Not checked yet

  Not checked in the free run.

May combine two claims.

Smolansky et al. (2023) found that assignment types rated to be least impacted are presentations and discussions that were either pre-recorded or live; assessments that require product design or creative/artistic work were rated as moderately affected by ChatGPT.

“For example, Smolansky et al. (2023) found that the assignment types rated to be least impacted are presentations and discussions that were either pre-recorded or live; assessments that require product design or creative/artistic work were also rated as moderately affected by ChatGPT.”

• cites Smolansky et al., 2023Not checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

Student overreliance on ChatGPT might impede the development of essential skills like teamwork, leadership, empathy, creativity, critical analysis, and independent thinking-competencies crucial for future job markets.

“This might impede the development of essential skills like teamwork, leadership, empathy, creativity, critical analysis, and independent thinking—competencies crucial for future job markets (Chan, 2023; Rajabi et al., 2023; Stutz et al., 2023).”

• cites Chan, 2023Not checked yet

  Not checked in the free run.

• cites Rajabi et al., 2023Not checked yet

  Not checked in the free run.

• cites Stutz et al., 2023Not checked yet

  Not checked in the free run.

May combine two claims.

GenAI can be used to advance writing such as proofreading, critique, and editing, create personalized assessments, and simulate conversations.

“For example, GenAI can be used to advance writing such as proofreading, critique, and editing (Currie et al., 2023), create personalized assessments, and simulate conversations (Cheung et al., 2023; Currie et al., 2023).”

• cites Currie et al., 2023Not checked yet

  Not checked in the free run.

• cites Cheung et al., 2023Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Chan (2023) conducted a mixed-method study aimed at developing an AI Ecological Education Policy Framework for higher education in Hong Kong.

“For example, Chan (2023) conducted a mixed-method study aimed at developing an AI Ecological Education Policy Framework for higher education in Hong Kong.”

• cites Chan, 2023Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Researchers emphasize the importance of training in new literacy in light of GenAI advancements.

“Researchers emphasize the importance of training in new literacy in light of GenAI advancements (Alexander et al., 2023; Guo & Wang, 2023).”

• cites Alexander et al., 2023Not checked yet

  Not checked in the free run.

• cites Guo & Wang, 2023Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The study found that ChatGPT did indeed pass some courses and performed well in certain types of assessments.

“The study found that ChatGPT did indeed pass some courses and performed well in certain types of assessments.”

• cites Nikolic et al., 2023Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

GenAI enables students to learn disciplinary knowledge or do tasks that they are familiar with, which fosters interdisciplinary learning such as project- and problem-based.

“GenAI enables students to learn a disciplinary knowledge or do some tasks (e.g., drawing, creating videos) that they are familiar, which fosters interdisciplinary learning such as project- and problem- based.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The use of AI tools may not necessarily be considered plagiarism if students are transparent in how they have been used in any submission.

“For example, the use of these tools may not necessarily be considered plagiarism if students are transparent in how they have been used in any submission (Perkins, 2023).”

• cites Perkins, 2023Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The AI Ecological Education Policy Framework comprises Pedagogical, Governance, and Operational dimensions, addressing training and support for teachers, staff, and students in AI literacy.

“This framework comprises Pedagogical, Governance, and Operational dimensions, addressing training and support for teachers, staff, and students in AI literacy.”

• cites Chan, 2023Not checked yet

  Not checked in the free run.

May combine two claims.

Twenty-six reviewed articles indicated that GenAI greatly disrupted traditional assessment approaches and methods, which significantly impacted the formulation of institutional policies.

“Twenty-six reviewed articles indicated that GenAI greatly disrupted traditional assessment approaches and methods, which significantly impacted the formulation of institutional policies.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Nikolic and colleagues (2023) investigated how ChatGPT influences the assessment of engineering education, with subjects including ten courses from seven universities in Australia.

“For example, Nikolic and colleagues (2023) investigated how ChatGPT influences the assessment of engineering education, the subjects included ten courses from seven universities in Australia.”

• cites Nikolic et al., 2023Not checked yet

  Not checked in the free run.

May combine two claims.

Interdisciplinary learning and assessment were advocated in 6% of the reviewed articles; however, most classroom teaching is still single-disciplinary-based.

“Interdisciplinary learning and assessment were advocated in the 6% of the reviewed articles; however, most classroom teaching is still single-disciplinary-based.”

• AssumptionRests on an assumption: Most classroom teaching is still single-disciplinary-based.

Classified as probably not a substantial claim.

The reviewed articles suggested that more diverse and innovative assessment is needed.

“The reviewed articles suggested that more diverse and innovative assessment is needed, which poses new challenges in professional development for teacher literacy, such as assessment, AI and digital literacy (Chan, 2023; Chiu et al., 2024).”

• cites Chan, 2023Not checked yet

  Not checked in the free run.

• cites Chiu et al., 2024Not checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

The reviewed articles suggested that the adoption of GenAI led to more interdisciplinary assessment.

“The reviewed articles suggested that the adoption of GenAI led to shifting educational focus, rethinking learning objectives, more interdisciplinary assessment, and disrupting traditional assessment methods.”

Classified as probably not a substantial claim.

The reviewed articles suggested that the adoption of GenAI led to shifting educational focus.

“The reviewed articles suggested that the adoption of GenAI led to shifting educational focus, rethinking learning objectives, more interdisciplinary assessment, and disrupting traditional assessment methods.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Some of the studies reviewed discussed the application of GenAI in a general context without detailing specific technologies.

“Secondly, some of the studies reviewed discussed the application of GenAI in a general context without detailing specific technologies.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

Various assessment methods, such as face-to-face tests, oral exams, and non-written tasks, can be employed to mitigate the impact of GenAI in academic assessments.

“As discussed, various assessment methods, such as face-to-face tests, oral exams, and non-written tasks, can be employed to mitigate the impact of GenAI in academic assessments.”

Classified as probably not a substantial claim. May combine two claims.

The reviewed articles suggested that assessment in GenAI-based learning should cater to holistic (both generic skills and disciplinary knowledge) and innovative teaching.

“The reviewed articles suggested that assessment in GenAI-based learning should cater to holistic (both generic skills and disciplinary knowledge) and innovative teaching (Heeneman et al., 2015; Popham, 2009).”

• cites Heeneman et al., 2015Not checked yet

  Not checked in the free run.

• cites Popham, 2009Not checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

The reviewed articles suggested that the adoption of GenAI led to disrupting traditional assessment methods.

“The reviewed articles suggested that the adoption of GenAI led to shifting educational focus, rethinking learning objectives, more interdisciplinary assessment, and disrupting traditional assessment methods.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The two coders coded the articles and charted the data continuously in an iterative discussion process using an inductive analysis approach.

“The two coders coded the articles and charted the data continuously in an iterative discussion process using an inductive analysis approach.”

Probably not a substantial claim: may not pass S4 (based). May combine two claims.

Research on GenAI application in higher education assessment methods is still limited, and opinions vary widely as the majority were conducted in 2023.

“However, research on its application in higher education assessment methods is still limited, and opinions vary widely as the majority were conducted in 2023.”

• AssumptionRests on an assumption: Existing literature has not properly investigated how GenAI would impact assessment in higher education.

Classified as probably not a substantial claim.

How GenAI transforms and influences higher education is still unclear.

“How GenAI transforms and influences higher education is still unclear (Al-Zahrani, 2023).”

• cites Al-Zahrani, 2023Not checked yet

  Not checked in the free run.

May combine two claims.

Recent research suggests ChatGPT can aid college students in completing tasks such as essay and proposal writing and take-home examinations, raising concerns about increased risks of cheating and undermining academic integrity.

“Recent research suggests that tools like ChatGPT can aid college students in completing tasks such as essay and proposal writing and take-home examinations, raising concerns about increased risks of cheating and undermining academic integrity (Chiu et al, 2023; Moorhouse et al., 2023).”

• cites Chiu et al, 2023Not checked yet

  Not checked in the free run.

• cites Moorhouse et al., 2023 sourceNot checked yet

  Not checked in the free run.

May combine two claims.

GenAI could assist students in getting disciplinary subject content by producing content-based assignments and providing answers to questions based on the discipline.

“Second, GenAI could assist students in getting disciplinary subject content by producing content-based assignments and providing answers to questions based on the discipline (Chan, 2023; Chaudhry et al., 2023).”

• cites Chan, 2023Not checked yet

  Not checked in the free run.

• cites Chaudhry et al., 2023Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Assessment related to programs and course objectives has a direct impact on learning and teaching for students and teachers.

“Assessment that is related to the programs and course objectives has a direct impact on learning and teaching for students and teachers (Heeneman et al., 2015; Popham, 2009).”

• cites Heeneman et al., 2015Not checked yet

  Not checked in the free run.

• cites Popham, 2009Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

The review used the PRISMA extension for scoping reviews to select articles and report results.

“We used the PRISMA extension for scoping reviews to select articles for review and report the results.”

• Also stated in: § Method · plain

  “We conducted this scoping review in accordance with the PRISMA Extension for Scoping Reviews (PRISMA-ScR) reporting guidelines and checklist (Tricco et al. 2018).”

• cites Tricco et al. 2018Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S4 (based).

The traditional assessment method is becoming increasingly inadequate for accurately assessing students' actual learning outcomes.

“The traditional assessment method is becoming increasingly inadequate for accurately assessing students’ actual learning outcomes.”

Classified as probably not a substantial claim. May combine two claims.

Scoping reviews are an ideal tool for determining the scope or coverage of a body of literature and giving a clear indication of the volume of the literature while providing a broader overview of its focus.

“Scoping reviews are an ideal tool for determining the scope or coverage of a body of literature on a given topic and giving a clear indication of the volume of the literature while providing a broader overview of its focus (Munn et al., 2018).”

• cites Munn et al., 2018Not checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

Although ChatGPT seemed to be powerful, it could not replace teacher feedback.

“These limitations indicated that although ChatGPT seemed to be powerful, it could not replace teacher feedback.”

Classified as probably not a substantial claim. May combine two claims.

Assessment will move from human-centered or machine-centered to mixed methods, transforming the traditional mode of assessment.

“This will move assessment from human-centered or machine-centered to mixed methods, transforming the traditional mode of assessment (Naidu & Sevnarayan, 2023).”

• cites Naidu & Sevnarayan, 2023Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Assessment acts as a driving force in teaching and learning.

“Assessment acts as a driving force in teaching and learning (Heeneman et al., 2015; Popham, 2009).”

• cites Heeneman et al., 2015Not checked yet

  Not checked in the free run.

• cites Popham, 2009Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

This holistic student development ensures that the workforce is well-equipped for the collaborative and interdisciplinary nature of modern challenges.

“This holistic student development ensures that the workforce is well-equipped for the collaborative and interdisciplinary nature of modern challenges.”

Classified as probably not a substantial claim.

Existing performance-based assessment systems adopted by higher education institutions to ensure students are learning and developing necessary skills for future job markets seem likely to be unserviceable with the arrival of ChatGPT.

“The existing performance-based assessment systems adopted by higher education institutions to ensure that students are learning and developing the necessary skills for future job markets seem likely to be unserviceable with the arrival of ChatGPT (Chaudhry et al., 2023).”

• cites Chaudhry et al., 2023Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Personality traits such as grit and perseverance help students adapt and persist in the face of unforeseen challenges.

“These attributes help students adapt and persist in the face of unforeseen challenges (Chiu, 2023; Dai et al., 2023).”

• cites Chiu, 2023Not checked yet

  Not checked in the free run.

• cites Dai et al., 2023Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

ChatGPT does not pose any threat to knowledge-based assessments conducted directly face-to-face on university campuses.

“ChatGPT does not pose any threat to knowledge-based assessments conducted directly face-to-face on university campuses (Ali et al., 2024)”

• cites Ali et al., 2024Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable).

The findings related to the roles of GenAI in this review may not be sufficiently detailed.

“Consequently, the findings related to the roles of GenAI in this review may not be sufficiently detailed.”

Classified as probably not a substantial claim. May combine two claims.

GenAI requires a significant evolution of traditional assessment methods, as its content-generating capabilities may diminish the effectiveness of standard exams and assignments.

“Gorichanaz (2023) suggests that GenAI requires a significant evolution of traditional assessment methods, as its content-generating capabilities may diminish the effectiveness of standard exams and assignments (Chiu et al., 2023).”

• cites Gorichanaz, 2023Not checked yet

  Not checked in the free run.

• cites Chiu et al., 2023Not checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

As a scoping review, the study cannot provide specific recommendations.

“Firstly, as a scoping review, the study cannot provide specific recommendations.”

Classified as probably not a substantial claim. May combine two claims.

The results of this review are preliminary but provide a different understanding of GenAI's role in modernizing and potentially revolutionizing the assessment landscape in higher education.

“Although the results of this review are preliminary, they provide a different understanding of GenAI’s role in modernizing and potentially revolutionizing the assessment landscape in higher education.”