Cited by 87Artificial Intelligence in EducationEducational Assessment and PedagogyArtificial Intelligence in Healthcare and Education

Race with the machines: assessing the capability of generative AI in solving authentic assessments

Open access
Nguyen Thanh, Binh, Vo, Diem Thi Hong, Nguyen Nhat, Minh, Pham, Thi Thu Tra, Thai Trung, Hieu, Ha Xuan, Son·2023·Australasian Journal of Educational Technology

Abstract

In this study, we introduce a framework designed to help educators assess the effectiveness of popular generative artificial intelligence (AI) tools in solving authentic assessments. We employed Bloom’s taxonomy as a guiding principle to create authentic assessments that evaluate the capabilities of generative AI tools. We applied this framework to assess the abilities of ChatGPT-4, ChatGPT-3.5, Google Bard and Microsoft Bing in solving authentic assessments in economics. We found that generative AI tools perform very well at the lower levels of Bloom's taxonomy while still maintaining a decent level of performance at the higher levels, with “create” being the weakest level of performance. Interestingly, these tools are better able to address numeric-based questions than text-based ones. Moreover, all the generative AI tools exhibit weaknesses in building arguments based on theoretical frameworks, maintaining the coherence of different arguments and providing appropriate references. Our study provides educators with a framework to assess the capabilities of generative AI tools, enabling them to make more informed decisions regarding assessments and learning activities. Our findings demand a strategic reimagining of educational goals and assessments, emphasising higher cognitive skills and calling for a concerted effort to enhance the capabilities of educators in preparing students for a rapidly transforming professional environment. Implications for practice or policy Our proposed framework enables educators to systematically evaluate the capabilities of widely used generative AI tools in assessments and assist them in the assessment design process. Tertiary institutions should re-evaluate and redesign programmes and course learning outcomes. The new focus on learning outcomes should address the higher levels of educational goals of Bloom’s taxonomy, specifically the “create” level.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.14742/ajet.8902

Cites

60 of 60 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Generative AI performs weakest at Bloom's create level and struggles with coherent argumentation.

    “Generative AI performs weakest at Bloom’s create level and struggles with coherent argumentation (Thanh et al., 2023), though intentional pedagogical design can support deeper engagement (Kwan et al., 2025; Zhou et al., 2024).”
  • Assessments should prioritise critical thinking, problem-solving, and creativity over rote memorisation.

    “Research suggests that assessments should prioritise critical thinking, problemsolving, and creativity over rote memorisation [ 26, 34].”
  • Generative AI tools such as ChatGPT-4, ChatGPT-3.5, Google Bard, and Microsoft Bing perform well at the lower levels of Bloom's taxonomy (e.g., remembering and understanding).

    “A study by [ 34] found that generative AI tools, such as ChatGPT-4, Chat -GPT-3.5, Google Bard, and Microsoft Bing, perform well at the lower levels of Bloom's taxonomy (e.g., remember -ing and understanding).”
  • Integrating multimodal generative models can enhance the authenticity of simulated assessment environments.

    “Integrating multimodal generative models, such as GPT-4 Vision for text and image generation, Meta's Voicebox for audio creation from text, and generative adversarial networks for digital avatar production, can further enhance the authenticity of simulated assessment environments [66].”

    From Promises and challenges of generative artificial intelligence for human learning (2024) · cites this paper as [66]

    The Introduction discusses how generative AI can complete authentic assessment tasks, and the literature review describes realism in assessments. Neither addresses using multimodal models to make simulated assessment environments more authentic. Searches for simulated environments, GPT-4 Vision, Voicebox, audio, and avatars found no supporting passage in our copy. (LLM, conf=0.74)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

2 of the 62 references couldn’t be matched to a known paper.

Research works printed but not linked: 2

  1. Samuelson, P. A., & Nordhaus, W. D. (2013). Economics (19th ed.). McGraw Hill.
  2. v2n1p197 · DOI

Cited by

Sorted by citations

Authors
Nguyen Thanh, B., Vo, D. T. H., Nguyen Nhat, M., Pham, T. T. T., Thai Trung, H., & Ha Xuan, S.
Affiliations
RMIT Vietnam (VN)
Venue
Australasian Journal of Educational Technology
Published
2023-12-22
Volume
39
Issue
5
Pages
59-81
ISSN
  • 1449-3098 (ISSN-L)
  • 1449-5554
OpenAlex
W4390096766
Cited by
87
References
62
Citations by year
2026: 30 · 2025: 33 · 2024: 22 · 2023: 2
FWCI
20.42
Type
journal article
Language
en
Open access
Locations
Australasian Journal of Educational Technology (publishedVersion, cc-by-nc-nd)
Topics
  • Physical Sciences › Computer Science › Information Systems › Artificial Intelligence in Education
  • Social Sciences › Education › Educational Assessment and Pedagogy
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
Keywords
generative AI, authentic assessment, assessment design, Bloom's taxonomy, ChatGPT-4, Google Bard, Bing Chat, economics education, higher-order thinking skills, argumentation, citation and referencing, learning outcomes

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review