Cited by 491Artificial Intelligence in EducationCreativity in Education and NeuroscienceInnovative Teaching and Learning Methods

ChatGPT improves creative problem-solving performance in university students: An experimental study

Open access
Urban, Marek, Děchtěrenko, Filip, Lukavský, Jiří, Hrabalová, Veronika, Svacha, Filip, Brom, Cyril, Urban, Kamila·2024·Computers & Education

Abstract

University students often employ generative artificial intelligence tools such as ChatGPT in resolution of ill-defined problem-solving tasks. However, the experimental evidence about effects of ChatGPT on complex problem-solving performance is still missing. In this preregistered experiment, the impact of ChatGPT on performance in a complex creative problem-solving task was investigated in 77 university students solving a task with ChatGPT in comparison to 68 students solving a task without it. ChatGPT use significantly improved self-efficacy for task resolution (d = 0.65) and enhanced the quality (d = 0.69), elaboration (d = 0.61), and originality (d = 0.55) of solutions. Moreover, participants with ChatGPT assistance perceived task as easier (d = 0.56) and requiring less mental effort (d = 0.58). However, use of ChatGPT did not make task resolution more interesting (d = 0.08), and the impact of ChatGPT on metacognitive monitoring accuracy was unclear. Although there were no significant differences in absolute accuracy between students solving the task with and without the assistance of ChatGPT, the absence of correlation between self-evaluation judgments and performance suggests that participants struggled to calibrate their self-evaluations when using ChatGPT. Notably, the perceived usefulness of ChatGPT appeared to inform self-evaluation judgments, resulting in higher inaccuracy. The implications for hybrid human-AI regulation (HHAIR) theory are discussed. To regulate effectively, students using AI tools should focus on valid metacognitive cues and ignore cues that could lead to inaccurate outcomes.

Integrity signals

Registry

Crossref · 10.1016/j.compedu.2024.105031

Cites

89 of 89 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Cites

Flagged first, then by citations

1 of the 90 references couldn’t be matched to a known paper.

Cited by

Sorted by citations

Authors
Urban, M., Děchtěrenko, F., Lukavský, J., Hrabalová, V., Svacha, F., Brom, C., & Urban, K.
Affiliations
  • Czech Academy of Sciences (CZ)
  • Charles University (CZ)
  • Slovak Academy of Sciences (SK)
Corresponding author
Urban, Marek
Venue
Computers & Education
Publisher
Elsevier BV
Published
2024-07
Volume
215
Pages
105031
ISSN
0360-1315 (ISSN-L)
OpenAlex
W4392932744
Cited by
396
References
90
Citations by year
2026: 171 · 2025: 181 · 2024: 43 · 2023: 1
FWCI
148.52
Type
journal article
Language
en
Open access
Locations
  • Computers & Education (publishedVersion)
  • PsyArXiv (OSF Preprints) (submittedVersion, cc-by)
  • PsyArXiv (OSF Preprints) (submittedVersion)
Funders
  • Vedecká grantová agentúra MŠVVaŠ SR a SAV (2/0026/21)
  • Akademie Věd České Republiky (PSU-480/Brno/2023)
  • Grantová Agentura České Republiky (GACR 24-11515S)
Topics
  • Physical Sciences › Computer Science › Information Systems › Artificial Intelligence in Education
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Creativity in Education and Neuroscience
  • Social Sciences › Psychology › Developmental and Educational Psychology › Innovative Teaching and Learning Methods
Keywords
ChatGPT, generative AI, creative problem solving, university students, self-efficacy, metacognitive monitoring, self-assessment, perceived usefulness, originality, perceived difficulty, AI-assisted problem solving

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review