The DOI matches its Crossref record; 91 of the 91 references it cites were found in a registry or an index.
ChatGPT improves creative problem-solving performance in university students: An experimental study
Open accessAbstract
University students often employ generative artificial intelligence tools such as ChatGPT in resolution of ill-defined problem-solving tasks. However, the experimental evidence about effects of ChatGPT on complex problem-solving performance is still missing. In this preregistered experiment, the impact of ChatGPT on performance in a complex creative problem-solving task was investigated in 77 university students solving a task with ChatGPT in comparison to 68 students solving a task without it. ChatGPT use significantly improved self-efficacy for task resolution (d = 0.65) and enhanced the quality (d = 0.69), elaboration (d = 0.61), and originality (d = 0.55) of solutions. Moreover, participants with ChatGPT assistance perceived task as easier (d = 0.56) and requiring less mental effort (d = 0.58). However, use of ChatGPT did not make task resolution more interesting (d = 0.08), and the impact of ChatGPT on metacognitive monitoring accuracy was unclear. Although there were no significant differences in absolute accuracy between students solving the task with and without the assistance of ChatGPT, the absence of correlation between self-evaluation judgments and performance suggests that participants struggled to calibrate their self-evaluations when using ChatGPT. Notably, the perceived usefulness of ChatGPT appeared to inform self-evaluation judgments, resulting in higher inaccuracy. The implications for hybrid human-AI regulation (HHAIR) theory are discussed. To regulate effectively, students using AI tools should focus on valid metacognitive cues and ignore cues that could lead to inaccurate outcomes.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Our findings demonstrate that GenAI can indeed play a constructive role in enhancing students' critical thinking and problem-solving capabilities, aligning with prior research, such as that by Essien et al. [3] and Urban et al. [18].”
From The Mediating Role of Generative AI Self-Regulation on Students’ Critical Thinking and Problem-Solving (2024) · cites this paper as Urban et al. [18]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“However, more complex tasks involving deeper understanding and application might require more active engagement and critical thinking, where reliance on AI alone may not yield significant improvements in knowledge transfer (Asare et al., 2023; Soderstrom and Bjork, 2015; Urban et al., 2024).”
From Beware of metacognitive laziness: Effects of generative artificial intelligence on learning motivation, processes, and performance (2025) · cites this paper as Urban et al., 2024
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“GenAI can make a task easier and improve its visible output without necessarily improving the student's independent capability.”
From From Assistance to Substitution: Generative AI and Undergraduate Research Skills (2026) · cites this paper as Urban et al. (2024)
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Assessment design | Output quality can improve without independent skill.”
From From Assistance to Substitution: Generative AI and Undergraduate Research Skills (2026) · cites this paper as Urban et al. (2024)
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The same study also found that the perceived usefulness of ChatGPT was associated with learners' self-evaluation judgements, resulting in higher inaccuracies in judgements of learning”
From Beware of metacognitive laziness: Effects of generative artificial intelligence on learning motivation, processes, and performance (2025) · cites this paper as Urban et al. (2024)
Experimental — accuracy not yet measured
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“For example, Urban et al. (2024) found that using ChatGPT reduced learners' perceived difficulty, leading to less effort in task solving.”
From Beware of metacognitive laziness: Effects of generative artificial intelligence on learning motivation, processes, and performance (2025) · cites this paper as Urban et al. (2024)
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Across both groups, participants generally valued the prospect of AI for its capacity to structure content, clarify terminology, facilitate conceptualization and summarization, and perform rapid quantitative analyses, all consistent with reported benefits in the recent literature [30,36].”
From Exploring the Impact of AI Tools on Cognitive Skills: A Comparative Analysis (2025) · cites this paper as [36]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Working with generative AI tools, such as ChatGPT, increases productivity in complex problem-solving tasks in both educational and professional settings (Noy & Zhang, 2023;Sun & Zhou, 2024;Urban et al., 2024).”
From “ ChatGPT can make mistakes. Check important info.” Epistemic beliefs and metacognitive accuracy in students' integration of ChatGPT content into academic writing (2025) · cites this paper as Urban et al., 2024
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“This complements the observations made by Urban et al. [18], who found that students engaging with ChatGPT exhibit enhanced creative problem-solving skills compared to those who do not use the tool.”
From The Mediating Role of Generative AI Self-Regulation on Students’ Critical Thinking and Problem-Solving (2024) · cites this paper as Urban et al. [18]
The cited source says what the claim attributes to it.
In the citing paper
“Urban et al. (2024) similarly found that ChatGPT improved creative problemsolving performance while raising uncertainty about metacognitive calibration.”
From From Assistance to Substitution: Generative AI and Undergraduate Research Skills (2026) · cites this paper as Urban et al., 2024
The cited source says what the claim attributes to it.
In the citing paper
“However, participants using ChatGPT reported lower mental effort and showed uncertainty in metacognitive calibration.”
From From Assistance to Substitution: Generative AI and Undergraduate Research Skills (2026) · cites this paper as Urban et al., 2024
The cited source says what the claim attributes to it.
In the citing paper
“Urban et al. (2024) conducted a preregistered experiment with 145 university students and found that ChatGPT developed solution quality, elaboration, and originality.”
From From Assistance to Substitution: Generative AI and Undergraduate Research Skills (2026) · cites this paper as Urban et al., 2024
The cited source says what the claim attributes to it.
In the citing paper
“Our observations of the potential for misjudging performance aligns with an experimental quantitative study conducted by Urban et al. (2024).”
From How university students work on assessment tasks with generative artificial intelligence: matters of judgement (2026) · cites this paper as Urban et al. (2024)
The cited source says what the claim attributes to it.
In the citing paper
“the authors observe: 'participants who thought the task easier also overestimated their performance more' (p.11).”
From How university students work on assessment tasks with generative artificial intelligence: matters of judgement (2026) · cites this paper as Urban et al. (2024)
The cited source says what the claim attributes to it.
In the citing paper
“This comparative study of performance on a complex creative problem-solving task (with or without ChatGPT) suggests GenAI use can increase self-efficacy but may interfere with estimation of task performance, possibly leading to overestimation.”
From How university students work on assessment tasks with generative artificial intelligence: matters of judgement (2026) · cites this paper as Urban et al. (2024)
The cited source says what the claim attributes to it.
In the citing paper
“Additionally, research has shown a wide range of effects of ChatGPT on creativity, including positive effects (Urban et al., 2024), entirely adverse impacts (Niloy et al., 2024), and negligible or, if the AI assistance is of low quality, detrimental effects (Medeiros et al., 2025).”
From Human Intelligence, Creativity, and Wisdom in the Age of Generative Artificial Intelligence (2025) · cites this paper as Urban et al., 2024
The cited source says what the claim attributes to it.
In the citing paper
“Urban et al. (2024) show that better performance can coexist with uncertainty about metacognitive calibration.”
From From Assistance to Substitution: Generative AI and Undergraduate Research Skills (2026) · cites this paper as Urban et al. (2024)
The cited source says what the claim attributes to it.
In the citing paper
“One of the most important findings across the literature is the difference between improving performance on a task and improving the capability that the task is intended to teach.”
From From Assistance to Substitution: Generative AI and Undergraduate Research Skills (2026) · cites this paper as Urban et al. (2024)
The cited source says what the claim attributes to it.
In the citing paper
“Urban et al. (2024) provide a related example: students produced stronger creative solutions with ChatGPT, but metacognitive calibration did not clearly improve.”
From From Assistance to Substitution: Generative AI and Undergraduate Research Skills (2026) · cites this paper as Urban et al. (2024)
The cited source says what the claim attributes to it.
In the citing paper
“Participant 12 (P_12) similarly tailored and refined multiple rounds of AI-generated text, illustrating the iterative co-creation process advocated by Urban et al. [36], though the final solution still required further validation.”
From Exploring the Impact of AI Tools on Cognitive Skills: A Comparative Analysis (2025) · cites this paper as [36]
The cited source says what the claim attributes to it.
In the citing paper
“In the context of generative AI use, Urban et al. (2024) found that metacognitive bias in university students solving non-routine problem-solving tasks was linked to their perceived usefulness of ChatGPT rather than their actual performance.”
From “ ChatGPT can make mistakes. Check important info.” Epistemic beliefs and metacognitive accuracy in students' integration of ChatGPT content into academic writing (2025) · cites this paper as Urban et al. (2024)
The cited source says what the claim attributes to it.
In the citing paper
“For the between-group comparison, the sample size was determined a priori using G*Power 3.1.9.6, with α = 0.05, β = 0.80, and an expected effect size of Cohen's d = 0.62.This effect size was estimated based on average between-group differences reported by Urban et al. (2024).The calculation indicated that a minimum of 42 participants per group would be required.”
From “ ChatGPT can make mistakes. Check important info.” Epistemic beliefs and metacognitive accuracy in students' integration of ChatGPT content into academic writing (2025) · cites this paper as Urban et al. (2024)
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Classroom discussions about the broader impacts of AI on education, society, and future careers, invite students to engage critically with the technology's role in their lives (Alshamy et al., 2025; Sekli et al., 2024; Urban et al., 2024).”
From Six Institutional Intervention Areas to Support Ethical and Effective Student Use of Generative AI in Higher Education: A Narrative Review (2026) · cites this paper as Urban et al., 2024
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The Measurement of Observer Agreement for Categorical Data
Landis, J. R., & Koch, G. G. (1977) · Biometrics · doi:10.2307/2529310 · Cited by 81,148
Self-efficacy: Toward a unifying theory of behavioral change
Bandura, A. (1977) · Psychological Review · doi:10.1037/0033-295x.84.2.191 · Cited by 43,925
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
Dwivedi, Y. K., Kshetri, N., Hughes, L., Slade, E. L., Jeyaraj, A., Kar, A. K., Baabdullah, A. M., Koohang, A., Raghavan, V., Ahuja, M., et al. (2023) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2023.102642 · Cited by 4,584
Self-Efficacy: An Essential Motive to Learn
Zimmerman, B. J. (2000) · Contemporary Educational Psychology · doi:10.1006/ceps.1999.1016 · Cited by 4,294
CREATIVE SELF-EFFICACY: ITS POTENTIAL ANTECEDENTS AND RELATIONSHIP TO CREATIVE PERFORMANCE.
Tierney, P., & Farmer, S. M. (2002) · Academy of Management Journal · doi:10.2307/3069429 · Cited by 2,787
Handbook of Motivation at School
Wentzel, K. R., & Wigfield, A. (2009) · Routledge eBooks · doi:10.4324/9780203879498 · Cited by 2,521
Experimental Evidence on the Productivity Effects of Generative Artificial Intelligence
Noy, S., & Zhang, W. (2023) · doi:10.2139/ssrn.4375283 · Cited by 2,157
Paas, F. G. W. C. (1992) · Journal of Educational Psychology · doi:10.1037/0022-0663.84.4.429 · Cited by 2,150
Experimental evidence on the productivity effects of generative artificial intelligence
Noy, S., & Zhang, W. (2023) · Science · doi:10.1126/science.adh2586 · Cited by 2,150
Cited by
Sorted by citations
Fan, Y., Tang, L., Le, H., Shen, K., Tan, S., Zhao, Y., Shen, Y., Li, X., & Gašević, D. (2025) · British Journal of Educational Technology · doi:10.1111/bjet.13544 · Cited by 805
Does ChatGPT enhance student learning? A systematic review and meta-analysis of experimental studies
Deng, R., Jiang, M., Yu, X., Lu, Y., & Liu, S. (2025) · Computers & Education · doi:10.1016/j.compedu.2024.105224 · Cited by 531
Wang, J., & Fan, W. (2025) · Humanities and Social Sciences Communications · doi:10.1057/s41599-025-04787-y · Cited by 322
RetractedDerakhshan, A., & Ghiasvand, F. (2024) · International Journal of Applied Linguistics · doi:10.1111/ijal.12561 · Cited by 304
Xu, X., Qiao, L., Cheng, N., Liu, H., & Zhao, W. (2025) · British Journal of Educational Technology · doi:10.1111/bjet.13599 · Cited by 316
Metadata
- Authors
- Urban, M., Děchtěrenko, F., Lukavský, J., Hrabalová, V., Svacha, F., Brom, C., & Urban, K.
- Affiliations
- Czech Academy of Sciences (CZ)
- Charles University (CZ)
- Slovak Academy of Sciences (SK)
- Corresponding author
- Urban, Marek
- ORCID
- Venue
- Computers & Education
- Publisher
- Elsevier BV
- Published
- 2024-07
- Volume
- 215
- Pages
- 105031
- ISSN
- 0360-1315 (ISSN-L)
- OpenAlex
- W4392932744
- Semantic Scholar
- df35c2ebe6f3a66f531ac4b9bf89a2377d23fe02
- Cited by
- 396
- References
- 90
- Citations by year
- 2026: 171 · 2025: 181 · 2024: 43
- FWCI
- 148.52
- Type
- journal article
- Language
- en
- Open access
- green
- doi.org
- Licence
- Locations
- Computers & Education (publishedVersion)
- PsyArXiv (OSF Preprints) (submittedVersion, cc-by)
- PsyArXiv (OSF Preprints) (submittedVersion)
- Funders
- Vedecká grantová agentúra MŠVVaŠ SR a SAV (2/0026/21)
- Akademie Věd České Republiky (PSU-480/Brno/2023)
- Grantová Agentura České Republiky (GACR 24-11515S)
- Topics
- Social Sciences › Education › Artificial Intelligence in Education
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Creativity in Education and Neuroscience
- Social Sciences › Psychology › Developmental and Educational Psychology › Innovative Teaching and Learning Methods
- Keywords
- ChatGPT, generative AI, creative problem solving, university students, self-efficacy, metacognitive monitoring, self-assessment, perceived usefulness, originality, perceived difficulty, AI-assisted problem solving
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
