Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 48 of the 48 references it cites were found in a registry or an index; 12 of the 31 sources its claims cite were checked for support.

Exploring the Impact of AI Tools on Cognitive Skills: A Comparative Analysis

Open access

Abstract

This study evaluates the impact of Generative AI (Artificial Intelligence) algorithms on human decision making in complex problem-solving tasks. Rather than assessing the algorithms in isolation, we focus on how their use shapes three critical cognitive components of decision making: analytical thinking, creative thinking, and systems thinking. In an experimental setting, student participants were randomly assigned to solve management consulting cases either with or without access to an AI algorithm. Their solutions were evaluated using a structured rubric that captures sub-skills across the three cognitive dimensions. The results of this exploratory study reveal that AI-supported participants achieved stronger outcomes in logical reasoning, structuring, and problem definition, but showed weaknesses in novel idea generation, multidisciplinary integration, and critical rejection of unsupported conclusions. These findings highlight how algorithmic support influences the configuration of human cognitive processes in decision making.

Overview

CoChat Summary

TL;DR

This exploratory experiment with 16 master’s/doctoral students and recent graduates found that ChatGPT (GPT-4) support improved some analytical and systems-thinking sub-skills but did not uniformly improve creativity or critical evaluation. Benefits depended on collaborative use; copy-pasting or minimal engagement often yielded weak or inconsistent outcomes.

Key findings

  • AI-supported participants achieved stronger outcomes in logical reasoning, structuring, and problem definition, but showed weaknesses in novel idea generation, multidisciplinary integration, and critical rejection of unsupported conclusions. In Task 1, GPT users scored higher in Logical Thinking (+29%), Decision Making (+13%), Alternative Approach (+25%), and Hypothesis Formulation (+13%), while lower in Argument Evaluation (−25%) and Questioning (−20%); in Task 2, Argument Evaluation was +25% for GPT users but Alternative Approach and Hypothesis Formulation were −21% and −50%.
  • Systems thinking: Task 1 GPT users outperformed non-users in Holistically and Parts (+50%), How Concepts Work Together (+25%), and Identify Patterns Over Time (+21%); Task 2 showed lower/no differences for these, though Systems Boundaries Definition was +38% for GPT users. Creative thinking: Task 2 Problem and Goals Definition and Representation was approximately 60% higher for GPT users; See the Big Picture was +38% in Task 1 but −13% in Task 2; Novelty +13%, Appropriateness −13%; average creativity difference approximately 7%.
  • The most frequent configuration was negative on all three skills (0-0-0), with five participants (two GPT users, three non-users). Collaborative GPT use often correlated with stronger sub-skills; copy-pasting tended lower, e.g., P_09 71% mean score vs P_01 26% and P_08 31%. GPT users finished 15–20 min earlier on average; post-survey rated AI usefulness 60–65%, with Creative Thinking slightly lower.

Methods

  • Experimental design: 16 participants (master’s/doctoral students and recent graduates in business administration/IT at a Finnish university; average age 31.6; 8 female/8 male) randomly assigned to solve one of two management consulting cases with ChatGPT (GPT-4) or without AI, up to 1.5 h; screen recordings and pre/post surveys collected.
  • Rubric: three cognitive skills—analytical thinking (11 sub-skills), creative thinking (8), systems thinking (10)—each sub-skill scored 1–3 by three co-authors, majority vote/consensus. Mixed-methods analysis: descriptive group-level statistics, individual sub-skill review, screen-recording behavioral analysis, notes, and post-experiment survey.

Limitations

  • Stated: definitions and measurement criteria should be refined; some sub-skills (e.g., Analogical Thinking, Mental Modeling) may need different assessment methods; small sample limits generalizability and robustness of statistical comparisons.
  • Evident: sample of 16 is exploratory and not statistically generalizable; results varied by task and were not consistent across both tasks; one GPT-user participant who opted not to use the tool was excluded from calculations; some sub-skill cells were empty when evaluators did not agree, and rubric scoring is subjective.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.3390/a18100631

Cites

48 of 48 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

11 Valid · 1 Not valid · 14 Not checked · 143 Own results and interpretations

93 more claims

Create a free account to see all 169 claims and to verify the 1 reference still open.

See all claims — free account
  • Probably not a substantial claim. 34
  • It may state two separate claims as one. 44

Create a free account to see these claims.

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

1 of 1

Citations

Cites

Research works printed but not linked: 2

  1. Sternberg, R.J. (1994). Creativity. Thinking and Problem Solving, Elsevier. [2nd ed.].
  2. Cosentino, M. (2013). Case in Point: Complete Case Interview Preparation, Burgee Press.

Cited by

See all 10

Metadata

Authors
Musazade, N., Mezei, J., & Wang, X.
Affiliations
Åbo Akademi University (FI)
Corresponding author
Mezei, József
Venue
Algorithms
Publisher
MDPI AG
Published
2025-10-07
Volume
18
Issue
10
Pages
631
ISSN
1999-4893 (ISSN-L)
OpenAlex
W4414917618
Cited by
10
References
38
Citations by year
2026: 9 · 2025: 1
FWCI
2.57
Type
journal article
Language
en
Open access
Locations
  • Algorithms (publishedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • Åbo Akademi University Research Portal (submittedVersion, cc-by)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
  • Social Sciences › Education › Artificial Intelligence in Education
  • Social Sciences › Decision Sciences › Management Science and Operations Research › Complex Systems and Decision Making
Keywords
generative AI, human-AI collaboration, decision-making, cognitive skills, analytical thinking, creative thinking, systems thinking, problem solving, logical reasoning, idea generation, multidisciplinary integration, critical evaluation of AI outputs

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-11, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-11 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol and qwen3.8-omni-flash).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.