CoChat
Sign Up
Cited by 0Student Assessment and FeedbackArtificial Intelligence in EducationProblem and Project Based Learning

AUTHENTIC ASSESSMENT AND CRITICAL THINKING IN UNIVERSITY STUDENTS IN THE AGE OF GENERATIVE ARTIFICIAL INTELLIGENCE: AN EXPLANATORY MODEL

Open access
Patricia del Carmen Arellano Gualle, Luis Alberto Flores Rodriguez, Jorge Armando Acosta Villarraga, Alex Segundo García Crisanto, Luis Angel Vega Palomino, Alexander Pacaya Taminche·2026·Acta Innovations
DOI 10.62441/actainnovations.vi.2132
Open this paper in CoChat — free accountOpen PDF

Abstract

The advent of Generative Artificial Intelligence (GenAI) is disrupting traditional thinking about assessment in universities, as it is possible that final written products will not be the only indicator of students' independent thinking. This study investigated the relationship between participation in an authentic assessment process and the use of information problem solving with a critical thinking approach in a task carried out in the university by using the GenAI. The publicly available GenAI-Assisted Information Problem Solving dataset was used to extract secondary data for analysis, which included GenAI dialogue, writing-process activity, annotations, student background characteristics and tutor-assigned proposal scores [6]. There were 230 complete cases used in the multivariable regression and moderation analysis. The relationship between authentic-assessment engagement and critical-thinking-oriented GenAI dialogue was positive (β = 0.174, p = 0.004), and the relationship between evaluative GenAI engagement and authentic-assessment engagement was positive, but stronger (β = 0.379, p < 0.001). Authentic-assessment engagement and evaluative GenAI engagement, however, did not have a statistically significant interaction (β = −0.102, p = 0.110). Process indicators also failed to significantly predict final proposal scores, with 3.8% of the variance in the model of proposal performance explained by the process indicators. The results indicate that authentic assessment can help to set the stage for more evaluative and revision-driven interactions with GenAI, and more process-level critical thinking does not necessarily lead to higher assessment scores. The study underscores the need to separate out the amount of AI use from the quality of the AI use and to include process evidence in the design of assessments in the GenAI era.

AI snapshot

Written from the paper’s full text.

TL;DR

Using secondary data from the publicly available GenAI-Assisted Information Problem Solving dataset, this study tests an explanatory model linking university students' engagement in an authentic assessment process to critical-thinking-oriented dialogue with generative AI. Authentic-assessment engagement and evaluative GenAI engagement both predicted critical-thinking-oriented dialogue, but neither predicted final tutor-assigned proposal scores.

Key findings

  • Authentic-assessment engagement was positively related to critical-thinking-oriented GenAI dialogue (β = 0.174, 95% CI [0.057, 0.292], p = 0.004), supporting H1.
  • Evaluative GenAI engagement showed a stronger positive association with critical-thinking-oriented dialogue (β = 0.379, 95% CI [0.258, 0.501], p < 0.001), supporting H2; the full model was significant, F(6, 223) = 9.89, p < 0.001, explaining 20.9% of variance (adjusted R² = 0.188).
  • The hypothesized moderation of authentic-assessment engagement by evaluative GenAI engagement was not significant (β = -0.102, 95% CI [-0.227, 0.023], p = 0.110), so H3 was not supported.
  • Process indicators did not significantly predict final proposal scores: the model explained 3.8% of variance (adjusted R² = 0.008), F(7, 222) ≈ 1.94, p = 0.064, with no significant process predictors (H4 not supported); age was the only significant covariate (β = 0.267, p = 0.005).

Methods

  • Quantitative explanatory design using secondary data from the GenAI-Assisted Information Problem Solving dataset (Li et al.), in which university students completed a data-science project-proposal task in the FLoRA learning environment with an embedded GenAI chatbot.
  • 230 complete cases were used for multivariable analysis (the dataset documentation reports 279 students with records; 275 proposal/score records). Mean age 24.87 years (SD = 3.50, range 17–52); 63.6% male, 31.3% female; 61.5% first-year; 78.5% reported prior GenAI knowledge and 25.1% prior data-science knowledge.
  • Hierarchical multiple regression predicting a critical-thinking dialogue index, controlling for prior data-science knowledge, prior GenAI knowledge, GenAI familiarity, and age; a moderation model added the authentic-assessment × evaluative GenAI interaction; a secondary regression examined final proposal scores. Two-tailed significance at p < 0.05.

Limitations

  • Stated by the paper: the secondary observational design cannot establish causality; the project-proposal task was not experimentally manipulated and all participants completed the same task; the critical-thinking outcome is a process-based dialogue indicator (judgment- and revision-oriented interactions) rather than a validated psychometric critical-thinking scale.
  • Stated as future-research needs: validated CT assessments, rubric-based process assessments, experimental and/or longitudinal designs, behavioral trace data, and investigation of explicit AI literacy instruction.
  • Evident from the text: the sample is predominantly male and early-stage, drawn from a single task and a single dataset, which limits generalizability; the paper also reports inconsistent record counts (279 in dataset documentation vs. 275 proposal/score records vs. 230 complete cases), and the proposal-score model was underpowered/non-significant, so its null result should be read cautiously.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.62441/actainnovations.vi.2132

Cites

0 citation errors · 0 not checkable

Statistics

Claims the paper makes

0 of 57 cited sources checked · 0 supported · 0 flagged

  • Findingown data

    The relationship between authentic-assessment engagement and critical-thinking-oriented GenAI dialogue was positive (β = 0.174, p = 0.004).

    “The relationship between authentic-assessment engagement and critical-thinking-oriented GenAI dialogue was positive (β = 0.174, p = 0.004)”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The FLoRA dataset's process traces (writing activity, source/annotation activity, GenAI dialogue) validly operationalize authentic-assessment engagement and critical-thinking-oriented information problem solving.
    • Rests on an assumption: The percentage of judgment-and revision-oriented interactions over total GenAI interactions validly measures evaluative GenAI engagement.
    • Rests on an assumption: Tutor-assigned proposal scores validly measure final project-proposal assessment performance.
    • Rests on an assumption: The GenAI dialogue annotations reliably distinguish judgment-and revision-oriented interactions from other interactions.
    • Rests on an assumption: The 230 complete cases are representative of the full dataset and missing data do not bias the estimated relationships.
  • Findingown data

    3.8% of the variance in the model of proposal performance was explained by the process indicators.

    “with 3.8% of the variance in the model of proposal performance explained by the process indicators.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The FLoRA dataset's process traces (writing activity, source/annotation activity, GenAI dialogue) validly operationalize authentic-assessment engagement and critical-thinking-oriented information problem solving.
    • Rests on an assumption: The percentage of judgment-and revision-oriented interactions over total GenAI interactions validly measures evaluative GenAI engagement.
    • Rests on an assumption: Tutor-assigned proposal scores validly measure final project-proposal assessment performance.
    • Rests on an assumption: The GenAI dialogue annotations reliably distinguish judgment-and revision-oriented interactions from other interactions.
    • Rests on an assumption: The 230 complete cases are representative of the full dataset and missing data do not bias the estimated relationships.
  • Findingown data

    Process indicators did not significantly predict final proposal scores.

    “Process indicators also failed to significantly predict final proposal scores”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The FLoRA dataset's process traces (writing activity, source/annotation activity, GenAI dialogue) validly operationalize authentic-assessment engagement and critical-thinking-oriented information problem solving.
    • Rests on an assumption: The percentage of judgment-and revision-oriented interactions over total GenAI interactions validly measures evaluative GenAI engagement.
    • Rests on an assumption: Tutor-assigned proposal scores validly measure final project-proposal assessment performance.
    • Rests on an assumption: The GenAI dialogue annotations reliably distinguish judgment-and revision-oriented interactions from other interactions.
    • Rests on an assumption: The 230 complete cases are representative of the full dataset and missing data do not bias the estimated relationships.
  • Findingown data

    Authentic-assessment engagement and evaluative GenAI engagement did not have a statistically significant interaction (β = -0.102, p = 0.110).

    “Authentic-assessment engagement and evaluative GenAI engagement, however, did not have a statistically significant interaction (β = -0.102,p = 0.110).”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The FLoRA dataset's process traces (writing activity, source/annotation activity, GenAI dialogue) validly operationalize authentic-assessment engagement and critical-thinking-oriented information problem solving.
    • Rests on an assumption: The percentage of judgment-and revision-oriented interactions over total GenAI interactions validly measures evaluative GenAI engagement.
    • Rests on an assumption: Tutor-assigned proposal scores validly measure final project-proposal assessment performance.
    • Rests on an assumption: The GenAI dialogue annotations reliably distinguish judgment-and revision-oriented interactions from other interactions.
    • Rests on an assumption: The 230 complete cases are representative of the full dataset and missing data do not bias the estimated relationships.
  • Findingown data

    The critical-thinking dialogue index was positively related to age (β = 0.160, p = 0.016).

    “The critical-thinking dialogue index was also positively related to age (β = 0.160, p = 0.016).”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The process-based critical-thinking dialogue indicator validly measures critical thinking.
  • May combine two claims. 19
  • Classified as probably not a substantial claim. 26

  • May combine two claims.

    Findingown data

    The relationship between evaluative GenAI engagement and authentic-assessment engagement was positive and stronger (β = 0.379, p < 0.001).

    “the relationship between evaluative GenAI engagement and authentic-assessment engagement was positive, but stronger (β = 0.379, p < 0.001)”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The FLoRA dataset's process traces (writing activity, source/annotation activity, GenAI dialogue) validly operationalize authentic-assessment engagement and critical-thinking-oriented information problem solving.
    • Rests on an assumption: The percentage of judgment-and revision-oriented interactions over total GenAI interactions validly measures evaluative GenAI engagement.
    • Rests on an assumption: Tutor-assigned proposal scores validly measure final project-proposal assessment performance.
    • Rests on an assumption: The GenAI dialogue annotations reliably distinguish judgment-and revision-oriented interactions from other interactions.
    • Rests on an assumption: The 230 complete cases are representative of the full dataset and missing data do not bias the estimated relationships.

  • May combine two claims.

    Findingown data

    The secondary regression model predicting final proposal scores explained 3.8% of the variance (adjusted R2 = 0.008) and did not reach the conventional 0.05 significance threshold (F(7, 222) ≈ 1.94, p = 0.064).

    “The model explained 3.8% of the variance (adjusted R² = 0.008) and did not reach the conventional 0.05 significance threshold, F(7, 222) ≈ 1.94, p = 0.064.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The final proposal score validly measures final assessment performance.

  • May combine two claims.

    Findingown data

    The multivariable model predicting the critical-thinking dialogue index was statistically significant (F(6, 223) = 9.89, p < 0.001) and accounted for 20.9% of the variance (adjusted R2 = 0.188).

    “The multivariable model predicting the critical-thinking dialogue index was statistically significant (F(6, 223) = 9.89, p < 0.001) and accounted for 20.9% of the variance (adjusted R² = 0.188).”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The process-based critical-thinking dialogue indicator validly measures critical thinking.

  • May combine two claims.

    Findingown data

    Prior data-science knowledge, prior GenAI knowledge, and GenAI familiarity were not statistically significant predictors of critical-thinking dialogue.

    “There was no evidence of prior data-science knowledge being statistically significant (all p > 0.26), nor was there for prior GenAI knowledge (all p > 0.34), or GenAI familiarity (all p > 0.43).”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The process-based critical-thinking dialogue indicator validly measures critical thinking.

  • May combine two claims.

    Findingown data

    The interaction model accounted for 21.9% of the variance in critical-thinking-oriented dialogue (adjusted R2 = 0.194), whereas the main-effects model accounted for 20.9% of the variance.

    “The interaction model accounted for 21.9% of the variance in critical-thinking-oriented dialogue (adjusted R² = 0.194), whereas the main-effects model accounted for 20.9% of the variance.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The process-based critical-thinking dialogue indicator validly measures critical thinking.

  • May combine two claims.

    Findingown data

    Authentic-assessment engagement, critical-thinking dialogue, and evaluative GenAI engagement were not significant predictors of final proposal score.

    “Authentic-assessment engagement (β = -0.159,p = 0.214), critical-thinking dialogue (β = 0.129, p = 0.375), and evaluative GenAI engagement (β = 0.151, p = 0.312) were not significant predictors.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The process-based critical-thinking dialogue indicator validly measures critical thinking.
    • Rests on an assumption: The authentic-assessment engagement measure validly captures authentic assessment engagement.
    • Rests on an assumption: The evaluative GenAI engagement measure validly captures evaluative GenAI engagement.
    • Rests on an assumption: The final proposal score validly measures final assessment performance.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The process-based approach aligns with recent studies that differentiate between AI exposure and engagement, as well as exposure with frequency.

    “This process-based approach aligns with recent studies that differentiate between AI exposure and engagement, as well as exposure with frequency [11,13].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [11]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [13]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S4 (based).

    Premisenone given

    The overall assessment score may not adequately reflect differences in reasoning processes.

    “Furthermore, the overall assessment score may not adequately reflect differences in reasoning processes.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Premisecitation

    Authenticity relates to whether students evaluate information, make judgments, synthesize and revise the information in the process of completing the task, not only to a realistic task format.

    “Authenticity should not only be seen as a realistic task format, but also relates to whether students evaluate information, make judgments, synthesize and revise the information in the process of completing the task [2,3].”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [2]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [3]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Premisecitation

    The results align with the literature, which shows that authentic assessment can promote higher-order skills and that the use of GenAI is more likely to enhance critical thinking when learners are expected to assess, reason, and edit the generated content.

    “The results align with the literature, which shows that authentic assessment can promote higher-order skills [1,3] and that the use of GenAI is more likely to enhance critical thinking when learners are expected to assess, reason, and edit the generated content [9,10].”
    Quote in the paperNot a substantial claim0 of 4 sources checked
    • cites [1]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [3]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [9]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [10]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The evidence that is available is in a diverse format, for different tasks, AI tools, and critical-thinking measures.

    “These constraints are significant as the evidence that is available is in a diverse format, for different tasks, AI tools, and critical-thinking measures [12,19].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [12]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [19]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

    Premisecitation

    Authenticity needs to be coupled with cognitive challenge, meaningful contextualization, and proper assessment criteria.

    “Authenticity needs to be coupled with cognitive challenge, meaningful contextualization, and proper assessment criteria [2,3].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [2]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [3]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Interpretationown data

    Evaluative GenAI engagement was significantly higher, indicating that cognitive nature of AI use might matter more than just interactions.

    “Evaluative GenAI engagement was significantly higher, indicating that cognitive nature of AI use might matter more than just interactions.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The process-based critical-thinking dialogue indicator validly measures critical thinking.
    • Rests on an assumption: The evaluative GenAI engagement measure validly captures evaluative GenAI engagement.

  • May combine two claims.

    Premisecitation

    Students' interactions with ChatGPT can be used to facilitate deep critical thinking when the interaction is active and reflective, but can also lead to cognitive offloading if the interaction is not critical.

    “Empirical research in recent years has demonstrated that students' interactions with ChatGPT can be used to facilitate deep critical thinking when the interaction is active and reflective, but can also lead to cognitive offloading if the interaction is not critical [11,12].”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [11]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [12]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Premisecitation

    Recent systematic reviews demonstrate that structured, inquiry-oriented and scaffolded use of GenAI can facilitate higher-order thinking, while unstructured use can facilitate cognitive offloading.

    “This interpretation is consistent with recent systematic reviews which demonstrate that structured, inquiry-oriented and scaffolded use of GenAI can facilitate higher-order thinking, while unstructured use can facilitate cognitive offloading [9,10].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [9]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [10]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Recent findings indicate that the cognitive quality of human-AI interactions might be more significant than the quantity of usage.

    “This is in line with recent findings that the cognitive quality of human-AI interactions might be more significant than the quantity of usage [11,19].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [11]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [19]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    230 complete cases were used for the current multivariable analysis.

    “230 complete cases were used for the current multivariable analysis.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The 230 complete cases are representative of the full sample and missing data are missing at random.

  • Probably not a substantial claim: may not pass S2 (checkable).

    Premisecitation

    The assessment process should allow for the observation of higher order reasoning and responsible use of AI during the assessment process.

    “This operationalization is in line with recent arguments that the assessment process should allow for the observation of higher order reasoning and responsible use of AI during the assessment process [15,17].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [15]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [17]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Another systematic review found ongoing conceptual and methodological confusion in the GenAI-critical-thinking literature.

    “Another systematic review found ongoing conceptual and methodological confusion in the GenAI-critical-thinking literature [9].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [9]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Premisecitation

    The use of authentic assessment does not guarantee academic integrity or reliable detection of the use of GenAI.

    “The use of authentic assessment does not guarantee academic integrity or reliable detection of the use of GenAI, as Kofinas et al. [5] demonstrated.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [5]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given

    For continuous variables, missingness, outliers, skewness, and distributional assumptions were checked.

    “For continuous variables, missingness, outliers, skewness, and distributional assumptions were checked.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationown data

    One area that seems to have a special significance is the cognitive quality of GenAI engagement.

    “One area that seems to have a special significance is the cognitive quality of GenAI engagement.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The process-based critical-thinking dialogue indicator validly measures critical thinking.
    • Rests on an assumption: The evaluative GenAI engagement measure validly captures evaluative GenAI engagement.

  • Probably not a substantial claim: may not pass S4 (based).

    Premisenone given

    These streams have not yet been integrated to a degree that is demonstrated by behavior.

    “But these streams have not yet been integrated to a degree that is demonstrated by behavior.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Premisenone given

    Less is known about the association between engagement in an authentic assessment process and critical-thinking oriented GenAI interaction, and whether indicators of the assessment process are related to the final assessment performance.

    “Less is known, in particular, about the association between engagement in an authentic assessment process and critical-thinking oriented GenAI interaction, and whether indicators of the assessment process are related to the final assessment performance.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data

    The weak proposal-score model is important; only 3.8% of the variance in final scores was accounted for by the process indicators and none of the process variables in the focus was significant in predicting final score.

    “Theoretically, the weak proposal-score model is important.Only 3.8% of the variance in final scores was accounted for by the process indicators and none of the process variables in the focus was significant in predicting final score.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The process-based critical-thinking dialogue indicator validly measures critical thinking.
    • Rests on an assumption: The authentic-assessment engagement measure validly captures authentic assessment engagement.
    • Rests on an assumption: The evaluative GenAI engagement measure validly captures evaluative GenAI engagement.
    • Rests on an assumption: The final proposal score validly measures final assessment performance.

  • Classified as probably not a substantial claim.

    Premisecitation

    The study employs the GenAI-Assisted Information Problem Solving dataset released by Li et al. which is publicly available.

    “The study employs the GenAI-Assisted Information Problem Solving dataset released by Li et al. [6] which is publicly available.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [6]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Classified as probably not a substantial claim.

    Premisecitation

    The final proposal score is based on a number of factors such as: goal clarity, topic appropriateness, business benefits and challenges, novelty/creativity, and report clarity.

    “The final proposal score is based on a number of factors such as: goal clarity, topic appropriateness, business benefits and challenges, novelty/creativity, and report clarity [6].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [6]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Classified as probably not a substantial claim.

    Premisecitation

    Systematic reviews reveal an urgent need for empirical research that links the use of AI to the design of the assessment tool to the critical-thinking process, instead of studying each of these areas separately.

    “In recent times, systematic reviews also reveal an urgent need for empirical research that links the use of AI to the design of the assessment tool to the critical-thinking process, instead of studying each of these areas separately [19,20].”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [19]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [20]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationown data

    The results further support the new paradigm of scaffolding, active involvement and explicit development of evaluative judgment in order to achieve effective integration of GenAI.

    “They further support the new paradigm of scaffolding, active involvement and explicit development of evaluative judgment in order to achieve effective integration of GenAI.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The process-based critical-thinking dialogue indicator validly measures critical thinking.
    • Rests on an assumption: The authentic-assessment engagement measure validly captures authentic assessment engagement.
    • Rests on an assumption: The evaluative GenAI engagement measure validly captures evaluative GenAI engagement.

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Premisecitation

    AI-supported assessment is a human-AI activity where the role of evaluative judgment and student agency is still central.

    “The framework thus fits in with the current recommendations to see AI-supported assessment as a human-AI activity where the role of evaluative judgment and student agency is still central [15,20].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [15]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [20]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductnone given

    The task of producing a project-proposal was not experimentally manipulated, and all participants were asked to complete the same task.

    “Second, the task of producing a project-proposal was not experimentally manipulated, and all participants were asked to complete the same task.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisecitation

    Research indicates that engaging with ChatGPT can enhance complex critical thinking, whereas output-focused interactions might offer limited cognitive benefits.

    “The findings align with research indicating that engaging with ChatGPT can enhance complex critical thinking, whereas output-focused interactions might offer limited cognitive benefits [11,12].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [11]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [12]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    The secondary observational design cannot determine causality.

    “The first is that the secondary observational design can't determine causality.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    The critical-thinking outcome is not a psychometric critical-thinking scale, but rather a process-based dialogue indicator, based on judgment-and revision-oriented interactions.

    “Third, the critical-thinking outcome is not a psychometric critical-thinking scale, but rather a process-based dialogue indicator, based on judgment-and revision-oriented interactions.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisecitation

    Judgment-oriented questions, revision activity, information evaluation, and synthesis are observable indicators relevant to critical thinking.

    “Judgment-oriented questions, revision activity, information evaluation, and synthesis are indicators that can be observed in student activities that are relevant to critical thinking [6,9].”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [6]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [9]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Premisenone given

    The literature shows significant advances in three somewhat interrelated areas: authentic assessment, critical thinking development, and transformation of assessment in the context of GenAI.

    “The literature shows significant advances in three somewhat interrelated areas: authentic assessment, critical thinking development, and transformation of assessment in the context of GenAI.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Interpretationcitation

    The discovery further supports the arguments for including process evidence in assessment, in addition to the final product, in the GenAI era.

    “The discovery further supports the arguments for including process evidence in assessment, in addition to the final product, in the GenAI era [4,5].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [4]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [5]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • Rests on an assumption: The process-based critical-thinking dialogue indicator validly measures critical thinking.
    • Rests on an assumption: The authentic-assessment engagement measure validly captures authentic assessment engagement.
    • Rests on an assumption: The evaluative GenAI engagement measure validly captures evaluative GenAI engagement.
    • Rests on an assumption: The final proposal score validly measures final assessment performance.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Critical thinking, problem solving, and collaboration are competencies associated with authentic assessment.

    “critical thinking, problem solving, and collaboration were identified as some of the competencies associated with authentic assessment by Vlachopoulos and Makri [3].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [3]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    The study points to the need for a paradigm shift in assessment that will make student thinking explicit without compromising the use of AI responsibly and the retention of human intellectual property.

    “In conclusion, the study points to the need for a paradigm shift in assessment that will make student thinking explicit without compromising the use of AI responsibly and the retention of human intellectual property.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The process-based critical-thinking dialogue indicator validly measures critical thinking.
    • Rests on an assumption: The authentic-assessment engagement measure validly captures authentic assessment engagement.
    • Rests on an assumption: The evaluative GenAI engagement measure validly captures evaluative GenAI engagement.
    • Rests on an assumption: The final proposal score validly measures final assessment performance.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The published documentation of the dataset provides the final number of students (279) who have records of interaction, annotations, dialogue, writing, proposal, and assessment.

    “The published documentation of the dataset provides the final number of students (279) who have records of interaction, annotations, dialogue, writing, proposal, and assessment [6].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [6]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

40 of 40

  • The paper relies on this without support.

    The FLoRA dataset's process traces (writing activity, source/annotation activity, GenAI dialogue) validly operationalize authentic-assessment engagement and critical-thinking-oriented information problem solving.

    “Authentic assessment was realized in the process of attending to the contextualized project-proposal process, which involved writing activity, source/annotation activity, problem formulation, synthesis and iterative development.”
    “Critical thinking was manifested in observable information-problem-solving activity, including the GenAI interactions of judgment-and revision-oriented.”

  • The paper relies on this without support.

    The percentage of judgment-and revision-oriented interactions over total GenAI interactions validly measures evaluative GenAI engagement.

    “Evaluative GenAI engagement was defined as the percentage of judgment-and revision-oriented interactions over the total number of GenAI interactions.”

  • The paper relies on this without support.

    Tutor-assigned proposal scores validly measure final project-proposal assessment performance.

    “The data set combines GenAI dialogue, system activity logs, writing and annotation traces, project proposals from the students, scores awarded by the tutors and student background data [6].”

  • The paper relies on this without support.

    The GenAI dialogue annotations reliably distinguish judgment-and revision-oriented interactions from other interactions.

    “Critical thinking was manifested in observable information-problem-solving activity, including the GenAI interactions of judgment-and revision-oriented.”

  • The paper relies on this without support.

    The 230 complete cases are representative of the full dataset and missing data do not bias the estimated relationships.

    “There were 230 complete cases used in the multivariable regression and moderation analysis.”

  • The paper relies on this without support.

    The project-proposal task constitutes a real authentic assessment context.

    “The conceptual model views the project-proposal activity as a real assessment context”

  • The paper relies on this without support.

    The process-based critical-thinking dialogue indicator validly measures critical thinking.

    “Third, the critical-thinking outcome is not a psychometric critical-thinking scale, but rather a process-based dialogue indicator, based on judgment-and revision-oriented interactions.”

  • The paper relies on this without support.

    The authentic-assessment engagement measure validly captures authentic assessment engagement.

    “Using hierarchical multiple regression, the independent contribution of authentic-assessment engagement and evaluative GenAI engagement to criticalthinking oriented dialogue was estimated while controlling for prior data-science knowledge, prior GenAI knowledge, GenAI familiarity, and age.”

  • The paper relies on this without support.

    The evaluative GenAI engagement measure validly captures evaluative GenAI engagement.

    “Using hierarchical multiple regression, the independent contribution of authentic-assessment engagement and evaluative GenAI engagement to criticalthinking oriented dialogue was estimated while controlling for prior data-science knowledge, prior GenAI knowledge, GenAI familiarity, and age.”

  • The paper relies on this without support.

    The final proposal score validly measures final assessment performance.

    “The final proposal score is based on a number of factors such as: goal clarity, topic appropriateness, business benefits and challenges, novelty/creativity, and report clarity [6].”

  • The paper relies on this without support.

    The 230 complete cases are representative of the full sample and missing data are missing at random.

    “230 complete cases were used for the current multivariable analysis.”

  • The paper relies on this without support.

    The statistical models are correctly specified and the associations are not due to unmeasured confounding.

    “Using hierarchical multiple regression, the independent contribution of authentic-assessment engagement and evaluative GenAI engagement to criticalthinking oriented dialogue was estimated while controlling for prior data-science knowledge, prior GenAI knowledge, GenAI familiarity, and age.”
12 of 12

Statistics recomputed

Every reported test result, recomputed from its test statistic and degrees of freedom. A result shows as not adding up only once an independent check has confirmed it.

  • ConsistentF(6, 223) = 9.89reported p < .001 · recomputed p < .001

    “F(6, 223) = 9.89, p < .001”

1 of 1

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

  • Sex categories do not sum to the reported total

    Minor

    The listed counts for men, women, and other/undisclosed students sum to 265, not 275. Because the final category includes undisclosed students, the table leaves 10 records unaccounted for.

    Reported 275 · recomputed 265

1 of 1

Cites

The publisher hasn’t deposited this paper’s reference list.

Related

Authors
Gualle, P. D. C. A., Rodriguez, L. A. F., Villarraga, J. A. A., Crisanto, A. S. G., Palomino, L. A. V., & Taminche, A. P.
Affiliations
  • Antenor Orrego Private University (PE)
  • Escuela Superior de Guerra (CO)
  • Universidad César Vallejo (PE)
ORCID
Venue
Acta Innovations
Publisher
Rotherham Press
Published
2026-10-18
Volume
63
Issue
2s
ISSN
2300-5599 (ISSN-L)
DOI
10.62441/actainnovations.vi.2132
OpenAlex
W7223399779
Cited by
0
References
0
FWCI
0.00
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
Acta Innovations (publishedVersion, cc-by)
Topics
  • Social Sciences › Education › Student Assessment and Feedback
  • Physical Sciences › Computer Science › Information Systems › Artificial Intelligence in Education
  • Social Sciences › Education › Problem and Project Based Learning
Keywords
authentic assessment, critical thinking, generative AI, university students, information problem solving, AI-assisted writing, student engagement with AI, assessment design

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-29.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-29.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review