CoChat
Sign Up
PreprintCited by 0Artificial Intelligence in EducationInnovative Teaching and Learning MethodsIntelligent Tutoring Systems and Adaptive Learning

Think Before You Accept: Can Written Justification Reduce Uncritical Uptake of AI Writing Suggestions?

Open access
Tao, Yan, Meyer, Jennifer, Kizilcec, Rene F.·2026·arXiv
DOI 10.48550/arxiv.2609.23936arXiv 2609.23936
Open this paper in CoChat — free accountOpen PDF

Abstract

Generative AI can offer students useful feedback, but its value depends on judging which suggestions are accurate and relevant. Prior research shows that strategic friction during human-AI interactions can promote critical uptake, but how to effectively implement such friction in academic contexts remains unclear. We examine whether requiring students to justify decisions to accept or reject AI suggestions can mitigate uncritical uptake in academic writing. In a randomized experiment embedded in a course activity (N=129), students wrote a data analysis proposal, received mixed-quality AI revision suggestions, and decided whether to accept or reject them. Students required to provide written justifications were 24 percentage points less likely to adopt flawed suggestions (65% vs. 41%), with no reduction in acceptance of sound suggestions (81% vs. 86%). However, thematic analysis revealed superficial engagement in the justification task and gaps in metacognitive monitoring and domain knowledge.

AI snapshot

Written from the paper’s full text.

TL;DR

Requiring students to write justifications for accepting or rejecting AI writing suggestions reduced uncritical uptake of flawed suggestions in a randomized classroom experiment, without reducing acceptance of sound suggestions. However, justifications were often superficial, and gaps in metacognitive monitoring and domain knowledge persisted.

Key findings

  • In the primary comparison (justification n = 64 vs. selection-only n = 65), requiring written justification reduced average acceptance of flawed AI suggestions from 65% to 41% (24 percentage points), with no significant reduction in acceptance of sound suggestions (81% vs. 86%).
  • A generalized linear mixed effects model found a significant condition × suggestion-quality interaction (β = −1.713, SE = 0.351, OR = 0.180, p < .001). For flawed suggestions, the estimated odds of acceptance under justification were 0.269 times those in selection-only, corresponding to approximately 73% lower odds; for sound suggestions, the difference was not statistically significant (OR = 1.491, p = .265).
  • Non-selective acceptance—accepting all eight suggestions—was 15.6% in the justification condition vs. 30.8% in selection-only and 32.3% in the batch condition. Justification differed significantly from batch (χ²(1) = 3.842, p = .050) and directionally from selection-only (χ²(1) = 3.339, p = .068).
  • Thematic analysis of 256 justifications for flawed suggestions from 64 students found frequent vague endorsement, heuristic reasoning, and echoing of the AI suggestion’s rationale. For the fabrication flaw, only 14 responses explicitly mentioned the factual error, while 18 of 24 who rejected it for contradicting or being irrelevant to their argument did not recognize the theory itself as fabricated or misrepresented.

Methods

  • The study embedded a randomized experiment in an in-class writing activity in an upper-division data science course at a U.S. university. Students wrote a short data analysis proposal, received eight GPT-4.1 revision suggestions of deliberately mixed quality (four sound, four flawed: fabrication, decontextualization, invalid inference, genre mismatch), and selected which to adopt.
  • Students were randomly assigned to justification, selection-only, or batch conditions. The primary analysis compared justification (n = 64) and selection-only (n = 65); the batch condition (n = 62) served as a secondary descriptive baseline. Analysis used chi-square tests and a mixed effects logistic regression predicting suggestion acceptance from condition, feedback quality, and their interaction, with random effects for student and suggestion.
  • Written justifications in the justification condition were analyzed using reflexive thematic analysis. A manual fidelity check found 147 of 152 suggestions (96.7%) conformed to the intended design.

Limitations

  • Stated: the design cannot separate the effect of justification from the time and effort it requires, or from possible suspicion cueing; the suggestion set was more adversarial than real-world AI feedback because half the suggestions were deliberately flawed; adoption decisions were a proximal outcome and revised draft quality was not assessed; the activity was ungraded, which may understate effects under graded conditions; findings come from a single course and writing context; and anonymous responses could not be verified as coming from distinct students.
  • Evident from the text: the thematic analysis focused only on flawed suggestions, so it does not characterize how students evaluated sound suggestions; the first author conducted the primary reflexive thematic analysis, with other authors contributing to refinement rather than independent coding consensus; and the study did not measure students’ domain knowledge before the activity, which the authors note would be needed to test whether friction mainly helps already-prepared students.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

DataCite · 10.48550/arxiv.2609.23936

Cites

1 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

20 of 52 cited sources checked · 15 supported · 0 flagged · accuracy not yet measured

  • Premisecitation

    A study of 156 Pakistani students found that 35% reported accepting AI suggestions for English academic writing without critical evaluation.

    “Similarly, a study of 156 Pakistani students found that 35% reported accepting AI suggestions for English academic writing without critical evaluation (Mujeeb et al., 2026).”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites Mujeeb et al., 2026 source

      The qualitative results (Related work; page unavailable) say that over a third of students admitted a tendency to accept AI suggestions uncritically, supporting the behavior described. But the Participants and Results sections report 400 students, not 156, and the retrieved text does not give an exact figure of 35%. (LLM, conf=0.91)

      Experimental — accuracy not yet measured
  • Interpretationcitation

    The relevant outcome for evaluating AI reliance is how well reliance tracks the quality of AI suggestions, not simply whether and how often students defer to AI.

    “Research on human-AI decision making suggests that reliance is appropriate when people accept correct AI output and reject incorrect output; thus, the relevant outcome is not simply whether and how often students defer to AI, but how well their reliance tracks the quality of AI suggestions (Raees et al., 2026;Schemmer et al., 2023;Bansal et al., 2021).”
    Quote in the paperNot a substantial claim2 of 3 sources checked
    • cites Raees et al., 2026

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Schemmer et al., 2023 source
    • cites Bansal et al., 2021 source
  • Premisecitation

    Writers who compose with an opinionated language model shift their expressed views toward the model's.

    “Writers who compose with an opinionated language model shift their expressed views toward the model's (Jakesch et al., 2023), and writers who accept substantial AI contributions report reduced ownership of the resulting text (Draxler et al., 2024).”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites Jakesch et al., 2023 source
  • Premisenone given

    Only Alshehri and colleagues have empirically examined adoption decision justification as part of an educational intervention for critical AI use in writing education.

    “To our knowledge, only Alshehri and colleagues (Alshehri et al., 2026) have empirically examined adoption decision justification as part of an educational intervention for critical AI use in writing education.”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites Alshehri et al., 2026 source
  • Interpretationcitation

    Defaulting to AI suggested revisions may constrain students' rhetorical agency by giving up control over what to say and how to say it.

    “Defaulting to AI suggested revisions may therefore constrain students' rhetorical agency by giving up control over what to say and how to say it (Tseng, 2026).”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites Tseng, 2026 source
  • May combine two claims. 40
  • Classified as probably not a substantial claim. 37
  • The quote only partly states this claim. 2

  • May combine two claims.

    Interpretationown data

    The findings echo students' self-reported experiences of uncritical AI uptake in prior survey and interview research and add behavioral evidence that students may still accept AI suggestions wholesale even when reminded that AI can make mistakes.

    “This echoes students' self-reported experiences of uncritical AI uptake in prior survey and interview research (Du & Chen, 2026;Mujeeb et al., 2026), and adds behavioral evidence that students may still accept AI suggestions wholesale even when they are reminded that AI can make mistakes.”
    Quote in the paperSubstantial claim1 of 2 sources checked
    • cites Du & Chen, 2026

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Mujeeb et al., 2026 source
    • Rests on an assumption: Accepting all eight suggestions is a valid indicator of non-selective or uncritical uptake.

  • Classified as probably not a substantial claim.

    Premisecitation

    Writers who accept substantial AI contributions report reduced ownership of the resulting text.

    “Writers who compose with an opinionated language model shift their expressed views toward the model's (Jakesch et al., 2023), and writers who accept substantial AI contributions report reduced ownership of the resulting text (Draxler et al., 2024).”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites Draxler et al., 2024 source

  • Classified as probably not a substantial claim.

    Premisecitation

    The point of contact between the writer and the model shapes how much of the writing remains the writer's own.

    “Studies of interactive writing systems note that the point of contact between the writer and the model (when suggestions appear, what the writer must do to take them, etc.) shapes how much of the writing remains the writer's own (Lee et al., 2022).”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites Lee et al., 2022 source

      Section 4.2.2 describes when suggestions appear and how writers accept them. Section 5.2.2 reports a correlation between the writer-authored fraction of the final text and writers’ feelings of ownership. The paper does not show that changing those interface choices shapes the writer-authored fraction; its observation about receiving more suggestions is tentative. (LLM, conf=0.82)

      Experimental — accuracy not yet measured

  • Classified as probably not a substantial claim.

    Premisecitation

    Reliance on AI is appropriate when people accept correct AI output and reject incorrect output.

    “Research on human-AI decision making suggests that reliance is appropriate when people accept correct AI output and reject incorrect output; thus, the relevant outcome is not simply whether and how often students defer to AI, but how well their reliance tracks the quality of AI suggestions (Raees et al., 2026;Schemmer et al., 2023;Bansal et al., 2021).”
    Quote in the paperSubstantial claim1 of 3 sources checked
    • cites Raees et al., 2026

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Schemmer et al., 2023 source

      Not checked in the free run.

    • cites Bansal et al., 2021 source

  • May combine two claims.

    Premisecitation

    Prompted self-explanation supports monitoring of understanding, identification of knowledge gaps, and integration of new information with prior knowledge.

    “This mechanism resembles prompted self-explanation, an instructional technique that encourages learners to articulate their reasoning and has been shown to support monitoring of understanding, identification of knowledge gaps, and integration of new information with prior knowledge (Bisra et al., 2018;Chi et al., 1989).”
    Quote in the paperSubstantial claim1 of 2 sources checked
    • cites Bisra et al., 2018

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Chi et al., 1989 source

      The Analyses of Explanations section reports that students who detected gaps in their understanding often went on to explain the material to themselves. The opening passage and Discussion describe self-explanations linking example steps to principles and building understanding. But this study examines students’ self-generated explanations, not prompted self-explanation as an instructional technique; it also describes monitoring as guiding explanations rather than showing that prompting improves monitoring. Page numbers were not provided in our copy. (LLM, conf=0.85)

      Experimental — accuracy not yet measured

  • May combine two claims.

    Premisecitation

    LLM feedback has been found comparable to human teacher feedback for surface-level issues such as grammar and clarity, but its quality is contested for content and argumentation.

    “Although LLM feedback has been found to be comparable to human teacher feedback for surface-level issues such as grammar and clarity, its quality is contested for content and argumentation (Craven & Fredrick, 2026;Hyland, 2026).”
    Quote in the paperSubstantial claim1 of 2 sources checked
    • cites Craven & Fredrick, 2026 source
    • cites Hyland, 2026

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Premisecitation

    The tendency to defer to AI is consistent with automation bias, in which people overrely on automated recommendations and accept them without sufficient independent verification.

    “This tendency is consistent with automation bias, in which people overrely on automated recommendations and accept them without sufficient independent verification (Goddard et al., 2012).”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites Goddard et al., 2012 source

  • May combine two claims.

    Premisecitation

    LLMs may produce feedback that lacks specificity, contains hallucinated information, or fails to account for the broader context and goals of a student's writing.

    “LLMs may produce feedback that lacks specificity, contains hallucinated information, or fails to account for the broader context and goals of a student's writing (Davis & Mahmoud, 2026;Tseng, 2026;Jia et al., 2024).”
    Quote in the paperSubstantial claim1 of 3 sources checked
    • cites Davis & Mahmoud, 2026

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Tseng, 2026 source
    • cites Jia et al., 2024

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Premisecitation

    Alshehri and colleagues' intervention, which required students to maintain AI-use logs with justifications alongside systematic instruction, found improvements in writing fluency and students' self-reported digital critical thinking.

    “They found improvements in writing fluency and students' self-reported digital critical thinking.”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites Alshehri et al., 2026 source

      The Intervention and control conditions section describes systematic instruction and AI-use logs documenting students’ decisions and justifications. The Abstract reports improved writing proficiency and Digital Critical Thinking Scale scores, and the Discussion identifies that scale as self-report. The paper assessed writing proficiency with an essay rubric, not writing fluency specifically, so that part of the claim is not established. (LLM, conf=0.88)

      Experimental — accuracy not yet measured

  • May combine two claims. The quote only partly states this claim.

    Findingown data

    The average acceptance rate of flawed suggestions decreased from 65% to 41% with required decision justification, with no corresponding reduction in the average acceptance of sound suggestions (81% vs 86%).

    “flawed suggestions were still accepted at an average rate of 41%”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Students sometimes relied on heuristic reasoning or simply echoed the rhetorical promises of an AI suggestion without carefully examining the truthfulness of its content or considering how the proposed change would affect their existing draft.

    “Students sometimes relied on heuristic reasoning or simply echoed the rhetorical promises of an AI suggestion without carefully examining the truthfulness of its content or considering how the proposed change would affect their existing draft.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    For all four flawed suggestions, acceptance was lower in the justification condition: Suggestion C (χ2 = 9.54, p = .002), Suggestion D (χ2 = 5.73, p = .017), Suggestion F (χ2 = 4.93, p = .026), and Suggestion H (χ2 = 7.45, p = .006).

    “For all four flawed suggestions, acceptance was lower in the justification condition: Suggestion C (χ 2 = 9.54, p = .002),Suggestion D (χ 2 = 5.73, p = .017),Suggestion F (χ 2 = 4.93, p = .026),and Suggestion H (χ 2 = 7.45, p = .006).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Only a small number of justifications for flawed suggestion acceptance engaged more deeply with the relationship between the suggestion and the student's own arguments, although these students still failed to identify the specific flaw in the AI suggestion.

    “Only a small number of justifications for flawed suggestion acceptance engaged more deeply with the relationship between the suggestion and their own arguments, although these students still failed to identify the specific flaw in the AI suggestion.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Heuristic reasoning justifications recognized a general principle underlying the AI suggestion and endorsed it based on a broad heuristic without evaluating the specific suggestion in the context of the student's own proposal.

    “These justifications recognized a general principle underlying the AI suggestion and endorsed it based on a broad heuristic that a particular type of addition or revision is generally beneficial, without evaluating the specific suggestion in the context of their own proposal.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Thematic analysis revealed gaps in metacognitive monitoring and domain knowledge.

    “However, thematic analysis revealed superficial engagement in the justification task and gaps in metacognitive monitoring and domain knowledge.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Non-selective acceptance was less frequent in the justification condition than in the batch condition (χ2(1) = 3.842, p = .050) and showed a similar directional pattern compared with the selection-only condition (χ2(1) = 3.339, p = .068).

    “Non-selective acceptance was less frequent in the justification condition, with evidence of a significant difference compared to the batch condition (χ 2 (1) = 3.842, p = .050)and a similar directional pattern compared to the selection-only condition (χ 2 (1) = 3.339, p = .068).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The responses of students who correctly identified the fabrication in Suggestion C drew on their prior knowledge of learning theories, illustrating the role of domain knowledge in evaluating AI suggestions.

    “Conversely, the responses of students who correctly identified the fabrication in Suggestion C drew on their prior knowledge of learning theories, further illustrating the role of domain knowledge in evaluating AI suggestions.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Students discriminated between suggestion types rather than rejecting more overall.

    “However, two features of our results argue against a pure suspicion-cueing explanation: the intervention did not lower acceptance of sound suggestions and students discriminated between suggestion types rather than rejecting more overall.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    The analysis covers 256 justifications for flawed suggestions from 64 students.

    “The analysis covers 256 justifications for flawed suggestions from 64 students.”
    Quote in the paperNot a substantial claim

  • The quote only partly states this claim.

    Findingown data

    Acceptance rates for sound and flawed AI-generated writing suggestions differed significantly between conditions based on chi-square tests.

    “Asterisks indicate statistically significant differences in acceptance rates between conditions based on chi-square tests (* p < .05,** p < .01,*** p < .001);items without asterisks showed no significant difference between conditions.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The findings revealed challenges in the intervention's effectiveness, including superficial engagement with the justification task and gaps in students' metacognitive monitoring and domain knowledge that can limit effective evaluation of AI suggestions.

    “Our findings also revealed challenges in the intervention's effectiveness, including superficial engagement with the justification task and gaps in students' metacognitive monitoring and domain knowledge that can limit effective evaluation of AI suggestions.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

    Interpretationreasoning

    Broader interventions may help develop the knowledge and skills needed for critical AI use, while embedded justification provides a lightweight mechanism for activating and applying those capacities during actual human-AI interaction.

    “Broader interventions may help develop the knowledge and skills needed for critical AI use, while embedded justification provides a lightweight mechanism for activating and applying those capacities during actual human-AI interaction.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductreasoning

    The analysis was primarily inductive: rather than applying a predetermined taxonomy of reasoning, the first author developed codes and themes through engagement with recurring patterns in students' justifications.

    “Consistent with this orientation, the analysis was primarily inductive: rather than applying a predetermined taxonomy of reasoning, the first author developed codes and themes through engagement with recurring patterns in students' justifications.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Interpretationnone given

    Whether a sound suggestion is helpful may depend on students' individual writing goals and preferences, making its appropriateness context dependent.

    “In contrast, whether a sound suggestion is helpful may depend on students' individual writing goals and preferences, making its appropriateness context dependent.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    After the course concluded, the anonymous activity data were repurposed for research with approval from the university Institutional Review Board.

    “After the course concluded, the anonymous activity data were repurposed for research with approval from the university Institutional Review Board.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data

    The justification task adds friction to the decision making process but may not provide sufficient guidance to activate and direct the metacognitive processes needed for effective evaluation.

    “These findings indicate that while the justification task adds friction to the decision making process, it may not provide sufficient guidance to activate and direct the metacognitive processes needed for effective evaluation.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    The first author manually reviewed a random 10% sample of generated feedback (19 responses; 152 AI suggestions) to assess fidelity to the intended experimental design.

    “The first author manually reviewed a random 10% sample of the generated feedback (19 responses; 152 AI suggestions) to assess fidelity to the intended experimental design.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductown data

    Attrition was minimal: one student assigned to the justification condition and four assigned to the batch condition did not complete the experimental task.

    “Attrition was minimal, with one student assigned to the justification condition and four assigned to the batch condition not completing the experimental task.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Premisenone given

    Prior research shows that strategic friction during human-AI interactions can promote critical uptake.

    “Prior research shows that strategic friction during human-AI interactions can promote critical uptake, but how to effectively implement such friction in academic contexts remains unclear.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data

    Requiring justification reduces, though does not eliminate, uncritical uptake of AI suggestions.

    “Our findings provide causal evidence that requiring justification reduces, though does not eliminate, uncritical uptake of AI suggestions.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning

    The effective ingredient is demanding an articulation of reasoning, not merely increasing the number of clicks.

    “Thus, the effective ingredient is demanding an articulation of reasoning, not merely increasing the number of clicks.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductown data

    Among sound suggestions, three Suggestion A instances somewhat overstated the theoretical claim while remaining relevant, one Suggestion E instance proposed an incompatible analytical approach, and one Suggestion G instance identified a writing issue that was not present in the draft.

    “Among sound suggestions, three Suggestion A instances somewhat overstated the theoretical claim while remaining relevant, one Suggestion E instance proposed an incompatible analytical approach, and one Suggestion G instance identified a writing issue that was not present in the draft.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Study design and conductown data

    There were no significant differences across experimental conditions in draft length, time spent drafting, or self-reported AI use, indicating the conditions were comparable on pre-experimental characteristics.

    “There were no significant differences across experimental conditions in draft length, time spent drafting, or self-reported AI use, indicating that the conditions were comparable on these pre-experimental characteristics.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Reflexive thematic analysis views themes as interpretive constructions generated through researchers' active engagement with and interpretation of the data.

    “reflexive thematic analysis views themes as interpretive constructions generated through researchers' active engagement with and interpretation of the data (Byrne, 2022).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Byrne, 2022

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Interpretationown data

    Requiring written justification increased students' selectivity primarily by reducing their acceptance of flawed AI suggestions while preserving their acceptance of sound suggestions.

    “These results suggest that requiring written justification increased students' selectivity primarily by reducing their acceptance of flawed AI suggestions while preserving their acceptance of sound suggestions.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    News media has reported higher education practitioners' integration of required justification for AI output adoption decision as a primary assignment requirement.

    “News media has reported higher education practitioners' integration of required justification for AI output adoption decision as a primary assignment requirement (Elkhodr & Gide, 2026), but its effectiveness, particularly in academic writing settings, remains unknown.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Elkhodr & Gide, 2026

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given

    The activity was administered through the university's Qualtrics platform in anonymous mode, and no identifying information was ever collected.

    “The activity was administered through the university's Qualtrics platform in anonymous mode, and no identifying information was ever collected.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data

    Students' fact checking for AI suggestions varied not only in whether they detected potential factual problems, but also in the strategies they used to evaluate the truthfulness of AI outputs.

    “These patterns illustrate that students' fact checking for AI suggestions varied not only in whether they detected potential factual problems, but also in the strategies they used to evaluate the truthfulness of AI outputs.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    Students may perceive automated writing tools as more knowledgeable or capable than themselves and therefore be more willing to defer to their suggestions.

    “Students may perceive automated writing tools as more knowledgeable or capable than themselves and therefore be more willing to defer to their suggestions (Ranalli, 2021).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Ranalli, 2021

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    All flawed suggestions conformed to their intended flaw types.

    “All flawed suggestions conformed to their intended flaw types.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationcitation

    Multiple researchers are not required but may contribute to reflexive thematic analysis by enriching and challenging interpretations rather than establishing consensus as evidence of a single correct interpretation.

    “Multiple researchers are therefore not required but may contribute to reflexive thematic analysis by enriching and challenging interpretations rather than establishing consensus as evidence of a single correct interpretation.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Byrne, 2022

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    An additional 62 students from the batch condition contributed to a supplementary comparison of overall suggestion acceptance selectivity.

    “An additional 62 students from the batch condition contributed to a supplementary comparison of overall suggestion acceptance selectivity.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductreasoning

    The first author returned to the justifications, comparing interpretations across rounds and across similarly worded responses to identify instances where knowledge of the AI suggestion may have led to inferences not sufficiently supported by students' responses.

    “The first author then returned to the justifications, comparing interpretations across rounds and across similarly worded responses to identify instances where knowledge of the AI suggestion may have led to inferences not sufficiently supported by students' responses.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductown data

    Overall, 147 of 152 suggestions (96.7%) conformed to the intended design, indicating high fidelity of the AI-generated suggestions.

    “Overall, 147 of 152 suggestions (96.7%) conformed to the intended design, indicating high fidelity of the AI-generated suggestions.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    Students were randomly assigned to one of three conditions with approximately equal allocation.

    “Students were randomly assigned to one of three conditions with approximately equal allocation.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductreasoning

    To mitigate the risk that prior knowledge of flaws could lead to overinterpreting students' justifications, the first author iteratively revisited the data with varying levels of contextual information.

    “To mitigate this risk, the first author iteratively revisited the data with varying levels of contextual information.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    Uncritical uptake of AI writing suggestions may arise from both students' cognitive tendency to defer to AI and the self-regulatory and metacognitive demands involved in evaluating AI suggestions.

    “Together, these factors suggest that uncritical uptake of AI writing suggestions may arise from both students' cognitive tendency to defer to AI and the self-regulatory and metacognitive demands involved in evaluating AI suggestions.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data

    Disaggregating judgments in the interface changes what the interaction costs, but does not by itself change how students decide.

    “Disaggregating judgments in the interface changes what the interaction costs, but it does not by itself change how students decide.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given

    The resulting themes, names, and definitions were reviewed and discussed among the authors, and the other authors questioned and contributed to the refinement of the first author's interpretations before the thematic structure was finalized.

    “The resulting themes, names, and definitions were then reviewed and discussed among the authors.Through these discussions, the other authors questioned and contributed to the refinement of the first author's interpretations, and the thematic structure was subsequently finalized.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    Written justification can be embedded directly into students' interactions with AI.

    “A key advantage of this intervention is that it can be embedded directly into students' interactions with AI.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Study design and conductown data

    The final primary analysis included 129 students: 64 in the justification condition and 65 in the selection-only condition.

    “The final analysis included 129 students in the primary comparison between the justification condition (n = 64) and the selection-only condition (n = 65).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data

    The reduced non-selective acceptance with required justification did not reflect a general reduction in AI acceptance, but greater discrimination between sound and flawed suggestions, with students becoming less likely to accept suggestions containing flaws.

    “Our findings also showed the reduced non-selective acceptance with required justification for suggestion adoption decisions did not reflect a general reduction in AI acceptance, but greater discrimination between sound and flawed suggestions, with students becoming less likely to accept suggestions containing flaws.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductown data

    The study did not examine how students formulate prompts or how they would respond to suggestions in a conversational AI tool.

    “We did not examine how students formulate prompts, or how they would respond to suggestions (in an conversational AI tool).”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    If the students best positioned to catch a fabricated citation are those who already know the relevant literature, then friction of this kind will mostly help the already-prepared.

    “If the students best positioned to catch a fabricated citation are those who already know the relevant literature, then friction of this kind will mostly help the already-prepared.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    Each Qualtrics response was treated as representing a unique student.

    “Finally, we treated each Qualtrics response as representing a unique student.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Study design and conductreasoning

    Testing whether friction interventions widen gaps would require measuring students' domain knowledge before the activity, which was not possible in the anonymous study design.

    “Testing this possibility directly would require measuring students' domain knowledge before the activity, which was not possible in our anonymous study design.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductown data

    N = 196 students were assigned; five students did not complete the task.

    “N = 196 assigned; five students did not complete the task”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductreasoning

    The activity was ungraded, which lowers the incentive for students to invest effort in a justification and may understate the effect of adding a justification requirement under graded assessment conditions.

    “The activity was also ungraded, which lowers the incentive for students to invest effort in a justification and may understate the effect of adding a justification requirement under graded assessment conditions.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: An ungraded activity lowers the incentive for students to invest effort in a justification.

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    Adoption decisions in this study are a proximal outcome.

    “Third, adoption decisions in this study are a proximal outcome.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    Such interventions could widen rather than narrow gaps in how well students use AI feedback.

    “Such interventions could therefore widen rather than narrow gaps in how well students use AI feedback.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based). May combine two claims.

    Interpretationnone given

    Prompt formulation and responding to suggestions may involve different forms of metacognitive monitoring and require different instructional support.

    “These steps may involve different forms of metacognitive monitoring and require different instructional support.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

    Interpretationreasoning

    Together, the findings highlight both the potential of embedding strategic friction directly into human-AI interactions and the need for complementary support to foster deeper and more effective critical engagement with AI.

    “Together, these findings highlight both the potential of embedding strategic friction directly into human-AI interactions and the need for complementary support to foster deeper and more effective critical engagement with AI.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Study design and conductnone given

    The findings come from a single course and one academic writing context.

    “Fourth, our findings come from a single course and one academic writing context.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    A configuration error prevented students who chose 'accept some' from selecting multiple suggestions in the batch condition.

    “a configuration error prevented students who chose”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    Justification gives students an opportunity to apply what they know, but does not necessarily support them around what they do not know.

    “Justification gives students an opportunity to apply what they know, but does not necessarily support them around what they do not know.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    The intervention only targeted one point in the writing process.

    “The intervention also only targeted one point in the writing process.”
    Quote in the paperNot a substantial claim
66 of 66

  • The paper relies on this without support.

    Accepting a flawed AI suggestion is a valid indicator of uncritical uptake of AI suggestions.

    “We compared students' responses across conditions along two behavioral dimensions of critical AI uptake: selectivity of adoption across suggestions of different quality (i.e., whether students adopted all suggestions or selectively adopted some) and quality of adoption decisions (i.e., whether students adopted flawed AI suggestions).”

  • The paper relies on this without support.

    The experimental activity's measures of adoption decisions generalize to academic writing contexts beyond the study activity.

    “We examine whether requiring students to justify decisions to accept or reject AI suggestions can mitigate uncritical uptake in academic writing.”

  • The authors state this as an assumption.

    Students' written justifications provide insight into their evaluative reasoning, though such accounts are partial and context dependent.

    “Epistemologically, the analysis adopted an experiential orientation, treating students' written justifications as accounts that provide insight into their evaluative reasoning while recognizing that such accounts are partial and context dependent.”

  • The paper relies on this without support.

    Accepting all eight suggestions is a valid indicator of non-selective or uncritical uptake.

    “We examined the frequency of non-selective acceptance, defined as accepting all eight suggestions.”

  • The paper relies on this without support.

    Shorter decision-making time indicates reduced engagement with the evaluation task.

    “We also compared decision making time between non-selective adopters and other students within each condition to examine whether nonselective adoption was associated with reduced engagement with the evaluation task.”

  • The paper relies on this without support.

    The AI suggestions classified as flawed have identifiable, established flaws, and the sound suggestions are sound.

    “our analysis focused on the former because the study design deliberately embedded identifiable flaws in selected AI suggestions.”

  • The paper relies on this without support.

    The batch condition approximates the interaction design of commercial AI writing tools.

    “We included a third condition (batch condition) as a secondary baseline that approximated the batch interactions (n = 66).”
    “Nearly one third of students in the batch condition, which approximated the interaction design of commercial AI writing tools, accepted all AI suggestions wholesale.”

  • The paper relies on this without support.

    Engaging with alternative perspectives is an important rhetorical practice in academic argumentation.

    “even though engaging with alternative perspectives is an important rhetorical practice in academic argumentation”

  • The paper relies on this without support.

    Current models produce flawed suggestions at a rate well below half.

    “Half of the eight suggestions had a deliberate flaw, which is well above the rate at which current models produce them.”

  • The paper relies on this without support.

    An ungraded activity lowers the incentive for students to invest effort in a justification.

    “The activity was also ungraded, which lowers the incentive for students to invest effort in a justification and may understate the effect of adding a justification requirement under graded assessment conditions.”

  • The paper relies on this without support.

    Writing a reason takes longer than clicking a checkbox.

    “Writing a reason takes longer than clicking a checkbox and being asked for a reason may itself signal that some suggestions are worth doubting.”
11 of 11

Cites

The publisher hasn’t deposited this paper’s reference list.

Research works printed but not linked: 12

  1. Kiran Bisra, Qing Liu, John C Nesbit, Farimah Salimi, and Philip H Winne. Inducing self- explanation: A meta-analysis. Educational Psychology Review, 30(3):703-725, 2018.
  2. Zana Buc ¸inca, Maja Barbara Malaya, and Krzysztof Z Gajos. To trust or to think: cognitive forcing functions can reduce overreliance on ai in ai-assisted decision-making. Proceedings of the ACM on Human-computer Interaction, 5(CSCW1):1-21, 2021.
  3. David Byrne. A worked example of braun and clarke's approach to reflexive thematic analysis. Quality & quantity, 56(3):1391-1412, 2022.
  4. Kimberly Davis and Qusay Mahmoud. Citation constraints and reference hallucinations in large language models. In The 39th Canadian Conference on Artificial Intelligence, pp. 464-475.
  5. PMLR, 2026.
  6. Jonathan St BT Evans. In two minds: dual-process accounts of reasoning. Trends in cognitive sciences, 7(10):454-459, 2003.
  7. Steve Graham. Changing how writing is taught. Review of research in education, 43(1):277-303, 2019.
  8. Ken Hyland. Efl teachers and feedback fatigue: Ai to the rescue? Language Teaching, 59(3): 310-326, 2026.
  9. Qinjin Jia, Jialin Cui, Ruijie Xi, Chengyuan Liu, Parvez Rashid, Ruochi Li, and Edward Gehringer. On assessing the faithfulness of llm-generated feedback on student assignments. In Proceedings of the 17th international conference on educational data mining, pp. 491-499, 2024.
  10. Bing Liu. How does self-regulated learning resist ai dependence? a mediating effect study based on college students who frequently use aigc tools. Sage Open, 16(1):21582440261420068, 2026.
  11. Jim Lo, Christy Wong, Agnes Ng, Pinna Wong, Denise Cheung, and Pauli Lai. Stretching ai's reach: Assessing an ai-driven feedback system for extended academic writing. Computers and Education: Artificial Intelligence, pp. 100511, 2025.
  12. Jim Ranalli. L2 student engagement with automated feedback on writing: Potential for learning and issues of trust. Journal of Second Language Writing, 52:100816, 2021.

Related

Authors
Tao, Y., Meyer, J., & Kizilcec, R. F.
Affiliations
  • Cornell University (US)
  • University of Vienna (AT)
Venue
arXiv
Publisher
arXiv
Published
2026
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2609.23936
arXiv
2609.23936
OpenAlex
W7214057524
Semantic Scholar
75945a23bd26a248b6b82cb5649a34d313fc5594
Cited by
0
References
0
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Information Systems › Artificial Intelligence in Education
  • Social Sciences › Psychology › Developmental and Educational Psychology › Innovative Teaching and Learning Methods
  • Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
Keywords
generative AI, written justification, human-AI interaction, academic writing, metacognitive monitoring, domain knowledge, student decision-making

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-30.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-30.

Cited sources are checked by gpt-6-sol.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review