The DOI matches its Crossref record; 30 of the 30 references it cites were found in a registry or an index; 9 of the 23 sources its claims cite were checked for support.
Beyond unstructured use: scaffolded integration of generative AI in assessment for learning
Open accessAbstract
This study examined how scaffolded integration of generative artificial intelligence (GenAI) within an assessment task influenced students’ experience, evaluative engagement, learning, confidence, and competence. A cross-sectional quantitative survey was completed by 164 first-year undergraduate students in a fully online health communication course. The assessment required students to generate, critically evaluate, and revise GenAI-assisted outputs, and reflect on their use. The questionnaire comprised Likert-scale and multiple-response items assessing students’ experiences, learning, confidence, competence, challenges, and perceived skill development. Composite scales were analysed descriptively and compared against neutral midpoints using non-parametric tests. Students reported positive technical experiences (64%) and learning benefits, particularly in critique writing (67%), critical thinking (83%), and responsible GenAI use (82%). Evaluative skills, including critical appraisal (71%) and source verification (69%), were frequently reported. Perceptions of reliability were cautious, and confidence and competence did not differ from neutral. Composite analyses indicated positive experience and learning without corresponding increases in confidence. Scaffolded integration of GenAI within assessment can support assessment for learning by promoting evaluative engagement and critical AI literacy without encouraging uncritical reliance. These findings position assessment design…
Overview
CoChat Summary
TL;DR
This cross-sectional survey of 164 first-year online health communication students examined scaffolded Microsoft Co-pilot integration in an assessment requiring critique and revision of GenAI outputs. Students reported positive technical experience and learning/evaluative benefits, but reliability perceptions were cautious and confidence/competence did not differ from neutral.
Key findings
- Composite Perceived GenAI Experience median was 3.5 (IQR 1.0), significantly above the neutral midpoint of 3 (Wilcoxon Z = −6.81, p < 0.001, r = 0.40); General Perceived Benefits median was 3.25 (IQR 1.0), also above neutral (Z = −4.98, p < 0.001, r = 0.31); GenAI Confidence and Competence median was 3.0 (IQR 1.0) and did not differ from neutral (Z = −0.42, p = 0.676).
- Item-level results: 64% reported positive technical experience, 55% rated usefulness positively, 46% rated content quality positively, and 34% rated outputs reliable versus 30% unreliable. Learning impacts were reported for final critique writing (67%), critical thinking (83%), and responsible GenAI use (82%).
- Challenges most reported were lack of clarity in GenAI’s reasoning or sources (58%), difficulty identifying bias (49%), irrelevant or inaccurate outputs (31%), and difficulty understanding prompts (13%). Skills most reported were critically evaluating outputs (71%), verifying source credibility (69%), effective prompting (59%), identifying bias (49%), and paraphrasing (44%).
- Spearman correlations showed a strong association between Perceived GenAI Experience and General Perceived Benefits (ρ = 0.68, p < 0.001), with moderate associations between Confidence/Competence and Experience (ρ = 0.45, p < 0.001) and between Confidence/Competence and Benefits (ρ = 0.39, p < 0.001).
Methods
- Cross-sectional quantitative survey administered after assessment completion from 04/04/25 to 18/04/25. Participants were 164 of 181 enrolled students (91% response rate), university students aged 18+ in a fully online core undergraduate Health Communication course; demographics were not collected, participation was voluntary, and responses were anonymised.
- The assessment embedded Microsoft Co-pilot at the initial drafting stage using a standardised prompt. Students critically evaluated the GenAI-generated draft, documented issues in structured comment boxes, produced a final revised critique in their own words, and completed a reflective component.
- The study-specific questionnaire had 15 items: Likert-type items Q1–Q13 on five-point scales and multiple-response items Q14–Q15. Analysis used descriptive frequencies/percentages, three composite scales (Experience: Q1, Q2, Q3, Q8; Benefits: Q9, Q10, Q11, Q13; Confidence/Competence: Q4, Q12), Cronbach’s alpha, Wilcoxon signed-rank tests against neutral 3, and Spearman correlations in SPSS version 23; no inferential statistics were applied to multiple-response items, and no adjustment for multiple comparisons was applied.
Limitations
- Stated: the study was conducted in a single first-year course, limiting generalisability; prior GenAI experience was not assessed; no objective behavioural or performance measures were included; the cross-sectional post-assessment design without a comparison group precludes causal inference; and assessment artefacts were not analysed, so findings reflect perceptions rather than direct behavioural evidence of evaluative judgement.
- Stated: composite scales had very high internal consistency (Cronbach’s α = 0.94–0.98), which may indicate item redundancy; the questionnaire was study-specific and not intended as a validated psychometric scale; and no formal adjustment for multiple testing was applied.
- Evident from the text: the study relies on self-report perceptions, collected no demographic data, and had small item non-response for multiple-response items (Q14 n = 159; Q15 n = 163).
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
9 Valid · 0 Not valid · 10 Not checked · 85 Own results and interpretations
Every source the paper cites for this claim was checked and supports it.
In the paper
“In GenAI-mediated environments, this capability becomes increasingly important, as fluent and plausible outputs may obscure inaccuracies, bias, or incomplete reasoning (Bearman et al., 2024).”
§ Beyond unstructured use: scaffolded integration of generative AI in assessment for learning > Introduction
What the sources say
- Supportedcites Bearman et al., 2024 source
The Introduction says GenAI outputs can appear plausible even when unsuitable, including when they are incorrect, biased, or too generic. It also warns that superficially persuasive information may prompt uncritical reliance without good evaluative judgement; its discussion of elegant prose versus a nuanced rationale supports the concern about reasoning. (LLM, conf=0.90)
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Emerging implementation evidence further suggests that structured assessment frameworks can support ethical GenAI integration, improve student engagement, and reduce academic misconduct (Furze et al., 2024).”
§ Beyond unstructured use: scaffolded integration of generative AI in assessment for learning > Introduction
What the sources say
- Supportedcites Furze et al., 2024 source
The Changes in pedagogical practices section (p. 10) reports increased student engagement following implementation of the AIAS, a structured assessment framework. It also describes new avenues for student creativity and engagement. (LLM, conf=0.93)
hedgedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Emerging implementation evidence further suggests that structured assessment frameworks can support ethical GenAI integration, improve student engagement, and reduce academic misconduct (Furze et al., 2024).”
§ Beyond unstructured use: scaffolded integration of generative AI in assessment for learning > Introduction
What the sources say
- Supportedcites Furze et al., 2024 source
The Academic misconduct section (p. 9) reports fewer misconduct cases after implementation of the structured AI Assessment Scale and suggests it had a positive impact on academic integrity. The Limitations section (p. 12) cautions that other factors may explain the decrease, so the evidence supports the citing sentence’s hedged claim that such a framework can reduce misconduct, not a definitive causal claim. (LLM, conf=0.89)
hedgedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Emerging implementation evidence further suggests that structured assessment frameworks can support ethical GenAI integration, improve student engagement, and reduce academic misconduct (Furze et al., 2024).”
§ Beyond unstructured use: scaffolded integration of generative AI in assessment for learning > Introduction
What the sources say
- Supportedcites Furze et al., 2024 source
The paper describes the AIAS as a framework for ethical GenAI use (Discussion, p. 11). It reports enhanced student engagement with GenAI and fewer reported GenAI-related misconduct cases after the pilot (p. 1), while cautioning that other factors may have contributed to the misconduct reduction (Limitations, p. 12). The citing sentence’s “suggests” reflects that uncertainty. (LLM, conf=0.91)
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“This perspective is supported by recent experimental evidence demonstrating that learning outcomes depend on how GenAI is integrated into learning activities rather than access to the technology itself; unrestricted GPT-4 access impaired subsequent unaided performance, whereas a guard-railed tutoring approach did not (Bastani et al., 2025)”
§ Beyond unstructured use: scaffolded integration of generative AI in assessment for learning > Introduction
What the sources say
- Supportedcites Bastani et al., 2025 source
The Abstract reports that students using GPT Base performed 17% worse than controls on a subsequent exam, while that negative effect was essentially eliminated with the guard-railed GPT Tutor. This supports the distinction between merely providing access and designing the tool to support learning. The Discussion limits the finding to two tutors used for mathematics in one high school. (LLM, conf=0.94)
boostedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
64 more claims
Create a free account to see all 104 claims and to verify the 2 references still open.
- It may state two separate claims as one. 19
- Probably not a substantial claim. 22
Citations
Cites
Assessment and Classroom Learning
Black, P., & Wiliam, D. (1998) · Assessment in Education: Principles, Policy & Practice · doi:10.1080/0969595980050102 · Cited by 8,152
ChatGPT for good? On opportunities and challenges of large language models for education
Kasneci, E., Sessler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., et al. (2023) · Learning and Individual Differences · doi:10.1016/j.lindif.2023.102274 · Cited by 7,326
ChatGPT: five priorities for research
van Dis, E. A. M., Bollen, J., Zuidema, W., van Rooij, R., & Bockting, C. L. (2023) · Nature · doi:10.1038/d41586-023-00288-7 · Cited by 1,842
Developing evaluative judgement: enabling students to make decisions about the quality of work
Tai, J., Ajjawi, R., Boud, D., Dawson, P., & Panadero, E. (2018) · Higher Education · doi:10.1007/s10734-017-0220-3 · Cited by 810
McDonald, N., Johri, A., Ali, A., & Collier, A. H. (2025) · Computers in Human Behavior: Artificial Humans · doi:10.1016/j.chbah.2025.100121 · Cited by 682
ChatGPT and large language models in academia: opportunities and challenges
Meyer, J. G., Urbanowicz, R. J., Martin, P. C. N., O’Connor, K., Li, R., Peng, P.-C., Bright, T. J., Tatonetti, N., Won, K. J., Gonzalez-Hernandez, G., et al. (2023) · BioData Mining · doi:10.1186/s13040-023-00339-9 · Cited by 569
Cao, Y., Li, S., Liu, Y., Yan, Z., Dai, Y., Yu, P. S., & Sun, L. (2023) · arXiv · doi:10.48550/arxiv.2303.04226 · Cited by 463
Developing evaluative judgement for a time of generative artificial intelligence
Bearman, M., Tai, J., Dawson, P., Boud, D., & Ajjawi, R. (2024) · Assessment & Evaluation in Higher Education · doi:10.1080/02602938.2024.2335321 · Cited by 312
Generative AI without guardrails can harm learning: Evidence from high school mathematics
Bastani, H., Bastani, O., Sungu, A., Ge, H., Kabakcı, Ö., & Mariman, R. (2025) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2422633122 · Cited by 309
Generative AI and Academic Integrity in Higher Education: A Systematic Review and Research Agenda
Bittle, K., & El-Gayar, O. (2025) · Information · doi:10.3390/info16040296 · Cited by 305
Research works printed but not linked: 3
- Bandura, A. (1997). Self-efficacy: The exercise of control. W.H. Freeman & Company.
- 10.48550/arXiv.2303.042 · arXiv · doi:10.48550/arxiv.2303.042
- Fawns, T., Henderson, M., Matthews, K. E., Oberg, G., Liang, Y., Walton, J., Corbin, T. A., Bearman, M. L., Buckingham Shum, S., McCluskey, T., McLean, J., Shibani, A., Bakharia, A., Lim, L.-A., Pepperell, N., et al. (2024). Gen AI and student perspectives of use and ambiguity: a multi-institutional study. In Proceedings ASCILITE.
Metadata
- Authors
- Cochrane, D., Littschwager, S., Tsang, E., & Jin, Y.
- Affiliations
- Massey University (NZ)
- Corresponding author
- Cochrane, Darryl
- ORCID
- Cochrane, Darryl
- Venue
- Cogent Education
- Publisher
- Informa UK Limited
- Published
- 2026-09-12
- Volume
- 13
- Issue
- 1
- ISSN
- 2331-186X (ISSN-L)
- OpenAlex
- W7212359061
- Cited by
- 0
- References
- 27
- FWCI
- 0.00
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0/ (vor)
- doi.org/10.1080/policies-text-data-mining (tdm)
- Locations
- Cogent Education (publishedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Topics
- Social Sciences › Education › Artificial Intelligence in Education
- Social Sciences › Education › Student Assessment and Feedback
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Keywords
- generative AI, assessment for learning, AI-assisted assessment, critical AI literacy, responsible AI use, critical thinking, source verification, first-year students, artificial intelligence in higher education
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-05, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-05 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Beyond unstructured use: scaffolded integration of generative AI in assessment for learning. https://app.cochat.ai/papers/10.1080/2331186x.2026.2731708
- The paper
- Cochrane, D., Littschwager, S., Tsang, E., & Jin, Y. (2026). Beyond unstructured use: scaffolded integration of generative AI in assessment for learning. Cogent Education. https://doi.org/10.1080/2331186x.2026.2731708
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.


