Participatory Co-Design and Evaluation of a Novel Approach to Generative AI-Integrated Coursework Assessment in Higher Education
Open accessThe DOI matches its Crossref record; 25 of the 25 references it cites were found in a registry or an index; 15 of the 34 sources its claims cite were checked for support.
Abstract
Generative AI tools offer opportunities for enhancing learning and assessment, but raise concerns about equity, academic integrity, and the ability to critically engage with AI-generated content. This study explores these issues within a psychology-oriented postgraduate programme at a UK university. We co-designed and evaluated a novel AI-integrated assessment aimed at improving critical AI literacy among students and teaching staff (pre-registration: osf.io/jqpce). Students were randomly allocated to two groups: the ‘compliant’ group used AI tools to assist with writing a blog and critically reflected on the outputs, while the ‘unrestricted’ group had free rein to use AI to produce the assessment. Teaching staff, blinded to group allocation, marked the blogs using an adapted rubric. Focus groups, interviews, and workshops were conducted to assess the feasibility, acceptability, and perceived integrity of the approach. Findings suggest that, when carefully scaffolded, integrating AI into assessments can promote both technical fluency and ethical reflection. A key contribution of this study is its participatory co-design and evaluation method, which was effective and transferable, and is presented as a practical toolkit for educators. This approach supports growing calls for authentic assessment that mirrors real-world tasks, while highlighting the ongoing need to balance academic integrity with skill development.
Overview
CoChat Summary
TL;DR
This study co-designed and evaluated an AI-integrated postgraduate assessment in which students used generative AI to draft and critique a blog, with random allocation to “compliant” or “unrestricted” AI-use groups. Findings suggest the approach was feasible and supported critical AI literacy, but markers could not reliably distinguish compliant from unrestricted AI-generated submissions.
Key findings
- Students and teaching staff generally found the co-designed AI-integrated assessment feasible and acceptable, with students valuing critical thinking development but reporting that AI did not reduce workload because outputs required substantial checking and refinement.
- Markers correctly identified 6 out of 14 students in the compliant group (42.9%) versus 3 out of 6 in the unrestricted group (50.0%); Fisher’s exact test gave an odds ratio of 0.75, p = 1.00, indicating no meaningful difference in marker accuracy.
- Assessment marks ranged from 35 (fail) to 78 (distinction); most markers’ scores fell within a ten-point range, but one assessment ranged from 58 to 78.
- Teaching staff initially found marking time-intensive due to evaluating both AI and student contributions, though efficiency improved with familiarity; most endorsed transparent, pedagogically grounded AI use despite integrity concerns.
Methods
- Participatory co-design and evaluation within a psychology-oriented postgraduate programme at a UK university (King’s College London), using the 2023–24 cohort of approximately 30 students; 12 participants took part: 8 students and 4 teaching staff.
- The adapted assessment required two AI-generated blog drafts, a final blog combining AI outputs with student revisions, and a critical commentary on AI-generated content; students were randomly allocated to a “compliant” group following the brief or an “unrestricted” group free to use AI by any means, and teaching staff marked blinded using an adapted rubric.
- Qualitative data came from semi-structured focus groups and interviews (45–60 min), workshop feedback, and co-design workshops using think-aloud methods; analysis used Braun and Clarke’s thematic analysis separately for students and staff, plus Fisher’s exact test for marker accuracy.
Limitations
- The paper states that students were not involved in the initial design phase, falling short of authentic co-production; qualitative feedback was collected after the initial pilot rather than a full module-wide rollout; and the single-institution, small-cohort, small-sample design limits generalisability.
- The paper also notes limitations of participatory approaches generally, including potential power imbalances, risks of tokenism, and over-reliance on stakeholder input at the expense of expert judgement.
- Evident from the text: the very small sample and short-term pilot limit statistical inference and long-term conclusions; reported group counts for marker identification (14 compliant and 6 unrestricted) do not align clearly with the stated 8 student participants, suggesting possible reporting or allocation ambiguity.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
11 Valid · 1 Not valid · 11 Not checked · 66 Own results and interpretations
Every source the paper cites for this claim was checked and supports it.
In the paper
“Academic integrity is also at risk, as AI can be used to ‘cheat’ in ways that evade detection (Yusuf et al., 2024).”
§ Participatory Co-Design and Evaluation of a Novel Approach to Generative AI-Integrated Coursework Assessment in Higher Education > 1. Introduction
What the sources say
- Supportedcites Yusuf et al., 2024 source
In “Perceived potential and concerns of GenAI in HE,” the paper lists respondents’ concern that GenAI tools promote cheating and plagiarism without being easily detected. It also lists threats to academic integrity, supporting the citing sentence as a statement of risk rather than a measured detection failure. (LLM, conf=0.91)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Initial studies suggest that, while students may benefit from AI-enhanced feedback, overreliance on these tools may undermine opportunities for deep learning and critical engagement (Kasneci et al., 2023; Suriano et al., 2025; Zawacki-Richter et al., 2019; Zhai et al., 2024).”
§ Participatory Co-Design and Evaluation of a Novel Approach to Generative AI-Integrated Coursework Assessment in Higher Education > 1. Introduction
What the sources say
- Partly supportedcites Kasneci et al., 2023 sourceExperimental — accuracy not yet measured
Section 4 warns that learners may rely too heavily on language models, reducing critical thinking and their interest in conducting their own investigations. Section 2.2 reports benefits of AI-generated feedback, but the warning is about reliance on models generally; the paper does not specifically link overreliance on feedback to diminished deep learning. (LLM, conf=0.86)
- Not checked — no full text of the sourcecites Suriano et al., 2025 source
- Partly supportedcites Zawacki-Richter et al., 2019 sourceExperimental — accuracy not yet measured
The Automated grading section describes AI-based feedback that can guide students and help improve their writing. The paper also urges attention to pedagogy, but the passages do not say that overreliance on AI feedback undermines deep learning or critical engagement. Page numbers are unavailable in our copy. (LLM, conf=0.82)
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“By embedding critical evaluation and editorial judgement, the assessment addressed concerns that AI could weaken core academic skills such as critical thinking and reflective analysis (Bittle & El-Gayar, 2025).”
§ Participatory Co-Design and Evaluation of a Novel Approach to Generative AI-Integrated Coursework Assessment in Higher Education > 4. Discussion
What the sources say
- Supportedcites Bittle & El-Gayar, 2025 source
The Path Forward (section 4.5) warns that reliance on GenAI could reduce students’ critical thinking skills, while Redesigning Assessments (section 4.3) calls for assessments that require higher-order thinking. The sentence’s description of the particular assessment is the citing authors’ own account; the source supports the concern and the rationale for addressing it through assessment design. (LLM, conf=0.86)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“While challenges remained in distinguishing AI-generated from student-authored content, most staff endorsed transparent and pedagogically grounded use of AI in academic settings.”
- Also stated in: § 4. Discussion · plain
“Teaching staff initially found marking more demanding and had limited success distinguishing unrestricted AI-generated content, but valued the assessment's potential to promote ethical and critical AI use.”
§ Participatory Co-Design and Evaluation of a Novel Approach to Generative AI-Integrated Coursework Assessment in Higher Education > 3. Results > 3.2. Feasibility, Acceptability, and Integrity
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § 4. Discussion · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Teaching staff found the assessment structure clear, although marking was initially time-intensive because of the dual task of evaluating both AI and student contributions.”
- Also stated in: § 4. Discussion · plain
“Teaching staff initially found marking more demanding and had limited success distinguishing unrestricted AI-generated content, but valued the assessment's potential to promote ethical and critical AI use.”
§ Participatory Co-Design and Evaluation of a Novel Approach to Generative AI-Integrated Coursework Assessment in Higher Education > 3. Results > 3.2. Feasibility, Acceptability, and Integrity
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § 4. Discussion · plain
24 more claims
Create a free account to see all 89 claims and to verify the 1 reference still open.
- Probably not a substantial claim. 40
- It may state two separate claims as one. 28
Citations
Cites
R: A Language and Environment for Statistical Computing
R Core Team (2025) · doi:10.32614/r.manuals · Cited by 352,181
Using thematic analysis in psychology
Braun, V., & Clarke, V. (2006) · Qualitative Research in Psychology · doi:10.1191/1478088706qp063oa · Cited by 194,414
Sample Size in Qualitative Interview Studies: Guided by Information Power
Malterud, K., Siersma, V. D., & Guassora, A. D. (2016) · Qualitative Health Research · doi:10.1177/1049732315617444 · Cited by 12,703 · cited 2×
“However, this study did not aim for statistical generalisability, but rather aimed to explore the feasibility and acceptability in context, using participatory methods grounded in information power (Malterud et al., 2016).”
ChatGPT for good? On opportunities and challenges of large language models for education
Kasneci, E., Sessler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., et al. (2023) · Learning and Individual Differences · doi:10.1016/j.lindif.2023.102274 · Cited by 7,326 · cited 3×
“Initial studies suggest that, while students may benefit from AI-enhanced feedback, overreliance on these tools may undermine opportunities for deep learning and critical engagement (Kasneci et al., 2023; Suriano et al., 2025; Zawacki-Richter et al., 2019; Zhai et al., 2024).”
Zawacki-Richter, O., Marín, V. I., Bond, M., & Gouverneur, F. (2019) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-019-0171-0 · Cited by 7,242 · cited 2×
“Initial studies suggest that, while students may benefit from AI-enhanced feedback, overreliance on these tools may undermine opportunities for deep learning and critical engagement (Kasneci et al., 2023; Suriano et al., 2025; Zawacki-Richter et al., 2019; Zhai et al., 2024).”
Staniszewska, S., Brett, J., Simera, I., Seers, K., Mockford, C., Goodlad, S., Altman, D. G., Moher, D., Barber, R., Denegri, S., et al. (2017) · BMJ · doi:10.1136/bmj.j3453 · Cited by 1,983 · cited 1×
“We used the Guidance for Reporting Involvement of Patients and the Public short form (GRIPP2-SF) check-list to report involvement in the study (reported in Table S1 in the Supplementary Materials; (Staniszewska et al., 2017).”
Zhai, C., Wibowo, S., & Li, L. D. (2024) · Smart Learning Environments · doi:10.1186/s40561-024-00316-7 · Cited by 1,567 · cited 1×
“Initial studies suggest that, while students may benefit from AI-enhanced feedback, overreliance on these tools may undermine opportunities for deep learning and critical engagement (Kasneci et al., 2023; Suriano et al., 2025; Zawacki-Richter et al., 2019; Zhai et al., 2024).”
The use of think-aloud methods in qualitative research an introduction to think-aloud methods
Charters, E. (2003) · Brock Education Journal · doi:10.26522/brocked.v12i2.38 · Cited by 748 · cited 2×
“The think aloud technique was used (Charters, 2003; Someren et al., 1994), whereby each section of the assessment materials was reviewed in turn.”
Bovill, C., Cook-Sather, A., Felten, P., Millard, L., & Moore-Cherry, N. (2016) · Higher Education · doi:10.1007/s10734-015-9896-4 · Cited by 685 · cited 1×
“This aligns with broader calls for more inclusive, responsive, and transparent innovation in educational assessment (Bovill et al., 2016; Healey & Healey, 2019).”
AI‐driven adaptive learning for sustainable educational transformation
Strielkowski, W., Grebennikova, V., Lisovskiy, A., Rakhimova, G., & Vasileva, T. (2025) · Sustainable Development · doi:10.1002/sd.3221 · Cited by 581 · cited 2×
“Interest in the role of AI in education has accelerated rapidly in recent years (Strielkowski et al., 2025), with growing attention being paid to its implications for assessment and feedback practices (e.g., Henderson et al., 2025; Usher, 2025).”
Yusuf, A., Pervin, N., & Román-González, M. (2024) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-024-00453-6 · Cited by 554 · cited 2×
“This echoes broader challenges in the literature, where AI use may complicate traditional definitions of scholarship and independent academic work (Kulkarni et al., 2024; Luo, 2024; Yusuf et al., 2024).”
An introduction to threshold concepts
Cousin, G. (2006) · Planet · doi:10.11120/plan.2006.00170004 · Cited by 443 · cited 1×
“Participatory evaluation involves contributors not just as participants, but as co-designers and co-evaluators (Fetterman et al., 2017), and has been used previously to explore AI-related resources and curriculum development (Cousin, 2006; Teodorowski et al., 2023).”
Luo (Jess), J. (2024) · Assessment & Evaluation in Higher Education · doi:10.1080/02602938.2024.2309963 · Cited by 366 · cited 2×
“This echoes broader challenges in the literature, where AI use may complicate traditional definitions of scholarship and independent academic work (Kulkarni et al., 2024; Luo, 2024; Yusuf et al., 2024).”
Generative AI and Academic Integrity in Higher Education: A Systematic Review and Research Agenda
Bittle, K., & El-Gayar, O. (2025) · Information · doi:10.3390/info16040296 · Cited by 305 · cited 2×
“There are also concerns that critical thinking, a key goal of higher education, could be weakened if students accept AI outputs without careful evaluation (Bittle & El-Gayar, 2025).”
Artificial intelligence for healthcare and medical education: a systematic review.
Li, S., Yin, C., Xu, Q., & Zhao, W. (2023) · PubMed Central · Cited by 253
Comparing Generative AI and teacher feedback: student perceptions of usefulness and trustworthiness
Henderson, M., Bearman, M., Chung, J., Fawns, T., Buckingham Shum, S., Matthews, K. E., & de Mello Heredia, J. (2026) · Assessment & Evaluation in Higher Education · doi:10.1080/02602938.2025.2502582 · Cited by 211 · cited 1×
“Interest in the role of AI in education has accelerated rapidly in recent years (Strielkowski et al., 2025), with growing attention being paid to its implications for assessment and feedback practices (e.g., Henderson et al., 2025; Usher, 2025).”
Student Interaction with ChatGPT Can Promote Complex Critical Thinking Skills
Suriano, R., Plebe, A., Acciai, A., & Fabio, R. A. (2025) · Learning and Instruction · doi:10.1016/j.learninstruc.2024.102011 · Cited by 208 · cited 1×
“Initial studies suggest that, while students may benefit from AI-enhanced feedback, overreliance on these tools may undermine opportunities for deep learning and critical engagement (Kasneci et al., 2023; Suriano et al., 2025; Zawacki-Richter et al., 2019; Zhai et al., 2024).”
Bradbury, K., Morton, K., Band, R., van Woezik, A., Grist, R., McManus, R. J., Little, P., & Yardley, L. (2018) · PLOS ONE · doi:10.1371/journal.pone.0196868 · Cited by 177 · cited 1×
“We documented this process using a Table of Changes (ToC) from the Person-Based Approach using the MoSCoW method, a prioritisation framework used to collaboratively decide which features, changes, or recommendations should be implemented [must, should, could, would like] (Bradbury et al., 2018).”
Critical thinking in the age of generative AI
Larson, B. Z., Moser, C., Caza, A., Muehlfeld, K., & Colombo, L. A. (2024) · Academy of Management Learning & Education · doi:10.5465/amle.2024.0338 · Cited by 161 · cited 1×
“…address concerns raised during the qualitative evaluation and reflected issues highlighted in previous research, such as that overreliance on AI could undermine opportunities for deep learning and reflective practice (Kasneci et al., 2023; Larson et al., 2024; Zawacki-Richter et al., 2019).”
Generative AI’s Impact on Critical Thinking: Revisiting Bloom’s Taxonomy
Gonsalves, C. (2026) · Journal of Marketing Education · doi:10.1177/02734753241305980 · Cited by 161 · cited 2×
“This challenge is prompting the development of pedagogical tools to enhance deeper engagement with AI content, including a revised version of Bloom’s Taxonomy (Gonsalves, 2024).”
Chi-square test: Analysis of contingency tables
Howell, D. C. (2011) · International Encyclopedia of Statistical Science · doi:10.1007/978-3-642-04898-2_174 · Cited by 108 · cited 1×
“Given the small sample size and expected cell counts of below five, we used Fisher’s exact test rather than the chi-square approximation (Howell, 2011), calculated using base R (R Core Team, 2025).”
The Future of Research in an Artificial Intelligence-Driven World
Kulkarni, M., Mantere, S., Vaara, E., van den Broek, E., Pachidi, S., Glaser, V. L., Gehman, J., Petriglieri, G., Lindebaum, D., Cameron, L. D., et al. (2024) · Journal of Management Inquiry · doi:10.1177/10564926231219622 · Cited by 80 · cited 2×
“This echoes broader challenges in the literature, where AI use may complicate traditional definitions of scholarship and independent academic work (Kulkarni et al., 2024; Luo, 2024; Yusuf et al., 2024).”
Usher, M. (2025) · Assessment & Evaluation in Higher Education · doi:10.1080/02602938.2025.2487495 · Cited by 75 · cited 1×
“Interest in the role of AI in education has accelerated rapidly in recent years (Strielkowski et al., 2025), with growing attention being paid to its implications for assessment and feedback practices (e.g., Henderson et al., 2025; Usher, 2025).”
Generative AI in the Classroom: Can Students Remain Active Learners?
Abdelghani, R., Sauzéon, H., & Oudeyer, P.-Y. (2023) · arXiv · doi:10.48550/arxiv.2310.03192 · Cited by 48
Teodorowski, P., Gleason, K., Gregory, J. J., Martin, M., Punjabi, R., Steer, S., Savasir, S., Vema, P., Murray, K., Ward, H., et al. (2023) · Research Involvement and Engagement · doi:10.1186/s40900-023-00480-z · Cited by 8 · cited 2×
“The participatory approach drew on academic, pedagogical, and lived experience to shape the teaching workshop and assessment materials, helping us spot practical challenges early and promote shared ownership of the development of the assessment (Fetterman et al., 2017; Teodorowski et al., 2023).”
Research works printed but not linked: 3
- Cook-Sather, A., Bovill, C., and Felten, P. (2014). Engaging students as partners in learning and teaching: A guide for faculty, John Wiley & Sons.
- Healey, 2019, Student engagement through partnership: A guide and update to the AdvanceHE framework, Advance HE, 12, 1
- Someren, M. W. v., Barnard, Y. F., and Sandberg, J. (1994). The think aloud method: A practical guide to modelling cognitive processes, Academic Press.
Cited by
Zhijuan, L. A. I., Zhang, T., Huang, N., Li, Y., You, T., Zhao, J., & Zheng, J. (2026) · Nurse Education Today · doi:10.1016/j.nedt.2026.107019 · Cited by 15
AI for Formative and Summative Assessment: A Balanced Approach
Dockens, A. L., & Shelton, K. (2025) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-5102-5.ch013 · Cited by 5
Castro, M. P., & Castillo-Martínez, I. M. (2026) · Frontiers in Education · doi:10.3389/feduc.2026.1729156 · Cited by 3
viet, Q. P., Nguyen, D. M., Xiem, N. T., & Nguyen, D. N. (2026) · Data in Brief · doi:10.1016/j.dib.2026.112952 · Cited by 3
“The development of AI has made it necessary for teaching approaches and students’ learning methods to change in order to adapt [ 2 ].”
Reyes-Sánchez, J. D. J. (2026) · IGI Global eBooks · doi:10.4018/979-8-3373-3638-1.ch015 · Cited by 1
Metadata
- Authors
- Martin, A. F., Tubaltseva, S., Harrison, A., & Rubin, G. J.
- Affiliations
- King's College London (GB)
- School of Liberal Arts, Richmond American University London, London W4 5AN, UK
- Corresponding author
- Martin, Alex F.
- Venue
- Behavioral Sciences
- Publisher
- MDPI AG
- Published
- 2025-06-12
- Volume
- 15
- Issue
- 6
- Pages
- 808
- ISSN
- 2076-328X (ISSN-L)
- PMC
- PMC12189063
- OpenAlex
- W4411284328
- Cited by
- 16
- References
- 25
- Citations by year
- 2026: 15 · 2025: 2
- FWCI
- 67.38
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Behavioral Sciences (publishedVersion, cc-by)
- Preprints.org (acceptedVersion, cc-by)
- PubMed (publishedVersion)
- PubMed Central (submittedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Research Portal (King's College London) (submittedVersion, cc-by)
- Funders
- Institute of Psychiatry, Psychology and Neuroscience at King’s College London’s Strategic College Teaching Fund (NIHR200890, NIHR207394)
- National Institute for Health and Care Research (NIHR200890, NIHR207394)
- Topics
- Social Sciences › Education › Higher Education Practises and Engagement
- Social Sciences › Education › Innovative Education and Learning Practices
- Social Sciences › Education › Higher Education Learning Practices
- Keywords
- generative AI, coursework assessment, academic integrity, critical AI literacy, participatory co-design, authentic assessment, AI-assisted writing, ethical reflection
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.