CoChat

The Health Communications Assessment Checklist: a teaching tool for uncovering bias in the age of AI

Open access
Ofrane, Rebecca H., Canada, Taliah, Birnbaum, Amanda S. · 2026 · Frontiers in Public Health

The DOI matches its Crossref record; 17 of the 17 references it cites were found in a registry or an index; 6 of the 12 sources its claims cite were checked for support.

Abstract

Introduction: The COVID-19 pandemic highlighted significant failures in United States public health communications, which were often perceived as unclear and biased, particularly among racialized and marginalized groups. As generative artificial intelligence (genAI) becomes integrated into public health practice, there is an urgent need to train the workforce to recognize and mitigate the societal biases these tools can reproduce. This study describes the development and classroom testing of a tool designed to teach these critical evaluation skills. Methods: We developed the “Health Communications Assessment Checklist,” a 14-question pedagogical tool based on the CommunicateHealth Framework for Equity-Centered Health Communication and the CDC’s Guiding Principles for health equity. The checklist was tested during the 2024–2025 academic year in two graduate-level courses ( n = 55 students) at a large public university. The single-lesson exercise required students to use ChatGPT-4o to generate health communication content and then apply the checklist to critique the output for clarity, relatability, actionability, and inclusivity. Results: Post-exercise surveys and classroom discussions indicated positive student engagement and self-reported skill-building in critical assessment. A key finding was that students leaned heavily on their diverse individual identities and lived experiences to identify subtle biases that the AI generated. Furthermore, the checklist acted as a self-assessment tool, helping students identify gaps in their own knowledge of communication implementation. Feedback led to iterative improvements of the tool. Conclusion: The Health Communications Assessment Checklist is a simple, modifiable, and promising pedagogical tool for public health education. By centering equity and reflexivity, it helps prepare the future workforce to critically manage AI-generated content, ensuring health communications are inclusive and effective in promoting health equity.

Overview

CoChat Summary

TL;DR

The paper describes the development and classroom testing of the Health Communications Assessment Checklist, a 14-question pedagogical tool for teaching students to critique AI-generated health communications for bias. Tested in two graduate public health courses, it was associated with positive engagement, self-reported skill-building, and iterative refinement of the checklist.

Key findings

  • The checklist was developed as a 14-question tool based on CommunicateHealth’s Framework for Equity-Centered Health Communication and the CDC’s Guiding Principles, focused on communications that are clear, relatable, actionable, and inclusive.
  • In testing during the 2024–2025 academic year across two graduate courses (n = 55 students; 49 eligible survey responses), students leaned heavily on their diverse identities and lived experiences to identify subtle biases in AI-generated content.
  • Student feedback indicated the checklist also acted as a self-assessment tool, helping them identify gaps in their own knowledge of communication implementation; suggestions focused more on how to implement recommended actions than on whether bias was present.
  • Students suggested adding an explicit accuracy and fact-checking prompt, and some expressed concern about others using generative AI without critical assessment skills.

Methods

  • The checklist was developed in the 2024–25 academic year from a review of existing tools, primarily CommunicateHealth’s equity-centered framework and CDC’s equity-centered communication principles.
  • It was tested in two graduate public health courses: Social and Behavioral Sciences in Health, with two in-person sections (n = 40), and Culminating Seminar, with one online section (n = 15), totaling n = 55 students.
  • The single-lesson exercise had students use ChatGPT-4o to generate health communication content, then apply the checklist to critique output for clarity, relatability, actionability, and inclusivity; post-exercise online questionnaire responses were analyzed using Braun and Clarke’s thematic analysis framework in Dedoose, with two coders independently reviewing responses and resolving discrepancies through consensus.

Limitations

  • The paper states a relatively small sample size in a single institution, addressed by assessing students on the same activity in different classroom settings and courses.
  • The paper states use of a single AI model—ChatGPT, specifically the free version—was intentional to compare output and student interaction with one model, not to analyze bias across different AI models.
  • The paper states the analysis relies on self-reported student questionnaire responses and classroom discussions rather than objective assessment of skill improvement; it also notes future evaluation could include pre- and post-surveys and longer-term follow-up.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.3389/fpubh.2026.1872506

Cites

17 of 17 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

3 Valid · 0 Not valid · 3 Not checked · 36 Own results and interpretations

6 more claims

Create a free account to see all 42 claims and to verify the 1 reference still open.

See all claims — free account
  • Probably not a substantial claim. 23
  • It may state two separate claims as one. 21

Create a free account to see these claims.

Citations

Cites

Research works printed but not linked: 1

  1. 2025, 5 Years Later: America Looks Back at the Impact of COVID-19

Metadata

Authors
Ofrane, R. H., Canada, T., & Birnbaum, A. S.
Affiliations
Montclair State University (US)
Corresponding author
Ofrane, Rebecca H.
Venue
Frontiers in Public Health
Published
2026-08-27
Volume
14
Pages
1872506-1872506
ISSN
2296-2565 (ISSN-L)
OpenAlex
W7204463223
Cited by
0
References
15
FWCI
0.00
Type
journal article
Language
en
Open access
Locations
  • Frontiers in Public Health (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
Topics
  • Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Public Health Policies and Education
  • Health Sciences › Health Professions › General Health Professions › Health and Medical Education
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
Keywords
generative AI, algorithmic bias, public health education, critical evaluation skills, GPT-4o, inclusivity, health equity, reflexivity, marginalized communities, employee training
MeSH terms
COVID-19, SARS-CoV-2, Generative Artificial Intelligence, Humans, Public Health, United States, Checklist, Health Communication

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.