CoChat

The DOI matches its Crossref record; 24 of the 26 references it cites were found in a registry or an index; 22 of the 37 sources its claims cite were checked for support.

DIALOGUE: A Generative AI-Based Pre–Post Simulation Study to Enhance Diagnostic Communication in Medical Students Through Virtual Type 2 Diabetes Scenarios

Open access

Abstract

DIALOGUE (DIagnostic AI Learning through Objective Guided User Experience) is a generative artificial intelligence (GenAI)-based training program designed to enhance diagnostic communication skills in medical students. In this single-arm pre–post study, we evaluated whether DIALOGUE could improve students’ ability to disclose a type 2 diabetes mellitus (T2DM) diagnosis with clarity, structure, and empathy. Thirty clinical-phase students completed two pre-test virtual encounters with an AI-simulated patient (ChatGPT, GPT-4o), scored by blinded raters using an eight-domain rubric. Participants then engaged in ten asynchronous GenAI scenarios with automated natural-language feedback. Seven days later, they completed two post-test consultations with human standardized patients, again evaluated with the same rubric. Mean total performance increased by 36.7 points (95% CI: 31.4–42.1; p < 0.001), and the proportion of high-performing students rose from 0% to 70%. Gains were significant across all domains, most notably in opening the encounter, closure, and diabetes specific explanation. Multiple regression showed that lower baseline empathy (β = −0.41, p = 0.005) and higher digital self-efficacy (β = 0.35, p = 0.016) independently predicted greater improvement; gender had only a marginal effect. Cluster analysis revealed three learner profiles, with the highest-gain group characterized by low empathy and high digital self-efficacy. Inter-rater reliability was excellent (ICC ≈ 0.90). These findings provide empirical evidence that GenAI-mediated training can meaningfully enhance diagnostic communication and may serve as a scalable, individualized adjunct to conventional medical education.

Overview

CoChat Summary

TL;DR

DIALOGUE, a single-arm pre–post study in 30 clinical-phase Mexican medical students, tested whether ten asynchronous ChatGPT virtual-patient T2DM diagnostic conversations plus automated feedback improved diagnostic communication. Total rubric scores rose from 49.96 ± 9.72 to 86.70 ± 17.56 (Δ = 36.74, p < 0.001, d = 2.58), with gains across all eight domains.

Key findings

  • Pre–post total performance increased by 36.74 points (95% CI: 31.39–42.09; p < 0.001; Cohen’s d = 2.58), and the proportion of high-performing students rose from 0% to 70%; inter-rater reliability was excellent (ICC ≈ 0.90).
  • Largest domain gains were in “Opening” (Δ = 1.92, d = 2.68), “Closure” (Δ = 1.79, d = 2.07), and “Diabetes specific explanation” (Δ = 2.06, d = 2.95); most improved subitems included teach-back (Δ = 2.12), goal negotiation (Δ = 1.58), and empathetic closure (Δ = 1.77).
  • Multiple regression explained approximately 37% of variance in Δ-scores (R² = 0.43, adjusted R² = 0.37; F(6, 23) = 4.13, p = 0.004); lower baseline empathy (β = −0.41, p = 0.005) and higher digital self-efficacy (β = 0.35, p = 0.016) independently predicted greater improvement, while gender had only a marginal bivariate effect (p = 0.044) that was not significant after adjustment.
  • K-means clustering (k = 3) identified three learner profiles: Cluster A (n = 10; Δ = 58.7 ± 10.2), Cluster B (n = 12; Δ = 33.4 ± 11.5), and Cluster C (n = 8; Δ = 16.2 ± 9.3), with between-cluster differences significant (p < 0.001); 93.3% (28/30) rated the GenAI sessions “very useful” or “extremely useful,” and 86.7% (26/30) were willing to integrate AI-based simulation into future training.

Methods

  • Prospective, single-arm, pre–post intervention conducted May–June 2025 at FESI-UNAM; 41 students assessed for eligibility, 32 enrolled, and 30 completed the full protocol (two lost to follow-up). Participants were undergraduate medical students in clinical semesters 2–7 recruited by convenience sampling.
  • Three phases: pre-test with two AI-simulated T2DM diagnostic encounters (ChatGPT GPT-4o-mini per methods; abstract says GPT-4o), asynchronous training consisting of a 20-min prompt engineering workshop and 40-min diagnostic communication workshop followed by ten independent ChatGPT simulated conversations with a “FEEDBACK” prompt, and post-test seven days later with two live human standardized-patient consultations lasting 5–7 min each, scored by blinded raters.
  • Primary outcome was a modified Kalamazoo Essential Elements Communication Checklist (Adapted) with eight domains scored on a 5-point Likert scale; analyses used paired t-tests or Wilcoxon signed-rank tests, Cohen’s d or rank-based r, ICCs, multiple linear regression with AIC stepwise selection, k-means clustering (k = 3), and inductive thematic analysis of over 300 AI-generated feedback responses.

Limitations

  • Stated by the authors: absence of a control group limits causal inference; short duration precludes conclusions about long-term retention or clinical transfer; single-institution deployment limits generalizability; AI feedback was not independently validated by clinical educators; LLM stochasticity may affect reproducibility; no comparison to faculty-led instruction or downstream clinical outcomes; participants were aware of study phase, potentially introducing performance expectancy bias.
  • Evident from the text (not framed as limitations by the authors): small completer sample (n = 30) and convenience sampling; post-test surveys rely on self-report; the sample was 73.3% female, though the authors state this reflects institutional demographics; baseline Scenario 1 vs Scenario 2 total scores differed modestly (p = 0.05), which the authors interpret as possible procedural learning or familiarity with the simulation format.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.3390/ejihpe15080152

Cites

24 of 24 references verified0 citation errors · 0 not checkable

Statistics

1 tests recomputed · 0 with the reported and recomputed p on opposite sides of .05

Figures

Figure 1 Participant flow through the DIALOGUE pre–post study, based on the 2025 CONSORT diagram. The figure outlines the number of students assessed for eligibility, enrolled, exposed to the AI-based remote training intervention, and completing the SP post-test. Final analyses were conducted on all participants who completed the protocol (Hopewell et al., 2025).
Figure 1 Participant flow through the DIALOGUE pre–post study, based on the 2025 CONSORT diagram. The figure outlines the number of students assessed for eligibility, enrolled, exposed to the AI-based remote training intervention, and completing the SP post-test. Final analyses were conducted on all participants who completed the protocol (Hopewell et al., 2025).CC BY · PubMed Central
Figure 2 Radar plot comparing mean baseline diagnostic communication scores across eight rubric domains for Scenario 1 and Scenario 2. Each axis represents one domain from the adapted Kalamazoo rubric: (D1) relationship and rapport, (D2) opening the encounter, (D3) information gathering, (D4) exploring patient perspective, (D5) information sharing, (D6) plan negotiation, (D7) closure, and (D8) diabetes specific explanation. Colored lines indicate mean Likert scale scores (range 1–5): green for Scenario 1 and purple for Scenario 2. Pre-intervention profiles were broadly similar across domains.
Figure 2 Radar plot comparing mean baseline diagnostic communication scores across eight rubric domains for Scenario 1 and Scenario 2. Each axis represents one domain from the adapted Kalamazoo rubric: (D1) relationship and rapport, (D2) opening the encounter, (D3) information gathering, (D4) exploring patient perspective, (D5) information sharing, (D6) plan negotiation, (D7) closure, and (D8) diabetes specific explanation. Colored lines indicate mean Likert scale scores (range 1–5): green for Scenario 1 and purple for Scenario 2. Pre-intervention profiles were broadly similar across domains.CC BY · PubMed Central
Figure 3 Radar plot comparing mean post-intervention diagnostic communication scores across eight rubric domains for Scenario 1 and Scenario 2. Each axis represents one domain from the adapted Kalamazoo rubric: (D1) relationship and rapport, (D2) opening the encounter, (D3) information gathering, (D4) exploring patient perspective, (D5) information sharing, (D6) plan negotiation, (D7) closure, and (D8) diabetes specific explanation. Colored lines indicate mean Likert scale scores (range 1–5): green for Scenario 1 and purple for Scenario 2. Post-intervention profiles show consistent improvement across domains.
Figure 3 Radar plot comparing mean post-intervention diagnostic communication scores across eight rubric domains for Scenario 1 and Scenario 2. Each axis represents one domain from the adapted Kalamazoo rubric: (D1) relationship and rapport, (D2) opening the encounter, (D3) information gathering, (D4) exploring patient perspective, (D5) information sharing, (D6) plan negotiation, (D7) closure, and (D8) diabetes specific explanation. Colored lines indicate mean Likert scale scores (range 1–5): green for Scenario 1 and purple for Scenario 2. Post-intervention profiles show consistent improvement across domains.CC BY · PubMed Central
Figure 4 Radar plot comparing mean post-intervention diagnostic communication scores across eight rubric domains for Scenario 1 and Scenario 2. Each axis represents one domain from the adapted Kalamazoo rubric: (D1) relationship and rapport, (D2) opening the encounter, (D3) information gathering, (D4) exploring patient perspective, (D5) information sharing, (D6) plan negotiation, (D7) closure, and (D8) diabetes specific explanation. Colored lines indicate mean Likert scale scores (range 1–5): orange for Scenario 1 and green for Scenario 2. Post-intervention profiles were broadly consistent across scenarios.
Figure 4 Radar plot comparing mean post-intervention diagnostic communication scores across eight rubric domains for Scenario 1 and Scenario 2. Each axis represents one domain from the adapted Kalamazoo rubric: (D1) relationship and rapport, (D2) opening the encounter, (D3) information gathering, (D4) exploring patient perspective, (D5) information sharing, (D6) plan negotiation, (D7) closure, and (D8) diabetes specific explanation. Colored lines indicate mean Likert scale scores (range 1–5): orange for Scenario 1 and green for Scenario 2. Post-intervention profiles were broadly consistent across scenarios.CC BY · PubMed Central
Figure 5 Violin plot comparing pre- and post-intervention diagnostic communication scores. Each dot represents an individual participant’s total score (range: 23–115), derived from the adapted Kalamazoo rubric. Violin plots illustrate the distribution and density of scores; overlaid boxplots show the median (horizontal bar) and interquartile range (box). Vertical lines indicate score range (minimum–maximum). Orange denotes pre-intervention scores; green denotes post-intervention scores. Asterisks (***) indicate statistically significant improvement after the intervention (paired-samples t-test, p < 0.001).
Figure 5 Violin plot comparing pre- and post-intervention diagnostic communication scores. Each dot represents an individual participant’s total score (range: 23–115), derived from the adapted Kalamazoo rubric. Violin plots illustrate the distribution and density of scores; overlaid boxplots show the median (horizontal bar) and interquartile range (box). Vertical lines indicate score range (minimum–maximum). Orange denotes pre-intervention scores; green denotes post-intervention scores. Asterisks (***) indicate statistically significant improvement after the intervention (paired-samples t-test, p < 0.001).CC BY · PubMed Central
Figure 6 Associations between baseline predictors and improvement in diagnostic-communication scores (Δ-score) among medical students (n = 30). Scatterplots (top and middle rows) display bivariate Pearson correlations between continuous baseline variables and the change in total rubric score (Δ-score = post−pre): (A) age (years), (B) school term, (C) grade point average (GPA), (D) digital self-efficacy (1–5 scale), (E) empathy score (Jefferson Scale of Empathy, JSE), and (F) self-reported motivation (1–10 scale). Solid blue lines represent linear regression trends; shaded areas show 95% confidence intervals. Box-and-whisker plots (bottom row) compare Δ-scores by (G) gender (female vs. male) and (H) prior patient interaction (yes vs. no). Each dot denotes an individual student. Only higher baseline empathy was significantly associated with lower Δ-scores in the bivariate analysis (R2 = −0.45, p = 0.013); gender showed a marginal difference (p = 0.044). No significant bivariate correlations were observed for age, school term, GPA, digital self-efficacy, motivation, or prior patient contact. (In the multivariate model reported in Section 3.4, digital self-efficacy emerged as an additional positive predictor of improvement.).
Figure 6 Associations between baseline predictors and improvement in diagnostic-communication scores (Δ-score) among medical students (n = 30). Scatterplots (top and middle rows) display bivariate Pearson correlations between continuous baseline variables and the change in total rubric score (Δ-score = post−pre): (A) age (years), (B) school term, (C) grade point average (GPA), (D) digital self-efficacy (1–5 scale), (E) empathy score (Jefferson Scale of Empathy, JSE), and (F) self-reported motivation (1–10 scale). Solid blue lines represent linear regression trends; shaded areas show 95% confidence intervals. Box-and-whisker plots (bottom row) compare Δ-scores by (G) gender (female vs. male) and (H) prior patient interaction (yes vs. no). Each dot denotes an individual student. Only higher baseline empathy was significantly associated with lower Δ-scores in the bivariate analysis (R2 = −0.45, p = 0.013); gender showed a marginal difference (p = 0.044). No significant bivariate correlations were observed for age, school term, GPA, digital self-efficacy, motivation, or prior patient contact. (In the multivariate model reported in Section 3.4, digital self-efficacy emerged as an additional positive predictor of improvement.).CC BY · PubMed Central
Figure 7 Forest plot showing effect sizes (Cohen’s d) across the eight diagnostic communication domains. All values reflect the magnitude of improvement from pre- to post-intervention. Dots represent the mean effect size per domain; horizontal lines indicate 95% confidence intervals. All domains demonstrated moderate to large effects, with the highest observed in D8 (diabetes specific explanation), D2 (opening), and D5 (information sharing).
Figure 7 Forest plot showing effect sizes (Cohen’s d) across the eight diagnostic communication domains. All values reflect the magnitude of improvement from pre- to post-intervention. Dots represent the mean effect size per domain; horizontal lines indicate 95% confidence intervals. All domains demonstrated moderate to large effects, with the highest observed in D8 (diabetes specific explanation), D2 (opening), and D5 (information sharing).CC BY · PubMed Central
Figure 8 Cluster heatmap and hierarchical dendrogram of learner profiles based on post-intervention diagnostic communication performance. Each row represents an individual student (N = 30), and each column represents one of the eight communication domains from the adapted Kalamazoo rubric: (D1) relationship and rapport, (D2) opening the encounter, (D3) information gathering, (D4) patient perspective, (D5) information sharing, (D6) plan negotiation, (D7) closure, and (D8) diabetes specific explanation. Scores were standardized (z-score transformation) to enable clustering and visualization. A hierarchical cluster analysis using Euclidean distance and complete linkage identified three distinct learner profiles: Cluster A (n = 10; high responders), Cluster B (n = 12; intermediate responders), and Cluster C (n = 8; low responders). The color gradient reflects standardized performance: red indicates higher relative scores and blue indicates lower relative scores across each domain. This visualization allows comparison of performance patterns across individuals and highlights consistent strengths or deficits within clusters.
Figure 8 Cluster heatmap and hierarchical dendrogram of learner profiles based on post-intervention diagnostic communication performance. Each row represents an individual student (N = 30), and each column represents one of the eight communication domains from the adapted Kalamazoo rubric: (D1) relationship and rapport, (D2) opening the encounter, (D3) information gathering, (D4) patient perspective, (D5) information sharing, (D6) plan negotiation, (D7) closure, and (D8) diabetes specific explanation. Scores were standardized (z-score transformation) to enable clustering and visualization. A hierarchical cluster analysis using Euclidean distance and complete linkage identified three distinct learner profiles: Cluster A (n = 10; high responders), Cluster B (n = 12; intermediate responders), and Cluster C (n = 8; low responders). The color gradient reflects standardized performance: red indicates higher relative scores and blue indicates lower relative scores across each domain. This visualization allows comparison of performance patterns across individuals and highlights consistent strengths or deficits within clusters.CC BY · PubMed Central
Figure 9 Association between baseline communication performance and improvement following the GenAI-based intervention. The scatterplot displays the relationship between pre-intervention rubric scores (x-axis) and Δ-scores (y-axis), calculated as post−pre total rubric score. Each point represents one student (N = 30), color-coded by cluster membership (Cluster A = red, B = pink, C = blue). The dashed regression line represents the best linear fit, and the shaded area indicates the 95% confidence interval. A negative trend was observed, suggesting that students with lower initial scores experienced greater improvements.
Figure 9 Association between baseline communication performance and improvement following the GenAI-based intervention. The scatterplot displays the relationship between pre-intervention rubric scores (x-axis) and Δ-scores (y-axis), calculated as post−pre total rubric score. Each point represents one student (N = 30), color-coded by cluster membership (Cluster A = red, B = pink, C = blue). The dashed regression line represents the best linear fit, and the shaded area indicates the 95% confidence interval. A negative trend was observed, suggesting that students with lower initial scores experienced greater improvements.CC BY · PubMed Central

Verification

Claims the paper makes

17 Valid · 5 Not valid · 11 Not checked · 112 Own results and interpretations

51 more claims · 4 with their cited sources checked

Create a free account to see all 145 claims and every source verdict.

See all claims — free account
  • It may state two separate claims as one. 63
  • Probably not a substantial claim. 38
  • Borderline: one of the tests for a substantial claim is unclear. 1

Create a free account to see these claims.

Statistics that don’t add up

Each reported p-value recomputed from its test statistic and degrees of freedom.

1 of 1

Statistics recomputed

Every reported test result, recomputed from its test statistic and degrees of freedom. A result shows as not adding up only once an independent check has confirmed it.

  • Doesn’t add upF(6, 23) = 4.13reported p = .004 · recomputed p = .006

    “F(6, 23) = 4.13, p = .004”

1 of 1

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

5 of 11 · the rest aren’t loaded

6 more findings

Create a free account to see every finding of this check.

See all findings — free account

Citations

Cites

Show all 26 references

Cited by

See all 18

Metadata

Authors
Suárez-García, R. X., Chavez-Castañeda, Q., Orrico-Pérez, R., Valencia-Marin, S., Castañeda-Ramírez, A. E., Quiñones-Lara, E., Ramos-Cortés, C. A., Gaytán-Gómez, A. M., Cortés-Rodríguez, J., Jarquín-Ramírez, J., Aguilar-Marchand, N. G., Valdés-Hernández, G., Campos-Martínez, T. E., Vilches-Flores, A., Leon-Cabrera, S., Méndez-Cruz, A. R., Jay-Jímenez, B. O., & Saldívar-Cerón, H. I.
Affiliations
  • Universidad Nacional Autónoma de México, Facultad de Estudios Superiores Iztacala (MX)
  • Unidad de Remisión de Diabetes Mellitus (URDM), Facultad de Estudios Superiores-Iztacala, Universidad Nacional Autónoma de México, Tlalnepantla 54090, Mexico
  • Instituto Politécnico Nacional (MX)
  • Carrera de Médico Cirujano, Facultad de Estudios Superiores-Iztacala, Universidad Nacional Autónoma de México, Tlalnepantla 54090, Mexico
Corresponding author
  • Jay-Jímenez, Brenda Ofelia
  • Saldívar-Cerón, Héctor Iván
Venue
European Journal of Investigation in Health, Psychology and Education
Publisher
MDPI AG
Published
2025-08-07
Volume
15
Issue
8
Pages
152
ISSN
  • 2174-8144 (ISSN-L)
  • 2254-9625
OpenAlex
W4413122463
Cited by
18
References
26
Citations by year
2026: 17 · 2025: 1
FWCI
4.33
Type
journal article
Language
en
Open access
Locations
  • European Journal of Investigation in Health Psychology and Education (publishedVersion, cc-by)
  • Preprints.org (acceptedVersion, cc-by)
  • Europe PMC (PubMed Central) (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
  • Consejo Mexiquense de Ciencia y Tecnología (COMECYT) (FICDTEM-2023-131, PAPIIT IA201725, PAPIME 203825, PAPIIT IN201724)
  • Universidad Nacional Autónoma de México (UNAM) (FICDTEM-2023-131, PAPIIT IA201725, PAPIME 203825, PAPIIT IN201724)
Topics
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Social Sciences › Education › Innovations in Medical Education
  • Social Sciences › Education › Simulation-Based Education in Healthcare
Keywords
diagnosis communication, type 2 diabetes, generative AI, medical students, ChatGPT, GPT-4o, empathy, digital self-efficacy, learner profiles, natural language feedback, standardized patients, communication skills training

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-05, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-05 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: DIALOGUE: A Generative AI-Based Pre–Post Simulation Study to Enhance Diagnostic Communication in Medical Students Through Virtual Type 2 Diabetes Scenarios. https://app.cochat.ai/papers/10.3390/ejihpe15080152
The paper
Suárez-García, R. X., Chavez-Castañeda, Q., Orrico-Pérez, R., Valencia-Marin, S., Castañeda-Ramírez, A. E., Quiñones-Lara, E., Ramos-Cortés, C. A., Gaytán-Gómez, A. M., Cortés-Rodríguez, J., Jarquín-Ramírez, J., Aguilar-Marchand, N. G., Valdés-Hernández, G., Campos-Martínez, T. E., Vilches-Flores, A., Leon-Cabrera, S., Méndez-Cruz, A. R., Jay-Jímenez, B. O., & Saldívar-Cerón, H. I. (2025). DIALOGUE: A Generative AI-Based Pre–Post Simulation Study to Enhance Diagnostic Communication in Medical Students Through Virtual Type 2 Diabetes Scenarios. European Journal of Investigation in Health, Psychology and Education. https://doi.org/10.3390/ejihpe15080152

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.