CoChat
Sign Up
Cited by 21Artificial Intelligence in EducationArtificial Intelligence in Healthcare and EducationInnovations in Medical Education

Enhancing professional communication training in higher education through artificial intelligence(AI)-integrated exercises: study protocol for a randomised controlled trial

Open access
Meinlschmidt, Gunther, Koc, Sara, Boerner, Emma, Tegethoff, Marion, Simacek, Thomas, Schirmer, Liam, Schneider, Michael·2025·BMC Medical Education
DOI 10.1186/s12909-025-07307-3PMC PMC12123891
Open this paper in CoChat — free accountOpen PDF

Abstract

Background: Effective communication skills are fundamental for health care professionals, yet conventional training methods face challenges in scalability and accessibility due to resource constraints. The emergence of artificial intelligence (AI), particularly generative AI, offers innovative ways for enhancing communication skills training by simulating realistic conversational scenarios and providing personalised, adaptive feedback. This manuscript is presented as study protocol for a cluster-randomised controlled trial that aims at evaluating the efficacy of an AI-supported higher education training protocol incorporating generative AI exercises to enhance communication competencies among psychology students. Methods: In this cluster-randomised controlled trial, psychology students enrolled in communication skill seminars at a medium sized university in a medium sized German city will participate. Classes will be assigned within a parallel group design to the AI condition (AI-enhanced exercises alongside teaching-as-usual, TAU, that includes classical exercises) or the control condition (TAU only). Additional non-randomised comparison classes will comprise students with TAU only, but not be part of main analyses. The primary outcome is the change in communication skills from baseline, assessed through questions reflecting the communication techniques emphasised in the training. Secondary outcomes include communication skills, self-efficacy and self-concept, motivation, attitudes toward AI, user experience with the AI tool, student evaluations of course quality, and feasibility aspects such as uptake and usability. Data will be collected via online surveys and the university’s teaching platform. Statistical analyses will employ mixed models to evaluate the intervention’s impact. Discussion: This study will provide empirical evidence on the effectiveness and feasibility of integrating AI into higher education communication skills training. Successful integration of AI-enhanced training could revolutionise educational practices by offering scalable, accessible, and personalised learning experiences. The findings may have broader implications for incorporating AI tools in various educational and professional training contexts, while addressing ethical considerations and promoting responsible use of AI in education. Trial registration: This trial has been pre-registered on the Open Science Framework (OSF) under identifier ‘th6f4’.

AI snapshot

Written from the paper’s full text.

TL;DR

This is a prospective study protocol for a cluster-randomised controlled trial evaluating whether AI-enhanced communication skills exercises improve communication competencies among psychology students compared with teaching-as-usual. No participant data have been collected at the time of writing; the paper reports planned objectives, design, outcomes, and analyses.

Key findings

  • The trial’s primary hypothesis is that AI-enhanced communication skills exercises will be linked to greater improvements in communication skills than conventional teaching methods only.
  • Secondary hypotheses concern higher communication skills self-efficacy and self-concept, motivation, more positive attitudes toward AI, more positive course-quality evaluations, high usability/user experience, and higher uptake of AI-enhanced exercises than conventional exercises.
  • The target is to enrol at least 50 participants per condition, recruiting from at least 10 seminars across two semesters, with randomisation at the seminar/class level.
  • A pilot phase during winter-term 2024/2025 was conducted to test technical feasibility, finalise instruments, and refine prompts; pilot participant-level data will not enter main analyses.

Methods

  • Design: cluster-randomised controlled trial with parallel groups; classes are randomised to AI-enhanced exercises alongside teaching-as-usual (TAU) or TAU only, with additional non-randomised TAU-only comparison classes not included in main analyses.
  • Participants: psychology students enrolled in communication skill seminars in a Bachelor of Science Psychology program at a medium-sized German university, aged at least 18, with sufficient German proficiency; recruitment runs from summer-term 2025 to winter-term 2025/2026.
  • Outcomes and analysis: primary outcome is change in communication skills from baseline to end of seminar, assessed via questions reflecting trained communication techniques; secondary outcomes include self-efficacy, self-concept, motivation, AI attitudes, usability, course evaluations, and feasibility. Mixed-effects models will account for class-level clustering, with intent-to-train and per-protocol analyses and covariates such as baseline scores.

Limitations

  • The paper states that reliance on AI technology means technical failures could impact learning and study outcomes, and that AI-generated interactions may not fully replicate human-to-human communication nuances.
  • The paper states that HAWKI currently supports text-based interactions only, so all AI exercises are text-based; voice or multimodal AI could better capture subtlety, tone, and emotional expression.
  • The paper states that focusing on psychology students at a single institution may limit generalisability; expanding to multiple institutions and diverse student populations would strengthen the evidence base.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1186/s12909-025-07307-3

Cites

30 of 30 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 20 cited sources checked · 0 supported · 0 flagged

  • Premisecitation

    Self-efficacy and positive self-concept can significantly predict persistence, engagement, and actual skill performance.

    “Research in educational psychology indicates that self-efficacy (i.e., confidence in one’s ability to perform a task) and a positive self-concept in a given domain can significantly predict persistence, engagement, and actual skill performance [16].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [16] Robbins SB, Lauver K, Le H, Davis D, Langley R, Carlstrom A. Do Psychosocial and Study Skill Factors Predict College Outcomes? A Meta-Analysis Psychological Bulletin. 2004;130:261–88. source

      Not checked in the free run.

  • Study design and conductcitation

    Each student's interaction with the AI-system is recorded and can be analysed using criteria derived from motivational interviewing best practices (MITI 4.2) by human raters and AI.

    “Each student’s interaction with the AI-system is recorded and can be analysed using criteria derived from motivational interviewing best practices (Motivational Interviewing Treatment Integrity, MITI, 4.2) [32] by human raters and AI.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [32] Moyers TB, Manuel JK, Ernst D. Motivational Interviewing Treatment Integrity Coding manual 4.2.1. Unpublished Manual. 2014. https://motivationalinterviewing.org/sites/default/files/miti4_2.pdf. Accessed 13 May 2025.

      Not checked in the free run.

  • Premisecitation

    Motivation and attitudes, including attitudes toward new technologies such as AI, can influence the degree of effort and openness a learner devotes to improving communication skills.

    “Similarly, motivation and attitudes, including attitudes toward new technologies such as AI, can influence the degree of effort and openness a learner devotes to improving communication skills [17].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [17] Parker K, Nunns M, Xiao Z, Ford T, Ukoumunne OC. Characteristics and practices of school-based cluster randomised controlled trials for improving health outcomes in pupils in the United Kingdom: a methodological systematic review. BMC Med Res Methodol. 2021;21:152. https://doi.org/10.1186/s12874-021-01348-0. source

      Not checked in the free run.

  • Study design and conductreasoning

    Raters responsible for evaluating communication skills exercises and correctness of free-text responses are blinded to participant condition to maintain data integrity and minimise bias.

    “The blinding strategy for this study includes rater blinding to maintain the integrity of the data and minimise bias.”
    • Also stated in: § Allocation concealment/blinding · plain
      “Raters responsible for evaluating communication skills exercises and correctness of free-text responses will be blinded to the condition to which the participants were assigned.”
    Quote in the paperSubstantial claim
  • Premisecitation

    Authenticity of AI interactions can be limited if AI-generated text fails to capture nuance of human responses or emotional expressions.

    “First, the authenticity of interactions can be limited if the AI’s generated text fails to capture the nuance of human responses or emotional expressions [12].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [12] Bender EM, Gebru T, McMillan-Major A, Shmitchell S. On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? In: Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency. Virtual Event Canada: ACM; 2021. p. 610–23. source

      Not checked in the free run.

  • Classified as probably not a substantial claim. 34
  • May combine two claims. 25

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given

    The pilot data served psychometric purposes; the self-developed communication-skills competence test was piloted and refined where required.

    “The pilot data also served psychometric purposes. We piloted the self-developed communication-skills competence test and refined it where required.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    Appropriate covariates, such as baseline scores and relevant demographic factors, are included in the models to adjust for potential confounding.

    “Appropriate covariates, such as baseline scores and relevant demographic factors, will be included in the models to adjust for potential confounding.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Study design and conductcitation

    Cluster-level randomisation preserves ecological validity and reduces risk of cross-condition interference compared to individual-level randomisation.

    “By implementing a cluster-level randomisation (i.e., randomising entire classes), we preserve ecological validity – each class follows a uniform teaching approach – and reduce risk of cross-condition interference [19].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [19] Vu T, Magis-Weinberg L, Jansen BRJ, Van Atteveldt N, Janssen TWP, Lee NC, et al. Motivation-Achievement Cycles in Learning: a Literature Review and Research Agenda. Educ Psychol Rev. 2022;34:39–71. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Premisecitation

    Both educators and learners must develop new literacies and competencies to effectively and ethically use AI tools.

    “Recent discussions highlight that both educators and learners must develop new literacies and competencies to effectively and ethically use these tools [5].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [5] Kasneci E, Sessler K, Küchemann S, Bannert M, Dementieva D, Fischer F, et al. ChatGPT for good? On opportunities and challenges of large language models for education. Learn Individ Differ. 2023;103:102274. https://doi.org/10.1016/j.lindif.2023.102274. source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Kobayashi et al. found significant improvements in physicians' communication skills, empathy, and reductions in burnout scores post-training with AI-based video analysis.

    “The study found significant improvements in physicians’ communication skills, empathy, and reductions in burnout scores post-training, indicating the potential benefits of integrating AI in clinical communication training.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [7] Kobayashi M, Katayama M, Hayashi T, Hashiyama T, Iyanagi T, Une S, et al. Effect of multimodal comprehensive communication skills training with video analysis by artificial intelligence for physicians on acute geriatric care: a mixed-methods study. BMJ Open. 2023;13. https://doi.org/10.1136/bmjopen-2022-06547. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    Data validation procedures include range checks and consistency checks to identify irregularities.

    “Data validation procedures will include range checks and consistency checks to identify any irregularities.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Interpretationreasoning

    The AI intervention is designed to enhance communication skills of psychology students, which is crucial in their professional training and future clinical interactions.

    “However, the intervention is designed to enhance the communication skills of psychology students, which is crucial in their professional training and future clinical interactions.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductown data

    A single-semester pilot study was conducted during winter-term 2024/2025 to determine technical feasibility, finalise study instruments, and rehearse organisational procedures.

    “Before initiating the main trial, we conducted a single-semester pilot study phase (during winter-term 2024/2025) to determine the technical feasibility of the AI-supported exercises, finalise the study instruments, and rehearse all organisational procedures.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

    Interpretationnone given

    The authenticity of AI-generated interactions may not fully replicate the nuances of human-to-human communication.

    “Additionally, the authenticity of AI-generated interactions, while improving, may not fully replicate the nuances of human-to-human communication.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationreasoning

    This approach ensures that all users, irrespective of their technical background, can efficiently engage with the AI without extensive training.

    “This approach ensures that all users, irrespective of their technical background, can efficiently engage with the AI without extensive training.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    The COVID-19 pandemic accelerated the adoption of digital learning tools, highlighting the potential of AI to maintain educational continuity and support remote learning, even in complex skill domains like communication.

    “The coronavirus disease 2019 pandemic accelerated the adoption of digital learning tools, highlighting the potential of AI to maintain educational continuity and support remote learning, even in complex skill domains like communication [4].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [4] Jiang R. How does artificial intelligence empower EFL teaching and learning nowadays? A review on artificial intelligence in the EFL context. Front Psychol. 2022;13. https://doi.org/10.3389/fpsyg.2022.1049401. source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Meta-analytic findings indicate that simulations are among the most effective means to facilitate learning of complex skills across domains, with technology-based and scaffolded simulations showing robust positive effects on learning outcomes.

    “Meta-analytic findings indicate that simulations are among the most effective means to facilitate the learning of complex skills across domains, with technology-based and scaffolded simulations showing robust positive effects on learning outcomes [11].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [11] Chernikova O, Heitzmann N, Stadler M, Holzberger D, Seidel T, Fischer F. Simulation-Based Learning in Higher Education: A Meta-Analysis. Rev Educ Res. 2020;90(4):499–541. https://doi.org/10.3102/0034654320933544. source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Proficient communication enables accurate assessment, enhances client engagement, and contributes to positive treatment outcomes.

    “Proficient communication enables accurate assessment, enhances client engagement, and contributes to positive treatment outcomes.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [1] Maguire P, Pitceathly C. Key communication skills and how to acquire them. BMJ. 2002;325:697. https://doi.org/10.1136/bmj.325.7366.697. source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Use of virtual standardised patients for mental health education with AI-driven tools supported experiential learning and skill transfer to real clinical settings.

    “Emerging evidence indicated that the use of virtual standardised patients for mental health education with AI-driven tools supported experiential learning and skill transfer to real clinical settings [10].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [10] Reger GM, Norr AM, Gramlich MA, Buchman JM. Virtual Standardized Patients for Mental Health Education. Curr Psychiatry Rep. 2021;23:57. https://doi.org/10.1007/s11920-021-01273-5. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductown data

    The set of questions for the primary outcome was piloted during the study pilot phase and adapted if necessary.

    “We use data collected during a study pilot phase during winter-term 2024/25, to pilot the set of questions and adapt the assessment if necessary.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    An AI-enabled virtual reality simulation improved interprofessional communication knowledge and self-efficacy among nursing students.

    “Similarly, an AI-enabled virtual reality simulation improved interprofessional communication knowledge and self-efficacy among nursing students, although the AI’s human-like features received lower ratings, suggesting areas for enhancement [8].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [8] Liaw SY, Tan JZ, Lim S, Zhou W, Yap J, Ratan R, et al. Artificial intelligence in virtual reality simulation for interprofessional communication training: Mixed method study. Nurse Educ Today. 2023;122. https://doi.org/10.1016/j.nedt.2023.105718. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductreasoning

    Missing data are minimised by online data collection and handled by mixed models using all available data points.

    “To minimise missing data, we are employing online data collection, which facilitates real-time entry and immediate correction of inconsistencies, enhancing data completeness. Additionally, we are implementing mixed models in our statistical analyses, which effectively handle missing data by using all available data points to estimate model parameters.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductnone given

    Randomisation is conducted at the seminar level by an independent party using coin-flip or computer-generated randomisation to ensure unbiased allocation.

    “Randomisation for the study will be conducted at the seminar level, with each seminar class being randomised according to the scheme outlined in Fig. 1. The randomisation process will be conducted by an independent party to ensure unbiased allocation, using coin-flip or computer-generated randomisation procedures, to determine to which condition (Condition 1 or Condition 2; Cluster Types A and B,”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Interpretationreasoning

    This approach is designed to maximise accessibility and effectiveness of training, adapting to diverse learning preferences.

    “This approach is designed to maximise the accessibility and effectiveness of the training, adapting to the diverse learning preferences and requirements of the students.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Large language models may inadvertently reproduce societal biases inherent in training data.

    “Second, managing biases is paramount, as large language models may inadvertently reproduce societal biases inherent in the training data [14].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [14] Hastings J. Preventing harm from non-conscious bias in medical generative AI. The Lancet Digital Health. 2024;6:e2-3. source

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning

    The output of the AI intervention is primarily educational and does not directly contribute to clinical decision-making.

    “The output of the AI intervention is primarily educational and does not directly contribute to clinical decision-making.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    The trial has been pre-registered on OSF under identifier 'th6f4'.

    “This trial has been pre-registered on the Open Science Framework (OSF) under identifier ‘th6f4’.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationreasoning

    The study focuses on psychology students at a single institution, which may limit the generalisability of the findings.

    “Moreover, the study focuses on psychology students at a single institution, which may limit the generalisability of the findings.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Technical constraints including stability of platform and need for real-time feedback require robust system monitoring.

    “Third, technical constraints – including the stability of the platform and the need for real-time feedback – require robust system monitoring.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [15] Andersen ES, Birk-Korch JB, Hansen RS, Fly LH, Röttger R, Arcani DMC, et al. Monitoring performance of clinical artificial intelligence in health care: a scoping review. JBI Evidence Synthesis. 2024. https://doi.org/10.11124/JBIES-24-00042. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    The trial is registered in the Open Science Framework (OSF) under identifier 'th6f4'.

    “This trial has been registered in the Open Science Framework (OSF) under identifier ‘th6f4’ on 29–11–2024, last update on 11–04–2025”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductown data

    During the pilot, the HAWKI platform was stress-tested under routine classroom conditions, with active sampling of transcripts, logging of response latencies, and flagging of AI failures.

    “During the pilot we stress-tested the HAWKI platform under routine classroom conditions. We actively sampled transcripts, logged response latencies and flagged any instance in which the AI (i) failed to adopt the instructed patient role, (ii) produced technically or ethically inappropriate feedback, or (iii) deviated from the task instructions.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Interpretationreasoning

    The present work lays essential groundwork for eventually creating fully multimodal AI training environments that more closely approximate real-world professional interactions.

    “Thus, the present work lays essential groundwork for eventually creating fully multimodal AI training environments that more closely approximate the complexity of real-world professional interactions.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    System prompts are included in the exercise protocol only if during the pilot phase they consistently elicited the intended user interactions.

    “System prompts are included in the exercise protocol only if during the pilot phase, they consistently elicited the intended user interactions.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductreasoning

    The finalised prompts remain unchanged during the main study, except for critical fixes, ensuring consistent AI behaviour and feedback for all participants.

    “While AI outputs were actively monitored and analysed through targeted sampling during a pilot phase during winter-term 2024/2025 (see above), the finalised prompts remain unchanged during the main study, except for critical fixes (e.g., to address system outages). This ensures consistent AI behaviour and feedback for all participants.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductreasoning

    Prompts that did not consistently elicit intended interactions were iteratively revised or discarded; only prompts with stable performance were retained and frozen for the main trial.

    “Prompts that did not consistently elicit the intended interactions were iteratively revised or discarded. Only prompts that demonstrated stable performance across all monitored sessions were retained, and we froze this final prompt set for the main trial (changes during the main trial are limited to critical bug-fixes).”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    Both intent-to-train (ITT) and per-protocol analyses are conducted to ensure robustness and comprehensive interpretation of results.

    “Both intent-to-train (ITT) and per-protocol analyses will be conducted to ensure robustness and comprehensive interpretation of the results.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    AI technologies (generative AI, LLMs) can simulate realistic conversations, provide instant feedback, and adapt to individual learning needs, offering accessible, flexible, and personalised learning opportunities.

    “These AI technologies can simulate realistic conversations, provide instant feedback, and adapt to individual learning needs, thereby offering accessible, flexible, and personalised learning opportunities [2, 3].”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [2] Tlili A, Shehata B, Adarkwah MA, Bozkurt A, Hickey DT, Huang R, et al. What if the devil is my guardian angel: ChatGPT as a case study of using chatbots in education. Smart Learn Environ. 2023;10:15. https://doi.org/10.1186/s40561-023-00237-x. source

      Not checked in the free run.

    • cites [3] Yusuf A, Pervin N, Román-González M. Generative AI and the future of higher education: a threat to academic integrity or reformation? Evidence from multicultural perspectives. Int J Educ Technol High Educ. 2024;21:21. https://doi.org/10.1186/s41239-024-00453-6. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Interpretationnone given

    The exercises shall allow repeated practice and receiving iterative feedback from the AI, enhancing the learning process.

    “The exercises shall allow repeated practice and receiving iterative feedback from the AI, enhancing the learning process.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationreasoning

    No significant physical or psychological risks are anticipated for participants; potential harms are minimal, primarily possible discomfort or stress.

    “Given the nature of this educational intervention study, no significant physical or psychological risks are anticipated for participants. The potential harms are minimal, primarily related to possible discomfort or stress associated with the exercises or evaluations.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductreasoning

    It is not feasible to blind participants to the type of intervention.

    “Given the nature of the study, it is not feasible to blind participants to the type of intervention.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    Between-class comparisons account for clustering at the class level using mixed-effects models.

    “For the main effect analyses, mixed-effects models will be employed. Between-class comparisons will be made, accounting for clustering at the class level.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductnone given

    The target sample size is at least 50 participants per condition to ensure robust statistical power for detecting medium effect sizes, allow stratified analyses, and enhance generalisability.

    “With regard to target sample size, our goal is to enrol at least 50 participants per condition to ensure robust statistical power for detecting medium effect sizes, allow stratified analyses, and to enhance the generalisability of our findings.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisecitation

    Communication competencies involve not only demonstration of specific conversational techniques but also psychological determinants underlying skill acquisition and performance.

    “Communication competencies involve not only the demonstration of specific conversational techniques but also the psychological determinants underlying skill acquisition and performance.”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [16] Robbins SB, Lauver K, Le H, Davis D, Langley R, Carlstrom A. Do Psychosocial and Study Skill Factors Predict College Outcomes? A Meta-Analysis Psychological Bulletin. 2004;130:261–88. source

      Not checked in the free run.

    • cites [17] Parker K, Nunns M, Xiao Z, Ford T, Ukoumunne OC. Characteristics and practices of school-based cluster randomised controlled trials for improving health outcomes in pupils in the United Kingdom: a methodological systematic review. BMC Med Res Methodol. 2021;21:152. https://doi.org/10.1186/s12874-021-01348-0. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    A rule is implemented so that if an instructor teaches multiple classes in one semester, the difference in the number of classes assigned to Cluster Type A vs. Cluster Type B does not exceed one.

    “while a rule is implemented so that, if a given instructor teaches multiple classes in one semester, the difference in the number of classes assigned to Cluster Type A vs. Cluster Type B does not exceed one.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductreasoning

    The study does not have an independent data monitoring committee or steering committee, due to its educational research context and low-risk nature.

    “This study will not have an independent data monitoring committee (DMC) due to its educational research context and relatively low-risk nature.”
    • Also stated in: § Composition, roles, and responsibilities of committees · plain
      “There is no steering committee or data monitoring committee overseeing the trial at this stage.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationnone given

    Reliance on AI technology means technical failures could impact the learning experience and study outcomes.

    “The reliance on AI technology means that any technical failures could impact the learning experience and study outcomes.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given

    The system is designed to be user-friendly, requiring no specific technical expertise beyond proficiency with Stud.IP.

    “The system is designed to be user-friendly, requiring no specific technical expertise beyond proficiency with the online learning platform Stud.IP, which is routinely used in classes at the university.”
    Quote in the paperNot a substantial claim
42 of 42

  • The paper relies on this without support.

    Conventional communication skills training methods often require substantial time, resources, and access to trained facilitators, potentially limiting scalability and accessibility.

    “While these methods are valuable, they often require substantial time, resources, and access to trained facilitators, potentially limiting scalability and accessibility.”

  • The paper relies on this without support.

    The HAWKI tool has previously shown reliable performance.

    “we chose the well-established HAWKI tool which has previously shown reliable performance.”

  • The paper relies on this without support.

    Ensuring realistic role-play scenarios depends on high-quality AI outputs and carefully designed system prompts that capture the nuances of interpersonal communication.

    “First, ensuring realistic role-play scenarios depends on high-quality AI outputs and carefully designed system prompts that capture the nuances of interpersonal communication.”
3 of 3

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Large language models have shown the potential to be interactive mentors and address medical students' diverse questions, uncertainties, and concerns in a personalized manner.

    “large language models (LLMs) have shown the potential to be interactive mentors and address medical students’ diverse questions, uncertainties, and concerns in a personalized manner [33].”

    From Development of a Clinical Clerkship Mentor Using Generative AI and Evaluation of Its Effectiveness in a Medical Student Trial Compared to Student Mentors: 2-Part Comparative Study (2025) · cites this paper as [33]

    The Background (page unavailable) says LLMs can simulate conversations, provide instant feedback, and adapt to individual learning needs, supporting the potential for interactive, personalised learning. It does not describe LLMs as mentors addressing medical students’ diverse questions, uncertainties, and concerns; the study protocol concerns communication training for psychology students. (LLM, conf=0.86)

    Experimental — accuracy not yet measured

Cites

Flagged first, then by citations

Research works printed but not linked: 1

  1. Kobayashi M, Katayama M, Hayashi T, Hashiyama T, Iyanagi T, Une S, et al. Effect of multimodal comprehensive communication skills training with video analysis by artificial intelligence for physicians on acute geriatric care: a mixed-methods study. BMJ Open. 2023;13. https://doi.org/10.1136/bmjopen-2022-06547. · DOI

Cited by

Sorted by citations

Related

Authors
Meinlschmidt, G., Koc, S., Boerner, E., Tegethoff, M., Simacek, T., Schirmer, L., & Schneider, M.
Affiliations
  • University of Basel (CH)
  • Universität Trier (DE)
  • RWTH Aachen University (DE)
Corresponding author
Meinlschmidt, Gunther
ORCID
Venue
BMC Medical Education
Publisher
Springer Science and Business Media LLC
Published
2025-05-30
Volume
25
Issue
1
Pages
804-804
ISSN
1472-6920 (ISSN-L)
DOI
10.1186/s12909-025-07307-3
PMC
PMC12123891
OpenAlex
W4410902394
Semantic Scholar
ea084090a9cbc22c834938b1b92b1bfe82216968
Cited by
15
References
29
Citations by year
2026: 8 · 2025: 7
FWCI
15.46
Type
journal article
Language
en
Open access
Licence
Locations
  • BMC Medical Education (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • RWTH Publications (RWTH Aachen) (submittedVersion, other-oa)
Funders
  • Teaching Incentives Fund of Trier University (TIF2024_01)
  • Universität Trier
Topics
  • Physical Sciences › Computer Science › Information Systems › Artificial Intelligence in Education
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Innovations in Medical Education
Keywords
artificial intelligence in education, generative AI, communication skills training, psychology students, health care professional communication, adaptive feedback, self-efficacy, attitudes toward AI, higher education
MeSH terms
Artificial Intelligence, Communication, Germany, Humans, Randomized Controlled Trials as Topic, Self Efficacy

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-08.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-08.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review