Evaluating feedback comments in entrustable professional activities: a cross-sectional study
Open accessAbstract
INTRODUCTION: Competency-based medical education (CBME) has transformed postgraduate medical training, prioritizing competency acquisition over traditional time-based curricula. Integral to CBME are Entrustable Professional Activities (EPAs), that aim to provide high-quality feedback for trainee development. Despite its importance, the quality of feedback within EPAs remains underexplored. METHODS: We employed a cross-sectional study to explore feedback quality within EPAs, and to examine factors influencing length of written comments and their relationship to quality. We collected and analyzed 1163 written feedback comments using the Quality of Assessment for Learning (QuAL) score. The QuAL aims to evaluate written feedback from low-stakes workplace assessments, based on 3 quality criteria (evidence, suggestion, connection). Afterwards, we performed correlation and regression analyses to examine factors influencing feedback length and quality. RESULTS: EPAs facilitated high-quality written feedback, with a significant proportion of comments meeting quality criteria. Task-oriented and actionable feedback was prevalent, enhancing value of low-stakes workplace assessments. From the statistical analyses, the type of assessment tool significantly influenced feedback length and quality, implicating that direct and video observations can yield superior feedback in comparison to case-based discussions. However, no correlation between entrustment scores and feedback quality was found, suggesting potential discrepancies between the feedback and the score on the entrustability scale. CONCLUSION: This study indicates the role of the EPAs to foster high-quality feedback within CBME. It also highlights the multifaceted feedback dynamics, suggesting the influence of factors such as feedback length and assessment tool on feedback quality. Future research should further explore contextual factors for enhancing medical education practices.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Longer feedback is often associated with higher perceived quality.
“Although longer feedback is often associated with higher perceived quality [26-28]”
From AI-Generated Versus Human Supervisor Feedback on Medical Students’ Clinical Clerkship Logs: Cross-Sectional Convergent Mixed Methods Study (2026) · cites this paper as [26]
The Discussion says longer comments tend to be of higher quality. However, the Methods and Results assessed quality using the QuAL score, not participants’ perceptions, so the source does not establish an association with perceived quality. (LLM, conf=0.84)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
A guideline of selecting and reporting intraclass correlation coefficients for reliability research
Koo, T. K., & Li, M. Y. (2016) · Journal of Chiropractic Medicine · doi:10.1016/j.jcm.2016.02.012 · Cited by 29,639 · see citing works
Content Analysis: An Introduction to Its Methodology
Krippendorff, K. (2019) · doi:10.4135/9781071878781 · Cited by 15,448 · see citing works
Elm, E. V., Altman, D. G., Egger, M., Pocock, S. J., Gøtzsche, P. C., & Vandenbroucke, J. P. (2007) · BMJ · doi:10.1136/bmj.39335.541782.ad · Cited by 10,766 · see citing works
Cohen, L., Manion, L., & Morrison, K. (2002) · Routledge eBooks · doi:10.4324/9780203224342 · Cited by 4,944 · see citing works
The Use of Triangulation in Qualitative Research
Carter, N., Bryant-Lukosius, D., DiCenso, A., Blythe, J., & Neville, A. J. (2014) · Oncology Nursing Forum · doi:10.1188/14.onf.545-547 · Cited by 3,915 · see citing works
Competency-based medical education: Theory to practice.
Frank, J. R., Snell, L. S., Cate, O. T., Holmboe, E. S., Carraccio, C., Swing, S. R., Harris, P., Glasgow, N. J., Campbell, C., Dath, D., et al. (2010) · Medical Teacher · doi:10.3109/0142159x.2010.501190 · Cited by 2,883 · see citing works
Sampling Methods in Research Methodology; How to Choose a Sampling Technique for ResearchPreprint
Taherdoost, H. (2016) · SSRN Electronic Journal · doi:10.2139/ssrn.3205035 · Cited by 1,989 · see citing works
Qualitative Research: The essential guide to theory and practice
Savin‐Baden, M., & Major, C. H. (2012) · Cited by 1,642 · see citing works
Carraccio, C. L., Wolfsthal, S. D., Englander, R., Ferentz, K. S., & Martin, C. (2002) · Academic Medicine · doi:10.1097/00001888-200205000-00003 · Cited by 979 · see citing works
ten Cate, O., Chen, H. C., Hoff, R. G., Peters, H., Bok, H. G. J., & van der Schaaf, M. F. (2015) · Medical Teacher · doi:10.3109/0142159x.2015.1060308 · Cited by 792 · see citing works
Core principles of assessment in competency-based medical education.
Lockyer, J., Carraccio, C., Chan, M.-K., Hart, D., Smee, S., Touchie, C., Holmboe, E. S., Frank, J. R., & Collaborators, O. B. O. T. I. (2017) · Medical Teacher · doi:10.1080/0142159x.2017.1315082 · Cited by 493 · see citing works
Assessment, feedback and the alchemy of learning
Watling, C., & Ginsburg, S. (2018) · Medical Education · doi:10.1111/medu.13645 · Cited by 434 · see citing works
The recommended description of an entrustable professional activity: AMEE Guide No. 140
ten Cate, O., & Taylor, D. R. (2020) · Medical Teacher · doi:10.1080/0142159x.2020.1838465 · Cited by 355 · see citing works
Guidelines: The do’s, don’ts and don’t knows of feedback for clinical education.
Lefroy, J., Watling, C., Teunissen, P. W., & Brand, P. (2015) · Perspectives on Medical Education · doi:10.1007/s40037-015-0231-7 · Cited by 347 · see citing works
The Ottawa Surgical Competency Operating Room Evaluation (O-SCORE)
Gofton, W. T., Dudek, N. L., Wood, T. J., Balaa, F. K., & Hamstra, S. J. (2012) · Academic Medicine · doi:10.1097/acm.0b013e3182677805 · Cited by 318 · see citing works
Feedback for learners in medical education: What is known? A scoping review.
Bing-You, R., Hayes, V., Varaklis, K., Trowbridge, R., Kemp, H., & McKelvy, D. (2017) · Academic Medicine · doi:10.1097/acm.0000000000001578 · Cited by 256 · see citing works
Entrustable professional activities in health care education: a scoping review
Shorey, S., Lau, T. C., Lau, S. T., & Ang, E. N. K. (2019) · Medical Education · doi:10.1111/medu.13879 · Cited by 238 · see citing works
Carraccio, C. L., & Englander, R. (2013) · Academic Medicine · doi:10.1097/acm.0b013e318299396f · Cited by 231 · see citing works
Implementing competency-based medical education: Moving forward
Frank, J. R., Snell, L. S., Englander, R., Holmboe, E. S., & Collaborators, O. B. O. T. I. (2017) · Medical Teacher · doi:10.1080/0142159x.2017.1315069 · Cited by 220 · see citing works
Overarching challenges to the implementation of competency-based medical education
Caverzagie, K. J., Nousiainen, M. T., Ferguson, P. C., ten Cate, O., Ross, S., Harris, K. A., Busari, J. O., Bould, M. D., Bouchard, J., Iobst, W. F., et al. (2017) · Medical Teacher · doi:10.1080/0142159x.2017.1315075 · Cited by 209 · see citing works
Twelve tips for the implementation of EPAs for assessment and entrustment decisions
Peters, H., Holzhausen, Y., Boscardin, C., ten Cate, O., & Chen, H. C. (2017) · Medical Teacher · doi:10.1080/0142159x.2017.1331031 · Cited by 180 · see citing works
Learning culture and feedback: an international study of medical athletes and musicians
Watling, C., Driessen, E., van der Vleuten, C. P. M., & Lingard, L. (2014) · Medical Education · doi:10.1111/medu.12407 · Cited by 162 · see citing works
Kogan, J. R., Hatala, R. A., Hauer, K. E., & Holmboe, E. S. (2017) · Perspectives on Medical Education · doi:10.1007/s40037-017-0376-7 · Cited by 161 · see citing works
Schut, S., Driessen, E. W., van Tartwijk, J., van der Vleuten, C. P. M., & Heeneman, S. (2018) · Medical Education · doi:10.1111/medu.13532 · Cited by 141 · see citing works
Nousiainen, M. T., Caverzagie, K. J., Ferguson, P. C., Frank, J. R., & Collaborators, O. B. O. T. I. (2017) · Medical Teacher · doi:10.1080/0142159x.2017.1315077 · Cited by 128 · see citing works
Hauer, K. E., Holmboe, E. S., & Kogan, J. R. (2011) · Medical Teacher · doi:10.3109/0142159x.2010.507710 · Cited by 118 · see citing works
Ferguson, P. C., Caverzagie, K. J., Nousiainen, M. T., Snell, L. S., & Collaborators, O. B. O. T. I. (2017) · Medical Teacher · doi:10.1080/0142159x.2017.1315079 · Cited by 105 · see citing works
APA handbook of research methods in psychology, Vol 3: Data analysis and research publication.
(2012) · American Psychological Association eBooks · doi:10.1037/13621-000 · Cited by 94 · see citing works
The impact of entrustment assessments on feedback and learning: Trainee perspectives
Martin, L. J., Sibbald, M., Vegas, D. B., Russell, D., & Govaerts, M. (2019) · Medical Education · doi:10.1111/medu.14047 · Cited by 87 · see citing works
Cracking the code: residents’ interpretations of written assessment comments
Ginsburg, S., van der Vleuten, C. P. M., Eva, K. W., & Lingard, L. (2017) · Medical Education · doi:10.1111/medu.13158 · Cited by 70 · see citing works
Chan, T. M., Sebok‐Syer, S. S., Sampson, C. S., & Monteiro, S. (2020) · Teaching and Learning in Medicine · doi:10.1080/10401334.2019.1708365 · Cited by 65 · see citing works
Ginsburg, S., Watling, C., Schumacher, D. J., Gingerich, A., & Hatala, R. A. (2021) · Academic Medicine · doi:10.1097/acm.0000000000004089 · Cited by 64 · see citing works
Eeckhout, T., Gerits, M., Bouquillon, D., & Schoenmakers, B. (2016) · Postgraduate Medical Journal · doi:10.1136/postgradmedj-2015-133633 · Cited by 37 · see citing works
Carraccio, C. L., Martini, A., Melle, E. V., & Schumacher, D. J. (2021) · Academic Medicine · doi:10.1097/acm.0000000000004075 · Cited by 25 · see citing works
Bonnie, L. H. A., Visser, M. R. M., Bont, J., Kramer, A. W. M., & van Dijk, N. (2018) · Education for Primary Care · doi:10.1080/14739879.2018.1532773 · Cited by 19 · see citing works
Woods, R. A., Singh, S., Thoma, B., Patocka, C., Cheung, W. J., Monteiro, S., & Chan, T. M. (2022) · Canadian Medical Education Journal · doi:10.36834/cmej.74860 · Cited by 15 · see citing works
What does “Timely” Mean to Residents? Challenging Feedback Assumptions in Postgraduate Education
Lip, A., Watling, C., & Ginsburg, S. (2023) · Perspectives on Medical Education · doi:10.5334/pme.1052 · Cited by 14 · see citing works
Choo, E., Woods, R. A., Walker, M. E., O’Brien, J., & Chan, T. M. (2023) · Canadian Medical Education Journal · doi:10.36834/cmej.75876 · Cited by 14 · see citing works
Andreou, V., Peters, S., Eggermont, J., & Schoenmakers, B. (2024) · BMC Medical Education · doi:10.1186/s12909-024-05530-y · Cited by 13 · see citing works
Bonnie, L. H. A., Nasori, M., Visser, M. R. M., Kramer, A. W. M., & van Dijk, N. (2021) · Education for Primary Care · doi:10.1080/14739879.2021.1951127 · Cited by 7 · see citing works
Flagged first, then by citations
Research works printed but not linked: 1
- [41]Woods R Singh S Thoma B, et al. Validity evidence for the quality of assessment for learning score: a quality metric for supervisor comments in competency based medical education. Can Med Educ J. 2022; 13(6): 19‐ 35. 36440075 PMC9684040
Cited by
Sorted by citations
Bremer, A. E., Scharenborg, S. W. J., Fluit, C. R. M. G., & van de Pol, M. H. J. (2025) · BMC Medical Education · doi:10.1186/s12909-025-07162-2 · Cited by 8 · see citing works
Al‐Diery, T., Marotti, S. B., Sim, Y. T., Jaam, M. J., Rowett, D. S., & Johnson, J. L. (2026) · Perspectives on Medical Education · doi:10.5334/pme.2259 · Cited by 3 · see citing works
Yousuf, A. M., Barakaat, A. A., Tahir, K., Sukhia, R. H., & Fida, M. (2025) · BMC Medical Education · doi:10.1186/s12909-025-07366-6 · Cited by 2 · see citing works
Kondo, T., Donkers, J., Nishigori, H., Rovers, S., & Heeneman, S. (2026) · JMIR Medical Education · doi:10.2196/90064 · Cited by 1 · see citing works
“Although longer feedback is often associated with higher perceived quality [26-28], the extent to which differences in length between AI and human feedback translate into quality advantages remains unclear [18,20].”
The Quality of Written Feedback Provided by Pediatric Hospitalists: A Needs Assessment
Smith, M., Walters, J., Savic, E., Collins, K., Dudas, R. A., Tackett, S. A., & Maniscalco, J. (2025) · Hospital Pediatrics · doi:10.1542/hpeds.2025-008460
Kondo, T., Donkers, J. H. H. L. M., Nishigori, H., Rovers, S. F. E., & Heeneman, S. (2025) · doi:10.2196/preprints.90064
Fernandes, N. V., Neves, C., Donato, H., Pêgo, J. M., & Ribeiro, J. C. (2026) · Medical Education Online · doi:10.1080/10872981.2026.2740871
- Authors
- Andreou, V., Peters, S., Eggermont, J., & Schoenmakers, B.
- Affiliations
- KU Leuven (BE)
- The University of Melbourne (AU)
- Corresponding author
- Andreou, Vasiliki
- Venue
- Journal of Medical Education and Curricular Development
- Publisher
- SAGE Publications
- Published
- 2024-01
- Volume
- 11
- Pages
- 23821205241275810-23821205241275810
- ISSN
- 2382-1205 (ISSN-L)
- PMC
- PMC11437546
- OpenAlex
- W4402759429
- Cited by
- 8
- References
- 40
- Citations by year
- 2026: 4 · 2025: 4
- FWCI
- 3.31
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Journal of Medical Education and Curricular Development (publishedVersion, cc-by-nc)
- PubMed Central (publishedVersion, cc-by-nc)
- PubMed (publishedVersion)
- Lirias (submittedVersion, cc-by-nc)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Minerva Access (University of Melbourne) (submittedVersion, cc-by-nc)
- Funders
- Fonds Wetenschappelijk Onderzoek (Grant [S003219N]-SBO SCAFFOLD)
- Topics
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Innovations in Medical Education
- Social Sciences › Education › Student Assessment and Feedback
- Health Sciences › Medicine › Family Practice › Clinical Reasoning and Diagnostic Skills
- Keywords
- entrustable professional activities, competency-based medical education, graduate medical education, assessment instruments, video observation, case-based discussion, entrustment, workplace-based assessment
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
