Guidelines: The do’s, don’ts and don’t knows of direct observation of clinical skills in medical education
Open accessThe DOI matches its Crossref record; 34 of the 34 references it cites were found in a registry or an index.
Abstract
Introduction Direct observation of clinical skills is a key assessment strategy in competency-based medical education. The guidelines presented in this paper synthesize the literature on direct observation of clinical skills. The goal is to provide a practical list of Do’s, Don’ts and Don’t Knows about direct observation for supervisors who teach learners in the clinical setting and for educational leaders who are responsible for clinical training programs. Methods We built consensus through an iterative approach in which each author, based on their medical education and research knowledge and expertise, independently developed a list of Do’s, Don’ts, and Don’t Knows about direct observation of clinical skills. Lists were compiled, discussed and revised. We then sought and compiled evidence to support each guideline and determine the strength of each guideline. Results A final set of 33 Do’s, Don’ts and Don’t Knows is presented along with a summary of evidence for each guideline. Guidelines focus on two groups: individual supervisors and the educational leaders responsible for clinical training programs. Guidelines address recommendations for how to focus direct observation, select an assessment tool, promote high quality assessments, conduct rater training, and create a learning culture conducive to direct observation. Conclusions High frequency, high quality direct observation of clinical skills can be challenging. These guidelines offer important evidence-based Do’s and Don’ts that can help improve the frequency and quality of direct observation.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Motivational processes affecting learning.
Dweck, C. S. (1986) · American Psychologist · doi:10.1037/0003-066x.41.10.1040 · Cited by 6,971
The assessment of clinical skills/competence/performance
Miller, G. E. (1990) · Academic Medicine · doi:10.1097/00001888-199009000-00045 · Cited by 4,883
Shute, V. J. (2008) · Review of Educational Research · doi:10.3102/0034654307313795 · Cited by 4,631
Feedback and Self-Regulated Learning: A Theoretical Synthesis
Butler, D. L., & Winne, P. H. (1995) · Review of Educational Research · doi:10.3102/00346543065003245 · Cited by 3,334
Competency-based medical education: Theory to practice.
Frank, J. R., Snell, L. S., Cate, O. T., Holmboe, E. S., Carraccio, C., Swing, S. R., Harris, P., Glasgow, N. J., Campbell, C., Dath, D., et al. (2010) · Medical Teacher · doi:10.3109/0142159x.2010.501190 · Cited by 2,883
Self-citationMcCambridge, J., Witton, J., & Elbourne, D. R. (2014) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2013.08.015 · Cited by 2,748 · cited 1×
“One of the benefits of direct observation is the opportunity to see how learners approach their authentic clinical work, but the benefits may be offset if the act of observation alters performance [128].”
BackgroundValidating the interpretations and uses of test scores
Kane, M. T. (2013) · Journal of Educational Measurement · doi:10.1111/jedm.12000 · Cited by 2,110
Steinert, Y., Mann, K., Centeno, A., Dolmans, D., Spencer, J., Gelula, M., & Prideaux, D. (2006) · Medical Teacher · doi:10.1080/01421590600902976 · Cited by 1,417
Assessing professional competence: from methods to programmes
Van Der Vleuten, C. P. M., & Schuwirth, L. W. T. (2005) · Medical Education · doi:10.1111/j.1365-2929.2005.02094.x · Cited by 1,268
Memory and Metamemory Considerations in the Training of Human Beings
Bjork, R. A. (1994) · Metacognition · doi:10.7551/mitpress/4561.003.0011 · Cited by 1,084
Programmatic assessment: From assessment of learning to assessment for learning
Schuwirth, L. W. T., & Van der Vleuten, C. P. M. (2011) · Medical Teacher · doi:10.3109/0142159x.2011.565828 · Cited by 902 · cited 2×
“tests of medical knowledge, simulated encounters, direct observations) in a program of assessment will provide a more well-rounded evaluation of the learner’s competence than any single assessment tool [133, 134].”
BackgroundA model for programmatic assessment fit for purpose.
van der Vleuten, C. P. M., Schuwirth, L. W. T., Driessen, E. W., Dijkstra, J., Tigelaar, D., Baartman, L. K. J., & van Tartwijk, J. (2012) · Medical Teacher · doi:10.3109/0142159x.2012.652239 · Cited by 892
Steinert, Y., Mann, K., Anderson, B., Barnett, B. M., Centeno, A., Naismith, L., Prideaux, D., Spencer, J., Tullo, E., Viggiano, T., et al. (2016) · Medical Teacher · doi:10.1080/0142159x.2016.1181851 · Cited by 845
Mattar, S. G., Alseidi, A. A., Jones, D. B., Jeyarajah, D. R., Swanstrom, L. L., Aye, R. W., Wexner, S. D., Martinez, J. M., Ross, S. B., Awad, M. M., et al. (2013) · Annals of Surgery · doi:10.1097/sla.0b013e3182a191ca · Cited by 788
Frank, J. R., Mungroo, R., Ahmad, Y., Wang, M., De Rossi, S., & Horsley, T. (2010) · Medical Teacher · doi:10.3109/0142159x.2010.500898 · Cited by 781 · cited 1×
“While direct observation of clinical skills is a key assessment strategy in competency-based medical education, it has always been essential to health professions education to ensure that all graduates are competent in essential domains [1, 2].”
BackgroundThe Mini-CEX: A Method for Assessing Clinical Skills
Norcini, J. J., Blank, L. L., Duffy, F. D., & Fortna, G. S. (2003) · Annals of Internal Medicine · doi:10.7326/0003-4819-138-6-200303180-00012 · Cited by 730 · cited 2×
“Potential approaches might be to target faculty to encourage direct observation of short encounters (thus minimizing the impact on efficiency) and to ground the direct observation in the faculty’s daily work [66, 129].”
BackgroundButler, R. (1987) · Journal of Educational Psychology · doi:10.1037/0022-0663.79.4.474 · Cited by 690
Ericsson, K. A. (2015) · Academic Medicine · doi:10.1097/acm.0000000000000939 · Cited by 542
Competency-based medical education in postgraduate medical education
Iobst, W. F., Sherbino, J., Cate, O. T., Richardson, D. L., Dath, D., Swing, S. R., Harris, P., Mungroo, R., Holmboe, E. S., Frank, J. R., et al. (2010) · Medical Teacher · doi:10.3109/0142159x.2010.500709 · Cited by 478 · cited 1×
“rect observation is an increasingly emphasized assessment method [3, 4] in which learners (medical students, graduate or postgraduate trainees) are observed by a supervisor while engaging in meaningful, authentic, realistic patient care and clinical activities [4, 5].”
Self-citationEntrustment Decision Making in Clinical Training
ten Cate, O., Hart, D., Ankel, F., Busari, J., Englander, R., Glasgow, N., Holmboe, E., Iobst, W., Lovell, E., Snell, L. S., et al. (2016) · Academic Medicine · doi:10.1097/acm.0000000000001044 · Cited by 478 · cited 1×
“Autonomy is reinforced when learners who are observed performing clinical activities with competence are granted the right to perform these activities with greater independence in subsequent encounters [53].”
BackgroundSelf-citationGuidelines: The do’s, don’ts and don’t knows of feedback for clinical education.
Lefroy, J., Watling, C., Teunissen, P. W., & Brand, P. (2015) · Perspectives on Medical Education · doi:10.1007/s40037-015-0231-7 · Cited by 347 · cited 3×
“Programs should also ensure their culture promotes learner receptivity to feedback by giving learners personal agency over their learning and ensuring longitudinal relationships between learners and their supervisors [14, 26].”
BackgroundMethodSargeant, J., Lockyer, J., Mann, K., Holmboe, E., Silver, I., Armson, H., Driessen, E., MacLeod, T., Yen, W., Ross, K., et al. (2015) · Academic Medicine · doi:10.1097/acm.0000000000000809 · Cited by 306
Self-citationProgrammatic assessment of competency-based workplace learning: when theory meets practice
Bok, H. G., Teunissen, P. W., Favier, R. P., Rietbroek, N. J., Theyse, L. F., Brommer, H., Haarhuis, J. C., van Beukelen, P., van der Vleuten, C. P., & Jaarsma, D. A. (2013) · BMC Medical Education · doi:10.1186/1472-6920-13-123 · Cited by 303 · cited 3×
“Learners perceive a conflict between direct observation as an educational tool and as an assessment method [57, 60].”
BackgroundLearning from clinical work: the roles of learning cues and credibility judgements
Watling, C., Driessen, E., van der Vleuten, C. P. M., & Lingard, L. (2012) · Medical Education · doi:10.1111/j.1365-2923.2011.04126.x · Cited by 261 · cited 1×
“Using assessors with content expertise and clinical acumen in the specific skill(s) being assessed is also important because learners are more likely to find feedback from these individuals credible and trustworthy [20, 64].”
BackgroundThe Effect of Bedside Case Presentations on Patients' Perceptions of Their Medical Care
Lehmann, L. S., Brancati, F. L., Chen, M.-C., Roter, D., & Dobs, A. S. (1997) · New England Journal of Medicine · doi:10.1056/nejm199704173361606 · Cited by 243
Feedback and the educational alliance: Examining credibility judgements and their consequences.
Telio, S., Regehr, G., & Ajjawi, R. (2016) · Medical Education · doi:10.1111/medu.13063 · Cited by 231
Faculty staff perceptions of feedback to residents after direct observation of clinical skills
Kogan, J. R., Conforti, L. N., Bernabeo, E. C., Durning, S. J., Hauer, K. E., & Holmboe, E. S. (2012) · Medical Education · doi:10.1111/j.1365-2923.2011.04137.x · Cited by 180
Self-citationStudent perceptions of assessment and feedback in longitudinal integrated clerkships
Bates, J., Konkin, J., Suddards, C., Dobson, S., & Pratt, D. (2013) · Medical Education · doi:10.1111/medu.12087 · Cited by 145 · cited 2×
“Longitudinal relationships afford learners the opportunity to have someone witness their learning progression and provide feedback in the context of a broader view of them as a learner [25].”
BackgroundKennedy, T. J. T., Regehr, G., Baker, G. R., & Lingard, L. A. (2009) · Medical Education · doi:10.1111/j.1365-2923.2009.03382.x · Cited by 141 · cited 1×
“Learners value autonomy and progressive independence [40, 41].”
Hauer, K. E., Holmboe, E. S., & Kogan, J. R. (2011) · Medical Teacher · doi:10.3109/0142159x.2010.507710 · Cited by 118
Self-citationSchultz, K. W., Kirby, J., Delva, D., Godwin, M., Verma, S., Birtwhistle, R., Knapper, C., & Seguin, R. (2004) · BMC Medical Education · doi:10.1186/1472-6920-4-12 · Cited by 103 · cited 2×
“in front of the patient undermines the patient’s therapeutic alliance with them, creates a tense atmosphere, and limits the ability to ask questions [73, 74].”
Understanding responses to feedback: the potential and limitations of regulatory focus theory
Watling, C., Driessen, E., van der Vleuten, C. P. M., Vanstone, M., & Lingard, L. (2012) · Medical Education · doi:10.1111/j.1365-2923.2012.04209.x · Cited by 99
Rules of Engagement: Residents’ Perceptions of the In-Training Evaluation Process
Watling, C. J., Kenyon, C. F., Zibrowski, E. M., Schulz, V., Goldszmidt, M. A., Singh, I., Maddocks, H. L., & Lingard, L. (2008) · Academic Medicine · doi:10.1097/acm.0b013e318183e78c · Cited by 86 · cited 1×
“While learners desire meaningful feedback focused on authentic clinical performance, [31, 39] they commonly experience tension between valuing direct observation as useful to learning and wanting to be autonomous and efficient [42].”
BackgroundWeston, P. S. J., & Smith, C. A. (2014) · Medical Teacher · doi:10.3109/0142159x.2013.836267 · Cited by 67 · cited 1×
“Although some learners find direct observation useful [24, 55], many view it (largely independent of the assessment tool used) as a ‘tick-box exercise’ or a curricular obligation [33, 56].”
Background
Cited by
Entrustment decision making: extending miller’s pyramid.
ten Cate, O., Carraccio, C., Damodaran, A., Gofton, W., Hamstra, S. J., Hart, D. E., Richardson, D., Ross, S., Schultz, K., Warm, E. J., et al. (2021) · Academic Medicine · doi:10.1097/acm.0000000000003800 · Cited by 160
Examining the educational impact of the mini-CEX: a randomised controlled study
Martinsen, S. S. S., Espeland, T., Berg, E. A. R., Samstad, E., Lillebo, B., & Slørdahl, T. S. (2021) · BMC Medical Education · doi:10.1186/s12909-021-02670-3 · Cited by 45
“This is encouraging, as concerns have been raised over supervisors basing their assessments of trainees’ clinical skills on proxy information, such as inferring history takings skills based on the case presentation [31, 32].”
BackgroundHauer, K. E., Chang, A., van Schaik, S. M., Lucey, C., Cowell, T., & Teherani, A. (2023) · Teaching and Learning in Medicine · doi:10.1080/10401334.2022.2111570 · Cited by 33
Self-citationEvaluating feedback comments in entrustable professional activities: a cross-sectional study
Andreou, V., Peters, S., Eggermont, J., & Schoenmakers, B. (2024) · Journal of Medical Education and Curricular Development · doi:10.1177/23821205241275810 · Cited by 8
Ohlsson, L., Abrandt Dahlgren, M., Kågedal, K., Setterud, H., & Tamás, É. (2024) · Journal of Problem Based Learning in Higher Education · doi:10.54337/ojs.jpblhe.v12i1.8316 · Cited by 3
Metadata
- Authors
- Kogan, J. R., Hatala, R., Hauer, K. E., & Holmboe, E.
- Venue
- Perspectives on Medical Education
- Publisher
- Ubiquity Press, Ltd.
- Published
- 2017-09-27
- Volume
- 6
- Issue
- 5
- Pages
- 286-305
- ISSN
- 2212-2761 (ISSN-L)
- 2212-277X
- OpenAlex
- W2757583948
- Cited by
- 155
- References
- 34
- FWCI
- 10.37
- Type
- journal article
- Language
- en
- Open access
- diamond
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.