Educational impact of in‐training assessment (ITA) in postgraduate medical education: a qualitative study of an ITA programme in actual practice
Abstract
Objectives To investigate the experiences and opinions of programme directors, clinical supervisors and trainees on an in‐training assessment (ITA) programme on a broad spectrum of competence for first year training in anaesthesiology. How does the programme work in practice and what are the benefits and barriers? What are the users' experiences and thoughts about its effect on training, teaching and learning? What are their attitudes towards this concept of assessment? Methods Semistructured interviews were conducted with programme directors, supervisors and trainees from 3 departments. Interviews were audiotaped and transcribed. The content of the interviews was analysed in a consensus process among the authors. Results The programme was of benefit in making goals and objectives clear, in structuring training, teaching and learning, and in monitoring progress and managing problem trainees. There was a generally positive attitude towards assessment. Trainees especially appreciated the coupling of theory with practice and, in general, the programme inspired an academic dialogue. Issues of uncertainty regarding standards of performance and conflict with service declined over time and experience with the programme, and departments tended to resolve practical problems through structured planning. Discussion Three interrelated factors appeared to influence the perceived value of assessment in postgraduate education: (1) the link between patient safety and individual practice when assessment is used as a licence to practise without supervision rather than as an end‐of‐training examination; (2) its benefits to educators and learners as an educational process rather than as merely a method of documenting competence, and (3) the attitude and rigour of assessment practice.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Similar results have been found for in-training assessment programmes during postgraduate medical education.
“Similar results have been found for in-training assessment programmes during postgraduate medical education.43”
From Test-enhanced learning in medical education (2008) · cites this paper as [43]
Cites
The assessment of clinical skills/competence/performance
Miller, G. E. (1990) · Academic Medicine · doi:10.1097/00001888-199009000-00045 · Cited by 4,883 · see citing works
Rasmussen, J. (1983) · IEEE Transactions on Systems, Man, and Cybernetics · doi:10.1109/tsmc.1983.6313160 · Cited by 3,434 · see citing works
The assessment of professional competence: Developments, research and practical implications
Van Der Vleuten, C. P. M. (1996) · Advances in Health Sciences Education · doi:10.1007/bf00596229 · Cited by 1,247 · see citing works
The effect of assessments and examinations on the learning of medical students
Newble, D. I., & Jaeger, K. (1983) · Medical Education · doi:10.1111/j.1365-2923.1983.tb00657.x · Cited by 488 · see citing works
Competency-based Residency Training
Long, D. M. (2000) · Academic Medicine · doi:10.1097/00001888-200012000-00009 · Cited by 271 · see citing works
The rising cost of NIH-funded biomedical research?
Kennedy, T. J. (1990) · Academic Medicine · doi:10.1097/00001888-199002000-00001 · Cited by 262 · see citing works
How Well Do Internal Medicine Faculty Members Evaluate the Clinical Skills of Residents?
NOEL, G. L., Herbers, J. E., Caplow, M. P., Cooper, G. S., Pangaro, L. N., & Harvey, J. (1992) · Annals of Internal Medicine · doi:10.7326/0003-4819-117-9-757 · Cited by 207 · see citing works
A statistical approach to measuring the competence of anaesthetic trainees at practical procedures
Kestin, I. G. (1995) · British Journal of Anaesthesia · doi:10.1093/bja/75.6.805 · Cited by 190 · see citing works
Durning, S. J., Cation, L. J., Markert, R. J., & Pangaro, L. N. (2002) · Academic Medicine · doi:10.1097/00001888-200209000-00020 · Cited by 185 · see citing works
The Certification and Recertification of Doctors: Issues in the Assessment of Clinical Competence
Newble, D. I., Jolly, B. C., & Wakeford, R. E. (1994) · BMJ · doi:10.1136/bmj.309.6961.1096 · Cited by 112 · see citing works
The role of assessment in expanding professional horizons
Ben‐David, M. F. (2000) · Medical Teacher · doi:10.1080/01421590050110731 · Cited by 111 · see citing works
Improving in-training evaluation programs
Turnbull, J. M., Gray, J., & MacFadyen, J. (1998) · Journal of General Internal Medicine · doi:10.1046/j.1525-1497.1998.00097.x · Cited by 99 · see citing works
The inter-rater reliability and internal consistency of a clinical evaluation exercise
Kroboth, F. J., Hanusa, B. H., Parker, S., Coulehan, J. L., Kapoor, W. N., Brown, F. H., Karpf, M., & Levey, G. S. (1992) · Journal of General Internal Medicine · doi:10.1007/bf02598008 · Cited by 82 · see citing works
Clinical Performance Assessments
PETRUSA, E. R. (2002) · Springer international handbooks of education · doi:10.1007/978-94-010-0462-6_26 · Cited by 76 · see citing works
Prescott, L. E., Norcini, J. J., McKinlay, P., & Rennie, J. S. (2002) · Medical Education · doi:10.1046/j.1365-2923.2002.01099.x · Cited by 74 · see citing works
Turnbull, J., MacFadyen, J., van Barneveld, C., et al. (2000) · Journal of General Internal Medicine · doi:10.1046/j.1525-1497.2000.06099.x · Cited by 67 · see citing works
Does a student log provide a means to better structure clinical education?
Dolmans, Schmidt, Beek, V. D., et al. (1999) · Medical Education · doi:10.1046/j.1365-2923.1999.00285.x · Cited by 62 · see citing works
Ringsted, C. V., Østergaard, D., Ravn, L., Pedersen, J., Berlac, P. A., & van der Vleuten, C. P. M. (2003) · Medical Teacher · doi:10.1080/01421590310001605642 · Cited by 48 · see citing works
Self-citationBeyond career choice: the role of learning style analysis in residency training
Baker, J. D., Cooke, J. E., Conroy, R. M., et al. (1988) · Medical Education · doi:10.1111/j.1365-2923.1988.tb00798.x · Cited by 45 · see citing works
Assessment of Clinical Performance: In-Training Evaluation
Turnbull, J., & van Barneveld, C. (2002) · Springer international handbooks of education · doi:10.1007/978-94-010-0462-6_29 · Cited by 42 · see citing works
The marvelous medical education machine or how medical education can be `unstuck' in time
Friedman, C. P. (2000) · Medical Teacher · doi:10.1080/01421590050110786 · Cited by 34 · see citing works
Assessing the New General Competencies for Resident Education
Reisdorff, E. J., Hayes, O. W., Carlson, D. J., et al. (2001) · Academic Medicine · doi:10.1097/00001888-200107000-00023 · Cited by 33 · see citing works
In‐training assessment in postgraduate training for general practice
Hays, R., & Wellard, R. (1998) · Medical Education · doi:10.1046/j.1365-2923.1998.00231.x · Cited by 32 · see citing works
Towards an acceptance of performance assessment
Finucane, P. M., Barron, S. R., Davies, H. A., Hadfield-Jones, R. S., & Kaigas, T. (2002) · Medical Education · doi:10.1046/j.1365-2923.2002.01314.x · Cited by 29 · see citing works
Ringsted, C. V., Østergaard, D., & Scherpbier, A. J. J. A. (2003) · Medical Teacher · doi:10.1080/0142159021000061431 · Cited by 28 · see citing works
Self-citationCohen, J. J. (1999) · Academic Medicine · doi:10.1097/00001888-199902000-00009 · Cited by 28 · see citing works
Ringsted, C. V., Østergaard, D., & van der Vleuten, C. P. M. (2003) · Acta Anaesthesiologica Scandinavica · doi:10.1046/j.1399-6576.2003.00255.x · Cited by 27 · see citing works
Self-citationThe importance of strong evaluation standards and procedures in training residents
Short, J. (1993) · Academic Medicine · doi:10.1097/00001888-199307000-00003 · Cited by 24 · see citing works
Ringsted, C. V., Østergaard, D., & Scherpbier, A. J. J. A. (2002) · Acta Anaesthesiologica Scandinavica · doi:10.1034/j.1399-6576.2002.460910.x · Cited by 15 · see citing works
Self-citationIn‐training assessment – its potential in enhancing clinical teaching
Spike, N., Alexander, H., Elliott, S. L., et al. (2000) · Medical Education · doi:10.1046/j.1365-2923.2000.00755.x · Cited by 15 · see citing works
Will competency assessment improve the training and skills of the trainee anaesthetist?
Patel, B. S., & Feerick, A. E. (2002) · Anaesthesia · doi:10.1046/j.1365-2044.2002.27092.x · Cited by 10 · see citing works
Chapman, M., & Wolff, A. (2002) · Anaesthesia · doi:10.1046/j.1365-2044.2002.27091.x · Cited by 10 · see citing works
UK doctors' attitudes to the General Medical Council's Performance Procedures, 1997-99
McManus, I. C., Winder, B., & Gordon, D. (2001) · Medical Education · doi:10.1046/j.1365-2923.2001.0350s1060.x · Cited by 10 · see citing works
Can resident evaluations demonstrate increases in residentsʼ skills over time?
Yudkowsky, R. (1999) · Academic Medicine · doi:10.1097/00001888-199910000-00056 · Cited by 9 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Test-enhanced learning in medical education
Larsen, D. P., Butler, A. C., & Roediger III, H. L. (2008) · Medical Education · doi:10.1111/j.1365-2923.2008.03124.x · Cited by 610 · see citing works
Programmatic assessment of competency-based workplace learning: when theory meets practice
Bok, H. G., Teunissen, P. W., Favier, R. P., Rietbroek, N. J., Theyse, L. F., Brommer, H., Haarhuis, J. C., van Beukelen, P., van der Vleuten, C. P., & Jaarsma, D. A. (2013) · BMC Medical Education · doi:10.1186/1472-6920-13-123 · Cited by 303 · see citing works
- Authors
- Ringsted, C., Henriksen, A. H., Skaarup, A. M., & Van der Vleuten, C. P. M.
- Affiliations
- Bispebjerg Hospital (DK)
- Corresponding author
- Ringsted, C
- Venue
- Medical Education
- Publisher
- Wiley
- Published
- 2004-05-10
- Volume
- 38
- Issue
- 7
- Pages
- 767-777
- ISSN
- 0308-0110 (ISSN-L)
- 1365-2923
- OpenAlex
- W2081520789
- Cited by
- 41
- References
- 34
- Citations by year
- 2023: 3 · 2022: 1 · 2021: 1 · 2020: 1 · 2019: 2 · 2018: 2 · 2017: 1 · 2015: 2 · 2014: 6 · 2013: 6 · 2012: 1
- FWCI
- 1.38
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Medical Education (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Innovations in Medical Education
- Health Sciences › Medicine › Family Practice › Clinical Reasoning and Diagnostic Skills
- Health Sciences › Medicine › Rehabilitation › Musculoskeletal Disorders and Rehabilitation
- Keywords
- in-training assessment, graduate medical education, anesthesiology education, trainee assessment, program directors, clinical supervisors, trainee experiences, competency assessment, patient safety, assessment standards, trainee supervision, teaching-learning process
- MeSH terms
- Anesthesiology, Attitude of Health Personnel, Clinical Competence, Curriculum, Denmark, Education, Medical, Graduate, Educational Measurement, Evaluation Studies as Topic, Humans, Inservice Training, Program Evaluation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
