Implementing workplace‐based assessment across the medical specialties in the United Kingdom
Open accessAbstract
Objectives To evaluate the reliability and feasibility of assessing the performance of medical specialist registrars (SpRs) using three methods: the mini‐clinical evaluation exercise (mini‐CEX), directly observed procedural skills (DOPS) and multi‐source feedback (MSF) to help inform annual decisions about the outcome of SpR training. Methods We conducted a feasibility study and generalisability analysis based on the application of these assessment methods and the resulting data. A total of 230 SpRs (from 17 specialties) in 58 UK hospitals took part from 2003 to 2004. Main outcome measures included: time taken for each assessment, and variance component analysis of mean scores and derivation of 95% confidence intervals for individual doctors’ scores based on the standard error of measurement. Responses to direct questions on questionnaires were analysed, as were the themes emerging from open‐comment responses. Results The methods can provide reliable scores with appropriate sampling. In our sample, all trainees who completed the number of assessments recommended by the Royal Colleges of Physicians had scores that were 95% certain to be better than unsatisfactory. The mean time taken to complete the mini‐CEX (including feedback) was 25 minutes. The DOPS required the duration of the procedure being assessed plus an additional third of this time for feedback. The mean time required for each rater to complete his or her MSF form was 6 minutes. Conclusions This is the first attempt to evaluate the use of comprehensive workplace assessment across the medical specialties in the UK. The methods are feasible to conduct and can make reliable distinctions between doctors’ performances. With adaptation, they may be appropriate for assessing the workplace performance of other grades and specialties of doctor. This may be helpful in informing foundation assessment.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Peer assessment in clinical training improves interpersonal skills and contributes to readiness for team-based environments.
“Cestone and Michaelsen [95] demonstrated that peer assessment in clinical training improves interpersonal skills and contributes to readiness for teambased environments.”
From Formative assessment and feedback in medical education: A practical guide: AMEE Guide No. 189 (2026) · cites this paper as [95]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
The Mini-CEX: A Method for Assessing Clinical Skills
Norcini, J. J., Blank, L. L., Duffy, F. D., & Fortna, G. S. (2003) · Annals of Internal Medicine · doi:10.7326/0003-4819-138-6-200303180-00012 · Cited by 730 · see citing works
The relationship between competence and performance: implications for assessing practice performance
Rethans, J.-J., Norcini, J. J., Maldonado, M. B., Blackmore, D. E., Jolly, B. C., LaDuca, T., Lew, S. R., Page, G. G., & Southgate, L. J. (2002) · Medical Education · doi:10.1046/j.1365-2923.2002.01316.x · Cited by 405 · see citing works
Use of Peer Ratings to Evaluate Physician Performance
Ramsey, P. G. (1993) · JAMA · doi:10.1001/jama.1993.03500130069034 · Cited by 352 · see citing works
Generalisability: a key to unlock professional assessment
Crossley, J., Davies, H., Humphris, G., & Jolly, B. (2002) · Medical Education · doi:10.1046/j.1365-2923.2002.01320.x · Cited by 169 · see citing works
Self-citationConstruct Validity of the MiniClinical Evaluation Exercise (MiniCEX)
Holmboe, E. S., Huot, S. J., Chung, J., Norcini, J. J., & Hawkins, R. E. (2003) · Academic Medicine · doi:10.1097/00001888-200308000-00018 · Cited by 167 · see citing works
Use of SPRAT for peer review of paediatricians in training
Archer, J. C., Norcini, J. J., & Davies, H. A. (2005) · BMJ · doi:10.1136/bmj.38447.610451.8f · Cited by 140 · see citing works
Current perspectives in assessment: the assessment of performance at work
Norcini, J. J. (2005) · Medical Education · doi:10.1111/j.1365-2929.2005.02182.x · Cited by 131 · see citing works
Assessing the performance of specialist registrars
Wragg, A., Wade, W. B., Fuller, G. N., Cowan, G. O., & Mills, P. G. (2003) · Clinical Medicine · doi:10.7861/clinmedicine.3-2-131 · Cited by 123 · see citing works
Self-citationFrank, J. R., & Langer, B. (2003) · World Journal of Surgery · doi:10.1007/s00268-003-7102-9 · Cited by 110 · see citing works
Crossley, J., Eiser, C., & Davies, H. A. (2005) · Medical Education · doi:10.1111/j.1365-2929.2005.02230.x · Cited by 89 · see citing works
Self-citationUse of peer ratings to evaluate physician performance
Ramsey, P. G. (1993) · JAMA · doi:10.1001/jama.269.13.1655 · Cited by 85 · see citing works
Ratings of the performances of practicing internists by hospital-based registered nurses
Wenrich, M. D., Carline, J. D., Giles, L. M., & Ramsey, P. G. (1993) · Academic Medicine · doi:10.1097/00001888-199309000-00014 · Cited by 85 · see citing works
‘I’m pickin' up good regressions': the governance of generalisability analyses
Crossley, J., Russell, J., Jolly, B. C., Ricketts, C., Roberts, C., Schuwirth, L., & Norcini, J. J. (2007) · Medical Education · doi:10.1111/j.1365-2923.2007.02843.x · Cited by 77 · see citing works
Self-citationExaminer Differences in the Mini-Cex
Norcini, J. J., Blank, L. L., Arnold, G. K., et al. (1997) · Advances in Health Sciences Education · doi:10.1023/a:1009734723651 · Cited by 48 · see citing works
Baltagi, B. H., Song, S. H., & Jung, B. C. (2002) · Econometrics Journal · doi:10.1111/1368-423x.t01-1-00094 · Cited by 42 · see citing works
Outcomes of assessments of registrars in the medical specialties
Tunbridge, M., Dickinson, D., & Swan, P. (2004) · Clinical Medicine · doi:10.7861/clinmedicine.4-1-66 · Cited by 3 · see citing works
Flagged first, then by citations
Research works printed but not linked: 2
- Crossley J, Russell J, Jolly B et al. IÕm pickinÕ up good regressions: the governance of generalisability analyses. Med Educ 2007;41 (10):926-34.
- Holmboe ES, Huot S, Chung J et al. Construct validity of the mini-clinical evaluation exercise (mini-CEX). Acad Med, 2003;78 (8):826-30.
Cited by
Sorted by citations
Massie, J., & Ali, J. M. (2016) · Advances in Health Sciences Education · doi:10.1007/s10459-015-9614-0 · Cited by 147 · see citing works
Driessen, E., & Scheele, F. (2013) · Medical Teacher · doi:10.3109/0142159x.2013.798403 · Cited by 110 · see citing works
Dijksterhuis, M. G. K., Schuwirth, L. W. T., Braat, D. D. M., Teunissen, P. W., & Scheele, F. (2013) · Medical Teacher · doi:10.3109/0142159x.2012.756576 · Cited by 79 · see citing works
Weston, P. S. J., & Smith, C. A. (2014) · Medical Teacher · doi:10.3109/0142159x.2013.836267 · Cited by 67 · see citing works
Examining the educational impact of the mini-CEX: a randomised controlled study
Martinsen, S. S. S., Espeland, T., Berg, E. A. R., Samstad, E., Lillebo, B., & Slørdahl, T. S. (2021) · BMC Medical Education · doi:10.1186/s12909-021-02670-3 · Cited by 45 · see citing works
Poeppelman, R. S., Cho, J., Nachbor, K., Sekhar, T. C., Pruett, J., Baim, A., Strul, S., Barsam, A., Langworthy, B., Waxman, E. L., et al. (2025) · Academic Medicine · doi:10.1097/acm.0000000000005985 · Cited by 4 · see citing works
- Authors
- Wilkinson, J. R., Crossley, J. G. M., Wragg, A., Mills, P., Cowan, G., & Wade, W.
- Affiliations
- Royal College of Physicians (GB)
- University of Sheffield (GB)
- Corresponding author
- Wilkinson, James R
- ORCID
- Wragg, Andrew
- Venue
- Medical Education
- Publisher
- Wiley
- Published
- 2008-03-12
- Volume
- 42
- Issue
- 4
- Pages
- 364-373
- ISSN
- 0308-0110 (ISSN-L)
- 1365-2923
- OpenAlex
- W2118672623
- Cited by
- 272
- References
- 16
- Citations by year
- 2026: 10 · 2025: 9 · 2024: 6 · 2023: 11 · 2022: 8 · 2021: 14 · 2020: 14 · 2019: 14 · 2018: 9 · 2017: 8 · 2016: 17 · 2015: 23 · 2014: 19 · 2013: 31 · 2012: 18
- FWCI
- 20.12
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Medical Education (publishedVersion)
- PubMed (publishedVersion)
- Funders
- Sir John Fisher Foundation
- Topics
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Innovations in Medical Education
- Health Sciences › Medicine › Family Practice › Clinical Reasoning and Diagnostic Skills
- Health Sciences › Medicine › Rehabilitation › Musculoskeletal Disorders and Rehabilitation
- Keywords
- workplace-based assessment, Mini-CEX, multisource feedback, assessment reliability, assessment feasibility, generalizability, standard error of measurement, variance component analysis, medical specialty training
- MeSH terms
- Analysis of Variance, Clinical Competence, Employee Performance Appraisal, Feasibility Studies, Feedback, United Kingdom, Medical Staff, Hospital, Medicine, Specialization, Workplace
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
