BEME systematic review: Predictive values of measurements obtained in medical schools and future performance in medical practice
The DOI matches its Crossref record; 6 of the 6 references it cites were found in a registry or an index.
Abstract
BACKGROUND: Effectiveness of medical education programs is most meaningfully measured as performance of its graduates. OBJECTIVES: To assess the value of measurements obtained in medical schools in predicting future performance in medical practice. SEARCH STRATEGY: The English literature from 1955 to 2004 was searched using MEDLINE, Embase, Cochrane's EPOC (Effective Practice and Organization of Care Group), Controlled Trial databases, ERIC, British Education Index, Psych Info, Timelit, Web of Science and hand searching of medical education journals. INCLUSION & EXCLUSIONS: Selected studies included students assessed or followed up to internship, residency and/or practice after postgraduate training. Assessment systems and instruments studied (Predictors) were the National Board Medical Examinations (NBME) I and II, preclinical and clerkship grade-point average, Observed Standardized Clinical Examination scores and Undergraduate Dean's rankings and honors society. Outcome measures were residency supervisor ratings, NBME III, residency in-training examinations, American Specialty Board examination scores, and on-the-job practice performance. DATA EXTRACTION: Data were extracted by using a modification of the BEME data extraction form study objectives, design, sample variables, statistical analysis and results. All included studies are summarized in a tabular form. DATA ANALYSIS AND SYNTHESIS: Quantitative meta-analysis and qualitative approaches were used for data analysis and synthesis including the methodological quality of the studies included. RESULTS: Of 569 studies retrieved with our search strategy, 175 full text studies were reviewed. A total of 38 studies met our inclusion criteria and 19 had sufficient data to be included in a meta-analysis of correlation coefficients. The highest correlation between predictor and outcome was NBME Part II and NBME Part III, r = 0.72, 95% CI 0.30-0.49 and the lowest between NBME I and supervisor rating during residency, r = 0.22, 95% CI 0.13-0.30. The approach to studying the predictive value of assessment tools varied widely between studies and no consistent approach could be identified. Overall, undergraduate grades and rankings were moderately correlated with internship and residency performance. Performance on similar instruments was more closely correlated. Studies assessing practice performance beyond postgraduate training programs were few. CONCLUSIONS: There is a need for a more consistent and systematic approach to studies of the effectiveness of undergraduate assessment systems and tools and their predictive value. Although existing tools do appear to have low to moderate correlation with postgraduate training performance, little is known about their relationship to longer-term practice patterns and outcomes.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Donabedian, A. (1988) · JAMA · doi:10.1001/jama.260.12.1743 · Cited by 5,895
The assessment of clinical skills/competence/performance
Miller, G. E. (1990) · Academic Medicine · doi:10.1097/00001888-199009000-00045 · Cited by 4,883
A Five-Stage Model of the Mental Activities Involved in Directed Skill Acquisition
Dreyfus, S. E., & Dreyfus, H. L. (1980) · doi:10.21236/ada084551 · Cited by 1,572
Factors associated with success in medical school: systematic review of the literature
Ferguson, E., James, D., & Madeley, L. (2002) · BMJ · doi:10.1136/bmj.324.7343.952 · Cited by 811
The relationship of academic measures in medical school to performance after graduation.
Markert, R. J. (1993) · Academic Medicine · doi:10.1097/00001888-199302000-00027 · Cited by 51
Can Success in the Surgical Residency Be Predicted from Preresidency Evaluation?
KRON, I. L., KAISER, D. L., NOLAN, S. P., RUDOLF, L. E., MULLER, W. H., & JONES, R. S. (1985) · Annals of Surgery · doi:10.1097/00000658-198512000-00005 · Cited by 35
Cited by
Second‐year medical students’ motivational beliefs, emotions, and achievement
Artino, A. R., La Rochelle, J. S., & Durning, S. J. (2010) · Medical Education · doi:10.1111/j.1365-2923.2010.03712.x · Cited by 295
Conducting a best evidence systematic review. Part 1: From idea to data coding. BEME Guide No. 13
Hammick, M., Dornan, T., & Steinert, Y. (2010) · Medical Teacher · doi:10.3109/01421590903414245 · Cited by 230
Triola, M. M., & Burk-Rafel, J. (2023) · Academic Medicine · doi:10.1097/acm.0000000000005227 · Cited by 111
(2015) · Educational Technology & Society · Cited by 68
McNulty, J. A., Espiritu, B. R., Hoyt, A. E., Ensminger, D. C., & Chandrasekhar, A. J. (2015) · Anatomical Sciences Education · doi:10.1002/ase.1442 · Cited by 45
Metadata
- Authors
- Hamdy, H., Prasad, K., Anderson, M. B., Scherpbier, A., Williams, R., Zwierstra, R., & Cuddihy, H.
- Venue
- Medical Teacher
- Publisher
- Informa UK Limited
- Published
- 2006-01
- Volume
- 28
- Issue
- 2
- Pages
- 103-116
- ISSN
- 0142-159X (ISSN-L)
- 1466-187X
- OpenAlex
- W1985089145
- Cited by
- 185
- References
- 6
- FWCI
- 0.97
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: BEME systematic review: Predictive values of measurements obtained in medical schools and future performance in medical practice. https://app.cochat.ai/papers/10.1080/01421590600622723
- The paper
- Hamdy, H., Prasad, K., Anderson, M. B., Scherpbier, A., Williams, R., Zwierstra, R., & Cuddihy, H. (2006). BEME systematic review: Predictive values of measurements obtained in medical schools and future performance in medical practice. Medical Teacher. https://doi.org/10.1080/01421590600622723
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.