Predicting doctor performance outcomes of curriculum interventions: problem‐based learning and continuing competence
Abstract
Context Problem‐based learning (PBL) is an educational strategy designed to enhance self‐assessment, self‐directed learning and lifelong learning. The present study examines a peer review programme to determine whether the impact of PBL on continuing competence can be detected in practice. Objectives This study aimed to establish whether McMaster graduates who graduated between 1972 and 1991 were any less likely to be identified as having issues of competence by a systematic peer review programme than graduates of other Ontario medical schools. Methods We identified a total of 1166 doctors who had graduated after 1972 and had completed a mandated peer review programme. Of these, 108 had graduated from McMaster and 857 from other Canadian schools. School of graduation was cross‐tabulated against peer rating. A secondary analysis examined predictors of ratings using multiple regression. Results We found that 4% of McMaster graduates and 5% of other graduates were deemed to demonstrate cause for concern or serious concern, and that 24% of McMaster doctors and 28% of other doctors were rated as excellent. These differences were not significant. Multiple regression indicated that certification by family medicine or a specialty, female gender and younger age were all predictors of practice outcomes, but school of graduation was not. Conclusions There is no evidence from this study that PBL graduates are better able to maintain competence than graduates of conventional schools. The study highlights potential problems in attempting to link undergraduate educational interventions to doctor performance outcomes.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
There was no difference between traditional and PBL curriculums in 19 studies.
“there was no difference between traditional and PBL curriculums in another 19 studies [60–78].”
From Effectiveness of problem-based learning methodology in undergraduate medical education: a scoping review (2022) · cites this paper as [69]
Cites
Systematic Review: The Relationship between Clinical Experience and Quality of Health Care
Choudhry, N., Fletcher, R., & Soumerai, S. (2005) · Annals of Internal Medicine · doi:10.7326/0003-4819-142-4-200502150-00008 · Cited by 1,348 · see citing works
Self-Assessment in the Health Professions: A Reformulation and Research Agenda
Eva, K. W., & Regehr, G. (2005) · Academic Medicine · doi:10.1097/00001888-200510001-00015 · Cited by 992 · see citing works
Koh, G. C.-H., Khoo, H. E., Wong, M. L., & Koh, D. (2008) · Canadian Medical Association Journal · doi:10.1503/cmaj.070565 · Cited by 659 · see citing works
Tamblyn, R. M., Abrahamowicz, M., Dauphinee, D. W., Wenghofer, E. F., Jacques, A., KLASS, D. J., Smee, S. M., Blackmore, D. E., Winslade, N. E., Girard, N., et al. (2007) · JAMA · doi:10.1001/jama.298.9.993 · Cited by 449 · see citing works
Self-citationHow Can I Know What I Don't Know? Poor Self Assessment in a Well-Defined Domain
Eva, K. W., Cunnington, J. P. W., Reiter, H., Keane, D. R., & Norman, G. R. (2004) · Advances in Health Sciences Education · doi:10.1023/b:ahse.0000038209.65714.d4 · Cited by 302 · see citing works
Self-citationMultisource feedback in the assessment of physician competencies
Lockyer, J. (2003) · Journal of Continuing Education in the Health Professions · doi:10.1002/chp.1340230103 · Cited by 284 · see citing works
Effect of problem-based, self-directed undergraduate education on life-long learning.
(1993) · Europe PMC (PubMed Central) · Cited by 259 · see citing works
RCT = results confounded and trivial: the perils of grand educational experiments
Norman, G. R. (2003) · Medical Education · doi:10.1046/j.1365-2923.2003.01586.x · Cited by 197 · see citing works
Self-citationGlobal minimum essential requirements: a road towards competence-oriented medical education
Schwarz, M. R., & Wojtczak, A. (2002) · Medical Teacher · doi:10.1080/01421590220120740 · Cited by 154 · see citing works
Certification and Specialization
Norcini, J. J., Kimball, H. R., & Lipner, R. S. (2000) · Academic Medicine · doi:10.1097/00001888-200012000-00016 · Cited by 127 · see citing works
McAuley, R. G., Paul, W. M., Morrison, G. H., Beckett, R., & Goldsmith, C. H. (1984) · Europe PMC (PubMed Central) · Cited by 114 · see citing works
CanMEDS 2000: Extract from the CanMEDS 2000 Project Societal Needs Working Group Report
(2000) · Medical Teacher · doi:10.1080/01421590050175505 · Cited by 109 · see citing works
Tamblyn, R., Abrahamowicz, M., Dauphinee, D., Girard, N., Bartlett, G., Grand'Maison, P., & Brailovsky, C. (2005) · BMJ · doi:10.1136/bmj.38636.582546.7c · Cited by 91 · see citing works
Day, S. C., Norcini, J. J., Webster, G. D., Viner, E. D., & Chirico, A. (1988) · PubMed · Cited by 80 · see citing works
A study of library use in problem-based and traditional medical curricula.
Marshall, J. G., Fitzgerald, D., Busby, L. P., et al. (1993) · PubMed · Cited by 65 · see citing works
Does clinical experience make up for failure to keep up to date?
Norman, G. R. (2005) · BMJ Evidence-Based Medicine · doi:10.1136/ebm.10.3.66-a · Cited by 16 · see citing works
Self-citationEDITORIAL. From the new Editor: continuity and change
Bligh, J. G. (1998) · Medical Education · doi:10.1046/j.1365-2923.1998.00260.x · Cited by 15 · see citing works
Does clinical experience make up for failure to keep up to date?
Geoffrey R. Norman, P., & Kevin W. Eva, P. (2005) · ACP Journal Club · doi:10.7326/acpjc-2005-142-3-a08 · Cited by 10 · see citing works
Where Medical Education matters
Bligh, J. G. (2016) · Medical Education · doi:10.1111/medu.12656 · Cited by 5 · see citing works
Where medical education matters
Bligh, J. (1998) · Medical Education · doi:10.1046/j.1365-2923.1998.00321.x · Cited by 3 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Trullàs, J. C., Blay, C., Sarri, E., & Pujol, R. (2022) · BMC Medical Education · doi:10.1186/s12909-022-03154-8 · Cited by 473 · see citing works
“The research compass”: an introduction to research in medical education: AMEE Guide No. 56
Ringsted, C., Hodges, B., & Scherpbier, A. (2011) · Medical Teacher · doi:10.3109/0142159x.2011.595436 · Cited by 231 · see citing works
Demirören, M., Turan, S., & Öztuna, D. (2016) · Medical Education Online · doi:10.3402/meo.v21.30049 · Cited by 193 · see citing works
Developing medical professionalism in future doctors: a systematic review
Passi, V., Doug, M., Peile, E., Thistlethwaite, J., & Johnson, N. (2010) · International Journal of Medical Education · doi:10.5116/ijme.4bda.ca2a · Cited by 178 · see citing works
- Authors
- Norman, G. R., Wenghofer, E., & Klass, D.
- Affiliations
- McMaster University (CA)
- Laurentian University (CA)
- College of Physicians and Surgeons of Ontario (CA)
- ORCID
- Wenghofer, Elizabeth
- Venue
- Medical Education
- Publisher
- Wiley
- Published
- 2008-07-17
- Volume
- 42
- Issue
- 8
- Pages
- 794-799
- ISSN
- 0308-0110 (ISSN-L)
- 1365-2923
- OpenAlex
- W2048998392
- Cited by
- 40
- References
- 20
- Citations by year
- 2026: 1 · 2024: 2 · 2023: 2 · 2022: 2 · 2021: 1 · 2020: 3 · 2019: 1 · 2017: 1 · 2016: 3 · 2015: 4 · 2014: 1 · 2013: 1 · 2012: 4
- FWCI
- 4.64
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Medical Education (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Innovations in Medical Education
- Social Sciences › Education › Problem and Project Based Learning
- Health Sciences › Medicine › Emergency Medicine › Hospital Admissions and Outcomes
- Keywords
- problem-based learning, continuing competence, peer review, medical graduates, McMaster University, medical curriculum, specialist certification, lifelong learning
- MeSH terms
- Curriculum, Education, Medical, Continuing, Ontario, Peer Group, Physicians, Problem-Based Learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
