The DOI matches its Crossref record.
Performances on the NBME I, II, and III by medical students in the problem-based learning and conventional tracks at the University of New Mexico
Abstract
BACKGROUND: Problem-based learning curricula are growing in popularity, and questions have been raised about the appropriateness of standardized examinations, such as the National Board of Medical Examiners (NBME) Parts I, II, and III examinations, for assessing students in these new curricula. METHOD: Data on students' performances on the NBME I were analyzed for 508 graduates of the conventional track and 167 graduates of the problem-based Primary Care Curriculum (PCC) track at the University of New Mexico School of Medicine from the classes of 1983-1992; on NBME II, for 447 and 144 graduates, respectively (classes of 1983-1991); and on NBME III, for 313 and 100 graduates, respectively (classes of 1983-1989). The analyses also included data on the students' total Medical College Admission Test (MCAT) scores, undergraduate science grade-point averages (SGPAs), and admission subgroups within tracks. The statistical methods included analysis of covariance, Student's t-test, and the Fisher exact test. RESULTS: The students who had requested the PCC track but had been randomized into the conventional track had the highest mean scores on all the study variables (for 34 students, 521 on the NBME I, and for 19 students, 551 on the NMBE III). The high-risk students who had requested but had not been accepted into the PCC track seemed to benefit from the highly structured conventional track with regard to their NMBE I performances (467 for 18 students). The PCC students--both those who had been randomized into the PCC and those who had been selected into the PCC--had significantly lower mean scores on the NBME I (455 for 85 students and 463 for 82 students compared with 505 for the 439 students who chose the conventional track), but significantly higher mean scores on the NBME III (521 for 38 students and 522 for 62 students compared with 483 for the seven high-risk students and 487 for the 276 students who chose the conventional track). For both tracks, strong relationships were found among the scores on the three NBME examinations. For the PCC students, significantly weaker relationships were found between mean SGPAs and mean scores on the NBME I, II, and III. For both tracks, MCAT scores, especially in the lowest and highest ranges, were most predictive of performances on the NBME I and II. CONCLUSION: In the short run, the more teacher-centered and structured conventional curriculum better prepared the students for the NBME I, while in the long run, the more student-centered problem-based curriculum better prepared the students for the NBME III:
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“there was no difference between traditional and PBL curriculums in another 19 studies [60–78].”
From Effectiveness of problem-based learning methodology in undergraduate medical education: a scoping review (2022) · cites this paper as [61]
Citations
Cites
Still being checked.
Cited by
Sorted by citations
Problem-Based Learning: What and How Do Students Learn?
Hmelo‐Silver, C. E. (2004) · Educational Psychology Review · doi:10.1023/b:edpr.0000034022.16470.f3 · Cited by 5,552
Effects of problem-based learning: a meta-analysis
Dochy, F., Segers, M., Van den Bossche, P., & Gijbels, D. (2003) · Learning and Instruction · doi:10.1016/s0959-4752(02)00025-7 · Cited by 1,713
Effectiveness of Problem-based Learning Curricula: Research and Theory
Colliver, J. A. (2000) · Academic Medicine · doi:10.1097/00001888-200003000-00017 · Cited by 917
Effects of Problem-Based Learning: A Meta-Analysis From the Angle of Assessment
Gijbels, D., Dochy, F., Van den Bossche, P., & Segers, M. (2005) · Review of Educational Research · doi:10.3102/00346543075001027 · Cited by 917
Walker, A., & Leary, H. (2009) · Interdisciplinary Journal of Problem-Based Learning · doi:10.7771/1541-5015.1061 · Cited by 581
Trullàs, J. C., Blay, C., Sarri, E., & Pujol, R. (2022) · BMC Medical Education · doi:10.1186/s12909-022-03154-8 · Cited by 473
Albanese, M. (2000) · Medical Education · doi:10.1046/j.1365-2923.2000.00753.x · Cited by 418
Problem-Based Learning: Effects on the Early Acquisition of Cognitive Skill in Medicine
Hmelo‐Silver, C. E. (1998) · Journal of the Learning Sciences · doi:10.1207/s15327809jls0702_2 · Cited by 302
Undergraduate medical education: comparison of problem-based learning and conventional teaching.
Nandi, P. L., Chan, J., Chan, C. P., Chan, P., & CHAN, L. P. (2000) · PubMed · Cited by 265
Antepohl, W., & Herzig, S. (1999) · Medical Education · doi:10.1046/j.1365-2923.1999.00289.x · Cited by 235
Problem-based learning in pre-clinical medical education: 22 years of outcome research
Hartling, L., Spooner, C., Tjosvold, L., & Oswald, A. (2010) · Medical Teacher · doi:10.3109/01421590903200789 · Cited by 191
(2003) · Cited by 183
Hoffman, K., Hosokawa, M., Blake, R., Headrick, L., & Johnson, G. (2006) · Academic Medicine · doi:10.1097/01.acm.0000232411.97399.c6 · Cited by 179
The effect of problem-based learning in nursing education: a meta-analysis
Shin, I.-S., & Kim, J.-H. (2013) · Advances in Health Sciences Education · doi:10.1007/s10459-012-9436-2 · Cited by 149
Essential readings in problem-based learning
Walker, A. (2015) · Purdue University Press eBooks · doi:10.2307/j.ctt6wq6fh · Cited by 110
Problem-Based Learning Outcomes: The Glass Half-Full
Distlehorst, L. H., Dawson, E. A., ROBBS, R. S., & Barrows, H. S. (2005) · Academic Medicine · doi:10.1097/00001888-200503000-00020 · Cited by 130
Blake, R. L., Hosokawa, M. C., & Riley, S. L. (2000) · Academic Medicine · doi:10.1097/00001888-200001000-00017 · Cited by 128
Problem‐based learning: measurable outcomes
Thomas, R. E. (1997) · Medical Education · doi:10.1046/j.1365-2923.1997.00671.x · Cited by 127
Problem‐based learning in academic health education. A systematic literature review
Polyzois, I., Claffey, N. M., & Mattheos, N. (2010) · European Journal Of Dental Education · doi:10.1111/j.1600-0579.2009.00593.x · Cited by 119
Aprendizagem Ativa na Educação em Saúde: Percurso Histórico e Aplicações
de Farias, P. A. M., Martin, A. L. D. A. R., & Cristo, C. S. (2015) · Revista Brasileira de Educação Médica · doi:10.1590/1981-52712015v39n1e00602014 · Cited by 108
Anderson, W. L., Mitchell, S. M., & Osgood, M. P. (2005) · Biochemistry and Molecular Biology Education · doi:10.1002/bmb.2005.49403306387 · Cited by 107
Do Computers Teach Better? A Media Comparison Study for Case-based Teaching in Radiology
Maleck, M., Fischer, M. R., Kammer, B., Zeiler, C., Mangel, E., Schenk, F., & Pfeifer, K.-J. (2001) · Radiographics · doi:10.1148/radiographics.21.4.g01jl091025 · Cited by 90
Instructional design and emerging teaching models in higher education
Enkenberg, J. (2001) · Computers in Human Behavior · doi:10.1016/s0747-5632(01)00021-8 · Cited by 92
Leary, H., Walker, A. E., Shelton, B. E., & Fitt, M. H. (2013) · Interdisciplinary Journal of Problem-based Learning · doi:10.7771/1541-5015.1331 · Cited by 85
Problem-Based Learning in Veterinary Education
Lane, E. A. (2008) · Journal of Veterinary Medical Education · doi:10.3138/jvme.35.4.631 · Cited by 79
Enarson, C. E., & Cariaga-Lo, L. (2001) · Medical Education · doi:10.1046/j.1365-2923.2001.01058.x · Cited by 78
Approaches to evidence-based teaching
Davies, P. R. (2000) · Medical Teacher · doi:10.1080/01421590078751 · Cited by 77
Winning, T. A., & Townsend, G. (2007) · Australian Dental Journal · doi:10.1111/j.1834-7819.2007.tb00458.x · Cited by 68
Kasim, R. M. (1999) · Advances in Health Sciences Education · doi:10.1023/a:1009871001258 · Cited by 55
Proposed model of case based learning for training undergraduate medical student in surgery
Jamkar, A. V., Burdick, W. P., Morahan, P. S., Yemul, V., Sarmukadum, & Singh, G. (2007) · Indian Journal of Surgery · doi:10.1007/s12262-007-0016-2 · Cited by 55
Psychological factors determining success in a medical career: A 10-year longitudinal study
Tartas, M., Walkiewicz, M., Majkowicz, M., & Budziński, W. (2011) · Medical Teacher · doi:10.3109/0142159x.2011.544795 · Cited by 54
Achievement of Students in a Conventional and Problem-Based Learning (PBL) Curriculum
Kaufman, D. M., & Mann, K. V. (1999) · Advances in Health Sciences Education · doi:10.1023/a:1009829831978 · Cited by 50
Assessing the Effectiveness of a New Curriculum: Part I
Dagenais, M. E., Hawley, D., & Lund, J. P. (2003) · Journal of Dental Education · doi:10.1002/j.0022-0337.2003.67.1.tb03618.x · Cited by 47
Meta‐analysis of quasi‐experimental research: are systematic narrative reviews indicated?
Colliver, J. A., Kucera, K., & Verhulst, S. J. (2008) · Medical Education · doi:10.1111/j.1365-2923.2008.03144.x · Cited by 45
Ge, W., Zhu, X.-Y., Lin, J., Jiang, J., Li, T., Lu, Y., Mi, Y., & Tung, T. (2025) · BMC Medical Education · doi:10.1186/s12909-025-06951-z · Cited by 44
Burch, V. C., Sikakana, C. N. T., Yeld, N., Seggie, J., & Schmidt, H. G. (2006) · Advances in Health Sciences Education · doi:10.1007/s10459-006-9006-6 · Cited by 42
Problem‐based learning: does it prepare medical students to become better doctors?
Schmidt, H. G. (1998) · The Medical Journal of Australia · doi:10.5694/j.1326-5377.1998.tb139020.x · Cited by 42
A meta-analysis for comparing effective teaching in clinical education
Huang, P., Haywood, M., O’Sullivan, A., & Shulruf, B. (2019) · Medical Teacher · doi:10.1080/0142159x.2019.1623386 · Cited by 38
Using Problem-Based Learning to Teach Forensic Psychiatry
Schultz-Ross, R. A., & Kline, A. E. (1999) · Academic Psychiatry · doi:10.1007/bf03340034 · Cited by 36
Outcomes in Medical Education: What is the Standard and Outcome of Care Delivered by our Graduates?
Tamblyn, R. M. (1999) · Advances in Health Sciences Education · doi:10.1023/a:1009893715930 · Cited by 34
Competency‐based training and assessment in Australian postgraduate clinical psychology education
Stevens, B. A., Hyde, J., Knight, R., Shires, A. G., & Alexander, R. (2015) · Clinical Psychologist · doi:10.1111/cp.12061 · Cited by 33
Context and Problem-based Learning
Overton, T. (2007) · New Directions in the Teaching of Physical Sciences · doi:10.11120/ndir.2007.00030007 · Cited by 30
Katsikitis, M., Hay, P., Barrett, R. J., & Wade, T. D. (2002) · Educational Psychology · doi:10.1080/01443410220138511 · Cited by 28
Problem-based learning in consultation psychiatry
Yates, W. R., & Gerdes, T. T. (1996) · General Hospital Psychiatry · doi:10.1016/0163-8343(96)00022-9 · Cited by 25
Context and problem-based learning
Overton, T. (2016) · New Directions in the Teaching of Physical Sciences · doi:10.29311/ndtps.v0i3.409 · Cited by 25
First Graduates' Perceptions on a Problem-Based and Task-Based Learning Curriculum
Özan, S., Karademir, S., Gürsel, Y., Taşkıran, C., & Musal, B. (2005) · Education for Health · doi:10.1080/13576280500148007 · Cited by 23
Using Problem-Based Learning to Enhance the Psychosocial Competence of Medical Students
Block, S. D. (1996) · Academic Psychiatry · doi:10.1007/bf03341552 · Cited by 23
Woloschuk, W., Mandin, H., Harasym, P. H., Lorscheider, F. L., & Brant, R. (2004) · Teaching and Learning in Medicine · doi:10.1207/s15328015tlm1602_1 · Cited by 22
Problem-Based Learning: What and How Do
Hmelo‐Silver, C. E. (2004) · Cited by 17
Meaningful occupation for occupational therapy students: a student‐centred curriculum
Salvatori, P. S. (1999) · Occupational Therapy International · doi:10.1002/oti.98 · Cited by 17
It is magic, myth and mindset What is Problem-Based Learning (PBL)?
Kwan, C. (2000) · Cited by 15
Use of remote response devices: an effective interactive method in the long- term learning
Muruzábal, M. M., Etxano, J., Slon, P., Barquín, P. G., Villanueva, A., Bastarrika, G., & Pueyo, J. C. (2014) · European Radiology · doi:10.1007/s00330-014-3468-3 · Cited by 14
Them, C., Schulc, E., Roner, A., & Behrens, J. (2003) · International Journal of Medical Informatics · doi:10.1016/s1386-5056(03)00096-0 · Cited by 12
Kumar, K., & Indurkhya, A. (2004) · Human Pathology · doi:10.1016/j.humpath.2004.09.019 · Cited by 12
Kumar, K. (2003) · Human Pathology · doi:10.1016/s0046-8177(03)00082-0 · Cited by 12
Effects of an applied supplemental course on student performance in elementary physiology.
Richardson, D. R., & Birge, B. (2000) · AJP Advances in Physiology Education · doi:10.1152/advances.2000.24.1.56 · Cited by 11
Casassus, P., Hivon, R., Gagnayre, R., & d’Ivernois, J. F. (1999) · Hematology and Cell Therapy · doi:10.1007/s00282-999-0137-0 · Cited by 10
Gómez‐Esquer, F., Martínez, I. R., Romero, F. M., & Barjola, P. (2009) · REDU Revista de Docencia Universitaria · doi:10.4995/redu.2009.6228 · Cited by 10
Problem-Based Learning in Medical Education
Dwinnell, B. G., & Adams, L. (1998) · Hospital Practice · doi:10.1080/21548331.1998.11443773 · Cited by 10
A META-ANALYSIS OF PROBLEM-BASED LEARNING:
Walker, A. E., Leary, H., & Lefler, M. (2015) · Purdue University Press eBooks · doi:10.2307/j.ctt6wq6fh.25 · Cited by 9
A Comparison of Unguided vs Guided Case-Based Instruction on the Surgery Clerkship
Adamas‐Rappaport, W. J., Waer, A. L., Teeple, M., Benjamin, M. A., Glazer, E. S., Sozanski, J., Poskus, D. M., & Ong, E. (2012) · Journal of surgical education · doi:10.1016/j.jsurg.2012.06.014 · Cited by 8
Shenouda, N. S., Swenson, R. L., & Fournier, J. T. (2003) · Teaching and Learning in Medicine · doi:10.1207/s15328015tlm1504_03 · Cited by 8
Singh, N., Gupta, R., & Mahalakshmi, V. N. (2018) · AJP Advances in Physiology Education · doi:10.1152/advan.00023.2018 · Cited by 7
Troncon, L. E. D. A., Figueiredo, J. F. C., Rodrigues, M. D. L. V., Piccinato, C. E., Peres, L. C., Cianflone, A. R. L., & Colares, M. D. F. A. (2004) · Revista Brasileira de Educação Médica · doi:10.1590/1981-5271v28.2-019 · Cited by 6
Curriculum Type and Sophomore Students' Preparation Time for the Usmle Step 1 Examination
RICHARDS, B. F., & Cariaga-Lo, L. (1994) · Evaluation & the Health Professions · doi:10.1177/016327879401700306 · Cited by 6
Amin, M. S., Zulla, R. T., Gaudet‐Amigo, G., Patterson, S. K., Murphy, N., & Ross, S. (2017) · Journal of Dental Education · doi:10.1002/j.0022-0337.2017.81.1.tb06248.x · Cited by 6
Intimidating Attendings: The Importance of Near-Peer Mentorship During Third-Year Surgical Clerkship
Johnson, B. M., Ayres, J. M., Minchew, H. M., Riffel, J. D., Dixon, K. S., Adkins, S. E., & Berbel, G. (2024) · Journal of Surgical Research · doi:10.1016/j.jss.2024.06.017 · Cited by 5
Naggar, M. A. A. E. A. E., & Maklady, F. A. H. (2014) · Intellectual Property Rights Open Access · doi:10.4172/2375-4516.1000106 · Cited by 5
Araújo, I. D. M., Araújo, R. D. C. T., & Grácio, M. C. C. (2001) · Revista Brasileira de Educação Médica · doi:10.1590/1981-5271v25.3-006 · Cited by 5
Expert Versus Novice Tutors: Impacts on Student Outcomes in Problem-Based Learning
Leary, H., Walker, A. E., Fitt, M. H., & Shelton, B. E. (2009) · Digital Commons - USU (Utah State University) · Cited by 5
To What Degree Do Problem-Based Learning Issues Change With Clinical Experience?
Hueston, W. J., Mallin, R., & Kern, D. H. (2002) · Teaching and Learning in Medicine · doi:10.1207/s15328015tlm1404_3 · Cited by 5
Factors That Influence Performance in a Problem-based Learning Tutorial
Alajmi, N. (2014) · e-publications@bond (Bond University) · Cited by 5
Lecturing skills as predictors of tutoring skills in a problem-based medical curriculum
Kassab, S. E., Hassan, N., Abu-Hijleh, M. F., & Sequeira, R. P. (2016) · Advances in Medical Education and Practice · doi:10.2147/amep.s96481 · Cited by 4
Pragmatic solutions for problems in the undergraduate medical programmes in Pakistan.
Bangash, M. A. (2002) · PubMed · Cited by 4
Teaching in the Classroom: Small Groups
Kelm, D. J., & Niven, A. S. (2019) · Respiratory medicine · doi:10.1007/978-3-030-10680-5_6 · Cited by 4
Patient-Based Teaching: A Clinical Instructional Method for Large Classrooms
Atre-Vaidya, N. (2000) · Academic Psychiatry · doi:10.1176/appi.ap.24.4.202 · Cited by 4
Recognizing Rural Health Resource and Education Needs
Roberts, L. W. (2022) · Academic Medicine · doi:10.1097/acm.0000000000004823 · Cited by 3
Aplicación de la metodología de aprendizaje basado en problemas a la docencia de Veterinaria Legal
Henríquez‐Hernández, L. A., Luzardo, O. P., Boada, L. D., Almeida‐González, M., & Peña, M. L. Z. (2015) · REDU Revista de Docencia Universitaria · doi:10.4995/redu.2015.5456 · Cited by 3
Problem-based learning in medical degree teaching: a scoping reviewPreprint
Trullàs, J. C., Blay, C., Sarri, E., & Pujol, R. (2021) · Research Square · doi:10.21203/rs.3.rs-514038/v1 · Cited by 3
Issues, Trends and Challenges for Research in Medical Education.
Kandiah, D. A. (2006) · Griffith Research Online · Cited by 2
Radionova, Z. V., & Hristova, P. (2010) · Perspectives of Innovations Economics and Business · doi:10.15208/pieb.2010.96 · Cited by 2
Woodward, C. A. (2002) · Springer international handbooks of education · doi:10.1007/978-94-010-0462-6_5 · Cited by 2
Relevance of Health Behavior Research to Training and Practice in the Health Professions
Bruhn, J. G. (1997) · Springer eBooks · doi:10.1007/978-1-4899-0484-3_1 · Cited by 2
Alzate, S., & Fabio, H. (2011) · Repositorio Institucional UN - Biblioteca Digital · Cited by 2
Soliman, M., Alnassar, S. A., Al-Shaikh, G. K., Ghonaim, M. A., Alhabib, T. A. S., & Al-Shelwi, M. (2015) · Journal of Taibah University Medical Sciences · doi:10.1016/j.jtumed.2015.02.003 · Cited by 1
Tarazona, J. L. (2005) · DOAJ (DOAJ: Directory of Open Access Journals) · Cited by 1
Middle School Science Teachers' Perception of Problem-Based Learning
Choikyunghee, & Rhee, H. (2008) · 교과교육학연구 · doi:10.24231/rici.2008.12.3.747 · Cited by 1
Verma, D., & Soni, R. (2022) · Towards Excellence · doi:10.37867/te140273 · Cited by 1
Integration of Multimodality and Critical Clinical Thinking Approaches in Medical Education
Haniya, S., Tapping, R. I., & Ferguson, D. C. (2023) · Advances in educational technologies and instructional design book series · doi:10.4018/978-1-6684-5124-3.ch005 · Cited by 1
Assessing the impact of a comprehensive sexually transmitted disease curriculum on learning outcomes
Chiang, S., Geisler, W. M., Jackson, J. R., & Rebar, R. W. (2004) · American Journal of Obstetrics and Gynecology · doi:10.1016/j.ajog.2004.07.064 · Cited by 1
Hunziker, A. W., & Hugi, R. (2007) · Annual Conference on Computers
Probleemgestuurd leren: een literatuuroverzicht
Beullens, J., & Jaspaert, H. (2001) · Tijdschrift voor Medisch Onderwijs · doi:10.1007/bf03056511
Mohammed, F. I., Obeidat, A. S., Alkaabba, A. F., Bashir, N., & Al‐Hoqail, I. A. (2011) · Educational Research Review · doi:10.5897/err11.086
Pańczyk, M., Belowska, J., & Gotlib, J. (2014) · EDULEARN proceedings
Comparing problem-based learning curriculum and the traditional curriculum based on the outcome
Kabir, S. A., Mohammad, S., Kabir, S. I., & Dissanayake, T. (2015) · International Journal of Medical Science Research and Practice
Activating children's thinking skills (ACTS) an intervention evaluation study.
Dewey, J. (2006) · Doctoral thesis, University of London.
Naggar, M. A. A. E. A. E. (2014) · Intellectual Property Rights Open Access · doi:10.4172/ipr.1000106
Trullàs, J. C., Blay, C., Sarri, E., & Pujol, R. (2022) · Figshare · doi:10.6084/m9.figshare.19194104
Metadata
- Authors
- Mennin, S. P., Friedman, M., Skipper, B., Kalishman, S., & Snyder, J.
- Affiliations
- University of New Mexico (US)
- ORCID
- Mennin, S P
- Venue
- Academic Medicine
- Publisher
- Oxford University Press (OUP)
- Published
- 1993-08
- Volume
- 68
- Issue
- 8
- Pages
- 616-24
- ISSN
- 1040-2446 (ISSN-L)
- OpenAlex
- W1982405425
- Cited by
- 103
- References
- 0
- Citations by year
- 2026: 1 · 2025: 1 · 2024: 1 · 2023: 1 · 2022: 4 · 2021: 1 · 2019: 2 · 2018: 1 · 2017: 1 · 2016: 2 · 2015: 7 · 2014: 5 · 2013: 2 · 2012: 1
- FWCI
- 6.83
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Academic Medicine (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Education › Problem and Project Based Learning
- Social Sciences › Education › Innovations in Medical Education
- Social Sciences › Education › Medical Education and Admissions
- Keywords
- problem-based learning, medical licensing examinations, medical students, primary care curriculum, MCAT scores, grade point average, at-risk students, medical school admissions, University of New Mexico School of Medicine, curriculum comparison, medical education assessment
- MeSH terms
- Curriculum, Education, Medical, Undergraduate, Educational Measurement, Educational Status, Forecasting, Licensure, Medical, New Mexico, Primary Health Care, Problem Solving, Random Allocation, School Admission Criteria, Science, Teaching, Program Evaluation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review