Assessment in Medical Education
The DOI matches its Crossref record; 74 of the 74 references it cites were found in a registry or an index.
Abstract
This article in the Medical Education series provides a conceptual framework for and a brief update on commonly used and emerging methods of assessment, discusses the strengths and limitations of each method, and identifies several major challenges in assessing professional competence and performance.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Such questions require real-time clinical reasoning that cannot be adequately prepared through AI-generated text alone (Epstein, 2007).”
From Designing AI-resilient assessment in higher education: a four-pillar conceptual framework (2026) · cites this paper as Epstein, 2007
In the citing paper
“There are many forms of examination in medicine, the most common of which is the written examination and the Objective Structured Clinical Examinations [56].”
From Applications and Challenges of Implementing Artificial Intelligence in Medical Education: Integrative Review (2019) · cites this paper as [56]
Citations
Cites
Educating the reflective practitioner
Schön, D. A. (1987) · Cited by 13,969
Defining and Assessing Professional Competence
Epstein, R. M. (2002) · JAMA · doi:10.1001/jama.287.2.226 · Cited by 3,107
Self-citationEricsson, K. A. (2004) · Academic Medicine · doi:10.1097/00001888-200410001-00022 · Cited by 2,988
Burnout and self-reported patient care in an internal medicine residency program
Shanafelt, T. D., Bradley, K. A., Wipf, J. E., & Back, A. L. (2002) · Annals of Internal Medicine · doi:10.7326/0003-4819-136-5-200203050-00008 · Cited by 2,165
Teaching Surgical Skills — Changes in the Wind
Reznick, R. K., & MacRae, H. (2006) · New England Journal of Medicine · doi:10.1056/nejmra054785 · Cited by 1,761
Diagnostic error in internal medicine
Graber, M. L., Franklin, N., & Gordon, R. (2005) · Archives of Internal Medicine · doi:10.1001/archinte.165.13.1493 · Cited by 1,549
The assessment of professional competence: Developments, research and practical implications
Van Der Vleuten, C. P. M. (1996) · Advances in Health Sciences Education · doi:10.1007/bf00596229 · Cited by 1,247
Epstein, R. M., Franks, P., Fiscella, K., Shields, C. G., Meldrum, S. C., Kravitz, R. L., & Duberstein, P. R. (2005) · Social Science & Medicine · doi:10.1016/j.socscimed.2005.02.001 · Cited by 1,211
Self-citationSchmidt, H. G., Norman, G. R., & Boshuizen, H. P. (1990) · Academic Medicine · doi:10.1097/00001888-199010000-00001 · Cited by 1,109
Epstein, R. M. (1999) · JAMA · doi:10.1001/jama.282.9.833 · Cited by 1,036
Self-citationAssessment of clinical competence
Wass, V., Van der Vleuten, C., Shatzer, J., & Jones, R. (2001) · The Lancet · doi:10.1016/s0140-6736(00)04221-5 · Cited by 1,024
Self-Assessment in the Health Professions: A Reformulation and Research Agenda
Eva, K. W., & Regehr, G. (2005) · Academic Medicine · doi:10.1097/00001888-200510001-00015 · Cited by 992
Educational Strategies to Promote Clinical Diagnostic Reasoning
Bowen, J. L. (2006) · New England Journal of Medicine · doi:10.1056/nejmra054782 · Cited by 931
Coping with complexity: educating for capability
Fraser, S. W., & Greenhalgh, T. (2001) · BMJ · doi:10.1136/bmj.323.7316.799 · Cited by 889
General Competencies And Accreditation In Graduate Medical Education
Batalden, P., Leach, D., Swing, S., Dreyfus, H., & Dreyfus, S. (2002) · Health Affairs · doi:10.1377/hlthaff.21.5.103 · Cited by 738
The Mini-CEX: A Method for Assessing Clinical Skills
Norcini, J. J., Blank, L. L., Duffy, F. D., & Fortna, G. S. (2003) · Annals of Internal Medicine · doi:10.7326/0003-4819-138-6-200303180-00012 · Cited by 730
Kravitz, R. L., Epstein, R. M., Feldman, M. D., Franz, C. E., Azari, R., Wilkes, M. S., Hinton, L., & Franks, P. (2005) · JAMA · doi:10.1001/jama.293.16.1995 · Cited by 586
Self-citationThe real test bias: Influences of testing on teaching and learning.
Frederiksen, N. (1984) · American Psychologist · doi:10.1037//0003-066x.39.3.193 · Cited by 531
The effect of assessments and examinations on the learning of medical students
NEWBLE, D. I., & JAEGER, K. (1983) · Medical Education · doi:10.1111/j.1365-2923.1983.tb00657.x · Cited by 490
Work and integrity: the crisis and promise of professionalism in America
(1995) · Choice Reviews Online · doi:10.5860/choice.33-0244 · Cited by 468
The Script Concordance Test: A Tool to Assess the Reflective Clinician
Charlin, B., Roy, L., Brailovsky, C., Goulet, F., & van der Vleuten, C. (2000) · Teaching and Learning in Medicine · doi:10.1207/s15328015tlm1204_5 · Cited by 373
Patient-Centered Communication and Diagnostic Testing
Epstein, R. M. (2005) · The Annals of Family Medicine · doi:10.1370/afm.348 · Cited by 373
Self-citationUse of Peer Ratings to Evaluate Physician Performance
Ramsey, P. G. (1993) · JAMA: The Journal of the American Medical Association · doi:10.1001/jama.1993.03500130069034 · Cited by 347
Competency based medical training: review.
Leung, W. (2002) · PubMed · Cited by 314
Different written assessment methods: what can be said about their strengths and weaknesses?
Schuwirth, L. W. T., & Van Der Vleuten, C. P. M. (2004) · Medical Education · doi:10.1111/j.1365-2929.2004.01916.x · Cited by 309
Chang, J. T., Hays, R. D., Shekelle, P. G., MacLean, C. H., Solomon, D. H., Reuben, D. B., Roth, C. P., Kamberg, C. J., Adams, J., Young, R. T., et al. (2006) · Annals of Internal Medicine · doi:10.7326/0003-4819-144-9-200605020-00010 · Cited by 296
Pitfalls in the pursuit of objectivity: issues of reliability
VAN DER VLEUTEN, C. P. M., NORMAN, G. R., & DE GRAAFF, E. (1991) · Medical Education · doi:10.1111/j.1365-2923.1991.tb00036.x · Cited by 283
Leung, W.-C., & Diwakar, V. (2002) · BMJ · doi:10.1136/bmj.325.7366.693 · Cited by 276
Assessing communication competence: a review of current tools.
Schirmer, J. M., Mauksch, L. B., Lang, F., Marvel, M. K., Zoppi, K., Epstein, R. M., Brock, D., & Pryzbylski, M. (2005) · PubMed · Cited by 252
Self-citationAssociation Between Licensure Examination Scores and Practice in Primary Care
Tamblyn, R. (2002) · JAMA · doi:10.1001/jama.288.23.3019 · Cited by 251
Norcini, J. J. (2003) · Medical Education · doi:10.1046/j.1365-2923.2003.01536.x · Cited by 243
Analytic global OSCE ratings are sensitive to level of training
Hodges, B., & McIlroy, J. H. (2003) · Medical Education · doi:10.1046/j.1365-2923.2003.01674.x · Cited by 232
The ability of the multiple mini-interview to predict preclerkship performance in medical school
Eva, K. W., Reiter, H. I., Rosenfeld, J., & Norman, G. R. (2004) · Academic Medicine · doi:10.1097/00001888-200410001-00012 · Cited by 225
BORDAGE, G., & ZACKS, R. (1984) · Medical Education · doi:10.1111/j.1365-2923.1984.tb01295.x · Cited by 213
Peer assessment of professional competence
Dannefer, E. F., Henson, L. C., Bierer, S. B., Grady‐Weliky, T. A., Meldrum, S., Nofziger, A. C., Barclay, C., & Epstein, R. M. (2005) · Medical Education · doi:10.1111/j.1365-2929.2005.02193.x · Cited by 174
Self-citationLeach, D. C. (2002) · JAMA · doi:10.1001/jama.287.2.243 · Cited by 171
Ram, Grol, Rethans, Schouten, Vleuten, & Kester (1999) · Medical Education · doi:10.1046/j.1365-2923.1999.00348.x · Cited by 171
Pitfalls in the pursuit of objectivity: issues of validity, efficiency and acceptability
NORMAN, G. R., VAN DER VLEUTEN, C. P. M., & DE GRAAFF, E. (1991) · Medical Education · doi:10.1111/j.1365-2923.1991.tb00037.x · Cited by 170
Preventing Errors in Clinical Practice: A Call for Self-Awareness
Borrell-Carrio, F. (2004) · The Annals of Family Medicine · doi:10.1370/afm.80 · Cited by 158
A practical guide to assessing clinical decision‐making skills using the key features approach
Farmer, E., & Page, G. (2005) · Medical Education · doi:10.1111/j.1365-2929.2005.02339.x · Cited by 148
Carraccio, C. L., & Englander, R. (2004) · Teaching and Learning in Medicine · doi:10.1207/s15328015tlm1604_13 · Cited by 133
Assessing Residents’ Competencies at Baseline: Identifying the Gaps
Lypson, M. L., Frohna, J. G., Gruppen, L. D., & Woolliscroft, J. O. (2004) · Academic Medicine · doi:10.1097/00001888-200406000-00013 · Cited by 132
A closer look at cueing effects in multiple‐choice questions
Schuwirth, L. W. T., van der Vleuten, C. P. M., & Donkers, H. H. L. M. (1996) · Medical Education · doi:10.1111/j.1365-2923.1996.tb00716.x · Cited by 131
Basing the Evaluation of Professionalism on Observable Behaviors: A Cautionary Tale
Ginsburg, S., Regehr, G., & Lingard, L. (2004) · Academic Medicine · doi:10.1097/00001888-200410001-00001 · Cited by 127
Why are sicker patients less satisfied with their medical care? Tests of two explanatory models.
Hall, J. A., Milburn, M. A., ROTER, D. L., & Daltroy, L. H. (1998) · Health Psychology · doi:10.1037//0278-6133.17.1.70 · Cited by 124
On the Internet: Thinking in Action
(2002) · Information Technology and People · doi:10.1108/itp.2002.15.4.363.2 · Cited by 123
Brailovsky, C., Charlin, B., Beausoleil, S., Coté, S., & van der Vleuten, C. P. M. (2001) · Medical Education · doi:10.1046/j.1365-2923.2001.00911.x · Cited by 123
Standardized or real patients to test clinical competence? The long case revisited
Wass, V., Jones, R., & Van der Vleuten, C. (2001) · Medical Education · doi:10.1046/j.1365-2923.2001.00928.x · Cited by 122
The role of assessment in expanding professional horizons
Ben-David, M. F. (2000) · Medical Teacher · doi:10.1080/01421590050110731 · Cited by 110
Do short cases elicit different thinking processes than factual knowledge questions do?
Schuwirth, L. W. T., Verheggen, M. M., Van Der Vleuten, C. P. M., Boshuizen, H. P. A., & Dinant, G. J. (2001) · Medical Education · doi:10.1046/j.1365-2923.2001.00771.x · Cited by 104
Mindful practice in action (II): Cultivating habits of mind.
Epstein, R. M. (2003) · Families, Systems, & Health · doi:10.1037/h0089495 · Cited by 91
Self-citationViolato, C., Marini, A., Toews, J., Lockyer, J., & Fidler, H. (1997) · Academic Medicine · doi:10.1097/00001888-199710001-00028 · Cited by 91
The Step 2 Clinical-Skills Examination
Papadakis, M. A. (2004) · New England Journal of Medicine · doi:10.1056/nejmp038246 · Cited by 89
Use of peer ratings to evaluate physician performance
Ramsey, P. G. (1993) · JAMA · doi:10.1001/jama.269.13.1655 · Cited by 81
Comprehensive Assessment of Professional Competence: The Rochester Experiment
Epstein, R. M., Dannefer, E. F., Nofziger, A. C., Hansen, J. T., Schultz, S. H., Jospe, N., Connard, L. W., Meldrum, S. C., & Henson, L. C. (2004) · Teaching and Learning in Medicine · doi:10.1207/s15328015tlm1602_12 · Cited by 78
Self-citationReznick, R. K., Blackmore, D., Cohen, R., Baumber, J., Rothman, A., Smee, S., Chalmers, A., Poldre, P., Birtwhistte, R., & Walsh, P. (1993) · Academic Medicine · doi:10.1097/00001888-199310000-00028 · Cited by 75
What Do Faculty Observe of Medical Students' Clinical Performance?
Pulito, A. R., Donnelly, M. B., Plymale, M., & Mentzer, Jr. (2006) · Teaching and Learning in Medicine · doi:10.1207/s15328015tlm1802_2 · Cited by 74
Schuwirth, L., & van der Vleuten, C. P. M. (2004) · Medical Education · doi:10.1111/j.1365-2929.2004.02055.x · Cited by 70
The structured oral examination as a method for assessing surgical residents
Anastakis, D. J., Cohen, R., & Reznick, R. K. (1991) · The American Journal of Surgery · doi:10.1016/0002-9610(91)90205-r · Cited by 68
The Effects of Type of Examination Anticipated on Test Preparation and Performance
Hakstian, A. R. (1971) · The Journal of Educational Research · doi:10.1080/00220671.1971.10884175 · Cited by 66
Gruppen, L. D., & Frohna, A. Z. (2002) · Springer international handbooks of education · doi:10.1007/978-94-010-0462-6_8 · Cited by 63
A new rating form for use by nurses in assessing residents’ humanistic behavior
Butterfield, P. S., & Mazzaferri, E. L. (1991) · Journal of General Internal Medicine · doi:10.1007/bf02598316 · Cited by 53
The long case versus objective structured clinical examinations
Norman, G. R. (2002) · BMJ · doi:10.1136/bmj.324.7340.748 · Cited by 51
Use of standardized patients in the assessment of medical practice.
Tamblyn, R. M. (1998) · Europe PMC (PubMed Central) · Cited by 46
Temporal and group‐related trends in peer assessment amongst medical students
Lurie, S. J., Nofziger, A. C., Meldrum, S., Mooney, C., & Epstein, R. M. (2006) · Medical Education · doi:10.1111/j.1365-2929.2006.02540.x · Cited by 44
Self-citationHow physician communication influences recognition of depression in primary care.
Carney, P. A., Eliassen, M. S., Wolford, G. L., Owen, M., BADGER, L. W., & Dietrich, A. J. (1999) · PubMed · Cited by 39
THINKING ABOUT STUDENT THINKING
FRIEDMAN, M. H., Connell, K. J., Olthoff, A. J., Sinacore, J. M., & Bordage, G. (1998) · Academic Medicine · doi:10.1097/00001888-199810000-00033 · Cited by 37
Evaluating humanistic attributes of internal medicine residents
Klessig, J. M., Robbins, A. S., Wieland, D., & Rubenstein, L. Z. (1989) · Journal of General Internal Medicine · doi:10.1007/bf02599551 · Cited by 32
Lurie, S. J., Lambert, D. R., Nofziger, A. C., Epstein, R. M., & Grady-Weliky, T. A. (2007) · Journal of General Internal Medicine · doi:10.1007/s11606-007-0117-4 · Cited by 31
Self-citationReevaluation of Clinical Competency
KLASS, D. J. (2000) · American Journal of Physical Medicine & Rehabilitation · doi:10.1097/00002060-200009000-00018 · Cited by 29
Maintenance of certification in the United States: A progress report
Batmangelich, S., & Adamowski, S. E. (2004) · Journal of Continuing Education in the Health Professions · doi:10.1002/chp.1340240303 · Cited by 25
Screening test length for sequential testing with a standardized-patient examination
Colliver, J. A., Vu, N. V., & Barrows, H. S. (1992) · Academic Medicine · doi:10.1097/00001888-199209000-00010 · Cited by 23
Jg, C., Woolliscroft, J. O., Em, H., Wolf, F. M., & Wk, D. (1984) · PubMed · Cited by 18
Learning Professional Processes: Public Knowledge and Personal Experience
(2002) · Routledge eBooks · doi:10.4324/9780203486016-11 · Cited by 5
Research works printed but not linked: 7
- 10.1037/0003-066x.39.3.193 · doi:10.1037/0003-066x.39.3.193 · Paper page
- Carney PA, 1999, J Fam Pract, 48, 958
- Tamblyn RM, 1998, CMAJ, 158, 205
- Nofziger AC, Davis B, Naumburg EH, Epstein RM. The impact of peer assessment on professional development. Presented at the Ottawa Conference on Medical Education and Assessment, Ottawa, July 15, 2002.
- Calhoun JG, 1984, Proc Annu Conf Res Med Educ, 23, 205
- 10.1037/0278-6133.17.1.70 · doi:10.1037/0278-6133.17.1.70 · Paper page
- Schirmer JM, 2005, Fam Med, 37, 184
Cited by
Chan, K. S., & Zary, N. (2019) · JMIR Medical Education · doi:10.2196/13930 · Cited by 877
Peer teaching in medical education: twelve reasons to move from theory to practice
Ten Cate, O., & Durning, S. (2007) · Medical Teacher · doi:10.1080/01421590701606799 · Cited by 695
Boscardin, C. K., Gin, B., Golde, P. B., & Hauer, K. E. (2024) · Academic Medicine · doi:10.1097/acm.0000000000005439 · Cited by 620
Kogan, J. R., Holmboe, E. S., & Hauer, K. E. (2009) · JAMA · doi:10.1001/jama.2009.1365 · Cited by 598
Norcini, J., Anderson, B., Bollela, V., Burch, V., Costa, M. J., Duvivier, R., Galbraith, R., Hays, R., Kent, A., Perrott, V., et al. (2011) · Medical Teacher · doi:10.3109/0142159x.2011.551559 · Cited by 574
Metadata
- Authors
- Epstein, R. M.
- Affiliations
- University of Rochester Medicine (US)
- Corresponding author
- Epstein, Ronald M.
- ORCID
- Epstein, Ronald M.
- Venue
- New England Journal of Medicine
- Publisher
- Massachusetts Medical Society
- Published
- 2007-01-25
- Volume
- 356
- Issue
- 4
- Pages
- 387-396
- ISSN
- 0028-4793 (ISSN-L)
- 1533-4406
- OpenAlex
- W2139001711
- Cited by
- 1,872
- References
- 74
- Citations by year
- 2026: 58 · 2025: 74 · 2024: 82 · 2023: 93 · 2022: 110 · 2021: 101 · 2020: 115 · 2019: 106 · 2018: 138 · 2017: 128 · 2016: 127 · 2015: 136 · 2014: 89 · 2013: 126 · 2012: 86
- FWCI
- 11.40
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- New England Journal of Medicine (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Education › Innovations in Medical Education
- Health Sciences › Medicine › Family Practice › Clinical Reasoning and Diagnostic Skills
- Social Sciences › Education › Medical Education and Admissions
- Keywords
- medical education assessment, assessment methods, professional competence, clinical performance assessment, conceptual framework
- MeSH terms
- Clinical Competence, Education, Medical, Educational Measurement, Humans, Patient Simulation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.