The DOI matches its Crossref record; 41 of the 41 references it cites were found in a registry or an index.
Criteria for good assessment: Consensus statement and recommendations from the Ottawa 2010 Conference
Open accessAbstract
In this article, we outline criteria for good assessment that include: (1) validity or coherence, (2) reproducibility or consistency, (3) equivalence, (4) feasibility, (5) educational effect, (6) catalytic effect, and (7) acceptability. Many of the criteria have been described before and we continue to support their importance here. However, we place particular emphasis on the catalytic effect of the assessment, which is whether the assessment provides results and feedback in a fashion that creates, enhances, and supports education. These criteria do not apply equally well to all situations. Consequently, we discuss how the purpose of the test (summative versus formative) and the perspectives of stakeholders (examinees, patients, teachers-educational institutions, healthcare system, and regulators) influence the importance of the criteria. Finally, we offer a series of practice points as well as next steps that should be taken with the criteria. Specifically, we recommend that the criteria be expanded or modified to take account of: (1) the perspectives of patients and the public, (2) the intimate relationship between assessment, feedback, and continued learning, (3) systems of assessment, and (4) accreditation systems.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
(1978) · Medical Entomology and Zoology · Cited by 53,488
Scholarship Reconsidered: Priorities of the Professoriate
Poston, L., & Boyer, E. L. (1992) · Academe · doi:10.2307/40250362 · Cited by 6,829
The assessment of clinical skills/competence/performance
Miller, G. E. (1990) · Academic Medicine · doi:10.1097/00001888-199009000-00045 · Cited by 4,883
The Social Transformation of American Medicine
Leavitt, J. W., & Starr, P. (1984) · The American Historical Review · doi:10.2307/1862729 · Cited by 3,247
The Role of Assessment in a Learning Culture
Shepard, L. A. (2000) · Educational Researcher · doi:10.3102/0013189x029007004 · Cited by 2,161 · cited 1×
“It would be a mistake to recognize the importance of assessment and yet not to connect it with the scholarship of teaching and learning (Shepard 2000).”
BackgroundAssessment in medical education
Epstein, R. M. (2007) · New England Journal of Medicine · doi:10.1056/nejmra054784 · Cited by 1,872
Medical Education in the United States and Canada.
(1911) · Archives of Internal Medicine · doi:10.1001/archinte.1911.00060010141008 · Cited by 1,625
A Five-Stage Model of the Mental Activities Involved in Directed Skill Acquisition
Dreyfus, S. E., & Dreyfus, H. L. (1980) · doi:10.21236/ada084551 · Cited by 1,572 · cited 1×
“Along the first dimension are the competencies that need to be assessed, along the second is the level of assessment required, and along the third is the trainee’s stage of development (Dreyfus & Dreyfus 1980; Miller 1990; Norcini et al. 2008).”
BackgroundSustainable Assessment: Rethinking assessment for the learning society
Boud, D. (2000) · Studies in Continuing Education · doi:10.1080/713695728 · Cited by 1,545
Assessing professional competence: from methods to programmes
Van Der Vleuten, C. P. M., & Schuwirth, L. W. T. (2005) · Medical Education · doi:10.1111/j.1365-2929.2005.02094.x · Cited by 1,268
The assessment of professional competence: Developments, research and practical implications
Van Der Vleuten, C. P. M. (1996) · Advances in Health Sciences Education · doi:10.1007/bf00596229 · Cited by 1,247
Assessment of clinical competence
Wass, V., Van der Vleuten, C., Shatzer, J., & Jones, R. (2001) · The Lancet · doi:10.1016/s0140-6736(00)04221-5 · Cited by 1,024
Workplace-based assessment as an educational tool: AMEE Guide No. 31
Norcini, J., & Burch, V. (2007) · Medical Teacher · doi:10.1080/01421590701775453 · Cited by 984
Self-citationMedical Education in the United States and Canada
Flexner, A. (1910) · Science · doi:10.1126/science.32.810.41 · Cited by 824
Kogan, J. R., Holmboe, E. S., & Hauer, K. E. (2009) · JAMA · doi:10.1001/jama.2009.1365 · Cited by 598
The social transformation of American medicine
Harris, J. E. (1983) · Social Science & Medicine · doi:10.1016/0277-9536(83)90170-3 · Cited by 520
Scholarship of Teaching: A model
Trigwell, K., Martin, E., Benjamin, J., & Prosser, M. (2000) · Higher Education Research & Development · doi:10.1080/072943600445628 · Cited by 466
Assessing Professional Behavior: Yesterday, Today, and Tomorrow
Arnold, L. (2002) · Academic Medicine · doi:10.1097/00001888-200206000-00006 · Cited by 462
AMEE Guide No. 25: The assessment of learning outcomes for the competent and reflective physician
Shumway, J. M., & Harden, R. M. (2003) · Medical Teacher · doi:10.1080/0142159032000151907 · Cited by 461
Lurie, S. J., Mooney, C. J., & Lyness, J. M. (2009) · Academic Medicine · doi:10.1097/acm.0b013e3181971f08 · Cited by 363
International Handbook of Research in Medical Education
(2002) · Springer International Handbooks of Education · doi:10.1007/978-94-010-0462-6 · Cited by 330
Formative assessment: a key to deep learning?
Rushton, A. (2005) · Medical Teacher · doi:10.1080/01421590500129159 · Cited by 291
Performance-Based Assessment: Lessons from the Health Professions
Swanson, D. B., Norman, G. R., & Linn, R. L. (1995) · Educational Researcher · doi:10.2307/1177072 · Cited by 290
Pay for Performance at the Tipping Point
Epstein, A. M. (2007) · New England Journal of Medicine · doi:10.1056/nejme078002 · Cited by 276
Workplace-based assessment as an educational tool
Norcini, J. J., Burch, V. C., & Gibbs, T. J. (2007) · Medical Entomology and Zoology · Cited by 266
Self-citationChapter 10: Socio-Cultural Aspects of Assessment
GIPPS, C. (1999) · Review of Research in Education · doi:10.3102/0091732x024001355 · Cited by 208
Assessing health professionals
Crossley, J., Humphris, G., & Jolly, B. (2002) · Medical Education · doi:10.1046/j.1365-2923.2002.01294.x · Cited by 208
The Professionalism Mini-Evaluation Exercise: A Preliminary Investigation
Cruess, R., McIlroy, J. H., Cruess, S., Ginsburg, S., & Steinert, Y. (2006) · Academic Medicine · doi:10.1097/00001888-200610001-00019 · Cited by 207
Driessen, E., Van Der Vleuten, C., Schuwirth, L., Van Tartwijk, J., & Vermunt, J. (2005) · Medical Education · doi:10.1111/j.1365-2929.2004.02059.x · Cited by 207
Practical Guide to the Evaluation of Clinical Competence
Huh, S. (2022) · Korean Medical Education Review · doi:10.17496/kmer.2022.24.2.176 · Cited by 161
Current perspectives in assessment: the assessment of performance at work
Norcini, J. J. (2005) · Medical Education · doi:10.1111/j.1365-2929.2005.02182.x · Cited by 149
Self-citationChanging education, changing assessment, changing research?
Schuwirth, L., & van der Vleuten, C. P. M. (2004) · Medical Education · doi:10.1111/j.1365-2929.2004.01851.x · Cited by 148
Trigwell, K. (2001) · The International Journal for Academic Development · doi:10.1080/13601440110033698 · Cited by 125
The role of assessment in expanding professional horizons
Ben‐David, M. F. (2000) · Medical Teacher · doi:10.1080/01421590050110731 · Cited by 113
Varkey, P., Karlapudi, S. P., Rose, S. H., NELSON, R. L., & Warner, M. (2009) · Academic Medicine · doi:10.1097/acm.0b013e31819731fb · Cited by 95
Practical Guide to Medical Student Assessment
Amin, Z., Seng, C. Y., & Eng, K. H. (2006) · WORLD SCIENTIFIC eBooks · doi:10.1142/6109 · Cited by 75
Educational Theory and Medical Education Practice: A Cautionary Note for Medical School Faculty
Colliver, J. A. (2002) · Academic Medicine · doi:10.1097/00001888-200212000-00012 · Cited by 74
The History of Medical Education
O’Malley, C. D. (1970) · University of California Press eBooks · doi:10.1525/9780520313446 · Cited by 62
Use of a structured interview to assess portfolio‐based learning
Burch, V. C., & Seggie, J. (2008) · Medical Education · doi:10.1111/j.1365-2923.2008.03128.x · Cited by 40
Self-citationDaviau, P. M. M. (2016) · BRILL eBooks · doi:10.1163/9789004316201_012 · Cited by 28
Achieving Large-Scale Change in Medical Education
Curry, L. (2002) · Springer international handbooks of education · doi:10.1007/978-94-010-0462-6_39 · Cited by 6
Cited by
The Next GME Accreditation System — Rationale and Benefits
Nasca, T. J., Philibert, I., Brigham, T., & Flynn, T. C. (2012) · New England Journal of Medicine · doi:10.1056/nejmsr1200117 · Cited by 1,611
Core principles of assessment in competency-based medical education.
Lockyer, J., Carraccio, C., Chan, M.-K., Hart, D., Smee, S., Touchie, C., Holmboe, E. S., Frank, J. R., & Collaborators, O. B. O. T. I. (2017) · Medical Teacher · doi:10.1080/0142159x.2017.1315082 · Cited by 493
Assessment, feedback and the alchemy of learning
Watling, C. J., & Ginsburg, S. (2019) · Medical Education · doi:10.1111/medu.13645 · Cited by 427
Cheng, A., Nadkarni, V. M., Mancini, M. B., Hunt, E. A., Sinz, E. H., Merchant, R. M., Donoghue, A., Duff, J. P., Eppich, W., Auerbach, M., et al. (2018) · Circulation · doi:10.1161/cir.0000000000000583 · Cited by 418
Curriculum Development for Medical Education
Thomas, P. (2016) · Johns Hopkins University Press eBooks · doi:10.1353/book.44600 · Cited by 415
Metadata
- Authors
- Norcini, J., Anderson, B., Bollela, V., Burch, V., Costa, M. J., Duvivier, R., Galbraith, R., Hays, R., Kent, A., Perrott, V., & Roberts, T.
- Venue
- Medical Teacher
- Publisher
- Informa UK Limited
- Published
- 2011-02-23
- Volume
- 33
- Issue
- 3
- Pages
- 206-214
- ISSN
- 0142-159X (ISSN-L)
- 1466-187X
- OpenAlex
- W2128363176
- Cited by
- 574
- References
- 41
- FWCI
- 24.11
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.