Patients don’t come with multiple choice options: essay-based assessment in UME
Open accessThe DOI matches its Crossref record; 7 of the 7 references it cites were found in a registry or an index.
Abstract
Curricular revision efforts have resulted in learner-centered programs that value content integration and active learning. Yet, less attention has been placed on assessment methods that are learner-centered and promote assessment for learning. The use of context rich short answer question (CR-SAQ) exams in the preclinical years of medical school was evaluated to determine if this format aligns with the criteria for assessment for learning. Medical students and preclinical faculty members were sent a survey comprised of closed and open-ended questions about their experience using CR-SAQ exams. Data were analyzed using a mixed-method design. Open-ended responses were evaluated using thematic analysis within the framework of criteria for assessment for learning. A total of 274 students (94%) and 24 faculty (75%) completed the survey. Fifty four percent of students reported preferring a CR-SAQ exam format over multiple choice questions (MCQ) format. Quantitative data and qualitative comments by students supported that CR-SAQ exams aligned with criteria for assessment for learning, including acceptability, authenticity, educational effect, and the cueing effect. Student concerns included preparation for USMLE Step 1 exam, as well as the validity and reproducibility of CR-SAQ assessments. Faculty largely agreed with the benefits of the CR-SAQ, but were concerned about feasibility, acceptability and reproducibility. The CR-SAQ exam format assessment strategy supports assessment for learning in an undergraduate medical education setting. Both benefits and drawbacks of this method are presented, however students and faculty describe a broader impact that this assessment method has on their development as a physician.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Qualitative data analysis for applied policy research
Ritchie, J., & Spencer, L. (2010) · Analyzing qualitative data · doi:10.4324/9780203413081_chapter_9 · Cited by 4,800
Assessment in medical education
Epstein, R. M. (2007) · New England Journal of Medicine · doi:10.1056/nejmra054784 · Cited by 1,872
The critical role of retrieval practice in long-term retention
Roediger, H. L., & Butler, A. C. (2011) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2010.09.003 · Cited by 1,796 · cited 1×
“(generation effect) [11], and allow for meaningful feedback to further the learning cycle (catalytic effect)”
BackgroundA five-dimensional framework for authentic assessment
Gulikers, J. T. M., Bastiaens, T. J., & Kirschner, P. A. (2004) · Educational Technology Research and Development · doi:10.1007/bf02504676 · Cited by 916
Programmatic assessment: From assessment of learning to assessment for learning
Schuwirth, L. W. T., & Van der Vleuten, C. P. M. (2011) · Medical Teacher · doi:10.3109/0142159x.2011.565828 · Cited by 902 · cited 3×
“More recently, there has been a drive towards writing ‘context-rich’ MCQs in which test items are embedded in a clinical vignette [8].”
BackgroundScouller, K. (1998) · Higher Education · doi:10.1023/a:1003196224280 · Cited by 724
Automated essay scoring and the future of educational assessment in medical education
Gierl, M. J., Latifi, S., Lai, H., Boulais, A., & De Champlain, A. (2014) · Medical Education · doi:10.1111/medu.12517 · Cited by 80 · cited 1×
“suggested that as automated essay scoring systems become optimized, more medical schools may consider this assessment approach [20].”
Background
Cited by
Morjaria, L., Burns, L., Bracken, K., Levinson, A. J., Ngo, Q. N., Lee, M., & Sibbald, M. (2024) · International Medical Education · doi:10.3390/ime3010004 · Cited by 33
“…remains an important assessment method for standardized assessment in medical certification, and comprises the bulk of existing research for LLM performance, undergraduate medical training often includes short-answer response problems for formative assessment of student progress [20,21].”
Morjaria, L., Burns, L., Bracken, K., Ngo, Q. N., Lee, M., Levinson, A. J., Smith, J., Thompson, P., & Sibbald, M. (2023) · Journal of Medical Education and Curricular Development · doi:10.1177/23821205231204178 · Cited by 31
“Short answer assessments are a widely used form of student evaluation in medical education, and are critical in evaluating student knowledge, communication skills, and other essential competencies required for delivering compassionate care.(10,11) The ability of ChatGPT to quickly and accurately apply known concepts i…”
BackgroundHollis-Sando, L., Pugh, C., Franke, K., Zerner, T., Tan, Y., Carneiro, G., Van den Hengel, A., Symonds, I., Duggan, P., & Bacchi, S. (2023) · Focus on Health Professional Education: A Multi-Professional Journal · doi:10.11157/fohpe.v24i1.531 · Cited by 7
“The performance on the 2-mark and 3-mark SAQs is salient in that previous research has suggested that this type of question may complement standard multiple-choice questions (Bird et al., 2019), although this point may be debated (Hift, 2014).”
Background
Metadata
- Authors
- Bird, J. B., Olvet, D. M., Willey, J. M., & Brenner, J.
- Venue
- Medical Education Online
- Publisher
- Informa UK Limited
- Published
- 2019-01-01
- Volume
- 24
- Issue
- 1
- Pages
- 1649959
- ISSN
- 1087-2981 (ISSN-L)
- OpenAlex
- W2969263256
- Cited by
- 59
- References
- 7
- FWCI
- 4.99
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Patients don’t come with multiple choice options: essay-based assessment in UME. https://app.cochat.ai/papers/10.1080/10872981.2019.1649959
- The paper
- Bird, J. B., Olvet, D. M., Willey, J. M., & Brenner, J. (2019). Patients don’t come with multiple choice options: essay-based assessment in UME. Medical Education Online. https://doi.org/10.1080/10872981.2019.1649959
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.