CoChat
Sign Up
Cited by 294Educational Assessment and PedagogyInnovations in Medical EducationClinical Reasoning and Diagnostic Skills

Assessment of higher order cognitive skills in undergraduate education: modified essay or multiple choice questions? Research paper

Open access
Palmer, Edward J, Devitt, Peter G·2007·BMC Medical Education
DOI 10.1186/1472-6920-7-49PMC PMC2148038
Open this paper in CoChat — free accountOpen PDF

Abstract

BACKGROUND: Reliable and valid written tests of higher cognitive function are difficult to produce, particularly for the assessment of clinical problem solving. Modified Essay Questions (MEQs) are often used to assess these higher order abilities in preference to other forms of assessment, including multiple-choice questions (MCQs). MEQs often form a vital component of end-of-course assessments in higher education. It is not clear how effectively these questions assess higher order cognitive skills. This study was designed to assess the effectiveness of the MEQ to measure higher-order cognitive skills in an undergraduate institution. METHODS: An analysis of multiple-choice questions and modified essay questions (MEQs) used for summative assessment in a clinical undergraduate curriculum was undertaken. A total of 50 MCQs and 139 stages of MEQs were examined, which came from three exams run over two years. The effectiveness of the questions was determined by two assessors and was defined by the questions ability to measure higher cognitive skills, as determined by a modification of Bloom's taxonomy, and its quality as determined by the presence of item writing flaws. RESULTS: Over 50% of all of the MEQs tested factual recall. This was similar to the percentage of MCQs testing factual recall. The modified essay question failed in its role of consistently assessing higher cognitive skills whereas the MCQ frequently tested more than mere recall of knowledge. CONCLUSION: Construction of MEQs, which will assess higher order cognitive skills cannot be assumed to be a simple task. Well-constructed MCQs should be considered a satisfactory replacement for MEQs if the MEQs cannot be designed to adequately test higher order skills. Such MCQs are capable of withstanding the intellectual and statistical scrutiny imposed by a high stakes exit examination.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1186/1472-6920-7-49

Cites

25 of 25 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Multiple-choice questions can be used just as well as OEQs to assess the foundations of expertise.

    “Multiplechoice questions can be used just as well as OEQs to assess the foundations of expertise.36”

    From Test-enhanced learning in medical education (2008) · cites this paper as [36]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Related

Authors
Palmer, E. J., & Devitt, P. G.
Affiliations
The University of Adelaide (AU)
Corresponding author
Palmer, Edward J
ORCID
Palmer, Edward J
Venue
BMC Medical Education
Publisher
Springer Science and Business Media LLC
Published
2007-11-28
Volume
7
Issue
1
Pages
49-49
ISSN
1472-6920 (ISSN-L)
DOI
10.1186/1472-6920-7-49
PMC
PMC2148038
OpenAlex
W2170435532
Cited by
294
References
25
Citations by year
2026: 14 · 2025: 13 · 2024: 20 · 2023: 27 · 2022: 22 · 2021: 23 · 2020: 24 · 2019: 16 · 2018: 16 · 2017: 23 · 2016: 17 · 2015: 12 · 2014: 11 · 2013: 11 · 2012: 13
FWCI
30.73
Type
journal article
Language
en
Open access
Licence
Locations
  • BMC Medical Education (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • Adelaide Research & Scholarship (AR&S) (University of Adelaide) (submittedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Topics
  • Social Sciences › Education › Educational Assessment and Pedagogy
  • Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Innovations in Medical Education
  • Health Sciences › Medicine › Family Practice › Clinical Reasoning and Diagnostic Skills
Keywords
multiple-choice questions, higher-order thinking skills, clinical problem solving, Bloom's taxonomy, item-writing flaws, test item quality, summative assessment, undergraduate medical education, exit exams, factual recall
MeSH terms
Cognition, Education, Medical, Undergraduate, Educational Measurement, Humans, Problem Solving, Psychometrics, South Australia, Reproducibility of Results

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review