CoChat
Cited by 9

The DOI matches its Crossref record; 99 of the 99 references it cites were found in a registry or an index.

Angoff methods in standard setting in health professional education: a systematic review and meta-analysis

Sridharan, Kannan, Sivaramakrishnan, Gowri · 2025 · BMC Medical Education

Abstract

BACKGROUND: Standard setting is a critical component of assessment in health professions education, ensuring fairness, defensibility, and validity in determining minimum competence. Among criterion-referenced approaches, the Angoff method and its variants are widely applied. However, the evidence base on their comparative performance remains fragmented, with no recent quantitative synthesis available. This systematic review and meta-analysis aimed to evaluate the outcomes of Angoff methods across different educational and assessment contexts. METHODS: A systematic electronic search was conducted for studies that applied Angoff or its variants [Modified, Yes/No, Three-level, Group (Yes/No), or Mastery] in health professions assessments and reported at least one of the following: cut score, pass rate, or inter-rater reliability. Pooled estimates were calculated using generalized linear mixed models and random-effects meta-analysis. Odds ratio (OR) was used as the effect estimate for pass rate in comparative studies. Meta-regression analyses explored heterogeneity by number of judges, and test length. RESULTS: A total of 91 studies were included in the systematic review. Mixed treatment comparison pooled estimates revealed significantly higher pass rates with Angoff (OR: 7.48) compared to the conventional fixed method. Mastery Angoff (88.2%) and Mastery Angoff with reality check (86.9%) were associated with higher cut scores, resulting in substantially lower pass rates (61.44% for Mastery Angoff). The Modified Angoff with reality check achieved excellent reliability (r = 0.917) while the Angoff (Yes/No) yielded the weakest and most variable results (r = 0.536), a finding confirmed by bootstrap analyses. Meta-regression indicated that each additional judge was associated with a 0.19-percentage point increase in cut scores (p = 0.003) while each additional test item was associated with a 0.05-percentage point increase in the pass rate (p = 0.001). CONCLUSION: For educators and policymakers, these findings underscore that the Angoff method is not a single technique but a flexible family of approaches with distinct outcomes. The choice of variants allows for the deliberate setting of more lenient or stringent standards, with Modified Angoff incorporating a reality check offering the most reliable and defensible results. This flexibility is a key strength but also a critical limitation, as an uninformed choice can lead to unintended pass rates. Therefore, selecting a specific Angoff variant must be a conscious decision aligned with the assessment's purpose and context.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1186/s12909-025-08300-6

Cites

99 of 99 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 99 references

Research works printed but not linked: 11

  1. Mortaz Hejri S, Jalili M. Standard setting in medical education: fundamental concepts and emerging challenges. Med J Islam Repub Iran. 2014;28:34.
  2. Park J. Possibility of using the yes/no Angoff method as a substitute for the percent Angoff method for estimating the cutoff score of the Korean medical licensing examination: a simulation study. J Educ Eval Health Prof. 2022;19:23. · DOI
  3. Anderson HG Jr, Nelson AA. Reliability and credibility of progress test criteria developed by alumni, faculty, and mixed alumni-faculty judge panels. Am J Pharm Educ. 2011;75(10):200. · DOI
  4. Chan SC, Amin SM, Lee TW. Implementing standard setting into the conjoint MAFP/FRACGP part 1 examination–process and issues. Malaysian Family Physician: Official J Acad Family Physicians Malaysia. 2016;11(2–3):2.
  5. Kamath MG, Pallath V, Ramnarayan K, Kamath A, Torke S, Gonsalves J. Standard setting of objective structured practical examination by modified Angoff method: A pilot study. Natl Med J India. 2016;29(3):160.
  6. Lee M, Hernandez E, Brook R, Ha E, Harris C, Plesa M, et al. Competency-based standard setting for a high-stakes objective structured clinical examination (OSCE): validity evidence. MedEdPublish. 2018;7:200. · DOI
  7. Park J, Yim MK, Kim NJ, Ahn DS, Kim YM. Similarity of the cut score in test sets with different item amounts using the modified Angoff, modified Ebel, and Hofstee standard-setting methods for the Korean medical licensing examination. J Educ Eval Health Prof. 2020;17:28. · doi:10.3352/jeehp.2020.17.28
  8. Park J. Possibility of using the yes/no Angoff method as a substitute for the percent Angoff method for estimating the cutoff score of the Korean medical licensing examination: a simulation study. J Educ Eval Health Prof. 2022;19:23. · DOI
  9. Sam AH, Millar KR, Westacott R, Melville CR, Brown CA. Standard setting very short answer questions (VSAQs) relative to single best answer questions (SBAQs): does having access to the answers make a difference? BMC Med Educ. 2022;22(1):640. · doi:10.1186/s12909-022-03693-0
  10. Tavakol M, O’Brien D, Stewart C. Determining intra-standard-setter inconsistency in the Angoff method using the three-parameter item response theory. Int J Med Educ. 2023;14:123. · doi:10.5116/ijme.64ed.e296
  11. Yim MK, Shin S. Using the Angoff method to set a standard on mock exams for the Korean nursing licensing examination. J Educ Eval Health Prof. 2020;17:14. · doi:10.3352/jeehp.2020.17.14

Metadata

Authors
Sridharan, K., & Sivaramakrishnan, G.
Venue
BMC Medical Education
Published
2025-12-29
Volume
25
Issue
1
ISSN
1472-6920
Cited by
9
References
99
Language
en
Licence

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Angoff methods in standard setting in health professional education: a systematic review and meta-analysis. https://app.cochat.ai/papers/10.1186/s12909-025-08300-6
The paper
Sridharan, K., & Sivaramakrishnan, G. (2025). Angoff methods in standard setting in health professional education: a systematic review and meta-analysis. BMC Medical Education. https://doi.org/10.1186/s12909-025-08300-6

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.