The DOI matches its Crossref record; 53 of the 53 references it cites were found in a registry or an index.
The impact of large scale licensing examinations in highly developed countries: a systematic review
Abstract
BACKGROUND: To investigate the existing evidence base for the validity of large-scale licensing examinations including their impact. METHODS: Systematic review against a validity framework exploring: Embase (Ovid Medline); Medline (EBSCO); PubMed; Wiley Online; ScienceDirect; and PsychINFO from 2005 to April 2015. All papers were included when they discussed national or large regional (State level) examinations for clinical professionals, linked to examinations in early careers or near the point of graduation, and where success was required to subsequently be able to practice. Using a standardized data extraction form, two independent reviewers extracted study characteristics, with the rest of the team resolving any disagreement. A validity framework was used as developed by the American Educational Research Association, American Psychological Association, and National Council on Measurement in Education to evaluate each paper's evidence to support or refute the validity of national licensing examinations. RESULTS: 24 published articles provided evidence of validity across the five domains of the validity framework. Most papers (n = 22) provided evidence of national licensing examinations relationships to other variables and their consequential validity. Overall there was evidence that those who do well on earlier or on subsequent examinations also do well on national testing. There is a correlation between NLE performance and some patient outcomes and rates of complaints, but no causal evidence has been established. CONCLUSIONS: The debate around licensure examinations is strong on opinion but weak on validity evidence. This is especially true of the wider claims that licensure examinations improve patient safety and practitioner competence.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Validity: on the meaningful interpretation of assessment data
Downing, S. M. (2003) · Medical Education · doi:10.1046/j.1365-2923.2003.01594.x · Cited by 1,434
Tamblyn, R., Abrahamowicz, M., Dauphinee, D., Wenghofer, E., Jacques, A., Klass, D., Smee, S., Blackmore, D., Winslade, N., Girard, N., et al. (2007) · JAMA · doi:10.1001/jama.298.9.993 · Cited by 448
Cook, D. A., Zendejas, B., Hamstra, S. J., Hatala, R., & Brydges, R. (2014) · Advances in Health Sciences Education · doi:10.1007/s10459-013-9458-4 · Cited by 317
Selection Criteria for Residency: Results of a National Program Directors Survey
Green, M., Jones, P., & Thomas, J. X. (2009) · Academic Medicine · doi:10.1097/acm.0b013e3181970c6b · Cited by 303
Prober, C. G., Kolars, J. C., First, L., & Melnick, D. E. (2015) · Academic Medicine · doi:10.1097/acm.0000000000000855 · Cited by 226
Doctor scores on national qualifying examinations predict quality of care in future practice
Wenghofer, E. F., KLASS, D. J., Abrahamowicz, M., Dauphinee, D. W., Jacques, A., Smee, S. M., Blackmore, D. E., Winslade, N. E., Reidel, K. E., Bartman, I., et al. (2009) · Medical Education · doi:10.1111/j.1365-2923.2009.03534.x · Cited by 135
Consequences Validity Evidence: Evaluating the Impact of Educational Assessments
Cook, D. A., & Lineberry, M. (2016) · Academic Medicine · doi:10.1097/acm.0000000000001114 · Cited by 129
Norcini, J. J., Boulet, J. R., Opalek, A., & Dauphinée, W. D. (2014) · Academic Medicine · doi:10.1097/acm.0000000000000310 · Cited by 124
Associations between residency selection strategies and doctor performance: a meta‐analysis
Kenny, S. A., McInnes, M. D. F., & Singh, V. (2013) · Medical Education · doi:10.1111/medu.12234 · Cited by 116
Kugler, A. D., & Sauer, R. M. (2005) · Journal of Labor Economics · doi:10.1086/430283 · Cited by 103
Research works printed but not linked: 4
- Archer J, Lynn N, Roberts M, Coombes L, Gale T, Regan de Bere S. A Systematic Review on the impact of licensing examinations for doctors in countries comparable to the UK: Final Report. http://www.gmc-uk.org/A_systematic_review_on_the_impact_of_licensing_examinations__for_doctors_in_countries_comparable_to_the_UK.pdf_61103496.pdf : General Medical Council;2015.
- Popay J, Roberts H, Sowden A, et al. Guidance on the Conduct of Narrative Synthesis in Systematic Reviews. http://www.researchgate.net/profile/Mark_Rodgers4/publication/233866356_Guidance_on_the_conduct_of_narrative_synthesis_in_systematic_reviews_A_product_from_the_ESRC_Methods_Programme/links/02e7e5231e8f3a6183000000.pdf2006 .
- USMLE CEUP. Comprehensive Review of USMLE Summary of Final Report and Recommendations. 2008.
- Sutherland KL, S. Regulation and Quality Improvement a review of the evidence. Health Found. 2006
Cited by
Design, delivery and effectiveness of health practitioner regulation systems: an integrative review
Leslie, K., Bourgeault, I. L., Carlton, A.-L., Balasubramanian, M., Mirshahi, R., Short, S. D., Carè, J., Cometto, G., & Lin, V. (2023) · Human Resources for Health · doi:10.1186/s12960-023-00848-y · Cited by 38
Metadata
- Authors
- Archer, J., Lynn, N., Coombes, L., Roberts, M., Gale, T., Price, T., & Regan de Bere, S.
- Venue
- BMC Medical Education
- Published
- 2016-08-19
- Volume
- 16
- Issue
- 1
- ISSN
- 1472-6920
- Cited by
- 42
- References
- 53
- Language
- en
- Funders
- General Medical Council (GMC189)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
