Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat
Sign Up
Cited by 6Innovations in Medical EducationPsychometric Methodologies and TestingMedical Education and Admissions

The DOI matches its Crossref record; 34 of the 34 references it cites were found in a registry or an index.

Psychometric evaluation of a national exam for clinical undergraduates

Open access
Han, Yuting, Jiang, Zhehan, Ouyang, Jinying, Xu, Lingling, Cai, Ting·2022·Frontiers in Medicine
DOI 10.3389/fmed.2022.1037897PMC PMC9794566
Open this paper in CoChat — free accountOpen PDF

Abstract

Background: As a high-stake national-level examination administrated before students’ clerkship in China, the Standardized Competence Test for Clinical Medicine Undergraduates (SCTCMU) has received much attention from the relevant educational departments and society at large. Investigating SCTCMU’s validity and reliability is critical to the national healthcare profession education. Materials and methods: Raw responses from SCTCMU, answered by 44,332 examines of 4th-year undergraduate medical students on 300 multiple-choice items, were used to evaluate the quality of the exam via psychometric methods based on item response theory (IRT). The core assumptions and model-data fit of IRT models were evaluated, as well as the item properties and information functions. Results: The IRT models were fitted to the observed assessment data, where all the required assumptions were met. The IRT analysis showed that most items had acceptable psychometric properties, and the passing score was located close to the lowest measurement error computed from the model outcomes. Conclusion: The proposed modern psychometric method provides a practical and informative approach to calibrating and analyzing medical education assessments. This work showcases a realistic depiction of the IRT analysis process and therefore facilitates the work of applied researchers wanting to conduct, interpret, and report IRT analyses on medical assessments.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.3389/fmed.2022.1037897

Cites

34 of 34 references verified0 citation errors · 0 not checkable

Statistics

Figures

FIGURE 1 The steps of using psychometric methods to evaluate the quality of medical education assessments.
FIGURE 1 The steps of using psychometric methods to evaluate the quality of medical education assessments.CC BY · PubMed Central
FIGURE 2 Statistical and item response theory (IRT) analysis results of Standardized Competence Test for Clinical Medicine Undergraduates (SCTCMU). (A) Total score distribution. (B) Test characteristic curve for the 3PL model. (C) Total information function and expected SE function. (D) Quality of item discriminations. (E) Difficulty and ability estimates distribution. (F) Guessing parameter distribution.
FIGURE 2 Statistical and item response theory (IRT) analysis results of Standardized Competence Test for Clinical Medicine Undergraduates (SCTCMU). (A) Total score distribution. (B) Test characteristic curve for the 3PL model. (C) Total information function and expected SE function. (D) Quality of item discriminations. (E) Difficulty and ability estimates distribution. (F) Guessing parameter distribution.CC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Research works printed but not linked: 5

  1. Baker, 2001, The Basics of Item Response Theory. Clearinghouse on Assessment and Evaluation.
  2. Revelle, 2022, psych: Procedures for Psychological, Psychometric, and Personality Research. R package Version 2.2.9.
  3. Angoff, 1971, Scales, norms, and equivalent scores., Educational Measurement., p. 508
  4. Linacre, 1989, Many-Faceted Rasch Measurement.
  5. 2014, Standards for Educational and Psychological Testing.

Cited by

See all 6

Related

See all

Metadata

Authors
Han, Y., Jiang, Z., Ouyang, J., Xu, L., & Cai, T.
Affiliations
Peking University (CN)
Corresponding author
Jiang, Zhehan
ORCID
Venue
Frontiers in Medicine
Publisher
Frontiers Media SA
Published
2022-12-14
Volume
9
Pages
1037897-1037897
ISSN
2296-858X (ISSN-L)
DOI
10.3389/fmed.2022.1037897
PMC
PMC9794566
OpenAlex
W4311496642
Cited by
5
References
34
Citations by year
2026: 1 · 2025: 1 · 2024: 3
FWCI
0.88
Type
journal article
Language
en
Open access
Licence
Locations
  • Frontiers in Medicine (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
  • National Natural Science Foundation of China
  • Peking University Health Science Center
Topics
  • Social Sciences › Education › Innovations in Medical Education
  • Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
  • Social Sciences › Education › Medical Education and Admissions
Keywords
national medical licensing examination, clinical competence assessment, medical undergraduates, test validity, test reliability, item response theory, item calibration, passing scores, measurement error, medical education assessment

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.