The DOI matches its Crossref record; the references it cites are not checked yet.
Towards outcome-based programme evaluation: Using student comparative self-assessments to determine teaching effectiveness
Open accessAbstract
BACKGROUND: Programme evaluation of medical education should be multi-dimensional. While structural and organisational aspects of teaching are frequently assessed, programme evaluation tools are rarely matched to specific learning objectives. AIMS: This study used one medical school's catalogue of specific learning objectives to implement and critically appraise a novel programme evaluation tool based on comparative student self-assessments. METHOD: Medical students enrolled in the clinical phase of the undergraduate curriculum in Göttingen were invited to self-rate their knowledge, skills and attitudes before and after each course. A newly developed formula controlling for student performance levels when entering a course was used to compute a percentage gain in knowledge, skills and attitudes. Data derived from a prospective, longitudinal intervention study on the development of electrocardiogram interpretation skills including 636 students from four consecutive cohorts were used to provide validity evidence of the new approach. RESULTS: The novel tool appeared superior to plain mean differences and effect sizes in detecting outstanding teaching as well as shortcomings of the curriculum. In addition, it adequately reflected objectively measured performance levels and was responsive to curriculum change. CONCLUSIONS: Comparative student self-assessment is a valid tool to appraise undergraduate medical curricula at the level of specific learning objectives.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The so-called student comparative self-assessment is a more valid tool to assess the actual increase in competence or knowledge, as it accounts for the initial level of participants' AI literacy.
“The so-called student comparative self-assessment [28,34] is a more valid tool to assess the actual increase in competence or knowledge, as it accounts for the initial level of participants' AI literacy.”
From Evaluating AI Courses: A Valid and Reliable Instrument for Assessing Artificial-Intelligence Learning through Comparative Self-Assessment (2023) · cites this paper as [28]
In ‘Development of the new programme evaluation tool’ (page unavailable), the authors calculate gain from pre- and post-course self-assessments and adjust for differing initial performance levels. Their study concerns medical education, not participants’ AI literacy. In ‘Limitations and suggestions for future research’ (page unavailable), they say further research is needed to establish the approach’s validity for factual knowledge, so the claim that it is a more valid measure of actual knowledge gain overstates the evidence. (LLM, conf=0.88)
Experimental — accuracy not yet measuredThe use of comparative self-assessment gains is particularly suitable for identifying between-subject differences, as well as differences between individual items.
“However, it should once again be emphasized that the use of comparative self-assessment gains is particularly suitable for identifying between-subject differences, as well as differences between individual items [28,34].”
From Evaluating AI Courses: A Valid and Reliable Instrument for Assessing Artificial-Intelligence Learning through Comparative Self-Assessment (2023) · cites this paper as [28]
The 'Comparison of different methods measuring performance increase' section (page unavailable) reports that CSA gain distinguished individual learning objectives, supporting the item-level part of the claim. The 'Limitations and suggestions for future research' section says cross-module comparison is an 'intriguing possibility,' not an established strength; the paper does not demonstrate particular suitability for identifying between-subject differences. (LLM, conf=0.83)
Experimental — accuracy not yet measuredThe retrospective/post method is often more suitable for assessing learning gains than the traditional pre/post test because it is subject to fewer biases.
“The retrospective/post method is often more suitable for assessing learning gains than the traditional pre/post test (one assessment before and one after a course, [28]) because it is subject to fewer biases.”
From Evaluating AI Courses: A Valid and Reliable Instrument for Assessing Artificial-Intelligence Learning through Comparative Self-Assessment (2023) · cites this paper as [28]
The original SNAIL scale was originally validated in English.
“the original scale by Laupichler et al. [28] was originally validated in English”
From Evaluating AI Courses: A Valid and Reliable Instrument for Assessing Artificial-Intelligence Learning through Comparative Self-Assessment (2023) · cites this paper as [28]
The Development of the new programme evaluation tool section describes a comparative self-assessment (CSA) tool, not the SNAIL scale. The Conclusion says its sample questions were translated from German into English, but does not address the language in which SNAIL was originally validated. Searches for SNAIL and Laupichler found no relevant passage in our copy. (LLM, conf=0.76)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
The citation lists haven’t been fetched yet
References 26 · Cited by 85
Research works printed but not linked: 5
- Cohen J, 1992, Psychol Bull, 112, 155, 10.1037/0033-2909.112.1.155 · DOI · Paper page
- Frank E, 2005, MedGenMed, 7, 2
- Gerrity MS, 1998, Acad Med, 73, S55
- Marsh HW, 1982, Br J Psychol, 52, 77
- Naftulin DH, 1973, J Med Educ, 48, 630
Metadata
- Authors
- Raupach, T., Münscher, C., Beißbarth, T., Burckhardt, G., & Pukrop, T.
- Affiliations
- Universitätsmedizin Göttingen (DE)
- Corresponding author
- Raupach, T.
- Venue
- Medical Teacher
- Publisher
- Informa UK Limited
- Published
- 2011-07-20
- Volume
- 33
- Issue
- 8
- Pages
- e446-e453
- ISSN
- 0142-159X (ISSN-L)
- 1466-187X
- OpenAlex
- W2140263226
- Cited by
- 85
- References
- 26
- Citations by year
- 2026: 2 · 2025: 13 · 2024: 8 · 2023: 10 · 2022: 19 · 2021: 9 · 2020: 5 · 2019: 1 · 2017: 1 · 2016: 3 · 2015: 3 · 2014: 4 · 2013: 3 · 2012: 3
- FWCI
- 3.13
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Medical Teacher (publishedVersion)
- PubMed (publishedVersion)
- UCL Discovery (University College London) (submittedVersion)
- GRO (submittedVersion)
- Topics
- Social Sciences › Education › Innovations in Medical Education
- Health Sciences › Health Professions › General Health Professions › Health and Medical Education
- Social Sciences › Education › Evaluation of Teaching Practices
- Keywords
- program evaluation, medical education, student self-assessment, teaching effectiveness, learning objectives, ECG interpretation, undergraduate medical curriculum, curriculum evaluation, knowledge, skills and attitudes, effect size
- MeSH terms
- Clinical Competence, Curriculum, Education, Medical, Undergraduate, Educational Measurement, Educational Status, Germany, Humans, Health Knowledge, Attitudes, Practice, Learning, Professional Competence, Schools, Medical, Self-Assessment, Statistics as Topic, Students, Medical, Teaching, Program Evaluation, Statistics, Nonparametric
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
