The DOI matches its Crossref record; 18 of the 18 references it cites were found in a registry or an index.
Estimating Learning Outcomes From Pre- and Posttest Student Self-Assessments: A Longitudinal Study
Abstract
PURPOSE: Learning outcome is an important measure for overall teaching quality and should be addressed by comprehensive evaluation tools. The authors evaluated the validity of a novel evaluation tool based on student self-assessments, which may help identify specific strengths and weaknesses of a particular course. METHOD: In 2011, the authors asked 145 fourth-year students at Göttingen Medical School to self-assess their knowledge on 33 specific learning objectives in a pretest and posttest as part of a cardiorespiratory module. The authors compared performance gain calculated from self-assessments with performance gain derived from formative examinations that were closely matched to these 33 learning objectives. RESULTS: Eighty-three students (57.2%) completed the assessment. There was good agreement between performance gain derived from subjective data and performance gain derived from objective examinations (Pearson r=0.78; P<.0001) on the group level. The association between the two measures was much weaker when data were analyzed on the individual level. Further analysis determined a quality cutoff for performance gain derived from aggregated student self-assessments. When using this cutoff, the evaluation tool was highly sensitive in identifying specific learning objectives with favorable or suboptimal objective performance gains. CONCLUSIONS: The tool is easy to implement, takes initial performance levels into account, and does not require extensive pre-post testing. By providing valid estimates of actual performance gain obtained during a teaching module, it may assist medical teachers in identifying strengths and weaknesses of a particular course on the level of specific learning objectives.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
CSA gain values can range from -100% to +100%, although in reality negative values are rare.
“CSA gain values can range from -100% to +100% (although in reality, negative values are rare).”
From Evaluating AI Courses: A Valid and Reliable Instrument for Assessing Artificial-Intelligence Learning through Comparative Self-Assessment (2023) · cites this paper as [34]
The so-called student comparative self-assessment is a more valid tool to assess the actual increase in competence or knowledge, as it accounts for the initial level of participants' AI literacy.
“The so-called student comparative self-assessment [28,34] is a more valid tool to assess the actual increase in competence or knowledge, as it accounts for the initial level of participants' AI literacy.”
From Evaluating AI Courses: A Valid and Reliable Instrument for Assessing Artificial-Intelligence Learning through Comparative Self-Assessment (2023) · cites this paper as [34]
The use of comparative self-assessment gains is particularly suitable for identifying between-subject differences, as well as differences between individual items.
“However, it should once again be emphasized that the use of comparative self-assessment gains is particularly suitable for identifying between-subject differences, as well as differences between individual items [28,34].”
From Evaluating AI Courses: A Valid and Reliable Instrument for Assessing Artificial-Intelligence Learning through Comparative Self-Assessment (2023) · cites this paper as [34]
Citations
Cites
The meaning and use of the area under a receiver operating characteristic (ROC) curve.
Hanley, J. A., & McNeil, B. J. (1982) · Radiology · doi:10.1148/radiology.143.1.7063747 · Cited by 21,856
Davis, D. A., Mazmanian, P. E., Fordis, M., Van Harrison, R., Thorpe, K. E., & Perrier, L. (2006) · JAMA · doi:10.1001/jama.296.9.1094 · Cited by 2,375
Use and Misuse of the Receiver Operating Characteristic Curve in Risk Prediction
Cook, N. R. (2007) · Circulation · doi:10.1161/circulationaha.106.672402 · Cited by 2,156
The misunderstood limits of folk science: An illusion of explanatory depth
Rozenblit, L., & Keil, F. (2002) · Cognitive Science · doi:10.1207/s15516709cog2605_1 · Cited by 1,032
A review of the validity and accuracy of self-assessments in health professions training
Gordon, M. J. (1991) · Academic Medicine · doi:10.1097/00001888-199112000-00012 · Cited by 537
Measuring Self-assessment: Current State of the Art
Ward, M., Gruppen, L. D., & Regehr, G. (2002) · Advances in Health Sciences Education · doi:10.1023/a:1014585522084 · Cited by 422
Colthart, I., Bagnall, G., Evans, A., Allbutt, H., Haig, A., Illing, J. C., & McKinstry, B. (2008) · Medical Teacher · doi:10.1080/01421590701881699 · Cited by 385
Evaluation of a Medical Faculty Development Program
Skeff, K. M., Stratos, G. A., & Bergen, M. R. (1992) · Evaluation & the Health Professions · doi:10.1177/016327879201500307 · Cited by 233
A longitudinal study of self‐assessment accuracy
Fitzgerald, J. T., White, C. B., & Gruppen, L. D. (2003) · Medical Education · doi:10.1046/j.1365-2923.2003.01567.x · Cited by 178
Course evaluation in medical education
Kogan, J. R., & Shea, J. A. (2007) · Teaching and Teacher Education · doi:10.1016/j.tate.2006.12.020 · Cited by 105
Raupach, T., Münscher, C., Beißbarth, T., Burckhardt, G., & Pukrop, T. (2011) · Medical Teacher · doi:10.3109/0142159x.2011.586751 · Cited by 85
Self-citationImpact of teaching and assessment format on electrocardiogram interpretation skills
Raupach, T., Hanneforth, N. N., Anders, S., Pukrop, T., ten Cate, O., & Harendza, S. (2010) · Medical Education · doi:10.1111/j.1365-2923.2010.03687.x · Cited by 78
Self-citationEnhancing Evaluation in an Undergraduate Medical Education Program
Gibson, K., Boyle, P. M., Black, D. A., Cunningham, M., Grimm, M., & McNeil, H. P. (2008) · Academic Medicine · doi:10.1097/acm.0b013e31817eb8ab · Cited by 54
Using Self-Assessments to Detect Workshop Success
D’Eon, M. F., Sadownik, L. A., Harrison, A., & Nation, J. G. (2008) · American Journal of Evaluation · doi:10.1177/1098214007312630 · Cited by 50
McOwen, K. S., Bellini, L. M., Morrison, G., & Shea, J. A. (2009) · Academic Medicine · doi:10.1097/acm.0b013e3181b6c996 · Cited by 18
Multidimensional Outcome Considerations in Assessing the Efficacy of Medical Educational Programs
Blumberg, P. (2003) · Teaching and Learning in Medicine · doi:10.1207/s15328015tlm1503_10 · Cited by 17
On Aggregation, Generalization, and Utility In Educational Evaluation
Hayman, J. L., Rayder, N., Stenner, A. J., & Madey, D. L. (1979) · Educational Evaluation and Policy Analysis · doi:10.3102/01623737001004031 · Cited by 12
Self-Assessments for Workshop Evaluations
D’Eon, M. F., & Eva, K. W. (2009) · American Journal of Evaluation · doi:10.1177/1098214009334366 · Cited by 5
Flagged first, then by citations
Research works printed but not linked: 3
- Herzig, 2007, GMS Z Med Ausbild, 24, Doc109
- Moltner, 2010, GMS Z Med Ausbild, 27, Doc44
- Raupach, 2012, GMS Z Med Ausbild, 29, Doc44
Cited by
Sorted by citations
Issleib, M., Kromer, A., Pinnschmidt, H. O., Süss-Havemann, C., & Kubitz, J. C. (2021) · Scandinavian Journal of Trauma Resuscitation and Emergency Medicine · doi:10.1186/s13049-021-00836-y · Cited by 81
Schiekirka-Schwake, S., & Raupach, T. (2015) · BMC Medical Education · doi:10.1186/s12909-015-0311-8 · Cited by 75
Self-citationLaupichler, M. C., Hadizadeh, D. R., Wintergerst, M. W. M., von der Emde, L., Paech, D., Dick, E. A., & Raupach, T. (2022) · BMC Medical Education · doi:10.1186/s12909-022-03866-x · Cited by 62
Self-citationMoll-Khosrawi, P., Falb, A., Pinnschmidt, H. O., Zöllner, C., & Issleib, M. (2022) · BMC Medical Education · doi:10.1186/s12909-022-03533-1 · Cited by 58
Laupichler, M. C., Aster, A., Perschewski, J.-O., & Schleiss, J. (2023) · Education Sciences · doi:10.3390/educsci13100978 · Cited by 48
“The so-called student comparative self-assessment [28,34] is a more valid tool to assess the actual increase in competence or knowledge, as it accounts for the initial level of participants’ AI literacy.”
BackgroundSchwerdtfeger, K., Wand, S., Schmid, O., Roeßler, M., Quintel, M., Leissner, K. B., & Russo, S. G. (2014) · BMC Medical Education · doi:10.1186/1472-6920-14-104 · Cited by 36
Schiekirka-Schwake, S., Anders, S., & Raupach, T. (2014) · BMC Medical Education · doi:10.1186/1472-6920-14-149 · Cited by 32
Self-citationKennedy, G., Rea, J. N. M., & Rea, I. M. (2019) · Medical Education Online · doi:10.1080/10872981.2019.1579558 · Cited by 35
Kirsch, J., & Spreckelsen, C. (2023) · BMC Medical Education · doi:10.1186/s12909-023-04258-5 · Cited by 37
Designing Learning Outcomes for Handoff Teaching of Medical Students Using Group Concept Mapping
Hynes, H., Slavi Stoyanov, A., Drachsler, H., Maher, B., Orrego, C., Stieger, L., Druener, S., Sopka, S., Schröder, H., & Henn, P. (2015) · Academic Medicine · doi:10.1097/acm.0000000000000642 · Cited by 30
Xu, X., Bos, N. A., & Wu, H. (2022) · Medical Teacher · doi:10.1080/0142159x.2022.2047168 · Cited by 28
Quantifying teaching quality in medical education: The impact of learning gain calculation
Westphale, S., Backhaus, J., & Koenig, S. (2021) · Medical Education · doi:10.1111/medu.14694 · Cited by 25
WittyFit—Live Your Work Differently: Study Protocol for a Workplace-Delivered Health Promotion
Dutheil, F., Duclos, M., Naughton, G. A., Dewavrin, S., Cornet, T., Huguet, P., Chatard, J., & Pereira, B. (2017) · JMIR Research Protocols · doi:10.2196/resprot.6267 · Cited by 27
Kostusiak, M., Hart, M. G., Barone, D. G., Hofmann, R., Kirollos, R. W., Santarius, T., & Trivedi, R. A. (2017) · Medical Teacher · doi:10.1080/0142159x.2017.1362102 · Cited by 22
Autopsy in undergraduate medical education—what do students really learn?
Anders, S., Mueller, M., Sperhake, J., Petersen‐Ewert, C., Schiekirka-Schwake, S., & Raupach, T. (2014) · International Journal of Legal Medicine · doi:10.1007/s00414-014-0974-4 · Cited by 19
Self-citationGerman Medical Students' Beliefs About the Effectiveness of Different Methods of Stopping Smoking
Raupach, T., Strobel, L., Beard, E., Krampe, H., Anders, S., & West, R. (2013) · Nicotine & Tobacco Research · doi:10.1093/ntr/ntt078 · Cited by 18
Self-citationPost mortem CT scans as a supplementary teaching method in gross anatomy
Buenting, M., Mueller, T., Raupach, T., Lüers, G. H., Wehrenberg, U., Gehl, A., & Anders, S. (2016) · Annals of Anatomy - Anatomischer Anzeiger · doi:10.1016/j.aanat.2016.05.003 · Cited by 18
Self-citationA Course-Based Undergraduate Research Experience Improves Outcomes in Mentored Research
Fendos, J., Cai, L., Yang, X., Ren, G., Li, L., Yan, Z., Lu, B., ., Y. P., Ma, J., Guo, B., et al. (2022) · CBE—Life Sciences Education · doi:10.1187/cbe.21-03-0065 · Cited by 17
Gross, I., & Gross, I. (2022) · IEEE Global Engineering Education Conference (EDUCON) · doi:10.1109/educon52537.2022.9766686 · Cited by 17
Slavi Stoyanov, A., Spoelstra, H., Bennett, D. M., Sweeney, C., Huffel, S. V., Shorten, G., O’Flynn, S., Cantillon‐Murphy, P., O’Tuathaigh, C. M. P., & Burgoyne, L. N. (2013) · Perspectives on Medical Education · doi:10.1007/s40037-013-0095-7 · Cited by 15
Kouz, K., Eisenbarth, S., Bergholz, A., & Mohr, S. (2020) · PLoS ONE · doi:10.1371/journal.pone.0239928 · Cited by 11
Marciniak, K., Scherg, A., Paal, P., Mason, S. R., & Elsner, F. (2023) · BMC Palliative Care · doi:10.1186/s12904-023-01217-1 · Cited by 13
Grebener, B.-L., Barth, J., Anders, S., Beißbarth, T., & Raupach, T. (2021) · Medical Teacher · doi:10.1080/0142159x.2020.1867714 · Cited by 9
Self-citationImproving evaluation at two medical schools
Schiekirka-Schwake, S., Dreiling, K., Pyka, K., Anders, S., von Steinbüchel, N., & Raupach, T. (2017) · The Clinical Teacher · doi:10.1111/tct.12686 · Cited by 7
Self-citationA Weighted Evaluation Study of Clinical Teacher Performance at Five Hospitals in the UK
Sam, A. H., Fung, C. Y., Barth, J., & Raupach, T. (2021) · Advances in Medical Education and Practice · doi:10.2147/amep.s322105 · Cited by 7
Self-citationHow self-reliant Peer Teaching can be set up to augment learning outcomes for university learners
Reith, D., Grein, M., Röthgen, D., Behrendt, J., Held, T., Malschützky, M. M., & Gross, I. (2022) · IEEE Global Engineering Education Conference (EDUCON) · doi:10.1109/educon52537.2022.9766811 · Cited by 7
Anders, S., Pyka, K., Mueller, T., von Streinbuechel, N., & Raupach, T. (2016) · Annals of Anatomy - Anatomischer Anzeiger · doi:10.1016/j.aanat.2016.07.008 · Cited by 6
Self-citationSelf-Assessment and Reflection in a 1st Semester Course for Software Engineering Students
Nielsen, J., Majgaard, G., & Sørensen, E. (2015) · Springer eBooks · doi:10.1007/978-3-319-05825-2_18 · Cited by 5
Cenaj, D., Schulte‐Uentrop, L., Kröger, L. F. L., Küllmei, J., Haus, J.-M., & Moll-Khosrawi, P. (2025) · Journal of Medical Education and Curricular Development · doi:10.1177/23821205251358090 · Cited by 4
Issleib, M., Geiger, A., Pinnschmidt, H. O., Hilz, P., Zöllner, C., & Moll-Khosrawi, P. (2025) · Virtual Reality · doi:10.1007/s10055-025-01216-y · Cited by 4
An Approach to Assessing Learning Outcomes over Consecutive Courses
Rothe, I., Winzker, M., & Salega, C. (2018) · IEEE Frontiers in Education Conference (FIE) · doi:10.1109/fie.2018.8658562 · Cited by 4
Gross, I., Reith, D., Grein, M., & Gros, I. (2019) · IEEE Global Engineering Education Conference (EDUCON) · doi:10.1109/educon.2019.8725111 · Cited by 4
Multipurpose Use of Quizzes in Teaching
Rothe, I., Pustowalow, W., & Winzker, M. (2021) · IEEE Global Engineering Education Conference (EDUCON) · doi:10.1109/educon46332.2021.9453875 · Cited by 4
Teaching Note—Teaching Students About Homelessness: A Model for Curriculum Integration
Aykanian, A., & Ryan-Dedominicis, T. (2021) · Journal of Social Work Education · doi:10.1080/10437797.2021.1997688 · Cited by 4
Position statement of the GMA committee “teaching evaluation”
Haverkamp, N., Barth, J., Schmidt, D., Dahmen, U., Keis, O., & Raupach, T. (2024) · PubMed Central · doi:10.3205/zma001674 · Cited by 2
Self-citationSubjective learning gain from a simulation-based health management course: a mixed methods study
Gu, R.-P., Zhao, F. R., Bai, J., Sun, S., & Zhu, A.-Y. (2024) · Frontiers in Public Health · doi:10.3389/fpubh.2024.1400135 · Cited by 2
Abina, A., Kovačič, D., Prucnal, M., Kiratzouli, V., & Zidanšek, A. (2025) · Sustainability · doi:10.3390/su17020412 · Cited by 2
Adams, J., Stosch, C., Mallmann, M. R., Adams, N.-B., & Ludwig, S. (2025) · Frontiers in Medicine · doi:10.3389/fmed.2025.1716282 · Cited by 2
Hausdörfer, T., Kanzow, P., Tschammler, C., Herrmann, M., & Wiegand, A. (2018) · Journal of Dental Education · doi:10.21815/jde.018.092 · Cited by 2
Outcome Assessment of a Writing Skill Improvement Initiative for Management Graduates
Deepa, R., & Gayathridevi, K. S. (2017) · International Journal of Educational Sciences · doi:10.1080/09751122.2017.1355607 · Cited by 2
Black Ice: Five ways to get a grip on grouped self-assessments of competence for program evaluation
Zhao, R., & D’Eon, M. F. (2020) · Canadian Medical Education Journal · doi:10.36834/cmej.69276 · Cited by 2
Fan, Y., Hanna, C., Folger, T. D., May, T. A., & Koskey, K. L. K. (2025) · Education Sciences · doi:10.3390/educsci15121709 · Cited by 1
Notfallkompetenzen für Medizinstudierende
Noll, C., Golon, S., Kloppenburg, K., Kluwe, L.-S., Triefenbach, A. M., Behrends, M., Mücke, U., Mikuteit, M., & Steffens, S. (2024) · Notfall + Rettungsmedizin · doi:10.1007/s10049-024-01307-4 · Cited by 1
Are Students Competent? Methods of Assessing Bachelor of Social Work Student Competence
Bailey-Residori, A. L. (2017) · Digital Commons - Olivet Distinctive Heritage, Scholarship, & Creativity (Olivet Nazarene University) · Cited by 1
A New Method for Setting Standard in Medical Education, Applying Previous Year Cumulative GPA
Aung, M. N., Jaroonvanichkul, V., Deerojanawong, J., Somboonwong, J., Ahmad, I., & Wannakrairot, P. (2019) · European Journal of Medical and Health Sciences · doi:10.24018/ejmed.2019.1.5.114 · Cited by 1
E-learning during the COVID-19 pandemic: the impact on medical students’ perceived academic progress
Vîlceleanu, B.-V., Bălan, A. M., Jugănaru, E., Merlușcă, E., Rașcu, A., & Oțelea, M. R. (2021) · Romanian Journal of Occupational Medicine · doi:10.2478/rjom-2021-0005 · Cited by 1
Diabetes among African Refugees: Addressing Risks Awareness through Culturally Sensitive Education
Maduma, D. (2018) · UA Campus Repository (The University of Arizona)
An Effective Algorithm to Enhance the Performance of Online Quiz System
Srinayana, A., & Praveenchandar, J. (2024) · International Conference on Sustainable Computing and Smart Systems (ICSCSS) · doi:10.1109/icscss60660.2024.10625061
Laupichler, M. C., Hadizadeh, D. R., Wintergerst, M. W. M., von der Emde, L., Paech, D., Dick, E., & Raupach, T. (2022) · Figshare · doi:10.6084/m9.figshare.21582652
Self-citationLaupichler, M. C., Hadizadeh, D. R., Wintergerst, M. W. M., von der Emde, L., Paech, D., Dick, E., & Raupach, T. (2022) · Figshare · doi:10.6084/m9.figshare.21582658
Self-citationIssleib, M., Kromer, A., Pinnschmidt, H., Süss-Havemann, C., & Kubitz, J. C. (2021) · Figshare · doi:10.6084/m9.figshare.13684982
Issleib, M., Kromer, A., Pinnschmidt, H., Süss-Havemann, C., & Kubitz, J. C. (2021) · Figshare · doi:10.6084/m9.figshare.13684988
Issleib, M., Kromer, A., Pinnschmidt, H., Süss-Havemann, C., & Kubitz, J. C. (2021) · Figshare · doi:10.6084/m9.figshare.13684985
Laupichler, M. C., Hadizadeh, D. R., Wintergerst, M. W. M., von der Emde, L., Paech, D., Dick, E., & Raupach, T. (2022) · Figshare · doi:10.6084/m9.figshare.21582661
Self-citationMarciniak, K., Scherg, A., Paal, P., Mason, S. R., & Elsner, F. (2024) · Figshare · doi:10.6084/m9.figshare.26603923
Aliyev, R., Backhaus, J., Westphale, S., & Koenig, S. (2026) · JMIR Medical Education · doi:10.2196/75516
Metadata
- Authors
- Schiekirka, S., Reinhardt, D., Beibarth, T., Anders, S., Pukrop, T., & Raupach, T.
- Affiliations
- Universitätsmedizin Göttingen (DE)
- Corresponding author
- Raupach, Tobias
- Venue
- Academic Medicine
- Publisher
- Oxford University Press (OUP)
- Published
- 2013-03
- Volume
- 88
- Issue
- 3
- Pages
- 369-375
- ISSN
- 1040-2446 (ISSN-L)
- OpenAlex
- W2329021261
- Cited by
- 56
- References
- 18
- Citations by year
- 2026: 1 · 2025: 5 · 2024: 5 · 2023: 3 · 2022: 9 · 2021: 10 · 2020: 2 · 2019: 3 · 2018: 3 · 2017: 5 · 2016: 2 · 2015: 3 · 2014: 3 · 2013: 2
- FWCI
- 3.40
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Academic Medicine (publishedVersion)
- PubMed (publishedVersion)
- GRO (submittedVersion)
- Topics
- Social Sciences › Education › Innovations in Medical Education
- Social Sciences › Education › Student Assessment and Feedback
- Social Sciences › Education › Educational Assessment and Pedagogy
- Keywords
- learning outcomes, student self-assessment, pretest-posttest, validity, performance improvement, formative assessment, learning objectives, medical education, teaching quality evaluation, medical students
- MeSH terms
- Adult, Education, Medical, Undergraduate, Educational Measurement, Female, Germany, Humans, Longitudinal Studies, Male, ROC Curve, Self-Assessment, Reproducibility of Results
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
