A Systematic Review of the Relationship Between In-Training Examination Scores and Specialty Board Examination Scores

McCrary, Hilary C.; Colbert-Getz, Jorie M.; Poss, W. Bradley; Smith, Brigitte K.

doi:10.4300/jgme-d-20-00111.1

Cited by 36Innovations in Medical EducationMedical Education and AdmissionsClinical Reasoning and Diagnostic Skills

A Systematic Review of the Relationship Between In-Training Examination Scores and Specialty Board Examination Scores

Open access

McCrary, Hilary C., Colbert-Getz, Jorie M., Poss, W. Bradley, Smith, Brigitte K.·2021·Journal of Graduate Medical Education

DOI 10.4300/jgme-d-20-00111.1PMC PMC7901636

Open this paper in CoChat — free account

Abstract

Background: In-training examinations (ITEs) are intended for low-stakes, formative assessment of residents' knowledge, but are increasingly used for high-stake purposes, such as to predict board examination failures. Objective: The aim of this review was to investigate the relationship between performance on ITEs and board examination performance across medical specialties. Methods: A search of the literature for studies assessing the strength of the relationship between ITE and board examination performance from January 2000 to March 2019 was completed. Results were categorized based on the type of statistical analysis used to determine the relationship between ITE performance and board examination performance. Results: Of 1407 articles initially identified, 89 articles underwent full-text review, and 32 articles were included in this review. There was a moderate-strong relationship between ITE and board examination performance, and ITE scores significantly predict board examination scores for the majority of studies. Performing well on an ITE predicts a passing outcome for the board examination, but there is less evidence that performing poorly on an ITE will result in failing the associated specialty board examination. Conclusions: There is a moderate to strong correlation between ITE performance and subsequent performance on board examinations. That the predictive value for passing the board examination is stronger than the predictive value for failing calls into question the “common wisdom” that ITE scores can be used to identify “at risk” residents. The graduate medical education community should continue to exercise caution and restraint in using ITE scores for moderate to high-stakes decisions.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Integrity signals

Registry

Crossref · 10.4300/jgme-d-20-00111.1

Cites

38 of 38 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Average human performance on mARC-QA (66%) is comparable to typical accuracy on board examinations and in-training assessments.
“comparable to typical accuracy on board examinations and in-training assessments70”
From Limitations of large language models in clinical problem-solving arising from inflexible reasoning (2025) · cites this paper as [70]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

A power primer.
Cohen, J. (1992) · Psychological Bulletin · doi:10.1037//0033-2909.112.1.155 · Cited by 8,771 · see citing works
Meaning and Values in Test Validation: The Science and Ethics of Assessment
Messick, S. (1989) · Educational Researcher · doi:10.3102/0013189x018002005 · Cited by 948 · see citing works
Appraising the Quality of Medical Education Research Methods: The Medical Education Research Study Quality Instrument and the Newcastle–Ottawa Scale-Education
Cook, D. A., & Reed, D. A. (2015) · Academic Medicine · doi:10.1097/acm.0000000000000786 · Cited by 781 · see citing works
Predictive Validity Evidence for Medical Education Research Study Quality Instrument Scores: Quality of Submissions to JGIM’s Medical Education Special Issue
Reed, D. A., Beckman, T. J., Wright, S. M., Levine, R. B., Kern, D. E., & Cook, D. A. (2008) · Journal of General Internal Medicine · doi:10.1007/s11606-008-0664-3 · Cited by 284 · see citing works
Predicting Performance on the American Board of Surgery Qualifying and Certifying Examinations
de Virgilio, C. (2010) · Archives of Surgery · doi:10.1001/archsurg.2010.177 · Cited by 193 · see citing works
Do Scores of the USMLE Step 1 and OITE Correlate with the ABOS Part I Certifying Examination?: A Multicenter Study
Dougherty, P. J., Walter, N. E., Schilling, P. L., Najibi, S., & Herkowitz, H. N. (2010) · Clinical Orthopaedics and Related Research · doi:10.1007/s11999-010-1327-3 · Cited by 125 · see citing works
Using the American Board of Surgery In-Training Examination to Predict Board Certification: A Cautionary Study
Jones, A. T., Biester, T. W., Buyske, J., Lewis, F. R., & Malangoni, M. A. (2014) · Journal of surgical education · doi:10.1016/j.jsurg.2014.04.004 · Cited by 85 · see citing works
The Relationship Between Internal Medicine Residency Graduate Performance on the ABIM Certifying Examination, Yearly In-Service Training Examinations, and the USMLE Step 1 Examination
Kay, C., Jackson, J. L., & Frank, M. (2014) · Academic Medicine · doi:10.1097/acm.0000000000000500 · Cited by 84 · see citing works
Passing the Boards
Klein, G. R., Austin, M. S., Randolph, S. R., Sharkey, P. F., & Hilibrand, A. S. (2004) · Journal of Bone and Joint Surgery · doi:10.2106/00004623-200405000-00032 · Cited by 80 · see citing works
The In-Training Examination: An Analysis of Its Predictive Value on Performance on the General Pediatrics Certification Examination
Althouse, L. A., & McGuinness, G. A. (2008) · The Journal of Pediatrics · doi:10.1016/j.jpeds.2008.03.012 · Cited by 74 · see citing works
Predicting Success on the Certification Examinations of the American Board of Anesthesiology
McClintock, J. C., & Gravlee, G. P. (2009) · Anesthesiology · doi:10.1097/aln.0b013e3181c62e2f · Cited by 65 · see citing works
The Predictive Validity of the Internal Medicine In-Training Examination
Babbott, S. F., Beasley, B. W., Hinchey, K. T., Blotzer, J. W., & Holmboe, E. S. (2007) · The American Journal of Medicine · doi:10.1016/j.amjmed.2007.05.003 · Cited by 64 · see citing works
Utility of AAOS OITE Scores in Predicting ABOS Part I Outcomes: AAOS Exhibit Selection
Swanson, D., Marsh, J. L., Hurwitz, S., DeRosa, G. P., Holtzman, K., Bucak, S. D., Baker, A., & Morrison, C. (2013) · The Journal of Bone and Joint Surgery-American Volume · doi:10.2106/jbjs.l.00457 · Cited by 47 · see citing works
A Comparative Study of Resident Performance on Standardized Training Examinations and the American Board of Ophthalmology Written Examination
Johnson, G. A., Bloom, J. N., Szczotka‐Flynn, L. B., Zauner, D., & Tomsak, R. L. (2010) · Ophthalmology · doi:10.1016/j.ophtha.2010.03.056 · Cited by 37 · see citing works
Relationship Between Resident-In-Training Examination in Psychiatry and Subsequent Certification Examination Performances
Juul, D., Schneidman, B. S., Sexson, S. B., Fernandez, F., Beresin, E. V., Ebert, M. H., Winstead, D. K., & Faulkner, L. R. (2009) · Academic Psychiatry · doi:10.1176/appi.ap.33.5.404 · Cited by 32 · see citing works
Resident Performance on the In-Training and Board Examinations in Obstetrics and Gynecology: Implications for the ACGME Outcome Project
Withiam-Leitch, M. L., & Olawaiye, A. B. (2008) · Teaching and Learning in Medicine · doi:10.1080/10401330801991642 · Cited by 31 · see citing works
The Predictive Validity of the ABFM's In-Training Examination.
O’Neill, T. R., Li, Z., Peabody, M. R., Lybarger, M., Royal, K. D., & Puffer, J. C. (2015) · PubMed · Cited by 30 · see citing works
Frequently Asked Questions
(2021) · American Public Health Association eBooks · doi:10.2105/9780875533162bm3 · Cited by 28 · see citing works
The Relationship Between Performance on the Infectious Diseases In-Training and Certification Examinations
Grabovsky, I., Hess, B., Haist, S. A., Lipner, R. S., Hawley, J. L., Woodward, S., & Engleberg, N. C. (2014) · Clinical Infectious Diseases · doi:10.1093/cid/ciu906 · Cited by 27 · see citing works
Otolaryngology Resident In‐Service Examination Scores Predict Passage of the Written Board Examination
Puscas, L. (2012) · Otolaryngology · doi:10.1177/0194599812444386 · Cited by 26 · see citing works
Performance on the Cardiovascular In-Training Examination in Relation to the ABIM Cardiovascular Disease Certification Examination
Indik, J. H., Duhigg, L. M., McDonald, F. S., Lipner, R. S., Rubright, J. D., Haist, S. A., Botkin, N. F., & Kuvin, J. T. (2017) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2017.04.020 · Cited by 26 · see citing works
Association between performance on Neurology In-Training and Certification Examinations
Juul, D., Flynn, F. G., Gutmann, L., Pascuzzi, R. M., Webb, L. C., Massey, J. M., DeKosky, S. T., Foertsch, M., & Faulkner, L. R. (2013) · Neurology · doi:10.1212/wnl.0b013e31827b92a3 · Cited by 22 · see citing works
Performance on the Adult Rheumatology In‐Training Examination and Relationship to Outcomes on the Rheumatology Certification Examination
Lohr, K. M., Clauser, A. L., Hess, B., Gelber, A. C., Valeriano‐Marcet, J., Lipner, R. S., Haist, S. A., Hawley, J. L., Zirkle, S., & Bolster, M. B. (2015) · Arthritis & Rheumatology · doi:10.1002/art.39281 · Cited by 22 · see citing works
Association between Orthopaedic In-Training Examination Subsection Scores and ABOS Part I Examination Performance
Ponce, B. A., Savage, J. A., Momaya, A. M., Seales, J., Oliver, J. R., McGwin, G., & Theiss, S. M. (2014) · Southern Medical Journal · doi:10.14423/smj.0000000000000208 · Cited by 18 · see citing works
Performance on the Nephrology In-Training Examination and ABIM Nephrology Certification Examination Outcomes
Jurich, D. P., Duhigg, L. M., Plumb, T. J., Haist, S. A., Hawley, J. L., Lipner, R. S., Smith, L., & Norby, S. M. (2018) · Clinical Journal of the American Society of Nephrology · doi:10.2215/cjn.05580517 · Cited by 18 · see citing works
Predictors of success on the written anesthesiology board certification examination
Kim, P. Y., Wallace, D., Allbritton, D. W., & Altose, M. D. (2012) · International Journal of Medical Education · doi:10.5116/ijme.5096.5d66 · Cited by 17 · see citing works
CREOG In-Training Examination Results: Contemporary Use to Predict ABOG Written Examination Outcomes
Lingenfelter, B. M., Jiang, X., Schnatz, P. F., O’Sullivan, D. M., Minassian, S. S., & Forstein, D. A. (2016) · Journal of Graduate Medical Education · doi:10.4300/jgme-d-15-00408.1 · Cited by 17 · see citing works
Pediatric Board Review Course for Residents “at Risk”
Aeder, L., Fogel, J., & Schaeffer, H. A. (2010) · Clinical Pediatrics · doi:10.1177/0009922809352679 · Cited by 16 · see citing works
Pulmonary and Critical Care In-Service Training Examination Score as a Predictor of Board Certification Examination Performance
Kempainen, R. R., Hess, B., Addrizzo-Harris, D. J., Schaad, D. C., Scott, C. S., Carlin, B. W., Shaw, R. C., Duhigg, L. M., & Lipner, R. S. (2016) · Annals of the American Thoracic Society · doi:10.1513/annalsats.201601-015oc · Cited by 15 · see citing works
The In-Service Examination Score As a Predictor of Success on the American Board of Preventive Medicine Certification Examination
Bedno, S. A., Soltis, M. A., Mancuso, J. D., Burnett, D. G., & Mallon, T. M. (2011) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2011.08.008 · Cited by 14 · see citing works
Medical Knowledge Assessment by Hematology and Medical Oncology In-Training Examinations Are Better Than Program Director Assessments at Predicting Subspecialty Certification Examination Performance
Collichio, F. A., Hess, B., Muchmore, E. A., Duhigg, L. M., Lipner, R. S., Haist, S. A., Hawley, J. L., Morrison, C. A., Clayton, C., Raymond, M. J., et al. (2016) · Journal of Cancer Education · doi:10.1007/s13187-016-0993-6 · Cited by 14 · see citing works
Do Chief Resident Scores on the In-Service Examination Predict Their Performance on the American Board of Urology Qualifying Examination?
Kerfoot, B. P., Baker, H. E., Connelly, D., Joseph, D. B., Matson, S., & Ritchey, M. L. (2011) · The Journal of Urology · doi:10.1016/j.juro.2011.03.126 · Cited by 13 · see citing works
A comparison of performance on the OMSITE and ABOMS written qualifying examination
Ellis, E., & Haug, R. H. (2000) · Journal of Oral and Maxillofacial Surgery · doi:10.1053/joms.2000.18275 · Cited by 12 · see citing works
Junior otolaryngology resident in‐service exams predict written board exam passage
Puscas, L. (2018) · The Laryngoscope · doi:10.1002/lary.27515 · Cited by 8 · see citing works
Relationship Between Performance on Child and Adolescent Psychiatry In-Training and Certification Examinations
Juul, D., Sexson, S. B., Brooks, B. A., Beresin, E. V., Bechtold, D. W., Lang, J. A., Faulkner, L. R., Tanguay, P. E., & Dingle, A. D. (2013) · Journal of Graduate Medical Education · doi:10.4300/jgme-d-12-00088.1 · Cited by 6 · see citing works
Does Taking the Fellowship In-Service Hematopathology Examination and Performance Relate to Success on the American Board of Pathology Hematology Examination?
Monaghan, S. A., Felgar, R. E., Kelly, M., Ali, A., Anastasi, J., Bellara, A. P., Rinder, H. M., Sargent, R. L., Wagner, J., Swerdlow, S. H., et al. (2016) · American Journal of Clinical Pathology · doi:10.1093/ajcp/aqw085 · Cited by 6 · see citing works
American Board of Dermatology Certification Examination: Preparation and Perceptions
Hayag, M. V., Berman, B., Weinstein, A., & Frankel, S. (2002) · Archives of Dermatology · doi:10.1001/archderm.138.4.544 · Cited by 2 · see citing works
Standardized Training Examinations among Ophthalmology Residents and the American Board of Ophthalmology Written Qualifying Examination First Attempt: The Morsani College of Medicine Experience
Drucker, M. D., & Carey, A. R. (2014) · Journal of Academic Ophthalmology · doi:10.1055/s-0034-1396085 · Cited by 2 · see citing works

Flagged first, then by citations

1 of the 39 references couldn’t be matched to a known paper.

Sorted by citations

Limitations of large language models in clinical problem-solving arising from inflexible reasoning
Kim, J., Podlasek, A., Shidara, K., Liu, F., Alaa, A., & Bernardo, D. (2025) · Scientific Reports · doi:10.1038/s41598-025-22940-0 · Cited by 78 · see citing works
Medical resident’s pursuing specialty and differences in clinical proficiency among medical residents in Japan: a nationwide cross-sectional study
Watari, T., Nishizaki, Y., Houchens, N., Kataoka, K., Sakaguchi, K., Shiraishi, Y., Shimizu, T., Yamamoto, Y., & Tokuda, Y. (2023) · BMC Medical Education · doi:10.1186/s12909-023-04429-4 · Cited by 23 · see citing works
Impact of an Online Question Bank on Resident In-Training Exam Performance
Scott, N. P., Martin, T. W., Schmidt, A., & Shanks, A. L. (2023) · Journal of Medical Education and Curricular Development · doi:10.1177/23821205231206221 · Cited by 11 · see citing works
The Association of ACGME Milestones With Performance on American Board of Surgery Assessments
Weaver, M. L., Carter, T., Yamazaki, K., Hamstra, S. J., Holmboe, E. S., Chaer, R. A., Park, Y. S., & Smith, B. K. (2023) · Annals of Surgery · doi:10.1097/sla.0000000000005998 · Cited by 10 · see citing works
Self-citation
Re-examining single-moment-in-time high-stakes examinations in specialist training: A critical narrative review
Sidhu, N. S., & Fleming, S. S. (2023) · Medical Teacher · doi:10.1080/0142159x.2023.2260081 · Cited by 10 · see citing works
Resident factors associated with American board of internal medicine certification exam failure
Seaberg, P. H., Kling, J. M., Klanderman, M. C., Mead‐Harvey, C., Williams, K. E., Labonte, H. R., Jain, A., Taylor, G., & Blair, J. E. (2022) · Medical Education Online · doi:10.1080/10872981.2022.2152162 · Cited by 9 · see citing works
Fellowship Board Pass Rates Rising: Analysis of Pathology Subspecialty Board Examination Performance
Jacobs, J. W., Booth, G. S., Usmani, A., Burner, J. D., & Adkins, B. D. (2022) · Archives of Pathology & Laboratory Medicine · doi:10.5858/arpa.2022-0129-oa · Cited by 7 · see citing works
Impact of Question Bank Use for In-Training Examination Preparation by OBGYN Residents – A Multicenter Study
Green, I. C., Weaver, A., Kircher, S., Levy, G., Brady, R. M., Flicker, A. B., Gala, R. B., Peterson, J., Decesare, J. Z., & Breitkopf, D. M. (2022) · Journal of surgical education · doi:10.1016/j.jsurg.2021.12.014 · Cited by 7 · see citing works
Burnout is not associated with trainee performance on the Vascular Surgery In-Training Exam
Cui, C. L., Reilly, M. A., Pillado, E. B., Li, R. D., Eng, J. S., Grafmuller, L. E., DiLosa, K. L., Conway, A. M., Escobar, G. A., Shaw, P. M., et al. (2024) · Journal of Vascular Surgery · doi:10.1016/j.jvs.2024.08.057 · Cited by 6 · see citing works
Are Age, Sex, or Race and Ethnicity Associated With Performance on the OBGYN NBME Exam and Clerkship?
Shvartsman, K., Abuhamdeh, S., Dong, T., Lutgendorf, M. A., Peterson, L., Ulfers, A., & Durning, S. J. (2025) · Journal of surgical education · doi:10.1016/j.jsurg.2025.103440 · Cited by 4 · see citing works
Assessing the impact of adopting a competency-based medical education framework and ACGME-I accreditation on educational outcomes in a family medicine residency program in Abu Dhabi Emirate, United Arab Emirates
AlKetbi, L. B., Nagelkerke, N. J. D., AlZarouni, A. A., AlKuwaiti, M., Aldhaheri, R., AlNeyadi, A. M., AlAlawi, S., & AlKuwaiti, M. (2024) · Frontiers in Medicine · doi:10.3389/fmed.2023.1257213 · Cited by 4 · see citing works
Taking Leave During Residency
McDeavitt, J. T., Appelbaum, N., Raddatz, M. M., Driscoll, S. W., & Kinney, C. L. (2022) · American Journal of Physical Medicine & Rehabilitation · doi:10.1097/phm.0000000000002004 · Cited by 4 · see citing works
USMLE Performance, Subsequent Standardized Testing, and ABIM Certification Exam Preparation for Internal Medicine Residency Programs: A Narrative Review
Smith, D. T., Matelski, A., Hall, M. A. K., Phadke, V. K., Vettese, T. E., Law, K. L., & Hemrajani, R. H. (2024) · Journal of General Internal Medicine · doi:10.1007/s11606-024-09229-0 · Cited by 3 · see citing works
Number of Patient Encounters in Emergency Medicine Residency Does Not Correlate with In-Training Exam Domain Scores
Kern, M. B., Jewell, C., Hekman, D., & Schnapp, B. H. (2022) · Western Journal of Emergency Medicine · doi:10.5811/westjem.2022.11.57997 · Cited by 3 · see citing works
Critical care medicine training in the age of COVID-19
Mickey, W. (2023) · Journal of Osteopathic Medicine · doi:10.1515/jom-2022-0244 · Cited by 2 · see citing works
Impact of COVID-19 on Pulmonary Critical Care Fellows In-Training Exam Performance
Alwakeel, M. I., Patrawalla, P. S., Carlin, B. W., Addrizzo-Harris, D., Mastronarde, J. G., Reitzner, J. B., Perez, H. L. R., Wang, X., & Chaisson, N. F. (2025) · CHEST Journal · doi:10.1016/j.chest.2025.04.015 · Cited by 2 · see citing works
The Use of COMLEX-USA Performance When Considering DO Applicants to General Surgery Residency Programs
Sandella, J. M., Young, G., Boulet, J. R., & Conrad-Schnetz, K. J. (2025) · Journal of surgical education · doi:10.1016/j.jsurg.2025.103642 · Cited by 2 · see citing works
Passing the Caduceus: A Medical Student Perspective on USMLE Step 1 Reform
Long, T. J. (2024) · Medical Science Educator · doi:10.1007/s40670-024-02187-6 · Cited by 1 · see citing works
Use of In-Training Examination Scores as a Fellowship Candidate Evaluation Metric: Time for a Change
Hong, C. X., Russell, C. B., Southworth, E., & Fairchild, P. S. (2024) · Urogynecology · doi:10.1097/spv.0000000000001489 · Cited by 1 · see citing works
Public Board Score Reporting Undermines Holistic Review for Residency Selection
Kelleher, M. E., Schumacher, D. J., Zhou, C. Y., Kwakye, D. B., Santen, S. A., Warm, E. J., & Kinnear, B. (2024) · Journal of General Internal Medicine · doi:10.1007/s11606-024-09133-7 · Cited by 1 · see citing works
Diseño de la línea de producción para la elaboración y envasado de puré de palta en el departamento de Piura
Suárez, A. D., Ruiz, D. D., Guzmán, B. E., Mendoza, G. G., & Bayona, K. L. J. (2014) · Institutional Repository of the University of Piura (University of Piura)
Additional file 2 of Medical resident’s pursuing specialty and differences in clinical proficiency among medical residents in Japan: a nationwide cross-sectional study
Watari, T., Nishizaki, Y., Houchens, N., Kataoka, K., Sakaguchi, K., Shiraishi, Y., Shimizu, T., Yamamoto, Y., & Tokuda, Y. (2023) · Figshare · doi:10.6084/m9.figshare.23564868
Additional file 3 of Medical resident’s pursuing specialty and differences in clinical proficiency among medical residents in Japan: a nationwide cross-sectional study
Watari, T., Nishizaki, Y., Houchens, N., Kataoka, K., Sakaguchi, K., Shiraishi, Y., Shimizu, T., Yamamoto, Y., & Tokuda, Y. (2023) · Figshare · doi:10.6084/m9.figshare.23564871
Additional file 4 of Medical resident’s pursuing specialty and differences in clinical proficiency among medical residents in Japan: a nationwide cross-sectional study
Watari, T., Nishizaki, Y., Houchens, N., Kataoka, K., Sakaguchi, K., Shiraishi, Y., Shimizu, T., Yamamoto, Y., & Tokuda, Y. (2023) · Figshare · doi:10.6084/m9.figshare.23564874
Additional file 5 of Medical resident’s pursuing specialty and differences in clinical proficiency among medical residents in Japan: a nationwide cross-sectional study
Watari, T., Nishizaki, Y., Houchens, N., Kataoka, K., Sakaguchi, K., Shiraishi, Y., Shimizu, T., Yamamoto, Y., & Tokuda, Y. (2023) · Figshare · doi:10.6084/m9.figshare.23564877
Additional file 1 of Medical resident’s pursuing specialty and differences in clinical proficiency among medical residents in Japan: a nationwide cross-sectional study
Watari, T., Nishizaki, Y., Houchens, N., Kataoka, K., Sakaguchi, K., Shiraishi, Y., Shimizu, T., Yamamoto, Y., & Tokuda, Y. (2023) · Figshare · doi:10.6084/m9.figshare.23564865
Can USMLE and COMLEX-USA Scores Predict At-Risk Emergency Medicine Residents' Performance on In-Training Examinations?
Plewa, M. C., Ledrick, D., Jenkins, K., Orqvist, A., & McCrea, M. (2024) · Cureus · doi:10.7759/cureus.58684
Navigating Discordance: Assessing Varied Applications of the American Academy of Periodontology In-Service Examination in Postdoctoral Programs
Darcey, C., Soh, M. Y., & Meyer, H. S. (2024) · MedEdPublish · doi:10.12688/mep.20169.1
MCQs in-training examination scores of the surgical residency program in Thailand: the relationship between medical school vs public health-based training institutions
Orrapin, S., Homchan, O.-U., Chotirosniramit, N., Jirapongcharoenlap, T., & Ditsatham, C. (2024) · BMC Medical Education · doi:10.1186/s12909-024-06063-0
Analyzing Patterns in Anesthesiology Residents' Exam Performance Using Data Mining Techniques
DACCPM, Karimian, M., Rahmatizadeh, S., Kohzadi, Z., Kohzadi, Z., Madadi, F., & Dabbagh, A. (2024) · Anesthesiology and Pain Medicine · doi:10.5812/aapm-151686
Board scores in the spotlight: Public reporting and the unintended consequences
Kelleher, M. E., Santen, S. A., Draper, C., Jordan, J., Gottlieb, M. A., & Kinnear, B. (2025) · AEM Education and Training · doi:10.1002/aet2.70006
Ophthalmic Knowledge Assessment Program Scores and Written Qualifying Examination Performance
Thanitcul, C., Schnabel, S. D., Bartley, G. B., Lee, A. G., Crump, H., Fajardo, D. E., Peters, K., Wilson, B., & Srikumaran, D. S. (2025) · JAMA Ophthalmology · doi:10.1001/jamaophthalmol.2025.1933
Association between Subspecialty In-Training Examination Scores and First Attempt Results in Neonatal-Perinatal Medicine Boards
Mir, I. N., Chokshi, R. K., Wambach, J., French, H. M., Frost, M. S., & Brion, L. P. (2025) · American Journal of Perinatology · doi:10.1055/a-2697-3426
Resilient Scores, Residual Scars
Heath, J. K., & Clancy, C. B. (2025) · CHEST Journal · doi:10.1016/j.chest.2025.05.016
Resident Coaching: A Path to Board Readiness
Geringer, J. L., Hojnoski, C. E., Luckasen, M. C., & Ford, M. B. (2025) · The Clinical Teacher · doi:10.1111/tct.70243
Additional file 6 of Medical resident’s pursuing specialty and differences in clinical proficiency among medical residents in Japan: a nationwide cross-sectional study
Watari, T., Nishizaki, Y., Houchens, N., Kataoka, K., Sakaguchi, K., Shiraishi, Y., Shimizu, T., Yamamoto, Y., & Tokuda, Y. (2023) · Figshare · doi:10.6084/m9.figshare.23564880

for “At-risk ” Residents to Im prove Their Scores on the In-train ing Exam
Shokar, G. S. (2003)
Validity of the in-training examination for predicting american board of internal medicine certifying examination scores
Grossman, R. S., Fincher, R.-M. E., Layne, R. D., Seelig, C. B., Berkowitz, L. R., & Levine, M. A. (1992) · Journal of General Internal Medicine · doi:10.1007/bf02599105 · Cited by 82 · see citing works
Resident Performance on the In-Training and Board Examinations in Obstetrics and Gynecology: Implications for the ACGME Outcome Project
Withiam-Leitch, M. L., & Olawaiye, A. B. (2008) · Teaching and Learning in Medicine · doi:10.1080/10401330801991642 · Cited by 31 · see citing works
Can we predict final outcome of internal medicine residents with in-training evaluation.
Chierakul, N., Pongprasobchai, S., Boonyapisit, K., Chinthammitr, Y., Pithukpakorn, M., Maneesai, A., Sriwijitkamol, A., Koomanachai, P., Koolvisoot, A., Tanwandee, T., et al. (2011) · PubMed · Cited by 1 · see citing works
Evaluation and enhancement of medical knowledge competency by monthly tests: a single institution experience
Assaly, R., Khan, A. R., Siddiqui, N. S., Thotakura, R. V., Hasan, S., Luni, F. K., Sodeman, T. C., Hinch, B. T., Kaw, D., Khuder, S., et al. (2015) · Advances in Medical Education and Practice · doi:10.2147/amep.s89951 · Cited by 4 · see citing works

Authors: McCrary, H. C., Colbert-Getz, J. M., Poss, W. B., & Smith, B. K.
Affiliations: University of Utah (US)
All authors are with the University of Utah School of Medicine.
Corresponding author: McCrary, Hilary C.
ORCID: McCrary, Hilary C.
Smith, Brigitte K.
Venue: Journal of Graduate Medical Education
Publisher: Journal of Graduate Medical Education
Published: 2020-12-29
Volume: 13
Issue: 1
Pages: 43-57
ISSN: 1949-8349 (ISSN-L)
1949-8357
DOI: 10.4300/jgme-d-20-00111.1
PMC: PMC7901636
OpenAlex: W3130133331
Cited by: 31
References: 39
Citations by year: 2025: 9 · 2024: 10 · 2023: 6 · 2022: 5 · 2014: 1
FWCI: 0.67
Type: journal article
Language: en
Open access: green
www.ncbi.nlm.nih.gov
Locations: Journal of Graduate Medical Education (publishedVersion)
PubMed Central (publishedVersion)
PubMed (publishedVersion)
Topics: Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Innovations in Medical Education
Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Medical Education and Admissions
Health Sciences › Medicine › Family Practice › Clinical Reasoning and Diagnostic Skills
Keywords: in-training examination, predictive validity, graduate medical education, high-stakes decision making
MeSH terms: Clinical Competence, Education, Medical, Graduate, Educational Measurement, Humans, Internship and Residency, Specialty Boards

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

A Systematic Review of the Relationship Between In-Training Examination Scores and Specialty Board Examination Scores

Abstract

Integrity signals

Claims the paper makes

Claims other papers make about this one

Self-check

Cites

Cited by

Related

Record

About these checks