CoChat
Sign Up
Cited by 21Innovations in Medical EducationMedical Education and AdmissionsTeacher Education and Assessments

The DOI matches its Crossref record; 1 of the 1 references it cites were found in a registry or an index.

The Predictive Validity of the National Board of Osteopathic Medical Examiners’ COMLEX-USA Examinations With Regard to Outcomes on American Board of Family Medicine Examinations

O’Neill, Thomas R., Peabody, Michael R., Song, Hao·2016·Academic Medicine
DOI 10.1097/acm.0000000000001254
Open this paper in CoChat — free account

Abstract

Purpose: To examine the predictive validity of the National Board of Osteopathic Medical Examiners’ Comprehensive Osteopathic Medical Licensing Examination of the United States of America (COMLEX-USA) series with regard to the American Board of Family Medicine’s (ABFM’s) In-Training Examination (ITE) and Maintenance of Certification for Family Physicians (MC-FP) Examination. Method: A repeated-measures design was employed, using test scores across seven levels of training for 1,023 DOs who took the MC-FP for the first time between April 2012 and November 2014 and for whom the ABFM had ITE scores for each of their residency years. Pearson and disattenuated correlations were calculated; Fisher r to z transformation was performed; and sensitivity, specificity, and positive and negative predictive values for the COMLEX-USA Level 2–Cognitive Evaluation (CE) with regard to the MC-FP were computed. Results: The Pearson and disattenuated correlations ranged from 0.55 to 0.69 and from 0.61 to 0.80, respectively. For MC-FP scores, only the correlation increase from the COMLEX-USA Level 2-CE to Level 3 was statistically significant (for Pearson correlations: z = 2.41, P = .008; for disattenuated correlations: z = 3.16, P < .001). The sensitivity, specificity, and positive and negative predictive values of the COMLEX-USA Level 2-CE with the MC-FP were 0.90, 0.39, 0.96, and 0.19, respectively. Conclusions: Evidence was found that the COMLEX-USA can assist family medicine residency program directors in predicting later resident performance on the ABFM’s ITE and MC-FP, which is becoming increasingly important as graduate medical education accreditation moves toward a single aligned model.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1097/acm.0000000000001254

Cites

1 of 1 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Research works printed but not linked: 52

  1. Callahan, 2010, The predictive validity of three versions of the MCAT in relation to performance in medical school, residency, and licensing examinations: A longitudinal study of 36 classes of Jefferson Medical College., Acad Med, 85, 980, 10.1097/ACM.0b013e3181cece3d · DOI
  2. Donnon, 2007, The predictive validity of the MCAT for medical school performance and medical board licensing examinations: A meta-analysis of the published research., Acad Med, 82, 100, 10.1097/01.ACM.0000249878.25186.b7 · DOI
  3. Violato, 2005, Does the medical college admission test predict clinical reasoning skills? A longitudinal study employing the Medical Council of Canada clinical reasoning examination., Acad Med, 80, S14, 10.1097/00001888-200510001-00007 · DOI
  4. Kleshinski, 2009, Impact of preadmission variables on USMLE step 1 and step 2 performance., Adv Health Sci Educ Theory Pract, 14, 69, 10.1007/s10459-007-9087-x · DOI
  5. Dunleavy, 2013, The predictive validity of the MCAT exam in relation to academic performance through medical school: A national cohort study of 2001–2004 matriculants., Acad Med, 88, 666, 10.1097/ACM.0b013e3182864299 · DOI
  6. de Oliveira, 2012, Factors affecting admission to anesthesiology residency in the United States: Choosing the future of our specialty., Anesthesiology, 117, 243, 10.1097/ALN.0b013e31825fb04b · DOI
  7. McDonald, 2008, Associations between United States Medical Licensing Examination (USMLE) and Internal Medicine In-Training Examination (IM-ITE) scores., J Gen Intern Med, 23, 1016, 10.1007/s11606-008-0641-x · DOI
  8. Fening, 2011, Correlation of USMLE Step 1 scores with performance on dermatology in-training examinations., J Am Acad Dermatol, 64, 102, 10.1016/j.jaad.2009.12.051 · DOI
  9. Thundiyil, 2010, Do United States Medical Licensing Examination (USMLE) scores predict in-training test performance for emergency medicine residents?, J Emerg Med, 38, 65, 10.1016/j.jemermed.2008.04.010 · DOI
  10. Alterman, 2011, The predictive value of general surgery application data for future resident performance., J Surg Educ, 68, 513, 10.1016/j.jsurg.2011.07.007 · DOI
  11. Jeffe, 2011, Factors associated with American Board of Medical Specialties member board certification among US medical school graduates., JAMA, 306, 961, 10.1001/jama.2011.1099 · DOI
  12. Sutton, 2014, Is USMLE Step 1 score a valid predictor of success in surgical residency?, Am J Surg, 208, 1029, 10.1016/j.amjsurg.2014.06.032 · DOI
  13. Swanson, 2009, Relationship between performance on part I of the American Board of Orthopaedic Surgery certifying examination and scores on USMLE Steps 1 and 2., Acad Med, 84, S21, 10.1097/ACM.0b013e3181b37fd2 · DOI
  14. McCaskill, 2007, USMLE step 1 scores as a significant predictor of future board passage in pediatrics., Ambul Pediatr, 7, 192, 10.1016/j.ambp.2007.01.002 · DOI
  15. Sevensma, 2008, COMLEX-USA and in-service examination scores: Tools for evaluating medical knowledge among residents., J Am Osteopath Assoc, 108, 713
  16. Cavalieri, 2003, Predictive validity of osteopathic medical licensing examinations for osteopathic medical knowledge measured by graduate written examinations., J Am Osteopath Assoc, 103, 337
  17. Pierce, 2013, Performance on COMLEX-USA exams predicts performance on EM residency in-training exams., Acad Emerg Med, 20, S219
  18. Sarko, 2010, COMLEX-1 and USMLE-1 are not interchangeable examinations., Acad Emerg Med, 17, 218, 10.1111/j.1553-2712.2009.00632.x · DOI
  19. Chick, 2010, Relationship between COMLEX and USMLE scores among osteopathic medical students who take both examinations., Teach Learn Med, 22, 3, 10.1080/10401330903445422 · DOI
  20. Li, 2014, Relationship between COMLEX-USA scores and performance on the American Osteopathic Board of Emergency Medicine part I certifying examination., J Am Osteopath Assoc, 114, 260
  21. Hubbard, 1967, An in-training examination for residents as a guide to learning., N Engl J Med, 276, 448, 10.1056/NEJM196702232760806 · DOI
  22. Leigh, 1990, Predictive validity of the American Board of Family Practice in-training examination., Acad Med, 65, 454, 10.1097/00001888-199007000-00009 · DOI
  23. Replogle, 2004, Assessing the predictive value of the American Board of Family Practice in-training examination., Fam Med, 36, 185
  24. O’Neill, 2015, The predictive validity of the ABFM’s in-training examination., Fam Med, 47, 349
  25. Garvin, 1984, Significance of the in-training examination in a surgical residency program., Surgery, 96, 109
  26. Biester, 1987, The American Board of Surgery in-training examination as a predictor of success on the qualifying examination., Curr Surg, 44, 194
  27. Jones, 2014, Using the American Board of Surgery in-training examination to predict board certification: A cautionary study., J Surg Educ, 71, 144, 10.1016/j.jsurg.2014.04.004 · DOI
  28. Grossman, 1992, Validity of the in-training examination for predicting American Board of Internal Medicine certifying examination scores., J Gen Intern Med, 7, 63, 10.1007/BF02599105 · DOI
  29. Waxman, 1994, Performance on the internal medicine second-year residency in-training examination predicts the outcome of the ABIM certifying examination., J Gen Intern Med, 9, 692, 10.1007/BF02599012 · DOI
  30. Rollins, 1998, Predicting pass rates on the American Board of Internal Medicine certifying examination., J Gen Intern Med, 13, 414, 10.1046/j.1525-1497.1998.00122.x · DOI
  31. Babbott, 2007, The predictive validity of the internal medicine in-training examination., Am J Med, 120, 735, 10.1016/j.amjmed.2007.05.003 · DOI
  32. Webb, 1996, How well does the psychiatry residency in-training examination predict performance on the American Board of Psychiatry and Neurology. Part I. Examination?, Am J Psychiatry, 153, 831, 10.1176/ajp.153.6.831 · DOI
  33. Goodman, 2002, RITE performance predicts outcome on the ABPN part I examination., Neurology, 58, 1144, 10.1212/WNL.58.8.1144 · DOI
  34. Juul, 2009, Relationship between resident-in-training examination in psychiatry and subsequent certification examination performances., Acad Psychiatry, 33, 404, 10.1176/appi.ap.33.5.404 · DOI
  35. Baumgartner, 1998, 1998 Joseph E. Whitley, MD, Award. Relationship between American College of Radiology in-training examination scores and American Board of Radiology written examination scores. Part 2. Multi-institutional study., Acad Radiol, 5, 374, 10.1016/S1076-6332(98)80156-6 · DOI
  36. Baumgartner, 1996, Relationship between American College of Radiology in-training examination scores and American Board of Radiology written examination scores., Acad Radiol, 3, 873, 10.1016/S1076-6332(96)80281-9 · DOI
  37. Althouse, 2008, The in-training examination: An analysis of its predictive value on performance on the general pediatrics certification examination., J Pediatr, 153, 425, 10.1016/j.jpeds.2008.03.012 · DOI
  38. Spellacy, 2006, Prediction of ABOG written examination performance from the third-year CREOG in-training examination results., J Reprod Med, 51, 621
  39. Withiam-Leitch, 2008, Resident performance on the in-training and board examinations in obstetrics and gynecology: Implications for the ACGME Outcome Project., Teach Learn Med, 20, 136, 10.1080/10401330801991642 · DOI
  40. Kearney, 2000, Performance on ABA-ASA in-training examination predicts success for RCPSC certification. American Board of Anesthesiology–American Society of Anesthesiologists. Royal College of Physicians and Surgeons of Canada., Can J Anaesth, 47, 914, 10.1007/BF03019676 · DOI
  41. Klein, 2004, Passing the boards: Can USMLE and orthopaedic in-training examination scores predict passage of the ABOS part-I examination?, J Bone Joint Surg Am, 86-A, 1092, 10.2106/00004623-200405000-00032 · DOI
  42. Rinder, 2011, Senior pathology resident in-service examination scores correlate with outcomes of the American Board of Pathology certifying examinations., Am J Clin Pathol, 136, 499, 10.1309/AJCPA7O4BBUGLSWW · DOI
  43. Cho, 2011, Predictive value of the Korean Academy of Family Medicine in-training examination for certifying examination., Korean J Fam Med, 32, 352, 10.4082/kjfm.2011.32.6.352 · DOI
  44. Norris, 2005, From specialty-based to practice-based: A new blueprint for the American Board of Family Medicine cognitive examination., J Am Board Fam Pract, 18, 546, 10.3122/jabfm.18.6.546 · DOI
  45. Spearman, 1904, The proof and measurement of association between two things., Am J Psychol, 15, 72, 10.2307/1412159 · DOI · Paper page
  46. Zimmerman, 1997, Properties of the Spearman correction for attenuation for normal and realistic non-normal distributions., Appl Psychol Meas, 21, 253, 10.1177/01466216970213005 · DOI
  47. Schumaker, 1996, Disattenuating correlation coefficients., Rasch Meas Trans, 10, 479
  48. Fisher, 1915, Frequency distribution of the values of the correlation coefficient in samples from an indefinitely large population., Biometrika, 10, 507
  49. Linacre, 1989, The “length” of a logit., Rasch Meas Trans, 3, 54
  50. Hawkins, 1999, The in-training examination in internal medicine: Resident perceptions and lack of correlation between resident scores and faculty predictions of resident performance., Am J Med, 106, 206, 10.1016/S0002-9343(98)00392-1 · DOI
  51. Taylor, 1990, A study of the ability of physician faculty members to predict resident performance., Fam Med, 22, 296
  52. Parker, 2004, Can family medicine residents predict their performance on the in-training examination?, Fam Med, 36, 705

Metadata

Authors
O’Neill, T. R., Peabody, M. R., & Song, H.
Affiliations
American Academy of Family Physicians (US)
Corresponding author
O’Neill, Thomas R.
ORCID
Peabody, Michael R.
Venue
Academic Medicine
Publisher
Oxford University Press (OUP)
Published
2016-11
Volume
91
Issue
11
Pages
1568-1575
ISSN
1040-2446 (ISSN-L)
DOI
10.1097/acm.0000000000001254
OpenAlex
W2417449430
Cited by
21
References
1
Citations by year
2025: 3 · 2024: 2 · 2023: 1 · 2022: 3 · 2021: 2 · 2020: 3 · 2019: 1 · 2018: 3 · 2016: 2
FWCI
1.99
Type
journal article
Language
en
Open access
closed
Locations
  • Academic Medicine (publishedVersion)
  • PubMed (publishedVersion)
Topics
  • Social Sciences › Education › Innovations in Medical Education
  • Social Sciences › Education › Medical Education and Admissions
  • Social Sciences › Education › Teacher Education and Assessments
Keywords
predictive validity, family medicine residency, in-training examination, resident performance prediction, osteopathic physicians, graduate medical education accreditation, cognitive assessment
MeSH terms
Certification, Clinical Competence, Family Practice, Humans, Internship and Residency, Licensure, Medical, Osteopathic Medicine, Specialty Boards, United States

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review