Cited by 51Ovarian function and disordersMedical Coding and Health InformationArtificial Intelligence in Healthcare

Identification of subjects with polycystic ovary syndrome using electronic health records

Open access
Castro, Victor, Shen, Yuanyuan, Yu, Sheng, Finan, Sean, Pau, Cindy Ta, Gainer, Vivian, Keefe, Candace C., Savova, Guergana +3 more·2015·Reproductive Biology and Endocrinology

Abstract

BACKGROUND: Polycystic ovary syndrome (PCOS) is a heterogeneous disorder because of the variable criteria used for diagnosis. Therefore, International Classification of Diseases 9 (ICD-9) codes may not accurately capture the diagnostic criteria necessary for large scale PCOS identification. We hypothesized that use of electronic medical records text and data would more specifically capture PCOS subjects. METHODS: Subjects with PCOS were identified in the Partners Healthcare Research Patients Data Registry by searching for the term "polycystic ovary syndrome" using natural language processing (n = 24,930). A training subset of 199 identified charts was reviewed and categorized based on likelihood of a true Rotterdam PCOS diagnosis, i.e. two out of three of the following: irregular menstrual cycles, hyperandrogenism and/or polycystic ovary morphology. Data from the history, physical exam, laboratory and radiology results were codified and extracted from notes of definite PCOS subjects. Thirty-two terms were used to build an algorithm for identifying definite PCOS cases and applied to the rest of the dataset. The positive predictive value cutoff was set at 76.8 % to maximize the number of subjects available for study. A true positive predictive value for the algorithm was calculated after review of 100 charts from subjects identified as definite PCOS cases with at least two documented Rotterdam criteria. The positive predictive value was compared to that calculated using 200 charts identified using the ICD-9 code for PCOS (256.4; n = 13,670). In addition, a cohort of previously recruited PCOS subjects was submitted for algorithm validation. RESULTS: Chart review demonstrated that 64 % were confirmed as definitely PCOS using the algorithm, with a 9 % false positive rate. 66 % of subjects identified by ICD-9 code for PCOS could be confirmed as definitely PCOS, with an 8.5 % false positive rate. There was no significant difference in the positive predictive values using the two methods (p = 0.2). However, the number of charts that had insufficient confirmatory data was lower using the algorithm (5 % vs 11 %; p < 0.04). Of 477 subjects with PCOS recruited and examined individually and present in the database as patients, 451 were found within the algorithm dataset. CONCLUSIONS: Extraction of text parameters along with codified data improves the confidence in PCOS patient cohorts identified using the electronic medical record. However, the positive predictive value was not significantly different when using ICD-9 codes or the specific algorithm. Further studies are needed to determine the positive predictive value of the two methods in additional electronic medical record datasets.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1186/s12958-015-0115-z

Cites

3 of 3 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The diagnosis of PCOS included ICD-9 and ICD-10 codes, which have been validated and used in previous epidemiologic studies.

    “The diagnosis of PCOS included the ICD-9 and ICD-10 codes, and these have been validated and used in previous epidemiologic studies.6,10,17,27,32,55, 56, 57”

    From Increased Prevalence of Adverse Health Outcomes Across the Lifespan in Those Affected by Polycystic Ovary Syndrome: A Canadian Population Cohort (2024) · cites this paper as [57]

    The Methods (Data source) and Results describe using ICD-9 codes to identify PCOS cases and checking diagnoses against chart findings. The Background discusses earlier studies of ICD-9 identification. The paper does not establish that ICD-10 PCOS codes were used or validated, and its Discussion says ICD-9 codes may be a reasonable proxy if validated in other health-record systems. Page numbers are unavailable in our copy. (LLM, conf=0.88)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 19

  1. [1]Revised 2003 consensus on diagnostic criteria and long-term health risks related to polycystic ovary syndrome (PCOS). Human Reproduction. 2004;19(1):41–7. 14688154
  2. [2]Johnson TRB, Kaplan LK, Ouyang P, Rizza RA. Evidence-based methodology workshop on polycystic ovary syndrome. https://prevention.nih.gov/docs/programs/pcos/FinalReport.pdf. Accessed October, 2015.
  3. [5]Welt CK Gudmundsson JA Arason G Adams J Palsdottir H Gudlaugsdottir G Characterizing discrete subsets of polycystic ovary syndrome as defined by the Rotterdam criteria: the impact of weight on phenotype and metabolic features J Clin Endocrinol Metab 2006 91 12 4842 8 10.1210/jc.2006-1327 17003085 · DOI
  4. [7]Barber TM Wass JA McCarthy MI Franks S Metabolic characteristics of women with polycystic ovaries and oligo-amenorrhoea but normal androgen levels: implications for the management of polycystic ovary syndrome Clin Endocrinol (Oxf) 2007 66 4 513 7 17371468
  5. [8]Cussons AJ Stuckey BG Walsh JP Burke V Norman RJ Polycystic ovarian syndrome: marked differences between endocrinologists and gynaecologists in diagnosis and management Clin Endocrinol (Oxf) 2005 62 3 289 95 10.1111/j.1365-2265.2004.02208.x 15730409 · DOI · Paper page
  6. [9]Auble B Elder D Gross A Hillman JB Differences in the management of adolescents with polycystic ovary syndrome across pediatric specialties J Pediatr Adolesc Gynecol 2013 26 4 234 8 10.1016/j.jpag.2013.03.007 23726135 PMC7024914 · DOI
  7. [10]Christensen SB Black MH Smith N Martinez MM Jacobsen SJ Porter AH Prevalence of polycystic ovary syndrome in adolescents Fertil Steril 2013 100 2 470 7 10.1016/j.fertnstert.2013.04.001 23756098 PMC3813299 · DOI
  8. [11]Lo JC Feigenbaum SL Yang J Pressman AR Selby JV Go AS Epidemiology and adverse cardiovascular risk profile of diagnosed polycystic ovary syndrome J Clin Endocrinol Metab 2006 91 4 1357 63 10.1210/jc.2005-2430 16434451 · DOI
  9. [12]Liddy ED, Turner AM, Bradley J. Modeling interventions to improve access to public health information. AMIA Annual Symposium proceedings / AMIA Symposium AMIA Symposium. 2003:909. PMC1480166 14728415
  10. [13]Murphy SN, Mendis M, Hackett K, Kuttan R, Pan W, Phillips LC et al. Architecture of the open-source clinical research chart from Informatics for Integrating Biology and the Bedside. AMIA Annual Symposium Proceedings; 2007: American Medical Informatics Association. PMC2655844 18693896
  11. [14]Savova GK Masanz JJ Ogren PV Zheng J Sohn S Kipper-Schuler KC Mayo clinical Text Analysis and Knowledge Extraction System (cTAKES): architecture, component evaluation and applications JAMIA 2010 17 5 507 13 20819853 PMC2995668
  12. [15]Shen Y Liao KP Cai T Sparse kernel machine regression for ordinal outcomes Biometrics 2015 71 1 63 70 10.1111/biom.12223 25196727 PMC4609171 · DOI
  13. [16]Welt CK Arason G Gudmundsson JA Adams J Palsdottir H Gudlaugsdottir G Defining constant versus variable phenotypic features of women with polycystic ovary syndrome using different ethnic groups and populations J Clin Endocrinol Metab 2006 91 11 4361 8 10.1210/jc.2006-1191 16940441 · DOI
  14. [17]Liao KP Cai T Gainer V Goryachev S Zeng-treitler Q Raychaudhuri S Electronic medical records for discovery research in rheumatoid arthritis Arthritis Care Res 2010 62 8 1120 7 10.1002/acr.20184 PMC3121049 20235204 · DOI
  15. [18]Perlis RH Iosifescu DV Castro VM Murphy SN Gainer VS Minnier J Using electronic medical records to enable large-scale studies in psychiatry: treatment resistant depression as a model Psychol Med 2012 42 1 41 50 10.1017/S0033291711000997 21682950 PMC3837420 · DOI · Paper page
  16. [19]Ananthakrishnan AN Cai T Savova G Cheng SC Chen P Perez RG Improving case definition of Crohn's disease and ulcerative colitis in electronic medical records using natural language processing: a novel informatics approach Inflamm Bowel Dis 2013 19 7 1411 20 10.1097/MIB.0b013e31828133fd 23567779 PMC3665760 · DOI
  17. [20]Broder-Fingert S Shah B Kessler M Pawelczak M David R Evaluation of adolescents for polycystic ovary syndrome in an urban population J Clin Res Pediatr Endocrinol 2009 1 4 188 93 21274294 PMC3005652
  18. [21]Legro RS Arslanian SA Ehrmann DA Hoeger KM Murad MH Pasquali R Diagnosis and treatment of polycystic ovary syndrome: an Endocrine Society clinical practice guideline J Clin Endocrinol Metab 2013 98 12 4565 92 10.1210/jc.2013-2350 24151290 PMC5399492 · DOI · Paper page
  19. [22]Popat VB Prodanov T Calis KA Nelson LM The menstrual cycle: a biological marker of general health in adolescents Ann N Y Acad Sci 2008 1135 43 51 10.1196/annals.1429.040 18574207 PMC2755071 · DOI

Cited by

Sorted by citations

Authors
Castro, V., Shen, Y., Yu, S., Finan, S., Pau, C. T., Gainer, V., Keefe, C. C., Savova, G., Murphy, S. N., Cai, T., & Welt, C. K.
Affiliations
  • Massachusetts General Hospital (US)
  • Harvard University (US)
  • Brigham and Women's Hospital (US)
  • Boston Children's Hospital (US)
  • University of Utah (US)
Corresponding author
Welt, Corrine K.
Venue
Reproductive Biology and Endocrinology
Published
2015-10-29
Volume
13
Issue
1
Pages
116-116
ISSN
1477-7827 (ISSN-L)
OpenAlex
W2104919320
Cited by
51
References
3
Citations by year
2026: 2 · 2025: 6 · 2024: 7 · 2023: 7 · 2022: 2 · 2021: 3 · 2020: 4 · 2019: 6 · 2018: 6 · 2017: 2 · 2016: 5
FWCI
4.94
Type
journal article
Language
en
Open access
Locations
  • Reproductive Biology and Endocrinology (publishedVersion, cc-by)
  • Europe PMC (PubMed Central) (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • Digital Access to Scholarship at Harvard (DASH) (Harvard University) (submittedVersion, cc-by)
Funders
  • National Institute of Child Health and Human Development (R01HD065029)
  • National Center for Research Resources (1 UL1 RR025758)
  • American Diabetes Association (1-10-CT-57)
  • Partners Center for Personalized Genetic Medicine
Topics
  • Health Sciences › Medicine › Reproductive Medicine › Ovarian function and disorders
  • Health Sciences › Health Professions › Health Information Management › Medical Coding and Health Information
  • Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
Keywords
polycystic ovary syndrome, electronic health records, natural language processing, ICD-9 codes, Rotterdam criteria, cohort identification, positive predictive value, clinical text mining, hyperandrogenism, menstrual irregularity, polycystic ovary morphology, false positive rate, algorithm validation, clinical information extraction
MeSH terms
Adult, Algorithms, Female, Humans, Middle Aged, Polycystic Ovary Syndrome, Databases, Factual, Electronic Health Records

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review