Identification of subjects with polycystic ovary syndrome using electronic health records
Open accessAbstract
BACKGROUND: Polycystic ovary syndrome (PCOS) is a heterogeneous disorder because of the variable criteria used for diagnosis. Therefore, International Classification of Diseases 9 (ICD-9) codes may not accurately capture the diagnostic criteria necessary for large scale PCOS identification. We hypothesized that use of electronic medical records text and data would more specifically capture PCOS subjects. METHODS: Subjects with PCOS were identified in the Partners Healthcare Research Patients Data Registry by searching for the term "polycystic ovary syndrome" using natural language processing (n = 24,930). A training subset of 199 identified charts was reviewed and categorized based on likelihood of a true Rotterdam PCOS diagnosis, i.e. two out of three of the following: irregular menstrual cycles, hyperandrogenism and/or polycystic ovary morphology. Data from the history, physical exam, laboratory and radiology results were codified and extracted from notes of definite PCOS subjects. Thirty-two terms were used to build an algorithm for identifying definite PCOS cases and applied to the rest of the dataset. The positive predictive value cutoff was set at 76.8 % to maximize the number of subjects available for study. A true positive predictive value for the algorithm was calculated after review of 100 charts from subjects identified as definite PCOS cases with at least two documented Rotterdam criteria. The positive predictive value was compared to that calculated using 200 charts identified using the ICD-9 code for PCOS (256.4; n = 13,670). In addition, a cohort of previously recruited PCOS subjects was submitted for algorithm validation. RESULTS: Chart review demonstrated that 64 % were confirmed as definitely PCOS using the algorithm, with a 9 % false positive rate. 66 % of subjects identified by ICD-9 code for PCOS could be confirmed as definitely PCOS, with an 8.5 % false positive rate. There was no significant difference in the positive predictive values using the two methods (p = 0.2). However, the number of charts that had insufficient confirmatory data was lower using the algorithm (5 % vs 11 %; p < 0.04). Of 477 subjects with PCOS recruited and examined individually and present in the database as patients, 451 were found within the algorithm dataset. CONCLUSIONS: Extraction of text parameters along with codified data improves the confidence in PCOS patient cohorts identified using the electronic medical record. However, the positive predictive value was not significantly different when using ICD-9 codes or the specific algorithm. Further studies are needed to determine the positive predictive value of the two methods in additional electronic medical record datasets.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The diagnosis of PCOS included ICD-9 and ICD-10 codes, which have been validated and used in previous epidemiologic studies.
“The diagnosis of PCOS included the ICD-9 and ICD-10 codes, and these have been validated and used in previous epidemiologic studies.6,10,17,27,32,55, 56, 57”
From Increased Prevalence of Adverse Health Outcomes Across the Lifespan in Those Affected by Polycystic Ovary Syndrome: A Canadian Population Cohort (2024) · cites this paper as [57]
The Methods (Data source) and Results describe using ICD-9 codes to identify PCOS cases and checking diagnoses against chart findings. The Background discusses earlier studies of ICD-9 identification. The paper does not establish that ICD-10 PCOS codes were used or validated, and its Discussion says ICD-9 codes may be a reasonable proxy if validated in other health-record systems. Page numbers are unavailable in our copy. (LLM, conf=0.88)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Legro, R. S., Kunselman, A. R., Dodson, W. C., & Dunaif, A. (1999) · The Journal of Clinical Endocrinology & Metabolism · doi:10.1210/jcem.84.1.5393 · Cited by 1,537 · see citing works
Prevalence of impaired glucose tolerance and diabetes in women with polycystic ovary syndrome.
Ehrmann, D. A., Barnes, R. B., Rosenfield, R. L., Cavaghan, M. K., & Imperial, J. (1999) · Diabetes Care · doi:10.2337/diacare.22.1.141 · Cited by 1,257 · cited 1× · see citing works
“Additional features are variable, with obesity exacerbating hyperandrogenism and risk for metabolic disorders including impaired glucose tolerance, type 2 diabetes and metabolic syndrome [3–5].”
BackgroundOligoanovulation with Polycystic Ovaries But Not Overt Hyperandrogenism
Dewailly, D., Catteau-Jonard, S., Reyss, A.-C., Leroy, M., & Pigny, P. (2006) · The Journal of Clinical Endocrinology & Metabolism · doi:10.1210/jc.2006-1054 · Cited by 198 · see citing works
Flagged first, then by citations
Research works printed but not linked: 19
- [1]Revised 2003 consensus on diagnostic criteria and long-term health risks related to polycystic ovary syndrome (PCOS). Human Reproduction. 2004;19(1):41–7. 14688154
- [2]Johnson TRB, Kaplan LK, Ouyang P, Rizza RA. Evidence-based methodology workshop on polycystic ovary syndrome. https://prevention.nih.gov/docs/programs/pcos/FinalReport.pdf. Accessed October, 2015.
- [5]Welt CK Gudmundsson JA Arason G Adams J Palsdottir H Gudlaugsdottir G Characterizing discrete subsets of polycystic ovary syndrome as defined by the Rotterdam criteria: the impact of weight on phenotype and metabolic features J Clin Endocrinol Metab 2006 91 12 4842 8 10.1210/jc.2006-1327 17003085 · DOI
- [7]Barber TM Wass JA McCarthy MI Franks S Metabolic characteristics of women with polycystic ovaries and oligo-amenorrhoea but normal androgen levels: implications for the management of polycystic ovary syndrome Clin Endocrinol (Oxf) 2007 66 4 513 7 17371468
- [8]Cussons AJ Stuckey BG Walsh JP Burke V Norman RJ Polycystic ovarian syndrome: marked differences between endocrinologists and gynaecologists in diagnosis and management Clin Endocrinol (Oxf) 2005 62 3 289 95 10.1111/j.1365-2265.2004.02208.x 15730409 · DOI · Paper page
- [9]Auble B Elder D Gross A Hillman JB Differences in the management of adolescents with polycystic ovary syndrome across pediatric specialties J Pediatr Adolesc Gynecol 2013 26 4 234 8 10.1016/j.jpag.2013.03.007 23726135 PMC7024914 · DOI
- [10]Christensen SB Black MH Smith N Martinez MM Jacobsen SJ Porter AH Prevalence of polycystic ovary syndrome in adolescents Fertil Steril 2013 100 2 470 7 10.1016/j.fertnstert.2013.04.001 23756098 PMC3813299 · DOI
- [11]Lo JC Feigenbaum SL Yang J Pressman AR Selby JV Go AS Epidemiology and adverse cardiovascular risk profile of diagnosed polycystic ovary syndrome J Clin Endocrinol Metab 2006 91 4 1357 63 10.1210/jc.2005-2430 16434451 · DOI
- [12]Liddy ED, Turner AM, Bradley J. Modeling interventions to improve access to public health information. AMIA Annual Symposium proceedings / AMIA Symposium AMIA Symposium. 2003:909. PMC1480166 14728415
- [13]Murphy SN, Mendis M, Hackett K, Kuttan R, Pan W, Phillips LC et al. Architecture of the open-source clinical research chart from Informatics for Integrating Biology and the Bedside. AMIA Annual Symposium Proceedings; 2007: American Medical Informatics Association. PMC2655844 18693896
- [14]Savova GK Masanz JJ Ogren PV Zheng J Sohn S Kipper-Schuler KC Mayo clinical Text Analysis and Knowledge Extraction System (cTAKES): architecture, component evaluation and applications JAMIA 2010 17 5 507 13 20819853 PMC2995668
- [15]Shen Y Liao KP Cai T Sparse kernel machine regression for ordinal outcomes Biometrics 2015 71 1 63 70 10.1111/biom.12223 25196727 PMC4609171 · DOI
- [16]Welt CK Arason G Gudmundsson JA Adams J Palsdottir H Gudlaugsdottir G Defining constant versus variable phenotypic features of women with polycystic ovary syndrome using different ethnic groups and populations J Clin Endocrinol Metab 2006 91 11 4361 8 10.1210/jc.2006-1191 16940441 · DOI
- [17]Liao KP Cai T Gainer V Goryachev S Zeng-treitler Q Raychaudhuri S Electronic medical records for discovery research in rheumatoid arthritis Arthritis Care Res 2010 62 8 1120 7 10.1002/acr.20184 PMC3121049 20235204 · DOI
- [18]Perlis RH Iosifescu DV Castro VM Murphy SN Gainer VS Minnier J Using electronic medical records to enable large-scale studies in psychiatry: treatment resistant depression as a model Psychol Med 2012 42 1 41 50 10.1017/S0033291711000997 21682950 PMC3837420 · DOI · Paper page
- [19]Ananthakrishnan AN Cai T Savova G Cheng SC Chen P Perez RG Improving case definition of Crohn's disease and ulcerative colitis in electronic medical records using natural language processing: a novel informatics approach Inflamm Bowel Dis 2013 19 7 1411 20 10.1097/MIB.0b013e31828133fd 23567779 PMC3665760 · DOI
- [20]Broder-Fingert S Shah B Kessler M Pawelczak M David R Evaluation of adolescents for polycystic ovary syndrome in an urban population J Clin Res Pediatr Endocrinol 2009 1 4 188 93 21274294 PMC3005652
- [21]Legro RS Arslanian SA Ehrmann DA Hoeger KM Murad MH Pasquali R Diagnosis and treatment of polycystic ovary syndrome: an Endocrine Society clinical practice guideline J Clin Endocrinol Metab 2013 98 12 4565 92 10.1210/jc.2013-2350 24151290 PMC5399492 · DOI · Paper page
- [22]Popat VB Prodanov T Calis KA Nelson LM The menstrual cycle: a biological marker of general health in adolescents Ann N Y Acad Sci 2008 1135 43 51 10.1196/annals.1429.040 18574207 PMC2755071 · DOI
Cited by
Sorted by citations
Deconstructing a Syndrome: Genomic Insights Into PCOS Causal Mechanisms and Classification
Dapas, M., & Dunaif, A. (2022) · Endocrine Reviews · doi:10.1210/endrev/bnac001 · Cited by 300 · see citing works
Large-scale identification of patients with cerebral aneurysms using natural language processing
Castro, V. M., Dligach, D., Finan, S., Yu, S., Can, A., Abd-El-Barr, M., Gainer, V., Shadick, N. A., Murphy, S., Cai, T., et al. (2017) · Neurology · doi:10.1212/wnl.0000000000003490 · Cited by 98 · see citing works
Self-citationBarrera, F. J., Brown, E. D. L., Rojo, A., Obeso, J., Plata, H., Lincango, E. P., Terry, N., Rodríguez-Gutiérrez, R., Hall, J. E., & Shekhar, S. (2023) · Frontiers in Endocrinology · doi:10.3389/fendo.2023.1106625 · Cited by 91 · see citing works
Predicting polycystic ovary syndrome with machine learning algorithms from electronic health records
Zad, Z., Jiang, V. S., Wolf, A. T., Wang, T., Cheng, J. J., Paschalidis, I. C., & Mahalingaiah, S. (2024) · Frontiers in Endocrinology · doi:10.3389/fendo.2024.1298628 · Cited by 97 · see citing works
Vine, D., Ghosh, M., Wang, T., & Bakal, J. (2024) · CJC Open · doi:10.1016/j.cjco.2023.12.010 · Cited by 53 · see citing works
Genetics of Polycystic Ovary Syndrome
Welt, C. K. (2021) · Endocrinology and Metabolism Clinics of North America · doi:10.1016/j.ecl.2020.10.006 · Cited by 43 · see citing works
Self-citationActkins, K. V., Singh, K., Hucks, D., Velez Edwards, D. R., Aldrich, M., Cha, J., Wellons, M., & Davis, L. K. (2021) · The Journal of Clinical Endocrinology & Metabolism · doi:10.1210/clinem/dgaa675 · Cited by 27 · see citing works
- Authors
- Castro, V., Shen, Y., Yu, S., Finan, S., Pau, C. T., Gainer, V., Keefe, C. C., Savova, G., Murphy, S. N., Cai, T., & Welt, C. K.
- Affiliations
- Massachusetts General Hospital (US)
- Harvard University (US)
- Brigham and Women's Hospital (US)
- Boston Children's Hospital (US)
- University of Utah (US)
- Corresponding author
- Welt, Corrine K.
- ORCID
- Venue
- Reproductive Biology and Endocrinology
- Published
- 2015-10-29
- Volume
- 13
- Issue
- 1
- Pages
- 116-116
- ISSN
- 1477-7827 (ISSN-L)
- PMC
- PMC4625743
- OpenAlex
- W2104919320
- Cited by
- 51
- References
- 3
- Citations by year
- 2026: 2 · 2025: 6 · 2024: 7 · 2023: 7 · 2022: 2 · 2021: 3 · 2020: 4 · 2019: 6 · 2018: 6 · 2017: 2 · 2016: 5
- FWCI
- 4.94
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- Reproductive Biology and Endocrinology (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Digital Access to Scholarship at Harvard (DASH) (Harvard University) (submittedVersion, cc-by)
- Funders
- National Institute of Child Health and Human Development (R01HD065029)
- National Center for Research Resources (1 UL1 RR025758)
- American Diabetes Association (1-10-CT-57)
- Partners Center for Personalized Genetic Medicine
- Topics
- Health Sciences › Medicine › Reproductive Medicine › Ovarian function and disorders
- Health Sciences › Health Professions › Health Information Management › Medical Coding and Health Information
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Keywords
- polycystic ovary syndrome, electronic health records, natural language processing, ICD-9 codes, Rotterdam criteria, cohort identification, positive predictive value, clinical text mining, hyperandrogenism, menstrual irregularity, polycystic ovary morphology, false positive rate, algorithm validation, clinical information extraction
- MeSH terms
- Adult, Algorithms, Female, Humans, Middle Aged, Polycystic Ovary Syndrome, Databases, Factual, Electronic Health Records
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review