Electronic Health Record Based Algorithm to Identify Patients with Autism Spectrum Disorder
Open accessThe DOI matches its Crossref record; 9 of the 9 references it cites were found in a registry or an index.
Abstract
OBJECTIVE: Cohort selection is challenging for large-scale electronic health record (EHR) analyses, as International Classification of Diseases 9th edition (ICD-9) diagnostic codes are notoriously unreliable disease predictors. Our objective was to develop, evaluate, and validate an automated algorithm for determining an Autism Spectrum Disorder (ASD) patient cohort from EHR. We demonstrate its utility via the largest investigation to date of the co-occurrence patterns of medical comorbidities in ASD. METHODS: We extracted ICD-9 codes and concepts derived from the clinical notes. A gold standard patient set was labeled by clinicians at Boston Children's Hospital (BCH) (N = 150) and Cincinnati Children's Hospital and Medical Center (CCHMC) (N = 152). Two algorithms were created: (1) rule-based implementing the ASD criteria from Diagnostic and Statistical Manual of Mental Diseases 4th edition, (2) predictive classifier. The positive predictive values (PPV) achieved by these algorithms were compared to an ICD-9 code baseline. We clustered the patients based on grouped ICD-9 codes and evaluated subgroups. RESULTS: The rule-based algorithm produced the best PPV: (a) BCH: 0.885 vs. 0.273 (baseline); (b) CCHMC: 0.840 vs. 0.645 (baseline); (c) combined: 0.864 vs. 0.460 (baseline). A validation at Children's Hospital of Philadelphia yielded 0.848 (PPV). Clustering analyses of comorbidities on the three-site large cohort (N = 20,658 ASD patients) identified psychiatric, developmental, and seizure disorder clusters. CONCLUSIONS: In a large cross-institutional cohort, co-occurrence patterns of comorbidities in ASDs provide further hypothetical evidence for distinct courses in ASD. The proposed automated algorithms for cohort selection open avenues for other large-scale EHR studies and individualized treatment of ASD.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Introduction to Information Retrieval
Manning, C. D., Raghavan, P., & Schütze, H. (2008) · doi:10.1017/cbo9780511809071 · Cited by 11,288
Baio, J., Wiggins, L., Christensen, D. L., Maenner, M. J., Daniels, J., Warren, Z., Kurzius-Spencer, M., Zahorodny, W., Robinson, C., Rosenberg, et al. (2018) · MMWR. Surveillance Summaries · doi:10.15585/mmwr.ss6706a1 · Cited by 6,078
(2013) · The Lancet · doi:10.1016/s0140-6736(12)62129-1 · Cited by 3,010
Wing, L., & Gould, J. (1979) · Journal of Autism and Developmental Disorders · doi:10.1007/bf01531288 · Cited by 2,587
Savova, G. K., Masanz, J. J., Ogren, P. V., Zheng, J., Sohn, S., Kipper-Schuler, K. C., & Chute, C. G. (2010) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.2009.001560 · Cited by 1,967
Self-citationThe Co-Morbidity Burden of Children and Young Adults with Autism Spectrum Disorders
Kohane, I. S., McMurry, A., Weber, G., MacFadden, D., Rappaport, L., Kunkel, L., Bickel, J., Wattanasin, N., Spence, S., Murphy, S., et al. (2012) · PLoS ONE · doi:10.1371/journal.pone.0033224 · Cited by 562
Self-citationComorbidity clusters in autism spectrum disorders: an electronic health record time-series analysis
Doshi-Velez, F., Ge, Y., & Kohane, I. (2014) · Pediatrics · doi:10.1542/peds.2013-0819 · Cited by 412 · cited 6×
“We perform a study similar to the one performed in [7] but with several improvements: 1) Patients are selected using the algorithm presented in this paper, providing a cleaner sample of ASD patients 2) we extend the analysis to multiple institutions, thus using a significantly larger dataset, 3) We compare the results…”
BackgroundMethodResultInfluentialSelf-citationElectronic Medical Records for Genetic Research: Results of the eMERGE Consortium
Kho, A. N., Pacheco, J. A., Peissig, P. L., Rasmussen, L., Newton, K. M., Weston, N., Crane, P. K., Pathak, J., Chute, C. G., Bielinski, S. J., et al. (2011) · Science Translational Medicine · doi:10.1126/scitranslmed.3001807 · Cited by 336
Self-citationElectronic health records-driven phenotyping: challenges, recent advances, and perspectives
Pathak, J., Kho, A. N., & Denny, J. C. (2013) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2013-002428 · Cited by 274
Self-citation
Cited by
Myers, S. M., Voigt, R. G., Colligan, R. C., Weaver, A. L., Storlie, C. B., Stoeckel, R. E., Port, J. D., & Katusic, S. K. (2019) · Journal of Autism and Developmental Disorders · doi:10.1007/s10803-018-3834-0 · Cited by 87
Vargason, T., Frye, R. E., McGuinness, D. L., & Hahn, J. (2019) · Autism Research · doi:10.1002/aur.2128 · Cited by 59
“The seven COC categories were defined based on ICD9-CM codes used in previous ASD clustering studies [Doshi-Velez et al., 2014; Lingren et al., 2016] and in other studies of COC prevalence [Meltzer, Johnson, Crosette, Ramos, & Mindell, 2010; Zerbo et al., 2015].”
BackgroundMethodResultWalters, C. E., Nitin, R., Margulis, K., Boorom, O., Gustavson, D. E., Bush, C. T., Davis, L. K., Below, J. E., Cox, N. J., Camarata, S. M., et al. (2020) · Journal of Speech, Language, and Hearing Research · doi:10.1044/2020_jslhr-19-00397 · Cited by 18
“However, recent works on autism (Bush et al., 2017; Leroy et al., 2018; Lingren et al., 2016), mental illness (Lyalina et al., 2013; McCoy et al., 2018), and even infectious diseases (Chiu & Hripcsak, 2017) demonstrate the feasibility of applying automated algorithmic procedures to EHRs for…”
Influential
Metadata
- Authors
- Lingren, T., Chen, P., Bochenek, J., Doshi-Velez, F., Manning-Courtney, P., Bickel, J., Wildenger Welchons, L., Reinhold, J., Bing, N., Ni, Y., Barbaresi, W., Mentch, F., Basford, M., Denny, J., Vazquez, L., Perry, C., Namjou, B., Qiu, H., Connolly, J., Abrams, D., Holm, I. A., Cobb, B. A., Lingren, N., Solti, I., Hakonarson, H., Kohane, I. S., Harley, J., & Savova, G.
- Venue
- PLOS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2016-07-29
- Volume
- 11
- Issue
- 7
- Pages
- e0159621
- ISSN
- 1932-6203 (ISSN-L)
- OpenAlex
- W2488962145
- Cited by
- 79
- References
- 9
- FWCI
- 6.92
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Electronic Health Record Based Algorithm to Identify Patients with Autism Spectrum Disorder. https://app.cochat.ai/papers/10.1371/journal.pone.0159621
- The paper
- Lingren, T., Chen, P., Bochenek, J., Doshi-Velez, F., Manning-Courtney, P., Bickel, J., Wildenger Welchons, L., Reinhold, J., Bing, N., Ni, Y., Barbaresi, W., Mentch, F., Basford, M., Denny, J., Vazquez, L., Perry, C., Namjou, B., Qiu, H., Connolly, J., . . . Savova, G. (2016). Electronic Health Record Based Algorithm to Identify Patients with Autism Spectrum Disorder. PLOS ONE. https://doi.org/10.1371/journal.pone.0159621
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.