The DOI matches its Crossref record; 108 of the 108 references it cites were found in a registry or an index.
Electronic health records-driven phenotyping: challenges, recent advances, and perspectives
Open accessAbstract
With the completion of the Human Genome Project1 as well as recent advances in genomic science and comparative biological studies, a new era of individualized medicine is evolving where novel biomedical discoveries are leading to more effective prevention, treatment, and diagnosis of disease. Although altered phenotypes are one of the most reliable manifestations of altered gene functions, research in extracting, representing, and analyzing phenotype–genotype relationships is still evolving. This has led to the emergence of a trans-discipline field, called ‘Phenomics,’2 that aims to capitalize on high-throughput computation and informatics technologies for the systematic study of phenotypes and how they might influence personal genomics.3 Many comparative phenomics studies in the recent past4,5 have demonstrated the power of positively correlating phenotypes with several measures of gene functions. However, despite the advances, research in phenomics is presented with various challenges, including (i) developing approaches for high-throughput extraction and representation of phenotypes, (ii) building techniques for storing, integrating, and querying phenotype data, and (iii) advancing phenotypic-driven analysis to derive phenotype–genotype associations. A significant barrier in the discovery of new genetic variants is the requirement to obtain the large sample sizes needed for an effective study (since variants may be rare within a population) leading to time-consuming and onerous sample collection efforts. Electronic health records (EHRs) can accelerate clinical research and genomic medicine, but are hindered by the limited number of validated processes and tools to enable accurate and rapid phenotype extraction.6 EHRs are increasing in ubiquity, functionality, and comprehensiveness across the USA, in part due to Meaningful Use standards7 implemented as part of the Health Information Technology for Economic and Clinical Health (HITECH) Act. One recent advance has been the coupling of DNA biorepositories to EHR data,8–13 combined with advances in informatics techniques, such as natural language processing (NLP), to enable genomic discoveries. The Electronic Medical Records and Genomics (eMERGE14,15) consortium—a network of nine academic medical centers—has demonstrated the effectiveness of EHR-derived phenotyping algorithms for cohort identification to conduct genome- and phenome-wide association studies.16–20 Algorithms to define phenotypes in eMERGE have typically followed an iterative path, and highlight the importance of intermittent chart review to validate phenotype accuracy, typically at more than one site (when a multisite implementation is planned16). Once finalized, eMERGE phenotype algorithms (as well as non-eMERGE algorithms) can be viewed at http://phekb.org. While eMERGE presents an exciting and encouraging demonstration of secondary use of EHR data, evaluating the strengths and limitations for EHRs has important implications for clinical and translational research, including clinical trials, observational cohorts, outcomes research, and comparative effectiveness research. These issues are further amplified within the realm of a learning healthcare system21 that emphasizes the ability to have real-time access to knowledge, digital capture of care experience, engaged and empowered patients, alignment of incentives to value, and a leadership-instilled culture of learning (figure 1). Hripcsak and Albers6 highlight several such challenges in leveraging EHRs for research, including data that is often incomplete, inaccurate, highly complex, and biased, and propose a combination of top-down knowledge engineering and bottom-up learning and analysis to address these issues (figure 2). Specifically, understanding the inherent complexity involved in a healthcare process model is critically important towards achieving a scalable and high-throughput phenotyping process. Different phases in a learning healthcare system (figure adapted with permission from Greene et al21). Electronic health record (EHR)-driven phenotyping and knowledge discovery (figure adapted with permission from Hripcsak and Albers6). Several have acknowledged that there is a recognized tension between facilitating data entry by allowing narrative text in EHRs, and utilizing these data for research.22 A key component for identifying patient cohorts in the EHR is to define inclusion and exclusion criteria that algorithmically select sets of patients based on stored clinical data. This process allows the definition of phenotypes over structured data (eg, demographics, diagnoses, medications, and laboratory measurements)23 as well as unstructured clinical text (eg, radiology reports,22,24 encounter notes, and discharge summaries25–30). In general, this process can be quite complex, involving heuristics encoded as rules or machine learning algorithms.31,32 Several NLP techniques33 have been developed specifically for clinical text, and includ
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Advancements in automated detection of SI, SB, and SA from clinical notes have shown varied success17,18,19,20,21,22,23,24,25,26.”
From Deep learning for identifying personal and family history of suicidal thoughts and behaviors from EHRs (2024) · cites this paper as [19]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“These approaches, ranging from rule-based algorithms to ML techniques, primarily focus on identifying suicidal tendencies without considering their temporal aspects, often conflating current and historical events.”
From Deep learning for identifying personal and family history of suicidal thoughts and behaviors from EHRs (2024) · cites this paper as [19]
Experimental — accuracy not yet measuredIn the citing paper
“Traditionally, the automated extraction of structured data elements from medical notes has relied on medical natural language processing (NLP) using rule-based or machine-learning approaches or a combination of both9,10.”
From A critical assessment of using ChatGPT for extracting structured data from clinical notes (2024) · cites this paper as [10]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The “Meaningful Use” Regulation for Electronic Health Records
Blumenthal, D., & Tavenner, M. (2010) · New England Journal of Medicine · doi:10.1056/nejmp1006114 · Cited by 2,059
Development of a Large-Scale De-Identified DNA Biobank to Enable Personalized Medicine
Roden, D. M., Pulley, J. M., Basford, M., Bernard, G. R., Clayton, E. W., Balser, J., & Masys, D. R. (2008) · Clinical Pharmacology & Therapeutics · doi:10.1038/clpt.2008.89 · Cited by 966
Next-generation phenotyping of electronic health records
Hripcsak, G., & Albers, D. J. (2013) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2012-001145 · Cited by 795
the eMERGE Team, McCarty, C. A., Chisholm, R. L., Chute, C. G., Kullo, I. J., Jarvik, G. P., Larson, E. B., Li, R., Masys, D. R., Ritchie, M. D., et al. (2011) · BMC Medical Genomics · doi:10.1186/1755-8794-4-13 · Cited by 701
The Electronic Medical Records and Genomics (eMERGE) Network: past, present, and future
Gottesman, O., Kuivaniemi, H., Tromp, G., Faucett, W. A., Li, R., Manolio, T. A., Sanderson, S. C., Kannry, J. L., Zinberg, R. E., Basford, M., et al. (2013) · Genetics in Medicine · doi:10.1038/gim.2013.72 · Cited by 691
Self-citationMiddleton, B., Bloomrosen, M., Dente, M. A., Hashmat, B., Koppel, R., Overhage, J. M., Payne, T. H., Rosenbloom, S. T., Weaver, C., & Zhang, J. (2013) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2012-001458 · Cited by 618
Validation of a common data model for active safety surveillance research
Overhage, J. M., Ryan, P. B., Reich, C. G., Hartzema, A. G., & Stang, P. E. (2012) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2011-000376 · Cited by 548
The Human Genome Project: Past, Present, and Future
Watson, J. D. (1990) · Science · doi:10.1126/science.2181665 · Cited by 537
MedEx: a medication information extraction system for clinical narratives
Xu, H., Stenner, S. P., Doan, S., Johnson, K. B., Waitman, L. R., & Denny, J. C. (2010) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m3378 · Cited by 532
Self-citationEvaluating temporal relations in clinical text: 2012 i2b2 Challenge
Sun, W., Rumshisky, A., & Uzuner, Ö. (2013) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2013-001628 · Cited by 497
Research works printed but not linked: 1
- Wolf W Doyle M Aufox S . DNA banking study in an ethnically diverse urban university hospital. Am J Hum Genet 2003;73:423.
Cited by
Sorted by citations
Human symptoms–disease network
Zhou, X., Menche, J., Barabási, A.-Ĺ., & Sharma, A. (2014) · Nature Communications · doi:10.1038/ncomms5212 · Cited by 659
Mining Electronic Health Records (EHRs): A Survey
Yadav, P., Steinbach, M., Kumar, V., & Simon, G. (2018) · ACM Computing Surveys · doi:10.1145/3127881 · Cited by 299
“In the realm of supervised temporal pattern mining, research has extended the temporal abstraction framework by mining recent temporal patterns for monitoring and event detection problems in patients suffering from T2DM (Batal et al. 2012).”
BackgroundMethodInfluentialA critical assessment of using ChatGPT for extracting structured data from clinical notes
Huang, J., Yang, D. M., Rong, R., Nezafati, K., Treager, C., Chi, Z., Wang, S., Cheng, X., Guo, Y., Klesse, L. J., et al. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01079-8 · Cited by 298
Advances in Electronic Phenotyping: From Rule-Based Definitions to Machine Learning Models
Banda, J. M., Seneviratne, M. G., Hernandez‐Boussard, T., & Shah, N. H. (2018) · Annual Review of Biomedical Data Science · doi:10.1146/annurev-biodatasci-080917-013315 · Cited by 228
“The widespread adoption of electronic health records (EHRs) over the past decade in the United States has generated vast repositories of clinical data (1).”
BackgroundInfluentialAutomated methods for the summarization of electronic health records
Perotte, R., & Elhadad, N. (2015) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocv032 · Cited by 227
“This knowledge representation effort has been successful in helping efforts like phenotyping combine terminological knowledge, expert reasoning, and machine learning to create actionable disease definitions.(102) Similarly in summarization work, it is important to make use of these available clinical knowledge represe…”
MethodBig Data in Medicine Is Driving Big Changes
Verspoor, K. M., & Martin-Sanchez, F. J. (2014) · Yearbook of Medical Informatics · doi:10.15265/iy-2014-0020 · Cited by 193
“The editors highlight the recent transition from the use of the EHR for clinical decision support to the use of the EHR directly for research [58], and a number of the articles make use of natural language processing to process unstructured content in the EHR for phenotyping [59-63].”
BackgroundVan Driest, S. L., Wells, Q. S., Stallings, S., Bush, W. S., Gordon, A., Nickerson, D. A., Kim, J. H., Crosslin, D. R., Jarvik, G. P., Carrell, D. S., et al. (2016) · JAMA · doi:10.1001/jama.2015.17701 · Cited by 170
“of ECGs per individual, median (range) 3 (1-14) 3 (1-31) .”
Self-citationA ‘Green Button’ For Using Aggregate Patient Data At The Point Of Care
Longhurst, C., Harrington, R. A., & Shah, N. H. (2014) · Health Affairs · doi:10.1377/hlthaff.2014.0099 · Cited by 159
Wang, Y., Chen, R. F., Ghosh, J., Denny, J. C., Kho, A. N., Chen, Y., Malin, B., & Sun, J. (2015) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/2783258.2783395 · Cited by 159
“Successful pheno-typing efforts on EHR data can enable many important applications, such as clinical predictive modeling [12, 27] and EHR-based genomic association studies [12, 17, 27].”
Self-citationTemporal Phenotyping from Longitudinal Electronic Health Records
Liu, C., Wang, F., Hu, J., & Xiong, H. (2015) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/2783258.2783352 · Cited by 147
“To address these challenges, before going into the stage of detailed applications, we should first do electronic phenotyping , which is basically a feature extraction process transforming the raw EHR data into clinically relevant features [7, 22].”
MethodProblems, challenges and promises: perspectives on precision medicine
Duffy, D. J. (2015) · Briefings in Bioinformatics · doi:10.1093/bib/bbv060 · Cited by 149
Learning statistical models of phenotypes using noisy labeled training data
Agarwal, V., Podchiyska, T., Banda, J. M., Goel, V. V., Leung, T. I., Minty, E. P., Sweeney, T. E., Ross, E., & Shah, N. H. (2016) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocw028 · Cited by 141
Subtyping: What It is and Its Role in Precision Medicine
Saria, S., & Goldenberg, A. (2015) · IEEE Intelligent Systems · doi:10.1109/mis.2015.60 · Cited by 129
Desiderata for computable representations of electronic health records-driven phenotype algorithms
Mo, H., Thompson, W. K., Rasmussen, L. V., Pacheco, J. A., Jiang, G., Kiefer, R. C., Zhu, Q., Xu, J., Montague, E. N. H., Carrell, D., et al. (2015) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocv112 · Cited by 135
“, identifying records for continuous trait analyses or marking records as a case, a control, or neither for given conditions).(8,9) Examples include identifying patients with hypothyroidism matched to hypothyroidismfree controls,(5) evaluating cardiac conduction duration in electrocardiograms of “heart-healthy” indivi…”
BackgroundSelf-citationWhy Is the Electronic Health Record So Challenging for Research and Clinical Care?
Holmes, J., Beinlich, J., Boland, M., Bowles, K., Chen, Y., Cook, T., Demiris, G., Draugelis, M., Fluharty, L., Gabriel, P., et al. (2021) · Methods of Information in Medicine · doi:10.1055/s-0041-1731784 · Cited by 115
“94 Fourth, EHR data typically offer a mixture of variable types (e.g., binary, ordinal, quantitative, and categorical).”
BackgroundPragmatic randomized clinical trials: best practices and statistical guidance
Gamerman, V., Cai, T., & Elsäßer, A. (2018) · Health Services and Outcomes Research Methodology · doi:10.1007/s10742-018-0192-5 · Cited by 117
Enabling phenotypic big data with PheNorm
Yu, S., Ma, Y., Gronsbell, J. L., Cai, T. A., Ananthakrishnan, A. N., Gainer, V. S., Churchill, S. E., Szolovits, P., Murphy, S. N., Kohane, I. S., et al. (2017) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocx111 · Cited by 108
“To overcome the scarcity of phenotypic data, genomic and other medical studies have begun to extract phenotypic information from electronic health records (EHRs) to augment existing biorepositories or quickly create new ones.(4,5) Notable efforts include the i2b2 effort led by Harvard University and Partners HealthCar…”
BackgroundRichesson, R., Sun, J., Pathak, J., Kho, A., & Denny, J. C. (2016) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2016.05.005 · Cited by 110
“The process of deriving research-grade phenotypes from clinical data using computer-executable algorithms is called computational phenotyping (phenotyping for short) [2].”
MethodSelf-citationComparison of Approaches for Heart Failure Case Identification From Electronic Health Record Data
Blecker, S., Katz, S. D., Horwitz, L. I., Kuperman, G. J., Park, H., Gold, A., & Sontag, D. A. (2016) · JAMA Cardiology · doi:10.1001/jamacardio.2016.3236 · Cited by 97
The Revival of the Notes Field: Leveraging the Unstructured Content in Electronic Health Records
Assale, M., Dui, L. G., Cina, A., Seveso, A., & Cabitza, F. (2019) · Frontiers in Medicine · doi:10.3389/fmed.2019.00066 · Cited by 100
“There are many works that face this topic trying to extract information from EHR to select a specific patients cohort automatically (4, 92–96).”
BackgroundBuilding the graph of medicine from millions of clinical narratives
Finlayson, S. G., LePendu, P., & Shah, N. H. (2014) · Scientific Data · doi:10.1038/sdata.2014.32 · Cited by 94
“Knowledge of such inter-relatedness can be readily leveraged for improving fundamental methods underlying a wide range of use cases, such as patient clustering 2 and outcome prediction 3 , analysing comorbidity patterns 4,5 , and cohort querying for clinical trial design 6 .”
MethodFederated Learning in Health care Using Structured Medical Data
Oh, W., & Nadkarni, G. N. (2023) · Advances in Kidney Disease and Health · doi:10.1053/j.akdh.2022.11.007 · Cited by 88
Barnado, A., Casey, C. G., Carroll, R. J., Wheless, L., Denny, J. C., & Crofford, L. J. (2016) · Arthritis Care & Research · doi:10.1002/acr.22989 · Cited by 80
“EHRs provide longitudinal information on a patient’s disease course that can be linked to genetic data for discovery research (2).”
BackgroundSelf-citationElectronic Health Record Based Algorithm to Identify Patients with Autism Spectrum Disorder
Lingren, T., Chen, P., Bochenek, J., Doshi‐Velez, F., Manning-Courtney, P., Bickel, J., Wildenger, L. K., Reinhold, J., Bing, N. M., Ni, Y., et al. (2016) · PLoS ONE · doi:10.1371/journal.pone.0159621 · Cited by 83
Self-citationRubbo, B., Fitzpatrick, N. K., Denaxas, S., Daskalopoulou, M., Yu, N., Patel, R. S., & Hemingway, H. (2015) · International Journal of Cardiology · doi:10.1016/j.ijcard.2015.03.075 · Cited by 79
Parameterizing time in electronic health record studies
Hripcsak, G. M., Albers, D. J., & Perotte, A. (2015) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocu051 · Cited by 75
High-fidelity phenotyping: richness and freedom from bias
Hripcsak, G. M., & Albers, D. J. (2017) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocx110 · Cited by 73
“1,2 Researchers have been carrying out EHR phenotyping since the beginning of informatics, from both structured data 3 and narrative data.”
BackgroundUnderstanding the care.data conundrum: New information flows for economic growth
Vezyridis, P., & Timmons, S. (2017) · Big Data & Society · doi:10.1177/2053951716688490 · Cited by 78
“…is also evident in calls to develop policy frameworks and information technology (IT) infrastructures that facilitate closer collaboration between the NHS and commercial bioscience firms (BIGT, 2003; HITF, 2004) in order to accelerate biomedical research and reduce costs (Pathak et al., 2013).”
BackgroundWeng, C., Shah, N. H., & Hripcsak, G. M. (2020) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2020.103433 · Cited by 72
Automated disease cohort selection using word embeddings from Electronic Health Records
Glicksberg, B. S., Miotto, R., Johnson, K. W., Shameer, K., Li, L., Chen, R., & Dudley, J. T. (2017) · Pacific Symposium on Biocomputing (PSB) · doi:10.1142/9789813235533_0014 · Cited by 67
Metzger, M.-H., Tvardik, N., Gicquel, Q., Bouvry, C., Poulet, E., & Potinet, V. (2016) · International Journal of Methods in Psychiatric Research · doi:10.1002/mpr.1522 · Cited by 64
“The recent development of semantic mining tools could be a more efficient and informative alternative to manual data collection (Hripcsak & Albers, 2013; Pathak, Kho, & Denny, 2013; Patrick, Nguyen, Wang, & Li, 2011; Proux et al., 2009).”
MethodBig biomedical data and cardiovascular disease research: opportunities and challenges
Denaxas, S., & Morley, K. I. (2015) · European Heart Journal - Quality of Care and Clinical Outcomes · doi:10.1093/ehjqcco/qcv005 · Cited by 66
“clinical text, electrocardiographic (ECG) monitoring data) to further enhance and fine-tune the accuracy of algorithms utilizing coded data.(4,51,52) For example, Mohebbi and Ghassemian(53) created an algorithm that consists of a linear discriminant analysis based feature reduction scheme and a SVM-based classifier an…”
MethodEdgcomb, J. B., & Zima, B. T. (2019) · Psychiatric Services · doi:10.1176/appi.ps.201800401 · Cited by 60
Khoury, P., Srinivasan, R., Kakumanu, S., Ochoa, S., Keswani, A., Sparks, R., & Rider, N. L. (2022) · The Journal of Allergy and Clinical Immunology In Practice · doi:10.1016/j.jaip.2022.01.047 · Cited by 67
“56 Applications such as clinical research cohort selection 57 and clinical phenotyping 58 became early and visible applications of AI.”
BackgroundDeveloping an Algorithm to Detect Early Childhood Obesity in Two Tertiary Pediatric Medical Centers
Brady, C. C., Namjou, B., Kennebeck, S., Bickel, J. P., Patibandla, N., Ni, Y., Driest, S. L. V., Chen, L., Roach, A., Cobb, B. A., et al. (2016) · Applied Clinical Informatics · doi:10.4338/aci-2016-01-ra-0015 · Cited by 56
Self-citationTemporal electronic phenotyping by mining careflows of breast cancer patients
Dagliati, A., Sacchi, L., Zambelli, A., Tibollo, V., Pavesi, L., Holmes, J. H., & Bellazzi, R. (2017) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2016.12.012 · Cited by 60
“In [4], [29] , authors discuss the importance of phenotyping through data mining methods and highlight the importance of taking into account the temporal dimension through proper temporal modeling and temporal abstraction to identify specific sub-cohorts of patients.”
BackgroundMachine Learning–Based Deep Phenotyping of Atopic Dermatitis
Maintz, L., Welchowski, T., Herrmann, N., Brauer, J., Kläschen, A. S., Fimmers, R., Schmid, M. C., Bieber, T. R. M., CK-CARE Study Group, Schmid‐Grendelmeier, P., et al. (2021) · JAMA Dermatology · doi:10.1001/jamadermatol.2021.3668 · Cited by 58
Clinical Research Informatics and Electronic Health Record Data
Horvath, M. M., Rusincovitch, S. A., & Richesson, R. (2014) · Yearbook of Medical Informatics · doi:10.15265/iy-2014-0009 · Cited by 59
“…of EHR data creates challenges for identifying the presence of any specific clinical condition – such as diabetes, heart disease, or obesity – in patient populations, and greatly limits opportunities to observe or learn from regional variation of disease patterns or trends over time [60].”
BackgroundMethodBuilding bridges across electronic health record systems through inferred phenotypic topics
Chen, Y., Ghosh, J., Bejan, C. A., Gunter, C. A., Gupta, S., Kho, A., Liebovitz, D., Sun, J., Denny, J. C., & Malin, B. (2015) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2015.03.011 · Cited by 57
“In an attempt to address this challenge, it was indicated that standardized information modeling and meaningful use standards could be leveraged for the presentation of a phenotyping algorithm across institutions [29] [30].”
Self-citationZhang, X., Yates, A., Vasilevsky, N., Gourdine, J.-P. F., Callahan, T. J., Carmody, L., Daniš, D., Joachimiak, M. P., Ravanmehr, V., Pfaff, E., et al. (2019) · npj Digital Medicine · doi:10.1038/s41746-019-0110-4 · Cited by 59
A Review of Automatic Phenotyping Approaches using Electronic Health Records
Alzoubi, H., Alzubi, R., Ramzan, N., West, D., Al-Hadhrami, T., & Alazab, M. (2019) · Electronics · doi:10.3390/electronics8111235 · Cited by 56
“Building an EHR-based phenotyping algorithm requires participation from clinical experts to review subsets within a patient’s record to label it as a case (healthy) or a control (affected) and to validate the algorithm output; these efforts take a considerable amount of time [21].”
BackgroundMethods for correcting inference based on outcomes predicted by machine learning
Wang, S., McCormick, T. H., & Leek, J. T. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2001238117 · Cited by 72
“Despite these potential biases, this approach to direct use of predicted outcomes in inferential models is popular in genomics (18), genetic (15), public health (10), and electronic health record phenotyping (6) among other applications.”
BackgroundMethodInfluentialOleynik, M., Kugic, A., Kasáč, Z., & Kreuzthaler, M. (2019) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocz149 · Cited by 56
Zheng, N. S., Feng, Q., Kerchberger, V. E., Zhao, J., Edwards, T. L., Cox, N. J., Stein, C. M., Roden, D. M., Denny, J. C., & Wei, W. (2020) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaa104 · Cited by 51
“Therefore, phenotyping with PheMap may generate more consistent and comparable phenotypes across institutes, as compared to high-throughput phenotyping approaches developed at a single institution, facilitating collaboration and cross validation without comprising patient privacy.(14,16) Our comparison of PheMap and X…”
BackgroundResultSelf-citationMonteith, S., Glenn, T., Geddes, J. R., Whybrow, P. C., & Bauer, M. (2016) · International Journal of Bipolar Disorders · doi:10.1186/s40345-016-0051-7 · Cited by 51
“There is considerable interest in using EMR to automate the process of defining phenotypic cohorts for genetic studies of bipolar disorder, since sample sizes of tens of thousands are needed (Pathak et al. 2013; Potash 2015).”
MethodInformatics and machine learning to define the phenotype
Basile, A. O., & Ritchie, M. D. (2018) · Expert Review of Molecular Diagnostics · doi:10.1080/14737159.2018.1439380 · Cited by 51
“However, given their incomplete, inaccurate, highly complex, dimensional and biased nature, EHR data present analytical difficulties [56].”
BackgroundHorowitz, C. R., Sabin, T., Ramos, M., Richardson, L. D., Hauser, D., Robinson, M., & Fei, K. (2019) · Genetics in Medicine · doi:10.1038/s41436-019-0498-x · Cited by 50
“Future studies may improve the sensitivity of EHR identification of study participants through improved algorithms and natural language processing.(37) It was difficult to use EHRs to identify AA patients, as race–ethnicity fields are commonly missing or inaccurate, even in settings that care mainly for minority patie…”
BackgroundGallego, B., Walter, S. R., Day, R. O., Dunn, A. G., Sivaraman, V., Shah, N. H., Longhurst, C., & Coiera, E. W. (2015) · Journal of Comparative Effectiveness Research · doi:10.2217/cer.15.12 · Cited by 49
Identifying Cases of Type 2 Diabetes in Heterogeneous Data Sources: Strategy from the EMIF Project
Roberto, G., Leal, I., Sattar, N. A., Loomis, A. K., Avillach, P., Egger, P., van Wijngaarden, R. P. T., Ansell, D., Reisberg, S., Tammesoo, M., et al. (2016) · PLoS ONE · doi:10.1371/journal.pone.0160648 · Cited by 48
“As demonstrated by the results presented here, in the context of a multi-national, multi-data source study, the “component algorithm strategy” represents an extremely flexible approach for generating EHR-driven[6] case-finding algorithms in a standardized fashion: on the one hand, it allows the local experts’ knowledg…”
BackgroundInfluentialRobinson, J. R., Denny, J. C., Roden, D. M., & Driest, S. L. V. (2017) · Clinical and Translational Science · doi:10.1111/cts.12522 · Cited by 51
“88,89 Curated phenotypes have been developed for individual diseases, but there is currently no high-throughput mechanism to produce cases and controls from thousands of detailed phenotype algorithms.”
BackgroundSelf-citationWenderfer, S. E., Chang, J. C., Davies, A. G., Luna, I. Y., Scobell, R., Sears, C., Magella, B., Mitsnefes, M. M., Stotter, B. R., Dharnidharka, V. R., et al. (2021) · Clinical Journal of the American Society of Nephrology · doi:10.2215/cjn.07810621 · Cited by 46
“(EHRs) are an important tool in clinical research and can efficiently identify and study large numbers of patients with uncommon diseases (12).”
BackgroundFreitas, J. K. D., Johnson, K. W., Golden, E., Nadkarni, G. N., Dudley, J. T., Böttinger, E. P., Glicksberg, B. S., & Miotto, R. (2021) · Patterns · doi:10.1016/j.patter.2021.100337 · Cited by 47
Pak, T. R., & Kasarskis, A. (2015) · Clinical Infectious Diseases · doi:10.1093/cid/civ670 · Cited by 43
“cal Records and Genomics) are standardizing comparison, validation, and deposition of these algorithms into a central repository [10].”
BackgroundA flexible symbolic regression method for constructing interpretable clinical prediction models
Cava, W. L., Lee, P. C., Ajmal, I., Ding, X., Solanki, P., Cohen, J. B., Moore, J. H., & Herman, D. S. (2023) · npj Digital Medicine · doi:10.1038/s41746-023-00833-8 · Cited by 50
“Most CDS is currently triggered based on manually curated rules or heuristics that identify patient cohorts with certain characteristics of interest 1 – 3 .”
InfluentialEVALUATING THE EFFECTIVENESS OF IT PROJECT MANAGEMENT IN HEALTHCARE DIGITALIZATION: A REVIEW
Akindote, O. J., Adegbite, A. O., Omotosho, A., Anyanwu, A., & Maduka, C. P. (2024) · International Medical Science Research Journal · doi:10.51594/imsrj.v4i1.698 · Cited by 42
Lhatoo, S. D., Bernasconi, N., Blümcke, I., Braun, K. P. J., Buchhalter, J., Denaxas, S., Galanopoulou, A. S., Josephson, C. B., Kobow, K., Lowenstein, D. H., et al. (2020) · Epilepsia · doi:10.1111/epi.16633 · Cited by 43
“Raw EHR data, however, suffer from multiple challenges 79 and require substantial preprocessing before they can be research-ready for statistical analysis, a process known as phenotyping .”
The path from big data to precision medicine
Huang, B. E., Mulyasasmita, W., & Rajagopal, G. (2016) · Expert Review of Precision Medicine and Drug Development · doi:10.1080/23808993.2016.1157686 · Cited by 44
“One focus area has been the development of robust and validated phenotyping algorithms [59,62 – 64], many of which are publically available at [65].”
BackgroundUnlocking the power of machine learning in big data: a scoping survey
Surur, F. M., Mamo, A. A., Gebresilassie, B. G., Mekonen, K. A., Golda, A., Behera, R. K., & Kumar, K. S. K. (2025) · Data Science and Management · doi:10.1016/j.dsm.2025.02.004 · Cited by 47
Functional evaluation of out-of-the-box text-mining tools for data-mining tasks
Jung, K., LePendu, P., Iyer, S. V., Bauer‐Mehren, A., Percha, B., & Shah, N. H. (2014) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2014-002902 · Cited by 39
“21,22 These may be positive mentions, indicating that the patient has the disease or is taking the drug, or negated mentions, such as when a condition is ruled out.”
BackgroundDanish clinical quality databases – an important and untapped resource for clinical research
Sørensen, H. T., Pedersen, L. H., Jorgensen, J., & Ehrenstein, V. (2016) · Clinical Epidemiology · doi:10.2147/clep.s113265 · Cited by 34
“A crucial element for personalized medicine in the future is comprehensive description of patients’ phenotypes (observed physical/ biological characteristics), since changes in phenotype are one of the most valid manifestations of altered gene function.(17) In combination with clinical databases and registries, the bi…”
Development of an automated phenotyping algorithm for hepatorenal syndrome
Koola, J., Davis, S. E., Nimri, O. A., Parr, S. K., Fabbri, D., Malin, B., Ho, S. B., & Matheny, M. E. (2018) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2018.03.001 · Cited by 34
“From this data, we evaluated nine different dimensionality reduction techniques: (1) using the full set of CUIs; (2) CUIs limited by semantic type; (3) CUIs aggregated by semantic similarity; (4) document embedding using the raw text; (5) document embedding using CUIs; (6) an a priori selection of CUIs based on domain…”
MethodAhmad, F., Ricket, I., Hammill, B., Eskenazi, L., Robertson, H. R., Curtis, L., Dobi, C. D., Girotra, S., Haynes, K., Kizer, J., et al. (2020) · Circulation. Cardiovascular Quality and Outcomes · doi:10.1161/circoutcomes.119.006292 · Cited by 30
“12,13,27,28 The ADAPTABLE computable phenotype successfully identified over 650 000 potentially eligible participants for the trial and helped the trial achieve its recruitment goal of 15 000 patients.”
BackgroundMaintz, L., Welchowski, T., Herrmann, N., Brauer, J., Traidl‐Hoffmann, C., Havenith, R., Müller, S., Rhyner, C., Dreher, A., CK‐CARE study group, et al. (2023) · Allergy · doi:10.1111/all.15647 · Cited by 33
Ostropolets, A., Zhang, L., & Hripcsak, G. M. (2020) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaa200 · Cited by 39
“For example, rule-based phenotyping with chart review validation 84 may not be feasible in real time, which creates a need for best practices for fast yet accurate patient identification.”
BackgroundAdekkanattu, P. M., Olfson, M., Susser, L. C., Patra, B. G., Vekaria, V., Coombes, B. J., Lepow, L., Fennessy, B., Charney, A. W., Ryu, E., et al. (2022) · Journal of Affective Disorders · doi:10.1016/j.jad.2022.12.044 · Cited by 32
Self-citationMugauri, H. D., Chimsimbe, M., Shambira, G., Shamhu, S., Nyamasve, J., Munyanyi, M., Gongora, R., Zizhou, S., Gabida, M., & Makurumidze, R. (2025) · Global Health Action · doi:10.1080/16549716.2025.2492913 · Cited by 32
“Despite these efforts, challenges such as limited infrastructure, financial constraints, and data privacy concerns remain significant barriers to widespread EHR adoption in the region [9].”
Sheng, J. Q., Hu, P. J., Liu, X., Huang, T., & Chen, Y. (2020) · Journal of Medical Internet Research · doi:10.2196/18372 · Cited by 38
“To that end, predictive analytics empowers in-depth analyses of the rich patient clinical data in EHRs for the improved care and management of patients with acute diseases and illnesses [5,15].”
BackgroundBarnado, A., Carroll, R. J., Casey, C. G., Wheless, L., Denny, J. C., & Crofford, L. J. (2018) · Arthritis Research & Therapy · doi:10.1186/s13075-018-1561-8 · Cited by 31
Self-citationResearch Use of Electronic Health Records: Patients’ Views on Alternative Approaches to Permission
Hammack‐Aviran, C. M., Brelsford, K. M., McKenna, K., Graham, R., Lampron, Z. M., & Beskow, L. M. (2020) · AJOB Empirical Bioethics · doi:10.1080/23294515.2020.1755383 · Cited by 29
“…laboratory values, vital signs, medications, lifestyle and environmental factors) (Hripcsak and Albers 2013; Boland, Hripcsak, et al. 2013; Pathak, Kho, and Denny 2013; Richesson CONTACT Laura M. Beskow laura.m.beskow@vanderbilt.edu Center for Biomedical Ethics and Society, Vanderbilt…”
BackgroundMethodCoughlin, J. W., Martin, L., Zhao, D., Goheer, A. A., Woolf, T. B., Holzhauer, K., Lehmann, H. P., Lent, M. R., McTigue, K. M., Clark, J. M., et al. (2022) · Journal of Medical Internet Research · doi:10.2196/34191 · Cited by 30
“Potential participants were identified using EHR-based eligibility criteria (ie, “computable phenotype” [38]) to query the EHR.”
Ontology-driven and weakly supervised rare disease identification from clinical notes
Dong, H., Suárez-Paniagua, V., Zhang, H., Wang, M., Casey, A., Davidson, E., Chen, J., Alex, B., Whiteley, W. N., & Wu, H. (2023) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-023-02181-9 · Cited by 38
“Compared to the effi-cient and gradually economical genotyping (i.e. sequencing genomics information), phenotyping usually needs high-throughput computational approaches for the extraction of diseases and traits from electronic health records (EHRs) [21, 22].”
BackgroundWagholikar, K. B., Fischer, C. M., Goodson, A., Herrick, C. D., Rees, M., Toscano, E., MacRae, C. A., Scirica, B. M., Desai, A. S., & Murphy, S. N. (2018) · Journal of Medical Systems · doi:10.1007/s10916-018-1066-7 · Cited by 27
DISCOVERING PATIENT PHENOTYPES USING GENERALIZED LOW RANK MODELS
Schuler, A., Liu, V. X., Wan, J., Callahan, A., Udell, M. R., Stark, D. E., & Shah, N. H. (2015) · Pacific Symposium on Biocomputing (PSB) · doi:10.1142/9789814749411_0014 · Cited by 27
Barnado, A., Carroll, R. J., Casey, C. G., Wheless, L., Denny, J. C., & Crofford, L. J. (2018) · Arthritis Care & Research · doi:10.1002/acr.23553 · Cited by 27
Self-citationLow, Y., Gallego, B., & Shah, N. H. (2015) · Journal of Comparative Effectiveness Research · doi:10.2217/cer.15.53 · Cited by 27
Dissenting from care.data: an analysis of opt-out forms
Vezyridis, P., & Timmons, S. (2016) · Journal of Medical Ethics · doi:10.1136/medethics-2016-103654 · Cited by 26
Hammam, N., Li, J., Evans, M., Kay, J. L., Izadi, Z., Anastasiou, C., Gianfrancesco, M. A., Yazdany, J., & Schmajuk, G. (2021) · Arthritis Research & Therapy · doi:10.1186/s13075-021-02615-7 · Cited by 26
“Other studies have also noted underestimation of specific clinical phenotypes using EHR data [21], including in other rheumatic conditions [22].”
Kotfila, C., & Uzuner, Ö. (2015) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2015.07.016 · Cited by 25
“With the proliferation of electronic health records (EHR) in recent years, automated phenotyping for cohort selection has become an area of growing interest for the biomedical informatics community (Pathak, Kho, & Denny, 2013).”
BackgroundLearning bundled care opportunities from electronic medical records
Chen, Y., Kho, A., Liebovitz, D., Ivory, C. H., Osmundson, S. S., Bian, J., & Malin, B. (2017) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2017.11.014 · Cited by 26
“unique ICD9 codes for this set of patients is 4,543. In recognition of the fact that multiple ICD-9 codes may describe the same clinical phenomena [33, 34], various phenotyping investigations (e.g., [35, 36]) have adopted alternative vocabularies for the secondary analysis of EMRs, such as the Phenome-Wide Association…”
BackgroundSelf-citationAlbers, D. J., Elhadad, N., Claassen, J., Perotte, R., Goldstein, A., & Hripcsak, G. M. (2018) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2018.01.004 · Cited by 24
“Phenotyping may be done manually as a set of rules or queries that assert a state based on raw data [10–14], or it may be automated using machine learning [15–19].”
BackgroundCheligeer, K., Wu, G., Laws, A., Quan, M. L., Li, A., Brisson, A.-M., Xie, J., & Xu, Y. (2024) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-024-02677-y · Cited by 26
“Using advanced algorithms and data mining techniques can facilitate the pCR detection from unstructured medical text [5].”
MethodElectronic Health Record Data in Cancer Learning Health Systems: Challenges and Opportunities
Post, A. R., Burningham, Z. R., & Halwani, A. S. (2022) · JCO Clinical Cancer Informatics · doi:10.1200/cci.21.00158 · Cited by 29
“In this future vision, EHRs will automatically mine and render genotypic and phenotypic data 86 together with the latest evidence for clinical decision making.”
R-U policy frontiers for health data de-identification
Xia, W., Heatherly, R. D., Ding, X., Li, J., & Malin, B. (2015) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocv004 · Cited by 23
Frigaard, M., Rubinsky, A. D., Lo, L. J., Malkina, A., Karliner, L., Kohn, M. A., & Peralta, C. A. (2019) · BMC Nephrology · doi:10.1186/s12882-018-1156-2 · Cited by 24
Toward cross‐platform electronic health record ‐driven phenotyping using Clinical Quality Language
Brandt, P., Kiefer, R. C., Pacheco, J. A., Adekkanattu, P. M., Sholle, E., Ahmad, F. S., Xu, J., Xu, Z., Ancker, J. S., Wang, F., et al. (2020) · Learning Health Systems · doi:10.1002/lrh2.10233 · Cited by 24
“7,8 Second, in order to assemble the cohort of interest, the phenotype definition must be executed against a clinical database.”
BackgroundSelf-citationKashyap, M., Seneviratne, M. G., Banda, J. M., Falconer, T., Ryu, B., Yoo, S., Hripcsak, G. M., & Shah, N. H. (2020) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaa032 · Cited by 23
“However, issues such as missingness, accuracy, and heterogeneity in EHR data present major challenges to effective phenotyping.(4)”
BackgroundEvent2vec: Learning Representations of Events on Temporal Sequences
Hong, S., Wu, M., Li, H., & Wu, Z. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-63564-4_3 · Cited by 22
“1 shows the daily treatment procedures of patients in hospital [13].”
BackgroundGoodloe, R., Farber‐Eger, E. H., Boston, J., Crawford, D. C., & Bush, W. S. (2017) · PubMed Central · Cited by 22
Feature extraction for phenotyping from semantic and knowledge resources
Ning, W., Chan, S., Beam, A. L., Yu, M. Y., Geva, A., Liao, K. P., Mullen, M. P., Mandl, K. D., Kohane, I. S., Cai, T., et al. (2019) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2019.103122 · Cited by 23
“Electronic health records (EHR) have become a valuable resource to accelerate biomedical research [1].”
BackgroundPhenotyping through Semi-Supervised Tensor Factorization (PSST).
Henderson, J., He, H., Malin, B., Denny, J. C., Kho, A., Ghosh, J., & Ho, J. C. (2018) · Europe PMC (PubMed Central) · Cited by 22
Self-citationConsiderations for Improving the Portability of Electronic Health Record-Based Phenotype Algorithms.
Rasmussen, L. V., Brandt, P., Jiang, G., Kiefer, R. C., Pacheco, J. A., Adekkanattu, P. M., Ancker, J. S., Wang, F., Xu, Z., Pathak, J., et al. (2019) · PubMed Central · Cited by 22
Self-citationPatient Stratification Using Electronic Health Records from a Chronic Disease Management Program
Chen, R., Sun, J., Dittus, R. S., Fabbri, D., Kirby, J., Laffer, C. L., McNaughton, C. D., & Malin, B. (2016) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2016.2514264 · Cited by 21
“, diet and exercise) [11] which are not captured consistently in EHRs [54], [55].”
BackgroundShung, D. L., Tsay, C. J., Laine, L., Chang, D., Li, F., Thomas, P. G., Partridge, C. M., Simonov, M., Hsiao, A., Tay, J. K., et al. (2020) · Journal of Gastroenterology and Hepatology · doi:10.1111/jgh.15313 · Cited by 20
“5 Such processes have been used to improve the accuracy of case de fi nition of in fl ammatory bowel disease patients as Crohn ’ s disease or ulcerative colitis, as well as to facilitate clinical trial recruitment and deploy randomized controlled trials.”
MethodMachine Learning Informed Diagnosis for Congenital Heart Disease in Large Claims Data Source
Marelli, A. J., Li, C., Liu, A., Nguyen, H. H., Moroz, H., Brophy, J. M., Guo, L., Buckeridge, D. L., Tang, J., Yang, A. Y., et al. (2023) · JACC Advances · doi:10.1016/j.jacadv.2023.100801 · Cited by 23
Sakhnini, A., Saliba, W., Schwartz, N., & Bisharat, N. (2017) · Medicine · doi:10.1097/md.0000000000007284 · Cited by 21
Williams, B. A., Voyce, S. J., Sidney, S., Roger, V. L., Plante, T. B., Larson, S. L., LaMonte, M. J., Labarthe, D. R., DeBarmore, B. M., Chang, A. R., et al. (2022) · Journal of the American Heart Association · doi:10.1161/jaha.121.024409 · Cited by 24
Patient perspectives on use of electronic health records for research recruitment
Beskow, L. M., Brelsford, K. M., & Hammack‐Aviran, C. M. (2019) · BMC Medical Research Methodology · doi:10.1186/s12874-019-0686-z · Cited by 22
Mining Electronic Health Records: A SurveyPreprint
Yadav, P., Steinbach, M. S., Kumar, V., & Simon, G. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1702.03222 · Cited by 20
Constructing Epidemiologic Cohorts from Electronic Health Record Data
Williams, B. A. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph182413193 · Cited by 22
Metadata
- Authors
- Pathak, J., Kho, A. N., & Denny, J. C.
- Affiliations
- Mayo Clinic (US)
- Vanderbilt University (US)
- Corresponding author
- Pathak, Jyotishman
- Venue
- Journal of the American Medical Informatics Association
- Publisher
- Oxford University Press (OUP)
- Published
- 2013-12
- Volume
- 20
- Issue
- e2
- Pages
- e206-e211
- ISSN
- 1067-5027 (ISSN-L)
- 1527-974X
- PMC
- PMC3861925
- OpenAlex
- W2151068025
- Cited by
- 274
- References
- 108
- Citations by year
- 2026: 16 · 2025: 16 · 2024: 18 · 2023: 18 · 2022: 24 · 2021: 22 · 2020: 34 · 2019: 28 · 2018: 24 · 2017: 18 · 2016: 21 · 2015: 23 · 2014: 12
- FWCI
- 32.21
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Journal of the American Medical Informatics Association (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- HighWire Press Open Archive (submittedVersion)
- Funders
- National Institutes of Health
- National Heart, Lung, and Blood Institute (U19HL065962)
- National Human Genome Research Institute (U01HG006379, U01HG006378, U01HG006388)
- National Institute of General Medical Sciences (R01GM105688, U19GM061388)
- U.S. National Library of Medicine (R01LM010685)
- Topics
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Health Sciences › Medicine › Health Informatics › Biomedical Text Mining and Ontologies
- Keywords
- electronic health records, phenotype extraction, genotype-phenotype correlation, natural language processing, cohort identification, rare variants, genomic medicine, meaningful use, HITECH Act, phenotype validation, clinical research informatics, phenomics, genome-wide association study, phenome-wide association study
- MeSH terms
- Academic Medical Centers, Algorithms, Humans, Phenotype, Genomics, Genetic Association Studies, Electronic Health Records
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
