The DOI matches its Crossref record; 71 of the 71 references it cites were found in a registry or an index.
Extracting information from the text of electronic medical records to improve case detection: a systematic review
Open accessAbstract
Background Electronic medical records (EMRs) are revolutionizing health-related research. One key issue for study quality is the accurate identification of patients with the condition of interest. Information in EMRs can be entered as structured codes or unstructured free text. The majority of research studies have used only coded parts of EMRs for case-detection, which may bias findings, miss cases, and reduce study quality. This review examines whether incorporating information from text into case-detection algorithms can improve research quality. Methods A systematic search returned 9659 papers, 67 of which reported on the extraction of information from free text of EMRs with the stated purpose of detecting cases of a named clinical condition. Methods for extracting information from text and the technical accuracy of case-detection algorithms were reviewed. Results Studies mainly used US hospital-based EMRs, and extracted information from text for 41 conditions using keyword searches, rule-based algorithms, and machine learning methods. There was no clear difference in case-detection algorithm accuracy between rule-based and machine learning methods of extraction. Inclusion of information from text resulted in a significant improvement in algorithm sensitivity and area under the receiver operating characteristic in comparison to codes alone (median sensitivity 78% (codes + text) vs 62% (codes), P = .03; median area under the receiver operating characteristic 95% (codes + text) vs 88% (codes), P = .025). Conclusions Text in EMRs is accessible, especially with open source information extraction algorithms, and significantly improves case detection when combined with codes. More harmonization of reporting within EMR studies is needed, particularly standardized reporting of algorithm accuracy metrics like positive predictive value (precision) and sensitivity (recall).
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Foundations of statistical natural language processing
Manning, C. D., & Schütze, H. (1999) · Cited by 9,816
Savova, G. K., Masanz, J. J., Ogren, P. V., Zheng, J., Sohn, S., Kipper-Schuler, K. C., & Chute, C. G. (2010) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.2009.001560 · Cited by 1,967
The Unified Medical Language System
Humphreys, B. L., McCray, A. T., & Lindberg, D. A. B. (1993) · Methods of Information in Medicine · doi:10.1055/s-0038-1634945 · Cited by 1,182
A Simple Algorithm for Identifying Negated Findings and Diseases in Discharge Summaries
Chapman, W. W., Bridewell, W., Hanbury, P., Cooper, G. F., & Buchanan, B. G. (2001) · Journal of Biomedical Informatics · doi:10.1006/jbin.2001.1029 · Cited by 1,118
A General Natural-language Text Processor for Clinical Radiology
Friedman, C., Alderson, P. O., Austin, J. H. M., Cimino, J. J., & Johnson, S. B. (1994) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.1994.95236146 · Cited by 778
Greenhalgh, T., & Hurwitz, B. (1999) · BMJ · doi:10.1136/bmj.318.7175.48 · Cited by 761
Automated Encoding of Clinical Documents Based on Natural Language Processing
Friedman, C., Shagina, L., Lussier, Y., & Hripcsak, G. (2004) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1552 · Cited by 551
MedEx: a medication information extraction system for clinical narratives
Xu, H., Stenner, S. P., Doan, S., Johnson, K. B., Waitman, L. R., & Denny, J. C. (2010) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m3378 · Cited by 531
The Unified Medical Language System
Humphreys, B. L., McCray, A. T., & Lindberg, D. A. B. (1993) · Yearbook of Medical Informatics · doi:10.1055/s-0038-1637976 · Cited by 507
Harkema, H., Dowling, J. N., Thornblade, T., & Chapman, W. W. (2009) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2009.05.002 · Cited by 406
Zeng, Q. T., Goryachev, S., Weiss, S., Sordo, M., Murphy, S. N., & Lazarus, R. (2006) · BMC Medical Informatics and Decision Making · doi:10.1186/1472-6947-6-30 · Cited by 360
Meystre, S. M., Friedlin, F. J., South, B. R., Shen, S., & Samore, M. H. (2010) · BMC Medical Research Methodology · doi:10.1186/1471-2288-10-70 · Cited by 329
Electronic medical records for discovery research in rheumatoid arthritis
Liao, K. P., Cai, T., Gainer, V., Goryachev, S., Zeng‐treitler, Q., Raychaudhuri, S., Szolovits, P., Churchill, S., Murphy, S., Kohane, I., et al. (2010) · Arthritis Care & Research · doi:10.1002/acr.20184 · Cited by 325
Portability of an algorithm to identify rheumatoid arthritis in electronic health records
Carroll, R. J., Thompson, W. K., Eyler, A. E., Mandelin, A. M., Cai, T., Zink, R. M., Pacheco, J. A., Boomershine, C. S., Lasko, T. A., Xu, H., et al. (2012) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2011-000583 · Cited by 235
The clinician's perspective on electronic health records and how they can affect patient care
Walsh, S. H. (2004) · BMJ · doi:10.1136/bmj.328.7449.1184 · Cited by 209
Currie, C. J., Poole, C. D., Evans, M., Peters, J. R., & Morgan, C. L. (2013) · The Journal of Clinical Endocrinology & Metabolism · doi:10.1210/jc.2012-3042 · Cited by 203
Bayes' rule: a tutorial introduction to Bayesian analysis
(2014) · Choice Reviews Online · doi:10.5860/choice.51-3301 · Cited by 174
Kushida, C. A., Nichols, D. A., Jadrnicek, R., Miller, R., Walsh, J. K., & Griffin, K. (2012) · Medical Care · doi:10.1097/mlr.0b013e3182585355 · Cited by 170
Recent trends in the incidence of recorded depression in primary care
Rait, G., Walters, K., Griffin, M., Buszewicz, M., Petersen, I., & Nazareth, I. (2009) · British Journal of Psychiatry · doi:10.1192/bjp.bp.108.058636 · Cited by 168
Ananthakrishnan, A. N., Cai, T., Savova, G., Cheng, S.-C., Chen, P., Perez, R. G., Gainer, V. S., Murphy, S. N., Szolovits, P., Xia, Z., et al. (2013) · Inflammatory Bowel Diseases · doi:10.1097/mib.0b013e31828133fd · Cited by 168
Importance of accurately identifying disease in studies using electronic health records
Manuel, D. G., Rosella, L. C., & Stukel, T. A. (2010) · BMJ · doi:10.1136/bmj.c4226 · Cited by 159
Widdifield, J., Bombardier, C., Bernatsky, S., Paterson, J. M., Green, D., Young, J., Ivers, N., Butt, D. A., Jaakkimainen, R. L., Thorne, J. C., et al. (2014) · BMC Musculoskeletal Disorders · doi:10.1186/1471-2474-15-216 · Cited by 159
Validation of Electronic Health Record Phenotyping of Bipolar Disorder Cases and Controls
Castro, V. M., Minnier, J., Murphy, S. N., Kohane, I. S., Churchill, S. E., Gainer, V. S., Cai, T., Hoffnagle, A. G., Dai, Y. G., Block, S. R., et al. (2014) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2014.14030423 · Cited by 149
Soler, J. K., Okkes, I. M., Wood, M., & Lamberts, H. (2008) · Family Practice · doi:10.1093/fampra/cmn028 · Cited by 144
Chung, C. P., Rohan, P., Krishnaswami, S., & McPheeters, M. L. (2013) · Vaccine · doi:10.1016/j.vaccine.2013.03.075 · Cited by 140
Chen, Y., Carroll, R. J., Hinz, E. M., Shah, A., Eyler, A. E., Denny, J. C., & Xu, H. (2013) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2013-001945 · Cited by 124
Pakhomov, S., Weston, S. A., Jacobsen, S. J., Chute, C. G., Meverden, R. A., & Roger, V. L. (2007) · PubMed · Cited by 122
Opportunities for and challenges of computerisation
Powsner, S. M., Wyatt, J. C., & Wright, P. (1998) · The Lancet · doi:10.1016/s0140-6736(98)08309-3 · Cited by 119 · cited 2×
“Among the main motivators for clinicians to code rather than use text is the increased ease of search, access, and retrieval.(5,6) A coded record allows the clinician to readily demonstrate that appropriate care has been provided, accurate diagnoses are made, and targets met.”
BackgroundNaïve Electronic Health Record phenotype identification for Rheumatoid arthritis.
Carroll, R. J., Eyler, A. E., & Denny, J. C. (2011) · PubMed Central · Cited by 111
Automatic Prediction of Rheumatoid Arthritis Disease Activity from the Electronic Medical Records
Lin, C., Karlson, E. W., Canhão, H. M., Miller, T. A., Dligach, D., Chen, P. J., Perez, R. N. G., Shen, Y., Weinblatt, M. E., Shadick, N. A., et al. (2013) · PLoS ONE · doi:10.1371/journal.pone.0069932 · Cited by 106
EMERSE: The Electronic Medical Record Search Engine.
Hanauer, D. A. (2006) · Europe PMC (PubMed Central) · Cited by 102
Xu, H., Fu, Z., Shah, A., Chen, Y., Peterson, N. B., Chen, Q., Mani, S., Levy, M. A., Dai, Q., & Denny, J. C. (2011) · PubMed · Cited by 98
The impact of health literacy on a patient's decision to adopt a personal health record.
Noblin, A. M., Wan, T. T. H., & Fottler, M. D. (2012) · PubMed · Cited by 96
Wu, S., Sohn, S., Ravikumar, K. E., Wagholikar, K. B., Jonnalagadda, S., Liu, H., & Juhn, Y. J. (2013) · Annals of Allergy Asthma & Immunology · doi:10.1016/j.anai.2013.07.022 · Cited by 88
Li, L., Chase, H. S., Patel, C. O., Friedman, C., & Weng, C. (2008) · PubMed · Cited by 81
Ford, E. M., Nicholson, A., Koeling, R., Tate, A. R., Carroll, J., Axelrod, L., Smith, H. E., Rait, G., Davies, K. A., Petersen, I., et al. (2013) · BMC Medical Research Methodology · doi:10.1186/1471-2288-13-105 · Cited by 80
Self-citationModeling Disease Severity in Multiple Sclerosis Using Electronic Health Records
Xia, Z., Secor, E. A., Chibnik, L. B., Bove, R. M., Cheng, S., Chitnis, T., Cagan, A., Gainer, V. S., Chen, P. J., Liao, K. P., et al. (2013) · PLoS ONE · doi:10.1371/journal.pone.0078927 · Cited by 79
Assessing the Difficulty and Time Cost of De-identification in Clinical Narratives
Phillips, W. F., Phansalkar, S., Sims, S. A., Hurdle, J. F., & Dorr, D. A. (2006) · Methods of Information in Medicine · doi:10.1055/s-0038-1634080 · Cited by 75
Imfeld, P., Brauchli Pernus, Y. B., Jick, S. S., & Meier, C. R. (2013) · Journal of Alzheimer's Disease · doi:10.3233/jad-121819 · Cited by 72
Wells, S., Hague, N., Thiru, K., & de Lusignan, S. (2003) · Methods of Information in Medicine · doi:10.1055/s-0038-1634239 · Cited by 71
Gulliford, M. C., Charlton, J., Ashworth, M. J., Rudd, A. G., & Toschke, A. M. (2009) · PLoS ONE · doi:10.1371/journal.pone.0007168 · Cited by 69
Wilke, R. A., Berg, R. L., Peissig, P. L., Kitchner, T., Sijercic, B., McCarty, C. A., & McCarty, D. J. (2007) · Clinical Medicine & Research · doi:10.3121/cmr.2007.726 · Cited by 69
Prospective recruitment of patients with congestive heart failure using an ad-hoc binary classifier
Pakhomov, S., Buntrock, J. D., & Chute, C. G. (2004) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2004.11.016 · Cited by 69
Stein, H. D., Nadkarni, P., Erdos, J., & Miller, P. L. (2000) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.2000.0070042 · Cited by 65
Second i2b2 workshop on natural language processing challenges for clinical records.
Uzuner, Ö. (2008) · PubMed · Cited by 63
Jones, M., DuVall, S. L., Spuhl, J., Samore, M. H., Nielson, C., & Rubin, M. A. (2012) · BMC Medical Informatics and Decision Making · doi:10.1186/1472-6947-12-34 · Cited by 60
Valkhoff, V. E., Coloma, P. M., Masclee, G. M. C., Gini, R., Innocenti, F., Lapi, F., Molokhia, M., Mosseveld, M., Nielsson, M. S., Schuemie, M. J., et al. (2014) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2014.02.020 · Cited by 59
Tate, A. R., Martin, A. G., Murray‐Thomas, T., Anderson, S. R., & Cassell, J. (2009) · BMC Medical Research Methodology · doi:10.1186/1471-2288-9-42 · Cited by 56
Self-citationAfzal, Z., Schuemie, M. J., van Blijderveen, J. C., Şen, E. Ö., Sturkenboom, M., & Kors, J. A. (2013) · BMC Medical Informatics and Decision Making · doi:10.1186/1472-6947-13-30 · Cited by 53
DeLisle, S., South, B. R., Anthony, J. A., Kalp, E. L., Gundlapallli, A., Curriero, F. C., Glass, G. E., Samore, M. H., & Perl, T. M. (2010) · PLoS ONE · doi:10.1371/journal.pone.0013377 · Cited by 47
Löve, Þ. J., Cai, T., & Karlson, E. W. (2010) · Seminars in Arthritis and Rheumatism · doi:10.1016/j.semarthrit.2010.05.002 · Cited by 46
Ye, Y., Tsui, F. R., D, M. M. W. M., Espino, J. U., & Li, Q. (2014) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2013-001934 · Cited by 45
Afzal, Z., Engelkes, M., Verhamme, K. M. C., Janssens, H. M., Sturkenboom, M., Kors, J. A., & Schuemie, M. J. (2013) · Pharmacoepidemiology and Drug Safety · doi:10.1002/pds.3438 · Cited by 43
Roch, A. M., Mehrabi, S., Krishnan, A., Schmidt, H., Kesterson, J. G., Beesley, C., Dexter, P. R., Palakal, M. J., & Schmidt, C. M. (2014) · HPB · doi:10.1111/hpb.12375 · Cited by 42
Use of computerized algorithm to identify individuals in need of testing for celiac disease
Ludvigsson, J. F., Pathak, J., Murphy, S., Durski, M. J., Kirsch, P. S., Chute, C. G., Ryu, E., & Murray, J. A. (2013) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2013-001924 · Cited by 42
Automated Outcome Classification of Emergency Department Computed Tomography Imaging Reports
Yadav, K., Sarioglu, E., Smith, M., & Choi, H. (2013) · Academic Emergency Medicine · doi:10.1111/acem.12174 · Cited by 41
DeLisle, S., Kim, B., Deepak, J., Siddiqui, T., Gundlapalli, A. V., Samore, M. H., & DʼAvolio, L. (2013) · PLoS ONE · doi:10.1371/journal.pone.0070944 · Cited by 40
Using natural language processing to improve accuracy of automated notifiable disease reporting.
Friedlin, J., Grannis, S. J., & Overhage, J. M. (2008) · PubMed · Cited by 39
Kalra, D., & Ingram, D. (2007) · Springer eBooks · doi:10.1007/1-84628-141-5_7 · Cited by 28
Automatically estimating the incidence of symptoms recorded in GP free text notes
Koeling, R., Tate, A. R., & Carroll, J. (2011) · International Workshop on Managing Interoperability and Complexity in Health Systems (MIXHS) · doi:10.1145/2064747.2064757 · Cited by 26
Self-citationGundlapalli, A. V., South, B. R., Phansalkar, S., Kinney, A. Y., Shen, S., DeLisle, S., Perl, T. M., & Samore, M. H. (2008) · PubMed Central · Cited by 22
Probabilistic Case Detection for Disease Surveillance Using Data in Electronic Medical Records
Tsui, F. R., D, M. M. W. M., Cooper, G. F., & Que, J. (2011) · Online Journal of Public Health Informatics · doi:10.5210/ojphi.v3i3.3793 · Cited by 22
A natural language processing algorithm to define a venous thromboembolism phenotype.
Hinz, E. M., Bastarache, L. A., & Denny, J. C. (2013) · Europe PMC (PubMed Central) · Cited by 20
Identifying patients with hypertension: a case for auditing electronic health record data.
Baus, A., HENDRYX, M. S., & Pollard, C. R. (2012) · PubMed Central · Cited by 17
Nielen, M. M. J., Ursum, J., Schellevis, F., & Korevaar, J. C. (2013) · BMC Family Practice · doi:10.1186/1471-2296-14-79 · Cited by 17
Predicting atrial fibrillation and flutter using Electronic Health Records
Karnik, S., Tan, S. L., Berg, B., Glurich, I., Zhang, J., Vidaillet, H. J., Jr., Page, C. D., & Chowdhary, R. (2012) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc.2012.6347254 · Cited by 16
An Efficient Pancreatic Cyst Identification Methodology Using Natural Language Processing
Mehrabi, S., Schmidt, C. M., Waters, J. A., Beesley, C., Krishnan, A., Kesterson, J., Dexter, P. R., Al-Haddad, M. A., Tierney, W. M., & Palakal, M. J. (2013) · Studies in health technology and informatics · doi:10.3233/978-1-61499-289-9-822 · Cited by 15
Zheng, H., Gaff, H. D., Smith, G. R., & DeLisle, S. (2014) · PLoS ONE · doi:10.1371/journal.pone.0100845 · Cited by 12
Unbiased Identification of Patients with Disorders of Sex Development
Hanauer, D. A., Gardner, M., & Sandberg, D. E. (2014) · PLoS ONE · doi:10.1371/journal.pone.0108702 · Cited by 10
Automated Identification of Diagnosis and Co-morbidity in Clinical Records
Cano, C., Blanco, A., & Peshkin, L. (2009) · Methods of Information in Medicine · doi:10.3414/me0615 · Cited by 5
Prevalence of treated hypertension in general practice in England and Wales, 1994 to 1998
Ryan, R. P., & Majeed, A. (2002) · UCL Discovery (University College London) · Cited by 4
Cited by
Applications of machine learning to diagnosis and treatment of neurodegenerative diseases
Myszczynska, M. A., Ojamies, P. N., Lacoste, A. M. B., Neil, D., Saffari, A., Mead, R., Hautbergue, G. M., Holbrook, J. D., & Ferraiuolo, L. (2020) · Nature Reviews Neurology · doi:10.1038/s41582-020-0377-8 · Cited by 595
Peiffer-Smadja, N., Rawson, T. M., Ahmad, R., Buchard, A., Georgiou, P., Lescure, F.-X., Birgand, G., & Holmes, A. H. (2020) · Clinical Microbiology and Infection · doi:10.1016/j.cmi.2019.09.009 · Cited by 561
Natural Language Processing of Clinical Notes on Chronic Diseases: Systematic Review
Sheikhalishahi, S., Miotto, R., Dudley, J. T., Lavelli, A., Rinaldi, F., & Osmani, V. (2019) · JMIR Medical Informatics · doi:10.2196/12239 · Cited by 558
“Ford et al [13] present a systematic review of 67 papers using IE techniques applied to medical records for the purpose of case detection (ie, finding occurrences of specific medical conditions).”
BackgroundLearning a Health Knowledge Graph from Electronic Medical Records
Rotmensch, M., Halpern, Y., Tlimat, A., Horng, S., & Sontag, D. (2017) · Scientific Reports · doi:10.1038/s41598-017-05778-z · Cited by 423
Koleck, T. A., Dreisbach, C., Bourne, P. E., & Bakken, S. (2019) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocy173 · Cited by 408
Metadata
- Authors
- Ford, E., Carroll, J. A., Smith, H. E., Scott, D., & Cassell, J. A.
- Venue
- Journal of the American Medical Informatics Association
- Publisher
- Oxford University Press (OUP)
- Published
- 2016-02-05
- Volume
- 23
- Issue
- 5
- Pages
- 1007-1015
- ISSN
- 1067-5027 (ISSN-L)
- 1527-974X
- OpenAlex
- W2283041611
- Cited by
- 408
- References
- 71
- FWCI
- 8.61
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.