The DOI matches its Crossref record; 42 of the 42 references it cites were found in a registry or an index.
Identifying Patient Smoking Status from Medical Discharge Records
Open accessAbstract
Clinical narrative records contain much useful information. However, most clinical narratives are in the form of fragmented English free text, showing the characteristics of a clinical sublanguage. This makes their linguistic processing, search, and retrieval challenging.1 Traditional natural language processing (NLP) tools are not designed for the fragmented free text found in narrative clinical records; therefore, they do not perform well on this type of data.2 Limited access to clinical records has been a barrier to the widespread development of medical language processing (MLP) technologies. In the absence of a standardized, publicly available ground truth that encourages the development of MLP systems and allows their head-to-head comparison, successful MLP efforts have been limited, e.g., MedLEE3 and Symtxt.4 A few MLP systems have been developed,5 and such efforts have successfully shown the usefulness of MLP in clinical settings.6–8 To improve the availability of clinical records and to contribute to the advancement of the state of the art in MLP, within the i2b2 (Informatics for Integrating Biology to the Bedside) project, the authors de-identified and released a set of clinical records from Partners HealthCare. These records provided the basis for the development of ground truth for two challenge questions: Automatic de-identification of clinical data, i.e., de-identification challenge. Automatic evaluation of the smoking status of patients based on medical records, i.e., smoking challenge. Representative teams from the MLP community participated in the two challenges and met at a workshop organized by the authors to discuss the results of the challenges. The workshop was co-sponsored by the American Medical Informatics Association and met in conjunction with its Fall Symposium in November 2006. This article provides an overview of the smoking challenge and the findings of the workshop. An overview of the de-identification challenge can be found in Uzuner et al.9 The smoking challenge continues the tradition of attempting to identify the state of the art in automatic language processing. Outside of the medical domain, there have been many efforts in this direction. Most of these efforts have been led by Message Understanding Conferences (MUC)10 and the National Institute of Standards and Technology (NIST).11 MUC organized shared tasks on named entity recognition. NIST organized a series of Text Retrieval Evaluation Conferences (TREC) on various domains including blogs and legal documents; they also organized a series of shared tasks on topic detection and tracking, speaker recognition, language recognition, spoken document retrieval, machine translation, and entity extraction. In the biomedical domain, three such prominent efforts were BioCreAtIvE12 for information extraction, ImageCLEF13,14 for image retrieval, and TREC Genomics15 for question answering and information retrieval. Keeping the goals of TREC,16 MUC,17 BioCreAtIvE,18 etc., in mind for the smoking challenge, we created a collection of actual medical discharge records. We invited the development of systems that can predict the smoking status of patients based on the narratives in these medical discharge records. We limited the scope of this task to understanding only the explicitly reported smoking information. In other words, information that implicitly reveals the smoking status was excluded from this study. Our smoking challenge continued the work on the application of classification techniques to the medical domain,19–22 and extended the MLP studies on medical discharge records.8,23–29 Information on the smoking status of patients is important for many health studies, e.g., studies on asthma; however, before this challenge, the only system for the automatic evaluation of the smoking status of patients from their records was the HITEx system.30 The data for the smoking challenge consisted exclusively of discharge summaries from Partners HealthCare. We preprocessed these records so that they were de-identified, tokenized, broken into sentences, converted into XML format, and separated into training and test sets. Institutional review boards of Partners HealthCare, Massachusetts Institute of Technology, and the State University of New York at Albany approved the challenge and the data preparation process. The data for the challenge were annotated by pulmonologists. The pulmonologists were asked to classify patient records into five possible smoking status categories. For the purposes of this challenge, we defined these categories as follows: A Past Smoker is a patient whose discharge summary asserts explicitly that the patient was a smoker one year or more ago but who has not smoked for at least one year. The assertion “past smoker” without any temporal qualifications means Past Smoker unless there is text that says that the patient stopped smoking less than one year ago. A Current Smoker is a patient whose discharge summary asserts e
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A Coefficient of Agreement for Nominal Scales
Cohen, J. (1960) · Educational and Psychological Measurement · doi:10.1177/001316446002000104 · Cited by 42,398
Data Mining: Practical Machine Learning Tools and Techniques
(2011) · Elsevier eBooks · doi:10.1016/c2009-0-19715-5 · Cited by 25,437
Content Analysis: An Introduction to its Methodology.
Shelley, M., & Krippendorff, K. (1984) · Journal of the American Statistical Association · doi:10.2307/2288384 · Cited by 24,312
Introduction to Modern Information Retrieval
Salton, G., & McGill, M. J. (1983) · Virtual Defense Library (Ministerio de Defensa) · Cited by 11,392
A review of assessing the accuracy of classifications of remotely sensed data
Congalton, R. G. (1991) · Remote Sensing of Environment · doi:10.1016/0034-4257(91)90048-b · Cited by 7,724
A re-examination of text categorization methods
Yang, Y., & Liu, X. (1999) · Proceedings of the 22nd annual international ACM SIGIR conference on Research and development in information retrieval · doi:10.1145/312624.312647 · Cited by 2,652
Message Understanding Conference-6: a brief history
Grishman, R., & Sundheim, B. (1996) · Proceedings of the 16th conference on Computational linguistics - · doi:10.3115/992628.992709 · Cited by 1,408
A Simple Algorithm for Identifying Negated Findings and Diseases in Discharge Summaries
Chapman, W. W., Bridewell, W., Hanbury, P., Cooper, G. F., & Buchanan, B. G. (2001) · Journal of Biomedical Informatics · doi:10.1006/jbin.2001.1029 · Cited by 1,118
Agreement, the F-Measure, and Reliability in Information Retrieval
Hripcsak, G. (2005) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1733 · Cited by 949
A General Natural-language Text Processor for Clinical Radiology
Friedman, C., Alderson, P. O., Austin, J. H. M., Cimino, J. J., & Johnson, S. B. (1994) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.1994.95236146 · Cited by 778
Research works printed but not linked: 7
- Chang, 2002, Drug discovery today, 7, 992, 10.1016/S1359-6446(02)02457-1 · doi:10.1016/s1359-6446(02)02457-1
- 2000, Journal of the American Medical Informatics Association, 7, 378, 10.1136/jamia.2000.0070378 · doi:10.1136/jamia.2000.0070378
- 1995, Proceedings / the ... Annual Symposium on Computer Application [sic] in Medical Care. Symposium on Computer Applications in Medical Care, 19, 284
- 2006, Journal of the American Medical Informatics Association, 13, 488, 10.1197/jamia.M2082 · doi:10.1197/jamia.m2082
- 2004, Medinfo, 11, 773
- 2005, BMC bioinformatics [electronic resource], 6, S1
- 2004, Medinfo, 11, 639
Cited by
Natural language processing: an introduction
Nadkarni, P. M., Ohno-Machado, L., & Chapman, W. W. (2011) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2011-000464 · Cited by 1,975
Savova, G. K., Masanz, J. J., Ogren, P. V., Zheng, J., Sohn, S., Kipper-Schuler, K. C., & Chute, C. G. (2010) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.2009.001560 · Cited by 1,967
Mining electronic health records: towards better research applications and clinical care
Jensen, P. B., Jensen, L. J., & Brunak, S. (2012) · Nature Reviews Genetics · doi:10.1038/nrg3208 · Cited by 1,680
2010 i2b2/VA challenge on concepts, assertions, and relations in clinical text
Uzuner, Ö., South, B. R., Shen, S., & DuVall, S. L. (2011) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2011-000203 · Cited by 1,252
Clinical information extraction applications: A literature review
Wang, Y., Wang, L., Rastegar-Mojarad, M., Moon, S., Shen, F., Afzal, N., Liu, S., Zeng, Y., Mehrabi, S., Sohn, S., et al. (2018) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2017.11.011 · Cited by 910
Savova, G. K., Kipper-Schuler, K. C., Hurdle, J. F., & Meystre, S. M. (2008) · Yearbook of Medical Informatics · doi:10.1055/s-0038-1638592 · Cited by 895
The Co-Morbidity Burden of Children and Young Adults with Autism Spectrum Disorders
Kohane, I. S., McMurry, A., Weber, G., MacFadden, D., Rappaport, L., Kunkel, L., Bickel, J., Wattanasin, N., Spence, S., Murphy, S., et al. (2012) · PLoS ONE · doi:10.1371/journal.pone.0033224 · Cited by 562
Self-citationA review of approaches to identifying patient phenotype cohorts using electronic health records
Shivade, C., Raghavan, P., Fosler-Lussier, E., Embi, P. J., Elhadad, N., Johnson, S. B., & Lai, A. M. (2014) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2013-001935 · Cited by 531
Perera, G., Broadbent, M., Callard, F., Chang, C.-K., Downs, J., Dutta, R., Fernandes, A., Hayes, R. D., Henderson, M., Jackson, R., et al. (2016) · BMJ Open · doi:10.1136/bmjopen-2015-008721 · Cited by 492
Evaluating temporal relations in clinical text: 2012 i2b2 Challenge
Sun, W., Rumshisky, A., & Uzuner, O. (2013) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2013-001628 · Cited by 473
Self-citationExtracting medication information from clinical text
Uzuner, Ö., Solti, I., & Cadag, E. (2010) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.2010.003947 · Cited by 466
Evaluating the State-of-the-Art in Automatic De-identification
Uzuner, O., Luo, Y., & Szolovits, P. (2007) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m2444 · Cited by 452
Self-citationA clinical text classification paradigm using weak supervision and deep representation
Wang, Y., Sohn, S., Liu, S., Shen, F., Wang, L., Atkinson, E. J., Amin, S., & Liu, H. (2019) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-018-0723-6 · Cited by 405
A comparison of word embeddings for the biomedical natural language processing
Wang, Y., Liu, S., Afzal, N., Rastegar-Mojarad, M., Wang, L., Shen, F., Kingsbury, P., & Liu, H. (2018) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2018.09.008 · Cited by 370
Biomedical Ontologies in Action: Role in Knowledge Management, Data Integration and Decision Support
Bodenreider, O. (2008) · Yearbook of Medical Informatics · doi:10.1055/s-0038-1638585 · Cited by 368
CLAMP – a toolkit for efficiently building customized clinical natural language processing pipelines
Soysal, E., Wang, J., Jiang, M., Wu, Y., Pakhomov, S., Liu, H., & Xu, H. (2018) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocx132 · Cited by 360
Using electronic health records to drive discovery in disease genomics
Kohane, I. S. (2011) · Nature Reviews Genetics · doi:10.1038/nrg2999 · Cited by 325
Self-citationMeystre, S. M., Friedlin, F. J., South, B. R., Shen, S., & Samore, M. H. (2010) · BMC Medical Research Methodology · doi:10.1186/1471-2288-10-70 · Cited by 329
2018 n2c2 shared task on adverse drug events and medication extraction in electronic health records
Henry, S., Buchan, K., Filannino, M., Stubbs, A., & Uzuner, O. (2020) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocz166 · Cited by 303
Self-citationChapman, W. W., Nadkarni, P. M., Hirschman, L., D'Avolio, L. W., Savova, G. K., & Uzuner, O. (2011) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2011-000465 · Cited by 296
Using Electronic Patient Records to Discover Disease Correlations and Stratify Patient Cohorts
Roque, F. S., Jensen, P. B., Schmock, H., Dalgaard, M., Andreatta, M., Hansen, T., Søeby, K., Bredkjær, S., Juul, A., Werge, T., et al. (2011) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002141 · Cited by 282
Unravelling the human genome–phenome relationship using phenome-wide association studies
Bush, W. S., Oetjens, M. T., & Crawford, D. C. (2016) · Nature Reviews Genetics · doi:10.1038/nrg.2015.36 · Cited by 274
emrQA: A Large Corpus for Question Answering on Electronic Medical Records
Pampari, A., Raghavan, P., Liang, J., & Peng, J. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1258 · Cited by 191
Evaluating the state of the art in coreference resolution for electronic medical records
Uzuner, O., Bodnari, A., Shen, S., Forbush, T., Pestian, J., & South, B. R. (2012) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2011-000784 · Cited by 164
Self-citationLu, M. T., Raghu, V. K., Mayrhofer, T., Aerts, H. J. W. L., & Hoffmann, U. (2020) · Annals of Internal Medicine · doi:10.7326/m20-1868 · Cited by 123
SECNLP: A survey of embeddings in clinical natural language processing
Kalyan, K. S., & Sangeetha, S. (2020) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2019.103323 · Cited by 111
Bejan, C. A., Angiolillo, J., Conway, D., Nash, R., Shirey-Rice, J. K., Lipworth, L., Cronin, R. M., Pulley, J., Kripalani, S., Barkin, S., et al. (2018) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocx059 · Cited by 106
Cross-Topic Learning for Work Prioritization in Systematic Review Creation and Update
Cohen, A. M., Ambert, K., & McDonagh, M. (2009) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m3162 · Cited by 88
“Clinical researchers read hundreds of discharge summaries, trying to determine the smoking status of the individuals in their study population.(17) Specific to the work presented here, experts in evidence-based medicine review hundreds or thousands of articles on specific classes of drugs to synthesize the evidence, l…”
BackgroundDetecting Social and Behavioral Determinants of Health with Structured and Free-Text Clinical Data
Feller, D. J., Bear Don't Walk IV, O. J., Zucker, J., Yin, M. T., Gordon, P., & Elhadad, N. (2020) · Applied Clinical Informatics · doi:10.1055/s-0040-1702214 · Cited by 86
Ensemble method–based extraction of medication and related information from clinical texts
Kim, Y., & Meystre, S. M. (2020) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocz100 · Cited by 56
Lybarger, K., Dobbins, N. J., Long, R., Singh, A., Wedgeworth, P., Uzuner, Ö., & Yetisgen, M. (2023) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocad073 · Cited by 48
The 2022 n2c2/UW shared task on extracting social determinants of health
Lybarger, K., Yetisgen, M., & Uzuner, Ö. (2023) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocad012 · Cited by 46
A scoping review of publicly available language tasks in clinical natural language processing
Gao, Y., Dligach, D., Christensen, L., Tesch, S., Laffin, R., Xu, D., Miller, T., Uzuner, O., Churpek, M. M., & Afshar, M. (2022) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocac127 · Cited by 28
Self-citationPrompt-based Extraction of Social Determinants of Health Using Few-shot Learning
Ramachandran, G. K., Fu, Y., Han, B., Lybarger, K., Dobbins, N., Uzuner, O., & Yetisgen, M. (2023) · Proceedings of the 5th Clinical Natural Language Processing Workshop · doi:10.18653/v1/2023.clinicalnlp-1.41 · Cited by 18
Self-citationBigBIO: A Framework for Data-Centric Biomedical Natural Language Processing
Fries, J. A., Weber, L., Seelam, N., Altay, G., Datta, D., Garda, S., Kang, M., Su, R., Kusa, W., Cahyawijaya, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2206.15076 · Cited by 17
Automated Identification of Eviction Status From Electronic Health Record Notes
Yao, Z., Tsai, J., Liu, W., Levy, D. A., Druhl, E., Reisman, J. I., & Yu, H. (2023) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocad081 · Cited by 15
Development of a social and environmental determinants of health informatics maturity model
Espinoza, J. C., Sehgal, S., Phuong, J., Bahroos, N., Starren, J., Wilcox, A., & Meeker, D. (2023) · Journal of Clinical and Translational Science · doi:10.1017/cts.2023.691 · Cited by 9
Fu, Y., Ramachandran, G. K., Dobbins, N. J., Park, N., Leu, M., Rosenberg, A. R., Lybarger, K., Xia, F., Uzuner, O., & Yetisgen, M. (2024) · arXiv · doi:10.48550/arxiv.2404.00826 · Cited by 5
Self-citation
Metadata
- Authors
- Uzuner, O., Goldstein, I., Luo, Y., & Kohane, I.
- Venue
- Journal of the American Medical Informatics Association
- Publisher
- Oxford University Press (OUP)
- Published
- 2008-01-01
- Volume
- 15
- Issue
- 1
- Pages
- 14-24
- ISSN
- 1067-5027 (ISSN-L)
- 1527-974X
- OpenAlex
- W2109206523
- Cited by
- 359
- References
- 42
- FWCI
- 5.03
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.