Characterizing Chronic Pain Episodes in Clinical Text at Two Health Care Systems: Comprehensive Annotation and Corpus Analysis
Open accessPublished version of a preprint
Abstract
Background: Chronic pain affects more than 20% of adults in the United States and is associated with substantial physical, mental, and social burden. Clinical text contains rich information about chronic pain, but no systematic appraisal has been performed to assess the electronic health record (EHR) narratives for these patients. A formal content analysis of the unstructured EHR data can inform clinical practice and research in chronic pain. Objective: We characterized individual episodes of chronic pain by annotating and analyzing EHR notes for a stratified cohort of adults with known chronic pain. Methods: We used the Rochester Epidemiology Project infrastructure to screen all residents of Olmsted County, Minnesota, for evidence of chronic pain, between January 1, 2005, and September 30, 2015. Diagnosis codes were used to assemble a cohort of 6586 chronic pain patients; people with cancer were excluded. The records of an age- and sex-stratified random sample of 62 patients from the cohort were annotated using an iteratively developed guideline. The annotated concepts included date, location, severity, causes, effects on quality of life, diagnostic procedures, medications, and other treatment modalities. Results: A total of 94 chronic pain episodes from 62 distinct patients were identified by reviewing 3272 clinical notes. Documentation was written by clinicians across a wide spectrum of specialties. Most patients (40/62, 65%) had 1 pain episode during the study period. Interannotator agreement ranged from 0.78 to 1.00 across the annotated concepts. Some pain-related concepts (eg, body location) had 100% (94/94) coverage among all the episodes, while others had moderate coverage (eg, effects on quality of life) (55/94, 59%). Back pain and leg pain were the most common types of chronic pain in the annotated cohort. Musculoskeletal issues like arthritis were annotated as the most common causes. Opioids were the most commonly captured medication, while physical and occupational therapies were the most common nonpharmacological treatments. Conclusions: We systematically annotated chronic pain episodes in clinical text. The rich content analysis results revealed complexity of the chronic pain episodes and of their management, as well as the challenges in extracting pertinent information, even for humans. Despite the pilot study nature of the work, the annotation guideline and corpus should be able to serve as informative references for other institutions with shared interest in chronic pain research using EHRs.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The corpus from study34 characterized chronic pain episodes in adult patients receiving treatment at Mayo Clinic and the Olmsted Medical Center, comprising 62 adults with noncancer chronic pain.
“For our fourth dataset we utilized a subset of the corpus from a study34 that characterized chronic pain episodes in adult patients receiving treatment at Mayo Clinic and the Olmsted Medical Center. An annotated corpus of 62 adults experiencing noncancer chronic pain was created using chronic pain ICD code for the anchor diagnosis and retrieving all the patient’s clinical notes 6 months before and”
From Social determinants of health extraction from clinical notes across institutions using large language models (2025) · cites this paper as [34]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Treede, R.-D., Rief, W., Barke, A., Aziz, Q., Bennett, M. I., Benoliel, R., Cohen, M., Evers, S., Finnerup, N. B., First, M. B., et al. (2019) · Pain · doi:10.1097/j.pain.0000000000001384 · Cited by 3,478 · see citing works
Prevalence of Chronic Pain and High-Impact Chronic Pain Among Adults — United States, 2016
Dahlhamer, J., Lucas, J., Zelaya, Nahin, R., Mackey, S., DeBar, L., Kerns, R., Von Korff, M., Porter, L., & Helmick, C. (2018) · MMWR. Morbidity and Mortality Weekly Report · doi:10.15585/mmwr.mm6736a2 · Cited by 2,598 · see citing works
Persistent Pain and Well-being: A World Health Organization Study in Primary Care
Gureje, O., Von Korff, M., Simon, G. E., & Gater, R. (1998) · JAMA · doi:10.1001/jama.280.2.147 · Cited by 1,496 · see citing works
A review of chronic pain impact on patients, their social environment and the health care system
Duenas, M., Ojeda, B., Salazar, A., Mico, J. A., & Failde, I. (2016) · Journal of Pain Research · doi:10.2147/jpr.s105892 · Cited by 985 · see citing works
Agreement, the F-Measure, and Reliability in Information Retrieval
Hripcsak, G. M. (2005) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1733 · Cited by 949 · see citing works
Rocca, W. A., Yawn, B. P., Sauver, J. L. S., Grossardt, B. R., & Melton, L. J. (2012) · Mayo Clinic Proceedings · doi:10.1016/j.mayocp.2012.08.012 · Cited by 791 · see citing works
Martell, B. A., O’Connor, P. G., Kerns, R. D., Becker, W. C., Morales, K. H., Kosten, T. R., & Fiellin, D. A. (2007) · Annals of Internal Medicine · doi:10.7326/0003-4819-146-2-200701160-00006 · Cited by 767 · see citing works
Sauver, J. L. S., Grossardt, B. R., Leibson, C. L., Yawn, B. P., Melton, L. J., & Rocca, W. A. (2012) · Mayo Clinic Proceedings · doi:10.1016/j.mayocp.2011.11.009 · Cited by 652 · see citing works
Data Resource Profile: The Rochester Epidemiology Project (REP) medical records-linkage system
Sauver, J. L. S., Grossardt, B. R., Yawn, B. P., Melton, L. J., Pankratz, J. J., Brue, S. M., & Rocca, W. A. (2012) · International Journal of Epidemiology · doi:10.1093/ije/dys195 · Cited by 613 · see citing works
The IASP classification of chronic pain for ICD-11: chronic cancer-related pain
Bennett, M., Kaasa, S., Barke, A., Korwisi, B., Rief, W., Treede, R., & Pain, T. I. T. F. T. C. O. C. (2018) · Pain · doi:10.1097/j.pain.0000000000001363 · Cited by 388 · see citing works
Reuben, D. B., Alvanzo, A. A. H., Ashikaga, T., Bogat, G. A., Callahan, C. M., Ruffing, V., & Steffens, D. C. (2015) · Annals of Internal Medicine · doi:10.7326/m14-2775 · Cited by 191 · see citing works
Tian, T., Zlateva, I., & Anderson, D. (2013) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2013-001856 · Cited by 163 · see citing works
Institute of Medicine (2011) · National Academies Press eBooks · doi:10.17226/13172 · Cited by 144 · see citing works
Heintzelman, N., Taylor, R. J., Simonsen, L., Lustig, R. L., Anderko, D., Haythornthwaite, J. A., Childs, L. C., & Bova, G. S. (2012) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2012-001076 · Cited by 75 · see citing works
BRIEF REPORT: Failure of an Electronic Medical Record Tool to Improve Pain Assessment Documentation
Saigh, O., Triola, M. M., & Link, R. N. (2006) · Journal of General Internal Medicine · doi:10.1111/j.1525-1497.2006.00330.x · Cited by 27 · see citing works
Dorflinger, L. M., Gilliam, W. P., Lee, A. W., & Kerns, R. D. (2014) · Translational Behavioral Medicine · doi:10.1007/s13142-014-0260-5 · Cited by 12 · see citing works
Annotation of pain and anesthesia events for surgery-related processes and outcomes extraction
Yim, W., Tedesco, D., Curtin, C. M., & Hernandez‐Boussard, T. (2017) · doi:10.18653/v1/w17-2325 · Cited by 2 · see citing works
Flagged first, then by citations
Research works printed but not linked: 1
- [13]Stubbs A MAE and MAI: Lightweight annotation and adjudication tools Proceedings of the 5th Linguistic Annotation Workshop 2011 The 5th Linguistic Annotation Workshop June 23-24, 2011 Portland, OR Stroudsburg, PA Association for Computational Linguistics 129 133
Cited by
Sorted by citations
Systematic design and data-driven evaluation of social determinants of health ontology (SDoHO)
Dang, Y., Li, F., Hu, X., Keloth, V. K., Zhang, M., Fu, S., Amith, M. “., Fan, J. W., Du, J., Yu, E., et al. (2023) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocad096 · Cited by 30 · see citing works
Self-citationKeloth, V. K., Selek, S., Chen, Q., Gilman, C., Fu, S., Dang, Y., Chen, X., Hu, X., Zhou, Y., He, H., et al. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01645-8 · Cited by 24 · see citing works
“…explored the use of deep neural network architectures such as CNN 29 , LSTM 30 , and BERT 21,23,31 to automatically classify SDoH categories 32 , or named entity recognition (NER) based approaches, utilizing tools like cTAKES 33 , CNN, andBERT-basedmodels 34 ,toextractSDoHfactorsfromclinicaltext.”
Self-citationKeloth, V., Selek, S., Chen, Q.-Y., Gilman, C., Fu, S.-Y., Dang, Y., Chen, X., Hu, X.-Y., Zhou, Y.-J., He, H., et al. (2024) · medRxiv · doi:10.1101/2024.05.21.24307726 · Cited by 6 · see citing works
Self-citationChaturvedi, J., Chance, N., Mirza, L., Vernugopan, V., Velupillai, S. U., Stewart, R. J., & Roberts, A. (2023) · JMIR Formative Research · doi:10.2196/45849 · Cited by 8 · see citing works
Phenotyping to predict 12-month health outcomes of older general medicine patients
Woodman, R. J., Bryant, K., Sorich, M. J., Thompson, C. H., Russell, P. T., Pilotto, A., & Mangoni, A. A. (2025) · Aging Clinical and Experimental Research · doi:10.1007/s40520-024-02924-2 · Cited by 7 · see citing works
Identifying Mentions of Pain in Mental Health Records Text: A Natural Language Processing Approach
Chaturvedi, J., Velupillai, S. U., Stewart, R. J., & Roberts, A. (2024) · Studies in health technology and informatics · doi:10.3233/shti231054 · Cited by 4 · see citing works
Nunes, D. A. P., Ferreira‐Gomes, J., Vaz, C., de Olíveira, D., Pimenta, S., Neto, F. L. M., & de Matos, D. M. (2022) · Revista Dor · doi:10.24875/dor.m22000014 · Cited by 2 · see citing works
Chronic Pain and Language: A Topic Modelling Approach to Personal Pain DescriptionsPreprint
Nunes, D. A. P., Gomes, J. F., Neto, F. L. M., & de Matos, D. M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.00402 · Cited by 4 · see citing works
Kondo, U., Ishida, M., Masukawa, K., Inoue, R., Nakayama, M., & Miyashita, M. (2026) · BMJ Supportive & Palliative Care · doi:10.1136/spcare-2026-006282 · Cited by 1 · see citing works
Towards User-centered Corpus Development: Lessons Learnt from Designing and Developing MedTator.
He, H., Fu, S., Wang, L., Wen, A., Liu, S., Moon, S., Miller, K., & Liu, H. (2022) · PubMed Central
Self-citationZhang, X., WILSON, C. L., Eyre, H., Reed, D. E., Wai, T. H., Kloehn, A. T., Rosser, E. W., Luo, G., & Zeliadt, S. B. (2026) · JMIR Research Protocols · doi:10.2196/96346
- Authors
- Carlson, L. A., Jeffery, M. M., Fu, S., He, H., McCoy, R. G., Wang, Y., Hooten, W. M., St Sauver, J., Liu, H., & Fan, J.
- Affiliations
- Mayo Clinic (US)
- Corresponding author
- Fan, Jungwei
- ORCID
- Venue
- JMIR Medical Informatics
- Publisher
- JMIR Publications Inc.
- Published
- 2020-11-16
- Volume
- 8
- Issue
- 11
- Pages
- e18659
- ISSN
- 2291-9694 (ISSN-L)
- PMC
- PMC7704279
- OpenAlex
- W3094522459
- Cited by
- 11
- References
- 17
- Citations by year
- 2026: 2 · 2025: 2 · 2024: 2 · 2023: 2 · 2022: 2 · 2021: 1
- FWCI
- 0.41
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- JMIR Medical Informatics (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- National Institutes of Health (U01TR002062, UL1TR002377)
- National Center for Advancing Translational Sciences
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Biomedical Text Mining and Ontologies
- Health Sciences › Health Professions › Health Information Management › Medical Coding and Health Information
- Health Sciences › Medicine › Pharmacology › Musculoskeletal pain and rehabilitation
- Keywords
- chronic pain, clinical text annotation, electronic health records, unstructured clinical notes, annotation guidelines, inter-annotator agreement, pain intensity, quality of life, opioids, non-pharmacological treatment, back pain, leg pain, musculoskeletal pain, arthritis, Rochester Epidemiology Project
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
