CoChat

The DOI matches its Crossref record; 34 of the 35 references it cites were found in a registry or an index.

HARVEST, a longitudinal patient record summarizer

Open access
Hirsch, Jamie S, Tanenbaum, Jessica S, Lipsky Gorman, Sharon, Liu, Connie, Schmitz, Eric, Hashorva, Dritan, Ervits, Artem, Vawdrey, David, Sturm, Marc, Elhadad, Noémie · 2015 · Journal of the American Medical Informatics Association

Abstract

Objective To describe HARVEST, a novel point-of-care patient summarization and visualization tool, and to conduct a formative evaluation study to assess its effectiveness and gather feedback for iterative improvements. Materials and methods HARVEST is a problem-based, interactive, temporal visualization of longitudinal patient records. Using scalable, distributed natural language processing and problem salience computation, the system extracts content from the patient notes and aggregates and presents information from multiple care settings. Clinical usability was assessed with physician participants using a timed, task-based chart review and questionnaire, with performance differences recorded between conditions (standard data review system and HARVEST). Results HARVEST displays patient information longitudinally using a timeline, a problem cloud as extracted from notes, and focused access to clinical documentation. Despite lack of familiarity with HARVEST, when using a task-based evaluation, performance and time-to-task completion was maintained in patient review scenarios using HARVEST alone or the standard clinical information system at our institution. Subjects reported very high satisfaction with HARVEST and interest in using the system in their daily practice. Discussion HARVEST is available for wide deployment at our institution. Evaluation provided informative feedback and directions for future improvements. Conclusions HARVEST was designed to address the unmet need for clinicians at the point of care, facilitating review of essential patient information. The deployment of HARVEST in our institution allows us to study patient record summarization as an informatics intervention in a real-world setting. It also provides an opportunity to learn how clinicians use the summarizer, enabling informed interface and content iteration and optimization to improve patient care.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1136/amiajnl-2014-002945

Cites

34 of 34 references verified0 citation errors · 0 not checkable

Statistics

Figures

Figure 1: Screenshot of iNYP, the standard clinical information review system at New York Presbyterian Hospital (NYPH) for a deidentified patient. The left frame displays the contents of the patient chart. Selecting a category opens that section in the main frame. In this example, the main frame displays the notes in chronological order with the author names, and selected note contents appear in the bottom frame.
Figure 1: Screenshot of iNYP, the standard clinical information review system at New York Presbyterian Hospital (NYPH) for a deidentified patient. The left frame displays the contents of the patient chart. Selecting a category opens that section in the main frame. In this example, the main frame displays the notes in chronological order with the author names, and selected note contents appear in the bottom frame.CC BY-NC · PubMed Central
Figure 2: HARVEST system architecture. The distributed online module parses and indexes notes and visits as they are generated from the electronic health record, so that, for all patients in the institution, the HBase database stores information about visits, notes, and problems extracted from the notes. An authenticated user can invoke the online web-based visualization for a particular medical record number, as part of the clinical information review system. CUI, concept unique identifier; MRN, medical record number.
Figure 2: HARVEST system architecture. The distributed online module parses and indexes notes and visits as they are generated from the electronic health record, so that, for all patients in the institution, the HBase database stores information about visits, notes, and problems extracted from the notes. An authenticated user can invoke the online web-based visualization for a particular medical record number, as part of the clinical information review system. CUI, concept unique identifier; MRN, medical record number.CC BY-NC · PubMed Central
Figure 3: Study protocol for groups of four subjects tested with four patient cases. This study consisted of a total of three groups of four subjects each (total 12 subjects and 12 patient cases). Light gray boxes indicate study condition with the use of HARVEST and dark gray boxes indicates use of iNYP alone. Overall, the study had 12 subjects and 12 patient cases.
Figure 3: Study protocol for groups of four subjects tested with four patient cases. This study consisted of a total of three groups of four subjects each (total 12 subjects and 12 patient cases). Light gray boxes indicate study condition with the use of HARVEST and dark gray boxes indicates use of iNYP alone. Overall, the study had 12 subjects and 12 patient cases.CC BY-NC · PubMed Central
Figure 4: Screenshot of HARVEST for the same deidentified patient as in the iNYP screenshot in figure 1. For the selected time range (3 months), stable angina, pulmonary hypertension, end-stage renal disease, and dyspnea are the most prominently documented problems. HARVEST also identified diabetes mellitus, hypertension, and dyslipidemia as important problems. The problem ‘dyspnea’ was selected. The Notes panel lists all notes in the selected time range that mention this problem. Out of these, a cardiology consult note is selected and displayed in the lower right panel, with all mentions of dyspnea (and synonyms) highlighted. On the timeline, documentation of dyspnea is highlighted by purple bars, indicating that dyspnea was a particularly salient issue at that time as well as 6 months later.
Figure 4: Screenshot of HARVEST for the same deidentified patient as in the iNYP screenshot in figure 1. For the selected time range (3 months), stable angina, pulmonary hypertension, end-stage renal disease, and dyspnea are the most prominently documented problems. HARVEST also identified diabetes mellitus, hypertension, and dyslipidemia as important problems. The problem ‘dyspnea’ was selected. The Notes panel lists all notes in the selected time range that mention this problem. Out of these, a cardiology consult note is selected and displayed in the lower right panel, with all mentions of dyspnea (and synonyms) highlighted. On the timeline, documentation of dyspnea is highlighted by purple bars, indicating that dyspnea was a particularly salient issue at that time as well as 6 months later.CC BY-NC · PubMed Central
Figure 5: Boxplot of overall questionnaire scores between the two scenarios (iNYP vs HARVEST). There were no statistically significant differences in scores between the two conditions.
Figure 5: Boxplot of overall questionnaire scores between the two scenarios (iNYP vs HARVEST). There were no statistically significant differences in scores between the two conditions.CC BY-NC · PubMed Central
Figure 6: Responses to Likert-type questions in the post-task questionnaire regarding helpfulness of the three features of HARVEST: access to the notes, the timeline, and the problem cloud. Most participants found the three features somewhat or very helpful (blue highlighted region to right of vertical bar), whereas fewer found the features somewhat or very unhelpful (pink and red highlighted region to left of vertical bar).
Figure 6: Responses to Likert-type questions in the post-task questionnaire regarding helpfulness of the three features of HARVEST: access to the notes, the timeline, and the problem cloud. Most participants found the three features somewhat or very helpful (blue highlighted region to right of vertical bar), whereas fewer found the features somewhat or very unhelpful (pink and red highlighted region to left of vertical bar).CC BY-NC · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 35 references

Research works printed but not linked: 3

  1. Weed, 2011, Medicine in denial
  2. Graedon, 2013, Is Larry Weed right, J Particip Med, 5, e13
  3. Lipsky-Gorman, 2011, ClinNote and HealthTermFinder: a pipeline for processing clinical notes

Cited by

See all 140

Metadata

Authors
Hirsch, J. S., Tanenbaum, J. S., Lipsky Gorman, S., Liu, C., Schmitz, E., Hashorva, D., Ervits, A., Vawdrey, D., Sturm, M., & Elhadad, N.
Affiliations
  • Columbia University (US)
  • Department of Biomedical Informatics, Columbia University, New York, USA
  • NewYork–Presbyterian Hospital (US)
Corresponding author
Hirsch, Jamie S
Venue
Journal of the American Medical Informatics Association
Published
2014-10-28
Volume
22
Issue
2
Pages
263-274
ISSN
  • 1067-5027 (ISSN-L)
  • 1527-974X
OpenAlex
W2167976315
Cited by
135
References
35
Citations by year
2026: 6 · 2025: 9 · 2024: 7 · 2023: 11 · 2022: 15 · 2021: 18 · 2020: 12 · 2019: 11 · 2018: 16 · 2017: 13 · 2016: 8 · 2015: 8
FWCI
17.67
Type
journal article
Language
en
Open access
Locations
  • Journal of the American Medical Informatics Association (publishedVersion, cc-by-nc)
  • PubMed Central (publishedVersion, cc-by-nc)
  • PubMed (publishedVersion)
Funders
  • National Science Foundation (1344668)
  • U.S. National Library of Medicine (LM007079, T15LM007079, R01LM010027)
Topics
  • Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
  • Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › Data Visualization and Analytics
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
Keywords
longitudinal patient records, clinical note extraction, natural language processing, temporal data visualization, clinical usability, task-based assessment, physician satisfaction, clinical information systems
MeSH terms
Attitude to Computers, Data Display, Humans, Natural Language Processing, Surveys and Questionnaires, Software, User-Computer Interface, Electronic Health Records

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: HARVEST, a longitudinal patient record summarizer. https://app.cochat.ai/papers/10.1136/amiajnl-2014-002945
The paper
Hirsch, J. S., Tanenbaum, J. S., Lipsky Gorman, S., Liu, C., Schmitz, E., Hashorva, D., Ervits, A., Vawdrey, D., Sturm, M., & Elhadad, N. (2015). HARVEST, a longitudinal patient record summarizer. Journal of the American Medical Informatics Association. https://doi.org/10.1136/amiajnl-2014-002945

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.