Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index; none of the 7 sources its claims cite is checked for support yet.

Does integrating nonurgent, clinically significant radiology alerts within the electronic health record impact closed-loop communication and follow-up?

Open access
O’Connor, Stacy D, Dalal, Anuj K, Sahni, V Anik, Lacson, Ronilda, Khorasani, Ramin · 2016 · Journal of the American Medical Informatics Association

Abstract

Objective To assess whether integrating critical result management software—Alert Notification of Critical Results (ANCR)—with an electronic health record (EHR)-based results management application impacts closed-loop communication and follow-up of nonurgent, clinically significant radiology results by primary care providers (PCPs). Materials and Methods This institutional review board-approved study was conducted at a large academic medical center. Postintervention, PCPs could acknowledge nonurgent, clinically significant ANCR-generated alerts (“alerts”) within ANCR or the EHR. Primary outcome was the proportion of alerts acknowledged via EHR over a 24-month postintervention. Chart abstractions for a random sample of alerts 12 months preintervention and 24 months postintervention were reviewed, and the follow-up rate of actionable alerts (eg, performing follow-up imaging, administering antibiotics) was estimated. Pre- and postintervention rates were compared using the Fisher exact test. Postintervention follow-up rate was compared for EHR-acknowledged alerts vs ANCR. Results Five thousand nine hundred and thirty-one alerts were acknowledged by 171 PCPs, with 100% acknowledgement (consistent with expected ANCR functionality). PCPs acknowledged 16% (688 of 4428) of postintervention alerts in the EHR, with the remaining in ANCR. Follow-up was documented for 85 of 90 (94%; 95% CI, 88%-98%) preintervention and 79 of 84 (94%; 95% CI, 87%-97%) postintervention alerts ( P > .99). Postintervention, 11 of 14 (79%; 95% CI, 52%-92%) alerts were acknowledged via EHR and 68 of 70 (97%; 95% CI, 90%-99%) in ANCR had follow-up ( P = .03). Conclusions Integrating ANCR and EHR provides an additional workflow for acknowledging nonurgent, clinically significant results without significant change in rates of closed-loop communication or follow-up of alerts.

Overview

CoChat Summary

TL;DR

Integrating ANCR-generated nonurgent, clinically significant radiology alerts into an EHR-based results management application gave primary care providers an additional acknowledgement workflow, but most alerts continued to be acknowledged in ANCR. The integration did not significantly change closed-loop communication or follow-up rates overall, although postintervention EHR-acknowledged alerts had lower follow-up than ANCR-acknowledged alerts.

Key findings

  • Of 5931 Level 3 alerts acknowledged by 171 PCPs, postintervention PCPs acknowledged 16% (688 of 4428) in the EHR and 84% (3740 of 4428) in ANCR; preintervention, 100% (1503 of 1503) were acknowledged in ANCR and 0% in the EHR. Overall acknowledgement was 100%.
  • Follow-up was documented for 94% of actionable alerts preintervention (85 of 90; 95% CI, 88%-98%) and 94% postintervention (79 of 84; 95% CI, 87%-97%; P > .99). The proportion of alerts judged actionable was 90% preintervention vs 84% postintervention (P = .29).
  • Postintervention, follow-up was 79% for actionable alerts acknowledged via EHR (11 of 14; 95% CI, 52%-92%) vs 97% for those acknowledged in ANCR (68 of 70; 95% CI, 90%-99%; P = .03).
  • Median time to acknowledgement was 0.6 hours preintervention and 0.6 hours postintervention (P = .003); postintervention, median time was 7 hours in the EHR vs 0.3 hours in ANCR (P < .001). Month-to-month variation in EHR acknowledgement was not statistically significant; median per-PCP EHR acknowledgement was 4% (IQR, 0-15).

Methods

  • IRB-approved pre-post study at a 793-bed tertiary academic medical center with a radiology department performing over 600 000 examinations annually. Included nonurgent, clinically significant Level 3 ANCR alerts for 171 PCPs at 13 affiliated outpatient practices during a 12-month preintervention period (May 2011-April 2012) and 24-month postintervention period (June 2012-May 2014); excluded Level 1/2 alerts, inpatient/ED alerts, and non-PCP providers.
  • Primary outcome was the proportion of Level 3 ANCR-generated alerts acknowledged in the EHR. Secondary outcome was follow-up rate of actionable alerts, defined as the proportion acted upon appropriately; a random sample of 200 acknowledged alerts (100 preintervention, 100 postintervention) was chart-abstracted and categorized by physicians with consensus review.
  • Pre/post follow-up rates were compared using the Fisher exact test; the Cochran-Armitage trend test evaluated trends across 3 years; the Wilcoxon rank sum test compared time to acknowledgement; a P-type statistical process control chart analyzed system adoption over time.

Limitations

  • Stated: the study was performed at 1 academic center with a single combination of critical radiology results software and EHR-based results management application.
  • Stated: there was wide variation in the proportion of ANCR-generated alerts each PCP acknowledged in the EHR; because clustering was minimal, PCP was not included as a covariate. Some variation may have been due to lack of training, and adoption may have improved with additional training.
  • Stated: the pre-postintervention design, rather than a randomized trial, cannot control for changes in follow-up due to factors other than the intervention; the authors note no other interventions targeting follow-up of nonurgent clinically significant radiology findings were introduced during the study period. Evident from the text: the postintervention EHR-acknowledged actionable alert subgroup was small (14 alerts), producing a wide confidence interval (52%-92%) and limited precision.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1093/jamia/ocv105

Cites

21 of 21 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

0 Valid · 0 Not valid · 5 Not checked · 25 Own results and interpretations

10 more claims

Create a free account to see all 30 claims and to verify the 7 references still open.

See all claims — free account
  • It may state two separate claims as one. 9
  • Probably not a substantial claim. 6

Create a free account to see these claims.

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

2 of 2

Citations

Cites

Show all 21 references

Research works printed but not linked: 1

  1. Charles, Adoption of Electronic Health Record Systems Among U.S. Non-federal Acute Care Hospitals: 2008-2013

Cited by

See all 25

Metadata

Authors
O’Connor, S. D., Dalal, A. K., Sahni, V. A., Lacson, R., & Khorasani, R.
Affiliations
  • Brigham and Women's Hospital (US)
  • Center for Evidence Based Imaging, Brookline, Massachusetts, USA
Venue
Journal of the American Medical Informatics Association
Published
2015-09-02
Volume
23
Issue
2
Pages
333-338
ISSN
  • 1067-5027 (ISSN-L)
  • 1527-974X
OpenAlex
W1920681141
Cited by
25
References
21
Citations by year
2025: 3 · 2024: 1 · 2023: 4 · 2022: 2 · 2021: 1 · 2020: 2 · 2019: 1 · 2018: 5 · 2017: 2 · 2016: 4
FWCI
4.52
Type
journal article
Language
en
Open access
Locations
  • Journal of the American Medical Informatics Association (publishedVersion)
  • PubMed (publishedVersion)
  • PubMed Central (publishedVersion)
  • HighWire Press Open Archive (submittedVersion)
Funders
U.S. National Library of Medicine (T15LM007092)
Topics
  • Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
  • Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Radiology practices and education
  • Health Sciences › Health Professions › Medical Laboratory Technology › Clinical Laboratory Practices and Quality Control
Keywords
electronic health record integration, closed-loop communication, primary care providers, clinical workflow
MeSH terms
Academic Medical Centers, Aftercare, Continuity of Patient Care, Humans, Monitoring, Physiologic, Radiology, Software, Reminder Systems, Systems Integration, Electronic Health Records, Physicians, Primary Care, Teach-Back Communication

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

A language model drew the claims from the paper on 2026-11-17, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-11-17 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Does integrating nonurgent, clinically significant radiology alerts within the electronic health record impact closed-loop communication and follow-up? https://app.cochat.ai/papers/10.1093/jamia/ocv105
The paper
O’Connor, S. D., Dalal, A. K., Sahni, V. A., Lacson, R., & Khorasani, R. (2016). Does integrating nonurgent, clinically significant radiology alerts within the electronic health record impact closed-loop communication and follow-up? Journal of the American Medical Informatics Association. https://doi.org/10.1093/jamia/ocv105

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.