Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 45 of the 46 references it cites were found in a registry or an index; none of the 36 sources its claims cite is checked for support yet.

A Multilevel Primary Care Intervention to Improve Follow-Up of Overdue Abnormal Cancer Screening Test Results: A Cluster Randomized Clinical Trial

Open access

Abstract

Importance: Realizing the benefits of cancer screening requires testing of eligible individuals and processes to ensure follow-up of abnormal results. Objective: To test interventions to improve timely follow-up of overdue abnormal breast, cervical, colorectal, and lung cancer screening results. Design, Setting, and Participants: Pragmatic, cluster randomized clinical trial conducted at 44 primary care practices within 3 health networks in the US enrolling patients with at least 1 abnormal cancer screening test result not yet followed up between August 24, 2020, and December 13, 2021. Intervention: Automated algorithms developed using data from electronic health records (EHRs) recommended follow-up actions and times for abnormal screening results. Primary care practices were randomized in a 1:1:1:1 ratio to (1) usual care, (2) EHR reminders, (3) EHR reminders and outreach (a patient letter was sent at week 2 and a phone call at week 4), or (4) EHR reminders, outreach, and navigation (a patient letter was sent at week 2 and a navigator outreach phone call at week 4). Patients, physicians, and practices were unblinded to treatment assignment. Main Outcomes and Measures: The primary outcome was completion of recommended follow-up within 120 days of study enrollment. The secondary outcomes included completion of recommended follow-up within 240 days of enrollment and completion of recommended follow-up within 120 days and 240 days for specific cancer types and levels of risk. Results: Among 11 980 patients (median age, 60 years [IQR, 52-69 years]; 64.8% were women; 83.3% were White; and 15.4% were insured through Medicaid) with an abnormal cancer screening test result for colorectal cancer (8245 patients [69%]), cervical cancer (2596 patients [22%]), breast cancer (1005 patients [8%]), or lung cancer (134 patients [1%]) and abnormal test results categorized as low risk (6082 patients [51%]), medium risk (3712 patients [31%]), or high risk (2186 patients [18%]), the adjusted proportion who completed recommended follow-up within 120 days was 31.4% in the EHR reminders, outreach, and navigation group (n = 3455), 31.0% in the EHR reminders and outreach group (n = 2569), 22.7% in the EHR reminders group (n = 3254), and 22.9% in the usual care group (n = 2702) (adjusted absolute difference for comparison of EHR reminders, outreach, and navigation group vs usual care, 8.5% [95% CI, 4.8%-12.0%], P < .001). The secondary outcomes showed similar results for completion of recommended follow-up within 240 days and by subgroups for cancer type and level of risk for the abnormal screening result. Conclusions and Relevance: A multilevel primary care intervention that included EHR reminders and patient outreach with or without patient navigation improved timely follow-up of overdue abnormal cancer screening test results for breast, cervical, colorectal, and lung cancer. Trial Registration: ClinicalTrials.gov Identifier: NCT03979495

Overview

CoChat Summary

TL;DR

Among 11,980 patients in 44 primary care practices, a multilevel intervention with EHR reminders plus patient outreach, with or without navigation, improved timely follow-up of overdue abnormal breast, cervical, colorectal, or lung cancer screening results compared with usual care or EHR reminders alone. EHR reminders alone did not improve follow-up.

Key findings

  • For the primary outcome, adjusted 120-day follow-up completion was 31.4% with EHR reminders, outreach, and navigation (n = 3455), 31.0% with EHR reminders and outreach (n = 2569), 22.7% with EHR reminders (n = 3254), and 22.9% with usual care (n = 2702); adjusted absolute difference for navigation vs usual care was 8.5% (95% CI, 4.8%-12.0%), P < .001, and for outreach vs usual care was 8.1% (95% CI, 4.5%-11.7%).
  • At 240 days, adjusted completion was 43.8% with navigation, 42.6% with outreach, 34.7% with EHR reminders, and 34.1% with usual care; navigation vs usual care adjusted difference was 9.7% (95% CI, 5.4%-14.2%).
  • In the as-treated population (n = 8180 after 3800 patients were excluded by manual chart review), 120-day adjusted completion was 38.9%, 33.7%, 23.6%, and 22.6%, respectively; navigation vs usual care adjusted difference was 16.3% (95% CI, 12.3%-20.4%).
  • Subgroup analyses showed greater odds of follow-up for cervical and colorectal abnormal results and for low- or medium-risk results; there was no statistically significant interaction by health system (P = .35) or enrollment period (P = .73), and no statistically significant difference for navigation vs outreach or EHR reminders vs usual care.

Methods

  • Pragmatic cluster randomized clinical trial at 44 primary care practices in 3 US networks (Brigham and Women’s Hospital, 15 practices; Dartmouth Health, 12; Massachusetts General Hospital, 17). Patients with at least 1 overdue abnormal breast, cervical, colorectal, or lung cancer screening result were enrolled between August 24, 2020, and December 13, 2021; 11,980 patients were included (median age, 60 years [IQR, 52-69]; 64.8% women; 83.3% White; 15.4% Medicaid). Abnormal results were colorectal (8245 [69%]), cervical (2596 [22%]), breast (1005 [8%]), and lung (134 [1%]); risk was low (6082 [51%]), medium (3712 [31%]), and high (2186 [18%]).
  • Practices were randomized 1:1:1:1 to usual care, EHR reminders, EHR reminders plus outreach (letter at week 2 and phone call at week 4), or EHR reminders plus outreach plus navigation (additional navigator call after 4 weeks assessing 9 social barrier domains). Automated EHR algorithms recommended follow-up actions and times.
  • Primary outcome: completion of recommended follow-up within 120 days of enrollment; secondary outcomes: 240-day completion and 120-/240-day completion by cancer type and risk level. Mixed logistic regression used patient as unit, random effects for practices and physicians, fixed effects for cancer type, calendar month, site, and covariates; adjusted proportions by marginal standardization; intention-to-treat primary and as-treated secondary.

Limitations

  • Stated: may not generalize beyond 3 large networks with integrated EHRs and population management infrastructure; local EHR implementation differences; the 3800-patient ITT vs as-treated difference reflects limitations in EHR data or algorithm accuracy; cervical cancer follow-up algorithms are complex and did not reflect most recent guidelines; systems may be reluctant to adapt such data systems.
  • Stated: pragmatic unblinded design; intervention did not actively engage specialists and tailoring outreach to risk was not feasible; cost of implementation was not evaluated; entire study occurred during the COVID-19 pandemic.
  • Evident from reported results: even the most intensive arm had many patients not completing follow-up (31.4% at 120 days), navigation added no statistically significant benefit over outreach alone, and the lung cancer subgroup was very small (134 patients).

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1001/jama.2023.18755

Cites

45 of 45 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

0 Valid · 0 Not valid · 18 Not checked · 43 Own results and interpretations

19 more claims

Create a free account to see all 61 claims and to verify the 36 references still open.

See all claims — free account
  • It may state two separate claims as one. 25
  • The quote only partly states this claim. 2
  • Probably not a substantial claim. 19

Create a free account to see these claims.

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

2 of 2

Citations

Cites

Show all 46 references

Cited by

See all 54

Metadata

Authors
Atlas, S. J., Tosteson, A. N. A., Wright, A., Orav, E. J., Burdick, T. E., Zhao, W., Hort, S. J., Wint, A. J., Smith, R. E., Chang, F. Y., Aman, D. G., Thillaiyapillai, M., Diamond, C. J., Zhou, L., & Haas, J. S.
Affiliations
  • Harvard University (US)
  • Dartmouth College (US)
  • Vanderbilt University Medical Center (US)
  • Brigham and Women's Hospital (US)
  • Dartmouth Health (US)
  • Columbia University Irving Medical Center (US)
Corresponding author
Atlas, Steven J.
Venue
JAMA
Published
2023-10-10
Volume
330
Issue
14
Pages
1348
ISSN
  • 0002-9955 (ISSN-L)
  • 0098-7484
OpenAlex
W4387472609
Cited by
54
References
46
Citations by year
2026: 27 · 2025: 17 · 2024: 10
FWCI
6.13
Type
journal article
Language
en
Open access
Locations
  • JAMA (publishedVersion)
  • PubMed Central (publishedVersion)
  • PubMed (publishedVersion)
Funders
National Cancer Institute (U01CA225451)
Topics
  • Health Sciences › Medicine › Oncology › Global Cancer Incidence and Screening
  • Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
  • Health Sciences › Medicine › Oncology › Colorectal Cancer Screening and Detection
Keywords
abnormal screening results, electronic health record reminders, patient outreach, care navigation, multilevel interventions, primary care practices, breast cancer screening, cervical cancer screening, colorectal cancer screening, lung cancer screening, follow-up completion, health networks, low-income patients, health equity, health information technology, quality improvement, patient engagement
MeSH terms
Aftercare, Aged, Female, Humans, Lung Neoplasms, Male, Mass Screening, Middle Aged, Neoplasms, Primary Health Care, Time Factors, United States, Colorectal Neoplasms, Reminder Systems, Early Detection of Cancer, Delayed Diagnosis, Electronic Health Records, Health Communication, Patient Navigation, Pragmatic Clinical Trials as Topic

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

A language model drew the claims from the paper on 2026-11-17, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-11-17 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: A Multilevel Primary Care Intervention to Improve Follow-Up of Overdue Abnormal Cancer Screening Test Results: A Cluster Randomized Clinical Trial. https://app.cochat.ai/papers/10.1001/jama.2023.18755
The paper
Atlas, S. J., Tosteson, A. N. A., Wright, A., Orav, E. J., Burdick, T. E., Zhao, W., Hort, S. J., Wint, A. J., Smith, R. E., Chang, F. Y., Aman, D. G., Thillaiyapillai, M., Diamond, C. J., Zhou, L., & Haas, J. S. (2023). A Multilevel Primary Care Intervention to Improve Follow-Up of Overdue Abnormal Cancer Screening Test Results: A Cluster Randomized Clinical Trial. JAMA. https://doi.org/10.1001/jama.2023.18755

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.