The DOI matches its Crossref record; 45 of the 46 references it cites were found in a registry or an index; none of the 36 sources its claims cite is checked for support yet.
A Multilevel Primary Care Intervention to Improve Follow-Up of Overdue Abnormal Cancer Screening Test Results: A Cluster Randomized Clinical Trial
Open accessAbstract
Importance: Realizing the benefits of cancer screening requires testing of eligible individuals and processes to ensure follow-up of abnormal results. Objective: To test interventions to improve timely follow-up of overdue abnormal breast, cervical, colorectal, and lung cancer screening results. Design, Setting, and Participants: Pragmatic, cluster randomized clinical trial conducted at 44 primary care practices within 3 health networks in the US enrolling patients with at least 1 abnormal cancer screening test result not yet followed up between August 24, 2020, and December 13, 2021. Intervention: Automated algorithms developed using data from electronic health records (EHRs) recommended follow-up actions and times for abnormal screening results. Primary care practices were randomized in a 1:1:1:1 ratio to (1) usual care, (2) EHR reminders, (3) EHR reminders and outreach (a patient letter was sent at week 2 and a phone call at week 4), or (4) EHR reminders, outreach, and navigation (a patient letter was sent at week 2 and a navigator outreach phone call at week 4). Patients, physicians, and practices were unblinded to treatment assignment. Main Outcomes and Measures: The primary outcome was completion of recommended follow-up within 120 days of study enrollment. The secondary outcomes included completion of recommended follow-up within 240 days of enrollment and completion of recommended follow-up within 120 days and 240 days for specific cancer types and levels of risk. Results: Among 11 980 patients (median age, 60 years [IQR, 52-69 years]; 64.8% were women; 83.3% were White; and 15.4% were insured through Medicaid) with an abnormal cancer screening test result for colorectal cancer (8245 patients [69%]), cervical cancer (2596 patients [22%]), breast cancer (1005 patients [8%]), or lung cancer (134 patients [1%]) and abnormal test results categorized as low risk (6082 patients [51%]), medium risk (3712 patients [31%]), or high risk (2186 patients [18%]), the adjusted proportion who completed recommended follow-up within 120 days was 31.4% in the EHR reminders, outreach, and navigation group (n = 3455), 31.0% in the EHR reminders and outreach group (n = 2569), 22.7% in the EHR reminders group (n = 3254), and 22.9% in the usual care group (n = 2702) (adjusted absolute difference for comparison of EHR reminders, outreach, and navigation group vs usual care, 8.5% [95% CI, 4.8%-12.0%], P < .001). The secondary outcomes showed similar results for completion of recommended follow-up within 240 days and by subgroups for cancer type and level of risk for the abnormal screening result. Conclusions and Relevance: A multilevel primary care intervention that included EHR reminders and patient outreach with or without patient navigation improved timely follow-up of overdue abnormal cancer screening test results for breast, cervical, colorectal, and lung cancer. Trial Registration: ClinicalTrials.gov Identifier: NCT03979495
Overview
CoChat Summary
TL;DR
Among 11,980 patients in 44 primary care practices, a multilevel intervention with EHR reminders plus patient outreach, with or without navigation, improved timely follow-up of overdue abnormal breast, cervical, colorectal, or lung cancer screening results compared with usual care or EHR reminders alone. EHR reminders alone did not improve follow-up.
Key findings
- For the primary outcome, adjusted 120-day follow-up completion was 31.4% with EHR reminders, outreach, and navigation (n = 3455), 31.0% with EHR reminders and outreach (n = 2569), 22.7% with EHR reminders (n = 3254), and 22.9% with usual care (n = 2702); adjusted absolute difference for navigation vs usual care was 8.5% (95% CI, 4.8%-12.0%), P < .001, and for outreach vs usual care was 8.1% (95% CI, 4.5%-11.7%).
- At 240 days, adjusted completion was 43.8% with navigation, 42.6% with outreach, 34.7% with EHR reminders, and 34.1% with usual care; navigation vs usual care adjusted difference was 9.7% (95% CI, 5.4%-14.2%).
- In the as-treated population (n = 8180 after 3800 patients were excluded by manual chart review), 120-day adjusted completion was 38.9%, 33.7%, 23.6%, and 22.6%, respectively; navigation vs usual care adjusted difference was 16.3% (95% CI, 12.3%-20.4%).
- Subgroup analyses showed greater odds of follow-up for cervical and colorectal abnormal results and for low- or medium-risk results; there was no statistically significant interaction by health system (P = .35) or enrollment period (P = .73), and no statistically significant difference for navigation vs outreach or EHR reminders vs usual care.
Methods
- Pragmatic cluster randomized clinical trial at 44 primary care practices in 3 US networks (Brigham and Women’s Hospital, 15 practices; Dartmouth Health, 12; Massachusetts General Hospital, 17). Patients with at least 1 overdue abnormal breast, cervical, colorectal, or lung cancer screening result were enrolled between August 24, 2020, and December 13, 2021; 11,980 patients were included (median age, 60 years [IQR, 52-69]; 64.8% women; 83.3% White; 15.4% Medicaid). Abnormal results were colorectal (8245 [69%]), cervical (2596 [22%]), breast (1005 [8%]), and lung (134 [1%]); risk was low (6082 [51%]), medium (3712 [31%]), and high (2186 [18%]).
- Practices were randomized 1:1:1:1 to usual care, EHR reminders, EHR reminders plus outreach (letter at week 2 and phone call at week 4), or EHR reminders plus outreach plus navigation (additional navigator call after 4 weeks assessing 9 social barrier domains). Automated EHR algorithms recommended follow-up actions and times.
- Primary outcome: completion of recommended follow-up within 120 days of enrollment; secondary outcomes: 240-day completion and 120-/240-day completion by cancer type and risk level. Mixed logistic regression used patient as unit, random effects for practices and physicians, fixed effects for cancer type, calendar month, site, and covariates; adjusted proportions by marginal standardization; intention-to-treat primary and as-treated secondary.
Limitations
- Stated: may not generalize beyond 3 large networks with integrated EHRs and population management infrastructure; local EHR implementation differences; the 3800-patient ITT vs as-treated difference reflects limitations in EHR data or algorithm accuracy; cervical cancer follow-up algorithms are complex and did not reflect most recent guidelines; systems may be reluctant to adapt such data systems.
- Stated: pragmatic unblinded design; intervention did not actively engage specialists and tailoring outreach to risk was not feasible; cost of implementation was not evaluated; entire study occurred during the COVID-19 pandemic.
- Evident from reported results: even the most intensive arm had many patients not completing follow-up (31.4% at 120 days), navigation added no statistically significant benefit over outreach alone, and the lung cancer subgroup was very small (134 patients).
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
0 Valid · 0 Not valid · 18 Not checked · 43 Own results and interpretations
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“adjusted absolute difference for comparison of EHR reminders, outreach, and navigation group vs usual care, 8.5% [95% CI, 4.8%-12.0%], P < .001”
plainn = 3,455- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“adjusted absolute difference for comparison of EHR reminders, outreach, and navigation group vs usual care, 8.5% [95% CI, 4.8%-12.0%], P < .001”
plainstates a causen = 980- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The cumulative follow-up completion proportions through 240 days using Kaplan-Meier estimates for the primary population showed patients in the EHR reminders and outreach group and patients in the EHR reminders, outreach, and navigation group were statistically significantly more likely to complete follow-up during the study period (log-rank test, P < .001; Figure 2).”
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The EHR reminders alone did not improve follow-up proportions and highlight the limited ability of passive reminders among patients who are overdue for follow-up.36”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Among 11 980 patients in 44 primary care practices, completion of follow-up for an abnormal breast, cervical, colorectal, or lung cancer screening test result within 120 days of study enrollment was higher among patients exposed to EHR reminders, outreach, and navigation (31.4%) or EHR reminders and outreach (31.0%) than those exposed to EHR reminders only (22.7%) or usual care (22.9%).”
- Also stated in: § Results · plain
“the adjusted proportion who completed recommended follow-up within 120 days was 31.4% in the EHR reminders, outreach, and navigation group (n = 3455), 31.0% in the EHR reminders and outreach group (n = 2569), 22.7% in the EHR reminders group (n = 3254), and 22.9% in the usual care group (n = 2702)”
plainstates a causen = 11,980- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § Results · plain
19 more claims
Create a free account to see all 61 claims and to verify the 36 references still open.
- It may state two separate claims as one. 25
- The quote only partly states this claim. 2
- Probably not a substantial claim. 19
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Abstract
“or (4) EHR reminders, outreach, and navigation (a patient letter was sent at week 2 and a navigator outreach phone call at week 4).”
§ Methods
“Patients remaining overdue for follow-up after 4 weeks received a reminder phone call by study staff informing them of the test result and providing contact information to schedule a follow-up visit. Patients in the EHR reminders, outreach, and navigation group remaining overdue after an additional 4 weeks received a follow-up call by a patient navigator”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
“Adjusted % b 52.2 46.9 37.3 35.9 18.3 (13.0 to 23.4) 11.0 (5.8 to 16.2) 1.4 (−3.6 to 6.3) 16.9 (11.7 to 22.0) 9.6 (4.5 to 14.8) 7.3 (1.8 to 12.7)”
Reported 18.3 · recomputed 16.3
Citations
Cites
Charlson, M. E., Pompei, P., Ales, K. L., & MacKenzie, C. R. (1987) · Journal of Chronic Diseases · doi:10.1016/0021-9681(87)90171-8 · Cited by 49,742
Reduced Lung-Cancer Mortality with Low-Dose Computed Tomographic Screening
The National Lung Screening Trial Research Team (2011) · New England Journal of Medicine · doi:10.1056/nejmoa1102873 · Cited by 11,064
Screening for Colorectal Cancer: US Preventive Services Task Force Recommendation Statement
US Preventive Services Task Force, Bibbins-Domingo, K., Grossman, D. C., Curry, S. J., Davidson, K. W., Epling, J. W., García, F. A. R., Gillman, M. W., Harper, D. M., Kemper, A. R., et al. (2016) · JAMA · doi:10.1001/jama.2016.5989 · Cited by 2,509
Adenoma detection rate and risk of colorectal cancer and death
Corley, D. A., Jensen, C. D., Marks, A. R., Zhao, W. K., Lee, J. K., Doubeni, C. A., Zauber, A. G., de Boer, J., Fireman, B. H., Schottinger, J. E., et al. (2014) · New England Journal of Medicine · doi:10.1056/nejmoa1309086 · Cited by 2,270
Self-citationScreening for breast cancer: U.S. Preventive Services Task Force recommendation statement
Siu, A. L., & on behalf of the U.S. Preventive Services Task Force (2016) · Annals of Internal Medicine · doi:10.7326/m15-2886 · Cited by 2,094
Screening for Colorectal Cancer: US Preventive Services Task Force Recommendation Statement
US Preventive Services Task Force, Davidson, K. W., Barry, M. J., Mangione, C. M., Cabana, M., Caughey, A. B., Davis, E. M., Donahue, K. E., Doubeni, C. A., Krist, A. H., et al. (2021) · JAMA · doi:10.1001/jama.2021.6238 · Cited by 2,034
Screening for Lung Cancer: US Preventive Services Task Force Recommendation Statement
US Preventive Services Task Force, Krist, A. H., Davidson, K. W., Mangione, C. M., Barry, M. J., Cabana, M., Caughey, A. B., Davis, E. M., Donahue, K. E., Doubeni, C. A., et al. (2021) · JAMA · doi:10.1001/jama.2021.1117 · Cited by 1,816
(2006) · Cancer · doi:10.1002/(issn)1097-0142 · Cited by 1,470
Screening for Cervical Cancer: US Preventive Services Task Force Recommendation Statement
US Preventive Services Task Force, Curry, S. J., Krist, A. H., Owens, D. K., Barry, M. J., Caughey, A. B., Davidson, K. W., Doubeni, C. A., Epling, J. W., Kemper, A. R., et al. (2018) · JAMA · doi:10.1001/jama.2018.10897 · Cited by 1,409
Perkins, R. B., Guido, R. S., Castle, P. E., Chelmow, D., Einstein, M. H., Garcia, F., Huh, W. K., Kim, J. J., Moscicki, A.-B., Nayar, R., et al. (2020) · Journal of Lower Genital Tract Disease · doi:10.1097/lgt.0000000000000525 · Cited by 1,406
Cited by
Ren, W., Liu, Z., Wu, Y., Zhang, Z., Hong, S., Liu, H., & on behalf of the Missing Data in Electronic health Records (MINDER) Group (2024) · Health Data Science · doi:10.34133/hds.0176 · Cited by 76
Patient Navigation Services for Breast and Cervical Cancer Screening and Follow-Up: A Meta-Analysis
Nelson, H. D., Cantor, A. G., Pappas, M., Blackie, K., Yu, Y., & Fu, R. (2025) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2025.1590 · Cited by 40
Atlas, S. J., Haas, J. S., Perez, G. K., Park, E. R., & Peppercorn, J. M. (2024) · JCO Oncology Practice · doi:10.1200/op.23.00818 · Cited by 21
Self-citationThe economics of diagnostic safety
Slawomirski, L., Kelly, D., de Bienassis, K., Kallas, K.-A., & Klazinga, N. (2025) · OECD Health Working Papers · doi:10.1787/fc61057a-en · Cited by 20
Li, C., Zhang, J., Pan, P., Zhang, J., Hou, X., Wang, Y., Chen, G., Muhammad, P., Reis, R. L., Ding, L., et al. (2024) · Advanced Science · doi:10.1002/advs.202400665 · Cited by 12
Metadata
- Authors
- Atlas, S. J., Tosteson, A. N. A., Wright, A., Orav, E. J., Burdick, T. E., Zhao, W., Hort, S. J., Wint, A. J., Smith, R. E., Chang, F. Y., Aman, D. G., Thillaiyapillai, M., Diamond, C. J., Zhou, L., & Haas, J. S.
- Affiliations
- Harvard University (US)
- Dartmouth College (US)
- Vanderbilt University Medical Center (US)
- Brigham and Women's Hospital (US)
- Dartmouth Health (US)
- Columbia University Irving Medical Center (US)
- Corresponding author
- Atlas, Steven J.
- ORCID
- Venue
- JAMA
- Publisher
- American Medical Association (AMA)
- Published
- 2023-10-10
- Volume
- 330
- Issue
- 14
- Pages
- 1348
- ISSN
- 0002-9955 (ISSN-L)
- 0098-7484
- PMC
- PMC10565610
- OpenAlex
- W4387472609
- Cited by
- 54
- References
- 46
- Citations by year
- 2026: 27 · 2025: 17 · 2024: 10
- FWCI
- 6.13
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- JAMA (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Funders
- National Cancer Institute (U01CA225451)
- Topics
- Health Sciences › Medicine › Oncology › Global Cancer Incidence and Screening
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Health Sciences › Medicine › Oncology › Colorectal Cancer Screening and Detection
- Keywords
- abnormal screening results, electronic health record reminders, patient outreach, care navigation, multilevel interventions, primary care practices, breast cancer screening, cervical cancer screening, colorectal cancer screening, lung cancer screening, follow-up completion, health networks, low-income patients, health equity, health information technology, quality improvement, patient engagement
- MeSH terms
- Aftercare, Aged, Female, Humans, Lung Neoplasms, Male, Mass Screening, Middle Aged, Neoplasms, Primary Health Care, Time Factors, United States, Colorectal Neoplasms, Reminder Systems, Early Detection of Cancer, Delayed Diagnosis, Electronic Health Records, Health Communication, Patient Navigation, Pragmatic Clinical Trials as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-11-17, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-11-17 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: A Multilevel Primary Care Intervention to Improve Follow-Up of Overdue Abnormal Cancer Screening Test Results: A Cluster Randomized Clinical Trial. https://app.cochat.ai/papers/10.1001/jama.2023.18755
- The paper
- Atlas, S. J., Tosteson, A. N. A., Wright, A., Orav, E. J., Burdick, T. E., Zhao, W., Hort, S. J., Wint, A. J., Smith, R. E., Chang, F. Y., Aman, D. G., Thillaiyapillai, M., Diamond, C. J., Zhou, L., & Haas, J. S. (2023). A Multilevel Primary Care Intervention to Improve Follow-Up of Overdue Abnormal Cancer Screening Test Results: A Cluster Randomized Clinical Trial. JAMA. https://doi.org/10.1001/jama.2023.18755
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.