CoChat
Sign Up
Cited by 25Suicide and Self-Harm StudiesEmergency and Acute Care StudiesHealthcare Quality and Management

Emergency department safety assessment and follow-up evaluation 2: an implementation trial to improve suicide prevention.

Open access
Boudreaux, Edwin D., Haskins, Brianna L., Larkin, Celine, Pelletier, Lori, Johnson, Sharon A., Stanley, Barbara, Brown, Gregory, Mattocks, Kristin +1 more·2020·Contemporary Clinical Trials
DOI 10.1016/j.cct.2020.106075PMC PMC7484364
Open this paper in CoChat — free account

Abstract

Background: Emergency departments (EDs) are important for preventing suicide. Historically, many patients with suicide risk are not detected during routine clinical care, and those who are often do not receive suicide-specific intervention. The original Emergency Department Safety Assessment and Follow-up Evaluation (ED-SAFE 1) study examined the implementation of universal suicide risk screening and a multi-component ED-initiated suicide prevention intervention. Purpose: The ED-SAFE 2 aims to study the impact of using a continuous quality improvement approach (CQI) to improve suicide related care, with a focus on improving universal suicide risk screening in adult ED patients and evaluating implementation of a new brief intervention called the Safety Planning Intervention (SPI) into routine clinical practice. CQI is a quality management process that uses data and collaboration to drive incremental, iterative improvements. The SPI is a personalized approach that focuses on early identification of warning signs and execution of systematic steps to manage suicidal thoughts. ED-SAFE 2 will provide data on the effectiveness of CQI procedures in improving suicide-related care processes, as well as the impact of these improvements on reducing suicide-related outcomes. Methods: Using a stepped wedge design, eight EDs collected data cross three study phases: Baseline (retrospective), Implementation (12 months), and Maintenance (12 months). Lean methods, a specific approach to pursuing CQI which focuses on increasing value and eliminating waste, were used to evaluate and improve suicide-related care. Conclusions: The results will build upon the success of the ED-SAFE 1 and will have a broad public health impact through promoting better suicide-related care processes and improved suicide prevention.

Integrity signals

Registry

Crossref · 10.1016/j.cct.2020.106075

Cites

31 of 31 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 37 cited sources checked · 0 supported · 0 flagged

  • Premisecitation

    ED-SAFE 1 found that common continuous quality improvement methods made it possible to raise suicide screening to about 85% of adults across eight varied emergency departments.

    “using common continuous quality improvement (CQI) strategies, it was feasible to increase suicide screening to reach, on average, 85% of the adult ED population across eight diverse EDs”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [9] Boudreaux ED Miller I Goldstein AB The Emergency Department Safety Assessment and Follow-up Evaluation (ED-SAFE): Method and design considerations. Contemp Clin Trials. 9 2013; 36(1): 14– 24. DOI: 10.1016/j.cct.2013.05.008. 23707435 PMC3979300 source

      Not checked in the free run.

  • Premisecitation

    In ED-SAFE 1, every site had already put in place clinical protocols to screen all patients for suicide risk using the Patient Safety Screener.

    “During ED-SAFE 1, all eight sites had adopted clinical protocols for universal suicide risk screening using the Patient Safety Screener [7].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [7] Boudreaux ED Camargo CA Jr. Arias SA Improving suicide risk screening and detection in the emergency department. Am J Prev Med. 4 2016; 50(4): 445– 453. DOI: 10.1016/j.amepre.2015.09.029 26654691 PMC4801719 source

      Not checked in the free run.

  • Premisecitation

    That same study also linked the Safety Planning Intervention with follow-up to over twice the odds of using outpatient behavioral health care.

    “and more than doubled the odds of engaging in outpatient behavioral health care”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [11] Joint Commission. (2016). Detecting and treating suicide ideation in all settings. Sentinel Event Alert, 56, 1– 7. 26915165

      Not checked in the free run.

  • Premisecitation

    In ED-SAFE 1, the rise in screening almost doubled risk detection, from 2.9% to 5.7% of adults.

    “such an increase in screening nearly doubled risk detection, rising from 2.9% to 5.7% of adults [7]”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [7] Boudreaux ED Camargo CA Jr. Arias SA Improving suicide risk screening and detection in the emergency department. Am J Prev Med. 4 2016; 50(4): 445– 453. DOI: 10.1016/j.amepre.2015.09.029 26654691 PMC4801719 source

      Not checked in the free run.

  • Premisecitation

    Data from the original ED-SAFE 1, drawing on chart reviews plus clinician and patient interviews, indicate that the baseline safety planning rate is below 25%.

    “original ED-SAFE 1 data based on chart reviews and interviews with clinicians and patients suggests it will be less than 25%.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites ED-SAFE 1

      Not checked in the free run.

  • Classified as probably not a substantial claim. 18
  • May combine two claims. 11
  • The quote only partly states this claim. 1

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingcitation

    All eight original ED-SAFE 1 emergency departments took part, representing diversity in size, demographics, and geography.

    “All eight of the original ED-SAFE 1 EDs participated, representing a diverse array of size, demographics, and geography [10].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [10] Miller IW Camargo CA Jr. Arias SA Suicide prevention in an emergency department population: The ED-SAFE study. JAMA Psychiatry. 2017; 74(6): 563– 570. DOI: 10.1001/jamapsychiatry 28456130 PMC5539839

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    The eight sites were divided into two groups by a median split of high versus low screening performance in ED-SAFE 1.

    “Eight sites were split into two groups based on a median split of high vs. low screening performance in ED-SAFE 1.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    Lean A3 plans from each site were independently rated by Master Lean coaches using quality ratings with descriptive anchors for Needs Improvement, Satisfactory, or Excellent.

    “The Lean A3 plans from each site were rated independently by Master Lean coaches (Pelletier, Johnson) using quality ratings that contained descriptive anchors aligning to “Needs Improvement, Satisfactory, or Excellent.””
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductreasoning

    The cohorts were randomly assigned to a start date (wedge) two months apart.

    “The cohorts were then randomly assigned to a start date (wedge) separated by two months.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Premisecitation

    A stepped wedge design is well suited to effectiveness trials and implementation studies when randomizing individual patients is impractical.

    “The study used a stepped wedge design [17], which is well adapted to effectiveness trials and implementation studies where randomization by patient is impractical.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [17] Hemming K Haines TP Chilton PJ Girling AJ The stepped wedge cluster randomised trial: rationale, design, analysis, and reporting. BMJ. 350 (2015): h391 DOI: 10.1136/bmj.h391 25662947 source

      Not checked in the free run.

  • May combine two claims.

    Study design and conductreasoning

    High- and low-performing sites were randomly matched into two-site cohorts, a step intended to avoid confounding between start order and performance that unbalanced randomization could produce.

    “Each high performing site was randomly paired with a low performing site to create a two-site cohort that prevented confounding of starting order and performance which might occur by random assignment without this balancing”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisecitation

    Historically, many patients at risk for suicide in routine clinical care are not detected, and those detected often do not receive suicide-specific intervention.

    “Historically, many patients with suicide risk are not detected during routine clinical care, and those who are often do not receive suicide-specific intervention.”
    • Also stated in: § BACKGROUND · plain
      “Historically, many at-risk patients are not detected during routine care, and, those who are, often do not receive suicide-specific intervention [7–8].”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [7] Boudreaux ED Camargo CA Jr. Arias SA Improving suicide risk screening and detection in the emergency department. Am J Prev Med. 4 2016; 50(4): 445– 453. DOI: 10.1016/j.amepre.2015.09.029 26654691 PMC4801719 source

      Not checked in the free run.

    • cites [8] Ahmedani BK Simon GE Stewart C et al. Health care contacts in the year before suicide death. J Gen Intern Med. 6 2014; 29(6): 870– 877. DOI: 10.1007/s11606-0142767-3 PMC4026491 24567199 source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    All clinical staff received training to place the safety plan in the medical record so that research assistants could locate it whenever it was completed.

    “All clinical staff were trained to post the safety plan to the medical record to ensure it could be identified by the RAs whenever it was done.”
    Quote in the paperNot a substantial claim

  • The quote only partly states this claim.

    Study design and conductreasoning

    A sample of 1,200 discharged patients (600 per phase) yields 90% power at alpha 0.05 to detect a rise in safety planning from 50% to 65%, corresponding to an odds ratio of 1.86.

    “A logistic regression of the binary response variable (safety plan: yes/no) on the binary predictor variable (treatment: Baseline/Implementation) with a sample size of 1,200 observations of discharged patients (600 Baseline/600 Implementation) achieves 90% power at a 0.05 significance level to detect a change in the probability of receiving a safety plan from the Baseline value of 50% to 65%. This”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductcitation

    Where possible, recommended scale construction principles were applied, including construct definitions, iterative item vetting and revision, piloting, and psychometric checks such as internal consistency for survey scales.

    “Where possible, recommended scale construction principles were used [26], including detailed construct definitions to guide item generation; vetting and revising items iteratively; piloting and revising items; and establishing relevant psychometrics, including internal consistency for survey scales.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [26] Pedhazur EJ Pedhazur-Schmelkin L Measurement, design, and analysis: an integrated approach. New York, NY: Psychology Press, Taylor & Francis Group; 1991.

      Not checked in the free run.

    • Rests on an assumption: No widely accepted standardized measures exist for several constructs in the model, including fidelity to Lean and clinician knowledge, attitudes, and efficacy regarding the Safety Planning Intervention.

  • May combine two claims.

    Study design and conductnone given

    Baseline documented current practices and patient outcomes and served as the control condition.

    “Baseline established the current practices and patient outcomes and acted as the control condition.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductreasoning

    Team Lean competence was further assured by checking the first improvement cycle against Lean standards: an updated A3, results from initial implementation of a change cycle, and at least one issue for a future cycle.

    “Team competence in Lean was further assured by comparing the output of the team’s first improvement cycle against Lean standards, including: (1) an updated A3 documenting the overall problem and goal, (2) results depicting initial implementation of a change cycle, and (3) identification of at least one issue the team would address in a future improvement cycle.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    Lean competence was established by requiring team members to score above 80% on a survey of Lean principles and practices.

    “Competence in Lean was established by requiring that Lean Team members score above an 80% threshold level on a competency survey assessing knowledge of Lean principles and practices.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductcitation

    Questions from existing surveys were adapted for the Lean Team effectiveness survey, which was administered twice, at the end of Implementation and Maintenance.

    “Questions from existing surveys were adapted [27], and the survey was carried out twice, at the end of Implementation and Maintenance phases.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [27] Shortell SM Marsteller JA Lin M The role of perceived team effectiveness in improving chronic illness care. Med Care. 11 2004; 42(11): 1040– 1048. DOI: https://www.jstor.org/stable/4640854 15586830 source

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Study design and conductown data

    The study assessed initial competence and fidelity over time for screening, the Safety Planning Intervention, and Lean.

    “Initial competence and longitudinal fidelity in screening, the SPI, and Lean were determined.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisecitation

    Both manifest risk (presenting with suicidal behavior) and latent risk (presenting with medical complaints but recent suicidal thoughts) are prevalent in emergency departments.

    “where both manifest risk, (individuals presenting with suicidal behavior), and latent risk, (individuals presenting with medical complaints but who have recently thought about suicide), are prevalent [2–6].”
    Quote in the paperSubstantial claim0 of 5 sources checked
    • cites [2] Stanley B Brown GK Safety planning intervention: a brief intervention to mitigate suicide risk. Cogn Behav Pract. 2012; 19: 256– 264. DOI: 10.1016/j.cbpra.2011.01.001 source

      Not checked in the free run.

    • cites [3] Rings JA Alexander PA Silvers VN Gutierrez PM Adapting the safety planning intervention for use in a Veterans’ psychiatric inpatient group setting. J Ment Health Couns. 2012; 34: 95– 109. DOI: 10.17744/mehc.34.2.a77036631424nmq7 source

      Not checked in the free run.

    • cites [4] Meyer H Life in the ‘lean’ lane: performance improvement at Denver Health. Health Aff (Millwood). 11 2010; 29(11): 2054– 2060. DOI: 10.1377/hlthaff.2010.0810 21041748 source

      Not checked in the free run.

    • cites [5] Toussaint J Gerard R On the mend: revolutionizing healthcare to save lives and transform the industry: Lean Enterprise Institute, Inc.; 2010.

      Not checked in the free run.

    • cites [6] Holden RJ Lean thinking in emergency departments: a critical review. Ann Emerg Med. 3 2011; 57(3): 265– 278. DOI: 10.1016/j.annemergmed.2010.08.001 21035904 PMC6548198 source

      Not checked in the free run.

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation

    The Joint Commission recommends universal suicide risk screening in emergency departments because emergency department patients have an overrepresentation of suicide risk.

    “Universal screening for suicide risk in the emergency department entails screening every patient regardless of presenting complaint and is a practice recommended by the Joint Commission [11] due to an overrepresentation of suicide risk in emergency department patients [22–24].”
    Quote in the paperSubstantial claim0 of 4 sources checked
    • cites [11] Joint Commission. (2016). Detecting and treating suicide ideation in all settings. Sentinel Event Alert, 56, 1– 7. 26915165

      Not checked in the free run.

    • cites [22] Ilgen MA Walton MA Cunningham RM Recent suicidal ideation among patients in an inner city emergency department. Suicide Life Threat Behav. 10 2009; 39(5): 508– 517. DOI: 10.1521/suli.2009.39.5.508. 19929151 source

      Not checked in the free run.

    • cites [23] Larkin GL Beautrais AL Emergency departments are underutilized sites for suicide prevention. Crisis. 2010; 31(1): 1– 6. DOI: 10.1027/0227-5910/a000001. 20197251 source

      Not checked in the free run.

    • cites [24] Boudreaux ED Jaques ML Brady KM Matson A Allen MH The patient safety screener: validation of a brief suicide risk screener for emergency department settings. Arch Suicide Res. 2015; 19(2): 151– 160. DOI: 10.1080/13811118.2015.1034604. 25826715 source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductcitation

    All SPI competence ratings used the Safety Plan Intervention Rating Scale (SPIRS), a standardized scale developed to assess competency in the intervention.

    “All ratings were conducted using the Safety Plan Intervention Rating Scale (SPIRS), a standardized rating scale developed to ascertain competency in the intervention [25].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [25] Brown G Stanley B Safety plan intervention rating scale: University of Pennsylvania and Columbia University; unpublished manual.

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    The Safety Planning Intervention has empirical support and can potentially be translated into clinical care.

    “The SPI was chosen because of its empirical support and potential for translation into clinical care [2]”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [2] Stanley B Brown GK Safety planning intervention: a brief intervention to mitigate suicide risk. Cogn Behav Pract. 2012; 19: 256– 264. DOI: 10.1016/j.cbpra.2011.01.001 source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    Standardized data collection procedures were used in all three phases.

    “Standardized data collection procedures were used across all three phases”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    Site trainers with an SPIRS total score of at least 16 were considered competent to provide training.

    “All site trainers who obtained an SPIRS total score of at least 16 were identified as competent to provide training.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Both the Joint Commission and the Suicide Prevention Resource Center recommend the Safety Planning Intervention as a best practice.

    “SPI is a best practice recommended by the Joint Commission [11] and the Suicide Prevention Resource Center.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [11] Joint Commission. (2016). Detecting and treating suicide ideation in all settings. Sentinel Event Alert, 56, 1– 7. 26915165

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Study design and conductreasoning

    Site trainer competence in the Safety Planning Intervention was determined by role playing a standardized patient, with audio-recorded role-plays independently rated by the site's Master SPI Trainer.

    “Competence in the SPI for the site trainer was determined through role playing of a standardized patient, a common method for establishing competence when actual observation or recordings with patients are impossible.”
    • Also stated in: § Intervention components. · plain
      “The role-play was audio-recorded and then independently rated by the site’s Master SPI Trainer.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductcitation

    Research assistants coded safety plans during random chart reviews using a strategy from Master SPI Trainers that rated the six steps as satisfactory.

    “During the random chart reviews, the safety plans were coded by RAs using a strategy developed by Master SPI Trainers that rated the six steps as satisfactory [31].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [31] Brown G Stanley B Coding manual for rating of safety plans in medical records: University of Pennsylvania and Columbia University; unpublished manual.

      Not checked in the free run.

24 of 24

  • The paper relies on this without support.

    No widely accepted standardized measures exist for several constructs in the model, including fidelity to Lean and clinician knowledge, attitudes, and efficacy regarding the Safety Planning Intervention.

    “Unfortunately, there are no widely accepted, standardized measures for several constructs in the model, such as fidelity to Lean and clinician knowledge, attitudes, and efficacy related to the SPI.”
1 of 1

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • VHA intervention study is attributed to the wrong reference

    Minor

    The cited reference [11] is a Joint Commission alert, not the described study of safety planning at VHA emergency departments. The reference list identifies the VHA intervention study as [12].

  • CDC definitions are attributed to an unrelated reference

    Minor

    The text attributes the outcome definitions to the CDC through reference [19], but [19] is a paper on healthcare quality improvement by Kaplan and colleagues.

2 of 2

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Related

Authors
Boudreaux, E. D., Haskins, B. L., Larkin, C., Pelletier, L., Johnson, S. A., Stanley, B., Brown, G., Mattocks, K., & Ma, Y.
Affiliations
  • University of Massachusetts Chan Medical School (US)
  • University of Connecticut (US)
  • Worcester Polytechnic Institute (US)
  • University of Pennsylvania (US)
Corresponding author
Boudreaux, Edwin D.
ORCID
Venue
Contemporary Clinical Trials
Publisher
Elsevier BV
Published
2020-08
Volume
95
Pages
106075
ISSN
1551-7144 (ISSN-L)
DOI
10.1016/j.cct.2020.106075
PMC
PMC7484364
OpenAlex
W3036349697
Semantic Scholar
db00052a7cf9b1ef06c868cf5e29f836d78916fa
Cited by
24
References
31
Citations by year
2026: 1 · 2025: 3 · 2024: 5 · 2023: 3 · 2022: 6 · 2021: 6
FWCI
1.50
Type
journal article
Language
en
Open access
Licence
Locations
  • Contemporary Clinical Trials (publishedVersion)
  • PubMed (publishedVersion)
  • PubMed Central (acceptedVersion)
  • eScholarship@UMassChan (submittedVersion)
Funders
Office of Extramural Research, National Institutes of Health
Topics
  • Social Sciences › Psychology › Clinical Psychology › Suicide and Self-Harm Studies
  • Health Sciences › Medicine › Emergency Medicine › Emergency and Acute Care Studies
  • Health Sciences › Health Professions › Health Information Management › Healthcare Quality and Management
Keywords
suicide prevention, emergency department, suicide risk screening, Safety Planning Intervention, continuous quality improvement, lean methods, implementation, suicide warning signs
MeSH terms
Suicide Prevention, Adult, Emergency Service, Hospital, Follow-Up Studies, Humans, Retrospective Studies, Suicidal Ideation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-06.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-06.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review