CoChat
Sign Up
Cited by 30Suicide and Self-Harm StudiesDelphi Technique in ResearchHealthcare Quality and Management

Prioritizing suicide prevention guideline recommendations in specialist mental healthcare: a Delphi study

Open access
Setkowski, Kim, van Balkom, Anton J. L. M., Dongelmans, Dave A., Gilissen, Renske·2020·BMC Psychiatry
DOI 10.1186/s12888-020-2465-0PMC PMC7007653
Open this paper in CoChat — free accountOpen PDF

Abstract

Background: The Delphi technique is a proven and reliable method to create common definitions and to achieve convergence of opinion. This study aimed to prioritize suicide prevention guideline recommendations and to develop a set of quality indicators (QIs) for suicide prevention in specialist mental healthcare. Methods: This study selected 12 key recommendations from the guideline to modify them into QIs. After feedback from two face-to-face workgroup sessions, 11 recommendations were rephrased and selected to serve as QIs. Next, a Delphi study with the 11 QIs was performed to achieve convergence of opinion among a panel of 90 participants (23 suicide experts, 23 members of patients’ advisory boards or experts with experiences in suicidal behavior and 44 mental healthcare professionals). The participants scored the 11 QIs on two selection criteria: relevance (it affects the number of suicides in the institution) and action orientation (institutions or professionals themselves can influence it) using a 5-point Likert scale. Also, data analysts working in mental healthcare institutions (MHIs) rated each QI on feasibility (is it feasible to monitor and extract from existing systems). Consensus was defined as 70% agreement with priority scores of four or five. Results: Out of the 11 recommendations, participants prioritized five recommendations as relevant and action-oriented in optimizing the quality of care for suicide prevention: 1) screening for suicidal thoughts and behavior, 2) safety plan, 3) early follow-up on discharge, 4) continuity of care and 5) involving family or significant others. Only one of the 11 recommendations early follow-up on discharge reached consensus on all three selection criteria (relevance, action orientation, and feasibility). Conclusions: The prioritization of relevant and action-oriented suicide prevention guideline recommendations is an important step towards the improvement of quality of care in specialist mental healthcare.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1186/s12888-020-2465-0

Cites

12 of 12 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Historically, use of safety plans in clinical practice seems based on clinicians' beliefs about effectiveness rather than empirical evidence.

    “Historically, the use of safety plans in clinical practice seems to be based on clinicians' beliefs about their effectiveness, 14,15 rather than on empirical evidence.16”

    From Safety planning-type interventions for suicide prevention: meta-analysis (2021) · cites this paper as [15]

    The Background (page unavailable) says safety planning did not yet have a substantial body of supporting research, which supports the claim that its historical use was not grounded in strong empirical evidence. But it also says safety planning was informed by research and clinical practice, and the Discussion describes some empirical evidence. The paper does not establish that clinicians’ beliefs, rather than evidence, drove its use; the citing sentence attaches other citations to that part. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

  • Implementation of SPTIs has been largely based on clinicians' beliefs about the value of the interventions.

    “So far, implementation has been largely based on clinicians' beliefs about the value of the interventions, 14,15”

    From Safety planning-type interventions for suicide prevention: meta-analysis (2021) · cites this paper as [15]

    The Results section (Round one: Rating QIs on relevance and action orientation; page unavailable) reports how a mixed panel rated proposed guideline recommendations. The Discussion (Not prioritized guideline recommendations; page unavailable) considers why some received lower ratings. Neither passage says that implementation to date has been largely based on clinicians’ beliefs about the interventions’ value; searches for that assertion and related terms found no such passage. (LLM, conf=0.70)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 28

  1. [4]de Beurs DP de Groot MH de Keijser J van Duijn E de Winter RF Kerkhof AJ Evaluation of benefit to patients of training mental health professionals in suicide guidelines: cluster randomised trial Br J Psychiatry 2016 208 5 477 483 10.1192/bjp.bp.114.156208 26678866 · DOI
  2. [5]Mokkenstorm Jan Franx Gerdien Gilissen Renske Kerkhof Ad Smit Johannes Suicide Prevention Guideline Implementation in Specialist Mental Healthcare Institutions in The Netherlands International Journal of Environmental Research and Public Health 2018 15 5 910 10.3390/ijerph15050910 PMC5981949 29751572 · DOI
  3. [6]Setkowski K, Mokkenstorm J, van Balkom AJLM, Franx G, Verbeek-Van Noord I, Dongelmans DA, et al. Feasibility and impact of data-driven learning within the suicide prevention action network of thirteen specialist mental healthcare institutions (SUPRANET Care) in the Netherlands: A study protocol. BMJ Open. 2018. PMC6104797 30127055
  4. [7]Profit J Gould JB Zupancic JA Stark AR Wall KM Kowalkowski MA Formal selection of measures for a composite index of NICU quality of care: baby-MONITOR J Perinatol 2011 31 11 702 710 10.1038/jp.2011.12 21350429 PMC3205234 · DOI
  5. [8]Chen L Huang LH Xing MY Feng ZX Shao LW Zhang MY Using the Delphi method to develop nursing-sensitive quality indicators for the NICU J Clin Nurs 2017 26 3–4 502 513 10.1111/jocn.13474 27404730 · DOI
  6. [10]van de Glind I Berben S Zeegers F Poppen H Hoogeveen M Bolt I A national research agenda for pre-hospital emergency medical services in the Netherlands: a Delphi-study Scand J Trauma Resusc Emerg Med 2016 24 2 10.1186/s13049-015-0195-y 26746873 PMC4706720 · DOI
  7. [11]Kallen MC Roos-Blom MJ Dongelmans DA Schouten JA Gude WT de Jonge E Development of actionable quality indicators and an action implementation toolbox for appropriate antibiotic use at intensive care units: a modified-RAND Delphi study PLoS One 2018 13 11 e0207991 10.1371/journal.pone.0207991 30496227 PMC6264509 · DOI
  8. [12]Kilbourne AM Keyser D Pincus HA Challenges and opportunities in measuring the quality of mental health care Can J Psychiatr 2010 55 9 549 557 10.1177/070674371005500903 PMC3084488 20840802 · DOI
  9. [13]Zalsman G Hawton K Wasserman D van Heeringen K Arensman E Sarchiapone M Suicide prevention strategies revisited: 10-year systematic review Lancet Psychiatry 2016 3 7 646 659 10.1016/S2215-0366(16)30030-X 27289303 · DOI · Paper page
  10. [16]Wilkinson ST Ballard ED Block MH Mathew SJ Murrough JW Feder A The effect of a single dose of intravenous ketamine on suicidal ideation: a systematic review and individual participant data meta-analysis Am J Pschiatry 2018 175 2 150 158 10.1176/appi.ajp.2017.17040472 PMC5794524 28969441 · DOI · Paper page
  11. [18]Linehan MM Korslund KE Harned MS Gallop RJ Lungu A Neacsiu AD Dialectical behavior therapy for high suicide risk in individuals with borderline personality disorder: a randomized clinical trial and component analysis JAMA Psychiatry 2015 72 5 475 482 10.1001/jamapsychiatry.2014.3039 25806661 · DOI · Paper page
  12. [22]Brent DA, Oquendo MA, Reynolds CF 3rd. Caring for Suicidal Patients. JAMA Psychiatry. 2019. 31188406
  13. [24]Hasson F Keeney S McKenna H Research guidelines for the Delphi survey technique J Adv Nurs 2000 32 4 1008 1015 11095242
  14. [26]Keeney S Hasson F McKenna H Consulting the oracle: ten lessons from using the Delphi technique in nursing research J Adv Nurs 2006 53 2 205 212 10.1111/j.1365-2648.2006.03716.x 16422719 · DOI · Paper page
  15. [27]Ogrinc G Davies L Goodman D Batalden P Davidoff F Stevens D SQUIRE 2.0 (standards for QUality improvement reporting excellence): revised publication guidelines from a detailed consensus process BMJ Qual Saf 2016 25 12 986 992 10.1136/bmjqs-2015-004411 PMC5256233 26369893 · DOI · Paper page
  16. [28]Dewa LH Murray K Thibaut B Ramtale SC Adam S Darzi A Identifying research priorities for patient safety in mental health: an international expert Delphi study BMJ Open 2018 8 3 e021361 10.1136/bmjopen-2017-021361 PMC5855203 29502096 · DOI
  17. [30]Huisman A Robben PB Kerkhof AJ An examination of the Dutch health care Inspectorate's supervision system for suicides of mental health care users Psychiatr Serv 2009 60 1 80 85 10.1176/ps.2009.60.1.80 19114575 · DOI
  18. [32]Chung DT Ryan CJ Hadzi-Pavlovic D Singh SP Stanton C Large MM Suicide rates after discharge from psychiatric facilities: a systematic review and meta-analysis JAMA Psychiatry 2017 74 7 694 702 10.1001/jamapsychiatry.2017.1044 28564699 PMC5710249 · DOI · Paper page
  19. [34]Mokkenstorm Jan K. Kerkhof Ad J.F.M. Smit Johannes H. Beekman Aartjan T.F. Is It Rational to Pursue Zero Suicides Among Patients in Health Care? Suicide and Life-Threatening Behavior 2017 48 6 745 754 10.1111/sltb.12396 29073324 PMC6586166 · DOI
  20. [35]Hogan MF Grumet JG Suicide prevention: an emerging priority for health care Health Aff (Millwood) 2016 35 6 1084 1090 10.1377/hlthaff.2015.1672 27269026 · DOI · Paper page
  21. [37]Lang M Uttaro T Caine E Carpinello S Felton C Implementing routine suicide risk screening for psychiatric outpatients with serious mental disorders: I Qualitative Results Arch Suicide Res 2009 13 2 160 168 10.1080/13811110902835098 19363752 PMC2883878 · DOI
  22. [39]Chan MKY Bhatti H Meader N Stockton S Evans J O'Connor RC Predicting suicide following self-harm: systematic review of risk factors and risk scales Br J Psychiatry 2016 209 4 277 283 10.1192/bjp.bp.115.170050 27340111 · DOI · Paper page
  23. [40]Carter G Milner A McGill K Pirkis J Kapur N Spittal MJ Predicitng suicidal behaviours using clinical instruments: systematic review and meta-analysis of positive predictive values for risk scales Br J Psychiatry 2017 210 6 387 395 10.1192/bjp.bp.116.182717 28302700 · DOI · Paper page
  24. [41]Coffey MJ Coffey CE Ahmedani BK Suicide in a health maintenance organization population JAMA Psychiatry 2015 72 3 294 296 10.1001/jamapsychiatry.2014.2440 25607598 · DOI
  25. [45]Verwey B Don't forget: contributions to the assessment and management of suicide attempters in the general hospital. Doctoral dissertation, University of Leiden. 2006.
  26. [49]Barrero SA Preventing suicide: a resource for the family Ann General Psychiatry 2008 7 1 10.1186/1744-859X-7-1 PMC2246136 18218088 · DOI
  27. [50]Wilkins N Thigpen S Lockman J Mackin J Madden M Perkins T Putting program evaluation to work: a framework for creating actionable knowledge for suicide prevention practice Transl Behav Med 2013 3 2 149 161 10.1007/s13142-012-0175-y 24073166 PMC3717978 · DOI
  28. [51]Trinanes Y Senra-Rivera C Seoane-Pesqueira G Gonzalez-Garcia A Alvarez-Ariza M de-Las-Heras-Linero E Perceived satisfaction and usefulness of suicide prevention information for patients and relatives Rev Calid Asist 2014 29 1 36 42 10.1016/j.cali.2013.09.004 24210519 · DOI

Cited by

Sorted by citations

Authors
Setkowski, K., van Balkom, A. J. L. M., Dongelmans, D. A., & Gilissen, R.
Affiliations
  • 113 Zelfmoordpreventie (NL)
  • Amsterdam UMC Location University of Amsterdam (NL)
Corresponding author
Setkowski, Kim
ORCID
Venue
BMC Psychiatry
Publisher
Springer Science and Business Media LLC
Published
2020-02-07
Volume
20
Issue
1
Pages
55-55
ISSN
1471-244X (ISSN-L)
DOI
10.1186/s12888-020-2465-0
PMC
PMC7007653
OpenAlex
W3005709756
Cited by
30
References
12
Citations by year
2026: 5 · 2025: 4 · 2024: 7 · 2023: 2 · 2022: 3 · 2021: 8 · 2020: 1
FWCI
1.40
Type
journal article
Language
en
Open access
Licence
Locations
  • BMC Psychiatry (publishedVersion, cc-by)
  • Europe PMC (PubMed Central) (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • Pure Amsterdam UMC (publishedVersion, other-oa)
  • Amsterdam UMC (VU Amsterdam) - Institutional Repository (publishedVersion, other-oa)
  • Data Archiving and Networked Services (DANS) (submittedVersion, other-oa)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
ZonMw (537001006)
Topics
  • Social Sciences › Psychology › Clinical Psychology › Suicide and Self-Harm Studies
  • Social Sciences › Sociology and Political Science › Delphi Technique in Research
  • Health Sciences › Health Professions › Health Information Management › Healthcare Quality and Management
Keywords
suicide prevention, quality indicators, guideline recommendations, consensus building, safety planning, continuity of care, family involvement, quality of care
MeSH terms
Suicide Prevention, Delphi Technique, Humans, Mental Health Services, Suicide, Practice Guidelines as Topic, Quality Indicators, Health Care, Consensus

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review