Cited by 2,045Delphi Technique in ResearchHealthcare Quality and ManagementHealth Education and Validation

Using and reporting the Delphi method for selecting healthcare quality indicators: a systematic review

Open access
Boulkedid, Rym, Abdoul, Hendy, Loustau, Marine, Sibony, Olivier, Alberti, Corinne·2011·PLoS ONE

Abstract

OBJECTIVE: Delphi technique is a structured process commonly used to developed healthcare quality indicators, but there is a little recommendation for researchers who wish to use it. This study aimed 1) to describe reporting of the Delphi method to develop quality indicators, 2) to discuss specific methodological skills for quality indicators selection 3) to give guidance about this practice. METHODOLOGY AND MAIN FINDING: Three electronic data bases were searched over a 30 years period (1978-2009). All articles that used the Delphi method to select quality indicators were identified. A standardized data extraction form was developed. Four domains (questionnaire preparation, expert panel, progress of the survey and Delphi results) were assessed. Of 80 included studies, quality of reporting varied significantly between items (9% for year's number of experience of the experts to 98% for the type of Delphi used). Reporting of methodological aspects needed to evaluate the reliability of the survey was insufficient: only 39% (31/80) of studies reported response rates for all rounds, 60% (48/80) that feedback was given between rounds, 77% (62/80) the method used to achieve consensus and 57% (48/80) listed quality indicators selected at the end of the survey. A modified Delphi procedure was used in 49/78 (63%) with a physical meeting of the panel members, usually between Delphi rounds. Median number of panel members was 17(Q1:11; Q3:31). In 40/70 (57%) studies, the panel included multiple stakeholders, who were healthcare professionals in 95% (38/40) of cases. Among 75 studies describing criteria to select quality indicators, 28 (37%) used validity and 17(23%) feasibility. CONCLUSION: The use and reporting of the Delphi method for quality indicators selection need to be improved. We provide some guidance to the investigators to improve the using and reporting of the method in future surveys.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1371/journal.pone.0020476

Cites

54 of 54 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 1

  1. [4]Donabedian A 1980 Explorations in quality assessment and monitoring: the definition of quality and approaches to its assessment Michigan Health Administration press

Cited by

Sorted by citations

Authors
Boulkedid, R., Abdoul, H., Loustau, M., Sibony, O., & Alberti, C.
Affiliations
  • Inserm (FR)
  • Assistance Publique – Hôpitaux de Paris (FR)
Corresponding author
Boulkedid, Rym
Venue
PLoS ONE
Published
2011-06-09
Volume
6
Issue
6
Pages
e20476
ISSN
1932-6203 (ISSN-L)
OpenAlex
W2147072925
Cited by
2,045
References
54
Citations by year
2026: 204 · 2025: 228 · 2024: 227 · 2023: 212 · 2022: 218 · 2021: 220 · 2020: 195 · 2019: 147 · 2018: 121 · 2017: 73 · 2016: 71 · 2015: 48 · 2014: 40 · 2013: 23 · 2012: 15
FWCI
13.63
Type
journal article
Language
en
Open access
Locations
  • PLoS ONE (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • RePEc: Research Papers in Economics (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • Figshare (submittedVersion, cc-by)
Topics
  • Social Sciences › Sociology and Political Science › Delphi Technique in Research
  • Health Sciences › Health Professions › Health Information Management › Healthcare Quality and Management
  • Social Sciences › Education › Health Education and Validation
Keywords
Delphi method, health care quality indicators, reporting quality, expert panel, consensus building, response rate, stakeholder engagement
MeSH terms
Delphi Technique, Humans, Research Design, Guidelines as Topic, Quality Indicators, Health Care

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review