Delphi studies in social and health sciences—Recommendations for an interdisciplinary standardized reporting (DELPHISTAR). Results of a Delphi study
Open accessThe DOI matches its Crossref record; 19 of the 19 references it cites were found in a registry or an index.
Abstract
Background: While different proposals exist for a guideline on reporting Delphi studies, none of them has yet established itself in the health and social sciences and across the range of Delphi variants. This seems critical because empirical studies demonstrate a diversity of modifications in the conduction of Delphi studies and sometimes even errors in the reporting. The aim of the present study is to close this gap and formulate a general reporting guideline. Method: In an international Delphi procedure, Delphi experts were surveyed online in three rounds to find consensus on a reporting guideline for Delphi studies in the health and social sciences. The respondents were selected via publications of Delphi studies. The preliminary reporting guideline, containing 65 items on five topics and presented for evaluation, had been developed based on a systematic review of the practice of Delphi studies and a systematic review of existing reporting guidelines for Delphi studies. Starting in the second Delphi round, the experts received feedback in the form of mean values, measures of dispersion, a summary of the open-ended responses and their own response in the previous round. The final draft of the reporting guideline contains the items on which at least 75% of the respondents agreed by assigning scale points 6 and 7 on a 7-point Likert scale. Results: 1,072 experts were invited to participate. A total of 91 experts completed the first Delphi round, 69 experts the second round, and 56 experts the third round. Of the 65 items in the first draft of the reporting guideline, consensus was ultimately reached for 38 items addressing the five topics: Title and Abstract (n = 3), Context (n = 7), Method (n = 20), Results (n = 4) and Discussion (n = 4). Items focusing on theoretical research and on dissemination were either rejected or remained subjects of dissent. Discussion: We assume a high level of acceptance and interdisciplinary suitability regarding the reporting guideline presented here and referred to as the "Delphi studies in social and health sciences–recommendations for an interdisciplinary standardized reporting" (DELPHISTAR). Use of this reporting guideline can substantially improve the ability to compare and evaluate Delphi studies.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials
the CONSORT Group, Schulz, K. F., Altman, D. G., & Moher, D. (2010) · BMC Medicine · doi:10.1186/1741-7015-8-18 · Cited by 13,371
Page, M. J., Moher, D., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., et al. (2021) · BMJ · doi:10.1136/bmj.n160 · Cited by 13,262
An Experimental Application of the DELPHI Method to the Use of Experts
Dalkey, N., & Helmer, O. (1963) · Management Science · doi:10.1287/mnsc.9.3.458 · Cited by 6,061
Research guidelines for the Delphi survey technique
Hasson, F. (2000) · Journal of Advanced Nursing · doi:10.1046/j.1365-2648.2000.01567.x · Cited by 5,951
The Delphi method as a research tool: an example, design considerations and applications
Okoli, C., & Pawlowski, S. D. (2004) · Information & Management · doi:10.1016/j.im.2003.11.002 · Cited by 4,771
The Delphi Technique: Making Sense of Consensus
Hsu, C.-C., & Sandford, B. A. (2007) · University of Massachusetts Amherst · doi:10.7275/pdz9-th90 · Cited by 4,500
Diamond, I. R., Grant, R. C., Feldman, B. M., Pencharz, P. B., Ling, S. C., Moore, A. M., & Wales, P. W. (2014) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2013.12.002 · Cited by 3,740
Jünger, S., Payne, S. A., Brine, J., Radbruch, L., & Brearley, S. G. (2017) · Palliative Medicine · doi:10.1177/0269216317690685 · Cited by 2,210
Boulkedid, R., Abdoul, H., Loustau, M., Sibony, O., & Alberti, C. (2011) · PLoS ONE · doi:10.1371/journal.pone.0020476 · Cited by 2,045
Consensus measurement in Delphi studies
von der Gracht, H. A. (2012) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2012.04.013 · Cited by 1,831
Delphi methodology in healthcare research: How to decide its appropriateness
Nasa, P., Jain, R., & Juneja, D. (2021) · World Journal of Methodology · doi:10.5662/wjm.v11.i4.116 · Cited by 1,821
Delphi Technique in Health Sciences: A Map
Niederberger, M., & Spranger, J. (2020) · Frontiers in Public Health · doi:10.3389/fpubh.2020.00457 · Cited by 1,072
Self-citationPage, M. J., Moher, D., Bossuyt, P., Boutron, I., Hoffmann, T., mulrow, C., Shamseer, L., Tetzlaff, J., Akl, E., Brennan, S. E., et al. (2020) · doi:10.31222/osf.io/gwdhk · Cited by 916
Plint, A. C., Moher, D., Morrison, A., Schulz, K., Altman, D. G., Hill, C., & Gaboury, I. (2006) · Medical Journal of Australia · doi:10.5694/j.1326-5377.2006.tb00557.x · Cited by 872
Gattrell, W. T., Logullo, P., van Zuuren, E. J., Price, A., Hughes, E. L., Blazey, P., Winchester, C. C., Tovey, D., Goldman, K., Hungin, A. P., et al. (2024) · PLOS Medicine · doi:10.1371/journal.pmed.1004326 · Cited by 701
Beiderbeck, D., Frevel, N., von der Gracht, H. A., Schmidt, S. L., & Schweitzer, V. M. (2021) · MethodsX · doi:10.1016/j.mex.2021.101401 · Cited by 642
Turner, L., Shamseer, L., Altman, D. G., Schulz, K. F., & Moher, D. (2012) · Systematic Reviews · doi:10.1186/2046-4053-1-60 · Cited by 628
Reporting guidelines for Delphi techniques in health sciences: A methodological review
Spranger, J., Homberg, A., Sonnberger, M., & Niederberger, M. (2022) · Zeitschrift für Evidenz, Fortbildung und Qualität im Gesundheitswesen · doi:10.1016/j.zefq.2022.04.025 · Cited by 262
Self-citation
Cited by
Using the Delphi Method to Establish Expert Consensus: A Practical Guide
Jorm, A. (2025) · Advancing Methods for Interdisciplinarity in the Social Sciences · doi:10.1007/978-981-96-8357-4 · Cited by 22
Determining quality dimensions for peer review reports using a Delphi approach
Sizo, A., Lino, A., Rocha, Á., & Reis, L. P. (2026) · Scientometrics · doi:10.1007/s11192-026-05603-3
Metadata
- Authors
- Niederberger, M., Schifano, J., Deckert, S., Hirt, J., Homberg, A., Köberich, S., Kuhn, R., Rommel, A., Sonnberger, M., & the DEWISS network
- Venue
- PLOS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2024-08-26
- Volume
- 19
- Issue
- 8
- Pages
- e0304651
- ISSN
- 1932-6203 (ISSN-L)
- OpenAlex
- W4401877826
- Cited by
- 220
- References
- 19
- FWCI
- 161.49
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.