Considering the implication of variations within Delphi research
Open accessThe DOI matches its Crossref record; 27 of the 27 references it cites were found in a registry or an index.
Abstract
Delphi research is an increasingly popular method within medical, social and psychological research. It tends to be employed where established theory or knowledge are lacking but where 'experts' are thought to hold relevant information. It consists of developing and administering sequential questionnaires that seek to move towards a position of relative consensus. Although the original authors of the technique established a specific method, the literature reveals modifications in the way this is applied. Variations include (i) restricting the ability of experts to respond to the original question, (ii) changing or varying the expert groups used and (iii) the point at which the research ends. This paper provides an overview of the technique and explains these variations and their implications before highlighting possible ways forward.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“It uses a process of anonymous and iterative feedback and voting to achieve consensus among a panel of independent experts by means of stepwise refinement of responses.”
From Systematic review: probiotics in the management of lower gastrointestinal symptoms in clinical practice - an evidence-based international guide (2013) · cites this paper as [22]
Citations
Cites
An Experimental Application of the DELPHI Method to the Use of Experts
Dalkey, N., & Helmer, O. (1963) · Management Science · doi:10.1287/mnsc.9.3.458 · Cited by 6,061
Qualitative Research: Consensus methods for medical and health services research
Jones, J., & Hunter, D. (1995) · BMJ · doi:10.1136/bmj.311.7001.376 · Cited by 3,471
Group Techniques for Program Planning: A Guide to Nominal Group and Delphi Processes
(1976) · The Journal of Applied Behavioral Science · doi:10.1177/002188637601200414 · Cited by 2,813
Delphi: a technique to harness expert opinion for critical decision‐making tasks in education
Clayton, M. J. (1997) · Educational Psychology · doi:10.1080/0144341970170401 · Cited by 840
Research Methods in Family Therapy
Tiesel, J. W., Sprenkle, D. H., & Moon, S. M. (1999) · Journal of Marriage and the Family · doi:10.2307/353770 · Cited by 660
Applying Delphi methodology in family therapy research
Jenkins, D. A., & Smith, T. E. (1994) · Contemporary Family Therapy · doi:10.1007/bf02197902 · Cited by 139
Developing a framework of, and quality indicators for, general practice management in Europe
Engels, Y. (2005) · Family Practice · doi:10.1093/fampra/cmi002 · Cited by 123
Fish, L. S., & Busby, D. M. (1996) · Cited by 87
The anticipated future of counselling psychology in the United States: A Delphi poll
Neimeyer, G. J., & Diamond, A. K. (2001) · Counselling Psychology Quarterly · doi:10.1080/09515070110057513 · Cited by 71
Turner, G. H., & Weiner, D. K. (2002) · Pain Medicine · doi:10.1046/j.1526-4637.2002.02030.x · Cited by 62
Research works printed but not linked: 3
- 2001, 14, 49, 10.1080/09515070125262 · doi:10.1080/09515070125262
- 2003, International journal of nursing studies, 40, 855
- 2001, 2, 74
Cited by
Enhancing rigour in the Delphi technique research
Hasson, F., & Keeney, S. (2011) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2011.04.005 · Cited by 760
“Perhaps, therefore a more realistic metaphor of a Delphi is that it merely represents a snapshot of expert opinion [64–66] for that group, at that moment in time.”
BackgroundChipchase, L., Schabrun, S., Cohen, L., Hodges, P., Ridding, M., Rothwell, J., Taylor, J., & Ziemann, U. (2012) · Clinical Neurophysiology · doi:10.1016/j.clinph.2012.05.003 · Cited by 257
“Delphi methodology uses a series of sequential questionnaires (commonly referred to as rounds) interspersed by controlled feedback to gain the most reliable consensus of opinion (Thompson, 2009; Vernon, 2009).”
MethodHungin, A. P. S., Mulligan, C., Pot, B., Whorwell, P., Agréus, L., Fracasso, P., Lionis, C., Mendive, J., Philippart de Foy, J.-M., Rubin, G., et al. (2013) · Alimentary Pharmacology & Therapeutics · doi:10.1111/apt.12460 · Cited by 212
Lange, T., Kopkow, C., Lützner, J., Günther, K.-P., Gravius, S., Scharf, H.-P., Stöve, J., Wagner, R., & Schmitt, J. (2020) · BMC Medical Research Methodology · doi:10.1186/s12874-020-0912-8 · Cited by 184
“Delphi consensus procedures have become widely used in various disciplines of medical research [9, 10], and are commonly used in the development of clinical practice guidelines and quality indicators [6], but also in the development of reporting guidelines [11], criteria for the appropriateness of interventions [8, 12…”
MethodThe Impact of Stress on Paramedic Performance During Simulated Critical Events
LeBlanc, V. R., Regehr, C., Tavares, W., Scott, A. K., MacDonald, R., & King, K. (2012) · Prehospital and Disaster Medicine · doi:10.1017/s1049023x12001021 · Cited by 179
Metadata
- Authors
- Thompson, M.
- Affiliations
- Royal National Hospital for Rheumatic Diseases (GB)
- Corresponding author
- Thompson, M.
- ORCID
- Thompson, M.
- Venue
- Family Practice
- Publisher
- Oxford University Press (OUP)
- Published
- 2009-08-14
- Volume
- 26
- Issue
- 5
- Pages
- 420-424
- ISSN
- 0263-2136 (ISSN-L)
- 1460-2229
- OpenAlex
- W2147017170
- Cited by
- 82
- References
- 27
- Citations by year
- 2026: 7 · 2025: 4 · 2024: 1 · 2023: 9 · 2022: 7 · 2021: 6 · 2020: 2 · 2019: 1 · 2018: 1 · 2017: 6 · 2016: 5 · 2015: 6 · 2014: 5 · 2013: 2 · 2012: 12
- FWCI
- 21.65
- Type
- journal article
- Language
- en
- Open access
- green
- eprints.uwe.ac.uk
- Licence
- cc-by
- Locations
- Family Practice (publishedVersion)
- PubMed (publishedVersion)
- UWE Research Repository (UWE Bristol) (submittedVersion, cc-by)
- HighWire Press Open Archive (submittedVersion)
- Topics
- Social Sciences › Sociology and Political Science › Delphi Technique in Research
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement
- Health Sciences › Nursing › Research and Theory › Health Education and Validation
- Keywords
- Delphi method, methodological variability, expert consensus, expert groups, stopping rules, research methodology
- MeSH terms
- Delphi Technique, Family Practice, Health Services Research, Humans
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.