The Delphi Technique: Making Sense of Consensus
Open accessThe DOI matches its DataCite record; 40 of the 40 references it cites were found in a registry or an index.
Abstract
The Delphi technique is a widely used and accepted method for gathering data from respondents within their domain of expertise. The technique is designed as a group communication process which aims to achieve a convergence of opinion on a specific real-world issue. The Delphi process has been used in various fields of study such as program planning, needs assessment, policy determination, and resource utilization to develop a full range of alternatives, explore or expose underlying assumptions, as well as correlate judgments on a topic spanning a wide range of disciplines. The Delphi technique is well suited as a method for consensus-building by using a series of questionnaires delivered using multiple iterations to collect data from a panel of selected subjects. Subject selection, time frames for conducting and completing a study, the possibility of low response rates, and unintentionally guiding feedback from the respondent group are areas which should be considered when designing and implementing a Delphi study. Accessed 68,465 times on https://pareonline.net from August 30, 2007 to December 31, 2019. For downloads from January 1, 2020 forward, please click on the PlumX Metrics link to the right.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“factors with a level of agreement ≥ 70% were extracted and included in the present review [19].”
From Healthcare professionals’ perspectives on artificial intelligence in patient care: a systematic review of hindering and facilitating factors on different levels (2025) · cites this paper as [19]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“To narrow the list into a usable checklist, we then used a three-step Delphi survey process [31].”
From PRISMA-S: an extension to the PRISMA Statement for Reporting Literature Searches in Systematic Reviews (2021) · cites this paper as [31]
The cited source says what the claim attributes to it.
In the citing paper
“The iteration characteristics of the Delphi technique have intrinsic limitations and might lead to groupthink, where participants might conform to dominant opinions of the group, which could affect objectivity.197”
From Definition and diagnostic criteria of clinical obesity (2025) · cites this paper as [197]
In the citing paper
“This anonymous and iterative group technique is designed to gather individual opinions from experts and transform these into a group consensus 6.”
From Diagnosis and treatment of chronic bacterial prostatitis and chronic prostatitis/chronic pelvic pain syndrome: a consensus guideline (2015) · cites this paper as [6]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Foundations of Behavioral Research
Stockton, R., & Kerlinger, F. N. (1974) · American Educational Research Journal · doi:10.2307/1162206 · Cited by 7,347
An Experimental Application of the DELPHI Method to the Use of Experts
Dalkey, N., & Helmer, O. (1963) · Management Science · doi:10.1287/mnsc.9.3.458 · Cited by 6,061
Research guidelines for the Delphi survey technique
Hasson, F. (2000) · Journal of Advanced Nursing · doi:10.1046/j.1365-2648.2000.01567.x · Cited by 5,951 · cited 1×
“As such, Ludwig (1994) specifically addresses subject motivation as the key to the successful implementation of a Delphi study and investigators need to play an active role in this area to help ensure as high a response rate as possible.”
The Delphi Method: An Experimental Study of Group Opinion
(1969) · RAND Corporation eBooks · doi:10.7249/rm5888 · Cited by 1,628
Gazing into the oracle : the Delphi method and its application to social policy and public health
Ziglio, E., & Adler, M. E. (1996) · Jessica Kingsley Publishers eBooks · Cited by 1,429
Group Techniques for Program Planning
de Ven, A. V., & Delbecq, A. L. (1975) · Cited by 907
Technological forecasting for decision making
Martino, J. P. (1972) · Cited by 813
Experimental Study of Channel Patterns
SCHUMM, S. A., & KHAN, H. R. (1972) · Geological Society of America Bulletin · doi:10.1130/0016-7606(1972)83[1755:esocp]2.0.co;2 · Cited by 776
An experimental study of group opinion
Dalkey, N. (1969) · Futures · doi:10.1016/s0016-3287(69)80025-x · Cited by 750
Miller, G. (2001) · Tourism Management · doi:10.1016/s0261-5177(00)00067-4 · Cited by 604
Research works printed but not linked: 7
- Adams, S. J. (2001). Projecting the next decade in safety management: A Delphi technique study. Professional Safety, 46 (10), 26-29.
- Green, P. J. (1982, March). The content of a college-level outdoor leadership course. Paper presented at the Conference of the Northwest District Association for the American Alliance for Health, Physical Education, Recreation, and Dance, Spokane, WA.
- Jones, C. G. (1975). A Delphi evaluation of agreement between organizations. In H. A. Linstone, & M. Turoff (Eds.). The Delphi method: Techniques and applications (pp. 160-167). Reading, MA: Addison-Wesley Publishing Company.
- Kaplan, L. M. (1971). The use of the Delphi method in organizational communication: A case study. Unpublished master's thesis, The Ohio State University, Columbus.
- Linstone, H. A., & Turoff, M. (1975). Introduction. In H. A. Linstone, & M. Turoff (Eds.). The Delphi method: Techniques and applications (pp. 3-12). Reading, MA: Addison-Wesley Publishing Company.
- Ludlow, J. (1975). Delphi inquiries and knowledge utilization. In H. A. Linstone, & M. Turoff (Eds.). The Delphi method: Techniques and applications (pp. 102-123). Reading, MA: Addison-Wesley Publishing Company.
- Scheibe, M., Skutsch, M., & Schofer, J. (1975). Experiments in Delphi methodology. In H. A. Linstone, & M. Turoff (Eds.). The Delphi method: Techniques and applications (pp. 262-287). Reading, MA: Addison-Wesley Publishing Company.
Cited by
Items 201–300 of 1,013
Luedtke, K., Boissonnault, W. G., Caspersen, N., Castien, R., Chaibi, A., Falla, D., Fernández‐de‐las‐Peñas, C., Hall, T. M., Hirsvang, J. R., Horre, T., et al. (2016) · Manual Therapy · doi:10.1016/j.math.2016.02.010 · Cited by 120
“is an expert for a Delphi Survey or how many experts are required (Hsu and Sandford, 2007).”
Safety leaders’ perceptions of safety culture in a large Australasian construction organisation
Biggs, S. E., Banks, T. D., Davey, J. D., & Freeman, J. (2012) · Safety Science · doi:10.1016/j.ssci.2012.04.012 · Cited by 154
“” The Delphi technique is a structured method used to gain consensus from a panel of experts [31], [32].”
MethodPotential of large language models in health care: Delphi study
Denecke, K., May, R., LLMHealthGroup, & Rivera Romero, O. (2024) · Journal of Medical Internet Research · doi:10.2196/52399 · Cited by 111
“The Delphi method is widely used to evaluate consensus, or lack thereof, among participants [23].”
MethodHiggins, T. R., Greene, D., & Baker, M. K. (2016) · The Journal of Strength and Conditioning Research · doi:10.1519/jsc.0000000000001559 · Cited by 130
Tartari, E., Tomczyk, S. M., Pires, D., Zayed, B., Rehse, A. P. C., Kariyo, P. C., Stempliuk, V. A., Zingg, W., Pittet, D., & Allegranzi, B. (2020) · Journal of Hospital Infection · doi:10.1016/j.jhin.2020.11.025 · Cited by 128
Edwards, J. G., Clarke, P. T. M., Pieracci, F. M., Bemelman, M., Black, E. A., Doben, A. R., Gasparri, M., Gross, R., Wu, J., Long, W. B., et al. (2019) · The Journal of Trauma: Injury, Infection, and Critical Care · doi:10.1097/ta.0000000000002282 · Cited by 114
Risk Management in Motorway PPP Projects: Empirical-based Guidelines
Carbonara, N., Costantino, N., Gunnigan, L., & Pellegrino, R. (2015) · Transport Reviews · doi:10.1080/01441647.2015.1012696 · Cited by 140
“The Delphi method employs multiple iterations to reach a consensus of opinion concerning a specific topic (Hsu & Sandford, 2007).”
MethodIntegration of Health Protection and Health Promotion: Rationale, Indicators, and Metrics
Sorensen, G., McLellan, D. L., Dennerlein, J. T., Pronk, N. P., Allen, J. D., Boden, L. I., Okechukwu, C. A., Hashimoto, D. M., Stoddard, A., & Wagner, G. R. (2013) · Journal of Occupational and Environmental Medicine · doi:10.1097/jom.0000000000000032 · Cited by 131
Delphi-based consensus study into a framework of community resilience to disaster
Alshehri, S. A., Rezgui, Y., & Li, H. (2014) · Natural Hazards · doi:10.1007/s11069-014-1423-x · Cited by 123
Postoperative ileus: in search of an international consensus on definition, diagnosis, and treatment
Gerö, D., Gié, O., Hübner, M., Demartines, N., & Hahnloser, D. (2016) · Langenbeck s Archives of Surgery · doi:10.1007/s00423-016-1485-1 · Cited by 120
“The Delphi technique is an iterative process used to collect and distill the judgments of a group of experts to reach consensus on specific questions or issues [9].”
MethodA Literature Review on Cloud Computing Adoption Issues in Enterprises
El-Gazzar, R. F. (2014) · IFIP advances in information and communication technology · doi:10.1007/978-3-662-43459-8_14 · Cited by 149
“In this regard, the Delphi method [95–97] would provide insights for IT managers about what are the most important issues, and their priorities, that should be considered when deciding to adopt cloud solutions.”
Simpson, R. C., Thomas, K. S., Leighton, P. A., & Murphy, R. (2013) · British Journal of Dermatology · doi:10.1111/bjd.12334 · Cited by 107
Morgan, R. L., Thayer, K. A., Santesso, N., Holloway, A. C., Blain, R., Eftim, S. E., Goldstone, A. E., Ross, P., Guyatt, G., & Schünemann, H. J. (2018) · Environment International · doi:10.1016/j.envint.2018.08.018 · Cited by 106
Azevedo, S. G., Carvalho, H., & Machado, V. A. C. (2016) · Benchmarking An International Journal · doi:10.1108/bij-07-2014-0072 · Cited by 132
DIET@NET: Best Practice Guidelines for dietary assessment in health research
Cade, J. E., Warthon-Medina, M., Albar, S. A., Alwan, N. A., Ness, A., Roe, M. A., Wark, P. A., Greathead, K., Burley, V. J., Finglas, P. M., et al. (2017) · BMC Medicine · doi:10.1186/s12916-017-0962-x · Cited by 127
Bank adoption of mobile banking: stakeholder perspective
Mullan, J., Bradley, L., & Loane, S. (2017) · International Journal of Bank Marketing · doi:10.1108/ijbm-09-2015-0145 · Cited by 134
Gündüz, M., & Elsherbeny, H. A. (2019) · Journal of Construction Engineering and Management · doi:10.1061/(asce)co.1943-7862.0001768 · Cited by 105
Development of an internationally accepted definition of reablement: a Delphi study
Metzelthin, S., Rostgaard, T., Parsons, M. J. G., & Burton, E. (2020) · Ageing and Society · doi:10.1017/s0144686x20000999 · Cited by 124
“There is no hard cut-off for the level of agreement in Delphi studies (Jorm, 2015), but other researchers have argued previously that 70 per cent is an adequate level of agreement (Hsu and Sandford, 2007; Feo et al., 2018).”
BackgroundDeveloping core outcome set for vitiligo clinical trials: international e‐Delphi consensus
Eleftheriadou, V., Thomas, K., van Geel, N., Hamzavi, I., Lim, H., Suzuki, T., Katayama, I., Anbar, T., Abdallah, M., Benzekri, L., et al. (2015) · Pigment Cell & Melanoma Research · doi:10.1111/pcmr.12354 · Cited by 105
Environmental Sustainability in Sport Facility Management: A Delphi Study
Mallen, C., Adams, L. J., Stevens, J., & Thompson, L. (2010) · European Sport Management Quarterly · doi:10.1080/16184741003774521 · Cited by 132
“This technique has been deemed a widely accepted and useable research method (Hsu & Sandford, 2007; Nielsen & Thangadurai, 2007).”
BackgroundMethodInfluentialCukier, S. N., Lalu, M. M., Bryson, G. L., Cobey, K. D., Grudniewicz, A., & Moher, D. (2020) · BMJ Open · doi:10.1136/bmjopen-2019-035561 · Cited by 119
Management of Acetaminophen Poisoning in the US and Canada: A Consensus Statement
Dart, R. C., Mullins, M. E., Matoushek, T., Ruha, A., Burns, M. M., Simone, K. E., Beuhler, M. C., Heard, K. J., Mazer-Amirshahi, M. E., Stork, C. M., et al. (2023) · JAMA Network Open · doi:10.1001/jamanetworkopen.2023.27739 · Cited by 127
Zhao, P., Ali, Z. M., & Ahmad, Y. (2023) · Sustainable Cities and Society · doi:10.1016/j.scs.2023.104990 · Cited by 126
How to Develop a Second Victim Support Program: A Toolkit for Health Care Organizations
Pratt, S. D., Kenney, L. K., Scott, S. D., & Wu, A. W. (2012) · The Joint Commission Journal on Quality and Patient Safety · doi:10.1016/s1553-7250(12)38030-6 · Cited by 124
Is England closing the international gap in cancer survival?
Walters, S., Benitez-Majano, S., Muller, P., Coleman, M. P., Allemani, C., Butler, J., Peake, M., Guren, M. G., Glimelius, B., Bergström, S., et al. (2015) · British Journal of Cancer · doi:10.1038/bjc.2015.265 · Cited by 103
Exploring consensus on how to measure smoking cessation. A Delphi study
Cheung, K. L., de Ruijter, D., Hiligsmann, M., Elfeddali, I., Hoving, C., Evers, S., & de Vries, H. (2017) · BMC Public Health · doi:10.1186/s12889-017-4902-7 · Cited by 115
“Furthermore, the Delphi methodology is practical, as experts from different parts of the world can be included due to its online character, and experts can complete each questionnaire at their own convenience, mitigating difficulties from non-matching schedules [26].”
BackgroundMethodThe role of traceability in end-to-end circular agri-food supply chains
Anastasiadis, F., Manikas, I., Apostolidou, I., & Wahbeh, S. Y. (2022) · Industrial Marketing Management · doi:10.1016/j.indmarman.2022.04.021 · Cited by 127
Roblek, V., Meško, M., Pušavec, F., & Likar, B. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.592528 · Cited by 151
The Integrated Nutrition Pathway for Acute Care (INPAC): Building consensus with a modified Delphi
Keller, H., McCullough, J., Davidson, B., Vesnaver, E., Laporte, M., Gramlich, L. M., Allard, J. P., Bernier, P., Duerksen, D. R., & Jeejeebhoy, K. N. (2015) · Nutrition Journal · doi:10.1186/s12937-015-0051-y · Cited by 111
Allison, K. H., Reisch, L. M., Carney, P. A., Weaver, D. L., Schnitt, S. J., O'Malley, F. P., Geller, B. M., & Elmore, J. G. (2014) · Histopathology · doi:10.1111/his.12387 · Cited by 101
“), and used a modified Delphi approach to guide the process of reaching agreement.(35) This involved compiling the independent reviews for each case, providing the three experts with their initial interpretations, and a re-review of the slides at a multi-headed microscope followed by a facilitated discussion.”
MethodCanto-Perello, J., Curiel‐Esparza, J., & Calvo, V. (2013) · Expert Systems with Applications · doi:10.1016/j.eswa.2013.02.031 · Cited by 107
O'Donnell, C. M., McLoughlin, L., Patterson, C. C., Clarke, M. J., McCourt, K. C., McBrien, M. E., McAuley, D. F., & Shields, M. O. (2017) · British Journal of Anaesthesia · doi:10.1016/j.bja.2017.09.002 · Cited by 113
An empirical survey on co-simulation: Promising standards, challenges and research needs
Schweiger, G., Gomes, C. Â. G., Engel, G. P., Hafner, I., Schöggl, J., Posch, A., & Nouidui, T. S. (2019) · Simulation Modelling Practice and Theory · doi:10.1016/j.simpat.2019.05.001 · Cited by 132
Madaras-Kelly, K., Jones, M., Remington, R., Hill, N., Huttner, B., & Samore, M. (2014) · Infection Control & Hospital Epidemiology · doi:10.1086/677633 · Cited by 99
The CompHP Core Competencies Framework for Health Promotion in Europe
Barry, M. M., Battel-Kirk, B., & Dempsey, C. (2012) · Health Education & Behavior · doi:10.1177/1090198112465620 · Cited by 114
“…end of the levels for consensus discussed in the literature (B. Green, Jones, Hughes, & Willmas, 1999; P. J. Green, 1982; Hasson et al., 2000; Hsu, 2007; Loughlin & Moore, 1979: National Commission for Health Education Credentialing (NCHEC), Society for Public Health Education (SOPHE), &…”
BackgroundO'Neill, L., Latchford, G., McCracken, L. M., & Graham, C. D. (2019) · Journal of Contextual Behavioral Science · doi:10.1016/j.jcbs.2019.08.008 · Cited by 87
“Choosing suitably qualified participants is crucial to the Delphi process (Hsu & Sandford, 2007).”
BackgroundTerminating Sequential Delphi Survey Data Collection
Kalaian, S. A., & Kasim, R. (2020) · doi:10.7275/g48q-je05 · Cited by 99
Spain, D., & Happé, F. (2020) · Journal of Rational-Emotive & Cognitive-Behavior Therapy · doi:10.1007/s10942-019-00335-1 · Cited by 97
“1 3 conducted frequently in health research, primarily to gather together the views of expert clinicians about poorly understood or under-evidenced topics, with the overall aim of establishing consensus about what best practice ‘should or could’ constitute (Hsu and Sandford 2007).”
BackgroundConsensus on international guidelines for management of groin hernias
the HerniaSurge Group, van Veenendaal, N., Simons, M. P., Hope, W. W., Tumtavitikul, S., & Bonjer, H. J. (2020) · Surgical Endoscopy · doi:10.1007/s00464-020-07516-5 · Cited by 114
Quality indicators for the care and outcomes of adults with atrial fibrillation
Arbelo, E., (Chair), Aktaa, S., Bollmann, A., D’Avila, A., Drossart, I., Dwight, J., Hills, M. T., Hindricks, G., Kusumoto, F. M., et al. (2021) · EP Europace · doi:10.1093/europace/euaa253 · Cited by 97
Chipchase, L. S., Buttrum, P. J., Dunwoodie, R., Hill, A. E., Mandrusiak, A., & Moran, M. (2012) · BMC Medical Education · doi:10.1186/1472-6920-12-112 · Cited by 112
“The anonymity minimised the possibility that a dominant group member or group pressure for conformity may influence the outcome as may occur in a face-to-face meeting [21,23].”
BackgroundMethodRosso, C., Martetschläger, F., Saccomanno, M. F., Voß, A., Lacheta, L., Panel, E. D. C., Ângelo, A. C., Antonogiannakis, E., Azevedo, C. D. C., Bak, K., et al. (2020) · Knee Surgery Sports Traumatology Arthroscopy · doi:10.1007/s00167-020-06286-w · Cited by 97
“A consensus process with an international panel of experienced clinicians using the modified Delphi technique was implemented [1, 13].”
MethodA methodological checklist for fMRI drug cue reactivity studies: development and expert consensus
Ekhtiari, H., Zare-Bidoky, M., Sangchooli, A., Janes, A. C., Kaufman, M. J., Oliver, J. A., Prisciandaro, J. J., Wüstenberg, T., Anton, R. F., Bach, P., et al. (2022) · Nature Protocols · doi:10.1038/s41596-021-00649-4 · Cited by 89
Booth, A., Clarke, M., Ghersi, D., Moher, D., Petticrew, M., & Stewart, L. (2011) · PLoS ONE · doi:10.1371/journal.pone.0027319 · Cited by 96
“The basic principle is for the panel to receive successive questionnaires, each one containing the anonymous responses to the previous round, and for them to modify their responses until a consensus is reached [7,8,9].”
BackgroundMethodThe top ten: best practice recommendations for aphasia
Simmons-Mackie, N., Worrall, L., Murray, L. L., Enderby, P., Rose, M. L., Paek, E. J., & Klippi, A. (2017) · Aphasiology · doi:10.1080/02687038.2016.1180662 · Cited by 96
“Methods were based on Delphi processes (Hasson, Keeney, & McKenna, 2000; Hsu & Sanford, 2007).”
MethodThompson, J. E., Fullerton, J. T., & Sawyer, A. J. (2011) · Midwifery · doi:10.1016/j.midw.2011.04.001 · Cited by 117
“A modified Delphi survey process was used in both rounds (Hsu and Sandford, 2007; Yousuf, 2007; Sinha et al., 2011).”
MethodDefining massive rotator cuff tears: a Delphi consensus study
Schumaier, A. P., Kovacevic, D., Schmidt, C. C., Green, A. J., Rokito, A. S., Jobin, C. M., Yian, E. H., Cuomo, F., Koh, J. L., Gilotra, M. N., et al. (2020) · Journal of Shoulder and Elbow Surgery · doi:10.1016/j.jse.2019.10.024 · Cited by 95
Manca, A., Hortobágyi, T., Carroll, T. J., Enoka, R. M., Farthing, J. P., Gandevia, S. C., Kidgell, D. J., Taylor, J. L., & Deriu, F. (2020) · Sports Medicine · doi:10.1007/s40279-020-01377-7 · Cited by 105
“The Delphi technique is a structured method to elicit opinions on given questions from a group of experts and stakeholders [27], and used increasingly in research, health, and medicine as a tool to address issues and develop consensual guidance on best practice.”
MethodInternational consensus definition of low anterior resection syndrome
Keane, C., Fearnhead, N. S., Bordeianou, L., Christensen, P., Espín, E., Laurberg, S., Mellgren, A. F., Messick, C. A., Orangio, G. R., Verjee, A., et al. (2020) · Colorectal Disease · doi:10.1111/codi.14957 · Cited by 107
Variation in Cardiologists’ Propensity to Test and Treat
Lucas, F. L., Sirovich, B., Gallagher, P. M., Siewers, A. E., & Wennberg, D. E. (2010) · Circulation Cardiovascular Quality and Outcomes · doi:10.1161/circoutcomes.108.840009 · Cited by 94
Ribeiro, L., Pindo, J. C., & Domínguez‐Granda, L. (2016) · The Science of The Total Environment · doi:10.1016/j.scitotenv.2016.09.004 · Cited by 94
“The additional inclusion of a new weighting system inferred from a Delphi panel of Portuguese experts (Hsu and Sandford, 2007) guarantees that the results are more precise in the hydro-environmental characterisation of case studies.”
BackgroundThoma, B., Chan, T. M., Paterson, Q. S., Milne, W. K., Sanders, J. L., & Lin, M. (2015) · Annals of Emergency Medicine · doi:10.1016/j.annemergmed.2015.03.002 · Cited by 98
International Consensus Statement: Spontaneous Cerebrospinal Fluid Rhinorrhea
Georgalas, C., Oostra, A., Ahmed, S. K., Castelnuovo, P., Dallan, I., van Furth, W. R., Harvey, R. J., Herman, P., Kombogiorgas, D., Locatelli, D., et al. (2020) · International Forum of Allergy & Rhinology · doi:10.1002/alr.22704 · Cited by 93
A Systematic Approach to Engineering Ethics Education
Li, J., & Fu, S. (2010) · Science and Engineering Ethics · doi:10.1007/s11948-010-9249-8 · Cited by 121
“It has been widely used and accepted as a method to achieve convergence of opinions concerning real-world knowledge solicited from experts within certain topical areas ( Hsu and Sandford 2007 ).”
MethodDurham, J., Albaghdadi, M., Baad‐Hansen, L., Breckons, M., Goulet, J. P., Lobbezoo, F., List, T., Michelotti, A., Nixdorf, D. R., Peck, C. C., et al. (2016) · Journal of Oral Rehabilitation · doi:10.1111/joor.12448 · Cited by 101
Cantwell, M., Walsh, D. M. J., Furlong, B., Moyna, N. M., McCaffrey, N., Boran, L., Smyth, S., & Woods, C. B. (2017) · European Journal of Cancer Care · doi:10.1111/ecc.12795 · Cited by 103
Cislaghi, B., & Heise, L. (2018) · Health Psychology · doi:10.1037/hea0000618 · Cited by 115
Shou, M., & Doménech, T. (2022) · Journal of Cleaner Production · doi:10.1016/j.jclepro.2022.133557 · Cited by 107
Validating indicators of sheep welfare through a consensus of expert opinion
Phythian, C. J., Michalopoulou, E., Jones, P. H., Winter, A., Clarkson, M. J., Stubbings, L., Grove‐White, D., Cripps, P. J., & Duncan, J. S. (2011) · animal · doi:10.1017/s1751731110002594 · Cited by 92
“However, Delphi questionnaires can be criticised on the grounds of poor response rates and long response times (Hsu and Sanford, 2007).”
BackgroundThe Delphi Technique as a Method to Obtain Consensus in Health Care Education Research
Latif, R. A., Dahlan, A., Mulud, Z. A., & Nor, M. Z. M. (2017) · Education in Medicine Journal · doi:10.21315/eimj2017.9.3.10 · Cited by 121
Mohandes, S. R., & Zhang, X. (2019) · Safety Science · doi:10.1016/j.ssci.2019.02.018 · Cited by 93
Individual social entrepreneurship orientation: towards development of a measurement scale
Satar, M. S., & Saqib, N. (2019) · Asia Pacific Journal of Innovation and Entrepreneurship · doi:10.1108/apjie-09-2018-0052 · Cited by 126
“At the outset a questionnaire was compiled based on extensive literature reviews (Hsu and Sandford, 2007).”
MethodThe international Pediatric Oncology Exercise Guidelines (iPOEG)
Wurz, A. J., McLaughlin, E., Lategan, C., Chamorro-Viña, C., Grimshaw, S. L., Hamari, L., Götte, M., Kesting, S., Rossi, F., van der Torre, P., et al. (2021) · Translational Behavioral Medicine · doi:10.1093/tbm/ibab028 · Cited by 105
“, actively practicing, publishing, and/or working in the field of pediatric exercise oncology; [30]), and snowball sampling.”
BackgroundNorouzian‐Maleki, S., Bell, S., Hosseini, S. B., & Faizi, M. (2014) · Ecological Indicators · doi:10.1016/j.ecolind.2014.07.033 · Cited by 109
“Round 3: Each participant receives a further questionnaire that includes the items and ratings summarized from the previous round and is asked to revise their judgments or “to specify the reasons for remaining outside the consensus” (Hsu and Sandford, 2007, p. 3).”
InfluentialTripathi, V., Stanton, C., Strobino, D. M., & Bartlett, L. (2015) · PLoS ONE · doi:10.1371/journal.pone.0129491 · Cited by 94
Development of the Barriers to Physical Activity Questionnaire for People with Mobility Impairments
Vasudevan, V., Rimmer, J. H., & Kviz, F. J. (2015) · Disability and health journal · doi:10.1016/j.dhjo.2015.04.007 · Cited by 108
Mirzazada, S., Padhani, Z. A., Jabeen, S., Fatima, M., Rizvi, A., Ansari, U., Das, J. K., & Bhutta, Z. A. (2020) · Conflict and Health · doi:10.1186/s13031-020-00285-x · Cited by 89
Hohn, M. M., & Durach, C. F. (2021) · International Journal of Operations & Production Management · doi:10.1108/ijopm-09-2020-0654 · Cited by 109
Perceived unmet needs of informal caregivers of people with dementia in Singapore
Vaingankar, J. A., Subramaniam, M., Picco, L., Eng, G. K., Shafie, S., Sambasivam, R., Zhang, Y. J., Sagayadevan, V., & Chong, S. A. (2013) · International Psychogeriatrics · doi:10.1017/s1041610213001051 · Cited by 89
“Two stage expert panel inputs using a modified Delphi approach (Hsu and Sanford, 2007) were used to (1) obtain professional caregivers’ opinion on the needs identified for the informal caregivers, and (2) identify additional needs to be included in the checklist of informal caregivers’ needs for…”
MethodGobat, N., Kinnersley, P., Gregory, J. W., & Robling, M. R. (2015) · Patient Education and Counseling · doi:10.1016/j.pec.2015.03.024 · Cited by 101
“In a typical Delphi study, the first iteration involves generating ideas that are then refined over subsequent rounds [49,50].”
BackgroundFuzzy delphi method: Issues and challenges
Saffie, N. A. M., Shukor, N. M., & Rasmani, K. A. (2016) · International Conference on Logistics, Informatics and Service Sciences · doi:10.1109/liss.2016.7854490 · Cited by 120
“Nonetheless, it can be observed from the literature that FDM is still open to criticism and some important issues need to be resolved.”
BackgroundInternational Expert Consensus on a Cell Therapy Communication Tool: DOSES
Murray, I. R., Chahla, J. A., Safran, M. R., Krych, A. J., Saris, D. B. F., Caplan, A. I., LaPrade, R. F., & on behalf of the Cell Therapies Communication Expert Group (2019) · Journal of Bone and Joint Surgery · doi:10.2106/jbjs.18.00915 · Cited by 96
Delphi Method: Strengths and Weaknesses
Fink‐Hafner, D., Dagen, T., Doušak, M., Novak, M., & Fink, M. H. (2019) · Cited by 174
Consensus-based standards for best supportive care in clinical trials in advanced cancer
Zafar, S. Y., Currow, D. C., Cherny, N. I., Strasser, F., Fowler, R., & Abernethy, A. P. (2012) · The Lancet Oncology · doi:10.1016/s1470-2045(11)70215-7 · Cited by 89
Technology roadmap for the Creative Industries
Abbasi, M. A., Vassilopoulou, P., & Stergioulas, L. K. (2017) · Creative Industries Journal · doi:10.1080/17510694.2016.1247627 · Cited by 88
Shulman, K. I., Almeida, O. P., Herrmann, N., Schaffer, A., Strejilevich, S. A., Paternoster, C., Amodeo, S., Dols, A., & Sajatovic, M. (2019) · Bipolar Disorders · doi:10.1111/bdi.12714 · Cited by 86
A review of Delphi forecasting research in tourism
Lin, V. S., & Song, H. (2014) · Current Issues in Tourism · doi:10.1080/13683500.2014.967187 · Cited by 108
“Green (1982, cited in Hsu & Sandford, 2007) suggested that for a consensus to be achieved, at least 70% of Delphi subjects needed to rate three or higher on a four-point scale and the median had to be 3.25 or higher.”
BackgroundInfluentialTarget plasma factor levels for personalized treatment in haemophilia: a Delphi consensus statement
Iorio, A., Iserman, E. C., Blanchette, V. S., Dolan, G., Ettingshausen, C. E., Hermans, C. R. J. R., Négrier, C., Oldenburg, J., Reininger, A. J., Rodríguez-Merchán, C. E., et al. (2017) · Haemophilia · doi:10.1111/hae.13215 · Cited by 86
A Delphi analysis on green performance evaluation indices for ports in China
Chen, Z., & Pak, M. (2017) · Maritime Policy & Management · doi:10.1080/03088839.2017.1327726 · Cited by 102
“Ten to fifteen subjects are sufficient if the Delphi study subjects’ backgrounds are homogeneous (Hsu and Sandford 2007).”
BackgroundMiriam Kunz, A., de Waal, M. W. M., Achterberg, W. P., Giménez‐Llort, L., Lobbezoo, F., Sampson, E. L., van Dalen‐Kok, A. H., Defrin, R., Invitto, S., Konstantinović, L. M., et al. (2019) · European Journal of Pain · doi:10.1002/ejp.1477 · Cited by 86
Özen, S., Sağ, E., Ben‐Chetrit, E., Gattorno, M., Gül, A., Hashkes, P. J., Koné‐Paut, I., Lachmann, H. J., Tsitsami, E., Twilt, M., et al. (2020) · Rheumatology · doi:10.1093/rheumatology/keaa863 · Cited by 89
Rezaei, M., Jafari‐Sadeghi, V., Cao, D., & Mahdiraji, H. A. (2021) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2021.120724 · Cited by 93
“…rounds of interviews, with no fear of losing their reputation and credibility, especially when the research moderator informed them of the other experts’ different viewpoints, which sound reasonable (Hjarnø, Syed and Aro, 2007; Hsu and Sandford, 2007; Nielsen and Thangadurai, 2007; Steurer, 2011).”
BackgroundGoldman, J. G., Volpe, D., Ellis, T. D., Hirsch, M. A., Johnson, J. A., Wood, J., Vinagre‐Aragón, A., Biundo, R., Rocco, A. D., Kasman, G. S., et al. (2024) · Journal of Parkinson s Disease · doi:10.3233/jpd-230117 · Cited by 123
Chen, D., Faibil, D., & Agyemang, M. (2020) · Environmental Science and Pollution Research · doi:10.1007/s11356-020-10360-8 · Cited by 93
“Hsu and Sandford (2007) conducted a study applying the Delphi method with six expert’s opinions to evaluate selection criteria for medical waste disposal. dos Muchangos et al. (2015) employed seven experts’ view in analyzing barriers to municipal solid waste.”
MethodDeveloping, implementing and disseminating a core outcome set for neonatal medicine
Webbe, J., Brunton, G., Ali, S., Duffy, J. M., Modi, N., & Gale, C. (2017) · BMJ Paediatrics Open · doi:10.1136/bmjpo-2017-000048 · Cited by 87
“The Delphi process allows for consensus to be reached from a selection of disparate expert opinions.(24) In each round, panel members will be asked to rank the outcomes.”
Screening Strategies for Pulmonary Hypertension in Patients With Interstitial Lung Disease
Rahaghi, F., Kolaitis, N. A., Adegunsoye, A., de Andrade, J. A., Flaherty, K. R., LANCASTER, L. H., Lee, J. S., Levine, D. J., Preston, I. R., Safdar, Z., et al. (2022) · CHEST Journal · doi:10.1016/j.chest.2022.02.012 · Cited by 90
“This version was developed by Delbecq et al(26) in 1975 and is now widely used in medical settings including pulmonology.(27-35) This study’s modified Delphi method used three rounds of surveys, all moderated by three coauthors (F.”
MethodHarris, P. N. A., McNamara, J. F., Lye, D. C., Davis, J. S., Bernard, L., Cheng, A. C., Doi, Y., Fowler, V. G., Kaye, K. S., Leibovici, L., et al. (2016) · Clinical Microbiology and Infection · doi:10.1016/j.cmi.2016.10.023 · Cited by 86
Sustainable supply chain management: Decision models for transformation and maturity
Reefke, H., & Sundaram, D. (2018) · Decision Support Systems · doi:10.1016/j.dss.2018.07.002 · Cited by 96
Brunnhofer, M., Gabriella, N., Schöggl, J.-P., Stern, T., & Posch, A. (2020) · Forest Policy and Economics · doi:10.1016/j.forpol.2019.02.006 · Cited by 82
Delphi method: Strengths and weaknesses
Fink‐Hafner, D., Dagen, T., Doušak, M., Novak, M., & Fink, M. H. (2019) · Advances in Methodology and Statistics · doi:10.51936/fcfm6982 · Cited by 126
“…which occurs in a group process which both distorts the data and deals with group and/or individual interests rather than focusing on problem-solving (Hsu and Sandford, 2007); • through the feedback given in Delphi, an individual expert may enrich his/her insight into empirical factors or…”
BackgroundMethodInfluentialGamification in medical education: identifying and prioritizing key elements through Delphi method
Wang, Y.-F., Hsu, Y.-F., Fang, K., & Kuo, L. (2024) · Medical Education Online · doi:10.1080/10872981.2024.2302231 · Cited by 141
“While the number of panels may vary substantially, most studies using the Delphi technique involve around 15 to 20 experts [34].”
BackgroundLee, J. S. (2019) · ReCALL · doi:10.1017/s0958344019000181 · Cited by 104
Postema, A. W., de Reijke, T. M., Ukimura, O., van den Bos, W., Azzouzi, A.-R., Barret, É., Baumunk, D., Blana, A., Bossi, A., Brausi, M. A., et al. (2016) · World Journal of Urology · doi:10.1007/s00345-016-1782-x · Cited by 81
“The Delphi method is a widely accepted method to achieve consensus among experts and is employed in economics, politics, military decision making and medicine [4].”
Nayahangan, L. J., Stefanidis, D., Kern, D. E., & Konge, L. (2018) · Medical Teacher · doi:10.1080/0142159x.2018.1472756 · Cited by 87
“This is acceptable if information about the topic is available (Hsu and Sandford 2007).”
BackgroundMethodPenm, J. H.-D., Vaillancourt, R., & Pouliot, A. (2018) · Research in Social and Administrative Pharmacy · doi:10.1016/j.sapharm.2018.07.020 · Cited by 81
Consensus Definition of Sport Specialization in Youth Athletes Using a Delphi Approach
Bell, D. R., Snedden, T. R., Biese, K. M., Nelson, E. O., Watson, A. M., Brooks, A., McGuine, T. A., Brown, R., & Kliethermes, S. (2021) · Journal of Athletic Training · doi:10.4085/1062-6050-0725.20 · Cited by 81
Nabbout, R., Auvin, S., Chiron, C., Irwin, J., Mistry, A., Bonner, N., Williamson, N., & Bennett, B. (2018) · Epilepsy & Behavior · doi:10.1016/j.yebeh.2017.08.029 · Cited by 80
“Finally, the Delphi consensus panel technique has received criticism for not adhering to the rigorous scientific standards as other methodologies; however, it has been successfully employed in various fields including gaining consensus, program planning, needs assessment, and exploring underlying assumptions [48].”
Consensus Core Point‐of‐care Ultrasound Applications for Pediatric Emergency Medicine Training
Shefrin, A. E., Warkentine, F. H., Constantine, E., Toney, A. G., Uya, A., Doniger, S. J., Sivitz, A. B., Horowitz, R., & Kessler, D. O. (2019) · AEM Education and Training · doi:10.1002/aet2.10332 · Cited by 80
“This method involves two or more iterations of a data collection instrument, in this case an electronic survey, to a group of stakeholders to reach consensus.(16,17) In designing this study consensus was defined to be reached when ≥80% of respondents agreed on a given item.”
MethodAgüera-Ortíz, L., García-Ramos, R., Pérez, F. J. G., López‐Álvarez, J., Rodríguez, J. M. M., Rodríguez, F. J. O., Pueyo, J. O., Valero, C. P., & Porta-Etessam, J. (2021) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2021.638651 · Cited by 95
“Although no formal sample size calculation is available for Delphi studies, a total number of 30–40 panelists was initially estimated as appropriate according to standard recommended practices (14).”
BackgroundThe impact of adopting lean construction in Egypt: Level of knowledge, application, and benefits
Shaqour, E. N. (2021) · Ain Shams Engineering Journal · doi:10.1016/j.asej.2021.07.005 · Cited by 133
Metadata
- Authors
- Hsu, C.-C., & Sandford, B. A.
- Venue
- University of Massachusetts Amherst
- Publisher
- University of Massachusetts Amherst
- Published
- 2007
- Volume
- 12
- Issue
- 10
- Pages
- 10
- OpenAlex
- W2583504027
- Cited by
- 4,500
- References
- 40
- Citations by year
- 2026: 160 · 2025: 361 · 2024: 381 · 2023: 406 · 2022: 373 · 2021: 381 · 2020: 383 · 2019: 375 · 2018: 358 · 2017: 287 · 2016: 298 · 2015: 240 · 2014: 176 · 2013: 129 · 2012: 91
- FWCI
- 516.01
- Type
- journal article
- Language
- en
- Open access
- Licence
- cc-by-nc-nd
- Locations
- ScholarWorks@UMassAmherst (University of Massachusetts Amherst) (submittedVersion, cc-by-nc-nd)
- University of Massachusetts (UMass) Amherst
- Practical assessment, research & evaluation
- Topics
- Social Sciences › Sociology and Political Science › Delphi Technique in Research
- Social Sciences › Decision Sciences › Management Science and Operations Research › Multi-Criteria Decision Making
- Social Sciences › Sociology and Political Science › Focus Groups and Qualitative Methods
- Keywords
- Delphi method, consensus building, expert opinion, participant selection, response rate, needs assessment, program planning, policy determination
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.