The Delphi Technique: Making Sense of Consensus
Open accessThe DOI matches its DataCite record; 40 of the 40 references it cites were found in a registry or an index.
Abstract
The Delphi technique is a widely used and accepted method for gathering data from respondents within their domain of expertise. The technique is designed as a group communication process which aims to achieve a convergence of opinion on a specific real-world issue. The Delphi process has been used in various fields of study such as program planning, needs assessment, policy determination, and resource utilization to develop a full range of alternatives, explore or expose underlying assumptions, as well as correlate judgments on a topic spanning a wide range of disciplines. The Delphi technique is well suited as a method for consensus-building by using a series of questionnaires delivered using multiple iterations to collect data from a panel of selected subjects. Subject selection, time frames for conducting and completing a study, the possibility of low response rates, and unintentionally guiding feedback from the respondent group are areas which should be considered when designing and implementing a Delphi study. Accessed 68,465 times on https://pareonline.net from August 30, 2007 to December 31, 2019. For downloads from January 1, 2020 forward, please click on the PlumX Metrics link to the right.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“factors with a level of agreement ≥ 70% were extracted and included in the present review [19].”
From Healthcare professionals’ perspectives on artificial intelligence in patient care: a systematic review of hindering and facilitating factors on different levels (2025) · cites this paper as [19]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“To narrow the list into a usable checklist, we then used a three-step Delphi survey process [31].”
From PRISMA-S: an extension to the PRISMA Statement for Reporting Literature Searches in Systematic Reviews (2021) · cites this paper as [31]
The cited source says what the claim attributes to it.
In the citing paper
“The iteration characteristics of the Delphi technique have intrinsic limitations and might lead to groupthink, where participants might conform to dominant opinions of the group, which could affect objectivity.197”
From Definition and diagnostic criteria of clinical obesity (2025) · cites this paper as [197]
In the citing paper
“This anonymous and iterative group technique is designed to gather individual opinions from experts and transform these into a group consensus 6.”
From Diagnosis and treatment of chronic bacterial prostatitis and chronic prostatitis/chronic pelvic pain syndrome: a consensus guideline (2015) · cites this paper as [6]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Foundations of Behavioral Research
Stockton, R., & Kerlinger, F. N. (1974) · American Educational Research Journal · doi:10.2307/1162206 · Cited by 7,347
An Experimental Application of the DELPHI Method to the Use of Experts
Dalkey, N., & Helmer, O. (1963) · Management Science · doi:10.1287/mnsc.9.3.458 · Cited by 6,061
Research guidelines for the Delphi survey technique
Hasson, F. (2000) · Journal of Advanced Nursing · doi:10.1046/j.1365-2648.2000.01567.x · Cited by 5,951 · cited 1×
“As such, Ludwig (1994) specifically addresses subject motivation as the key to the successful implementation of a Delphi study and investigators need to play an active role in this area to help ensure as high a response rate as possible.”
The Delphi Method: An Experimental Study of Group Opinion
(1969) · RAND Corporation eBooks · doi:10.7249/rm5888 · Cited by 1,628
Gazing into the oracle : the Delphi method and its application to social policy and public health
Ziglio, E., & Adler, M. E. (1996) · Jessica Kingsley Publishers eBooks · Cited by 1,429
Group Techniques for Program Planning
de Ven, A. V., & Delbecq, A. L. (1975) · Cited by 907
Technological forecasting for decision making
Martino, J. P. (1972) · Cited by 813
Experimental Study of Channel Patterns
SCHUMM, S. A., & KHAN, H. R. (1972) · Geological Society of America Bulletin · doi:10.1130/0016-7606(1972)83[1755:esocp]2.0.co;2 · Cited by 776
An experimental study of group opinion
Dalkey, N. (1969) · Futures · doi:10.1016/s0016-3287(69)80025-x · Cited by 750
Miller, G. (2001) · Tourism Management · doi:10.1016/s0261-5177(00)00067-4 · Cited by 604
Research works printed but not linked: 7
- Adams, S. J. (2001). Projecting the next decade in safety management: A Delphi technique study. Professional Safety, 46 (10), 26-29.
- Green, P. J. (1982, March). The content of a college-level outdoor leadership course. Paper presented at the Conference of the Northwest District Association for the American Alliance for Health, Physical Education, Recreation, and Dance, Spokane, WA.
- Jones, C. G. (1975). A Delphi evaluation of agreement between organizations. In H. A. Linstone, & M. Turoff (Eds.). The Delphi method: Techniques and applications (pp. 160-167). Reading, MA: Addison-Wesley Publishing Company.
- Kaplan, L. M. (1971). The use of the Delphi method in organizational communication: A case study. Unpublished master's thesis, The Ohio State University, Columbus.
- Linstone, H. A., & Turoff, M. (1975). Introduction. In H. A. Linstone, & M. Turoff (Eds.). The Delphi method: Techniques and applications (pp. 3-12). Reading, MA: Addison-Wesley Publishing Company.
- Ludlow, J. (1975). Delphi inquiries and knowledge utilization. In H. A. Linstone, & M. Turoff (Eds.). The Delphi method: Techniques and applications (pp. 102-123). Reading, MA: Addison-Wesley Publishing Company.
- Scheibe, M., Skutsch, M., & Schofer, J. (1975). Experiments in Delphi methodology. In H. A. Linstone, & M. Turoff (Eds.). The Delphi method: Techniques and applications (pp. 262-287). Reading, MA: Addison-Wesley Publishing Company.
Cited by
Items 101–200 of 1,013
Jousse-Joulin, S., D'Agostino, M. A., Nicolas, C., Naredo, E., Ohrndorf, S., Backhaus, M., Tamborrini, G., Chary-Valckenaere, I., Terslev, L., Iagnocco, A., et al. (2019) · Annals of the Rheumatic Diseases · doi:10.1136/annrheumdis-2019-215024 · Cited by 204
Eco-innovation pathways to a circular economy: Envisioning priorities through a Delphi approach
de Jesus, A., Antunes, P., Santos, R., & Mendonça, S. (2019) · Journal of Cleaner Production · doi:10.1016/j.jclepro.2019.04.049 · Cited by 204
Identifying a core set of science teaching practices: A delphi expert panel approach
Kloser, M. (2014) · Journal of Research in Science Teaching · doi:10.1002/tea.21171 · Cited by 204
Hung, H.-L., Altschuld, J. W., & Lee, Y.-F. (2008) · Evaluation and Program Planning · doi:10.1016/j.evalprogplan.2008.02.005 · Cited by 205
“Strengths and limitations have also been discussed extensively (Eggers & Jones, 1998; Franklin & Hart, 2007; Gordon, 1994; Hartman, 1981; Hsu & Sandford, 2007; Lang, 1994; Linstone & Turoff, 1975; Mitchell, 1991; Powell, 2003; Price, 2005; Williams & Webb, 1994; Yousuf, 2007).”
GPT models in construction industry: Opportunities, limitations, and a use case validation
Saka, A., Taiwo, R., Saka, N., Salami, B. A., Ajayi, S., Akande, K., & Kazemi, H. (2024) · Developments in the Built Environment · doi:10.1016/j.dibe.2023.100300 · Cited by 202
Barriers to big data analytics in manufacturing supply chains: A case study from Bangladesh
Moktadir, M. A., Ali, S. M., Paul, S. K., & Shukla, N. (2019) · Computers & Industrial Engineering · doi:10.1016/j.cie.2018.04.013 · Cited by 198
“It is a very dynamic method for assessing data in which experts/evaluators share their practical experiences to reach a convergence of opinions (Gordon, 2009; Hsu & Sandford, 2007; Pawlowski & Okoli, 2004).”
‘More of an art than a science’? The development, design and mechanics of the Delphi Technique
Drumm, S., Bradley, C., & Moriarty, F. (2022) · Research in Social and Administrative Pharmacy · doi:10.1016/j.sapharm.2021.06.027 · Cited by 195
Social entrepreneurship orientation: development of a measurement scale
Kraus, S., Niemand, T., Halberstadt, J., Shaw, E., & Syrjä, P. (2017) · International Journal of Entrepreneurial Behavior & Research · doi:10.1108/ijebr-07-2016-0206 · Cited by 195
Loblaw, D. A., Prestrud, A. A., Somerfield, M. R., Oliver, T. K., Brouwers, M. C., Nam, R. K., Lyman, G. H., & Basch, E. (2012) · Journal of Clinical Oncology · doi:10.1200/jco.2012.42.0489 · Cited by 189
“In addition, quantitative data can be collected, allowing for the application and reporting of statistical analyses.(16) The strategy lends itself to explicit reporting,(17) which is critical for guideline development.”
Caumo, W., Antunes, L., Lorenzzi Elkfury, J., Herbstrith, E., Busanello Sipmann, R., Souza, A., Torres, I., Souza dos Santos, V., & Neblett, R. (2017) · Journal of Pain Research · doi:10.2147/jpr.s131479 · Cited by 188
Expert Opinion Is Necessary: Delphi Panel Methodology Facilitates a Scientific Approach to Consensus
Hohmann, E., Brand, J. C., Rossi, M. J., & Lubowitz, J. H. (2018) · Arthroscopy: The Journal of Arthroscopic & Related Surgery · doi:10.1016/j.arthro.2017.11.022 · Cited by 186
Competency Definitions, Development and Assessment: A Brief Review
Wong, S.-C. (2020) · International Journal of Academic Research in Progressive Education and Development · doi:10.6007/ijarped/v9-i3/8223 · Cited by 186
“The Delphi process can be concluded if key ideas are established (Hsu & Sandford, 2007; Okoli & Pawlowski, 2004).”
BackgroundTowards a definition of neurodisability: a Delphi survey
Morris, C., Janssens, A., Tomlinson, R., Williams, J., & Logan, S. (2013) · Developmental Medicine & Child Neurology · doi:10.1111/dmcn.12218 · Cited by 185
Kirkpatrick, S. I., Baranowski, T., Subar, A. F., Tooze, J. A., & Frongillo, E. A. (2019) · Journal of the Academy of Nutrition and Dietetics · doi:10.1016/j.jand.2019.06.010 · Cited by 183
Gillespie, A. M., Obregon, R., El Asawi, R., Richey, C., Manoncourt, E., Joshi, K., Naqvi, S., Pouye, A., Safi, N., Chitnis, K., et al. (2016) · Global Health: Science and Practice · doi:10.9745/ghsp-d-16-00226 · Cited by 184
members, T. F. A. P., Kuhn-Thiel, A. M., Weiß, C., & Wehling, M. (2014) · Drugs & Aging · doi:10.1007/s40266-013-0146-0 · Cited by 184
“Statistical analysis based on Round 1 input was performed as follows: the Likert scale is often favoured for consensus procedures, as well as means, median and mode (‘central tendency’ indicators) [26, 27].”
MethodVEILLARD, J., COWLING, K., BITTON, A., RATCLIFFE, H., KIMBALL, M., BARKLEY, S., MERCEREAU, L., WONG, E., TAYLOR, C., HIRSCHHORN, L. R., et al. (2017) · The Milbank Quarterly · doi:10.1111/1468-0009.12301 · Cited by 183
The Delphi Method Research Strategy in Studies of Information Systems
Skinner, R., Nelson, R. R., Chin, W. W., & Land, L. (2015) · Communications of the Association for Information Systems · doi:10.17705/1cais.03702 · Cited by 180
Hurst, J. R., Verma, N., Lowe, D., Baxendale, H. E., Jolles, S., Kelleher, P., Longhurst, H. J., Patel, S. Y., Renzoni, E. A., Sander, C. R., et al. (2017) · The Journal of Allergy and Clinical Immunology: In Practice · doi:10.1016/j.jaip.2017.01.021 · Cited by 180
Consensus Based Definition of Growth Restriction in the Newborn
Beune, I. M., Bloomfield, F. H., Ganzevoort, W., Embleton, N. D., Rozance, P. J., van Wassenaer-Leemhuis, A. G., Wynia, K., & Gordijn, S. J. (2018) · The Journal of Pediatrics · doi:10.1016/j.jpeds.2017.12.059 · Cited by 180
Farrell, B., Tsang, C., Raman-Wilms, L., Irving, H., Conklin, J., & Pottie, K. (2015) · PLOS ONE · doi:10.1371/journal.pone.0122246 · Cited by 178
Terlouw, L. G., Moelker, A., Abrahamsen, J., Acosta, S., Bakker, O. J., Baumgartner, I., Boyer, L., Corcos, O., van Dijk, L. J., Duran, M., et al. (2020) · United European Gastroenterology Journal · doi:10.1177/2050640620916681 · Cited by 178
Syeed, M. M. M., Hossain, M. S., Karim, M. R., Uddin, M. F., Hasan, M., & Khan, R. H. (2023) · Environmental and Sustainability Indicators · doi:10.1016/j.indic.2023.100247 · Cited by 174
Defining Physical Literacy for Application in Australia: A Modified Delphi Method
Keegan, R. J., Barnett, L. M., Dudley, D. A., Telford, R. D., Lubans, D. R., Bryant, A. S., Roberts, W. M., Morgan, P. J., Schranz, N. K., Weissensteiner, J. R., et al. (2019) · Journal of Teaching in Physical Education · doi:10.1123/jtpe.2018-0264 · Cited by 173
“Consensus from a Delphi process should not be taken to mean that a ‘correct’ answer has necessarily been found, but rather that experts have been engaged in seeking a convergence of opinion and state-of-the-art knowledge (Hsu & Sandford, 2007; Keeney et al., 2011).”
BackgroundMethodInfluentialThomas, S. L., Wakerman, J., & Humphreys, J. S. (2015) · International Journal for Equity in Health · doi:10.1186/s12939-015-0228-1 · Cited by 175
“Resulting anonymity prevented dominance by any individual and allowed all opinions to be considered [21].”
BackgroundJolivet, R. R., Moran, A. C., O’Connor, M., Chou, D., Bhardwaj, N., Newby, H., Requejo, J., Schaaf, M., Say, L., & Langer, A. (2018) · BMC Pregnancy and Childbirth · doi:10.1186/s12884-018-1763-8 · Cited by 175
“steering committee decided on an ideal panel size of 1015 individuals based on the literature regarding suggested Delphi method group size [20, 21].”
MethodUnlocking the drivers of big data analytics value in firms
Côrte-Real, N., Ruivo, P., Oliveira, T., & Popovič, A. (2019) · Journal of Business Research · doi:10.1016/j.jbusres.2018.12.072 · Cited by 175
Expert oriented approach for analyzing the blockchain adoption barriers in humanitarian supply chain
Sahebi, I. G., Masoomi, B., & Ghorbani, S. (2020) · Technology in Society · doi:10.1016/j.techsoc.2020.101427 · Cited by 175
A principal component analysis is conducted for a case series quality appraisal checklist
Guo, B., Moga, C., Harstall, C., & Schopflocher, D. (2016) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2015.07.010 · Cited by 174
“One of the limitations of this study is the small number (seven) of Delphi panelists; in general, 10 to 15 panelists are considered sufficient for a Delphi study that is focused and the background of the Delphi participants is homogenous [29].”
BackgroundDelphi consensus recommendations for a treatment algorithm in pulmonary sarcoidosis
Rahaghi, F. F., Baughman, R. P., Saketkoo, L. A., Sweiss, N. J., Barney, J. B., Birring, S. S., Costabel, U., Crouser, E. D., Drent, M., Gerke, A. K., et al. (2020) · European Respiratory Review · doi:10.1183/16000617.0146-2019 · Cited by 167
Rosenthal, R., Hoffmann, H., Clavien, P., Bucher, H. C., & Dell‐Kuster, S. (2015) · World Journal of Surgery · doi:10.1007/s00268-015-3003-y · Cited by 162
“Since there is abundant literature on the classification of postoperative complications, we refrained from a first round with open question [13].”
Büyüközkan, G., Mukul, E., & Kongar, E. (2021) · Socio-Economic Planning Sciences · doi:10.1016/j.seps.2020.100929 · Cited by 165
De Sanctis, V., Bossi, P., Sanguineti, G., Trippa, F., Ferrari, D., Bacigalupo, A., Ripamonti, C. I., Buglione, M., Pergolizzi, S., Langendjik, J. A., et al. (2016) · Critical Reviews in Oncology/Hematology · doi:10.1016/j.critrevonc.2016.01.010 · Cited by 163
354 Focal therapy in prostate cancer – international multidisciplinary consensus on trial design
Van Den Bos, W., Muller, B. G., Ahmed, H., Bangma, C. H., Barret, E., Crouzet, S., Eggener, S. E., Gill, I. S., Joniau, S., Kovacs, G., et al. (2014) · European Urology Supplements · doi:10.1016/s1569-9056(14)60349-3 · Cited by 164
Manyara, A. M., Purvis, A., Ciani, O., Collins, G. S., & Taylor, R. S. (2024) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2024.111485 · Cited by 163
“It has been used to build consensus in various exercises such as needs assessment, policy determination, estimation of disease prevalence, and development of clinical and health reporting guidelines [6–9].”
Dubé, E., Gagnon, D., Ouakki, M., Bettinger, J. A., Guay, M., Halperin, S., Wilson, K., Graham, J., Witteman, H. O., MacDonald, S., et al. (2016) · PLOS ONE · doi:10.1371/journal.pone.0156118 · Cited by 163
“Our study used two rounds of stakeholder questionnaires in an approach based on the Delphi method, which is well suited for consensus-building [22, 23].”
MethodVolkmar, G., Fischer, P. M., & Reinecke, S. (2022) · Journal of Business Research · doi:10.1016/j.jbusres.2022.04.007 · Cited by 163
Defining and identifying concepts of medication literacy: An international perspective
Pouliot, A., Vaillancourt, R., Stacey, D., & Suter, P. (2018) · Research in Social and Administrative Pharmacy · doi:10.1016/j.sapharm.2017.11.005 · Cited by 160
Development of a List of High-Risk Operations for Patients 65 Years and Older
Schwarze, M. L., Barnato, A. E., Rathouz, P. J., Zhao, Q., Neuman, H. B., Winslow, E. R., Kennedy, G. D., Hu, Y.-Y., Dodgion, C. M., Kwok, A. C., et al. (2015) · JAMA Surgery · doi:10.1001/jamasurg.2014.1819 · Cited by 159
EL MÉTODO DELPHI EN LA INVESTIGACIÓN ACTUAL EN EDUCACIÓN: UNA REVISIÓN TEÓRICA Y METODOLÓGICA
López-Gómez, E. (2017) · Educación XX1 · doi:10.5944/educxx1.20169 · Cited by 159
Research Pearls: Expert Consensus Based Evidence Using the Delphi Method
Hohmann, E., Cote, M. P., & Brand, J. C. (2018) · Arthroscopy: The Journal of Arthroscopic & Related Surgery · doi:10.1016/j.arthro.2018.10.004 · Cited by 158
Capturing the benefits of industry 4.0: a business network perspective
Schroeder, A., Ziaee Bigdeli, A., Galera Zarco, C., & Baines, T. (2019) · Production Planning & Control · doi:10.1080/09537287.2019.1612111 · Cited by 159
“The Delphi method involves the inherent risk of creating specific topic-related information instead of consolidated generalizable insights (Hsu and Sandford 2007).”
BackgroundMethodInfluentialBarrena-Martínez, J., López-Fernández, M., & Romero-Fernández, P. M. (2019) · The International Journal of Human Resource Management · doi:10.1080/09585192.2017.1332669 · Cited by 155
Thorne, J. E., Jabs, D. A., Belfort, R., Dick, A. D., Gangaputra, S. S., Nussenblatt, R., Okada, A. A., Rosenbaum, J. T., Trusko, B. E., & for The Standardization of Uveitis Nomenclature (SUN) Project (2013) · Methods of Information in Medicine · doi:10.3414/me12-01-0063 · Cited by 172
Al-araibi, A. A. M., Mahrin, M. N. B., & Yusoff, R. C. M. (2019) · Education and Information Technologies · doi:10.1007/s10639-018-9780-9 · Cited by 152
“Thus, it enables the study to be conducted efficiently and effectively (Hsu and Sandford 2007).”
BackgroundMethodCavernous angiomas: deconstructing a neurosurgical disease
Awad, I. A., & Polster, S. P. (2019) · Journal of neurosurgery · doi:10.3171/2019.3.jns181724 · Cited by 150
Laupichler, M. C., Aster, A., & Raupach, T. (2023) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2023.100126 · Cited by 149
“…both a common and an acceptable modification of the Delphi process format to use a structured questionnaire in Round 1 that is based upon an extensive review of the literature ” (Hsu, 2007, p. 2), we wanted to ensure that the preliminary themes identified reflected the most important AI constructs.”
MethodA checklist to assess the quality of survey studies in psychology
Protogerou, C., & Hagger, M. S. (2020) · Methods in Psychology · doi:10.1016/j.metip.2020.100031 · Cited by 146
Development and Evaluation of the Dutch Clinician-Administered PTSD Scale for DSM-5 (CAPS-5)
Boeschoten, M. A., van der Aa, N., Bakker, A., ter Heide, F. J. J., Hoofwijk, M. C., Jongedijk, R. A., van Minnen, A., Elzinga, B. M., & Olff, M. (2018) · European Journal of Psychotraumatology · doi:10.1080/20008198.2018.1546085 · Cited by 168
“…crowd of Dutch psychotrauma professionals (clinicians as well as researchers) and was based on the principles of a cross-cultural adaptation process (Beaton et al., 2000) as well as the Delphi method (Dalkey, 1969; Dalkey & Helmer, 1963; Hsu & Sandford, 2007) for consensusbuilding by expert-rounds.”
MethodDigital Marketing Actions That Achieve a Better Attraction and Loyalty of Users: An Analytical Study
García, J. L. L., Lizcano, D., Ramos, C., & Matos, N. (2019) · Future Internet · doi:10.3390/fi11060130 · Cited by 134
Fullerton, J. T., Thompson, J. E., & Severino, R. (2011) · Midwifery · doi:10.1016/j.midw.2011.03.005 · Cited by 172
“A modified Delphi survey process was used in both phases (Hsu and Sandford, 2007; Yousuf, 2007).”
MethodPowell, B. J., Stanick, C. F., Halko, H. M., Dorsey, C. N., Weiner, B. J., Barwick, M. A., Damschroder, L. J., Wensing, M., Wolfenden, L., & Lewis, C. C. (2017) · Implementation Science · doi:10.1186/s13012-017-0649-x · Cited by 144
“Next steps will include developing consensus on the relative priorities for these categories and criteria through a Delphi [18] study; developing rating criteria with concrete, measurable anchors; and assessing inter-rater reliability and known-groups validity of the criteria [3].”
Values and behaviors of effective lean managers: Mixed-methods exploratory research
van Dun, D. H., Hicks, J., & Wilderom, C. P. M. (2016) · European Management Journal · doi:10.1016/j.emj.2016.05.001 · Cited by 203
“In order to refine and potentially supplement the literaturebased list of values and behaviors associated with effective lean managers, a Delphi study, a “method for consensus-building” among a group of experts (Hsu & Sandford, 2007, p. 1), was conducted.”
BackgroundMethodConsensus criteria for the diagnosis of scabies: A Delphi study of international experts
Engelman, D., Fuller, L. C., Steer, A. C., & for the International Alliance for the Control of Scabies Delphi panel (2018) · PLOS Neglected Tropical Diseases · doi:10.1371/journal.pntd.0006549 · Cited by 142
“A panel of 10–15 members is usually recommended [15].”
BackgroundChalmers, J., & Armour, M. (2019) · Springer eBooks · doi:10.1007/978-981-10-5251-4_99 · Cited by 185
Brustia, R., Monsel, A., Skurzak, S., Schiffer, E., Carrier, F. M., Patrono, D., Kaba, A., Detry, O., Malbouisson, L. M. S., Andraus, W., et al. (2021) · Transplantation · doi:10.1097/tp.0000000000003808 · Cited by 170
D’Souza, N., Babberich, M. P. M. D. N. T., D’Hoore, A. J. L., Tiret, E., Xynos, E., Beets‐Tan, R. G. H., Nagtegaal, I. D., Blomqvist, L., Holm, T., Glimelius, B. L. G., et al. (2019) · Annals of Surgery · doi:10.1097/sla.0000000000003251 · Cited by 148
Research priorities for public mental health in Europe: recommendations of the ROAMER project
Forsman, A. K., Wahlbeck, K., Aaro, L. E., Alonso, J., Barry, M. M., Brunn, M., Cardoso, G., Cattan, M., de Girolamo, G., Eberhard-Gran, M., et al. (2015) · The European Journal of Public Health · doi:10.1093/eurpub/cku232 · Cited by 139
Biegstraaten, M., Cox, T. M., Belmatoug, N., Berger, M. G., Collin-Histed, T. L., Dahl, S. V., Rocco, M. D., Fraga, C., Giona, F., Giraldo, P., et al. (2016) · Blood Cells Molecules and Diseases · doi:10.1016/j.bcmd.2016.10.008 · Cited by 143
“A modified Delphi procedure was used to develop group consensus on management goals for GD1 [20].”
MethodAgarwal, S., Sripad, P., Johnson, C., Kirk, K., Bellows, B., Ana, J., Blaser, V., Kumar, M. B., Buchholz, K., Casseus, A., et al. (2019) · Human Resources for Health · doi:10.1186/s12960-019-0422-0 · Cited by 136
“The first two consultations followed a modified Delphi approach [39].”
MethodLessons Learned from a Two-Round Delphi-based Scenario Study
Schmalz, U., Spinler, S., & Ringbeck, J. (2021) · MethodsX · doi:10.1016/j.mex.2020.101179 · Cited by 136
Shared mobility in post-COVID era: New challenges and opportunities
Shokouhyar, S., Shokoohyar, S., Sobhani, A., & Gorizi, A. J. (2021) · Sustainable Cities and Society · doi:10.1016/j.scs.2021.102714 · Cited by 158
Expert Consensus: Telehealth Skills for Health Care Professionals
Galpin, K., Sikka, N. K., King, S. L., Horvath, K. A., Shipman, S. A., Evans, N., Henderson, K., Kitts, A. B., Krupinski, E. A., Kvedar, J. C., et al. (2020) · Telemedicine Journal and e-Health · doi:10.1089/tmj.2020.0420 · Cited by 137
Ecosystem Services Modeling as a Tool for Defining Priority Areas for Conservation
Duarte, G. T., Ribeiro, M. C., & Paglia, A. P. (2016) · PLoS ONE · doi:10.1371/journal.pone.0154573 · Cited by 161
“We used the Delphimethod [46], in which we send the questionnaire with a text description of the survey to experts via e-mail.”
MethodDefining metabolically healthy obesity in children: a scoping review
Damanhoury, S., Newton, A. S., Rashid, M., Hartling, L., Byrne, J. L. S., & Ball, G. D. C. (2018) · Obesity Reviews · doi:10.1111/obr.12721 · Cited by 155
“The summary of responses was shared with participants in each subsequent round to inform responses and encourage consensus (25,26).”
BackgroundMethodInfluentialVernooij, R. W. M., Alonso-Coello, P., Brouwers, M., Martínez García, L., & Panel, C. (2017) · PLOS Medicine · doi:10.1371/journal.pmed.1002207 · Cited by 131
Hamilton, C. B., Hoens, A. M., McQuitty, S., McKinnon, A., English, K., Backman, C. L., Azimi, T., Khodarahmi, N., & LI, L. C. (2018) · PLoS ONE · doi:10.1371/journal.pone.0206588 · Cited by 148
Luke, D. A., Sarli, C. C., Suiter, A. M., Carothers, B. J., Combs, T. B., Allen, J. L., Beers, C. E., & Evanoff, B. A. (2018) · Clinical and Translational Science · doi:10.1111/cts.12495 · Cited by 131
Meshkat, B., Cowman, S., Gethin, G., Ryan, K., Wiley, M. M., Brick, A., Clarke, E., & Mulligan, E. (2014) · Journal of Hospital Administration · doi:10.5430/jha.v3n4p1 · Cited by 150
Who is a Delphi ‘expert’? Reflections on a bioeconomy expert selection procedure from Ireland
Devaney, L., & Henchion, M. M. (2018) · Futures · doi:10.1016/j.futures.2018.03.017 · Cited by 146
Barriers to Blockchain Adoption in the Circular Economy: A Fuzzy Delphi and Best-Worst Approach
Rejeb, A., Rejeb, K., Keogh, J. G., & Zailani, S. H. M. (2022) · Sustainability · doi:10.3390/su14063611 · Cited by 170
“The fuzzy Delphi method (FDM) represents a formal communication technique or strategy initially envisioned as a methodical, interactive, and predictive procedure based on an expert panel [86,87].”
MethodLopetuso, L. R., Deleu, S., Godny, L., Petito, V., Puca, P., Facciotti, F., Sokol, H., Ianiro, G., Masucci, L., Abreu, M., et al. (2023) · Gut · doi:10.1136/gutjnl-2023-329948 · Cited by 129
Légaré, F., Borduas, F., Freitas, A., Jacques, A., Godin, G., Luconi, F., Grimshaw, J., & the CPD-KT team (2014) · PLoS ONE · doi:10.1371/journal.pone.0091013 · Cited by 161
“As there are no definite criteria for determining consensus in a Delphi study [12], content validity was set a priori when at least 75% of participants had reached agreement on the relevance of an item.”
BackgroundUnderstanding cloud computing adoption issues: A Delphi study approach
El-Gazzar, R. F., Hustad, E., & Olsen, D. H. (2016) · Journal of Systems and Software · doi:10.1016/j.jss.2016.04.061 · Cited by 164
Jordans, I. P. M., Verberkt, C., de Leeuw, R. A., Bilardo, C. M., den Bosch, T. V., Bourne, T., Brölmann, H. A. M., Dueholm, M., Hehenkamp, W. J. K., Jastrow, N., et al. (2021) · Ultrasound in Obstetrics and Gynecology · doi:10.1002/uog.24815 · Cited by 159
Measuring Best Practices for Workplace Safety, Health, and Well-Being
Sorensen, G., Sparer, E. H., Williams, J. A. R., Gundersen, D. A., Boden, L. I., Dennerlein, J. T., Hashimoto, D. M., Katz, J. N., McLellan, D. L., Okechukwu, C. A., et al. (2018) · Journal of Occupational and Environmental Medicine · doi:10.1097/jom.0000000000001286 · Cited by 175
Revisiting the Delphi technique - Research thinking and practice: A discussion paper
Hasson, F., Keeney, S., & McKenna, H. (2025) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2025.105119 · Cited by 185
Stärk, K. D. C., Alonso, S., Dadios, N., Dupuy, C., Ellerbroek, L., Georgiev, M., Hardstaff, J., Huneau‐Salaün, A., Laugier, C., Mateus, A., et al. (2013) · Food Control · doi:10.1016/j.foodcont.2013.11.009 · Cited by 123
Measuring the adequacy of antenatal health care: a national cross-sectional study in Mexico
Heredia-Pi, I., Servan-Mori, E., Darney, G., Reyes-Morales, H., & Lozano, R. (2016) · Bulletin of the World Health Organization · doi:10.2471/blt.15.168302 · Cited by 123
Palermo, T. M., Janicke, D. M., McQuaid, E. L., Mullins, L. L., Robins, P. M., & Wu, Y. P. (2014) · Journal of Pediatric Psychology · doi:10.1093/jpepsy/jsu015 · Cited by 127
“…an approach that systematically collects perspectives from a larger representative segment of pediatric psychologists to build consensus, such as using Delphi methods (Hsu & Sandford, 2007), would potentially generate additional ideas and provide more confidence in their perceived importance.”
BackgroundEvaluating investments in renewable energy under policy risks
Gatzert, N., & Vogl, N. (2016) · Energy Policy · doi:10.1016/j.enpol.2016.04.027 · Cited by 150
Water Quality Indices as Tools for Decision Making and Management
Gitau, M. W., Chen, J., & Ma, Z. (2016) · Water Resources Management · doi:10.1007/s11269-016-1311-0 · Cited by 121
Developing a sustainable HRM system from a contextual perspective
Diaz‐Carrion, R., López‐Fernández, M., & Romero‐Fernandez, P. M. (2018) · Corporate Social Responsibility and Environmental Management · doi:10.1002/csr.1528 · Cited by 120
Artificial intelligence in medical education curriculum: An e-Delphi study for competencies
Çalışkan, S. A., Demir, K., & Karaca, O. (2022) · PLOS ONE · doi:10.1371/journal.pone.0271872 · Cited by 115
Consensus-based technical recommendations for clinical translation of renal BOLD MRI
Bane, O., Mendichovszky, I. A., Milani, B., Dekkers, I. A., Deux, J., Eckerbom, P., Grenier, N., Hall, M. E., Inoue, T., Laustsen, C., et al. (2019) · Magnetic Resonance Materials in Physics Biology and Medicine · doi:10.1007/s10334-019-00802-x · Cited by 127
Standards of practice in empirical bioethics research: towards a consensus
Ives, J., Dunn, M., Molewijk, B. C., Schildmann, J., Bærøe, K., Frith, L., Huxtable, R., Landeweer, E. G. M., Mertz, M., Provoost, V., et al. (2018) · BMC Medical Ethics · doi:10.1186/s12910-018-0304-3 · Cited by 121
Baines, R. L., & Regan de Bere, S. (2018) · Health Expectations · doi:10.1111/hex.12618 · Cited by 116
Magua, W., Zhu, X., Bhattacharya, A., Filut, A., Potvien, A., Leatherberry, R., Lee, Y.-G., Jens, M., Malikireddy, D. R., Carnes, M. L., et al. (2017) · Journal of Women s Health · doi:10.1089/jwh.2016.6021 · Cited by 130
Gattrell, W. T., Hungin, A. P., Price, A., Winchester, C. C., Tovey, D., Hughes, E. L., van Zuuren, E. J., Goldman, K., Logullo, P., Matheis, R., et al. (2022) · Research Integrity and Peer Review · doi:10.1186/s41073-022-00122-0 · Cited by 116
“The use of structured, systematic approaches to reach consensus is supported by the observation that, in an unstructured group meeting, there is the risk of a single individual dominating the discussion and decisions may be portrayed as unanimous when, in reality, there is dissent within the group [31].”
BackgroundStrohmeier, S. (2020) · The International Journal of Human Resource Management · doi:10.1080/09585192.2018.1443963 · Cited by 114
“As a result, 37 usable questionnaires (92.5% response rate) were obtained after the second round (see Table 1), which clearly exceeds the acceptable response rates (e.g. Hsu & Sandford, 2007).”
BackgroundResultInfluentialSmith, M. D., Vicenzino, B., Bahr, R., Bandholm, T., Cooke, R., Mendonça, L. D. M., Fourchet, F., Glasgow, P. D., Gribble, P. A., Herrington, L. C., et al. (2021) · British Journal of Sports Medicine · doi:10.1136/bjsports-2021-104087 · Cited by 135
How to use the Delphi method to aid in decision making and build consensus in pharmacy education
Olsen, A. A., Wolcott, M. D., Haines, S. T., Janke, K. K., & McLaughlin, J. E. (2021) · Currents in Pharmacy Teaching and Learning · doi:10.1016/j.cptl.2021.07.018 · Cited by 115
Fanti, S., Minozzi, S., Morigi, J. J., Giesel, F., Ceci, F., Uprimny, C., Hofman, M. S., Eiber, M., Schwarzenbock, S., Castellucci, P., et al. (2017) · European Journal of Nuclear Medicine and Molecular Imaging · doi:10.1007/s00259-017-3725-1 · Cited by 113
Prioritizing Zoonoses: A Proposed One Health Tool for Collaborative Decision-Making
Rist, C., Arriola, C. S., & Rubin, C. S. (2014) · PLoS ONE · doi:10.1371/journal.pone.0109986 · Cited by 146
Bussières, A., Cancelliere, C., Ammendolia, C., Comer, C. M., Zoubi, F. A., Châtillon, C.-É., Chernish, G., Cox, J. M., Gliedt, J. A., Haskett, D., et al. (2021) · Journal of Pain · doi:10.1016/j.jpain.2021.03.147 · Cited by 136
A critical review of the motivational factors for festival attendance based on meta-analysis
맹해영, Jang, H. Y., & Li, J. M. (2015) · Tourism Management Perspectives · doi:10.1016/j.tmp.2015.10.003 · Cited by 150
Merfeld, K., Wilhelms, M.-P., Henkel, S., & Kreutzer, K. (2019) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2019.03.012 · Cited by 112
Marone, E. M., Freyrie, A., Ruotolo, C., Michelagnoli, S., Antonello, M., Speziale, F., Veroux, P., Gargiulo, M., & Gaggiano, A. (2019) · Annals of Vascular Surgery · doi:10.1016/j.avsg.2019.05.049 · Cited by 118
An Interdisciplinary Definition of Pornography: Results from a Global Delphi Panel
McKee, A., Byron, P., Litsou, K., & Ingham, R. (2019) · Archives of Sexual Behavior · doi:10.1007/s10508-019-01554-4 · Cited by 112
“Hsu and Sandford (2007) note that, in designing a Delphi instrument, researchers must be careful not to lead the respondents.”
BackgroundInternational consensus statement on microbiome testing in clinical practice
Porcari, S., Mullish, B. H., Asnicar, F., Ng, S. C., Zhao, L., Hansen, R., O’Toole, P. W., Raes, J., Hold, G. L., Putignani, L., et al. (2024) · The Lancet. Gastroenterology & hepatology · doi:10.1016/s2468-1253(24)00311-x · Cited by 147
“18 For each statement, experts were asked to rate their agreement anonymously, according to a 5 point Likert scale (1=agree strongly, 2=agree with reservation, 3=undecided, 4=disagree, and 5=disagree strongly).”
Metadata
- Authors
- Hsu, C.-C., & Sandford, B. A.
- Venue
- University of Massachusetts Amherst
- Publisher
- University of Massachusetts Amherst
- Published
- 2007
- Volume
- 12
- Issue
- 10
- Pages
- 10
- OpenAlex
- W2583504027
- Cited by
- 4,500
- References
- 40
- Citations by year
- 2026: 160 · 2025: 361 · 2024: 381 · 2023: 406 · 2022: 373 · 2021: 381 · 2020: 383 · 2019: 375 · 2018: 358 · 2017: 287 · 2016: 298 · 2015: 240 · 2014: 176 · 2013: 129 · 2012: 91
- FWCI
- 516.01
- Type
- journal article
- Language
- en
- Open access
- Licence
- cc-by-nc-nd
- Locations
- ScholarWorks@UMassAmherst (University of Massachusetts Amherst) (submittedVersion, cc-by-nc-nd)
- University of Massachusetts (UMass) Amherst
- Practical assessment, research & evaluation
- Topics
- Social Sciences › Sociology and Political Science › Delphi Technique in Research
- Social Sciences › Decision Sciences › Management Science and Operations Research › Multi-Criteria Decision Making
- Social Sciences › Sociology and Political Science › Focus Groups and Qualitative Methods
- Keywords
- Delphi method, consensus building, expert opinion, participant selection, response rate, needs assessment, program planning, policy determination
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.