The Delphi Technique: Making Sense of Consensus
Open accessThe DOI matches its DataCite record; 40 of the 40 references it cites were found in a registry or an index.
Abstract
The Delphi technique is a widely used and accepted method for gathering data from respondents within their domain of expertise. The technique is designed as a group communication process which aims to achieve a convergence of opinion on a specific real-world issue. The Delphi process has been used in various fields of study such as program planning, needs assessment, policy determination, and resource utilization to develop a full range of alternatives, explore or expose underlying assumptions, as well as correlate judgments on a topic spanning a wide range of disciplines. The Delphi technique is well suited as a method for consensus-building by using a series of questionnaires delivered using multiple iterations to collect data from a panel of selected subjects. Subject selection, time frames for conducting and completing a study, the possibility of low response rates, and unintentionally guiding feedback from the respondent group are areas which should be considered when designing and implementing a Delphi study. Accessed 68,465 times on https://pareonline.net from August 30, 2007 to December 31, 2019. For downloads from January 1, 2020 forward, please click on the PlumX Metrics link to the right.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“factors with a level of agreement ≥ 70% were extracted and included in the present review [19].”
From Healthcare professionals’ perspectives on artificial intelligence in patient care: a systematic review of hindering and facilitating factors on different levels (2025) · cites this paper as [19]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“To narrow the list into a usable checklist, we then used a three-step Delphi survey process [31].”
From PRISMA-S: an extension to the PRISMA Statement for Reporting Literature Searches in Systematic Reviews (2021) · cites this paper as [31]
The cited source says what the claim attributes to it.
In the citing paper
“The iteration characteristics of the Delphi technique have intrinsic limitations and might lead to groupthink, where participants might conform to dominant opinions of the group, which could affect objectivity.197”
From Definition and diagnostic criteria of clinical obesity (2025) · cites this paper as [197]
In the citing paper
“This anonymous and iterative group technique is designed to gather individual opinions from experts and transform these into a group consensus 6.”
From Diagnosis and treatment of chronic bacterial prostatitis and chronic prostatitis/chronic pelvic pain syndrome: a consensus guideline (2015) · cites this paper as [6]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Foundations of Behavioral Research
Stockton, R., & Kerlinger, F. N. (1974) · American Educational Research Journal · doi:10.2307/1162206 · Cited by 7,347
An Experimental Application of the DELPHI Method to the Use of Experts
Dalkey, N., & Helmer, O. (1963) · Management Science · doi:10.1287/mnsc.9.3.458 · Cited by 6,061
Research guidelines for the Delphi survey technique
Hasson, F. (2000) · Journal of Advanced Nursing · doi:10.1046/j.1365-2648.2000.01567.x · Cited by 5,951 · cited 1×
“As such, Ludwig (1994) specifically addresses subject motivation as the key to the successful implementation of a Delphi study and investigators need to play an active role in this area to help ensure as high a response rate as possible.”
The Delphi Method: An Experimental Study of Group Opinion
(1969) · RAND Corporation eBooks · doi:10.7249/rm5888 · Cited by 1,628
Gazing into the oracle : the Delphi method and its application to social policy and public health
Ziglio, E., & Adler, M. E. (1996) · Jessica Kingsley Publishers eBooks · Cited by 1,429
Group Techniques for Program Planning
de Ven, A. V., & Delbecq, A. L. (1975) · Cited by 907
Technological forecasting for decision making
Martino, J. P. (1972) · Cited by 813
Experimental Study of Channel Patterns
SCHUMM, S. A., & KHAN, H. R. (1972) · Geological Society of America Bulletin · doi:10.1130/0016-7606(1972)83[1755:esocp]2.0.co;2 · Cited by 776
An experimental study of group opinion
Dalkey, N. (1969) · Futures · doi:10.1016/s0016-3287(69)80025-x · Cited by 750
Miller, G. (2001) · Tourism Management · doi:10.1016/s0261-5177(00)00067-4 · Cited by 604
Research works printed but not linked: 7
- Adams, S. J. (2001). Projecting the next decade in safety management: A Delphi technique study. Professional Safety, 46 (10), 26-29.
- Green, P. J. (1982, March). The content of a college-level outdoor leadership course. Paper presented at the Conference of the Northwest District Association for the American Alliance for Health, Physical Education, Recreation, and Dance, Spokane, WA.
- Jones, C. G. (1975). A Delphi evaluation of agreement between organizations. In H. A. Linstone, & M. Turoff (Eds.). The Delphi method: Techniques and applications (pp. 160-167). Reading, MA: Addison-Wesley Publishing Company.
- Kaplan, L. M. (1971). The use of the Delphi method in organizational communication: A case study. Unpublished master's thesis, The Ohio State University, Columbus.
- Linstone, H. A., & Turoff, M. (1975). Introduction. In H. A. Linstone, & M. Turoff (Eds.). The Delphi method: Techniques and applications (pp. 3-12). Reading, MA: Addison-Wesley Publishing Company.
- Ludlow, J. (1975). Delphi inquiries and knowledge utilization. In H. A. Linstone, & M. Turoff (Eds.). The Delphi method: Techniques and applications (pp. 102-123). Reading, MA: Addison-Wesley Publishing Company.
- Scheibe, M., Skutsch, M., & Schofer, J. (1975). Experiments in Delphi methodology. In H. A. Linstone, & M. Turoff (Eds.). The Delphi method: Techniques and applications (pp. 262-287). Reading, MA: Addison-Wesley Publishing Company.
Cited by
Items 401–500 of 1,013
Álvarez, S., Penela, A. C., Mantecón, I. M., & Rubio, A. (2016) · Journal of Cleaner Production · doi:10.1016/j.jclepro.2016.02.028 · Cited by 78
“Consensus of opinion is achieved through 236 multiple iterations between the experts (Hsu and Sandford, 2007).”
BackgroundApplying Delphi method for strategic design of social entrepreneurship
Kavoura, A., & Andersson, T. (2016) · Library Review · doi:10.1108/lr-06-2015-0062 · Cited by 62
Lambert, S. D., Ould Brahim, L., Morrison, M., Girgis, A., Yaffe, M., Belzile, E., Clayberg, K., Robinson, J., Thorne, S., Bottorff, J. L., et al. (2019) · Supportive Care in Cancer · doi:10.1007/s00520-018-4314-y · Cited by 62
“The analyses in rounds 2 and 3 identified the level of consensus for each research topic as defined by having (a) 80% of stakeholders’ ratings fall within the two highest or two lowest response categories on the 4-point scale and (b) an interquartile range (IQR) of 1 or less [13, 24].”
InfluentialMedia entrepreneurship: a consensual definition
Khajeheian, D. (2017) · AD-minister · doi:10.17230/ad-minister.30.5 · Cited by 67
“The Delphi technique is a widely used and accepted method for gathering data from respondents within their domain of expertise (Hsu and Sandford, 2007).”
Methodvan Luijtelaar, A., Greenwood, B. M., Ahmed, H. U., Barqawi, A. B., Barret, É., Bomers, J. G. R., Brausi, M. A., Choyke, P. L., Cooperberg, M. R., Eggener, S. E., et al. (2019) · World Journal of Urology · doi:10.1007/s00345-019-02636-7 · Cited by 61
“To achieve consensus among a panel of experts in the field, the Delphi method was used [16].”
Saporiti, N., Cannas, V. G., Pozzi, R., & Rossi, T. (2023) · International Journal of Production Economics · doi:10.1016/j.ijpe.2023.108888 · Cited by 73
Prioritization by consensus of enhancements for sustainable mobility in urban areas
Curiel‐Esparza, J., Mazario-Diez, J. L., Canto-Perello, J., & Martin-Utrillas, M. (2015) · Environmental Science & Policy · doi:10.1016/j.envsci.2015.10.015 · Cited by 69
Frayne, D. J., Verbiest, S., Chelmow, D. P., Clarke, H. F., Dunlop, A. L., Hosmer, J., Menard, M. K., Moos, M.-K., Ramos, D., Stuebe, A. M., et al. (2016) · Obstetrics and Gynecology · doi:10.1097/aog.0000000000001379 · Cited by 77
“A twostage web-based Delphi survey and a face-to-face meeting of key opinion leaders of the Clinical Workgroup were used to develop consensus for a framework to measure preconception wellness.(23) Measures subjected to the Delphi survey were identified by systematic literature review and stakeholder feedback.”
MethodReview of the Delphi method in library and information science research
Lund, B. D. (2020) · Journal of Documentation · doi:10.1108/jd-09-2019-0178 · Cited by 92
Christiansen, S. N., Filippou, G., Scirè, C. A., Balint, P. V., Bruyn, G. A. W., Dalbeth, N., Dejaco, C., Sedie, A. D., Filippucci, E., Hammer, H. B., et al. (2021) · Seminars in Arthritis and Rheumatism · doi:10.1016/j.semarthrit.2020.11.011 · Cited by 64
Rezaei, M., Giovando, G., Rezaei, S., & Sadraei, R. (2022) · British Food Journal · doi:10.1108/bfj-08-2021-0948 · Cited by 68
Gückenberger, M., Belka, C., Bezjak, A., Bradley, J. D., Daly, M. E., Ruysscher, D. D., Dziadziuszko, R., Finn, C. F., Flentje, M. P., Gore, E., et al. (2020) · International Journal of Radiation Oncology*Biology*Physics · doi:10.1016/j.ijrobp.2020.05.012 · Cited by 60
McCurtin, A., Healy, C., Kelly, L., Murphy, F., Ryan, J., & Walsh, J. (2017) · International Journal of Language & Communication Disorders · doi:10.1111/1460-6984.12324 · Cited by 72
Gómez, J. A. D., & Gómez, G. (2017) · Tourism Management · doi:10.1016/j.tourman.2017.05.015 · Cited by 60
Arora, S., Kapoor, A., & Shen, W. (2018) · Batteries · doi:10.3390/batteries4030030 · Cited by 64
“There are several ways including minimax deviation [42], minimum mean-square error method [43] and Delphi method [44] to determine what a customer wants as an output of the system.”
MethodMehta, A. B., Kuter, D. J., Salek, S., Belmatoug, N., Bembi, B., Bright, J. R., Dahl, S. V., Deodato, F., Rocco, M. D., Göker-Alpan, Ö., et al. (2018) · Internal Medicine Journal · doi:10.1111/imj.14156 · Cited by 60
Leenhardt, R., Buisson, A., Bourreille, A., Marteau, P. R., Koulaouzidis, A., Li, C., Keuchel, M., Rondonotti, E., Tóth, E., Plevris, J. N., et al. (2019) · United European Gastroenterology Journal · doi:10.1177/2050640619895864 · Cited by 69
“The Delphi method is one of the most accurate and validated process for answering questions that cause uncertainty even among experts.(13,14) The present study used a Delphi consensus model based on alternative feedback rounds between a core group (CG) and an expert group (EG) until a consensus was reached (Figure 1).”
MethodJiménez, E. A. L., & Ouariachi, T. (2020) · Journal of Information Communication and Ethics in Society · doi:10.1108/jices-03-2020-0034 · Cited by 59
Kelker, H. P., Yoder, K., Musey, P. I., Harris, M., Johnson, O., Sarmiento, E. J., Vyas, P. K., Henderson, B., Adams, Z. W., & Welch, J. L. (2021) · BMC Emergency Medicine · doi:10.1186/s12873-021-00425-3 · Cited by 65
“, quarantine, safety concerns), wellbeing, burnout, and resilience [38, 39].”
Mori, Y., East, J. E., Hassan, C., Halvorsen, N., Berzin, T. M., Byrne, M. F., Renteln, D. A. V., Hewett, D. G., Repici, A., Ramchandani, M. J., et al. (2023) · Digestive Endoscopy · doi:10.1111/den.14531 · Cited by 73
“Statement 3 : We recommend that a broad range of high-quality cost-effectiveness research should be undertaken to understand whether AI implementation bene fi ts populations and societies in different health-care systems.”
BackgroundDriving behaviour change with cybersecurity awareness
Chaudhary, S. (2024) · Computers & Security · doi:10.1016/j.cose.2024.103858 · Cited by 97
An effective Building Neighborhood Green Index model for measuring urban green space
Liu, Y., Meng, Q., Zhang, J., Zhang, L., Jancsó, T., & Vatseva, R. (2015) · International Journal of Digital Earth · doi:10.1080/17538947.2015.1037870 · Cited by 59
van Rijssen, L. B., Gerritsen, A., Henselmans, I., Sprangers, M. A. G., Jacobs, M., Bassi, C., Busch, O. R., Castillo, C. F.-D., Fong, Z. V., He, J. P., et al. (2017) · Annals of Surgery · doi:10.1097/sla.0000000000002633 · Cited by 64
Ghanouni, P., Jarus, T., Zwicker, J. G., Lucyshyn, J. M., Mow, K., & Ledingham, A. (2018) · Journal of Autism and Developmental Disorders · doi:10.1007/s10803-018-3737-0 · Cited by 70
“Through this iterative and controlled feedback process, we developed and validated the scenarios until consensus was reached (Hsu and Sandford 2007).”
MethodZervogianni, V., Fletcher‐Watson, S., Herrera, G., Goodwin, M. S., Pérez-Fuster, P., Brosnan, M., & Grynszpan, O. (2020) · Autism · doi:10.1177/1362361319898331 · Cited by 79
“The Delphi methodology was therefore chosen as an ideal format for systematically capturing and integrating opinion from a diverse group of experts, who were not co-located and remained anonymous from each other (Goodman, 1987; Hsu & Sandford, 2007).”
MethodMagnusson, D., Rethorn, Z. D., Bradford, E. H., Maxwell, J. L., Ingman, M., Davenport, T. E., & Bezner, J. R. (2020) · Physical Therapy · doi:10.1093/ptj/pzaa056 · Cited by 67
What should we know about dementia in the 21st Century? A Delphi consensus study
Annear, M. J., Toye, C., McInerney, F., Eccleston, C., Tranter, B., Elliott, K.-E., & Robinson, A. (2015) · BMC Geriatrics · doi:10.1186/s12877-015-0008-1 · Cited by 58
“In Delphi studies, participant selection is often regarded as the most important step in the process because it relates directly to the quality of the data generated [12,18].”
BackgroundMethodInfluentialLane, R., Halcomb, E., McKenna, L., Zwar, N. A., Naccarella, L., Davies, G. P., & Russell, G. M. (2016) · Australian Health Review · doi:10.1071/ah15239 · Cited by 60
Cross-cultural neuropsychological assessment in the European Union: a Delphi expert study
Franzen, S., Papma, J. M., van den Berg, E., & Nielsen, T. R. (2020) · Archives of Clinical Neuropsychology · doi:10.1093/arclin/acaa083 · Cited by 65
Sörelius, K., Wyss, T. R., Adam, D. J., Beck, A. W., Bérard, X., Budtz-Lilly, J. W., Chakfé, N., Clough, R. E., Czerny, M., D’Oria, M., et al. (2022) · European Journal of Vascular and Endovascular Surgery · doi:10.1016/j.ejvs.2022.11.024 · Cited by 65
Velozo, C. A., Seel, R. T., Magasi, S., Heinemann, A. W., & Romero, S. (2012) · Archives of Physical Medicine and Rehabilitation · doi:10.1016/j.apmr.2012.06.001 · Cited by 65
Refrigerated container versus bulk: evidence from the banana cold chain
Arduino, G., Murillo, D. G. C., & Parola, F. (2013) · Maritime Policy & Management · doi:10.1080/03088839.2013.851421 · Cited by 69
Aggressive driving behaviour in young drivers (aged 16 through 25) involved in fatal crashes
Lambert-Bélanger, A., Dubois, S. J., Weaver, B., Mullen, N. W., & Bédard, M. (2012) · Journal of Safety Research · doi:10.1016/j.jsr.2012.10.011 · Cited by 68
Diaz, J. M., Webb, S. T., Warner, L. A. S., & Monaghan, P. F. (2018) · Urban forestry & urban greening · doi:10.1016/j.ufug.2018.02.014 · Cited by 73
“The Delphi technique is a widely used and accepted method for achieving convergence of opinion concerning realworld knowledge solicited from experts within certain topic areas (Hsu and Sandford, 2007).”
MethodImplementation of Pharmacogenetics in Primary Care: A Multi-Stakeholder Perspective
Rigter, T., Jansen, M. E., de Groot, J. M., Janssen, S. W. J., Rodenburg, W., & Cornel, M. C. (2020) · Frontiers in Genetics · doi:10.3389/fgene.2020.00010 · Cited by 66
Martindale, R. G., Berlana, D., Boullata, J. I., Cai, W., Calder, P. C., Deshpande, G., Evans, D. C., García‐de‐Lorenzo, A., Goulet, O., Li, A., et al. (2020) · Journal of Parenteral and Enteral Nutrition · doi:10.1002/jpen.1746 · Cited by 70
Framing maternal morbidity: WHO scoping exercise
Vanderkruik, R. C., Tunçalp, Ö., Chou, D., & Say, L. (2013) · BMC Pregnancy and Childbirth · doi:10.1186/1471-2393-13-213 · Cited by 57
“A modified Delphi method was used among ten identified reproductive and maternal health experts within the WHO headquarters in Geneva, Switzerland to revise and finalize the questions within the questionnaire [15,16].”
MethodTechnique Standards for Skin Lesion Imaging: A Delphi Consensus Statement
Katragadda, C., Finnane, A., Soyer, H. P., Marghoob, A. A., Halpern, A., Malvehy, J., Kittler, H., Hofmann-Wellenhof, R., Da Silva, D., Abraham, I., et al. (2017) · JAMA Dermatology · doi:10.1001/jamadermatol.2016.3949 · Cited by 57
Systematic review of the elements and service standards of delightful service
Wang, J. C., Wang, Y., & Tai, Y.-F. (2016) · International Journal of Contemporary Hospitality Management · doi:10.1108/ijchm-08-2014-0400 · Cited by 76
Los desafíos éticos del periodismo en la era de la inteligencia artificial
Ruiz, M. J. U., Rubio, L. M. C., & Verdú, F. J. M. (2021) · Estudios sobre el Mensaje Periodístico · doi:10.5209/esmp.69708 · Cited by 97
Paton, B. M., Read, P. J., van Dyk, N., Wilson, M. G., Pollock, N., Court, N., Giakoumis, M., Head, P., Kayani, B., Kelly, S., et al. (2023) · British Journal of Sports Medicine · doi:10.1136/bjsports-2021-105384 · Cited by 75
“52 The criteria for expert inclusion were: a high level of expertise assessing, managing, rehabilitating and/or researching HSIs, based on: the number and type of HSI seen per year, years worked with athletes who sustain HSI, willingness to complete the digital survey and or attend the consensus…”
Becuwe, H., Roblin, N. P., Tondeur, J., Thys, J., Castelein, E., & Voogt, J. M. (2017) · Australasian Journal of Educational Technology · doi:10.14742/ajet.2789 · Cited by 72
“The Delphi method was chosen because it is specifically directed towards generating consensus in a group of respondents (Hsu & Sandford, 2007; Shaikh & Khoja, 2014).”
MethodLeenhardt, R., Li, C., Koulaouzidis, A., Cavallaro, F., Cholet, F., Eliakim, R., Fernández-Urién, I., Kopylov, U., McAlindon, M. E., Németh, A., et al. (2019) · Endoscopy International Open · doi:10.1055/a-0761-9742 · Cited by 69
“The literature on this technique is expanding, mainly addressing what it is and how it should be used [13, 17].”
BackgroundMethodButcher, N. J., Monsour, A., Mew, E. J., Szatmari, P., Pierro, A., Kelly, L. E., Farid-Kapadia, M., Chee-a-tow, A., Saeed, L., Monga, S., et al. (2019) · Trials · doi:10.1186/s13063-019-3248-0 · Cited by 57
Ascarya, A., Sukmana, R., Rahmawati, S., & Masrifah, A. R. (2022) · Journal of Islamic accounting and business research · doi:10.1108/jiabr-09-2020-0267 · Cited by 130
Prioritization of future genetics education for general practitioners: a Delphi study
Houwink, E. J. F., Henneman, L., Westerneng, M., van Luijk, S. J., Cornel, M. C., Dinant, G., & van der Vleuten, C. P. M. (2012) · Genetics in Medicine · doi:10.1038/gim.2011.15 · Cited by 64
“In the first round, the experts were asked to rate the educational urgency for GPs of each of the 29 topics on a 7-point Focus group findings (10) 29 statements about GPs’genetic educational needs”
BackgroundDevelopment of a multi-scale model for customer perceived value of electric vehicles
Miao, R., Xu, F., Zhang, K., & Jiang, Z. (2014) · International Journal of Production Research · doi:10.1080/00207543.2014.890757 · Cited by 62
“The Delphi technique, by definition, is a group process involving an interaction between the researcher and a group of identified experts on a specified topic, usually through a series of questionnaires (Hsu and Sandford 2007).”
MethodHaddad, M., Menchetti, M., McKeown, E., Tylee, A. T., & MANN, A. H. (2015) · BMC Psychiatry · doi:10.1186/s12888-014-0381-x · Cited by 72
“It was initially used to assist forecasting the occurrence of future events, but is more commonly used within health and education research for the purposes of establishing core professional competencies, defining outcome measurements, validating content of measurement tools and critically assessing quality criteria […”
MethodHsu, C.-C., & Sandford, B. A. (2012) · IGI Global eBooks · doi:10.4018/978-1-4666-0074-4.ch011 · Cited by 62
“In common survey practices, investigators attempt to identify “what is,” but the Delphi is an effort to assess “what should/could be” (Miller, 2006; Hsu & Sandford, 2007a).”
BackgroundSelf-citationLatif, R. A., Mohamed, R., Dahlan, A., & Nor, M. Z. M. (2016) · Education in Medicine Journal · doi:10.5959/eimj.v8i3.421 · Cited by 84
Wijetilleka, S., Jayne, D. R. W., Mukhtyar, C. B., Ala, A., Bright, P., Chinoy, H., Harper, L., Kazmi, M. A., Kiani‐Alikhan, S., Li, C. K., et al. (2018) · Rheumatology · doi:10.1093/rheumatology/key394 · Cited by 62
Spinelli, A., Anania, G., Arezzo, A., Berti, S., Bianco, F., Bianchi, P. P., Giuli, M. D., Nardi, P. D., Paolis, P. D., Foppa, C., et al. (2020) · Updates in Surgery · doi:10.1007/s13304-020-00837-z · Cited by 63
“One criticism of using the modified Delphi method is the loss of anonymity in the voting process, particularly in the third round [83, 84].”
MethodEstablishing a Research Agenda for Art Therapy: A Delphi Study
Kaiser, D., & Deaver, S. (2013) · Art Therapy · doi:10.1080/07421656.2013.819281 · Cited by 55
Thomas, S. L., Wakerman, J., & Humphreys, J. S. (2014) · BMC Family Practice · doi:10.1186/1471-2296-15-143 · Cited by 68
“The anonymity of the Delphi process prevents dominance by some individuals and allows for less popular or less forceful opinions to be considered [13].”
BackgroundHIRAID: An evidence-informed emergency nursing assessment framework
Munroe, B., Curtis, K. A., Murphy, M., Strachan, L., & Buckley, T. A. (2015) · Australasian Emergency Nursing Journal · doi:10.1016/j.aenj.2015.02.001 · Cited by 68
Development of a sustainability assessment framework for geothermal energy projects
Shortall, R., Davíðsdóttir, B., & Axelsson, G. (2015) · Energy Sustainable Development/Energy for sustainable development · doi:10.1016/j.esd.2015.02.004 · Cited by 66
Challenges in mathematical cognition: A collaboratively-derived research agenda
Alcock, L., Ansari, D., Batchelor, S., Bisson, M., Smedt, B. D., Gilmore, C. K., Göbel, S. M., Hannula‐Sormunen, M. M., Hodgen, J., Inglis, M., et al. (2016) · Journal of Numerical Cognition · doi:10.5964/jnc.v2i1.10 · Cited by 77
Audigé, L., Flury, M. P., Müller, A. M., & Durchholz, H. (2016) · Journal of Shoulder and Elbow Surgery · doi:10.1016/j.jse.2016.04.036 · Cited by 62
“We implemented a consensus process with an international panel of experienced clinicians using the modified Delphi technique.(17) Three successive online surveys were completed using the electronic datacapture system REDCap.”
MethodUnderstanding key skills for information security managers
Haqaf, H., & Koyuncu, M. (2018) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2018.07.013 · Cited by 69
Content validation of a quantitative stakeholder engagement measure
Goodman, M. S., Ackermann, N., Bowen, D. J., & Thompson, V. L. S. (2019) · Journal of Community Psychology · doi:10.1002/jcop.22239 · Cited by 65
Henzler, D., Schmidt, S., Koçar, A., Herdegen, S., Lindinger, G. L., Maris, M. T., Bak, M. A. R., Willems, D. L., Tan, H. L., Lauerer, M., et al. (2025) · BMC Health Services Research · doi:10.1186/s12913-025-12664-2 · Cited by 54
“In accordance with common practice for quantitative survey studies, factors with a level of agreement ≥ 70% were extracted and included in the pres-ent review [19].”
Genetic Counseling Supervisor Competencies: Results of a Delphi Study
Higgins, S. E., Veach, P. M., MacFarlane, I. M., Borders, L. D., LeRoy, B. S., & Callanan, N. P. (2012) · Journal of Genetic Counseling · doi:10.1007/s10897-012-9512-z · Cited by 54
“Procedure & Data Analysis An online Delphi method was chosen because it is a communication process that allows experts in various locations to collaborate in an anonymous fashion (Hsu and Sandford 2007; Pickard 2007).”
BackgroundMethodInfluentialInitiating consensus: stakeholders define entrepreneurship in education
Attali, M. O., & Yemini, M. (2016) · Educational Review · doi:10.1080/00131911.2016.1153457 · Cited by 68
“…derives from the fact that the participants are anonymous and isolated from one another, thereby reducing the pressure and the biases commonly resulting from face-to-face group discussions, such as those caused by individuals who dominate the discussion (Clayton 1997; Hsu and Sandford 2007).”
BackgroundMethodCacciamani, G. E., Sholklapper, T. N., Dell’Oglio, P., Rocco, B. M. C., Annino, F., Antonelli, A., Amenta, M., Borghesi, M., Bove, P., Bozzini, G., et al. (2022) · European Urology Focus · doi:10.1016/j.euf.2022.01.018 · Cited by 58
Developing, Testing, and Communicating Earthquake Forecasts: Current Practices and Future Directions
Mizrahi, L., Dallo, I., van der Elst, N. J., Christophersen, A., Spassiani, I., Werner, M. J., Iturrieta, P. C., Bayona, J. A., Iervolino, I., Schneider, M., et al. (2024) · Reviews of Geophysics · doi:10.1029/2023rg000823 · Cited by 76
Kubota, Y., Okuyama, T., Uchida, M., Umezawa, S., Nakaguchi, T., Sugano, K., Ito, Y., Katsuki, F., Nakano, Y., Nishiyama, T., et al. (2016) · Psycho-Oncology · doi:10.1002/pon.4000 · Cited by 53
“Both the Confidence Scale and the Knowledge Scale were developed using the Delphi technique [14] by our research group for this study because there were no established measures.”
MethodNursing physical assessment for patient safety in general wards: reaching consensus on core skills
Douglas, C., Booker, C., Fox, R., Windsor, C. A., Osborne, S. R., & Gardner, G. E. (2016) · Journal of Clinical Nursing · doi:10.1111/jocn.13201 · Cited by 58
CTS position statement: Pharmacotherapy in patients with COPD—An update
Bourbeau, J., Bhutani, M., Hernandez, P., Marciniuk, D. D., Aaron, S. D., Balter, M. S., Beauchesne, M.-F., D’Urzo, A. D., Goldstein, R. S., Kaplan, A. G., et al. (2017) · Canadian Journal of Respiratory Critical Care and Sleep Medicine · doi:10.1080/24745332.2017.1395588 · Cited by 64
“For this reason, key messages in our position statement were determined using a survey with features of a Delphi technique.(23)”
MethodHow to deal with knowledge management misalignment: a taxonomy based on a 3D fuzzy methodology
Centobelli, P., Cerchione, R., & Esposito, E. (2018) · Journal of Knowledge Management · doi:10.1108/jkm-10-2016-0456 · Cited by 58
Schumacher, K. A., & Agbemabiese, L. (2019) · Resources Conservation and Recycling · doi:10.1016/j.resconrec.2019.06.033 · Cited by 61
Chang, S.-I., Chang, L.-M., & Liao, J.-C. (2020) · Information & Management · doi:10.1016/j.im.2020.103335 · Cited by 80
Development of a Tool for Food Literacy Assessment in Children (TFLAC)
Amin, S. A., Lehnerd, M. E., Cash, S. B., Economos, C. D., & Sacheck, J. M. (2019) · Journal of Nutrition Education and Behavior · doi:10.1016/j.jneb.2018.12.006 · Cited by 65
Baltic Health Tourism: Uniqueness and Commonalities
Smith, M. (2015) · Scandinavian Journal of Hospitality and Tourism · doi:10.1080/15022250.2015.1024819 · Cited by 65
Prescribing for Australians living with dementia: study protocol using the Delphi technique
Page, A. T., Potter, K., Clifford, R. M., McLachlan, A. J., & Etherton‐Beer, C. (2015) · BMJ Open · doi:10.1136/bmjopen-2015-008048 · Cited by 61
“We have chosen to use a Delphi technique as the most appropriate study design to develop explicit criteria that are consensus based where an insufficient quantity or grade of evidence exists to develop evidence-based criteria.(40) Other methods commonly used to achieve consensus (eg, a focus group) would not be feasib…”
MethodCritical BIM qualification criteria for construction pre-qualification and selection
Mahamadu, A., Mahdjoubi, L., & Booth, C. A. (2017) · Architectural Engineering and Design Management · doi:10.1080/17452007.2017.1296812 · Cited by 64
Currie, K., King, C., Nuttall, T. J., Smith, M., & Flowers, P. (2018) · Veterinary Record · doi:10.1136/vr.104639 · Cited by 58
Esmaeilizadeh, S., Shaghaghi, A., & Taghipour, H. (2020) · Journal of Material Cycles and Waste Management · doi:10.1007/s10163-020-01005-6 · Cited by 59
Luedtke, K., Basener, A., Bedei, S., Castien, R., Chaibi, A., Falla, D., Fernández-de-las-Peñas, C., Gustafsson, M., Hall, T., Jull, G., et al. (2020) · BMJ Open · doi:10.1136/bmjopen-2019-029855 · Cited by 52
Chan, K., Nam, S., Evans, B., de Oliveira, C., Chambers, A., Gavura, S., Hoch, J. S., Mercer, R. E., Dai, W. F., Beca, J. M., et al. (2020) · BMJ Open · doi:10.1136/bmjopen-2019-032884 · Cited by 52
Evaluating critical institutional factors of Industry 4.0 for education reform
AlMalki, H. A., & DURUGBO, C. M. (2023) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2023.122327 · Cited by 69
Maher, T. M., Whyte, M. K. B., Hoyles, R. K., Parfrey, H., Ochiai, Y., Mathieson, N., Turnbull, A., Williamson, N., & Bennett, B. M. (2015) · Advances in Therapy · doi:10.1007/s12325-015-0249-6 · Cited by 55
“with IPF) were identified by the panel chairperson, based on common adopted criteria [33] and independently recruited to participate.”
Ghana Men Study Group, Quaye, S., Raymond, H. F., Atuahene, K., Amenyah, R., Aberle-Grasse, J. M., McFarland, W., & El-Adas, A. (2015) · AIDS and Behavior · doi:10.1007/s10461-014-0943-4 · Cited by 51
“The Delphi technique is used for building consensus and convergence of opinion from respondents within their domain of expertise on a specific real-world issue [18].”
MethodKaljouw, E., Pieters, B. R., Kovács, G., & Hoskin, P. (2015) · Radiotherapy and Oncology · doi:10.1016/j.radonc.2015.10.021 · Cited by 52
van der Feltz-Cornelis, C. M., Elfeddali, I., Werneke, U., Malt, U. F., Van den Bergh, O., Schaefert, R., Kop, W. J., Lobo, A., Sharpe, M., Söllner, W., et al. (2018) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2018.00151 · Cited by 52
Matthay, E. C., Gottlieb, L. M., Rehkopf, D. H., Tan, M. L., Vlahov, D., & Glymour, M. M. (2021) · Epidemiologic Reviews · doi:10.1093/epirev/mxab005 · Cited by 56
“For example, the Delphi technique is a structured communication approach to elicit consensus from a panel of experts and can be used to rank or score policies on the basis of stringency or effectiveness (67, 68).”
MethodPeeraer, J., & Petegem, P. V. (2014) · Evaluation and Program Planning · doi:10.1016/j.evalprogplan.2014.09.005 · Cited by 51
“As suggested by Hsu and Sandford (2007), as a result of round two, areas of disagreement and agreement were identified (see next section on defining consensus and priority). For the third and final round, that was launched in September 2012, after the follow-up key players meeting, all key players were invited to re-r…”
BackgroundLee, J. S., & Lee, K. (2018) · British Journal of Educational Technology · doi:10.1111/bjet.12652 · Cited by 73
Anrys, P., Petit, A.-E., Thevelin, S., Sallevelt, B. T. G. M., Maanen, A. C. D., Soiza, R. L., Correa-Pérez, A., Dalleur, O., Brauwer, I. D., Petrović, M., et al. (2021) · Journal of the American Medical Directors Association · doi:10.1016/j.jamda.2021.03.019 · Cited by 65
Kaufmann, E., Bartoloméi, F., Boon, P., Chabardès, S., Colon, A., Erőss, L., Fabó, D., Gonçalves-Ferreira, A., Imbach, L., Paesschen, W. V., et al. (2020) · Seizure · doi:10.1016/j.seizure.2020.08.015 · Cited by 50
“Consensus group methodologies, such as the Delphi method [40], are used to synthesize expert opinions in a systematic way when evidence is lacking or questions are not manageable with experimental and epidemiological methods.”
MethodEnhancing the Decision-Making Process through Industry 4.0 Technologies
Rosin, F., Forget, P., Lamouri, S., & Pellerin, R. (2022) · Sustainability · doi:10.3390/su14010461 · Cited by 76
What makes a landscape contemplative?
Olszewska-Guizzo, A., Marques, P. F., Ryan, R. L., & Barbosa, F. (2016) · Environment and Planning B Urban Analytics and City Science · doi:10.1177/0265813516660716 · Cited by 55
“The evaluation of landscapes according to set criteria fits well into that concept (Hsu and Sandford, 2007).”
BackgroundDEFINING RAPID REVIEWS: A MODIFIED DELPHI CONSENSUS APPROACH
Kelly, S. E., Moher, D., & Clifford, T. J. (2016) · International Journal of Technology Assessment in Health Care · doi:10.1017/s0266462316000489 · Cited by 50
Lyons, V. H., Moore, M., Guiney, R., Ayyagari, R. C., Thompson, L. L., Rivara, F. P., Fleming, R., Crawley, D., Harper, D., & Vavilala, M. S. (2017) · Journal of School Health · doi:10.1111/josh.12510 · Cited by 57
Sleijser‐Koehorst, M. L. S., Bijker, L., Cuijpers, P. J., Scholten-Peeters, G. G. M., & Coppieters, M. W. (2018) · Pain · doi:10.1097/j.pain.0000000000001441 · Cited by 50
“The number of experts is considered less important than their expertise and experience.(9,10,16) Delphi studies with expert panels of approximately 20 experts typically result in stable and reliable results.”
BackgroundSetting new learning targets for the 21st century science education in Malaysia
Osman, K., & Marimuthu, N. (2010) · Procedia - Social and Behavioral Sciences · doi:10.1016/j.sbspro.2010.03.581 · Cited by 68
Shawcross, D. L., Dunk, A. A., Jalan, R., KIRCHEIS, G., de Knegt, R. J., Laleman, W., Ramage, J. K., Wedemeyer, H. H., & Morgan, I. E. J. (2015) · European Journal of Gastroenterology & Hepatology · doi:10.1097/meg.0000000000000529 · Cited by 68
Determinants of Financial Sustainability for Farm Credit Applications—A Delphi Study
Henning, J. I. F., & Jordaan, H. (2016) · Sustainability · doi:10.3390/su8010077 · Cited by 70
“Every feedback process provides an opportunity for each of the participants to reassess his or her initial opinion in relation to the responses of the other responders [14], who remain anonymous during the entire process [21–23].”
BackgroundMethodGonzalez-Salazar, M. A., Venturini, M., Poganietz, W., Finkenrath, M., Kirsten, T., G., H. R. A., & Spina, P. R. (2016) · Applied Energy · doi:10.1016/j.apenergy.2016.07.120 · Cited by 49
McCulloch, K. L., de Joya, A. L., Hays, K., Donnelly, E. M., Johnson, T. K., Nirider, C. D., Roth, H., Saliga, S., & Ward, I. (2016) · Neurology Report · doi:10.1097/npt.0000000000000145 · Cited by 52
ESPACOMP Medication Adherence Reporting Guidelines (EMERGE): a reactive-Delphi study protocol
Helmy, R., Zullig, L. L., Dunbar-Jacob, J., Hughes, D. A., Vrijens, B., Wilson, I. B., & De Geest, S. (2017) · BMJ Open · doi:10.1136/bmjopen-2016-013496 · Cited by 49
Metadata
- Authors
- Hsu, C.-C., & Sandford, B. A.
- Venue
- University of Massachusetts Amherst
- Publisher
- University of Massachusetts Amherst
- Published
- 2007
- Volume
- 12
- Issue
- 10
- Pages
- 10
- OpenAlex
- W2583504027
- Cited by
- 4,500
- References
- 40
- Citations by year
- 2026: 160 · 2025: 361 · 2024: 381 · 2023: 406 · 2022: 373 · 2021: 381 · 2020: 383 · 2019: 375 · 2018: 358 · 2017: 287 · 2016: 298 · 2015: 240 · 2014: 176 · 2013: 129 · 2012: 91
- FWCI
- 516.01
- Type
- journal article
- Language
- en
- Open access
- Licence
- cc-by-nc-nd
- Locations
- ScholarWorks@UMassAmherst (University of Massachusetts Amherst) (submittedVersion, cc-by-nc-nd)
- University of Massachusetts (UMass) Amherst
- Practical assessment, research & evaluation
- Topics
- Social Sciences › Sociology and Political Science › Delphi Technique in Research
- Social Sciences › Decision Sciences › Management Science and Operations Research › Multi-Criteria Decision Making
- Social Sciences › Sociology and Political Science › Focus Groups and Qualitative Methods
- Keywords
- Delphi method, consensus building, expert opinion, participant selection, response rate, needs assessment, program planning, policy determination
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.