The Delphi Technique: Making Sense of Consensus
Open accessThe DOI matches its DataCite record; 40 of the 40 references it cites were found in a registry or an index.
Abstract
The Delphi technique is a widely used and accepted method for gathering data from respondents within their domain of expertise. The technique is designed as a group communication process which aims to achieve a convergence of opinion on a specific real-world issue. The Delphi process has been used in various fields of study such as program planning, needs assessment, policy determination, and resource utilization to develop a full range of alternatives, explore or expose underlying assumptions, as well as correlate judgments on a topic spanning a wide range of disciplines. The Delphi technique is well suited as a method for consensus-building by using a series of questionnaires delivered using multiple iterations to collect data from a panel of selected subjects. Subject selection, time frames for conducting and completing a study, the possibility of low response rates, and unintentionally guiding feedback from the respondent group are areas which should be considered when designing and implementing a Delphi study. Accessed 68,465 times on https://pareonline.net from August 30, 2007 to December 31, 2019. For downloads from January 1, 2020 forward, please click on the PlumX Metrics link to the right.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“factors with a level of agreement ≥ 70% were extracted and included in the present review [19].”
From Healthcare professionals’ perspectives on artificial intelligence in patient care: a systematic review of hindering and facilitating factors on different levels (2025) · cites this paper as [19]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“To narrow the list into a usable checklist, we then used a three-step Delphi survey process [31].”
From PRISMA-S: an extension to the PRISMA Statement for Reporting Literature Searches in Systematic Reviews (2021) · cites this paper as [31]
The cited source says what the claim attributes to it.
In the citing paper
“The iteration characteristics of the Delphi technique have intrinsic limitations and might lead to groupthink, where participants might conform to dominant opinions of the group, which could affect objectivity.197”
From Definition and diagnostic criteria of clinical obesity (2025) · cites this paper as [197]
In the citing paper
“This anonymous and iterative group technique is designed to gather individual opinions from experts and transform these into a group consensus 6.”
From Diagnosis and treatment of chronic bacterial prostatitis and chronic prostatitis/chronic pelvic pain syndrome: a consensus guideline (2015) · cites this paper as [6]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Foundations of Behavioral Research
Stockton, R., & Kerlinger, F. N. (1974) · American Educational Research Journal · doi:10.2307/1162206 · Cited by 7,347
An Experimental Application of the DELPHI Method to the Use of Experts
Dalkey, N., & Helmer, O. (1963) · Management Science · doi:10.1287/mnsc.9.3.458 · Cited by 6,061
Research guidelines for the Delphi survey technique
Hasson, F. (2000) · Journal of Advanced Nursing · doi:10.1046/j.1365-2648.2000.01567.x · Cited by 5,951 · cited 1×
“As such, Ludwig (1994) specifically addresses subject motivation as the key to the successful implementation of a Delphi study and investigators need to play an active role in this area to help ensure as high a response rate as possible.”
The Delphi Method: An Experimental Study of Group Opinion
(1969) · RAND Corporation eBooks · doi:10.7249/rm5888 · Cited by 1,628
Gazing into the oracle : the Delphi method and its application to social policy and public health
Ziglio, E., & Adler, M. E. (1996) · Jessica Kingsley Publishers eBooks · Cited by 1,429
Group Techniques for Program Planning
de Ven, A. V., & Delbecq, A. L. (1975) · Cited by 907
Technological forecasting for decision making
Martino, J. P. (1972) · Cited by 813
Experimental Study of Channel Patterns
SCHUMM, S. A., & KHAN, H. R. (1972) · Geological Society of America Bulletin · doi:10.1130/0016-7606(1972)83[1755:esocp]2.0.co;2 · Cited by 776
An experimental study of group opinion
Dalkey, N. (1969) · Futures · doi:10.1016/s0016-3287(69)80025-x · Cited by 750
Miller, G. (2001) · Tourism Management · doi:10.1016/s0261-5177(00)00067-4 · Cited by 604
Research works printed but not linked: 7
- Adams, S. J. (2001). Projecting the next decade in safety management: A Delphi technique study. Professional Safety, 46 (10), 26-29.
- Green, P. J. (1982, March). The content of a college-level outdoor leadership course. Paper presented at the Conference of the Northwest District Association for the American Alliance for Health, Physical Education, Recreation, and Dance, Spokane, WA.
- Jones, C. G. (1975). A Delphi evaluation of agreement between organizations. In H. A. Linstone, & M. Turoff (Eds.). The Delphi method: Techniques and applications (pp. 160-167). Reading, MA: Addison-Wesley Publishing Company.
- Kaplan, L. M. (1971). The use of the Delphi method in organizational communication: A case study. Unpublished master's thesis, The Ohio State University, Columbus.
- Linstone, H. A., & Turoff, M. (1975). Introduction. In H. A. Linstone, & M. Turoff (Eds.). The Delphi method: Techniques and applications (pp. 3-12). Reading, MA: Addison-Wesley Publishing Company.
- Ludlow, J. (1975). Delphi inquiries and knowledge utilization. In H. A. Linstone, & M. Turoff (Eds.). The Delphi method: Techniques and applications (pp. 102-123). Reading, MA: Addison-Wesley Publishing Company.
- Scheibe, M., Skutsch, M., & Schofer, J. (1975). Experiments in Delphi methodology. In H. A. Linstone, & M. Turoff (Eds.). The Delphi method: Techniques and applications (pp. 262-287). Reading, MA: Addison-Wesley Publishing Company.
Cited by
Items 501–600 of 1,013
Chen, L., Ng, E., Huang, S.-C., & Fang, W. (2017) · Sustainability · doi:10.3390/su9101751 · Cited by 58
“This study has adopted the Delphi method, which uses experts-level analysis beyond partial errors on criteria selection [104,105].”
MethodPolisena, J., Castaldo, R., Ciani, O., Federici, C., Borsci, S., Ritrovato, M., Clark, D. J., & Pecchia, L. (2018) · International Journal of Technology Assessment in Health Care · doi:10.1017/s0266462318000314 · Cited by 55
Laurie, S., & Mortimer, K. (2019) · Journal of Marketing Management · doi:10.1080/0267257x.2019.1576755 · Cited by 49
“The Delphi technique enables these people to contribute to a discussion and debate anonymously and with equal status, in short spurts of time and without leaving their desks (Hsu and Sandford, 2007; Yousuf, 2007). Wakefield and Watson (2013) refer to it as a ‘virtual focus group’ (p.”
BackgroundMethodInfluentialAgrawal, T. K., & Pal, R. (2019) · Sustainability · doi:10.3390/su11061698 · Cited by 81
“Traditionally, a Delphi study starts with an open-ended questionnaire; however, as a common and well-accepted modification in format, a structured or semi-structured questionnaire can be designed based on literature analysis (mentioned in [18]; practiced in [25]).”
BackgroundBhat, S., Gijo, E. V., Antony, J., & Cross, J. (2022) · The TQM Journal · doi:10.1108/tqm-10-2021-0302 · Cited by 65
Hospital preparedness and response in CBRN emergencies: TIER assessment tool
Olivieri, C., Ingrassia, P. L., Della Corte, F., Carenzo, L., Sapori, J.-M., Gabilly, L., Segond, F., Grieger, F., Arnod-Prin, P., Larrucea, X., et al. (2017) · European Journal of Emergency Medicine · doi:10.1097/mej.0000000000000399 · Cited by 48
Karabacak, B., Yildirim, S. O., & Baykal, N. (2016) · International Journal of Critical Infrastructure Protection · doi:10.1016/j.ijcip.2016.10.001 · Cited by 48
“Controlled opinion feedback was supplied to the respondents between the phases [14].”
MethodClifford, A. M., Ryan, J., Walsh, C., & McCurtin, A. (2017) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-017-0415-7 · Cited by 52
“Direction for a representative sample size was guided by recommendations for powering Delphi consensus, which suggests a minimum of 15 respondents [23].”
Application of neural network for forecasting of exchange rates and forex trading
Maknickienė, N., & Maknickas, A. (2012) · Business and management · doi:10.3846/bm.2012.017 · Cited by 48
“Jel classification: C15, C32, C53”
The Royal North Shore Hospital Emergency Department airway registry: Closing the audit loop
Fogg, T., Alkhouri, H., & Vassiliadis, J. (2015) · Emergency Medicine Australasia · doi:10.1111/1742-6723.12496 · Cited by 51
Willis, C. D., Greene, J. K., Abramowicz, A., & Riley, B. (2016) · Health Promotion and Chronic Disease Prevention in Canada · doi:10.24095/hpcdp.36.6.01 · Cited by 49
“Such prioritization processes may involve in-person workshops or modified Delphi processes for gathering large group perspectives.(54) 2.”
BackgroundDevelopment of an atrophic acne scar risk assessment tool
Tan, J. K. L., Thiboutot, D. M., Gollnick, H. P. M., Kang, S., Layton, A., Leyden, J. J., Torres, V., Guillemot, J., & Dréno, B. (2017) · Journal of the European Academy of Dermatology and Venereology · doi:10.1111/jdv.14325 · Cited by 59
Early-career researchers’ views on ethical dimensions of patient engagement in research
Bélisle‐Pipon, J., Rouleau, G., & Birko, S. (2018) · BMC Medical Ethics · doi:10.1186/s12910-018-0260-y · Cited by 54
“Three rounds were used: a ‘classic’ first round [19] of open-ended questions regarding panelists’ experience with PER and its definition, values, feasibility, and utility; a second round to deepen and clarify salient issues emerging from Round 1; and a third round consisting of four second-round questions”
MethodCassidy, J. T., Baker, J. F., & Lenehan, B. J. (2018) · Global Spine Journal · doi:10.1177/2192568217750125 · Cited by 48
“According to the SOSG, these 2 systematic reviews provided a framework for the development of the SINS via a systematic method of distillation of expert opinion known as the Delphi technique.(46) The 6 different parameters were assigned differing numerical grades with a minimum score being 0 and a maximum score being…”
MethodRogozinska, E., D'Amico, M., Khan, K., Cecatti, J., Teede, H., Yeo, S., Vinter, C., Rayanagoudar, G., Barakat, R., Perales, M., et al. (2016) · BJOG: An International Journal of Obstetrics & Gynaecology · doi:10.1111/1471-0528.13764 · Cited by 46
Designing micro-fintech models for Islamic micro financial institutions in Indonesia
Ascarya, A., & Sakti, A. (2022) · International Journal of Islamic and Middle Eastern Finance and Management · doi:10.1108/imefm-05-2020-0233 · Cited by 124
Sakhnini, S., & Blonder, R. (2015) · International Journal of Science Education · doi:10.1080/09500693.2015.1035687 · Cited by 60
“In order to present information concerning the collective judgments of the respondents (Hasson, Keeney, & McKenna, 2000) in the Delphi rounds, different descriptive statistical tests can be used (Hsu & Sandford, 2007).”
BackgroundMethodInfluentialSalavati, M., Krijnen, W. P., Rameckers, E. A. A., Looijestijn, P. L., Maathuis, C. G. B., van der Schans, C. P., & Steenbergen, B. (2015) · Research in Developmental Disabilities · doi:10.1016/j.ridd.2015.07.013 · Cited by 47
Cross-cultural validation of the Patient–Practitioner Orientation Scale (PPOS)
Pereira, C. M., Amaral, C. F. S., Ribeiro, M. M. F., Paro, H. B. M. D. S., Pinto, R. D. M. C., Reis, L. E. T., Silva, C. H. M., & Krupat, E. (2013) · Patient Education and Counseling · doi:10.1016/j.pec.2012.11.014 · Cited by 61
“Anonymous experts (reviewers) involved in the Delphi process receive feedback and statistical representation of their answers in the successive rounds [29,30].”
BackgroundMethodPatyk, K. A., Duncan, C., Nol, P., Sonne, C., Laidre, K. L., Obbard, M. E., Wiig, Ø., Aars, J., Regehr, E. V., Gustafson, L., et al. (2015) · The Science of The Total Environment · doi:10.1016/j.scitotenv.2015.02.007 · Cited by 55
Muller, B. G., van den Bos, W., Brausi, M. A., Cornud, F., Gontero, P., Kirkham, A. P. S., Pinto, P. A., Polascik, T. J., Rastinehad, A. R., de Reijke, T. M., et al. (2013) · British Journal of Urology · doi:10.1111/bju.12548 · Cited by 47
Prevention of Fetal Alcohol Spectrum Disorder: Current Canadian Efforts and Analysis of Gaps
Poole, N., Schmidt, R. A., Green, C., & Hemsing, N. (2016) · Substance Abuse Research and Treatment · doi:10.4137/sart.s34545 · Cited by 47
Identifying the features of an exercise addiction: A Delphi study
Macfarlane, L., Owens, G. R., & Cruz, B. D. P. (2016) · Journal of Behavioral Addictions · doi:10.1556/2006.5.2016.060 · Cited by 57
“…Phone: +64 9 373 7599 ext. 86875; Fax: +64 9 373 7902; E-mail: lucy. macfarlane@auckland.ac.nz This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium for non-commercial…”
BackgroundSobhanifard, Y. (2017) · British Food Journal · doi:10.1108/bfj-12-2016-0604 · Cited by 62
EAN guideline on palliative care of people with severe, progressive multiple sclerosis
Solari, A., Giordano, A., Sastre‐Garriga, J., Köpke, S., Rahn, A. C., Kleiter, I., Aleksovska, K., Battaglia, M. A., Bay, J., Copetti, M., et al. (2020) · European Journal of Neurology · doi:10.1111/ene.14248 · Cited by 59
“Agreement upon the direction (‘for’ or ‘against’) and the strength (‘strong’ or ‘weak’) of each recommendation was obtained via teleconferences and electronic discussions, using a modified Delphi method [26].”
Shoukohyar, S., & Seddigh, M. R. (2020) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2020.120059 · Cited by 50
Linking disaster risk reduction and human development
Raikes, J., Smith, T. F., Baldwin, C. L., & Henstra, D. (2021) · Climate Risk Management · doi:10.1016/j.crm.2021.100291 · Cited by 60
Criteria for Site Selection of Temporary Shelters after Earthquakes: a Delphi Panel
Soltani, A., Ardalan, A., Boloorani, A. D., Haghdoost, A. A., & Hosseinzadeh‐Attar, M. J. (2015) · PLoS Currents · doi:10.1371/currents.dis.07ae4415115b4b3d71f99ba8b304b807 · Cited by 46
van Leeuwen, M., Kieffer, J. M., Young, T., Annunziata, M. A., Arndt, V., Arrarás, J. I., Autran, D., Hani, H. B., Chakrabarti, M., Chinot, O. L., et al. (2022) · Journal of Cancer Survivorship · doi:10.1007/s11764-021-01160-1 · Cited by 53
The measurements of frailty and their possible application to spinal conditions: a systematic review
Moskven, E., Charest-Morin, R., Flexman, A. M., & Street, J. (2022) · The Spine Journal · doi:10.1016/j.spinee.2022.03.014 · Cited by 50
Preliminary psychometric scale development using the mixed methods Delphi technique
Dragostinov, Y., Harðardóttir, D., McKenna, P. E., Robb, D. A., Nesset, B., Ahmad, M., Romeo, M., Lim, M. Y., Yu, C., Jang, Y., et al. (2022) · Methods in Psychology · doi:10.1016/j.metip.2022.100103 · Cited by 69
Development of telehealth principles and guidelines for older adults: A modified Delphi approach
Wardlow, L., Leff, B. A., Biese, K., Roberts, C., Archbald‐Pannone, L. R., Ritchie, C. S., DeCherrie, L. V., Sikka, N. K., Gillespie, S. M., & Aging, T. C. F. T. A. (2022) · Journal of the American Geriatrics Society · doi:10.1111/jgs.18123 · Cited by 55
Knapp, S., Schweiger, O., Kraberg, A. C., Asmus, H., Asmus, R. M., Brey, T., Frickenhaus, S., Gutt, J., Kühn, I., Liess, M., et al. (2016) · The Science of The Total Environment · doi:10.1016/j.scitotenv.2016.09.002 · Cited by 50
Survey of diagnostic and treatment practices for multiple sclerosis in Europe
Fernández, Ó., Delvecchio, M., Edan, G., Fredrikson, S., Gionvannoni, G., Hartung, H., Havrdová, E., Kappos, L., Pozzilli, C., Soerensen, P. S., et al. (2017) · European Journal of Neurology · doi:10.1111/ene.13236 · Cited by 50
“consensus building that employs multiple rounds of questions and analysis of responses by a panel of subject experts to identify areas of agreement [5].”
BackgroundBennouna, C., Mansourian, H., & Stark, L. (2017) · Conflict and Health · doi:10.1186/s13031-017-0108-y · Cited by 54
Modeling and Simulation in Engineering Education: A Learning Progression
Magana, A. J. (2017) · Journal of Professional Issues in Engineering Education and Practice · doi:10.1061/(asce)ei.1943-5541.0000338 · Cited by 53
Sokol‐Hessner, L., Folcarelli, P. H., Annas, C. L., Brown, S. M., Fernández, L., Roche, S. D., Lee, B. S., Sands, K., Atlas, T., Benoît, D., et al. (2018) · The Joint Commission Journal on Quality and Patient Safety · doi:10.1016/j.jcjq.2018.02.003 · Cited by 49
Torrecilla-Salinas, C. J., Troyer, O. D., Escalona, M. J., & Mejías, M. (2018) · Information Technology and Management · doi:10.1007/s10799-018-0290-7 · Cited by 56
“It is important to note, as Hsu and Sandford [35] report, that starting round 1 directly with a well-structured questionnaire becomes an acceptable modification of the Delphi method.”
BackgroundMethodInfluentialEstablishing consensus of position-specific predictors for elite youth soccer in England
Roberts, S. J., McRobert, A. P., Lewis, C. J., & Reeves, M. J. (2019) · Science and Medicine in Football · doi:10.1080/24733938.2019.1581369 · Cited by 46
The Delphi Method in Information Systems Research (2004‑2017)
Alarabiat, A., & Ramos, I. M. P. (2019) · The Electronic Journal of Business Research Methods · doi:10.34190/jbrm.17.2.04 · Cited by 60
“Adding to this procedure, the selection of experts may be informed by an analysis of their scientific publications (Hsu and Sandford, 2007b).”
BackgroundMethodInfluentialCumin, J., Peláez, S., & Mottron, L. (2021) · Autism · doi:10.1177/13623613211042719 · Cited by 67
The Training Needs of Staff Supporting Individuals Ageing with Intellectual Disability
Wark, S., Hussain, R., & Edwards, H. E. (2014) · Journal of Applied Research in Intellectual Disabilities · doi:10.1111/jar.12087 · Cited by 51
Glackin, B. C., Rodenhiser, R. W., & Herzog, B. (2014) · The Journal of Academic Librarianship · doi:10.1016/j.acalib.2014.04.007 · Cited by 58
A Socio-technical Framework for Threat Modeling a Software Supply Chain
Sabbagh, B. A., & Kowalski, S. J. (2015) · IEEE Security & Privacy · doi:10.1109/msp.2015.72 · Cited by 45
Nguyen, T. K., Senan, S., Bradley, J. D., Franks, K., Giuliani, M., Gückenberger, M., Landis, M. S., Loo, B. W., Louie, A. V., Onishi, H., et al. (2017) · Practical Radiation Oncology · doi:10.1016/j.prro.2017.10.008 · Cited by 49
Ankolekar, A., Vanneste, B. G. L., Gurp, E. B., van Roermund, J. G. H., Limbergen, E. J. V., van de Beek, K., Marcelissen, T. A. T., Zambon, V., Oelke, M., Dekker, A. L. A. J., et al. (2019) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-019-0862-4 · Cited by 52
“Incorporating these varying perspectives and preferences may not be feasible, so at times developers may need to consult evidencebased guidelines or establish consensus through methods such as focus groups or a Delphi study [34].”
BackgroundFrench, S. D., Nielsen, M., Hall, L., Nicolson, P. J. A., van Tulder, M. W., Bennell, K. L., Hinman, R. S., Maher, C. G., Jull, G. A., & Hodges, P. W. (2019) · Pain · doi:10.1097/j.pain.0000000000001663 · Cited by 46
Xiao, Y., Chen, Z., & McNeil, L. (2021) · Maritime Policy & Management · doi:10.1080/03088839.2021.1894364 · Cited by 56
“The statistical analysis can offer evidence to ensure the credibility of the survey (Hsu and Sandford 2007; Arof 2015).”
BackgroundPaulsen, P., Smulders, F. J. M., Winkelmayer, R., Vodnansky, M., & Bauer, A. (2011) · Wageningen Academic Publishers eBooks · doi:10.3920/978-90-8686-723-3 · Cited by 45
Hanson, C. L., Oliver, E. J., Dodd-Reynolds, C. J., Pearsons, A., & Kelly, P. (2020) · International Journal of Behavioral Nutrition and Physical Activity · doi:10.1186/s12966-020-01050-2 · Cited by 56
Sodi, A., Banfi, S., Testa, F., Della Corte, M., Passerini, I., Pelo, E., Rossi, S., Simonelli, F., & Italian IRD Working Group (2021) · Orphanet Journal of Rare Diseases · doi:10.1186/s13023-021-01868-4 · Cited by 44
“The initial questionnaire was supported by a comprehensive review of the literature to identify the current understanding of the disease and the place of voretigene neparvovec gene supplementation in its treatment [35].”
MethodSimulated Placements as Partial Replacement of Clinical Training Time: A Delphi Consensus Study
Bridge, P., Adeoye, J., Edge, C., Garner, V., Humphreys, A., Ketterer, S.-J., Linforth, J., Manning-Stanley, A. S., Newsham, D., Prescott, D., et al. (2022) · Clinical Simulation in Nursing · doi:10.1016/j.ecns.2022.04.009 · Cited by 45
Revising the advanced access model pillars: a multimethod study
Bréton, M., Gaboury, I., Beaulieu, C., Sasseville, M., Hudon, C., Malham, S., Maillet, L., Duhoux, A., Rodrigues, I., & Haggerty, J. (2022) · CMAJ Open · doi:10.9778/cmajo.20210314 · Cited by 19
Pluemer, J., Freyvert, Y., Pratt, N., Robinson, J. E., Cooke, J. A., Tataryn, Z. L., Pierre, C. A., Godolias, P., Frieler, S., von Glinski, A., et al. (2022) · Journal of Neurosurgery Spine · doi:10.3171/2022.11.spine22719 · Cited by 44
Spinal Muscular Atrophy Update in Best Practices
Schroth, M. K., Deans, J., Arya, K., Castro, D., Vivo, D. C. D., Gibbons, M. A., Ionita, C., Kuntz, N. L., Lakhotia, A., Knierbein, E. E. N., et al. (2024) · Neurology Clinical Practice · doi:10.1212/cpj.0000000000200310 · Cited by 52
“Data were collected using 3 iterative online survey rounds 30 (Alchemer, Louisville, CO).”
Zlateva, I., Anderson, D., Coman, E., Khatri, K., Tian, T., & Fifield, J. (2015) · BMC Health Services Research · doi:10.1186/s12913-015-0893-1 · Cited by 53
Personal learning environments and university teacher roles explored using Delphi
Shaikh, Z. A., & Khoja, S. A. (2014) · Australasian Journal of Educational Technology · doi:10.14742/ajet.324 · Cited by 55
“These allow researchers to include open-ended questions, allow participants to give their input (suggestions, comments, addition, deletion of items), and are used in situations when stable results are also important besides achieving strong consensus (Hsu & Sandford, 2007; Na, 2006).”
BackgroundProt‐Labarthe, S., Weil, T., Angoulvant, F., Boulkedid, R., Alberti, C., & Bourdon, O. (2014) · PLoS ONE · doi:10.1371/journal.pone.0101171 · Cited by 56
“We validated the propositions included in POPI by a two round Delphi method [16,17].”
MethodMINIMIZING COMMUNICATION RISK IN CONSTRUCTION: A DELPHI STUDY OF THE KEY ROLE OF PROJECT MANAGERS
Cerić, A. (2014) · Journal of Civil Engineering and Management · doi:10.3846/13923730.2013.802739 · Cited by 59
Li, Y., Ehiri, J. E., Hu, D., Zhang, Y., Wang, Q., Zhang, S., & Cao, J. (2014) · BMC Infectious Diseases · doi:10.1186/1471-2334-14-268 · Cited by 50
“The Delphi process benefitted from the use of a survey instrument that was developed following a systematic review of the global TB control literature [11,47-60], review of government TB control program documents [28], local and international expert consultations.”
MethodLangereis, E. J., Borgo, A., Crushell, E., Harmatz, P., van Hasselt, P. M., Jones, S. A., Kelly, P. M., Lampe, C., van der Lee, J. H., Odent, T., et al. (2013) · Orphanet Journal of Rare Diseases · doi:10.1186/1750-1172-8-155 · Cited by 44
Abdullah, M. R. T. L., Hussin, Z. B., Asra, & Zakaria, A. R. (2013) · The University of Malaya Research Repository (University of Malaya) · Cited by 44
on behalf of the ROMIO study group, BROOKES, S. T., Chalmers, K. A., AVERY, K. N. L., Coulman, K. D., & Blazeby, J. M. (2018) · Trials · doi:10.1186/s13063-017-2405-6 · Cited by 52
A BSC-ANP approach to organizational outsourcing decision support-A case study
Modak, M., Ghosh, K. K., & Pathak, K. (2018) · Journal of Business Research · doi:10.1016/j.jbusres.2018.01.040 · Cited by 44
Anderson, A. J., Dixon, M. A., Oshiro, K. F., Wicker, P. V., Cunningham, G. B., & Heere, B. (2018) · Sport Management Review · doi:10.1016/j.smr.2018.06.015 · Cited by 62
Measuring Intercultural Learning through CLIL
Parra, M. E. G. (2020) · Journal of New Approaches in Educational Research · doi:10.7821/naer.2020.1.457 · Cited by 66
“The panel of experts evaluated the aforesaid tools by developing a series of alternatives, exploring or exposing underlying assumptions, as well as through the correlation of judgments on a topic spanning a wide range of disciplines (Hsu & Sandford, 2007).”
Basinska, K., Wellens, N. I. H., Simon, M., Zeller, A., Kressig, R. W., & Zúñiga, F. (2020) · Journal of Advanced Nursing · doi:10.1111/jan.14644 · Cited by 55
Operationalising elaboration theory for simulation instruction design: a Delphi study
Haji, F. A., Khan, R., Regehr, G., Ng, G., de Ribaupierre, S., & Dubrowski, A. (2015) · Medical Education · doi:10.1111/medu.12726 · Cited by 43
Comprehensive analysis of lifeboat accidents using the Fuzzy Delphi method
Danacı, M., & Yıldırım, U. (2023) · Ocean Engineering · doi:10.1016/j.oceaneng.2023.114371 · Cited by 45
Traynor, A. P., Boyle, C. J., & Janke, K. K. (2013) · American Journal of Pharmaceutical Education · doi:10.5688/ajpe7710221 · Cited by 46
Rowe, M., Frantz, J. M., & Bozalek, V. G. (2013) · BMC Medical Education · doi:10.1186/1472-6920-13-51 · Cited by 49
“There are no criteria upon which to determine the nature of the “expert”, the optimal panel size or even selection criteria of the panelists in a Delphi study [12].”
BackgroundStänder, S., Zeidler, C., Riepe, C., Steinke, S., Fritz, F., Bruland, P., Soto‐Rey, I., Storck, M., Agner, T., Augustin, M., et al. (2015) · Journal of the European Academy of Dermatology and Venereology · doi:10.1111/jdv.13296 · Cited by 55
Aesthetic Judgement and Visual Impact of Architectural Forms: A Study of Library Buildings
Jennath, A., & Nidhish, P. J. (2016) · Procedia Technology · doi:10.1016/j.protcy.2016.05.226 · Cited by 55
Miller, B. S., Turcu, A. F., Nanba, A. T., Hughes, D. T., Cohen, M. S., Gauger, P. G., & Auchus, R. J. (2017) · World Journal of Surgery · doi:10.1007/s00268-017-4311-1 · Cited by 47
“In 2016, the Primary Aldosteronism Surgical Outcomes (PASO) group, an international group of 31 leading experts in the field including six endocrine surgeons, established new definitions [24] for biochemical and clinical cure of PA using the Delphi method [25] of establishing consensus.”
BackgroundConceptual model of managing health care volunteers in disasters: a mixed method study
Salmani, I., Seyedin, H., Ardalan, A., & Farajkhoda, T. (2019) · BMC Health Services Research · doi:10.1186/s12913-019-4073-6 · Cited by 47
Cachecho, S., Elfassy, C., Hamdy, R. C., Rosenbaum, P., & Dahan‐Oliel, N. (2019) · American Journal of Medical Genetics Part C Seminars in Medical Genetics · doi:10.1002/ajmg.c.31739 · Cited by 43
“The modified Delphi method was selected over other consensus techniques because it overcomes geographical barriers and allows participants to remain anonymous and have equal opportunity to share without bias (Black et al., 1999; de Meyrick, 2003; Hsu & Sandford, 2007).”
MethodCurrie, C. C., Ohrbach, R. K., de Leeuw, R., Forssell, H., Imamura, Y., Jääskeläinen, S. K., Koutris, M., Nasri‐Heir, C., Tan, H. L., Renton, T. F., et al. (2020) · Journal of Oral Rehabilitation · doi:10.1111/joor.13123 · Cited by 49
Leadership behaviours that foster organisational identification during change
Aitken, K., & von Treuer, K. M. (2020) · Journal of Organizational Change Management · doi:10.1108/jocm-01-2020-0029 · Cited by 68
Durdyev, S., Mohandes, S. R., Mahdiyar, A., & Ismail, S. B. (2021) · Engineering Construction & Architectural Management · doi:10.1108/ecam-11-2020-0945 · Cited by 49
Riemens, J., Lemieux, A.-A., Lamouri, S., & Garnier, L. (2021) · Sustainability · doi:10.3390/su132111700 · Cited by 65
“Considering proper planning management and motivation are key when conducting a Delphi study [43], particular attention has been paid to the distinctive research steps, due to the wide variety of stakeholders involved in the panel.”
BackgroundMethodInfluentialExamining the OHS of green building construction projects: A hybrid fuzzy-based approach
Durdyev, S., Mohandes, S. R., Tokbolat, S., Sadeghi, H., & Zayed, T. M. (2022) · Journal of Cleaner Production · doi:10.1016/j.jclepro.2022.130590 · Cited by 43
Reliability and Educational Value of Laparoscopic Sleeve Gastrectomy Surgery Videos on YouTube
Toolabi, K., Parsaei, R., Elyasinia, F., & Zamanian, A. (2019) · Obesity Surgery · doi:10.1007/s11695-019-03907-3 · Cited by 42
Developing consensus in Histopathology: the role of the Delphi method
Taze, D., Hartley, C., Morgan, A. W., Chakrabarty, A., Mackie, S. L., & Griffin, K. J. (2022) · Histopathology · doi:10.1111/his.14650 · Cited by 53
“This provides an opportunity for the participants to reconsider their initial ranking if they wish to do so, especially if they are deemed to be an outlier for a given statement.(2,16) Researchers can expand their use by choosing to include “open text boxes” thereby enabling participants the option to add in comments…”
BackgroundMethodDigitalized transcranial electrical stimulation: A consensus statement
Brunoni, A. R., Ekhtiari, H., Antal, A., Auvichayapat, P., Baeken, C., Benseñor, I. M., Bikson, M., Boggio, P. S., Borroni, B., Brighina, F., et al. (2022) · Clinical Neurophysiology · doi:10.1016/j.clinph.2022.08.018 · Cited by 47
ACURATE: A guide for reporting sham controls in trials using acupuncture
Lee, Y., Kim, S., Lee, H., Chae, Y., & Lee, M. S. (2023) · Journal of Evidence-Based Medicine · doi:10.1111/jebm.12524 · Cited by 47
Best nutrition counseling practices for the treatment of anorexia nervosa: A Delphi study
Mittnacht, A. M., & Bulik, C. M. (2015) · International Journal of Eating Disorders · doi:10.1002/eat.22319 · Cited by 42
Schultz, H. Y., Dutz, J. P., Furukawa, F., Goodfield, M. J. D., Kuhn, A., Lee, L. A., Nyberg, F., Szepietowski, J. C., Sontheimer, R. D., & Werth, V. P. (2014) · Journal of Investigative Dermatology · doi:10.1038/jid.2014.316 · Cited by 42
“Dr. Joseph Merola (Harvard Medical School, Boston, MA) spoke about the Delphi technique (Dalkey and Helmer, 1963; Hsu and Sandford, 2007).”
BackgroundMcDonald, L., Hitzig, S. L., Pillemer, K. A., Lachs, M. S., Beaulieu, M. J., Brownell, P., Burnes, D. P. R., Caspi, E., Mont, J. D., Gadsby, R., et al. (2015) · Journal of Elder Abuse & Neglect · doi:10.1080/08946566.2014.995869 · Cited by 44
“As such, this feedback process allows and encourages persons participating in the Delphi process to reassess initial judgments about the information provided in previous iterations (Hsu & Sandford, 2007).”
BackgroundMethodInfluentialWells, C., Kolt, G. S., Marshall, P. W., & Bialocerkowski, A. (2014) · Physical Therapy · doi:10.2522/ptj.20130568 · Cited by 58
“21,31,32 Items of consensus were ranked in order of importance by a comparison of median scores for questions with a Likert response scale.”
InfluentialConsensus for Dry Needling for Plantar Heel Pain (Plantar Fasciitis): A Modified Delphi Study
Cotchett, M., Landorf, K. B., Munteanu, S. E., & Raspovic, A. (2011) · Acupuncture in Medicine · doi:10.1136/aim.2010.003145 · Cited by 45
“Therapists involved in the instruction and facilitation of dry needling courses were considered leaders in the profession with substantial knowledge of the study topic. In addition, the Delphi panellists also considered authors of peer-reviewed articles to be leaders within the profession, although there is no guideli…”
BackgroundNayahangan, L. J., Konge, L., Schroeder, T. V., Paltved, C., Lindorff‐Larsen, K., Nielsen, B. U., & Eiberg, J. (2017) · European Journal of Vascular and Endovascular Surgery · doi:10.1016/j.ejvs.2017.01.011 · Cited by 46
Regillo, C. D., Callanan, D., Diana, V., Fine, H. F., Holekamp, N. M., Kuppermann, B. D., Singer, M. A., & Singh, R. P. (2017) · Ophthalmic surgery, lasers & imaging retina · doi:10.3928/23258160-20170329-03 · Cited by 42
The direct and indirect costs of managing chronic obstructive pulmonary disease in Greece
Souliotis, K. N., Kousoulakou, H., Hillas, G., Tzanakis, Ν., Toumbis, M., & Vassilakopoulos, T. (2017) · International Journal of COPD · doi:10.2147/copd.s132825 · Cited by 42
“The data elicitation method was the Delphi technique, which aims at consensus building via multiple rounds of questionnaire completion.(12) Questionnaires were completed remotely, data collected upon completion of first round were analyzed, and results were presented to study participants during a face-to-face meeting.”
MethodIndonesian dengue burden estimates: review of evidence by an expert panel
Wahyono, T. Y. M., Nealon, J., Beucher, S., Prayitno, A., Moureau, A., Nawawi, S., Thabrany, H., & Nadjib, M. (2017) · Epidemiology and Infection · doi:10.1017/s0950268817001030 · Cited by 42
“stopped upon reaching predefined criteria [8].”
Solutions for counteracting human deception in social engineering attacks
Campbell, C. C. (2019) · Information Technology & People · doi:10.1108/itp-12-2017-0422 · Cited by 42
Health Care Transition Outcomes in Inflammatory Bowel Disease: A Multinational Delphi Study
van den Brink, G., van Gaalen, M. A. C., de Ridder, L., van der Woude, C. J., & Escher, J. C. (2019) · Journal of Crohn s and Colitis · doi:10.1093/ecco-jcc/jjz044 · Cited by 44
de Groot, N., Bonsel, G. J., Birnie, E., & Valentine, N. (2019) · PLoS ONE · doi:10.1371/journal.pone.0212633 · Cited by 45
Kruse, T. M., Veltri, A., & Branscum, A. J. (2019) · Journal of Safety Research · doi:10.1016/j.jsr.2019.10.005 · Cited by 54
Miller, R., Chrissian, A. A., Lee, G., Rahman, N. M., Wahidi, M. M., Tremblay, A., Hsia, D. W., Almeida, F. A., Shojaee, S., Mudambi, L., et al. (2020) · Journal of Bronchology & Interventional Pulmonology · doi:10.1097/lbr.0000000000000707 · Cited by 44
Ozair, M. M., Baharuddin, K. A., Mohamed, S. A., Esa, W., & Yusoff, M. S. B. (2017) · Education in Medicine Journal · doi:10.21315/eimj2017.9.2.1 · Cited by 63
Metadata
- Authors
- Hsu, C.-C., & Sandford, B. A.
- Venue
- University of Massachusetts Amherst
- Publisher
- University of Massachusetts Amherst
- Published
- 2007
- Volume
- 12
- Issue
- 10
- Pages
- 10
- OpenAlex
- W2583504027
- Cited by
- 4,500
- References
- 40
- Citations by year
- 2026: 160 · 2025: 361 · 2024: 381 · 2023: 406 · 2022: 373 · 2021: 381 · 2020: 383 · 2019: 375 · 2018: 358 · 2017: 287 · 2016: 298 · 2015: 240 · 2014: 176 · 2013: 129 · 2012: 91
- FWCI
- 516.01
- Type
- journal article
- Language
- en
- Open access
- Licence
- cc-by-nc-nd
- Locations
- ScholarWorks@UMassAmherst (University of Massachusetts Amherst) (submittedVersion, cc-by-nc-nd)
- University of Massachusetts (UMass) Amherst
- Practical assessment, research & evaluation
- Topics
- Social Sciences › Sociology and Political Science › Delphi Technique in Research
- Social Sciences › Decision Sciences › Management Science and Operations Research › Multi-Criteria Decision Making
- Social Sciences › Sociology and Political Science › Focus Groups and Qualitative Methods
- Keywords
- Delphi method, consensus building, expert opinion, participant selection, response rate, needs assessment, program planning, policy determination
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.