The Delphi Technique: Making Sense of Consensus
Open accessThe DOI matches its DataCite record; 40 of the 40 references it cites were found in a registry or an index.
Abstract
The Delphi technique is a widely used and accepted method for gathering data from respondents within their domain of expertise. The technique is designed as a group communication process which aims to achieve a convergence of opinion on a specific real-world issue. The Delphi process has been used in various fields of study such as program planning, needs assessment, policy determination, and resource utilization to develop a full range of alternatives, explore or expose underlying assumptions, as well as correlate judgments on a topic spanning a wide range of disciplines. The Delphi technique is well suited as a method for consensus-building by using a series of questionnaires delivered using multiple iterations to collect data from a panel of selected subjects. Subject selection, time frames for conducting and completing a study, the possibility of low response rates, and unintentionally guiding feedback from the respondent group are areas which should be considered when designing and implementing a Delphi study. Accessed 68,465 times on https://pareonline.net from August 30, 2007 to December 31, 2019. For downloads from January 1, 2020 forward, please click on the PlumX Metrics link to the right.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“factors with a level of agreement ≥ 70% were extracted and included in the present review [19].”
From Healthcare professionals’ perspectives on artificial intelligence in patient care: a systematic review of hindering and facilitating factors on different levels (2025) · cites this paper as [19]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“To narrow the list into a usable checklist, we then used a three-step Delphi survey process [31].”
From PRISMA-S: an extension to the PRISMA Statement for Reporting Literature Searches in Systematic Reviews (2021) · cites this paper as [31]
The cited source says what the claim attributes to it.
In the citing paper
“The iteration characteristics of the Delphi technique have intrinsic limitations and might lead to groupthink, where participants might conform to dominant opinions of the group, which could affect objectivity.197”
From Definition and diagnostic criteria of clinical obesity (2025) · cites this paper as [197]
In the citing paper
“This anonymous and iterative group technique is designed to gather individual opinions from experts and transform these into a group consensus 6.”
From Diagnosis and treatment of chronic bacterial prostatitis and chronic prostatitis/chronic pelvic pain syndrome: a consensus guideline (2015) · cites this paper as [6]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Foundations of Behavioral Research
Stockton, R., & Kerlinger, F. N. (1974) · American Educational Research Journal · doi:10.2307/1162206 · Cited by 7,347
An Experimental Application of the DELPHI Method to the Use of Experts
Dalkey, N., & Helmer, O. (1963) · Management Science · doi:10.1287/mnsc.9.3.458 · Cited by 6,061
Research guidelines for the Delphi survey technique
Hasson, F. (2000) · Journal of Advanced Nursing · doi:10.1046/j.1365-2648.2000.01567.x · Cited by 5,951 · cited 1×
“As such, Ludwig (1994) specifically addresses subject motivation as the key to the successful implementation of a Delphi study and investigators need to play an active role in this area to help ensure as high a response rate as possible.”
The Delphi Method: An Experimental Study of Group Opinion
(1969) · RAND Corporation eBooks · doi:10.7249/rm5888 · Cited by 1,628
Gazing into the oracle : the Delphi method and its application to social policy and public health
Ziglio, E., & Adler, M. E. (1996) · Jessica Kingsley Publishers eBooks · Cited by 1,429
Group Techniques for Program Planning
de Ven, A. V., & Delbecq, A. L. (1975) · Cited by 907
Technological forecasting for decision making
Martino, J. P. (1972) · Cited by 813
Experimental Study of Channel Patterns
SCHUMM, S. A., & KHAN, H. R. (1972) · Geological Society of America Bulletin · doi:10.1130/0016-7606(1972)83[1755:esocp]2.0.co;2 · Cited by 776
An experimental study of group opinion
Dalkey, N. (1969) · Futures · doi:10.1016/s0016-3287(69)80025-x · Cited by 750
Miller, G. (2001) · Tourism Management · doi:10.1016/s0261-5177(00)00067-4 · Cited by 604
Research works printed but not linked: 7
- Adams, S. J. (2001). Projecting the next decade in safety management: A Delphi technique study. Professional Safety, 46 (10), 26-29.
- Green, P. J. (1982, March). The content of a college-level outdoor leadership course. Paper presented at the Conference of the Northwest District Association for the American Alliance for Health, Physical Education, Recreation, and Dance, Spokane, WA.
- Jones, C. G. (1975). A Delphi evaluation of agreement between organizations. In H. A. Linstone, & M. Turoff (Eds.). The Delphi method: Techniques and applications (pp. 160-167). Reading, MA: Addison-Wesley Publishing Company.
- Kaplan, L. M. (1971). The use of the Delphi method in organizational communication: A case study. Unpublished master's thesis, The Ohio State University, Columbus.
- Linstone, H. A., & Turoff, M. (1975). Introduction. In H. A. Linstone, & M. Turoff (Eds.). The Delphi method: Techniques and applications (pp. 3-12). Reading, MA: Addison-Wesley Publishing Company.
- Ludlow, J. (1975). Delphi inquiries and knowledge utilization. In H. A. Linstone, & M. Turoff (Eds.). The Delphi method: Techniques and applications (pp. 102-123). Reading, MA: Addison-Wesley Publishing Company.
- Scheibe, M., Skutsch, M., & Schofer, J. (1975). Experiments in Delphi methodology. In H. A. Linstone, & M. Turoff (Eds.). The Delphi method: Techniques and applications (pp. 262-287). Reading, MA: Addison-Wesley Publishing Company.
Cited by
Items 801–900 of 1,013
Identifying Indicators Contributing to the Social Vulnerability Index via a Scoping Review
Kim, K.-M., Kang, J., & Hwang, C. (2025) · Land · doi:10.3390/land14020263 · Cited by 45
CHD and respiratory syncytial virus: global expert exchange recommendations
Tulloh, R. M. R., Medrano, C., Checchia, P. A., Stapper, C., Sumitomo, N., Gorenflo, M., Bae, E. J., Juanico-Enríquez, A., Gil‐Jaurena, J., Wu, M., et al. (2017) · Cardiology in the Young · doi:10.1017/s1047951117000609 · Cited by 33
“The Delphi method for consensus is a wellestablished process whereby a questionnaire is presented to a panel of experts to elicit their opinions, and then multiple iterations of review and refinement are used to build group agreement.(20,26) The classical Delphi method allows the respondents to remain anonymous to bot…”
MethodThienprayoon, R., Mark, M. S. J., & Grossoehme, D. H. (2017) · Journal of Palliative Medicine · doi:10.1089/jpm.2017.0333 · Cited by 34
“The list of remaining domains, their ratings, minority opinions, and items achieving consensus was distributed to panelists.(9) Participants reflected upon their responses in relation to the group.”
Development and validation of oral health‐related quality of life measure in oral submucous fibrosis
Gondivkar, S. M., Bhowate, R. R., Gadbail, A. R., Gaikwad, R. N., Gondivkar, R., Sarode, S. C., & Sarode, G. (2018) · Oral Diseases · doi:10.1111/odi.12857 · Cited by 40
Blieck, Y., Ooghe, I., Zhu, C., DePryck, K., Struyven, K., Pynoo, B., & Laer, H. V. (2018) · Studies in Continuing Education · doi:10.1080/0158037x.2018.1457023 · Cited by 37
Sii, S. S. Z., Barton, K., Pasquale, L. R., Yamamoto, T., King, A. J., & Azuara‐Blanco, A. (2018) · American Journal of Ophthalmology · doi:10.1016/j.ajo.2018.07.014 · Cited by 32
The Most Important Therapy Outcomes for School-Aged Children Who Stutter: An Exploratory Study
Cooke, K., & Millard, S. K. (2018) · American Journal of Speech-Language Pathology · doi:10.1044/2018_ajslp-odc11-17-0195 · Cited by 37
Schmidt, M. W., Kowalewski, K., Schmidt, M., Wennberg, E., Garrow, C. R., Paik, S. P., Benner, L., Schijven, M. P., Müller-Stich, B. P., & Nickel, F. (2018) · Surgical Endoscopy · doi:10.1007/s00464-018-6480-x · Cited by 34
van der Lee, L., Hill, A., & Patman, S. M. (2018) · Journal of Evaluation in Clinical Practice · doi:10.1111/jep.13077 · Cited by 31
Outsourcing in humanitarian logistics – status quo and future directions
Gossler, T., Wakolbinger, T., & Burkart, C. (2020) · International Journal of Physical Distribution & Logistics Management · doi:10.1108/ijpdlm-12-2018-0400 · Cited by 38
Common Clinical Practice for Low Back Pain Treatment: A Modified Delphi Study
Varrassi, G., Moretti, B., Pace, M. C., Evangelista, P., & Iolascon, G. (2021) · Pain and Therapy · doi:10.1007/s40122-021-00249-w · Cited by 49
“Similar methods have been applied in previous studies to achieve consensus of opinion in a real-world context, including in health care environments [12, 13].”
ResultIntelligibility and comprehensibility: A Delphi consensus study
Pommée, T., Balaguer, M., Mauclair, J., Pinquier, J., & Woisard, V. (2021) · International Journal of Language & Communication Disorders · doi:10.1111/1460-6984.12672 · Cited by 57
Padayachee, K. (2016) · Decision Support Systems · doi:10.1016/j.dss.2016.09.012 · Cited by 40
Validity of the Model for Assessment of Telemedicine: A Delphi study
Kidholm, K., Jensen, L. K., Kjølhede, T., Nielsen, E., & Hørup, M. B. (2016) · Journal of Telemedicine and Telecare · doi:10.1177/1357633x16686553 · Cited by 35
Kegelaers, J. J., Wylleman, P., Brandt, K. D., Rossem, N. V., & Rosier, N. (2018) · European Sport Management Quarterly · doi:10.1080/16184742.2017.1384505 · Cited by 35
“The Delphi method was used as it is well suited to reach consensus on a certain topic among a group of experts (Hsu & Sandford, 2007).”
MethodOn Modeling Linked Open Statistical Data
Kalampokis, E., Zeginis, D., & Tarabanis, K. A. (2018) · Journal of Web Semantics · doi:10.1016/j.websem.2018.11.002 · Cited by 31
Embedding Sustainability in Tourism Education: Bridging Curriculum Gaps for a Sustainable Future
Ara, A. F.-V., Guereño-Omil, B., & Varela, N. A. (2024) · Sustainability · doi:10.3390/su16219286 · Cited by 48
Pincombe, B. M., Blunden, S. L., Pincombe, A. H., & Dexter, P. (2012) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2012.05.001 · Cited by 32
“Most Delphi processes have a qualitative stopping condition: consensus is considered to have occurred when moderators assess expert responses to have changed little between rounds [54-56].”
BackgroundNaweed, A., Rainbird, S., & Chapman, J. (2015) · Ergonomics · doi:10.1080/00140139.2014.1001448 · Cited by 33
“Initial findings were then refined into overarching groupings and discussed by the researchers using the Delphi method, allowing them to agree coding definitions (Hsu 2007; Powell 2003).”
MethodApplication of the Delphi technique in healthcare maintenance
Njuangang, S., Liyanage, C., & Akintoye, A. (2017) · International Journal of Health Care Quality Assurance · doi:10.1108/ijhcqa-02-2017-0042 · Cited by 41
Khalil, A. A., Perry, H., Duffy, J. M., Reed, K., Baschat, A., Deprest, J. A., Hecher, K., Lewi, L., Lopriore, E., & Oepkes, D. (2017) · Trials · doi:10.1186/s13063-017-2042-0 · Cited by 31
“Web-based Delphi tools facilitate international data collection and are largely considered acceptable to the user [21, 22].”
BackgroundIzaryk, K., & Skarakis‐Doyle, E. (2017) · American Journal of Speech-Language Pathology · doi:10.1044/2017_ajslp-16-0046 · Cited by 37
Bougie-assisted cricothyroidotomy: Delphi-derived essential steps for the novice learner
Dharamsi, A., Gray, S. H., Hicks, C. M., Sherbino, J., McGowan, M., & Petrosoniak, A. (2018) · Canadian Journal of Emergency Medicine · doi:10.1017/cem.2018.386 · Cited by 29
Nace, D. A., Perera, S., Hanlon, J. T., Saracco, S., Anderson, G., Schweon, S. J., Klein‐Fedyshin, M., Wessel, C. B., Mulligan, M. A., Drinka, P. J., et al. (2018) · Journal of the American Medical Directors Association · doi:10.1016/j.jamda.2018.05.030 · Cited by 33
“[26, 27] The Delphi panel was comprised of a national group of skilled and practicing PA/LTC physicians.”
BackgroundDauncey, J. W., Olupot‐Olupot, P., & Maitland, K. (2019) · BMC Health Services Research · doi:10.1186/s12913-019-4129-7 · Cited by 33
“The Delphi technique and member checking would have a been useful techniques to establish consensus responses, strengthening conclusions and enhancing inter-disciplinary view-points [29], but this was not undertaken in our study.”
MethodManagement of Open Innovation in Healthcare for Cost Accounting Using EHR
Biancone, P., Secinaro, S. F., Brescia, V., & Calandra, D. (2019) · Journal of Open Innovation Technology Market and Complexity · doi:10.3390/joitmc5040099 · Cited by 40
“The Delphi method is applied in the first phase, defining the structure of the interview [70,71].”
MethodL&D professionals in organisations: much ambition, unfilled promise
Garavan, T. N., Heneghan, S., O’Brien, F., Gubbins, C., Lai, Y., Carbery, R., Duggan, J., Lannon, R., Sheehan, M., & Grant, K. (2019) · European journal of training and development · doi:10.1108/ejtd-09-2019-0166 · Cited by 46
Cruz-Martínez, R. R., Wentzel, J., Asbjørnsen, R. A., Noort, P. D., van Niekerk, J. M., Sanderman, R., & van Gemert‐Pijnen, J. E. W. C. (2020) · Journal of Medical Internet Research · doi:10.2196/16157 · Cited by 35
Competency Gap in the Labor Market: Evidence from Vietnam
Le, Q. T.-T., Tam Ho Dan, D. O. A. N., Nguyen, Q. L. H. T. T., & NGUYEN, D. T. P. (2020) · Journal of Asian Finance Economics and Business · doi:10.13106/jafeb.2020.vol7.no9.697 · Cited by 36
“This is the technique used for consensus achievement (Hsu & Sandford, 2007; Kembro, Näslund, & Olhager, 2017).”
MethodGuerreiro, M. P., Strawbridge, J., Cavaco, A. M. D. N., Félix, I. B., Marques, M. M., & Cadogan, C. A. (2021) · BMC Medical Education · doi:10.1186/s12909-021-02720-w · Cited by 34
“The Delphi technique is a widely used method for achieving consensus of opinion from experts within a particular field [19].”
MethodThe benefits and harms of open notes in mental health: A Delphi survey of international experts
Blease, C. R., Kharko, A., Hägglund, M., O’Neill, S. F., Wachenheim, D., Salmi, L., Harcourt, K., Locher, C., DesRoches, C. M., & Torous, J. B. (2021) · PLoS ONE · doi:10.1371/journal.pone.0258056 · Cited by 30
Voûte, M. T., Stathis, A. O., Schneider, P. A., Thomas, S. D., Brodmann, M., Armstrong, E. J., Holden, A. H., & Varcoe, R. L. (2021) · JACC: Cardiovascular Interventions · doi:10.1016/j.jcin.2021.07.056 · Cited by 30
Medical Education e-Professionalism (MEeP) framework; from conception to development
Guraya, S. S., Frcs, S. Y. G., Harkin, D. W., Ryan, Á., Nor, M. Z. M., & Yusoff, M. S. B. (2021) · Medical Education Online · doi:10.1080/10872981.2021.1983926 · Cited by 34
“Delphi technique Delphi technique collects anonymous opinions from a panel of experts to forecast future trends on a given research topic [36].”
MethodLearning-based dynamic capabilities in closed-loop supply chains: an expert study
Ritola, I., Krikke, H. R., & Caniëls, M. C. J. (2022) · The International Journal of Logistics Management · doi:10.1108/ijlm-01-2021-0044 · Cited by 30
Ogunbayo, B. F., Aigbavboa, C. O., Thwala, W. D., & Akinradewo, O. I. (2022) · Built Environment Project and Asset Management · doi:10.1108/bepam-06-2021-0080 · Cited by 30
Chatzitheodoridis, F., Melfou, K., Kontogeorgos, A., & Kalogiannidis, S. (2022) · Smart Cities · doi:10.3390/smartcities6010002 · Cited by 34
A Consensus Definition and Core Competencies for Being an Advocate for Pharmacy
Bzowyckyj, A. S., & Janke, K. K. (2013) · American Journal of Pharmaceutical Education · doi:10.5688/ajpe77224 · Cited by 33
Vasopressor and Inotrope Use in Canadian Emergency Departments: Evidence Based Consensus Guidelines
Djogovic, D., MacDonald, S., Wensel, A., Green, R., Loubani, O. M., Archambault, P. M., Bordeleau, S., Messenger, D. W., Szulewski, A., Davidow, J., et al. (2015) · Canadian Journal of Emergency Medicine · doi:10.1017/cem.2014.77 · Cited by 30
Measuring the value of public libraries in the digital age: What the power people need to know
Halpin, E., Rankin, C., Chapman, E. L., & Walker, C. (2013) · Journal of Librarianship and Information Science · doi:10.1177/0961000613497746 · Cited by 46
Connor, J. A., LaGrasta, C., & Hickey, P. A. (2015) · American Journal of Critical Care · doi:10.4037/ajcc2015230 · Cited by 30
Hromi‐Fiedler, A. J., Buccini, G., Gubert, M. B., Doucet, K., & Pérez‐Escamilla, R. (2018) · Maternal and Child Nutrition · doi:10.1111/mcn.12659 · Cited by 31
A Taxonomy of Arts Interventions for People With Dementia
Cousins, E. M., Tischler, V., Garabedian, C. E., & Dening, T. (2019) · The Gerontologist · doi:10.1093/geront/gnz024 · Cited by 43
An ERM Maturity ModelPreprint
Monda, B., & Giorgino, M. (2013) · SSRN Electronic Journal · doi:10.2139/ssrn.2198944 · Cited by 34
Pillemer, K. A., Chen, E. K., Riffin, C. A., Prigerson, H. G., & Reid, M. C. (2015) · American Journal of Public Health · doi:10.2105/ajph.2015.302675 · Cited by 33
Identifying content for simulation-based curricula in urology: a national needs assessment
Nayahangan, L. J., Hansen, R. B., Lindorff‐Larsen, K., Paltved, C., Nielsen, B. U., & Konge, L. (2017) · Scandinavian Journal of Urology · doi:10.1080/21681805.2017.1352618 · Cited by 29
“The Delphi technique is widely used as a method to achieve convergence of opinion from a panel of selected participants [10].”
MethodDetermining procedures for simulation-based training in radiology: a nationwide needs assessment
Nayahangan, L. J., Nielsen, K. R., Albrecht‐Beste, E., Nielsen, M. B., Paltved, C., Lindorff‐Larsen, K., Nielsen, B. U., & Konge, L. (2018) · European Radiology · doi:10.1007/s00330-017-5244-7 · Cited by 33
“E-mails were sent to each participant to avoid the influence of dominant individuals [12].”
BackgroundHIDRAdisk: an innovative visual tool to assess the burden of hidradenitis suppurativa
Chiricozzi, A., Bettoli, V., Pità, O. D., Dini, V., Fabbrocini, G., Monfrecola, G., Musumeci, M. L., Parodi, A., Sampogna, F., Pennella, A., et al. (2018) · Journal of the European Academy of Dermatology and Venereology · doi:10.1111/jdv.15122 · Cited by 29
Validation of a Classroom Management Questionnaire for pre and Inservice Teachers of English
Díaz, C., González, G., Ramírez, L. J., & Parra, J. M. (2018) · Revista Colombiana de Educación · doi:10.17227/rce.num75-8111 · Cited by 29
Development and refinement of three animal-based broiler chicken welfare indicators
Souza, A., Soriano, V., Schnaider, M. A., Rucinque, D. S., & Molento, C. F. M. (2018) · Animal Welfare · doi:10.7120/09627286.27.3.263 · Cited by 29
Counseling Competencies in Audiology: A Modified Delphi Study
Meibos, A., Muñoz, K. F., & Twohig, M. P. (2019) · American Journal of Audiology · doi:10.1044/2018_aja-18-0141 · Cited by 37
“To account for possible attrition between rounds, a target sample of 30 experts was chosen to stay within the recommended expectations of 10 to 15 panel members for a homogenous panel (Bardecki, 1984; Clayton, 1997; Hsu & Sandford, 2007b).”
Ramesh, G. N., Hunt, R. H., Лазебник, Л. Б., Marakhouski, Y., Mănuc, M., Aye, K. S., Bordin, D. S., Бакулина, Н. В., Искаков, Б. С., Хамраев, А. А., et al. (2018) · Euroasian Journal of Hepato-Gastroenterology · doi:10.5005/jp-journals-10018-1281 · Cited by 31
Core set of unfavorable events of shoulder arthroplasty: an international Delphi consensus process
Audigé, L., Schwyzer, H., Äärimaa, V., Alta, T. D. W., Amaral, M. V. G., Armstrong, A., van Noort, A., Bale, S., Beyth, S., Bischof, A., et al. (2019) · Journal of Shoulder and Elbow Surgery · doi:10.1016/j.jse.2019.07.021 · Cited by 29
Design and Validation of Mathematical Literacy Instruments for Assessment for Learning in Indonesia
Suciati, S., Munadi, S., Sugiman, S., & Febriyanti, W. D. R. (2020) · European Journal of Educational Research · doi:10.12973/eu-jer.9.2.865 · Cited by 29
Developing Strategy: A Guide For Nurse Managers to Manage Nursing Staff's Work-related Problems
Allah, A. R. G., Elshrief, H., & Ageiz, M. H. (2020) · Asian Nursing Research · doi:10.1016/j.anr.2020.07.004 · Cited by 29
Defining a humanitarian shelter and settlements research agenda
Opdyke, A., Goldwyn, B., & Javernick‐Will, A. N. (2020) · International Journal of Disaster Risk Reduction · doi:10.1016/j.ijdrr.2020.101950 · Cited by 32
“The Delphi technique is particularly helpful for goal setting and addressing strategic objectives [42].”
MethodAI adoption in the hiring process – important criteria and extent of AI adoption
Bhatt, P. (2022) · foresight · doi:10.1108/fs-07-2021-0144 · Cited by 48
The Influence of International Agreements on Disaster Risk Reduction
Raikes, J., Smith, T. F., Baldwin, C. L., & Henstra, D. (2022) · International Journal of Disaster Risk Reduction · doi:10.1016/j.ijdrr.2022.102999 · Cited by 36
Identifying Biomarkers of Spinal Muscular Atrophy for Further Development
Glascock, J., Darras, B. T., Crawford, T. O., Sumner, C. J., Kolb, S. J., DiDonato, C. J., Elsheikh, B., Howell, K., Farwell, W. R., Valente, M., et al. (2023) · Journal of Neuromuscular Diseases · doi:10.3233/jnd-230054 · Cited by 31
“Surveys and phone calls were reiterative in order to gain increasing consensus with each discussion [144]; initial, broad questions were designed to evoke more specific follow-up questions.”
BackgroundVictar, H. C., Perera, B. A. K. S., Palihakkara, A. D., & Dewagoda, K. G. (2023) · Smart and Sustainable Built Environment · doi:10.1108/sasbe-10-2022-0231 · Cited by 38
Challinor, J. M., Hollis, R., Freidank, C., & Verhoeven, C. (2013) · Cancer Nursing · doi:10.1097/ncc.0000000000000100 · Cited by 32
Kim, M. K. (2011) · Computers in Human Behavior · doi:10.1016/j.chb.2011.11.018 · Cited by 30
Nanotechnology applications as a context for teaching the essential concepts of NST
Sakhnini, S., & Blonder, R. (2016) · International Journal of Science Education · doi:10.1080/09500693.2016.1152518 · Cited by 30
“The Delphi technique is a multiple-iteration procedure for obtaining the most reliable consensus from a group of experts without the need for actual meeting of the expert group (Hsu & Sandford, 2007).”
InfluentialFrankena, T. K., Naaldenberg, J., Cardol, M., Meijering, J. V., Leusink, G. L., & Valk, H. M. J. V. S. L. (2016) · BMJ Open · doi:10.1136/bmjopen-2016-011861 · Cited by 37
“All quantitative analyses were conducted in SPSS Statistics 20.”
Leading for safety: A weighted safety leadership model in shipping
Kim, T.-E., & Gausdal, A. H. (2017) · Reliability Engineering & System Safety · doi:10.1016/j.ress.2017.05.002 · Cited by 39
Finn, G. M., Hitch, G., Apampa, B., Hennessy, C., Smith, C. F., Stewart, J., & Gard, P. R. (2018) · Journal of Anatomy · doi:10.1111/joa.12787 · Cited by 29
Pediatric Emergency Care Research Networks: A Research Agenda
Stoner, M. J., Mahajan, P., Bressan, S., Lam, S. H. F., Chumpitazi, C. E., Kornblith, A. E., Linakis, S. W., Roland, D., Freedman, S. B., Nigrovic, L. E., et al. (2018) · Academic Emergency Medicine · doi:10.1111/acem.13656 · Cited by 29
Mansour, J. C., Chavin, K. D., Morris‐Stiff, G. J., Warner, S. G., Cardona, K., Fong, Z. V., Maker, A. V., Libutti, S. K., Warren, R. S., Hill, C. R. S., et al. (2018) · HPB · doi:10.1016/j.hpb.2018.09.020 · Cited by 29
Jandhyala, R. (2020) · Current Medical Research and Opinion · doi:10.1080/03007995.2020.1734920 · Cited by 29
Wong, E. C. H., Lee, K., & Tong, M. C. F. (2020) · American Journal of Speech-Language Pathology · doi:10.1044/2019_ajslp-19-00118 · Cited by 35
Ganisen, S., Mohammad, I. S., Jawahar Nesan, L., Mohammed, A. H., & Kanniyapan, G. (2015) · Jurnal Teknologi · doi:10.11113/jt.v77.6871 · Cited by 28
“Regarding any set standards of selecting Delphi panel of experts, there is, in fact, no exact criterion currently listed in the literature concerning the selection of Delphi panellists [62].”
BackgroundEducator's blueprint: A primer on consensus methods in medical education research
Gottlieb, M. A., Caretta‐Weyer, H. A., Chan, T. M., & Humphrey-Murto, S. M. (2023) · AEM Education and Training · doi:10.1002/aet2.10891 · Cited by 37
Sarr, R., Spain, D., Quinton, A. M. G., Happe, F. G. E., Brewin, C. R., Radcliffe, J., Jowett, S., Miles, S., González, R. A., Albert, I., et al. (2024) · British Journal of Psychology · doi:10.1111/bjop.12731 · Cited by 36
“Delphi studies have a minimum of 10 to 15 participants in the first round, relying on the quality of the expertise of participants over the quantity (Hsu & Sandford, 2007).”
BackgroundMethodInfluentialYu, Y., Sun, X., Zhuang, Y., Dong, X., Liu, H., Jiang, P., Yu, Z., & Zhang, Y. (2011) · The International Journal of Health Planning and Management · doi:10.1002/hpm.1117 · Cited by 28
Expert Perspectives on the Future of Physical Education in Higher Education
Lorusso, J. R., & Richards, K. A. R. (2017) · Quest · doi:10.1080/00336297.2017.1359789 · Cited by 28
“…synthesized discussion is then fed-back to participants in an anonymous format and participants are asked to respond by indicating their agreement and/or disagreement with the items, or by offering narrative explanations for their positions that differ greatly from the group (Hsu & Sandford, 2007).”
BackgroundMethodMaking a Wiser Crowd: Benefits of Individual Metacognitive Control on Crowd Performance
Bennett, S. T., Benjamin, A. S., Mistry, P. K., & Mark Steyvers, B. (2018) · Computational Brain & Behavior · doi:10.1007/s42113-018-0006-4 · Cited by 30
Areas of agreement in the management of childhood non-infectious chronic anterior uveitis in the UK
Solebo, A. L., Rahi, J. S., Dick, A. D., Ramanan, A. V., Ashworth, J. L., & Edelsten, C. (2019) · British Journal of Ophthalmology · doi:10.1136/bjophthalmol-2018-313789 · Cited by 28
Core outcome set for studies investigating management of selective fetal growth restriction in twins
Townsend, R., Duffy, J. M., Sileo, F. G., Perry, H., Ganzevoort, W., Reed, K., Baschat, A., Deprest, J. A., Gratacós, E., Hecher, K., et al. (2019) · Ultrasound in Obstetrics and Gynecology · doi:10.1002/uog.20388 · Cited by 31
Priorities for peri‐operative research in Africa
Biccard, B. M. (2020) · Anaesthesia · doi:10.1111/anae.14934 · Cited by 35
Sawhney, M., Bilal, M., Pohl, H., Kushnir, V. M., Khashab, M. A., Schulman, A. R., Berzin, T. M., Chahal, P., Muthusamy, V. R., Varadarajulu, S. S., et al. (2020) · Gastrointestinal Endoscopy · doi:10.1016/j.gie.2020.05.014 · Cited by 29
“The Delphi method is a validated and structured technique to obtain expert consensus, and it is particularly well suited for the present situation where there is limited outcome data, and guidance for procedural timing is urgently needed.(9,10) Conducting new studies to assess outcomes related to delaying procedures a…”
MethodEstablishing Consensus for Essential Elements in Returning to Learn Following a Concussion
McAvoy, K., Eagan‐Johnson, B., Dymacek, R., Hooper, S. R., McCart, M., & Tyler, J. S. (2020) · Journal of School Health · doi:10.1111/josh.12949 · Cited by 36
My home is your castle: forecasting the future of accommodation sharing
Gassmann, S.-E., Nunkoo, R., Tiberius, V., & Kraus, S. (2021) · International Journal of Contemporary Hospitality Management · doi:10.1108/ijchm-06-2020-0596 · Cited by 33
Delivering Tourism Sustainability and Competitiveness in Seaside and Marine Resorts with GIS
Jurkus, E., Taminskas, J., Povilanskas, R., Kontautienė, V., Baltranaitė, E., Dailidė, R., & Urbis, A. (2021) · Journal of Marine Science and Engineering · doi:10.3390/jmse9030312 · Cited by 35
Orrego, C., Ballester, M., Heymans, M., García, E. C., Groene, O., de Guzmán, E. N., Hernandez, H. P., Suñol, R., & COMPAR‐EU Group (2021) · Health Expectations · doi:10.1111/hex.13303 · Cited by 28
Gondi, S., Ellis, S. D., Gupta, M., Ellerbeck, E. F., Richter, K. P., Burns, J. M., & Gupta, A. (2021) · PLoS ONE · doi:10.1371/journal.pone.0255578 · Cited by 32
Preoperative Management of Medications for Psychiatric Diseases
Oprea, A. D., Keshock, M. C., O’Glasser, A. Y., Cummings, K. C., Edwards, A. F., Zimbrean, P. C., Urman, R. D., & Mauck, K. F. (2022) · Mayo Clinic Proceedings · doi:10.1016/j.mayocp.2021.11.011 · Cited by 34
Maguire, T., Ryan, J., Fullam, R. S., & McKenna, B. (2022) · Journal of Psychiatric and Mental Health Nursing · doi:10.1111/jpm.12827 · Cited by 31
“One potential limitation is the arbitrary setting of the consensus measure at 70% of the participants; however, there is evidence to suggest that agreement at 70% and above is sufficient (Feo et al., 2018; Hsu & Sandford, 2007).”
BackgroundComer, C. M., Ammendolia, C., Battié, M. C., Bussières, A., Fairbank, J. C. T., Haig, A. J., Melloh, M., Redmond, A. C., Schneider, M., Standaert, C. J., et al. (2022) · BMC Musculoskeletal Disorders · doi:10.1186/s12891-022-05485-5 · Cited by 34
Wei, J., Fang, X., Qiao, J., Liu, H., Cui, H., Wei, Y., Xiao-hong, J., Xu, B. L., Han, Q., & Jing, X. (2022) · Nurse Education in Practice · doi:10.1016/j.nepr.2022.103452 · Cited by 31
Kim, M. K. (2012) · Educational Technology Research and Development · doi:10.1007/s11423-012-9247-4 · Cited by 34
“In particular the Delphi survey procedure was employed (Goodman 1987; Hsu and Sandford 2007; Okoli and Pawlowski 2004).”
MethodYang, H., Khin, L., Kennedy, O. R., & Banomyong, R. (2011) · Journal of Service Science and Management · doi:10.4236/jssm.2011.43045 · Cited by 28
“Delphi Technique Delphi, a systematic interactive forecasting technique which depends on a panel of independent, carefully selected expert respondents [21], was used to identify characteristics of efficient sea-port operations.”
BackgroundMethodInfluentialBuilding an ontology for assistive technology using the Delphi method
Danial‐Saad, A., Kuflik, T., Weiss, P. L., & Schreuer, N. (2012) · Disability and Rehabilitation Assistive Technology · doi:10.3109/17483107.2012.723238 · Cited by 31
“This allowed validation and rating of the domainontology of both the user’s profile of characteristics and the pointing device features [37,67].”
MethodRao, J. K., Anderson, L. A., Sukumar, B., Beauchesne, D. A., Stein, T., & Frankel, R. M. (2010) · BMC Health Services Research · doi:10.1186/1472-6963-10-97 · Cited by 31
“This method was first developed in the early 1950s as a tool for setting military priorities and since then, has been used to solve a variety of problems, such as helping groups develop educational priorities, performance indicators, and treatment guidelines [ 12-16 ].”
BackgroundMethodInfluentialHvidt, E. A., Hansen, D. G., Ammentorp, J., Bjerrum, L., Cold, S., Gulbrandsen, P., Olesen, F., Pedersen, S. S., Søndergaard, J., Timmermann, C., et al. (2017) · European Journal of General Practice · doi:10.1080/13814788.2017.1326479 · Cited by 32
Lyon, A. R., Lewis, C. C., Melvin, A., Boyd, M. R., Nicodimos, S., Liu, F. F., & Jungbluth, N. J. (2015) · Implementation Science · doi:10.1186/s13012-016-0495-2 · Cited by 30
Leahy, M. J., Valle, R. J. D., Landon, T. J., Iwanaga, K., Sherman, S. G., Reyes, A., & Chan, F. H. (2018) · Journal of Vocational Rehabilitation · doi:10.3233/jvr-170914 · Cited by 32
“The use of these iterative procedures is common in Delphi studies to achieve group consensus among experts, with consensus being reflected in the convergence of variance or standard deviations in subsequent iterations (Hsu & Standford, 2007).”
Consensus minimum data set for lung cancer multidisciplinary teams: Results of a Delphi process
Stone, E. C. A., Rankin, N. M., Phillips, J. L., Fong, K. M., Currow, D. C., Miller, A., Largey, G., Zielinski, R., Flynn, P., & Shaw, T. J. (2018) · Respirology · doi:10.1111/resp.13307 · Cited by 28
Poulin, P. A., Shergill, Y., Romanow, H. C., Busse, J. W., Chambers, C. T., Cooper, L., Forgeron, P. A., Harper, A. O., Hudspith, M., Iorio, A., et al. (2018) · Canadian Journal of Pain · doi:10.1080/24740527.2018.1433959 · Cited by 33
A Critical Analysis of How Central European Spas Create Health Tourism Experiencescapes
Dryglas, D., & Smith, M. (2023) · Tourism Planning & Development · doi:10.1080/21568316.2023.2259357 · Cited by 35
“The first Delphi round is generally unstructured, thus allowing participants to express their opinions about the issues that they consider important (Okoli & Pawlowski, 2004) and starts with open-ended questions (Hsu & Sandford, 2019).”
Outsourcing: decision-making methods and criteria during design and engineering
Shishank, S., & Dekkers, R. (2012) · Production Planning & Control · doi:10.1080/09537287.2011.648544 · Cited by 34
Laur, C. V., & Keller, H. (2015) · Journal of Multidisciplinary Healthcare · doi:10.2147/jmdh.s93103 · Cited by 35
Metadata
- Authors
- Hsu, C.-C., & Sandford, B. A.
- Venue
- University of Massachusetts Amherst
- Publisher
- University of Massachusetts Amherst
- Published
- 2007
- Volume
- 12
- Issue
- 10
- Pages
- 10
- OpenAlex
- W2583504027
- Cited by
- 4,500
- References
- 40
- Citations by year
- 2026: 160 · 2025: 361 · 2024: 381 · 2023: 406 · 2022: 373 · 2021: 381 · 2020: 383 · 2019: 375 · 2018: 358 · 2017: 287 · 2016: 298 · 2015: 240 · 2014: 176 · 2013: 129 · 2012: 91
- FWCI
- 516.01
- Type
- journal article
- Language
- en
- Open access
- Licence
- cc-by-nc-nd
- Locations
- ScholarWorks@UMassAmherst (University of Massachusetts Amherst) (submittedVersion, cc-by-nc-nd)
- University of Massachusetts (UMass) Amherst
- Practical assessment, research & evaluation
- Topics
- Social Sciences › Sociology and Political Science › Delphi Technique in Research
- Social Sciences › Decision Sciences › Management Science and Operations Research › Multi-Criteria Decision Making
- Social Sciences › Sociology and Political Science › Focus Groups and Qualitative Methods
- Keywords
- Delphi method, consensus building, expert opinion, participant selection, response rate, needs assessment, program planning, policy determination
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.