The Delphi Technique: Making Sense of Consensus
Open accessThe DOI matches its DataCite record; 40 of the 40 references it cites were found in a registry or an index.
Abstract
The Delphi technique is a widely used and accepted method for gathering data from respondents within their domain of expertise. The technique is designed as a group communication process which aims to achieve a convergence of opinion on a specific real-world issue. The Delphi process has been used in various fields of study such as program planning, needs assessment, policy determination, and resource utilization to develop a full range of alternatives, explore or expose underlying assumptions, as well as correlate judgments on a topic spanning a wide range of disciplines. The Delphi technique is well suited as a method for consensus-building by using a series of questionnaires delivered using multiple iterations to collect data from a panel of selected subjects. Subject selection, time frames for conducting and completing a study, the possibility of low response rates, and unintentionally guiding feedback from the respondent group are areas which should be considered when designing and implementing a Delphi study. Accessed 68,465 times on https://pareonline.net from August 30, 2007 to December 31, 2019. For downloads from January 1, 2020 forward, please click on the PlumX Metrics link to the right.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“factors with a level of agreement ≥ 70% were extracted and included in the present review [19].”
From Healthcare professionals’ perspectives on artificial intelligence in patient care: a systematic review of hindering and facilitating factors on different levels (2025) · cites this paper as [19]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“To narrow the list into a usable checklist, we then used a three-step Delphi survey process [31].”
From PRISMA-S: an extension to the PRISMA Statement for Reporting Literature Searches in Systematic Reviews (2021) · cites this paper as [31]
The cited source says what the claim attributes to it.
In the citing paper
“The iteration characteristics of the Delphi technique have intrinsic limitations and might lead to groupthink, where participants might conform to dominant opinions of the group, which could affect objectivity.197”
From Definition and diagnostic criteria of clinical obesity (2025) · cites this paper as [197]
In the citing paper
“This anonymous and iterative group technique is designed to gather individual opinions from experts and transform these into a group consensus 6.”
From Diagnosis and treatment of chronic bacterial prostatitis and chronic prostatitis/chronic pelvic pain syndrome: a consensus guideline (2015) · cites this paper as [6]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Foundations of Behavioral Research
Stockton, R., & Kerlinger, F. N. (1974) · American Educational Research Journal · doi:10.2307/1162206 · Cited by 7,347
An Experimental Application of the DELPHI Method to the Use of Experts
Dalkey, N., & Helmer, O. (1963) · Management Science · doi:10.1287/mnsc.9.3.458 · Cited by 6,061
Research guidelines for the Delphi survey technique
Hasson, F. (2000) · Journal of Advanced Nursing · doi:10.1046/j.1365-2648.2000.01567.x · Cited by 5,951 · cited 1×
“As such, Ludwig (1994) specifically addresses subject motivation as the key to the successful implementation of a Delphi study and investigators need to play an active role in this area to help ensure as high a response rate as possible.”
The Delphi Method: An Experimental Study of Group Opinion
(1969) · RAND Corporation eBooks · doi:10.7249/rm5888 · Cited by 1,628
Gazing into the oracle : the Delphi method and its application to social policy and public health
Ziglio, E., & Adler, M. E. (1996) · Jessica Kingsley Publishers eBooks · Cited by 1,429
Group Techniques for Program Planning
de Ven, A. V., & Delbecq, A. L. (1975) · Cited by 907
Technological forecasting for decision making
Martino, J. P. (1972) · Cited by 813
Experimental Study of Channel Patterns
SCHUMM, S. A., & KHAN, H. R. (1972) · Geological Society of America Bulletin · doi:10.1130/0016-7606(1972)83[1755:esocp]2.0.co;2 · Cited by 776
An experimental study of group opinion
Dalkey, N. (1969) · Futures · doi:10.1016/s0016-3287(69)80025-x · Cited by 750
Miller, G. (2001) · Tourism Management · doi:10.1016/s0261-5177(00)00067-4 · Cited by 604
Research works printed but not linked: 7
- Adams, S. J. (2001). Projecting the next decade in safety management: A Delphi technique study. Professional Safety, 46 (10), 26-29.
- Green, P. J. (1982, March). The content of a college-level outdoor leadership course. Paper presented at the Conference of the Northwest District Association for the American Alliance for Health, Physical Education, Recreation, and Dance, Spokane, WA.
- Jones, C. G. (1975). A Delphi evaluation of agreement between organizations. In H. A. Linstone, & M. Turoff (Eds.). The Delphi method: Techniques and applications (pp. 160-167). Reading, MA: Addison-Wesley Publishing Company.
- Kaplan, L. M. (1971). The use of the Delphi method in organizational communication: A case study. Unpublished master's thesis, The Ohio State University, Columbus.
- Linstone, H. A., & Turoff, M. (1975). Introduction. In H. A. Linstone, & M. Turoff (Eds.). The Delphi method: Techniques and applications (pp. 3-12). Reading, MA: Addison-Wesley Publishing Company.
- Ludlow, J. (1975). Delphi inquiries and knowledge utilization. In H. A. Linstone, & M. Turoff (Eds.). The Delphi method: Techniques and applications (pp. 102-123). Reading, MA: Addison-Wesley Publishing Company.
- Scheibe, M., Skutsch, M., & Schofer, J. (1975). Experiments in Delphi methodology. In H. A. Linstone, & M. Turoff (Eds.). The Delphi method: Techniques and applications (pp. 262-287). Reading, MA: Addison-Wesley Publishing Company.
Cited by
Items 701–800 of 1,013
ABIDE Delphi study: topics to discuss in diagnostic consultations in memory clinics
Fruijtier, A. D., Visser, L. N. C., van Maurik, I. S., Zwan, M. D., Bouwman, F. H., van der Flier, W. M., & Smets, E. M. A. (2019) · Alzheimer's Research & Therapy · doi:10.1186/s13195-019-0531-y · Cited by 36
Zapardiel, I., Alvarez, J., Barahona, M., Barri, P. N., Boldo, A., Bresco, P., Gasca, I., Jaunarena, I., Kucukmetin, A., Mancebo, G., et al. (2020) · Annals of Surgical Oncology · doi:10.1245/s10434-020-09222-x · Cited by 37
The constraints of theory: What is the impact of the Theory of Constraints on Operations Strategy?
Pacheco, D. A. D. J., Antunes, J. A. V., & de Matos, C. A. (2020) · International Journal of Production Economics · doi:10.1016/j.ijpe.2020.107955 · Cited by 65
Oehrlein, E. M., Schoch, S. C., Burcu, M., McBeth, J. F., Bright, J. L., Pashos, C. L., Willke, R. J., Love, T. R., Mattingly, T. J., & Perfetto, E. M. (2022) · Value in Health · doi:10.1016/j.jval.2022.04.1738 · Cited by 39
Aronson, B. D., Janke, K. K., & Traynor, A. P. (2012) · American Journal of Pharmaceutical Education · doi:10.5688/ajpe767125 · Cited by 38
“There is no consensus in the literature regarding the optimal number of subjects needed for a Delphi panel.(35) Delbecq and colleagues suggested that 10 to 15 subjects is sufficient if they are homogenous.”
BackgroundCabrelli, A. L., Stow, A., & Hughes, L. (2014) · Biodiversity and Conservation · doi:10.1007/s10531-014-0760-0 · Cited by 41
A modified Delphi study of screening for fetal alcohol spectrum disorders in Australia
Watkins, R. E., Elliott, E. J., Halliday, J. L., O’Leary, C., D’Antoine, H., Russell, E., Hayes, L., Peadon, E. M., Wilkins, A., Jones, H. M., et al. (2013) · BMC Pediatrics · doi:10.1186/1471-2431-13-13 · Cited by 40
Application of computer vision for construction progress monitoring: a qualitative investigation
Moragane, H. P. M. N. L. B., Perera, B. A. K. S., Palihakkara, A. D., & Ekanayake, B. (2022) · Construction Innovation · doi:10.1108/ci-05-2022-0130 · Cited by 41
Rezaei, M., Sadraei, R., Jafari‐Sadeghi, V., & Vrontis, D. (2023) · Asia Pacific Journal of Management · doi:10.1007/s10490-023-09896-3 · Cited by 43
Analysis of Experts' and Trainers' Views on Cyberbullying
Jäger, T., Amado, J., Matos, A., & Pessôa, T. (2010) · Australian Journal of Guidance and Counselling · doi:10.1375/ajgc.20.2.169 · Cited by 41
Shortall, R., Davíðsdóttir, B., & Axelsson, G. (2015) · Renewable and Sustainable Energy Reviews · doi:10.1016/j.rser.2015.04.175 · Cited by 43
Tornatore, C., Phillips, J. T., Khan, O. A., Miller, A., & Hughes, M. A. (2016) · Neurology Clinical Practice · doi:10.1212/cpj.0000000000000254 · Cited by 39
“Current US treatment consensus patterns emphasize (1) MRI in multiple sclerosis (MS) diagnosis and subsequent treatment decisions, (2) treatment of early disease, (3) aggressive initial treatment of highly active MS, and (4) close patient monitoring for clinical response and adverse effects of disease-modifying drugs.”
BackgroundGarrigós‐Simón, F. J., Cruz, T. F. G., & Contreras-Pacheco, O. E. (2016) · Entrepreneurship and Regional Development · doi:10.1080/08985626.2016.1255437 · Cited by 42
Muñiz‐Rodríguez, L., Alonso, P., Rodríguez-Muñiz, L. J., & Valcke, M. (2017) · Journal of Education for Teaching International Research and Pedagogy · doi:10.1080/02607476.2017.1296539 · Cited by 49
“Feedback is probably the most important element in the Delphi method and it is the driver of consensus development (Hsu and Sandford 2007).”
BackgroundMethodInfluentialHip Surveillance for Children With Cerebral Palsy: A Survey of the POSNA Membership
Shore, B. J., Shrader, M. W., Narayanan, U. G., Miller, F., Graham, H. K., & Mulpuri, K. (2017) · Journal of Pediatric Orthopaedics · doi:10.1097/bpo.0000000000001050 · Cited by 35
Ethical Principles and Guidelines of Global Health Nursing Practice
McDermott‐Levy, R., Leffers, J. M., & Mayaka, J. (2018) · Nursing Outlook · doi:10.1016/j.outlook.2018.06.013 · Cited by 35
Agrò, E. F., Gammie, A., Kessler, T. M., van Koeveringe, G. A., Serati, M., Solomon, E., Wachter, S. D., & Kirschner-Hermanns, R. K. M. (2018) · European Urology Focus · doi:10.1016/j.euf.2018.07.031 · Cited by 35
Fung, J. S. T., Akech, S., Kissoon, N. “., Wiens, M. O., English, M., & Ansermino, J. M. (2019) · PLoS ONE · doi:10.1371/journal.pone.0211274 · Cited by 37
Diaz, J. M., Chaudhary, A. K., Jayaratne, K. S. U., & Assan, E. (2020) · Evaluation and Program Planning · doi:10.1016/j.evalprogplan.2020.101790 · Cited by 35
Ten commandments for successful implementation of Design for Six Sigma
Gijo, E. V., Bhat, S., Antony, J., & Park, S. H. (2021) · The TQM Journal · doi:10.1108/tqm-01-2021-0014 · Cited by 41
Assessing Urban Quality of Life: Developing the Criteria for Saudi Cities
Al-Qawasmi, J., Saeed, M., Asfour, O. S., & Aldosary, A. S. (2021) · Frontiers in Built Environment · doi:10.3389/fbuil.2021.682391 · Cited by 44
“IQDs are commonly used to determine consensus, with a smaller IQD indicating larger consensus (De Meyrick 2003; Hsu and Sandford 2007).”
MethodMartin-Utrillas, M., Reyes-Medina, M., Curiel‐Esparza, J., & Canto-Perello, J. (2014) · Clean Technologies and Environmental Policy · doi:10.1007/s10098-014-0834-4 · Cited by 35
Salud America ! Developing a National Latino Childhood Obesity Research Agenda
Ramírez, A. G., Chalela, P., Gallion, K. J., Green, L. W., & Ottoson, J. M. (2011) · Health Education & Behavior · doi:10.1177/1090198110372333 · Cited by 38
“It also removes the shortcomings of face-to-face group discussions, such as the influence of dominant individuals, manipulation, or group pressure for conformity (Dalkey, 1972; Delbecq, Van de Ven, & Gustafson, 1975; Hsu & Sandford, 2007).”
BackgroundPanel, O. B. O. T. P. I. T. C., de Leonardo, C. G., Moral, R. R., Caballero, F., Cavaco, A. M. D. N., Moore, P., Dupuy, L., Cyrino, A. D. P. P., Cortés, M. T., Gorostegui, M. E. P., et al. (2016) · BMC Medical Education · doi:10.1186/s12909-016-0610-8 · Cited by 47
“experts involved (number of participants, professional profile, level of expertise and type of participation in the process), the type of questionnaire proposed, the definition of a sufficient level of agreement and subsequent applicability [79, 80].”
BackgroundTowards an operational definition of oral frailty: A e-Delphi study
Parisius, K. G. H., Verhoeff, M. C., Lobbezoo, F., Avivi‐Arber, L., Duyck, J. A. J., Hirano, H., Iijima, K., Janssens, B. E., Kossioni, A., Lin, C., et al. (2023) · Archives of Gerontology and Geriatrics · doi:10.1016/j.archger.2023.105181 · Cited by 34
Spinal Muscular Atrophy Update in Best Practices: Recommendations for Treatment Considerations
Schroth, M. K., Deans, J., Goebel, D. X. B., Burnette, W. B., Darras, B. T., Elsheikh, B., Felker, M. V., Klein, A., Krueger, J. M., Proud, C. M., et al. (2024) · Neurology Clinical Practice · doi:10.1212/cpj.0000000000200374 · Cited by 46
IAPRD new consensus classification of myoclonus
Latorre, A., van der Veen, S., Peña, A. B., Truong, D. D., Erro, R., Frucht, S. J., Ganos, C., Hallett, M., Pérez‐Dueñas, B., Rossi, M., et al. (2024) · Parkinsonism & Related Disorders · doi:10.1016/j.parkreldis.2024.107216 · Cited by 39
Understanding the visual skills and strategies of train drivers in the urban rail environment
Naweed, A., & Balakrishnan, G. (2014) · Work · doi:10.3233/wor-131705 · Cited by 35
“The data were analysed using the Delphi technique, a widely used and accepted method for achieving convergence of findings concerned with real-world expert knowledge [20, 21].”
Unlocking skills in hospitals: better jobs, more care
Duckett, S. J., & Breadon, P. (2014) · Cited by 44
Hollaar, V. R. Y., van der Maarel‐Wierink, C. D., van der Putten, G.-J., van der Sanden, W. J. M., de Swart, B. J., & Baat, C. D. (2016) · BMC Geriatrics · doi:10.1186/s12877-016-0231-4 · Cited by 34
“Nowadays, the method has been modified to gather and structure information on a particular issue from experts in an electronical way, using an iterative process to reach consensus on that issue [23, 24].”
MethodAn expert consensus on core competencies in integrated care for psychiatrists
Sunderji, N., Waddell, A., Gupta, M., Soklaridis, S., & Steinberg, R. J. (2016) · General Hospital Psychiatry · doi:10.1016/j.genhosppsych.2016.05.003 · Cited by 39
Nayahangan, L. J., Clementsen, P. F., Paltved, C., Lindorff‐Larsen, K., Nielsen, B. U., & Konge, L. (2016) · Respiration · doi:10.1159/000446926 · Cited by 38
Transforming Roles of Nursing Professional Development Practitioners
Warren, J. I., & Harper, M. G. (2017) · Journal for Nurses in Professional Development · doi:10.1097/nnd.0000000000000320 · Cited by 34
Holton, A. E., Gallagher, P., Ryan, C. A., Fahey, T. P., & Cousins, G. (2017) · BMJ Open · doi:10.1136/bmjopen-2017-017453 · Cited by 39
Structured reporting for fibrosing lung disease: a model shared by radiologist and pulmonologist
Consort, T. I. S. R. O. F. L. D., Sverzellati, N., Odone, A., Silva, M., Polverosi, R., Florio, C., Cardinale, L., Cortese, G., Addonisio, G., Zompatori, M., et al. (2017) · La radiologia medica · doi:10.1007/s11547-017-0835-6 · Cited by 34
“Survey analysis was prospectively performed round by round [15].”
Dars, S., Uden, H., Kumar, S., & Banwell, H. A. (2018) · PeerJ · doi:10.7717/peerj.4667 · Cited by 43
“…with symptomatic 97 flexible pes planus 98 Methods 99 Study design 100 The study was a three-round modified Delphi panel design where participants’ opinion was 101 sought in Round one, with responses collated and analysed for consensus (Hasson et al. 2000; 102 Hsu & Sandford 2007; Vernon 2009).”
Cerezo-Narváez, A., Carmenado, I. D. L. R., Pastor-Fernández, A., Blanco, J. L. Y., & Otero-Mateo, M. (2019) · Education Sciences · doi:10.3390/educsci9010044 · Cited by 51
“On the other side, the Delphi technique is a widely accepted method for gathering data, but only if respondents are within their domain of expertise [130].”
MethodDevane, D., Barrett, N., Gallen, A., O’Reilly, M. F., Nadin, M., Conway, G., Biesty, L., & Smith, V. A. (2019) · BMC Pregnancy and Childbirth · doi:10.1186/s12884-019-2346-z · Cited by 37
“It has been described as an optimal design for facilitating consensus-building on a topic under investigation [7].”
BackgroundAlgurén, B., Ramirez, J. P., Salt, M. J., Sillett, N., Myers, S. N., Alvarez-Cote, A., Butcher, N. J., Canêo, L. F., Céspedes, J., Chaplin, J. E., et al. (2020) · Archives of Disease in Childhood · doi:10.1136/archdischild-2020-320345 · Cited by 33
Wall, D. R., Meah, N., York, K., Bhoyrul, B., Bokhari, L., Abraham, L. S., Adams, R., Bergfeld, W. F., Betz, R. C., Blume‐Peytavi, U., et al. (2021) · JAMA Dermatology · doi:10.1001/jamadermatol.2020.5839 · Cited by 36
Anderson, A. M., Comer, C. M., Smith, T. O., Drew, B. T., Pandit, H. G., Antcliff, D., Redmond, A. C., & McHugh, G. A. (2021) · BMC Musculoskeletal Disorders · doi:10.1186/s12891-021-04160-5 · Cited by 39
“Providing panellists with a summary of the results of the preceding round is an established approach for encouraging panellists to reconsider their initial judgement and hence facilitate the development of consensus [42, 48].”
BackgroundMarketing messages for post-pandemic destination recovery- A Delphi study
Singh, S., Nicely, A., Day, J., & Cai, L. A. (2021) · Journal of Destination Marketing & Management · doi:10.1016/j.jdmm.2021.100676 · Cited by 49
Intercultural communicative competence: Hospitality industry and education perspectives
Liu, Y., Liu, J., & King, B. E. M. (2022) · Journal of Hospitality Leisure Sport & Tourism Education · doi:10.1016/j.jhlste.2022.100371 · Cited by 66
Proposed model of integrated Islamic commercial and social finance for Islamic bank in Indonesia
Ascarya, A., Suharto, U., & Husman, J. A. (2022) · Eurasian economic review : · doi:10.1007/s40822-022-00201-z · Cited by 74
“Delphi and ANP methods require at least one focus group discussion (FGD) of knowledgeable respondents (Hsu & Sanford, 2007; Saaty, 2005), while each FGD could comprise of 6–12 respondents (Dilshad & Latif, 2013).”
BackgroundMethodJanke, K. K., Kelley, K. A., Sweet, B. V., & Kuba, S. E. (2016) · American Journal of Pharmaceutical Education · doi:10.5688/ajpe8010167 · Cited by 40
“Statements defined from early rounds can be rated by the panelists to determine consensus in later rounds.(9,12) Several characteristics of the Delphi process encourage quality participation and input.”
BackgroundAnalyzing risk management in Mediterranean Countries: The Syrian perspective
Santeramo, F. G., Pasquale, J. D., Contò, F., Tudisca, S., & Sgroi, F. (2012) · Munich Personal RePEc Archive (Ludwig Maximilian University of Munich) · Cited by 38
London International Consensus and Delphi study on hamstring injuries part 1: classification
Paton, B. M., Court, N., Giakoumis, M., Head, P., Kayani, B., Kelly, S., Kerkhoffs, G. M. M. J., Moore, J. A., Moriarty, P. D., Murphy, S., et al. (2023) · British Journal of Sports Medicine · doi:10.1136/bjsports-2021-105371 · Cited by 35
Lettrich, M. D., Asaro, M. J., Borggaard, D. L., Dick, D. M., Griffis, R. B., Litz, J. A., Orphanides, C., Palka, D. L., Soldevilla, M. S., Balmer, B. C., et al. (2023) · PLoS ONE · doi:10.1371/journal.pone.0290643 · Cited by 41
Wackerle, A. M., Marcaccio, S. E., Apseloff, N. A., Getgood, A. M. J., Musahl, V., Tapasvi, S. R., Alaia, M. J., Amendola, A., Becker, R., Brown, C. H., et al. (2025) · Journal of ISAKOS Joint Disorders & Orthopaedic Sports Medicine · doi:10.1016/j.jisako.2025.100900 · Cited by 33
Applying the Delphi technique to define ‘medication wastage’
West, L. M., Diack, L., Cordina, M., & Stewart, D. C. (2015) · European Journal of Hospital Pharmacy · doi:10.1136/ejhpharm-2014-000593 · Cited by 39
Selecting utilities placement techniques in urban underground engineering
Curiel‐Esparza, J., & Canto-Perello, J. (2013) · Archives of Civil and Mechanical Engineering · doi:10.1016/j.acme.2013.02.001 · Cited by 33
Camacho, G. B., Cepolina, S., Dobrijevic, A., Cerbán, M. D. M., Djordjevic, O., & González, C. (2015) · International Conference on Industrial Engineering and Systems Management · doi:10.1109/iesm.2015.7380209 · Cited by 33
“This is a structured communication technique, originally developed as a systematic, interactive forecasting method which relies on a panel of experts [22].”
MethodMurphy, M., Hollinghurst, S. P., & Salisbury, C. (2016) · Health Expectations · doi:10.1111/hex.12462 · Cited by 32
Improving Operational Risk Management Using Business Performance Management Technologies
Weeserik, B. P., & Spruit, M. R. (2018) · Sustainability · doi:10.3390/su10030640 · Cited by 33
“However, a Delphi expert panel would have consumed too much time [82,83].”
BackgroundA Delphi study to identify the core components of nurse to nurse handoff
O’Rourke, J., Abraham, J., Riesenberg, L. A., Matson, J., & Lopez, K. D. (2018) · Journal of Advanced Nursing · doi:10.1111/jan.13565 · Cited by 40
Signs and symptoms of disordered eating in pregnancy: a Delphi consensus study
Bannatyne, A. J., Hughes, R., Stapleton, P., Watt, B. D., & MacKenzie‐Shalders, K. L. (2018) · BMC Pregnancy and Childbirth · doi:10.1186/s12884-018-1849-3 · Cited by 40
“This could have been achieved through the implementation of a consultation meeting [55]; however, the anonymity of the panels likely prevented power-imbalances and group think that may have developed via direct contact [56, 57].”
BackgroundEnabling Community Participation for Social Innovation in the Energy Sector
Budiman, I. (2018) · Indonesian Journal of Energy · doi:10.33116/ije.v1i2.23 · Cited by 33
Utama, W. P., Chan, A. P. C., Zahoor, H., Gao, R., & Jumas, D. Y. (2019) · Engineering Construction & Architectural Management · doi:10.1108/ecam-01-2018-0016 · Cited by 32
The contribution of social media to project management
Kanagarajoo, M. V., Fulford, R. G., & Standing, C. (2019) · International Journal of Productivity and Performance Management · doi:10.1108/ijppm-09-2018-0316 · Cited by 50
The Adoption of the Electronic Health Record by Physicians
Al-Rayes, S., Alumran, A. K., & Al-Fayez, W. (2019) · Methods of Information in Medicine · doi:10.1055/s-0039-1695006 · Cited by 38
Researching the Affects That Online Pornography Has on U.K. Adolescents Aged 11 to 16
Martellozzo, E., Monaghan, A., Davidson, J. C., & Adler, J. R. (2020) · SAGE Open · doi:10.1177/2158244019899462 · Cited by 60
“…employed a Delphi style approach between the three stages, in which the findings of one stage were checked and verified—both in terms of data reliability and by comparison with the literature—by the research team, then by application to the next stage in the cycle (Hsu & Sandford, 2007).”
MethodFallon, B., Lefebvre, R., Collin‐Vézina, D., Houston, E. A. L., Joh-Carnella, N., Malti, T., Filippelli, J., Schumaker, K., Manel, W., Kartusch, M., et al. (2020) · Child Abuse & Neglect · doi:10.1016/j.chiabu.2020.104706 · Cited by 39
“Research team members and agency partners further developed and refined the questions using a modified Delphi method which is an iterative process that requires consensus among experts to enhance the credibility of measures (Hsu & Sandford, 2007).”
Khanina, A., Urbancic, K. F., Haeusler, G. M., Kong, D. C. M., Douglas, A. P., Tio, S. Y., Worth, L. J., Slavin, M. A., & Thursky, K. (2020) · Journal of Antimicrobial Chemotherapy · doi:10.1093/jac/dkaa409 · Cited by 34
Norris, E., Hastings, J., Marques, M. M., Finnerty, A. N., Zink, S., & Michie, S. (2021) · Journal of Biomedical Semantics · doi:10.1186/s13326-021-00240-6 · Cited by 37
“Consensus exercises consist of two or more sequential rounds of questions and feedback to experts, aiming to achieve consensus across the group, such as in Delphi exercises [49] or using Nominal Group Technique [50].”
MethodDhaubanjar, S., Lutz, A., Gernaat, D., Nepal, S., Smolenaars, W. J., Pradhananga, S., Biemans, H., Ludwig, F., Shrestha, A. B., & Immerzeel, W. W. (2021) · The Science of The Total Environment · doi:10.1016/j.scitotenv.2021.147142 · Cited by 38
International Collaboration in the Field of Educational Research: A Delphi Study
Bond, M., Marín, V. I., & Bedenlier, S. (2021) · Journal of New Approaches in Educational Research · doi:10.7821/naer.2021.7.614 · Cited by 48
“Hsu and Sandford (2007) describe the main features of this technique as being an iterative one, including several rounds of questioning experts on a specific topic –whilst keeping their identity from each other to avoid bias, through which consensus on the topic or question is eventually reached.”
Hajiheydari, N., Delgosha, M. S., Wang, Y., & Olya, H. (2021) · Industrial Management & Data Systems · doi:10.1108/imds-04-2021-0209 · Cited by 40
Hamm, D., & Su, C.-H. J. (2021) · Journal of Hospitality and Tourism Management · doi:10.1016/j.jhtm.2021.10.001 · Cited by 41
Khan, M. R., Alam, M. J., Tabassum, N. H., & Khan, N. A. (2022) · Sustainability · doi:10.3390/su142114215 · Cited by 52
Expanding the Uses of AHRQ's Prevention Quality Indicators
Davies, S. M., McDonald, K. M., Schmidt, E. M., Schultz, E. M., Geppert, J., & Romano, P. S. (2011) · Medical Care · doi:10.1097/mlr.0b013e3182159e65 · Cited by 33
“Delphi panels are traditionally large, although an optimal size has not been established.(25) Delphi panels review information only in writing, whereas the Nominal group technique includes group discussion, effectively limiting Nominal group panels to 9 to 15 participants.”
Christmann, G. B., Balgar, K., & Mahlkow, N. (2014) · Social Sciences · doi:10.3390/socsci3010142 · Cited by 33
“experts from coastal cities and communities in Germany, the Netherlands, Denmark and Poland was conducted [58–60].”
BackgroundValidation of a clinical leadership qualities framework for managers in aged care: a D elphi study
Jeon, Y., Conway, J., Chenoweth, L., Weise, J. C., Thomas, T. H. T., & Williams, A. (2014) · Journal of Clinical Nursing · doi:10.1111/jocn.12682 · Cited by 35
Berendt, J., Stiel, S., Simon, S. T., Schmitz, A., van Oorschot, B., Stachura, P., & Ostgathe, C. (2016) · The Oncologist · doi:10.1634/theoncologist.2016-0063 · Cited by 37
Kreibich, R., Oertel, B., & Wölk, M. (2012) · SSRN Electronic Journal · doi:10.2139/ssrn.2094215 · Cited by 33
Fard, S. M. H., & Doratlı, N. (2022) · Sustainability · doi:10.3390/su14052495 · Cited by 33
Lenet, T., McIsaac, D. I., Hallet, J. H., Jerath, A., Lalu, M. M., Nicholls, S. G., Presseau, J., Tinmouth, A. T., Verret, M., Wherrett, C. G., et al. (2023) · JAMA Network Open · doi:10.1001/jamanetworkopen.2023.49559 · Cited by 52
Developing competencies for multidisciplinary hospice and palliative care professionals in Korea
Kang, J., Kim, Y., Yoo, Y.-S., Choi, J. Y., Koh, S. J., Jho, H. J., Choi, Y. S., Park, J., Moon, D. H., Kim, D. Y., et al. (2013) · Supportive Care in Cancer · doi:10.1007/s00520-013-1850-3 · Cited by 32
A Quality Scorecard for the Administration of Online Education Programs
Shelton, K. (2010) · Online Learning · doi:10.24059/olj.v14i4.163 · Cited by 69
DIAGNOSIS AND TREATMENT OF INTERSTITIAL CYSTITIS/BLADDER PAIN SYNDROME
Hanno, P. M., Burks, D., Clemens, J. Q., Dmochowski, R. R., Erickson, D. R., Forrest, J. B., Gordon, B., GRAY, M. L., Mayer, R., Newman, D. K., et al. (2011) · Cited by 32
Preventing Adverse Events in Cataract Surgery: Recommendations From a Massachusetts Expert Panel
Nanji, K. C., Roberto, S. A., Morley, M. G., & Bayes, J. (2017) · Anesthesia & Analgesia · doi:10.1213/ane.0000000000002529 · Cited by 31
Framework for Managing Integration Challenges of Last Planner System in IMPs
El-Sabek, L. M., & McCabe, B. Y. (2018) · Journal of Construction Engineering and Management · doi:10.1061/(asce)co.1943-7862.0001468 · Cited by 43
Using the Delphi method to establish nursing‐sensitive quality indicators for ICU nursing in China
Yang, S. J., Huang, L., Zhao, X., Xing, M., Shao, L., Zhang, M., Shao, R., Wei, J., & Gao, C. (2019) · Research in Nursing & Health · doi:10.1002/nur.21925 · Cited by 32
Banno, M., Tsujimoto, Y., & Kataoka, Y. (2019) · BMJ Open · doi:10.1136/bmjopen-2018-024942 · Cited by 36
“9 Participants may regulate their initial ratings based on feedback from the group in a number of accompanying loops.(6) 9 The Delphi technique consists of any type of self-administered questionnaire with no meetings, whereas the modified Delphi technique consists of the use of a self-administered questionnaire, combi…”
MethodRoye, B. D., Campbell, M. L., Matsumoto, H., Pahys, J. M., Welborn, M. C., Sawyer, J. R., Fletcher, N. D., McIntosh, A. L., Sturm, P. F., Gómez, J. A., et al. (2019) · Journal of Pediatric Orthopaedics · doi:10.1097/bpo.0000000000001379 · Cited by 35
Parry, I. S., Schneider, J. C., Yelvington, M., Sharp, P., Serghiou, M. A., Ryan, C. M., Richardson, E., Pontius, K., Niszczak, J., McMahon, M., et al. (2019) · Journal of Burn Care & Research · doi:10.1093/jbcr/irz150 · Cited by 32
Hamada, S., Pirracchio, R., Beauchesne, J., Benlaldj, M. N., Meaudre, É., Léone, M., Pottecher, J., Abback, P.-S., Gauss, T., Boutonnet, M., et al. (2020) · The Journal of Trauma: Injury, Infection, and Critical Care · doi:10.1097/ta.0000000000002624 · Cited by 35
Darzi, A., Karam, S. G., Spencer, F. A., Spyropoulos, A. C., Mbuagbaw, L. C. E., Woller, S. C., Zakai, N. A., Streiff, M. B., Gould, M. K., Cushman, M., et al. (2020) · Blood Advances · doi:10.1182/bloodadvances.2020001937 · Cited by 34
Castro, M. P., Scheede, C. R., & Zermeño, M. G. G. (2020) · IEEE International Conference on Technology and Entrepreneurship (ICTE) · doi:10.1109/icte-v50708.2020.9114372 · Cited by 32
“In this context, the Delphi method seeks to generate a consensus in the participating group, using questionnaires to collect the data [48].”
MethodXu, C., Ming-yan, Y., Kang, M., & Duan, G. (2020) · BioMed Research International · doi:10.1155/2020/2345607 · Cited by 32
“The Delphi technique used to testify the appropriateness of the ARG movements was used to collect data from experts [17].”
MethodAndrés‐Jensen, L., Attarbaschi, A., Bárdi, E., Barzilai‐Birenboim, S., Bhojwani, D., Hagleitner, M. M., Halsey, C., Harila‐Saari, A. H., van Litsenburg, R. R. L., Hudson, M. M., et al. (2021) · The Lancet Haematology · doi:10.1016/s2352-3026(21)00136-8 · Cited by 34
Safran, M. R., Murray, I. R., Andrade, T., Aoki, S. K., Ayeni, O. R., Balakumar, J., Bonin, N., Brick, M. J., Cakic, J. N., Domb, B. G., et al. (2022) · Arthroscopy The Journal of Arthroscopic and Related Surgery · doi:10.1016/j.arthro.2022.03.027 · Cited by 33
Madsen, G. R., Wilkens, R., Boysen, T., Burisch, J. M., Bryant, R. V., Carter, D., Gecse, K. B., Maaser, C., Maconi, G., Novak, K. L., et al. (2022) · Alimentary Pharmacology & Therapeutics · doi:10.1111/apt.16950 · Cited by 32
Telehealth as a Means of Enabling Health Equity
Kuziemsky, C. E., Hunter, I. M., Udayasankaran, J. G., Ranatunga, P., Kulatunga, G. G., John, S., John, O., Florez-Arango, J. F., Ito, M., Ho, K., et al. (2022) · Yearbook of Medical Informatics · doi:10.1055/s-0042-1742500 · Cited by 41
“The Delphi method is an iterative process that allows a group of stakeholders to reach a structured consensus on a topic or question [26, 27].”
BackgroundMethodOrganizational readiness for knowledge translation in chronic care: a Delphi study
Attieh, R., Gagnon, M., Estabrooks, C. A., Légaré, F., Ouimet, M., Vazquez, P., & Nuño-Solínis, R. (2014) · BMC Health Services Research · doi:10.1186/s12913-014-0534-0 · Cited by 38
“The modified Delphi is a systemic and interactive group data collection procedure for obtaining forecasts from a panel of selected independent expert participants; it uses a series of questionnaires delivered in multiple iterations [10,11].”
BackgroundMethodResultA study of digital curator competences: A survey of experts
Madrid, M. M. (2013) · The International Information & Library Review · doi:10.1080/10572317.2013.10766382 · Cited by 47
“But there are recent studies that acquired the option of replacing the traditional open-ended questionnaire with a structured one, thus a modified version of the Delphi study (Hsu and Sandford, 2007; Murry & Hammons, 1995).”
BackgroundMethodJopp, R., Lacy, T. D., Mair, J., & Fluker, M. (2012) · Asia Pacific Journal of Tourism Research · doi:10.1080/10941665.2012.688515 · Cited by 33
“These weaknesses as stated by Hsu and Sandford ( 2007 ) include the potential for low response rates due to the multiple feedback process and the possibility of indentifying general statements versus specific topic-related information.”
BackgroundLisi, A. J., Salsbury, S. A., Hawk, C., Vining, R. D., Wallace, R. B., Branson, R., Long, C. R., Burgo-Black, A. L., & Goertz, C. (2018) · Journal of Manipulative and Physiological Therapeutics · doi:10.1016/j.jmpt.2017.10.001 · Cited by 33
“As the current VA guideline for LBP offered few details on interdisciplinary care processes that include chiropractic care for veterans with LBP conditions, we chose to conduct a modified Delphi process study to seek consensus among clinicianswhowork in real-worldVA settings on best practices for establishing interpro…”
MethodBurn patients’ pain experiences and perceptions
Duchin, E. R., Moore, M., Carrougher, G. J., Min, E. K., Gordon, D. B., Stewart, B. T., Sabel, J. I., Jo-Nes, A., & Pham, T. N. (2021) · Burns · doi:10.1016/j.burns.2021.01.010 · Cited by 31
Oxley, E., Nash, H. M., & Weighall, A. (2024) · International Journal of Research & Method in Education · doi:10.1080/1743727x.2024.2317851 · Cited by 51
“There is no general agreement within the literature around how to classify consensus in a Delphi study (Hsu and Sandford 2007).”
The association between obesity and efficacy of psoriasis therapies: An expert consensus panel
Burshtein, J. G., Armstrong, A. W., CHOW, M. J., DeBusk, L., Glick, B. P., Gottlieb, A. B., Gold, L. S., Korman, N. J., Lio, P., Merola, J. F., et al. (2024) · Journal of the American Academy of Dermatology · doi:10.1016/j.jaad.2024.12.016 · Cited by 38
Metadata
- Authors
- Hsu, C.-C., & Sandford, B. A.
- Venue
- University of Massachusetts Amherst
- Publisher
- University of Massachusetts Amherst
- Published
- 2007
- Volume
- 12
- Issue
- 10
- Pages
- 10
- OpenAlex
- W2583504027
- Cited by
- 4,500
- References
- 40
- Citations by year
- 2026: 160 · 2025: 361 · 2024: 381 · 2023: 406 · 2022: 373 · 2021: 381 · 2020: 383 · 2019: 375 · 2018: 358 · 2017: 287 · 2016: 298 · 2015: 240 · 2014: 176 · 2013: 129 · 2012: 91
- FWCI
- 516.01
- Type
- journal article
- Language
- en
- Open access
- Licence
- cc-by-nc-nd
- Locations
- ScholarWorks@UMassAmherst (University of Massachusetts Amherst) (submittedVersion, cc-by-nc-nd)
- University of Massachusetts (UMass) Amherst
- Practical assessment, research & evaluation
- Topics
- Social Sciences › Sociology and Political Science › Delphi Technique in Research
- Social Sciences › Decision Sciences › Management Science and Operations Research › Multi-Criteria Decision Making
- Social Sciences › Sociology and Political Science › Focus Groups and Qualitative Methods
- Keywords
- Delphi method, consensus building, expert opinion, participant selection, response rate, needs assessment, program planning, policy determination
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.