The Delphi Technique: Making Sense of Consensus
Open accessThe DOI matches its DataCite record; 40 of the 40 references it cites were found in a registry or an index.
Abstract
The Delphi technique is a widely used and accepted method for gathering data from respondents within their domain of expertise. The technique is designed as a group communication process which aims to achieve a convergence of opinion on a specific real-world issue. The Delphi process has been used in various fields of study such as program planning, needs assessment, policy determination, and resource utilization to develop a full range of alternatives, explore or expose underlying assumptions, as well as correlate judgments on a topic spanning a wide range of disciplines. The Delphi technique is well suited as a method for consensus-building by using a series of questionnaires delivered using multiple iterations to collect data from a panel of selected subjects. Subject selection, time frames for conducting and completing a study, the possibility of low response rates, and unintentionally guiding feedback from the respondent group are areas which should be considered when designing and implementing a Delphi study. Accessed 68,465 times on https://pareonline.net from August 30, 2007 to December 31, 2019. For downloads from January 1, 2020 forward, please click on the PlumX Metrics link to the right.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“factors with a level of agreement ≥ 70% were extracted and included in the present review [19].”
From Healthcare professionals’ perspectives on artificial intelligence in patient care: a systematic review of hindering and facilitating factors on different levels (2025) · cites this paper as [19]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“To narrow the list into a usable checklist, we then used a three-step Delphi survey process [31].”
From PRISMA-S: an extension to the PRISMA Statement for Reporting Literature Searches in Systematic Reviews (2021) · cites this paper as [31]
The cited source says what the claim attributes to it.
In the citing paper
“The iteration characteristics of the Delphi technique have intrinsic limitations and might lead to groupthink, where participants might conform to dominant opinions of the group, which could affect objectivity.197”
From Definition and diagnostic criteria of clinical obesity (2025) · cites this paper as [197]
In the citing paper
“This anonymous and iterative group technique is designed to gather individual opinions from experts and transform these into a group consensus 6.”
From Diagnosis and treatment of chronic bacterial prostatitis and chronic prostatitis/chronic pelvic pain syndrome: a consensus guideline (2015) · cites this paper as [6]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Foundations of Behavioral Research
Stockton, R., & Kerlinger, F. N. (1974) · American Educational Research Journal · doi:10.2307/1162206 · Cited by 7,347
An Experimental Application of the DELPHI Method to the Use of Experts
Dalkey, N., & Helmer, O. (1963) · Management Science · doi:10.1287/mnsc.9.3.458 · Cited by 6,061
Research guidelines for the Delphi survey technique
Hasson, F. (2000) · Journal of Advanced Nursing · doi:10.1046/j.1365-2648.2000.01567.x · Cited by 5,951 · cited 1×
“As such, Ludwig (1994) specifically addresses subject motivation as the key to the successful implementation of a Delphi study and investigators need to play an active role in this area to help ensure as high a response rate as possible.”
The Delphi Method: An Experimental Study of Group Opinion
(1969) · RAND Corporation eBooks · doi:10.7249/rm5888 · Cited by 1,628
Gazing into the oracle : the Delphi method and its application to social policy and public health
Ziglio, E., & Adler, M. E. (1996) · Jessica Kingsley Publishers eBooks · Cited by 1,429
Group Techniques for Program Planning
de Ven, A. V., & Delbecq, A. L. (1975) · Cited by 907
Technological forecasting for decision making
Martino, J. P. (1972) · Cited by 813
Experimental Study of Channel Patterns
SCHUMM, S. A., & KHAN, H. R. (1972) · Geological Society of America Bulletin · doi:10.1130/0016-7606(1972)83[1755:esocp]2.0.co;2 · Cited by 776
An experimental study of group opinion
Dalkey, N. (1969) · Futures · doi:10.1016/s0016-3287(69)80025-x · Cited by 750
Miller, G. (2001) · Tourism Management · doi:10.1016/s0261-5177(00)00067-4 · Cited by 604
Research works printed but not linked: 7
- Adams, S. J. (2001). Projecting the next decade in safety management: A Delphi technique study. Professional Safety, 46 (10), 26-29.
- Green, P. J. (1982, March). The content of a college-level outdoor leadership course. Paper presented at the Conference of the Northwest District Association for the American Alliance for Health, Physical Education, Recreation, and Dance, Spokane, WA.
- Jones, C. G. (1975). A Delphi evaluation of agreement between organizations. In H. A. Linstone, & M. Turoff (Eds.). The Delphi method: Techniques and applications (pp. 160-167). Reading, MA: Addison-Wesley Publishing Company.
- Kaplan, L. M. (1971). The use of the Delphi method in organizational communication: A case study. Unpublished master's thesis, The Ohio State University, Columbus.
- Linstone, H. A., & Turoff, M. (1975). Introduction. In H. A. Linstone, & M. Turoff (Eds.). The Delphi method: Techniques and applications (pp. 3-12). Reading, MA: Addison-Wesley Publishing Company.
- Ludlow, J. (1975). Delphi inquiries and knowledge utilization. In H. A. Linstone, & M. Turoff (Eds.). The Delphi method: Techniques and applications (pp. 102-123). Reading, MA: Addison-Wesley Publishing Company.
- Scheibe, M., Skutsch, M., & Schofer, J. (1975). Experiments in Delphi methodology. In H. A. Linstone, & M. Turoff (Eds.). The Delphi method: Techniques and applications (pp. 262-287). Reading, MA: Addison-Wesley Publishing Company.
Cited by
James, P. A., Oparil, S., Carter, B. L., Cushman, W. C., Dennison-Himmelfarb, C., Handler, J., Lackland, D. T., LeFevre, M. L., MacKenzie, T. D., Ogedegbe, O., et al. (2014) · JAMA · doi:10.1001/jama.2013.284427 · Cited by 7,950
Diamond, I. R., Grant, R. C., Feldman, B. M., Pencharz, P. B., Ling, S. C., Moore, A. M., & Wales, P. W. (2014) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2013.12.002 · Cited by 3,740
“Quality score components Were criteria for participants reproducible? Yes 65 No 33 Was the number of rounds to be performed stated? Yes 81 No 17 Were criteria for dropping items clear? Yes or not applicable 59 No 39 Stopping criteria other than rounds specified? Yes 15 No 83 Quality score (items) 0 3 1 16 2 37 3 38 4…”
BackgroundMethodRethlefsen, M. L., Kirtley, S., Waffenschmidt, S., Ayala, A. P., Moher, D., Page, M. J., & Koffel, J. (2019) · doi:10.31219/osf.io/sfc38 · Cited by 3,696
Rethlefsen, M. L., Kirtley, S., Waffenschmidt, S., Ayala, A. P., Moher, D., Page, M. J., Koffel, J. B., PRISMA-S Group, Blunt, H., Brigham, T., et al. (2021) · Systematic Reviews · doi:10.1186/s13643-020-01542-z · Cited by 3,714
“To narrow the list into a usable checklist, we then used a three-step Delphi survey process [31].”
Development of a critical appraisal tool to assess the quality of cross-sectional studies (AXIS)
Downes, M. J., Brennan, M. L., Williams, H. C., & Dean, R. S. (2016) · BMJ Open · doi:10.1136/bmjopen-2016-011458 · Cited by 2,376
Definition and diagnostic criteria of clinical obesity
Rubino, F., Cummings, D. E., Eckel, R. H., Cohen, R. V., Wilding, J. P. H., Brown, W. A., Stanford, F. C., Batterham, R. L., Farooqi, I. S., Farpour-Lambert, N. J., et al. (2025) · The Lancet Diabetes & Endocrinology · doi:10.1016/s2213-8587(24)00316-4 · Cited by 1,811
Consensus definition of fetal growth restriction: a Delphi procedure: Consensus definition of FGR
Gordijn, S. J., Beune, I. M., Thilaganathan, B., Papageorghiou, A., Baschat, A. A., Baker, P. N., Silver, R. M., Wynia, K., & Ganzevoort, W. (2016) · Ultrasound in Obstetrics & Gynecology · doi:10.1002/uog.15884 · Cited by 1,638
Early Detection of Prostate Cancer: AUA Guideline
Carter, H. B., Albertsen, P. C., Barry, M. J., Etzioni, R., Freedland, S. J., Greene, K. L., Holmberg, L., Kantoff, P., Konety, B. R., Murad, M. H., et al. (2013) · Journal of Urology · doi:10.1016/j.juro.2013.04.119 · Cited by 1,242
European consensus conference on faecal microbiota transplantation in clinical practice
Cammarota, G., Ianiro, G., Tilg, H., Rajilić-Stojanović, M., Kump, P., Satokari, R., Sokol, H., Arkkila, P., Pintus, C., Hart, A., et al. (2017) · Gut · doi:10.1136/gutjnl-2016-313017 · Cited by 1,207
Hayden, J. K., Smiley, R. A., Alexander, M., Kardong-Edgren, S., & Jeffries, P. R. (2014) · Journal of Nursing Regulation · doi:10.1016/s2155-8256(15)30062-4 · Cited by 1,186
Rodríguez-Mañas, L., Féart, C., Mann, G., Viña, J., Chatterji, S., Chodzko-Zajko, W., Gonzalez-Colaço Harmand, M., Bergman, H., Carcaillon, L., Nicholson, C., et al. (2013) · The Journals of Gerontology: Series A · doi:10.1093/gerona/gls119 · Cited by 1,185
“Statistical Analysis Statistical analyses were performed in accordance with previously established procedures (14).”
MethodGuckenberger, M., Lievens, Y., Bouma, A. B., Collette, L., Dekker, A., deSouza, N. M., Dingemans, A.-M. C., Fournier, B., Hurkmans, C., Lecouvet, F. E., et al. (2020) · The Lancet Oncology · doi:10.1016/s1470-2045(19)30718-1 · Cited by 1,111
Medical Management of Kidney Stones: AUA Guideline
Pearle, M. S., Goldfarb, D. S., Assimos, D. G., Curhan, G., Denu-Ciocca, C. J., Matlaga, B. R., Monga, M., Penniston, K. L., Preminger, G. M., Turk, T. M. T., et al. (2014) · Journal of Urology · doi:10.1016/j.juro.2014.05.006 · Cited by 1,099
Delphi Technique in Health Sciences: A Map
Niederberger, M., & Spranger, J. (2020) · Frontiers in Public Health · doi:10.3389/fpubh.2020.00457 · Cited by 1,072
“Some authors define Delphi techniques more specifically and focus on reaching consensus between the experts (2, 9, 10).”
BackgroundGötestam Skorpen, C., Hoeltzenbein, M., Tincani, A., Fischer-Betz, R., Elefant, E., Chambers, C., da Silva, J., Nelson-Piercy, C., Cetin, I., Costedoat-Chalumeau, N., et al. (2016) · Annals of the Rheumatic Diseases · doi:10.1136/annrheumdis-2015-208840 · Cited by 1,068
Williams, T. A., Lenders, J. W. M., Mulatero, P., Burrello, J., Rottenkolber, M., Adolf, C., Satoh, F., Amar, L., Quinkler, M., Deinum, J., et al. (2017) · The Lancet Diabetes & Endocrinology · doi:10.1016/s2213-8587(17)30135-3 · Cited by 969
Sinha, I. P., Smyth, R. L., & Williamson, P. R. (2011) · PLoS Medicine · doi:10.1371/journal.pmed.1000393 · Cited by 891
“These have also been variously highlighted as important aspects of methodology in other commentaries about the Delphi technique [13–15].”
MethodGates, M., Gates, A., Pieper, D., Fernandes, R. M., Tricco, A. C., Moher, D., Brennan, S. E., Li, T., Pollock, M., Lunny, C., et al. (2022) · BMJ · doi:10.1136/bmj-2022-070849 · Cited by 856
Bishop, D. V. M., Snowling, M. J., Thompson, P. A., Greenhalgh, T., & CATALISE consortium (2016) · PLOS ONE · doi:10.1371/journal.pone.0158753 · Cited by 831
Reis, R. S., Salvo, D., Ogilvie, D., Lambert, E. V., Goenka, S., & Brownson, R. C. (2016) · The Lancet · doi:10.1016/s0140-6736(16)30728-0 · Cited by 719
Diagnosis and Treatment of Overactive Bladder (Non-Neurogenic) in Adults: AUA/SUFU Guideline
Gormley, E. A., Lightner, D. J., Burgio, K. L., Chai, T. C., Clemens, J. Q., Culkin, D. J., Das, A. K., Foster, H. E., Scarpero, H. M., Tessier, C. D., et al. (2012) · Journal of Urology · doi:10.1016/j.juro.2012.09.079 · Cited by 710
Diagnosis and Treatment of Interstitial Cystitis/Bladder Pain Syndrome: AUA Guideline Amendment
Hanno, P. M., Erickson, D., Moldwin, R., & Faraday, M. M. (2015) · Journal of Urology · doi:10.1016/j.juro.2015.01.086 · Cited by 692
“Expert Opinion with consensus achieved using a modified Delphi technique if differences of opinion emerged.(1) For a”
MethodAUA Guideline for the Diagnosis and Treatment of Interstitial Cystitis/Bladder Pain Syndrome
Hanno, P. M., Burks, D. A., Clemens, J. Q., Dmochowski, R. R., Erickson, D., FitzGerald, M. P., Forrest, J. B., Gordon, B., Gray, M., Mayer, R. D., et al. (2011) · Journal of Urology · doi:10.1016/j.juro.2011.03.064 · Cited by 687
Use of Delphi in health sciences research: A narrative review
Shang, Z. (2023) · Medicine · doi:10.1097/md.0000000000032829 · Cited by 657
Melloul, E., Hübner, M., Scott, M., Snowden, C., Prentis, J., Dejong, C. H. C., Garden, O. J., Farges, O., Kokudo, N., Vauthey, J., et al. (2016) · World Journal of Surgery · doi:10.1007/s00268-016-3700-1 · Cited by 626
“A 3-round Web-based Delphi approach was used in this consensus process [21].”
MethodCaudle, K. E., Dunnenberger, H. M., Freimuth, R. R., Peterson, J. F., Burlison, J. D., Whirl-Carrillo, M., Scott, S. A., Rehm, H. L., Williams, M. S., Klein, T. E., et al. (2017) · Genetics in Medicine · doi:10.1038/gim.2016.87 · Cited by 571
“26–50% 8 (14) 6 (11) 5 (11) 5 (11) 7 (19)”
Loyola-Gonzalez, O. (2019) · IEEE Access · doi:10.1109/access.2019.2949286 · Cited by 559
“by using the opinions of several experts in the application area? A statistical method for validating the suitability of an applied machine learning model by using the opinions of several experts in the application area is the Delphi method [165], [166].”
BackgroundMethodInfluentialEvaluation and Treatment of Cryptorchidism: AUA Guideline
Kolon, T. F., Herndon, C. D. A., Baker, L. A., Baskin, L. S., Baxter, C. G., Cheng, E. Y., Diaz, M., Lee, P. A., Seashore, C. J., Tasian, G. E., et al. (2014) · Journal of Urology · doi:10.1016/j.juro.2014.05.005 · Cited by 536
Cammarota, G., Ianiro, G., Kelly, C. R., Mullish, B. H., Allegretti, J. R., Kassam, Z., Putignani, L., Fischer, M., Keller, J. J., Costello, S. P., et al. (2019) · Gut · doi:10.1136/gutjnl-2019-319548 · Cited by 506
Akers, A., Al-Shahi Salman, R., A. Awad, I., Dahlem, K., Flemming, K., Hart, B., Kim, H., Jusue-Torres, I., Kondziolka, D., Lee, C., et al. (2017) · Neurosurgery · doi:10.1093/neuros/nyx091 · Cited by 495
Francis, A. W., Dawson, D. L., & Golijani-Moghaddam, N. (2016) · Journal of Contextual Behavioral Science · doi:10.1016/j.jcbs.2016.05.003 · Cited by 496
Belton, I., MacDonald, A., Wright, G., & Hamlin, I. (2019) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2019.07.002 · Cited by 474
Experts' views on digital competence: Commonalities and differences
Janssen, J., Stoyanov, S., Ferrari, A., Punie, Y., Pannekeet, K., & Sloep, P. (2013) · Computers & Education · doi:10.1016/j.compedu.2013.06.008 · Cited by 473
Methodology Update: Delphi Studies
McPherson, S., Reese, C., & Wendler, M. C. (2018) · Nursing Research · doi:10.1097/nnr.0000000000000297 · Cited by 467
“The number of participant-experts for a study using the Delphi method can vary significantly (Hsu & Sandford, 2007; Mangan, 2011).”
InfluentialThe American Congress of Rehabilitation Medicine Diagnostic Criteria for Mild Traumatic Brain Injury
Silverberg, N. D., Iverson, G. L., Cogan, A., Dams-O-Connor, K., Delmonico, R., Graf, M. J. P., Iaccarino, M. A., Kajankova, M., Kamins, J., McCulloch, K. L., et al. (2023) · Archives of Physical Medicine and Rehabilitation · doi:10.1016/j.apmr.2023.03.036 · Cited by 480
“13,14 The identification, invitation, and characteristics of the expert panel members were described in the previously published article.”
BackgroundVan Melle, E., Frank, J. R., Holmboe, E. S., Dagnone, D., Stockley, D., & Sherbino, J. (2019) · Academic Medicine · doi:10.1097/acm.0000000000002743 · Cited by 451
Ventral Hernia Management: Expert Consensus Guided by Systematic Review
Liang, M. K., Holihan, J. L., Itani, K., Alawadi, Z. M., Gonzalez, J. R. F., Askenasy, E. P., Ballecer, C., Chong, H. S., Goldblatt, M. I., Greenberg, J. A., et al. (2017) · Annals of Surgery · doi:10.1097/sla.0000000000001701 · Cited by 441
Sheth, V. H., Shah, N. P., Jain, R., Bhanushali, N., & Bhatnagar, V. (2024) · The Journal of Prosthetic Dentistry · doi:10.1016/j.prosdent.2022.05.019 · Cited by 440
Multi-criteria decision-making for flood risk management: A survey of the current state of the art
de Brito, M. M., & Evers, M. (2016) · Natural Hazards and Earth System Sciences · doi:10.5194/nhess-16-1019-2016 · Cited by 435
“On the other hand, the results can be influenced by dominant stakeholders and noise in the responses (Hsu and Sandford, 2007).”
BackgroundMethodWaggoner, J., Carline, J. D., & Durning, S. J. (2016) · Academic Medicine · doi:10.1097/acm.0000000000001092 · Cited by 429
Delphi Panels: Research Design, Procedures, Advantages, and Challenges
R Avella, J. (2016) · International Journal of Doctoral Studies · doi:10.28945/3561 · Cited by 426
Eubank, B. H., Mohtadi, N. G., Lafave, M. R., Wiley, J. P., Bois, A. J., Boorman, R. S., & Sheps, D. M. (2016) · BMC Medical Research Methodology · doi:10.1186/s12874-016-0165-8 · Cited by 423
“Subject anonymity can reduce the effects of dominant individuals, and reduce manipulation or coercion to conform to certain viewpoints [65, 66].”
BackgroundRethlefsen, M. L., Kirtley, S., Waffenschmidt, S., Ayala, A. P., Moher, D., Page, M. J., Koffel, J. B., & Group, P.-S. (2021) · Journal of the Medical Library Association · doi:10.5195/jmla.2021.962 · Cited by 409
A National Assessment of Pediatric Readiness of Emergency Departments
Gausche-Hill, M., Ely, M., Schmuhl, P., Telford, R., Remick, K. E., Edgerton, E. A., & Olson, L. M. (2015) · JAMA Pediatrics · doi:10.1001/jamapediatrics.2015.138 · Cited by 416
Identification and analysis of reverse logistics barriers using fuzzy Delphi method and AHP
Bouzon, M., Govindan, K., Rodriguez, C. M. T., & Campos, L. M. S. (2016) · Resources, Conservation and Recycling · doi:10.1016/j.resconrec.2015.05.021 · Cited by 415
Moreira, D. G., Costello, J. T., Brito, C. J., Adamczyk, J. G., Ammer, K., Bach, A. J. E., Costa, C. M. A., Eglin, C., Fernandes, A. A., Fernández-Cuevas, I., et al. (2017) · Journal of Thermal Biology · doi:10.1016/j.jtherbio.2017.07.006 · Cited by 406
“…median = 8; range 3 to 80), the multiple nationalities (13) and different professional backgrounds (4) of the panelists allow a thorough and broad analysis of the use of IRT, illustrating the positive characteristics of the current Delphi procedure (Boulkedid et al., 2011; Hsu and Sandford, 2007).”
BackgroundMethodInfluentialUrban Neighborhood Green Index – A measure of green spaces in urban areas
Gupta, K., Kumar, P., Pathan, S. K., & Sharma, K. P. (2012) · Landscape and Urban Planning · doi:10.1016/j.landurbplan.2012.01.003 · Cited by 404
Montazeri, A., Mohammadi, S., M.Hesari, P., Ghaemi, M., Riazi, H., & Sheikhi-Mobarakeh, Z. (2023) · Systematic Reviews · doi:10.1186/s13643-023-02410-2 · Cited by 433
Diagnosis and management of pemphigus: Recommendations of an international panel of experts
Murrell, D. F., Peña, S., Joly, P., Marinovic, B., Hashimoto, T., Diaz, L. A., Sinha, A. A., Payne, A. S., Daneshpazhooh, M., Eming, R., et al. (2020) · Journal of the American Academy of Dermatology · doi:10.1016/j.jaad.2018.02.021 · Cited by 391
Understanding current and future issues in collaborative consumption: A four-stage Delphi study
Barnes, S. J., & Mattsson, J. (2016) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2016.01.006 · Cited by 384
“…inhibitors to collaborative consumption are ...”; and Q3: “The most important developments to collaborative consumption in the next 10 years will be...” Hsu and Sandford (2007) suggest that the list of rated items can be reduced by focusing on those where more than 70% of respondents rated them…”
BackgroundRees, J., Abrahams, M., Doble, A., Cooper, A., & the Prostatitis Expert Reference Group (PERG) (2015) · BJU International · doi:10.1111/bju.13101 · Cited by 371
Consolidated principles for screening based on a systematic review and consensus process
Dobrow, M. J., Hagens, V., Chafe, R., Sullivan, T., & Rabeneck, L. (2018) · Canadian Medical Association Journal · doi:10.1503/cmaj.171154 · Cited by 359
Fernández-de-las-Peñas, C., & Dommerholt, J. (2018) · Pain Medicine · doi:10.1093/pm/pnx207 · Cited by 336
Gillessen, S., Omlin, A., Attard, G., de Bono, J. S., Efstathiou, E., Fizazi, K., Halabi, S., Nelson, P. S., Sartor, O., Smith, M. R., et al. (2015) · Annals of Oncology · doi:10.1093/annonc/mdv257 · Cited by 322
“Questions and options for answers were later revised and sent a second time to all panel members, including all inputs received shown in an anonymised fashion, so all the panellists could see every comment from their colleagues [11].”
Sonographic examination of uterine niche in non‐pregnant women: a modified Delphi procedure
Jordans, I. P. M., de Leeuw, R. A., Stegwee, S. I., Amso, N. N., Barri‐Soldevila, P. N., van den Bosch, T., Bourne, T., Brölmann, H. A. M., Donnez, O., Dueholm, M., et al. (2019) · Ultrasound in Obstetrics & Gynecology · doi:10.1002/uog.19049 · Cited by 317
Fisher, B. A., Jonsson, R., Daniels, T., Bombardieri, M., Brown, R. M., Morgan, P., Bombardieri, S., Ng, W.-F., Tzioufas, A. G., Vitali, C., et al. (2017) · Annals of the Rheumatic Diseases · doi:10.1136/annrheumdis-2016-210448 · Cited by 311
Measuring severe maternal morbidity: validation of potential measures
Main, E. K., Abreo, A., McNulty, J., Gilbert, W., McNally, C., Poeltler, D., Lanner-Cusin, K., Fenton, D., Gipps, T., Melsop, K., et al. (2016) · American Journal of Obstetrics and Gynecology · doi:10.1016/j.ajog.2015.11.004 · Cited by 313
AMEYAW, E. E., HU, Y., SHAN, M., CHAN, A. P. C., & LE, Y. (2016) · JOURNAL OF CIVIL ENGINEERING AND MANAGEMENT · doi:10.3846/13923730.2014.945953 · Cited by 307
Blockchain in Additive Manufacturing and its Impact on Supply Chains
Kurpjuweit, S., Schmidt, C. G., Klöckner, M., & Wagner, S. M. (2021) · Journal of Business Logistics · doi:10.1111/jbl.12231 · Cited by 305
Utilization and Harmonization of Adult Accelerometry Data: Review and Expert Consensus
WIJNDAELE, K., WESTGATE, K., STEPHENS, S. K., BLAIR, S. N., BULL, F. C., CHASTIN, S. F. M., DUNSTAN, D. W., EKELUND, U., ESLIGER, D. W., FREEDSON, P. S., et al. (2015) · Medicine & Science in Sports & Exercise · doi:10.1249/mss.0000000000000661 · Cited by 297
“Consistent with Delphi principles (16,38), responses were anonymous.”
BackgroundReinero, C., Visser, L. C., Kellihan, H. B., Masseau, I., Rozanski, E., Clercx, C., Williams, K., Abbott, J., Borgarelli, M., & Scansen, B. A. (2020) · Journal of Veterinary Internal Medicine · doi:10.1111/jvim.15725 · Cited by 302
Murray, I. R., Geeslin, A. G., Goudie, E. B., Petrigliano, F. A., & LaPrade, R. F. (2017) · Journal of Bone and Joint Surgery · doi:10.2106/jbjs.16.00793 · Cited by 298
Business models for sustainable consumption in the circular economy: An expert study
Tunn, V. S. C., Bocken, N. M. P., van den Hende, E. A., & Schoormans, J. P. L. (2019) · Journal of Cleaner Production · doi:10.1016/j.jclepro.2018.11.290 · Cited by 298
Towards an expert consensus to delineate a clinical syndrome of chronic breathlessness
Johnson, M. J., Yorke, J., Hansen-Flaschen, J., Lansing, R., Ekström, M., Similowski, T., & Currow, D. C. (2017) · European Respiratory Journal · doi:10.1183/13993003.02277-2016 · Cited by 298
“The first two stages were conducted in accordance with the Delphi method.[25] For stage 3, the results were presented and discussed during an international workshop.”
Urodynamic Studies in Adults: AUA/SUFU Guideline
Winters, J. C., Dmochowski, R. R., Goldman, H. B., Herndon, C. D. A., Kobashi, K. C., Kraus, S. R., Lemack, G. E., Nitti, V. W., Rovner, E. S., & Wein, A. J. (2012) · Journal of Urology · doi:10.1016/j.juro.2012.09.081 · Cited by 290
Wirrell, E. C., Laux, L., Donner, E., Jette, N., Knupp, K., Meskis, M. A., Miller, I., Sullivan, J., Welborn, M., & Berg, A. T. (2017) · Pediatric Neurology · doi:10.1016/j.pediatrneurol.2017.01.025 · Cited by 283
Protocol - realist and meta-narrative evidence synthesis: Evolving Standards (RAMESES)
Greenhalgh, T., Wong, G., Westhorp, G., & Pawson, R. (2011) · BMC Medical Research Methodology · doi:10.1186/1471-2288-11-115 · Cited by 283
“avoidance of ‘groupthink’, openness to dissenting voices) [30,32,33].”
BackgroundAntonini, A., Stoessl, A. J., Kleinman, L. S., Skalicky, A. M., Marshall, T. S., Sail, K. R., Onuk, K., & Odin, P. L. A. (2018) · Current Medical Research and Opinion · doi:10.1080/03007995.2018.1502165 · Cited by 282
Ajayi, S. O., Oyedele, L. O., Bilal, M., Akinade, O. O., Alaka, H. A., Owolabi, H. A., & Kadiri, K. O. (2015) · Resources, Conservation and Recycling · doi:10.1016/j.resconrec.2015.06.001 · Cited by 282
“The technique is a widely used and accepted method of enquiry that is used to achieve convergence of opinion from people within a domain of expertise (Hsu, 2007).”
Mitrev, N., Vande Casteele, N., Seow, C. H., Andrews, J. M., Connor, S. J., Moore, G. T., Barclay, M., Begun, J., Bryant, R., Chan, W., et al. (2017) · Alimentary Pharmacology & Therapeutics · doi:10.1111/apt.14368 · Cited by 282
“for incorporating TDM for anti-TNF agents into modern IBD management.(14-19) Points addressed included scenarios for performing”
MethodInternational Consensus Definition of Low Anterior Resection Syndrome
Keane, C., Fearnhead, N. S., Bordeianou, L. G., Christensen, P., Basany, E. E., Laurberg, S., Mellgren, A., Messick, C., Orangio, G. R., Verjee, A., et al. (2020) · Diseases of the Colon & Rectum · doi:10.1097/dcr.0000000000001583 · Cited by 275
Lemay, K. R., Tulloch, H. E., Pipe, A. L., & Reed, J. L. (2019) · Journal of Cardiopulmonary Rehabilitation and Prevention · doi:10.1097/hcr.0000000000000379 · Cited by 280
Wong, G., Greenhalgh, T., Westhorp, G., & Pawson, R. (2014) · Health Services and Delivery Research · doi:10.3310/hsdr02300 · Cited by 276
Key Elements for, and Indicators of, a Successful Transition: An International Delphi Study
Suris, J.-C., & Akre, C. (2015) · Journal of Adolescent Health · doi:10.1016/j.jadohealth.2015.02.007 · Cited by 272
Castration-Resistant Prostate Cancer: AUA Guideline
Cookson, M. S., Roth, B. J., Dahm, P., Engstrom, C., Freedland, S. J., Hussain, M., Lin, D. W., Lowrance, W. T., Murad, M. H., Oh, W. K., et al. (2013) · Journal of Urology · doi:10.1016/j.juro.2013.05.005 · Cited by 275
Toward a Consensus on Centralization in Surgery
Vonlanthen, R., Lodge, P., Barkun, J. S., Farges, O., Rogiers, X., Soreide, K., Kehlet, H., Reynolds, J. V., Käser, S. A., Naredi, P., et al. (2018) · Annals of Surgery · doi:10.1097/sla.0000000000002965 · Cited by 274
Expert Consensus on Characteristics of Wisdom: A Delphi Method Study
Jeste, D. V., Ardelt, M., Blazer, D., Kraemer, H. C., Vaillant, G., & Meeks, T. W. (2010) · The Gerontologist · doi:10.1093/geront/gnq022 · Cited by 269
Towards a standardised definition for fundamental care: A modified Delphi study
Feo, R., Conroy, T., Jangland, E., Muntlin Athlin, Å., Brovall, M., Parr, J., Blomberg, K., & Kitson, A. (2018) · Journal of Clinical Nursing · doi:10.1111/jocn.14247 · Cited by 265
“There is no single definition of consensus in the Delphi method (Jorm, 2015); however, previously, researchers have argued that 70% agreement is adequate (Hsu & Sandford, 2007).”
MethodChipchase, L., Schabrun, S., Cohen, L., Hodges, P., Ridding, M., Rothwell, J., Taylor, J., & Ziemann, U. (2012) · Clinical Neurophysiology · doi:10.1016/j.clinph.2012.05.003 · Cited by 257
Pollock, M., Fernandes, R. M., Pieper, D., Tricco, A. C., Gates, M., Gates, A., & Hartling, L. (2019) · Systematic Reviews · doi:10.1186/s13643-019-1252-9 · Cited by 228
Bach-Mortensen, A. M., Lange, B. C. L., & Montgomery, P. (2018) · Implementation Science · doi:10.1186/s13012-018-0789-7 · Cited by 253
“To do so, one may draw on stakeholder inclusive methods such as consensus meetings and Delphi panels [105, 106] to determine what types of support TSOs require to appropriately deliver EBIs.”
BackgroundSharlip, I. D., Belker, A. M., Honig, S., Labrecque, M., Marmar, J. L., Ross, L. S., Sandlow, J. I., & Sokal, D. C. (2012) · Journal of Urology · doi:10.1016/j.juro.2012.09.080 · Cited by 247
Walton, H., Spector, A., Tombor, I., & Michie, S. (2017) · British Journal of Health Psychology · doi:10.1111/bjhp.12260 · Cited by 238
“This systematic method could be used for building a consensus (Hsu & Sandford, 2007) regarding which psychometric and implementation qualities are most important, and which qualities should be given the most weighting when developing and evaluating fidelity of delivery and engagement measures.”
MethodExpert appraisal of criteria for assessing gaming disorder: an international Delphi study
Castro‐Calvo, J., King, D. L., Stein, D. J., Brand, M., Carmi, L., Chamberlain, S. R., Demetrovics, Z., Fineberg, N. A., Rumpf, H., Yücel, M., et al. (2021) · Addiction · doi:10.1111/add.15411 · Cited by 236
“The Delphi technique is a method for systematically gathering data from expert respondents, for assessing the extent of agreement and for resolving disagreement [29,30].”
BackgroundMethodLongmuir, P. E., Boyer, C., Lloyd, M., Borghese, M. M., Knight, E., Saunders, T. J., Boiarskaia, E., Zhu, W., & Tremblay, M. S. (2017) · Journal of Sport and Health Science · doi:10.1016/j.jshs.2015.11.004 · Cited by 234
Khalil, A., Beune, I., Hecher, K., Wynia, K., Ganzevoort, W., Reed, K., Lewi, L., Oepkes, D., Gratacos, E., Thilaganathan, B., et al. (2019) · Ultrasound in Obstetrics & Gynecology · doi:10.1002/uog.19013 · Cited by 235
Artificial Intelligence and Parametric Construction Cost Estimate Modeling: State-of-the-Art Review
Elmousalami, H. H. (2020) · Journal of Construction Engineering and Management · doi:10.1061/(asce)co.1943-7862.0001678 · Cited by 229
Misery, L., Ständer, S., Szepietowski, J., Reich, A., Wallengren, J., Evers, A., Takamori, K., Brenaut, E., Gall-Ianotto, C., Fluhr, J., et al. (2017) · Acta Dermato Venereologica · doi:10.2340/00015555-2397 · Cited by 233
Definition of a COPD self-management intervention: International Expert Group consensus
Effing, T. W., Vercoulen, J. H., Bourbeau, J., Trappenburg, J., Lenferink, A., Cafarella, P., Coultas, D., Meek, P., van der Valk, P., Bischoff, E. W. M. A., et al. (2016) · European Respiratory Journal · doi:10.1183/13993003.00025-2016 · Cited by 229
“Data were collected using features of a Delphi technique [14], which was complemented with an additional group meeting [15].”
InfluentialNiederberger, M., Schifano, J., Deckert, S., Hirt, J., Homberg, A., Köberich, S., Kuhn, R., Rommel, A., Sonnberger, M., & the DEWISS network (2024) · PLOS ONE · doi:10.1371/journal.pone.0304651 · Cited by 220
Marotta, A., Martinelli, F., Nanni, S., Orlando, A., & Yautsiukhin, A. (2017) · Computer Science Review · doi:10.1016/j.cosrev.2017.01.001 · Cited by 226
“Delphi method [160] can be helpful to increase the credibility in the results of the interview.”
MethodStrang, J. F., Meagher, H., Kenworthy, L., de Vries, A. L. C., Menvielle, E., Leibowitz, S., Janssen, A., Cohen-Kettenis, P., Shumer, D. E., Edwards-Leeper, L., et al. (2018) · Journal of Clinical Child & Adolescent Psychology · doi:10.1080/15374416.2016.1228462 · Cited by 226
“Various formats have been used for the Delphi method, including in-person work sessions and online survey formats (Hsu & Sandford, 2007).”
Guckenberger, M., Belka, C., Bezjak, A., Bradley, J., Daly, M. E., DeRuysscher, D., Dziadziuszko, R., Faivre-Finn, C., Flentje, M., Gore, E., et al. (2020) · Radiotherapy and Oncology · doi:10.1016/j.radonc.2020.04.001 · Cited by 225
Andratschke, N., Willmann, J., Appelt, A. L., Alyamani, N., Balermpas, P., Baumert, B. G., Hurkmans, C., Høyer, M., Langendijk, J. A., Kaidar-Person, O., et al. (2022) · The Lancet Oncology · doi:10.1016/s1470-2045(22)00447-8 · Cited by 215
Lenters, L. M., Wazny, K., Webb, P., Ahmed, T., & Bhutta, Z. A. (2013) · BMC Public Health · doi:10.1186/1471-2458-13-s3-s23 · Cited by 215
“The purpose of the Delphi exercise was to gather and synthesize expert opinion around the plausible impact estimates of interventions in various settings [53].”
BackgroundReefke, H., & Sundaram, D. (2017) · Omega · doi:10.1016/j.omega.2016.02.003 · Cited by 212
Meah, N., Wall, D., York, K., Bhoyrul, B., Bokhari, L., Sigall, D. A., Bergfeld, W. F., Betz, R. C., Blume-Peytavi, U., Callender, V., et al. (2020) · Journal of the American Academy of Dermatology · doi:10.1016/j.jaad.2020.03.004 · Cited by 211
International Journal of Engineering and Management Research
(2018) · International Journal of Engineering and Management Research · doi:10.21276/ijemr · Cited by 211
Focal Therapy in Prostate Cancer: International Multidisciplinary Consensus on Trial Design
van den Bos, W., Muller, B. G., Ahmed, H. U., Bangma, C. H., Barret, É., Crouzet, S., Eggener, S. E., Gill, I. S., Joniau, S., Kovács, G., et al. (2014) · European Urology · doi:10.1016/j.eururo.2014.01.001 · Cited by 208
“building that uses a sequence of questionnaires to collect data from selected subjects [14].”
Sustainable ecotourism indicators with fuzzy Delphi method – A Philippine perspective
Ocampo, L., Ebisa, J. A., Ombe, J., & Geen Escoto, M. (2018) · Ecological Indicators · doi:10.1016/j.ecolind.2018.05.060 · Cited by 205
Metadata
- Authors
- Hsu, C.-C., & Sandford, B. A.
- Venue
- University of Massachusetts Amherst
- Publisher
- University of Massachusetts Amherst
- Published
- 2007
- Volume
- 12
- Issue
- 10
- Pages
- 10
- OpenAlex
- W2583504027
- Cited by
- 4,500
- References
- 40
- Citations by year
- 2026: 160 · 2025: 361 · 2024: 381 · 2023: 406 · 2022: 373 · 2021: 381 · 2020: 383 · 2019: 375 · 2018: 358 · 2017: 287 · 2016: 298 · 2015: 240 · 2014: 176 · 2013: 129 · 2012: 91
- FWCI
- 516.01
- Type
- journal article
- Language
- en
- Open access
- Licence
- cc-by-nc-nd
- Locations
- ScholarWorks@UMassAmherst (University of Massachusetts Amherst) (submittedVersion, cc-by-nc-nd)
- University of Massachusetts (UMass) Amherst
- Practical assessment, research & evaluation
- Topics
- Social Sciences › Sociology and Political Science › Delphi Technique in Research
- Social Sciences › Decision Sciences › Management Science and Operations Research › Multi-Criteria Decision Making
- Social Sciences › Sociology and Political Science › Focus Groups and Qualitative Methods
- Keywords
- Delphi method, consensus building, expert opinion, participant selection, response rate, needs assessment, program planning, policy determination
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.