The Delphi method: Substance, context, a critique and an annotated bibliography

Pill, Juri

doi:10.1016/0038-0121(71)90041-3

Cited by 579Delphi Technique in ResearchMulti-Criteria Decision MakingReliability and Agreement in Measurement

The Delphi method: Substance, context, a critique and an annotated bibliography

Open access

Pill, Juri·1971·Socio-Economic Planning Sciences

DOI 10.1016/0038-0121(71)90041-3

Open this paper in CoChat — free account

Integrity signals

Registry

Crossref · 10.1016/0038-0121(71)90041-3

Cites

42 of 42 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

The Delphi-type methods involved 14 individuals, an appropriate number for these methods.
“Our Delphi-type methods involved 14 individuals, an appropriate number for these methods, 51 but a number that makes the choice of participants important.”
From Behaviour change techniques: the development and evaluation of a taxonomic method for reporting and describing behaviour change interventions (a suite of five studies involving consensus methods, randomised controlled trials and analysis of qualitative data) (2015) · cites this paper as [51]

Untitled work
(1932) · Medical Entomology and Zoology · Cited by 9,568 · see citing works
An Experimental Application of the DELPHI Method to the Use of Experts
Dalkey, N., & Helmer, O. (1963) · Management Science · doi:10.1287/mnsc.9.3.458 · Cited by 6,061 · see citing works
Science and the Modern World
Fleming, N. B. (2000) · BRILL eBooks · doi:10.1163/9789004495692_008 · Cited by 2,166 · see citing works
Techniques of attitude scale construction.
Edwards, A. L. (1957) · Appleton-Century-Crofts eBooks · doi:10.1037/14423-000 · Cited by 1,670 · see citing works
The Delphi Method: An Experimental Study of Group Opinion
Dalkey, N. C. (1969) · RAND Corporation eBooks · doi:10.7249/rm5888 · Cited by 1,645 · see citing works
A Basis for Scaling Qualitative Data
Guttman, L. (1944) · American Sociological Review · doi:10.2307/2086306 · Cited by 1,333 · see citing works
Consumption Theory in Terms of Revealed Preference
Samuelson, P. A. (1948) · Economica · doi:10.2307/2549561 · Cited by 1,012 · see citing works
Science and the Modern World
Ravenel, M. P. (1926) · American Journal of Public Health · doi:10.2105/ajph.16.4.414-c · Cited by 925 · see citing works
Alternative Approaches to the Theory of Choice in Risk-Taking Situations
Arrow, K. J. (1951) · Econometrica · doi:10.2307/1907465 · Cited by 729 · see citing works
The design of a policy Delphi
Turoff, M. (1970) · Technological Forecasting and Social Change · doi:10.1016/0040-1625(70)90161-7 · Cited by 623 · see citing works
The measurement of attitude
Thurstone, L. L., & Chave, E. J. (1929) · University of Michigan Library Repository · Cited by 576 · see citing works
On the Epistemology of the Inexact Sciences
Helmer, O., & Rescher, N. (1959) · Management Science · doi:10.1287/mnsc.6.1.25 · Cited by 532 · see citing works
ANALYSIS OF THE FUTURE: THE DELPHI METHOD
Helmer, O. (1967) · Defense Technical Information Center (DTIC) · Cited by 381 · see citing works
Techniques of Attitude Scale Construction
Barrett, D. N., & Edwards, A. L. (1957) · The American Catholic Sociological Review · doi:10.2307/3708688 · Cited by 358 · see citing works
REPORT ON A LONG-RANGE FORECASTING STUDY,
Gordon, T. J., & Helmer, O. (1964) · RAND Corporation eBooks · Cited by 316 · see citing works
Assets and liabilities in group problem solving: The need for an integrative function.
Maier, N. R. F. (1967) · Psychological Review · doi:10.1037/h0024737 · Cited by 306 · see citing works
The Way of Zen
Watts, A. W. (1957) · Cited by 269 · see citing works
An Internal Consistency Check for Scale Values Determined by the Method of Successive Intervals
Edwards, A. L., & Thurstone, L. L. (1952) · Psychometrika · doi:10.1007/bf02288780 · Cited by 112 · see citing works
The Prediction of Social and Technological Events
Kaplan, A., Skogstad, A. L., & Girshick, M. A. (1950) · Public Opinion Quarterly · doi:10.1086/266153 · Cited by 105 · see citing works
Theory of Experimental Inference
Hempel, C. G., & Churchman, C. W. (1949) · The Journal of Philosophy · doi:10.2307/2019448 · Cited by 103 · see citing works
THE USE OF THE DELPHI TECHNIQUE IN PROBLEMS OF EDUCATIONAL INNOVATIONS
Helmer, O. (1966) · Cited by 95 · see citing works
The DELPHI Method, II: Structure of Experiments
Brown, B., Cochran, S. W., & Dalkey, N. C. (1969) · RAND Corporation eBooks · doi:10.7249/rm5957 · Cited by 70 · see citing works
Technical Memorandum TM-51-
River, B. (2006) · Cited by 65 · see citing works
SYSTEMATIC USE OF EXPERT OPINIONS
Helmer, O. (1967) · Defense Technical Information Center (DTIC) · Cited by 59 · see citing works
Theory of Experimental Inference
Maziarz, E. A. (1948) · New Scholasticism · doi:10.5840/newscholas194822429 · Cited by 59 · see citing works
Technological forecasting for industry and government
Peterson, R. M., Erickson, W. J., Kamrany, N. M., Steel, T. B., & von Buelow, R. (1970) · Futures · doi:10.1016/0016-3287(70)90018-2 · Cited by 58 · see citing works
PREDICTING THE FUTURE
Dalkey, N. C. (1968) · Defense Technical Information Center (DTIC) · Cited by 28 · see citing works
Experiments in Group Prediction
Dalkey, N. C. (1968) · Cited by 24 · see citing works
The Systematic Use of Expert Judgment in Operations Research
Helmer-Hirschberg, O. (1963) · Cited by 19 · see citing works
COST-EFFECTIVENESS: SOME TRENDS ANALYSIS
Quade, E. S. (1967) · Defense Technical Information Center (DTIC) · Cited by 18 · see citing works
AN APPROACH TO THE STUDY OF A DEVELOPING ECONOMY BY OPERATIONAL GAMING
Helmer, O., & Quade, E. S. (1963) · Defense Technical Information Center (DTIC) · Cited by 15 · see citing works
Prospects of Technological Progress *
Helmer, O. (2018) · Routledge eBooks · doi:10.4324/9781351106450-8 · Cited by 6 · see citing works
Part III: The Education Innovation Study
(1967) · American Behavioral Scientist · doi:10.1177/000276426701000704 · Cited by 5 · see citing works
Analysis of the Exploratory Development Project Evaluation Experiment
Dean, B. V., Mathis, S. J., Lev, B., Svestka, J. A., & Adler, M. (1970) · Scholarly Commons @ Case Western Reserve University · Cited by 4 · see citing works
On a computer-aided systems approach to personnel administration
Taft, M. I., & Reisman, A. (1971) · Socio-Economic Planning Sciences · doi:10.1016/0038-0121(71)90011-5 · Cited by 4 · see citing works
THE FUTURE AS AN OBJECT OF RESEARCH
Rescher, N. (1967) · Defense Technical Information Center (DTIC) · Cited by 4 · see citing works
THE FUTURE OF SCIENCE
Helmer, O. (1967) · Cited by 4 · see citing works
The Prospects for Technological Progress
Berliner, J. S. (2019) · Routledge eBooks · doi:10.4324/9780429314735-18 · Cited by 3 · see citing works
On a Computer-Aided Systems Approach to Personnel Administration
Reisman, A., & Taft, M. I. (1968) · Scholarly Commons @ Case Western Reserve University · Cited by 3 · see citing works
Present worth
Jeynes, P. H. (1952) · Electrical Engineering · doi:10.1109/ee.1952.6437737 · Cited by 2 · see citing works
THE YEAR 2000
Haydon, B. (1967) · Cited by 1 · see citing works
METHODOLOGY OF SOCIETAL STUDIES
Helmer, O. (1967) · Defense Technical Information Center (DTIC) · Cited by 1 · see citing works

Flagged first, then by citations

3 of the 45 references couldn’t be matched to a known paper.

Sorted by citations

The behavior change technique taxonomy (v1) of 93 hierarchically clustered techniques: building an international consensus for the reporting of behavior change interventions
Michie, S., Richardson, M., Johnston, M., Abraham, C., Francis, J., Hardeman, W., Eccles, M. P., Cane, J., & Wood, C. E. (2013) · Annals of Behavioral Medicine · doi:10.1007/s12160-013-9486-6 · Cited by 7,629 · see citing works
The SCARE 2020 guideline: updating consensus Surgical CAse REport (SCARE) guidelines
Agha, R. A., Franchi, T., Sohrabi, C., Mathew, G., Kerwan, A., Thoma, A., Beamish, A. J., Noureldin, A., Rao, A., Vasudevan, B., et al. (2020) · International Journal of Surgery · doi:10.1016/j.ijsu.2020.10.034 · Cited by 5,594 · see citing works
The Delphi Technique: Making Sense of Consensus
Hsu, C.-C., & Sandford, B. A. (2007) · University of Massachusetts Amherst · doi:10.7275/pdz9-th90 · Cited by 4,500 · see citing works
Qualitative Research: Consensus methods for medical and health services research
Jones, J., & Hunter, D. (1995) · BMJ · doi:10.1136/bmj.311.7001.376 · Cited by 3,471 · see citing works
The SCARE 2023 guideline: updating consensus Surgical CAse REport (SCARE) guidelines
Sohrabi, C., Mathew, G., Maria, N., Kerwan, A., Franchi, T., Agha, R. A., & Collaborators (2023) · International Journal of Surgery · doi:10.1097/js9.0000000000000373 · Cited by 3,112 · see citing works
The Delphi technique: myths and realities
Powell, C. (2003) · Journal of Advanced Nursing · doi:10.1046/j.1365-2648.2003.02537.x · Cited by 2,611 · see citing works
The SCARE 2018 statement: Updating consensus Surgical CAse REport (SCARE) guidelines
Agha, R., Borrelli, M. R., Farwana, R., Koshy, K., Fowler, A. J., Orgill, D. P., Zhu, H., Alsawadi, A., Ashraf, N., Rao, A., et al. (2018) · International Journal of Surgery · doi:10.1016/j.ijsu.2018.10.028 · Cited by 2,344 · see citing works
STROCSS 2021: Strengthening the reporting of cohort, cross-sectional and case-control studies in surgery
Mathew, G., Agha, R., Albrecht, J., Goel, P., Mukherjee, I., Pai, P., Dcruz, A. K., Nixon, I. J., Klappenbach, R., Enam, S. A., et al. (2021) · International Journal of Surgery · doi:10.1016/j.ijsu.2021.106165 · Cited by 2,144 · see citing works
Consensus development methods, and their use in clinical guideline development.
Murphy, M., Black, Lamping, McKee, M., Sanderson, Askham, & Marteau (1998) · Health Technology Assessment · doi:10.3310/hta2030 · Cited by 2,042 · see citing works
Searching for an Operational Definition of Frailty: A Delphi Method Based Consensus Statement. The Frailty Operative Definition-Consensus Conference Project
Rodríguez‐Mañas, L., Féart, C., Mann, G. E., Viña, J., Chatterji, S., Chodzko-Zajko, W. J., Harmand, M. G., Bergman, H., Carcaillon‐Bentata, L., Nicholson, C., et al. (2012) · The Journals of Gerontology Series A · doi:10.1093/gerona/gls119 · Cited by 1,187 · see citing works
Combining Probability Distributions: A Critique and an Annotated Bibliography
Genest, C., & Zidek, J. V. (1986) · Statistical Science · doi:10.1214/ss/1177013825 · Cited by 1,000 · see citing works
Qualitative Research: Application of the Delphi Method to CEM Research
Hallowell, M. R., & Gambatese, J. (2009) · Journal of Construction Engineering and Management · doi:10.1061/(asce)co.1943-7862.0000137 · Cited by 931 · see citing works
The max-min Delphi method and fuzzy Delphi method via fuzzy integration
Ishikawa, A., Amagasa, M., Shiga, T., Tomizawa, G., Tatsuta, R., & Mieno, H. (1993) · Fuzzy Sets and Systems · doi:10.1016/0165-0114(93)90251-c · Cited by 915 · see citing works
The PROCESS 2018 statement: Updating Consensus Preferred Reporting Of CasE Series in Surgery (PROCESS) guidelines
Agha, R., Borrelli, M. R., Farwana, R., Koshy, K., Fowler, A. J., Orgill, D. P., Zhu, H., Alsawadi, A., Noureldin, A., Rao, A., et al. (2018) · International Journal of Surgery · doi:10.1016/j.ijsu.2018.10.031 · Cited by 908 · see citing works
The PROCESS 2020 Guideline: Updating Consensus Preferred Reporting Of CasE Series in Surgery (PROCESS) Guidelines
Agha, R., Sohrabi, C., Mathew, G., Franchi, T., Kerwan, A., O’Neill, N., Thoma, A., Beamish, A. J., Noureldin, A., Rao, A., et al. (2020) · International Journal of Surgery · doi:10.1016/j.ijsu.2020.11.005 · Cited by 808 · see citing works
Statistical Methods for Eliciting Probability Distributions
Garthwaite, P. H., Kadane, J. B., & O’Hagan, A. (2005) · Journal of the American Statistical Association · doi:10.1198/016214505000000105 · Cited by 800 · see citing works
Behaviour change techniques: the development and evaluation of a taxonomic method for reporting and describing behaviour change interventions (a suite of five studies involving consensus methods, randomised controlled trials and analysis of qualitative data)
Michie, S., Wood, C. E., Johnston, M., Abraham, C., Francis, J. J., & Hardeman, W. (2015) · Health Technology Assessment · doi:10.3310/hta19990 · Cited by 680 · see citing works
The design of a policy Delphi
Turoff, M. (1970) · Technological Forecasting and Social Change · doi:10.1016/0040-1625(70)90161-7 · Cited by 623 · see citing works
Prediction of adverse maternal outcomes in pre-eclampsia: development and validation of the fullPIERS model
von Dadelszen, P., Payne, B. A., Li, J., Ansermino, J. M., Pipkin, F. B., Côté, A., Douglas, M. J., Gruslin, A., Hutcheon, J. A., Joseph, K. S., et al. (2010) · The Lancet · doi:10.1016/s0140-6736(10)61351-7 · Cited by 585 · see citing works
Delphi: myths and reality
Mullen, P. M. (2003) · Journal of Health Organization and Management · doi:10.1108/14777260310469319 · Cited by 574 · see citing works
Wiley Encyclopedia of Operations Research and Management Science
Cochran, J. (2010) · Wiley eBooks · doi:10.1002/9780470400531 · Cited by 568 · see citing works
STROCSS 2021: Strengthening the reporting of cohort, cross-sectional and case-control studies in surgery
Mathew, G., Agha, R., Albrecht, J., Goel, P., Mukherjee, I., Pai, P., Dcruz, A. K., Nixon, I. J., Roberto, K., Enam, S. A., et al. (2021) · Annals of Medicine and Surgery · doi:10.1016/j.amsu.2021.103026 · Cited by 555 · see citing works
An evaluation of Delphi
Woudenberg, F. (1991) · Technological Forecasting and Social Change · doi:10.1016/0040-1625(91)90002-w · Cited by 530 · see citing works
Delphi as a method to establish consensus for diagnostic criteria
Graham, B. (2003) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(03)00211-7 · Cited by 481 · see citing works
Delphi Panels: Research Design, Procedures, Advantages, and Challenges
Avella, J. R. (2016) · International journal of doctoral studies · doi:10.28945/3561 · Cited by 435 · see citing works
Estimating Burden and Disease Costs of Exposure to Endocrine-Disrupting Chemicals in the European Union
Trasande, L., Zoeller, R. T., Hass, U., Kortenkamp, A., Grandjean, P., Myers, J. P., DiGangi, J., Bellanger, M. M., Hauser, R. B., Legler, J., et al. (2015) · The Journal of Clinical Endocrinology & Metabolism · doi:10.1210/jc.2014-4324 · Cited by 382 · see citing works
Which Clinical Anesthesia Outcomes Are Both Common and Important to Avoid? The Perspective of a Panel of Expert Anesthesiologists
Macario, A., Weinger, M. B., Truong, P., & Lee, M. (1999) · Anesthesia & Analgesia · doi:10.1097/00000539-199905000-00023 · Cited by 377 · see citing works
Camberwell Assessment of Need for the Elderly (CANE)
Reynolds, T., Thornicroft, G., Abas, M., Woods, B. T., Hoe, J., Leese, M. N., & Orrell, M. W. (2000) · The British Journal of Psychiatry · doi:10.1192/bjp.176.5.444 · Cited by 366 · see citing works
The role of primary care in preventing ambulatory care sensitive conditions
Caminal, J. (2004) · European Journal of Public Health · doi:10.1093/eurpub/14.3.246 · Cited by 337 · see citing works
Core Outcome Measures in Effectiveness Trials (COMET) initiative: protocol for an international Delphi study to achieve consensus on how to select outcome measurement instruments for outcomes included in a ‘core outcome set’
Prinsen, C. A. C., Vohra, S., Rose, M. R., King-Jones, S., Ishaque, S., Bhaloo, Z., Adams, D., & Terwee, C. B. (2014) · Trials · doi:10.1186/1745-6215-15-247 · Cited by 332 · see citing works
An environmental evaluation system for water resource planning
Dee, N., Baker, J., Drobny, N., Duke, K., Whitman, I., & Fahringer, D. (1973) · Water Resources Research · doi:10.1029/wr009i003p00523 · Cited by 302 · see citing works
Strengthening evaluation and implementation by specifying components of behaviour change interventions: a study protocol
Michie, S., Abraham, C., Eccles, M. P., Francis, J. J., Hardeman, W., & Johnston, M. (2011) · Implementation Science · doi:10.1186/1748-5908-6-10 · Cited by 301 · see citing works
Assessment of clinical signs of atopic dermatitis: A systematic review and recommendation
Schmitt, J. M., Langan, S., Deckert, S., Svensson, Å., von Kobyletzki, L., Thomas, K. S., & Spuls, P. I. (2013) · Journal of Allergy and Clinical Immunology · doi:10.1016/j.jaci.2013.07.008 · Cited by 295 · see citing works
Seeking worldwide professional consensus on the principles of end-of-life care for the critically ill. The Consensus for Worldwide End-of-Life Practice for Patients in Intensive Care Units (WELPICUS) study
Sprung, C. L., Truog, R. D., Curtis, J. R., Joynt, G. M., Baras, M., Michalsen, A., Briegel, J., Kesecioglu, J., Efferen, L., De Robertis, E., et al. (2014) · American Journal of Respiratory and Critical Care Medicine · doi:10.1164/rccm.201403-0593cc · Cited by 250 · see citing works
Identifying the clinical domains of fibromyalgia: Contributions from clinician and patient delphi exercises
Mease, P. J., Arnold, L. M., Crofford, L. J., Williams, D. A., Russell, I. J., Humphrey, L., Abetz, L. N., & Martin, S. A. (2008) · Arthritis Care & Research · doi:10.1002/art.23826 · Cited by 231 · see citing works
Triage
Christian, M. D., Sprung, C. L., King, M. A., Dichter, J. R., Kissoon, N. “., Devereaux, A. V., & Gomersall, C. D. (2014) · CHEST Journal · doi:10.1378/chest.14-0736 · Cited by 220 · see citing works
Sustainable ecotourism indicators with fuzzy Delphi method – A Philippine perspective
Ocampo, L. A., Ebisa, J. A., Ombe, J., & Escoto, M. G. (2018) · Ecological Indicators · doi:10.1016/j.ecolind.2018.05.060 · Cited by 220 · see citing works
Medical Assessment by a Delphi Group Opinion Technic
Milholland, A. V., Wheeler, S. G., & Heieck, J. J. (1973) · New England Journal of Medicine · doi:10.1056/nejm197306142882405 · Cited by 218 · see citing works
Competencies for distance education professionals
Thach, E. C., & Murphy, K. L. (1995) · Educational Technology Research and Development · doi:10.1007/bf02300482 · Cited by 215 · see citing works
Standardized outcome measures for pregnancy and childbirth, an ICHOM proposal
Nijagal, M. A., Wissig, S., Stowell, C. J., Olson, E. M., Amer‐Wåhlin, I., Bonsel, G. J., Brooks, A., Coleman, M., Karalasingam, S. D., Duffy, J. M., et al. (2018) · BMC Health Services Research · doi:10.1186/s12913-018-3732-3 · Cited by 205 · see citing works
Development of a standardized definition for Hirschsprung's-associated enterocolitis: a Delphi analysis
Pastor, A. C., Osman, F., Teitelbaum, D. H., Caty, M. G., & Langer, J. C. (2009) · Journal of Pediatric Surgery · doi:10.1016/j.jpedsurg.2008.10.052 · Cited by 200 · see citing works
Defining Physical Literacy for Application in Australia: A Modified Delphi Method
Keegan, R. J., Barnett, L. M., Dudley, D. A., Telford, R. D., Lubans, D. R., Bryant, A. S., Roberts, W., Morgan, P. J., Schranz, N. K., Weissensteiner, J. R., et al. (2019) · Journal of Teaching in Physical Education · doi:10.1123/jtpe.2018-0264 · Cited by 187 · see citing works
Consensus proposal for taxonomy and definition of the autoinflammatory diseases (AIDs): a Delphi study
Ben‐Chetrit, E., Gattorno, M., Gül, A., Kastner, D. L., Lachmann, H. J., Touitou, I., & Ruperto, N. (2018) · Annals of the Rheumatic Diseases · doi:10.1136/annrheumdis-2017-212515 · Cited by 180 · see citing works
Core Outcome Domains for Controlled Trials and Clinical Recordkeeping in Eczema: International Multiperspective Delphi Consensus Process
Schmitt, J. M., panel, O. B. O. T. H. O. M. I. E. (. D., Langan, S., Stamm, T., & Williams, H. C. (2010) · Journal of Investigative Dermatology · doi:10.1038/jid.2010.303 · Cited by 175 · see citing works
A proposed set of metrics for standardized outcome reporting in the management of low back pain
Clement, R. C., Welander, A., Stowell, C. J., Cha, T. D., Chen, J. L., Davies, M., Fairbank, J. C. T., Foley, K. T., Gehrchen, M., Hägg, O., et al. (2015) · Acta Orthopaedica · doi:10.3109/17453674.2015.1036696 · Cited by 172 · see citing works
EL MÉTODO DELPHI EN LA INVESTIGACIÓN ACTUAL EN EDUCACIÓN: UNA REVISIÓN TEÓRICA Y METODOLÓGICA
Gómez, E. L. (2017) · Educación XX1 · doi:10.5944/educxx1.20169 · Cited by 172 · see citing works
Defining an International Standard Set of Outcome Measures for Patients With Hip or Knee Osteoarthritis: Consensus of the International Consortium for Health Outcomes Measurement Hip and Knee Osteoarthritis Working Group
Rolfson, O., Wissig, S., van Maasakkers, L., Stowell, C. J., Ackerman, I. N., Ayers, D. C., Barber, T., Benzakour, T., Bozic, K. J., Budhiparama, N. C., et al. (2016) · Arthritis Care & Research · doi:10.1002/acr.22868 · Cited by 168 · see citing works
Determinants of forward stage transitions: a Delphi study
de Vet, E. (2004) · Health Education Research · doi:10.1093/her/cyg111 · Cited by 167 · see citing works
Definitions of terms, processes and a minimum dataset for transperineal prostate biopsies: a standardization approach of the G insburg S tudy G roup for E nhanced P rostate D iagnostics
Kuru, T. H., Wadhwa, K., Chang, R. T. M., Echeverria, L. M. C., Roethke, M. C., Polson, A., Rottenberg, G., Koo, B., Lawrence, E. M., Seidenader, J., et al. (2013) · British Journal of Urology · doi:10.1111/bju.12132 · Cited by 158 · see citing works
STROCSS 2021: Strengthening the reporting of cohort, cross-sectional and case-control studies in surgery
Mathew, G., & Agha, R. (2021) · International Journal of Surgery Open · doi:10.1016/j.ijso.2021.100430 · Cited by 155 · see citing works
Impeding challenges on industry 4.0 in circular economy: Palm oil industry in Malaysia
Abdul-Hamid, A.-Q., Ali, M. H., Tseng, M., Lan, S., & Kumar, M. (2020) · Computers & Operations Research · doi:10.1016/j.cor.2020.105052 · Cited by 152 · see citing works
Evaluating regional low-carbon tourism strategies using the fuzzy Delphi- analytic network process approach
Zhang, J. (2016) · Journal of Cleaner Production · doi:10.1016/j.jclepro.2016.09.122 · Cited by 148 · see citing works
Consultant surgeons' opinion of the skills required of basic surgical trainees
Baldwin, P. J., Paisley, A., & Brown, S. P. (1999) · British journal of surgery · doi:10.1046/j.1365-2168.1999.01169.x · Cited by 142 · see citing works
Limitations in validating emergency department triage scales
Twomey, M., Wallis, L. A., & Myers, J. E. (2007) · Emergency Medicine Journal · doi:10.1136/emj.2007.046383 · Cited by 140 · see citing works
The mechanics of implementation strategies and measures: advancing the study of implementation mechanisms
Lewis, C. C., Klasnja, P., Lyon, A. R., Powell, B. J., Lengnick‐Hall, R., Buchanan, G. J. R., Meza, R. D., Chan, M. C., Boynton, M. H., & Weiner, B. J. (2022) · Implementation Science Communications · doi:10.1186/s43058-022-00358-3 · Cited by 139 · see citing works
Methods for Deriving Criteria for Assessing the Quality of Medical Care
Donabedian, A. (1980) · Medical Care Review · doi:10.1177/107755878003700702 · Cited by 136 · see citing works
The Delphi Technique and Its Uses In Social Science Research*
Strauss, H. J., & Zeigler, L. H. (1975) · The Journal of Creative Behavior · doi:10.1002/j.2162-6057.1975.tb00574.x · Cited by 135 · see citing works
Descripción y usos del método Delphi en investigaciones del área de la salud
Varela-Ruíz, M., Díaz-Bravo, L., & García-Durán, R. (2012) · Investigación en Educación Médica · doi:10.22201/fm.20075057e.2012.02.00007 · Cited by 130 · see citing works
Developing weighting system for refurbishment building assessment scheme in Malaysia through analytic hierarchy process (AHP) approach
Kamaruzzaman, S. N., Lou, E. C. W., Wong, P. F., Wood, F. R., & Ani, A. I. C. (2017) · Energy Policy · doi:10.1016/j.enpol.2017.10.023 · Cited by 130 · see citing works
Setting the Scope of Concept Inventories for Introductory Computing Subjects
Goldman, K., Gross, P., Heeren, C., Herman, G. L., Kaczmarczyk, L. C., Loui, M. C., & Zilles, C. B. (2010) · ACM Transactions on Computing Education · doi:10.1145/1789934.1789935 · Cited by 122 · see citing works
Identifying important and difficult concepts in introductory computing courses using a delphi process
Goldman, K., Gross, P., Heeren, C., Herman, G. L., Kaczmarczyk, L. C., Loui, M. C., & Zilles, C. B. (2008) · ACM Technical Symposium on Computer Science Education (SIGCSE) · doi:10.1145/1352135.1352226 · Cited by 122 · see citing works
Evaluation of Postoperative Handover Using a Tool to Assess Information Transfer and Teamwork
Nagpal, K., Abboudi, M., Fischler, L., Schmidt, T., Vats, A., Manchanda, C., Sevdalis, N., Scheidegger, D. H., Vincent, C. A., & Moorthy, K. (2011) · Annals of Surgery · doi:10.1097/sla.0b013e318211d849 · Cited by 117 · see citing works
Advancing implementation science through measure development and evaluation: a study protocol
Lewis, C. C., Weiner, B. J., Stanick, C., & Fischer, S. M. (2015) · Implementation Science · doi:10.1186/s13012-015-0287-0 · Cited by 115 · see citing works
The Delphi technique: an untapped approach of participatory research
Kezar, A. J., & Maxey, D. (2014) · International Journal of Social Research Methodology · doi:10.1080/13645579.2014.936737 · Cited by 114 · see citing works
Calculation of the disease burden associated with environmental chemical exposures: application of toxicological information in health economic estimation
Grandjean, P., & Bellanger, M. M. (2017) · Environmental Health · doi:10.1186/s12940-017-0340-3 · Cited by 112 · see citing works
Delphi-based consensus study into a framework of community resilience to disaster
Alshehri, S. A., Rezgui, Y., & Li, H. (2014) · Natural Hazards · doi:10.1007/s11069-014-1423-x · Cited by 110 · see citing works
Providing continuity of care for people with severe mental illness
Crawford, M., de Jonge, E., Freeman, G. K., & Weaver, T. D. (2004) · Social Psychiatry and Psychiatric Epidemiology · doi:10.1007/s00127-004-0732-x · Cited by 110 · see citing works
Nature-based Tourism and Environmental Sustainability in South Africa
Spenceley, A. (2005) · Journal of Sustainable Tourism · doi:10.1080/09669580508668483 · Cited by 108 · see citing works
THE POLICY DELPHI: PURPOSE, STRUCTURE, AND APPLICATION
Needham, R. D., & de Loë, R. C. (1990) · Canadian Geographies / Géographies canadiennes · doi:10.1111/j.1541-0064.1990.tb01258.x · Cited by 108 · see citing works
How to e-mental health: a guideline for researchers and practitioners using digital technology in the context of mental health
Seiferth, C., Vogel, L., Aas, B., Brandhorst, I., Carlbring, P., Conzelmann, A., Esfandiari, N., Finkbeiner, M., Hollmann, K., Lautenbacher, H., et al. (2023) · Nature Mental Health · doi:10.1038/s44220-023-00085-1 · Cited by 99 · see citing works
An exploration of the relationship between Internet self-efficacy and sources of Internet self-efficacy among Taiwanese university students
Chuang, S., Lin, F.-M., & Tsai, C. (2015) · Computers in Human Behavior · doi:10.1016/j.chb.2015.01.044 · Cited by 96 · see citing works
Innovation mechanisms in German precision farming
Busse, M., Doernberg, A., Siebert, R., Kuntosch, A., Schwerdtner, W., König, B., & Bokelmann, W. (2013) · Precision Agriculture · doi:10.1007/s11119-013-9337-2 · Cited by 96 · see citing works
Food waste prevention along the food supply chain: A multi-actor approach to identify effective solutions
Díaz-Ruiz, R., Costa‐Font, M., López‐i‐Gelats, F., & Roig, J. M. G. (2019) · Resources Conservation and Recycling · doi:10.1016/j.resconrec.2019.05.031 · Cited by 94 · see citing works
Developing and testing a framework for the assessment of neighbourhood liveability in two contrasting countries: Iran and Estonia
Norouzian‐Maleki, S., Bell, S., Hosseini, S. B., & Faizi, M. (2014) · Ecological Indicators · doi:10.1016/j.ecolind.2014.07.033 · Cited by 92 · see citing works
Instituting Surveillance Guidelines and Adverse Outcomes in Preeclampsia
Menzies, J., Magee, L. A., Li, J., MacNab, Y. C., Yin, R., Stuart, H., Baraty, B., Lam, E. M., Hamilton, T., Lee, S. K., et al. (2007) · Obstetrics and Gynecology · doi:10.1097/01.aog.0000266977.26311.f0 · Cited by 92 · see citing works
Consensus strategies for the nonoperative management of patients with blunt splenic injury
Olthof, D. C., van der Vlies, C. H., Joosse, P., van Delden, O. M., Jurkovich, G. J., & Goslings, J. C. (2013) · The Journal of Trauma: Injury, Infection, and Critical Care · doi:10.1097/ta.0b013e3182921627 · Cited by 91 · see citing works
Determinants of awareness, initiation and maintenance of physical activity among the over-fifties: a Delphi study
van Stralen, M. M., Lechner, L., Mudde, A. N., de Vries, H., & Bolman, C. A. W. (2008) · Health Education Research · doi:10.1093/her/cyn045 · Cited by 91 · see citing works
Improving the detection of chronic migraine: Development and validation of Identify Chronic Migraine (ID-CM)
Lipton, R. B., Serrano, D., Buse, D. C., Pavlovic, J. M., Blumenfeld, A. M., Dodick, D. W., Aurora, S. K., Becker, W. J., Diener, H., Wang, S., et al. (2015) · Cephalalgia · doi:10.1177/0333102415583982 · Cited by 86 · see citing works
Laterality indices consensus initiative (LICI): A Delphi expert survey report on recommendations to record, assess, and report asymmetry in human behavioural and brain research
Vingerhoets, G., Verhelst, H., Gerrits, R., Badcock, N. A., Bishop, D., Carey, D. P., Flindall, J. W., Grimshaw, G. M., Harris, L. J., Hausmann, M., et al. (2023) · Laterality Asymmetries of Body Brain and Cognition · doi:10.1080/1357650x.2023.2199963 · Cited by 78 · see citing works
Best Practice Guidelines for Hip Arthroscopy in Femoroacetabular Impingement: Results of a Delphi Process
Lynch, T. S., Minkara, A. A., Aoki, S. K., Bedi, A., Bharam, S., Clohisy, J. C. F., Harris, J. D., Larson, C. M., Nepple, J. J., Nho, S. J., et al. (2019) · Journal of the American Academy of Orthopaedic Surgeons · doi:10.5435/jaaos-d-18-00041 · Cited by 77 · see citing works
Establishing consensus on the best practice guidelines for the use of bracing in adolescent idiopathic scoliosis
Roye, B. D., Simhon, M. E., Matsumoto, H., Bakarania, P., Berdishevsky, H., Dolan, L. A., Grimes, K., Grivas, T. B., Hresko, M. T., Karol, L. A., et al. (2020) · Spine Deformity · doi:10.1007/s43390-020-00060-1 · Cited by 76 · see citing works
STROCSS 2021: strengthening the reporting of cohort, cross-sectional and case-control studies in surgery
Mathew, G., & Agha, R. (2021) · IJS Short Reports · doi:10.1097/sr9.0000000000000035 · Cited by 73 · see citing works
Domains of quality of life: results of a three-stage Delphi consensus procedure among patients, family of patients, clinicians, scientists and the general public
Pietersma, S., de Vries, M., & Marle, M. E. V. D. A. (2013) · Quality of Life Research · doi:10.1007/s11136-013-0578-3 · Cited by 73 · see citing works
PIERS Proteinuria: Relationship With Adverse Maternal and Perinatal Outcome
Payne, B. A., Magee, L. A., Côté, A., Hutcheon, J. A., Li, J., Kyle, P. M., Menzies, J. M., Moore, M. P., Parker, C., Pullar, B., et al. (2011) · Journal of Obstetrics and Gynaecology Canada · doi:10.1016/s1701-2163(16)34907-6 · Cited by 73 · see citing works
Quality indicators for Delphi studies
Rodríguez, J. L., & Lertxundi, A. L. (2023) · Futures & Foresight Science · doi:10.1002/ffo2.172 · Cited by 72 · see citing works
The Prediction of Adverse Maternal Outcomes in Preeclampsia
von Dadelszen, P., Magee, L. A., Devarakonda, R. M., Hamilton, T., Ainsworth, L. M., Yin, R., Norena, M., Walley, K. R., Gruslin, A., Moutquin, J., et al. (2004) · Journal of Obstetrics and Gynaecology Canada · doi:10.1016/s1701-2163(16)30137-2 · Cited by 72 · see citing works
Monitoring Vital Signs: Development of a Modified Early Warning Scoring (Mews) System for General Wards in a Developing Country
Kyriacos, U., Jelsma, J. M., James, M. F., & Jordan, S. E. (2014) · PLoS ONE · doi:10.1371/journal.pone.0087073 · Cited by 70 · see citing works
Costs and clinical outcomes of primary prophylaxis of variceal bleeding in patients with hepatic cirrhosis: a decision analytic model
Saab, S., DeRosa, V. P., Nieto, J. M., Durazo, F. A., Han, S. H. B., & Roth, B. E. (2003) · The American Journal of Gastroenterology · doi:10.1016/s0002-9270(03)00056-x · Cited by 70 · see citing works
A psychometric evaluation of the digital logic concept inventory
Herman, G. L., Zilles, C. B., & Loui, M. C. (2014) · Computer Science Education · doi:10.1080/08993408.2014.970781 · Cited by 69 · see citing works
The Delphi Process in Dental Research
Cramer, C. K., Klasser, G. D., Epstein, J. B., & Sheps, S. B. (2008) · Journal of Evidence Based Dental Practice · doi:10.1016/j.jebdp.2008.09.002 · Cited by 68 · see citing works
Identifying Core Concepts of Cybersecurity: Results of Two Delphi Processes
Parekh, G., DeLatte, D., Herman, G. L., Oliva, L., Phatak, D. S., Scheponik, T., & Sharman, A. T. (2017) · IEEE Transactions on Education · doi:10.1109/te.2017.2715174 · Cited by 68 · see citing works
Scientific inquiry or political critique?
Goldschmidt, P. G. (1975) · Technological Forecasting and Social Change · doi:10.1016/0040-1625(75)90059-1 · Cited by 67 · see citing works
A review of program evaluation and fund allocation methods within the service and government sectors
Zanakis, S. H., Mandakovic, T., Gupta, S. K., Sahay, S., & Hong, S.-W. (1995) · Socio-Economic Planning Sciences · doi:10.1016/0038-0121(95)98607-w · Cited by 67 · see citing works
Guidelines for Using the Delphi Technique to Develop Habitat Suitability Index Curves
Crance, J. H. (1987) · Cited by 67 · see citing works
Lack of consensus on tests and criteria for generalized joint hypermobility, Ehlers-Danlos syndrome: hypermobile type and joint hypermobility syndrome
Remvig, L., Flycht, L., Christensen, K. B., & Juul‐Kristensen, B. (2014) · American Journal of Medical Genetics Part A · doi:10.1002/ajmg.a.36402 · Cited by 66 · see citing works
Review of the Delphi method in library and information science research
Lund, B. D. (2020) · Journal of Documentation · doi:10.1108/jd-09-2019-0178 · Cited by 65 · see citing works
The Delphi technique and judgmental forecasting
Stewart, T. R. (1987) · Climatic Change · doi:10.1007/bf00138797 · Cited by 65 · see citing works
Principles in practice: Toward a conceptual framework for resilient urban design
Lak, A., Hasankhan, F., & Garakani, S. A. (2020) · Journal of Environmental Planning and Management · doi:10.1080/09640568.2020.1714561 · Cited by 64 · see citing works
Validating green building social sustainability indicators in China using the fuzzy delphi method
Tseng, M., Li, S.-X., Lin, C.-W. R., & Chiu, A. S. F. (2022) · Journal of Industrial and Production Engineering · doi:10.1080/21681015.2022.2070934 · Cited by 64 · see citing works
A procedure for assessing recreational carrying capacity of coastal resort areas
Sowman, M. R. (1987) · Landscape and Urban Planning · doi:10.1016/0169-2046(87)90044-2 · Cited by 64 · see citing works

Authors: Pill, J.
Affiliations: Case Western Reserve University (US)
Corresponding author: Pill, Juri
Venue: Socio-Economic Planning Sciences
Publisher: Elsevier BV
Published: 1971-02
Volume: 5
Issue: 1
Pages: 57-71
ISSN: 0038-0121 (ISSN-L)
DOI: 10.1016/0038-0121(71)90041-3
OpenAlex: W1968468300
Cited by: 579
References: 45
Citations by year: 2026: 6 · 2025: 11 · 2024: 14 · 2023: 25 · 2022: 25 · 2021: 27 · 2020: 31 · 2019: 30 · 2018: 27 · 2017: 27 · 2016: 22 · 2015: 31 · 2014: 29 · 2013: 27 · 2012: 10
FWCI: 26.64
Type: journal article
Language: en
Open access: green
commons.case.edu
Licence: public-domain
www.elsevier.com/tdm/userlicense/1.0/ (tdm)
Locations: Socio-Economic Planning Sciences (publishedVersion)
Scholarly Commons @ Case Western Reserve University (submittedVersion, public-domain)
RePEc: Research Papers in Economics (submittedVersion)
Topics: Social Sciences › Sociology and Political Science › Delphi Technique in Research
Social Sciences › Decision Sciences › Management Science and Operations Research › Multi-Criteria Decision Making
Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement
Keywords: Delphi method, consensus building, expert panel, subjective scaling, uncertainty, historical development, evaluation methodology, annotated bibliography

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

The Delphi method: Substance, context, a critique and an annotated bibliography

Integrity signals

Claims the paper makes

Claims other papers make about this one

Cites

Cited by

Related

Record

About these checks