Preparing, conducting, and analyzing Delphi surveys: Cross-disciplinary practices, new directions, and advancements
Open accessAbstract
Delphi is a scientific method to organize and structure an expert discussion aiming to generate insights on controversial topics with limited information. The technique has seen a rise in publication frequency in various disciplines, especially over the past decades. In April 2021, the term Delphi method yielded 28,200 search hits in Google Scholar for the past five years alone. Given the increasing level of uncertainty caused by rapid technological and social change around the globe, collective expert opinions and assessments are likely to gain even more importance. Therefore, the paper at hand presents technical recommendations derived from a Delphi study that was conducted amid the outbreak of the COVID-19 pandemic in 2020. The paper comprehensively demonstrates how to prepare, conduct, and analyze a Delphi study. In this regard, it combines several methodological advancements of the recent past (e.g., dissent analyses, scenario analyses) with state-of-the-art impulses from other disciplines like strategic management (e.g., fuzzy clustering), psychology (e.g., sentiment analyses), or clinical trials (e.g., consensus measurement). By offering insights on the variety of possibilities to exploit Delphi-based data, we aim to support researchers across all disciplines in conducting Delphi studies and potentially expand and improve the method's field of application.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The study closely followed the guidelines proposed by Beiderbeck et al. (2021) to assure reliability and validity.
“In order to assure the reliability and validity of the current study, we closely followed the guidelines proposed by Beiderbeck et al. (2021), including the general phases of preparing, conducting, and analysing the Delphi study.”
From Artificial Intelligence in Education: Implications for Policymakers, Researchers, and Practitioners (2024) · cites this paper as Beiderbeck et al., 2021
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Russell, J. A. (1980) · Journal of Personality and Social Psychology · doi:10.1037/h0077714 · Cited by 16,334 · cited 1× · see citing works
“To account for the individual wellbeing of participants there are numerous alternatives to PANAS, such as the "profile of mood states" or "circumplex model of affect" [75] .”
MethodDellinger, R. P., Levy, M. M., Carlet, J. M., Bion, J., Parker, M. M., Jaeschke, R., Reinhart, K., Angus, D. C., Brun‐Buisson, C., Beale, R. J., et al. (2007) · Intensive Care Medicine · doi:10.1007/s00134-007-0934-2 · Cited by 6,722 · see citing works
An Experimental Application of the DELPHI Method to the Use of Experts
Dalkey, N., & Helmer, O. (1963) · Management Science · doi:10.1287/mnsc.9.3.458 · Cited by 6,061 · see citing works
Research guidelines for the Delphi survey technique
Hasson, F. (2000) · Journal of Advanced Nursing · doi:10.1046/j.1365-2648.2000.01567.x · Cited by 5,951 · see citing works
Global prevalence of dementia: a Delphi consensus study
Ferri, C. P., Prince, M. J., Brayne, C., Brodaty, H., Fratiglioni, L., Ganguli, M., Hall, K. S., Hasegawa, K., Hendrie, H. C., Huang, Y., et al. (2005) · The Lancet · doi:10.1016/s0140-6736(05)67889-0 · Cited by 5,336 · see citing works
The Delphi method as a research tool: an example, design considerations and applications
Okoli, C., & Pawlowski, S. D. (2004) · Information & Management · doi:10.1016/j.im.2003.11.002 · Cited by 4,771 · see citing works
Mokkink, L. B., Terwee, C. B., Patrick, D. L., Alonso, J., Stratford, P. W., Knol, D. L., Bouter, L. M., & de Vet, H. C. W. (2010) · Quality of Life Research · doi:10.1007/s11136-010-9606-8 · Cited by 4,260 · see citing works
Frailty Consensus: A Call to Action
Morley, J. E., Vellas, B. J., van Kan, G. A., Anker, S. D., Bauer, J. M., Bernabei, R., Cesari, M., Chumlea, W. C., Doehner, W., Evans, J., et al. (2013) · Journal of the American Medical Directors Association · doi:10.1016/j.jamda.2013.03.022 · Cited by 4,078 · see citing works
Diamond, I. R., Grant, R. C., Feldman, B. M., Pencharz, P. B., Ling, S. C., Moore, A. M., & Wales, P. W. (2014) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2013.12.002 · Cited by 3,740 · see citing works
Qualitative Research: Consensus methods for medical and health services research
Jones, J., & Hunter, D. (1995) · BMJ · doi:10.1136/bmj.311.7001.376 · Cited by 3,471 · see citing works
Content Analysis in Mass Communication: Assessment and Reporting of Intercoder Reliability
Lombard, M., Snyder-Duch, J., & Bracken, C. C. (2002) · Human Communication Research · doi:10.1111/j.1468-2958.2002.tb00826.x · Cited by 3,068 · see citing works
Group Techniques for Program Planning: A Guide to Nominal Group and Delphi Processes
(1976) · The Journal of Applied Behavioral Science · doi:10.1177/002188637601200414 · Cited by 2,811 · see citing works
The Technology Acceptance Model: Past, Present, and Future
Lee, Y., Kozar, K. A., & Larsen, K. R. T. (2003) · Communications of the Association for Information Systems · doi:10.17705/1cais.01250 · Cited by 2,572 · see citing works
Thompson, E. R. (2007) · Journal of Cross-Cultural Psychology · doi:10.1177/0022022106297301 · Cited by 2,399 · see citing works
Jünger, S., Payne, S. A., Brine, J., Radbruch, L., & Brearley, S. G. (2017) · Palliative Medicine · doi:10.1177/0269216317690685 · Cited by 2,210 · see citing works
Consensus methods: characteristics and guidelines for use
Fink, A., Kosecoff, J., Chassin, M., & Brook, R. H. (1984) · American Journal of Public Health · doi:10.2105/ajph.74.9.979 · Cited by 2,089 · see citing works
Boulkedid, R., Abdoul, H., Loustau, M., Sibony, O., & Alberti, C. (2011) · PLoS ONE · doi:10.1371/journal.pone.0020476 · Cited by 2,045 · see citing works
Consensus development methods, and their use in clinical guideline development.
Murphy, M., Black, Lamping, McKee, M., Sanderson, Askham, & Marteau (1998) · Health Technology Assessment · doi:10.3310/hta2030 · Cited by 2,042 · see citing works
Consensus measurement in Delphi studies
von der Gracht, H. A. (2012) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2012.04.013 · Cited by 1,831 · see citing works
Bishop, D. V. M., Snowling, M. J., Thompson, P. A., Greenhalgh, T., & and the CATALISE‐2 consortium (2017) · Journal of Child Psychology and Psychiatry · doi:10.1111/jcpp.12721 · Cited by 1,641 · see citing works
Conducting Semi‐Structured Interviews
Adams, W. C. (2015) · doi:10.1002/9781119171386.ch19 · Cited by 1,533 · see citing works
A critical review of the Delphi technique as a research methodology for nursing
Keeney, S., Hasson, F., & McKenna, H. (2001) · International Journal of Nursing Studies · doi:10.1016/s0020-7489(00)00044-4 · Cited by 1,333 · see citing works
Identifying Software Project Risks: An International Delphi Study
Schmidt, R. C., Lyytinen, K., Keil, M., & Cule, P. E. (2001) · Journal of Management Information Systems · doi:10.1080/07421222.2001.11045662 · Cited by 1,252 · see citing works
Trust in a specific technology: An investigation of its components and measures
Mcknight, D. H., Carter, M., Thatcher, J. B., & Clay, P. F. (2011) · ACM Transactions on Management Information Systems · doi:10.1145/1985347.1985353 · Cited by 1,185 · see citing works
Heckman, J. (1990) · American Economic Review · Cited by 1,175 · see citing works
The Delphi technique: a worthwhile research approach for nursing?
McKenna, H. (1994) · Journal of Advanced Nursing · doi:10.1111/j.1365-2648.1994.tb01207.x · Cited by 1,151 · see citing works
Raincloud plots: a multi-platform tool for robust data visualization
Allen, M., Poggiali, D., Whitaker, K. J., Marshall, T. R., & Kievit, R. (2019) · Wellcome Open Research · doi:10.12688/wellcomeopenres.15191.1 · Cited by 1,125 · see citing works
Consensus development methods, and their use in clinical guideline development.
Murphy, M. P., Black, N. A., Lamping, D. L., et al. (1998) · PubMed · Cited by 965 · see citing works
The Delphi technique: a methodological discussion
Williams, P. L., & Webb, C. (1994) · Journal of Advanced Nursing · doi:10.1111/j.1365-2648.1994.tb01066.x · Cited by 933 · see citing works
Sinha, I. P., Smyth, R. L., & Williamson, P. R. (2011) · PLoS Medicine · doi:10.1371/journal.pmed.1000393 · Cited by 891 · see citing works
Delphi: a technique to harness expert opinion for critical decision‐making tasks in education
Clayton, M. J. (1997) · Educational Psychology · doi:10.1080/0144341970170401 · Cited by 850 · see citing works
Franke, T., Attig, C., & Wessel, D. (2019) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2018.1456150 · Cited by 828 · see citing works
Enhancing rigour in the Delphi technique research
Hasson, F., & Keeney, S. (2011) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2011.04.005 · Cited by 771 · see citing works
Core issues in sustainable supply chain management – a Delphi study
Seuring, S. A., & Müller, M. (2007) · Business Strategy and the Environment · doi:10.1002/bse.607 · Cited by 628 · see citing works
Decision Making under Deep Uncertainty
Marchau, V., Walker, W. E., Bloemen, P. J. T. M., & Popper, S. W. (2019) · Springer eBooks · doi:10.1007/978-3-030-05252-2 · Cited by 609 · see citing works
The impact of ERP on supply chain management: Exploratory findings from a European Delphi study
Akkermans, H. A., Bogerd, P., Yücesan, E., & Wassenhove, L. N. V. (2002) · European Journal of Operational Research · doi:10.1016/s0377-2217(02)00550-7 · Cited by 596 · see citing works
Mullen, P. M. (2003) · Journal of Health Organization and Management · doi:10.1108/14777260310469319 · Cited by 574 · see citing works
Delphi: A brief look backward and forward
Linstone, H. A., & Turoff, M. (2010) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2010.09.011 · Cited by 508 · see citing works
Raincloud plots: a multi-platform tool for robust data visualization
Allen, M., Poggiali, D., Whitaker, K. J., Marshall, T. R., van Langen, J., & Kievit, R. (2021) · Wellcome Open Research · doi:10.12688/wellcomeopenres.15191.2 · Cited by 502 · see citing works
Robertson, M., Line, M. A., Jones, S., et al. (2000) · Higher Education Research & Development · doi:10.1080/07294360050020499 · Cited by 499 · see citing works
Belton, I., MacDonald, A., Wright, G., & Hamlin, I. (2019) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2019.07.002 · Cited by 474 · see citing works
The Delphi technique: Past, present, and future prospects — Introduction to the special issue
Rowe, G., & Wright, G. (2011) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2011.09.002 · Cited by 421 · see citing works
RT Delphi: An efficient, “round-less” almost real time Delphi method
Gordon, T. J., & Pease, A. (2005) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2005.09.005 · Cited by 420 · see citing works
Consensus in the delphi method: What makes a decision change?
Barrios, M., Guilera, G., Nuño, L., & Gómez‐Benito, J. (2020) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2020.120484 · Cited by 403 · see citing works
Stability and agreement criteria for the termination of Delphi studies
Dajani, J. S., Sincoff, M. Z., & Talley, W. K. (1979) · Technological Forecasting and Social Change · doi:10.1016/0040-1625(79)90007-6 · Cited by 365 · see citing works
Gnatzy, T., Warth, J., von der Gracht, H. A., & Darkow, I. (2011) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2011.04.006 · Cited by 303 · see citing works
Roßmann, B., Canzaniello, A., von der Gracht, H. A., & Hartmann, E. (2017) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2017.10.005 · Cited by 282 · see citing works
Maintenance in digitalised manufacturing: Delphi-based scenarios for 2030
Bokrantz, J., Skoogh, A., Berlin, C., & Stahre, J. (2017) · International Journal of Production Economics · doi:10.1016/j.ijpe.2017.06.010 · Cited by 272 · see citing works
Review of Delphi-based scenario studies: Quality and design considerations
Nowack, M., Endrikat, J., & Guenther, E. (2011) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2011.03.006 · Cited by 271 · see citing works
de Loë, R. C., Melnychuk, N., Murray, D., & Plummer, R. (2016) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2015.12.009 · Cited by 255 · see citing works
The future of manufacturing: A Delphi-based scenario analysis on Industry 4.0
Culot, G., Orzes, G., Sartor, M., & Nassimbeni, G. (2020) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2020.120092 · Cited by 250 · see citing works
Teo, T. P., & Zhou, M. (2014) · Journal of Computing in Higher Education · doi:10.1007/s12528-014-9080-3 · Cited by 241 · see citing works
Comparing face-to-face meetings, nominal groups, Delphi and prediction markets on an estimation task
Graefe, A., & Armstrong, J. S. (2010) · International Journal of Forecasting · doi:10.1016/j.ijforecast.2010.05.004 · Cited by 240 · see citing works
The future of omnichannel retail: A four-stage Delphi study
von Briel, F. (2018) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2018.02.004 · Cited by 240 · see citing works
Investigating non-response bias in mail surveys.
Sheikh, K., & Mattingly, S. (1981) · Journal of Epidemiology & Community Health · doi:10.1136/jech.35.4.293 · Cited by 231 · see citing works
Biases in future-oriented Delphi studies: A cognitive perspective
Winkler, J., & Moser, R. (2016) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2016.01.021 · Cited by 219 · see citing works
Chang, P.-T., Huang, L., & Lin, H.-J. (2000) · Fuzzy Sets and Systems · doi:10.1016/s0165-0114(98)00067-0 · Cited by 207 · see citing works
Improving the Delphi process: Lessons from social psychological research
Bolger, F., & Wright, G. (2011) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2011.07.007 · Cited by 201 · see citing works
Future applications of blockchain in business and management: A Delphi study
White, G. R. T. (2017) · Strategic Change · doi:10.1002/jsc.2144 · Cited by 195 · see citing works
The Delphi Technique: A Research Strategy for Career and Technical Education
Stitt-Gohdes, W. L., & Crews, T. B. (2004) · Journal of Career and Technical Education · doi:10.21061/jcte.v20i2.636 · Cited by 181 · see citing works
International Business and Trade in the Next Decade: Report from a Delphi Study
Czinkota, M. R., & Ronkainen, I. A. (1997) · Journal of International Business Studies · doi:10.1057/palgrave.jibs.8490121 · Cited by 175 · see citing works
Scenario construction via Delphi and cross-impact analysis
Bañuls, V. A., & Turoff, M. (2011) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2011.03.014 · Cited by 169 · see citing works
Expert biases in technology foresight. Why they are a problem and how to mitigate them
Bonaccorsi, A. J., Apreda, R., & Fantoni, G. (2019) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2019.119855 · Cited by 162 · see citing works
The Delphi technique in forecasting– A 42-year bibliographic analysis (1975–2017)
Flostrand, A., PITT, L. F., & Bridson, S. (2019) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2019.119773 · Cited by 159 · see citing works
Real-Time Delphi in practice — A comparative analysis of existing software-based tools
Aengenheyster, S., Cuhls, K. E., Gerhold, L., et al. (2017) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2017.01.023 · Cited by 157 · see citing works
Desirability bias in foresight: Consequences for decision quality based on Delphi results
Ecken, P., Gnatzy, T., & von der Gracht, H. A. (2011) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2011.05.006 · Cited by 140 · see citing works
Lee, A. C., Cousens, S. N., Darmstadt, G. L., Blencowe, H. K., Pattinson, R., Moran, N. F., Hofmeyr, G. J., Haws, R. A., Bhutta, S. Z., & Lawn, J. E. (2011) · BMC Public Health · doi:10.1186/1471-2458-11-s3-s10 · Cited by 138 · see citing works
Lessons Learned from a Two-Round Delphi-based Scenario Study
Schmalz, U., Spinler, S., & Ringbeck, J. (2020) · MethodsX · doi:10.1016/j.mex.2020.101179 · Cited by 138 · see citing works
The Delphi Technique and Its Uses In Social Science Research*
Strauss, H. J., & Zeigler, L. H. (1975) · The Journal of Creative Behavior · doi:10.1002/j.2162-6057.1975.tb00574.x · Cited by 135 · see citing works
Modified Delphi Investigation of Exercise Science in Physical Education Teacher Education
Bulger, S. M., & Housner, L. D. (2007) · Journal of Teaching in Physical Education · doi:10.1123/jtpe.26.1.57 · Cited by 133 · see citing works
Who is a Delphi ‘expert’? Reflections on a bioeconomy expert selection procedure from Ireland
Devaney, L., & Henchion, M. M. (2018) · Futures · doi:10.1016/j.futures.2018.03.017 · Cited by 130 · see citing works
Who is an expert for foresight? A review of identification methods
Mauksch, S., von der Gracht, H. A., & Gordon, T. J. (2020) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2020.119982 · Cited by 123 · see citing works
Warth, J., von der Gracht, H. A., & Darkow, I. (2012) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2012.04.005 · Cited by 120 · see citing works
Förster, B., & von der Gracht, H. A. (2013) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2013.07.012 · Cited by 104 · see citing works
Spickermann, A., Zimmermann, M., & von der Gracht, H. A. (2013) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2013.04.009 · Cited by 87 · see citing works
The impact of COVID-19 on the European football ecosystem – A Delphi-based scenario analysis
Beiderbeck, D., Frevel, N., von der Gracht, H. A., Schmidt, S. L., & Schweitzer, V. M. (2021) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2021.120577 · Cited by 87 · see citing works
Self-citationSimon, S. T., Weingärtner, V., Higginson, I. J., Voltz, R., & Bausewein, C. (2013) · Journal of Pain and Symptom Management · doi:10.1016/j.jpainsymman.2013.06.013 · Cited by 77 · see citing works
Markmann, C., Spickermann, A., von der Gracht, H. A., & Brem, A. (2020) · Futures & Foresight Science · doi:10.1002/ffo2.56 · Cited by 77 · see citing works
Psychological biases and heuristics in the context of foresight and scenario processes
Schirrmeister, E., Göhring, A., & Warnke, P. (2020) · Futures & Foresight Science · doi:10.1002/ffo2.31 · Cited by 73 · see citing works
Mixed Methods: Combining Expert Interviews, Cross-Impact Analysis and Scenario DevelopmentPreprint
Muskat, M., Blackman, D. A., & Muskat, B. (2013) · SSRN Electronic Journal · doi:10.2139/ssrn.2202179 · Cited by 73 · see citing works
Winkler, J., Kuklinski, C. P. J.-W., & Moser, R. (2014) · Journal of Business Research · doi:10.1016/j.jbusres.2014.11.001 · Cited by 64 · see citing works
Energy-constrained and low-carbon scenarios for the transportation and logistics industry
von der Gracht, H. A., & Darkow, I. (2016) · The International Journal of Logistics Management · doi:10.1108/ijlm-12-2013-0150 · Cited by 51 · see citing works
Door-to-door travel in 2035 – A Delphi study
Schmalz, U., Ringbeck, J., & Spinler, S. (2020) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2020.120096 · Cited by 50 · see citing works
Delphi, the issue of reliability
Kastein, M. R., Jacobs, M., van der Hell, R. H., et al. (1993) · Technological Forecasting and Social Change · doi:10.1016/0040-1625(93)90075-i · Cited by 49 · see citing works
Hirschinger, M., Spickermann, A., Hartmann, E., von der Gracht, H. A., & Darkow, I. (2015) · Journal of Supply Chain Management · doi:10.1111/jscm.12074 · Cited by 47 · see citing works
Extracting scenario archetypes: A quantitative text analysis of documents about the future
Fergnani, A., & Jackson, M. (2019) · Futures & Foresight Science · doi:10.1002/ffo2.17 · Cited by 40 · see citing works
To what extent will blockchain drive the machine economy? Perspectives from a prospective study
Schweizer, A., Knoll, P., Urbach, N., von der Gracht, H. A., & Hardjono, T. (2020) · IEEE Transactions on Engineering Management · doi:10.1109/tem.2020.2979286 · Cited by 39 · see citing works
Self-citationKendall, J. W. (1977) · Technological Forecasting and Social Change · doi:10.1016/0040-1625(77)90017-8 · Cited by 35 · see citing works
The Future of Logistics: Scenarios for 2025
von der Gracht, H. A. (2008) · Cited by 35 · see citing works
Loye, D. (1980) · Technological Forecasting and Social Change · doi:10.1016/0040-1625(80)90001-3 · Cited by 32 · see citing works
The future of professional football
Merkel, S., Schmidt, S. L., & Schreyer, D. (2016) · Sport Business and Management An International Journal · doi:10.1108/sbm-10-2014-0043 · Cited by 31 · see citing works
Self-citationPanula‐Ontto, J., Luukkanen, J., Kaivo-oja, J. R. L., et al. (2018) · Energy Policy · doi:10.1016/j.enpol.2018.04.009 · Cited by 29 · see citing works
Consensus Development Methods, and their Use in Creating Clinical Guidelines
Black, N., Murphy, M., Lamping, D. L., et al. (2001) · SAGE Publications Ltd eBooks · doi:10.4135/9781848608344.n24 · Cited by 22 · see citing works
Measuring Wellbeing in the SOEPPreprint
Schimmack, U. (2008) · SSRN Electronic Journal · doi:10.2139/ssrn.1306888 · Cited by 16 · see citing works
Münch, C., & von der Gracht, H. A. (2021) · Futures & Foresight Science · doi:10.1002/ffo2.88 · Cited by 12 · see citing works
Kawamoto, C. T., Wright, J. T. C., Spers, R. G., & de Carvalho, D. E. (2019) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2018.12.020 · Cited by 9 · see citing works
Dellinger, R. P., Levy, M. M., Carlet, J. M., Bion, J., Parker, M. M., Jaeschke, R., Reinhart, K., Angus, D. C., Brun-Buisson, C., Beale, R., et al. (2008) · Critical Care Medicine · doi:10.1097/01.ccm.0000298158.12101.41 · Cited by 9 · see citing works
Flagged first, then by citations
Research works printed but not linked: 4
- [3]Aggarwal C.C. Data mining: the textbook - mining discrete sequences Springer Int. Publ. 746 2015 10.1007/978-3-319-14142-8 · DOI
- [34]Gheorghiu R. Dragomir B. Andreescu L. Cuhls K. Rose A. Curaj A. Weber M. New Horizons: Data from a Delphi Survey in Support of European Union Future Policies in Research and Innovation 2017 10.2777/654172 · DOI
- [50]Kovaleva A. Beierlein C. Kemper C. Rammstedt B. Eine Kurzskala zur Messung von Kontrollüberzeugung Gesis – Leibniz-Institut für Sozialwissenschaften (Vol. 19) 2012 http://www.gesis.org/fileadmin/upload/forschung/publikationen/gesis_reihen/gesis_arbeitsberichte/WorkingPapers_2012-19.pdf
- [77]Scheibe M. Skutsch M. Schofer J. Experients in Delphi methodology Linstone H.A. Turoff M. The Delphi Method: Techniques and Applications (pp. 262–287) 1975 Addison-Wesley Reading http://link.springer.com/10.1007/s00256-011-1145-z · DOI
Cited by
Sorted by citations
A multisociety Delphi consensus statement on new fatty liver disease nomenclature
Rinella, M. E., Lazarus, J. V., Ratziu, V., Francque, S. M., Sanyal, A. J., Kanwal, F., Romero, D., Abdelmalek, M. F., Anstee, Q. M., Arab, J. P., et al. (2023) · Hepatology · doi:10.1097/hep.0000000000000520 · Cited by 3,476 · see citing works
A multisociety Delphi consensus statement on new fatty liver disease nomenclature
Rinella, M. E., Lazarus, J. V., Ratziu, V., Francque, S. M., Sanyal, A. J., Kanwal, F., Romero, D., Abdelmalek, M. F., Anstee, Q. M., Arab, J. P., et al. (2023) · Journal of Hepatology · doi:10.1016/j.jhep.2023.06.003 · Cited by 3,100 · see citing works
A multisociety Delphi consensus statement on new fatty liver disease nomenclature
Rinella, M. E., Lazarus, J. V., Ratziu, V., Francque, S. M., Sanyal, A. J., Kanwal, F., Romero, D., Abdelmalek, M. F., Anstee, Q. M., Arab, J. P., et al. (2024) · Annals of Hepatology · doi:10.1016/j.aohep.2023.101133 · Cited by 844 · see citing works
Reporting guidelines for Delphi techniques in health sciences: A methodological review
Spranger, J., Homberg, A., Sonnberger, M., & Niederberger, M. (2022) · Zeitschrift für Evidenz, Fortbildung und Qualität im Gesundheitswesen · doi:10.1016/j.zefq.2022.04.025 · Cited by 262 · see citing works
Artificial Intelligence in Education: Implications for Policymakers, Researchers, and Practitioners
Ifenthaler, D., Majumdar, R., Gorissen, P., Judge, M., Mishra, S., Raffaghelli, J., & Shimada, A. (2024) · Technology, Knowledge and Learning · doi:10.1007/s10758-024-09747-0 · Cited by 238 · see citing works
Raichurkar, P., Denehy, L., Solomon, M., Koh, C., Pillinger, N., Hogan, S., McBride, K., Carey, S., Bartyn, J., Hirst, N., et al. (2023) · Annals of Surgical Oncology · doi:10.1245/s10434-023-14192-x · Cited by 53 · see citing works
Oo, C. T. L., Wider, W., Pang, N. T. P., Koh, E. B. Y., Vasanthi, R. K., Thet, K. Z. Z., Ramalho, R., Özdemir, B. N., & Mahboob, K. (2026) · International Journal of Qualitative Studies on Health and Well-being · doi:10.1080/17482631.2026.2621802 · Cited by 5 · see citing works
“Fifteen mental health professionals were selected through purposive sampling to ensure a well-informed and contextually relevant expert panel (Beiderbeck et al., 2021).”
Punpeng, G. (2025) · Cogent Arts & Humanities · doi:10.1080/23311983.2025.2570062
“…and widely accepted measure of consensus due to its resistance to outliers and its effectiveness in capturing the consistency of expert responses (Beiderbeck et al., 2021; Von der gracht, 2012); and (3) at least 70% of panellists assigning a rating of 4 or 5. this combination of statistical…”
- Authors
- Beiderbeck, D., Frevel, N., von der Gracht, H. A., Schmidt, S. L., & Schweitzer, V. M.
- Affiliations
- WHU – Otto Beisheim School of Management (DE)
- Corresponding author
- Beiderbeck, Daniel
- Venue
- MethodsX
- Publisher
- Elsevier BV
- Published
- 2021
- Volume
- 8
- Pages
- 101401
- ISSN
- 2215-0161 (ISSN-L)
- PMC
- PMC8374446
- OpenAlex
- W3170399214
- Cited by
- 642
- References
- 97
- Citations by year
- 2026: 179 · 2025: 183 · 2024: 153 · 2023: 85 · 2022: 39 · 2021: 3
- FWCI
- 96.96
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- MethodsX (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Social Sciences › Sociology and Political Science › Delphi Technique in Research
- Social Sciences › Decision Sciences › Management Science and Operations Research › Multi-Criteria Decision Making
- Social Sciences › Sociology and Political Science › Focus Groups and Qualitative Methods
- Keywords
- Delphi method, expert opinion, consensus building, scenario analysis, fuzzy clustering, sentiment analysis, consensus measurement, COVID-19 pandemic, methodological recommendations, interdisciplinary applications
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
