CoChat
Sign Up
Cited by 22Delphi Technique in ResearchMulti-Criteria Decision MakingReliability and Agreement in Measurement

Using the Delphi Method to Establish Expert Consensus: A Practical Guide

Open access
Jorm, Anthony·2025·Advancing Methods for Interdisciplinarity in the Social Sciences
DOI 10.1007/978-981-96-8357-4
Open this paper in CoChat — free accountOpen PDF

Abstract

This Open Access book provides guidance on how to do a Delphi study to establish a consensus across the broad range of disciplines.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1007/978-981-96-8357-4

Cites

184 of 184 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 145

  1. fied Delphi-analytic hierarchy process study. BMC Cancer, 24(1), Article 120. https://doi.org/10.1186/s12885-024-11877-8 · DOI
  2. Sackman, H. (1975). Summary evaluation of Delphi. Policy Analysis, 1(4), 693-718. http://www.jstor.org/stable/42784280
  3. Simoiu, C., Sumanth, C., Mysore, A., & Goel, S. (2019). Studying the "wisdom of crowds" at scale. In Seventh AAAI conference on human computation and crowdsourcing (HCOMP-19).
  4. R1 · DOI
  5. Burgman, M. A. (2016). Trusting judgements: How to get the best out of experts. Cambridge University Press.
  6. Cao, J. Q., Surgeoner, B., Manna, M., Boileau, J. F., Gelmon, K. A., Brackstone, M., Brezden-Masley, C., Jerzak, K. J., Prakash, I., Sehdev, S., Wong, S. M., Bouganim, N., Cescon, D. W., Chia, S., Dayes, I. S., Joy, A. A., & Henning, J. W. (2024). Guidance for Canadian breast cancer practice: National consensus recommendations for clinical staging of patients newly diagnosed with breast A. JORM cancer. Current Oncology, 31(11), 7226-7243. https://doi.org/10.3390/ · DOI
  7. curroncol31110533 · DOI
  8. Hasson, F., Keeney, S., & McKenna, H. (2000). Research guidelines for the Delphi survey technique. Journal of Advanced Nursing, 32(4), 1008-1015.
  9. Jorm, A. (2025). Expert consensus in science. Palgrave Macmillan. https://doi. org/10.1007/978-981-97-9222-1 · DOI
  10. Saunders, H., Anderson, C., Feldman, F., Holroyd-Leduc, J., Jain, R., Liu, B., Macaulay, S., Marr, S., Silvius, J., Weldon, J., Bayoumi, A. M., Straus, S. E., Tricco, A. C., & Isaranuwatchai, W. (2023). Developing a fall prevention inter- vention economic model. PLoS One, 18(1 January), Article e0280572. https:// doi.org/10.1371/journal.pone.0280572 · DOI
  11. Bailey, E., Bellairs-Walsh, I., Reavley, N., Gooding, P., Hetrick, S., Rice, S., Boland, A., & Robinson, J. (2024). Best practice for integrating digital inter- ventions into clinical care for young people at risk of suicide: A Delphi study. BMC Psychiatry, 24(1), 71. https://doi.org/10.1186/s12888-023-05448-7 · DOI
  12. Baron, R. S. (2005). So right it's wrong: Groupthink and the ubiquitous nature of polarized group decision making. Advances in Experimental Social Psychology, 37, 219-253. https://doi.org/10.1016/S0065-2601(05)37004-3 · DOI · Paper page
  13. Barrington, H., Young, B., & Williamson, P. R. (2021). Patient participation in Delphi surveys to develop core outcome sets: Systematic review. BMJ Open, 11(9), e051066. https://doi.org/10.1136/bmjopen-2021-051066 · DOI
  14. Barrios, M., Guilera, G., Nuño, L., & Gómez-Benito, J. (2021). Consensus in the delphi method: What makes a decision change? Technological Forecasting and Social Change, 163, 120484. https://doi.org/10.1016/j. techfore.2020.120484 · DOI
  15. Becker, J., Brackbill, D., & Centola, D. (2017). Network dynamics of social influ- ence in the wisdom of crowds. Proceedings of the National Academy of Sciences USA, 114(26), E5070-E5076. https://doi.org/10.1073/pnas.1615978114 · DOI · Paper page
  16. Belton, I., Wright, G., Sissons, A., Bolger, F., Crawford, M. M., Hamlin, I., Taylor Browne Lūka, C., & Vasilichi, A. (2021). Delphi with feedback of rationales: How large can a Delphi group be such that participants are not overloaded, de-motivated, or disengaged? Technological Forecasting and Social Change, 170, 120897. https://doi.org/10.1016/j.techfore.2021.120897 · DOI
  17. Berk, L., Jorm, A. F., Kelly, C. M., Dodd, S., & Berk, M. (2011). Development of guidelines for caregivers of people with bipolar disorder: A Delphi expert con- sensus study. Bipolar Disorders, 13(5-6), 556-570. https://doi. org/10.1111/j.1399-5618.2011.00942.x · DOI
  18. Biddle, L., Rifkin-Zybutz, R., Derges, J., Turner, N., Bould, H., Sedgewick, F., Gooberman-Hill, R., Moran, P., & Linton, M. J. (2022). Developing good practice indicators to assist mental health practitioners to converse with young people about their online activities and impact on mental health: A two-panel mixed-methods Delphi study. BMC Psychiatry, 22(1), 485. https://doi. org/10.1186/s12888-022-04093-w REFERENCES · DOI
  19. Boccardi, M., Bocchetta, M., Apostolova, L. G., Barnes, J., Bartzokis, G., Corbetta, G., DeCarli, C., deToledo-Morrell, L., Firbank, M., Ganzola, R., Gerritsen, L., Henneman, W., Killiany, R. J., Malykhin, N., Pasqualetti, P., Pruessner, J. C., Redolfi, A., Robitaille, N., Soininen, H., et al. (2015). Delphi definition of the EADC-ADNI harmonized protocol for hippocampal segmen- tation on magnetic resonance. Alzheimer's & Dementia, 11(2), 126-138. https://doi.org/10.1016/j.jalz.2014.02.009 · DOI
  20. Boel, A., Navarro-Compán, V., Landewé, R., & van der Heijde, D. (2021). Two different invitation approaches for consecutive rounds of a Delphi survey led to comparable final outcome. Journal of Clinical Epidemiology, 129, 31-39. https://doi.org/10.1016/j.jclinepi.2020.09.034 · DOI · Paper page
  21. Bolger, F., Stranieri, A., Wright, G., & Yearwood, J. (2011). Does the Delphi process lead to increased accuracy in group-based judgmental forecasts or does it simply induce consensus amongst judgmental forecasters? Technological Forecasting and Social Change, 78(9), 1671-1680. https://doi.org/10.1016/j. techfore.2011.06.002 · DOI
  22. Boulkedid, R., Abdoul, H., Loustau, M., Sibony, O., & Alberti, C. (2011). Using and reporting the Delphi method for selecting healthcare quality indicators: A systematic review. PLoS One, 6(6), e20476. https://doi.org/10.1371/journal. pone.0020476 · DOI · Paper page
  23. Brookes, S. T., Macefield, R. C., Williamson, P. R., McNair, A. G., Potter, S., Blencowe, N. S., Strong, S., & Blazeby, J. M. (2016). Three nested random- ized controlled trials of peer-only or multiple stakeholder group feedback within Delphi surveys during core outcome and information set development. Trials, 17(1), 409. https://doi.org/10.1186/s13063-016-1479-x · DOI
  24. Burgman, M. A. (2016). Trusting judgements: How to get the best out of experts. Cambridge University Press.
  25. Campbell, S. M., Hann, M., Roland, M. O., Quayle, J. A., & Shekelle, P. G. (1999). The effect of panel membership and feedback on ratings in a two-round Delphi survey: Results of a randomized controlled trial. Medical Care, 37(9), 964-968. https://doi.org/10.1097/00005650-199909000-00012 · DOI
  26. Chalmers, K. J., Jorm, A. F., Kelly, C. M., Reavley, N. J., Bond, K. S., Cottrill, F. A., & Wright, J. (2020). Offering mental health first aid to a person after a potentially traumatic event: A Delphi study to redevelop the 2008 guidelines. BMC Psychology, 8(1), 105. https://doi.org/10.1186/s40359-020-00473-7 · DOI
  27. Chan, J., Searle, E. F., Khodyakov, D., Denson, L., Echeverri, A., Browne, E. M., Chiarelli, Y., Dickey, L. L., Erickson, D. S., Flannery, J., Kaplan, L. J., Markovitz, S., Popescu, S. V., & Shenoy, E. S. (2024). They will come, you must build it: A modified Delphi process applied to preparing acute care facili- ties infrastructure for high-consequence infectious diseases. Health Security, 22(5), 384-393. https://doi.org/10.1089/hs.2024.0013 · DOI
  28. Charles, K. R., Hall, L., Ullman, A. J., & Schults, J. A. (2022). Methodology minute: Utilizing the RAND/UCLA appropriateness method to develop guidelines for infection prevention. American Journal of Infection Control, 50(3), 345-348. https://doi.org/10.1016/j.ajic.2021.12.012 · DOI
  29. Cormack, C. J., Childs, J., & Kent, F. (2024). Competencies required by sonog- raphers teaching ultrasound interprofessionally: A Delphi consensus study. BMC Medical Education, 24(1), Article 970. https://doi.org/10.1186/ s12909-024-05933-x REFERENCES · DOI
  30. Daniel, C., & Hernandez, T. (2024). What retail apocalypse? A Delphi forecast of commercial space demand in the Toronto region. Journal of Retailing and Consumer Services, 77, 103670. https://doi.org/10.1016/j. jretconser.2023.103670 · DOI
  31. Davis-Stober, C. P., Budescu, D. V., Dana, J., & Broomell, S. B. (2014). When is a crowd wise? Decision, 1, 79-101. https://doi.org/10.1037/dec0000004 · DOI
  32. Dayé, C. (2018). How to train your oracle: The Delphi method and its turbulent youth in operations research and the policy sciences. Social Studies of Science, 48(6), 846-868. https://doi.org/10.1177/0306312718798497 · DOI
  33. de Oliveira, S., & Nisbett, R. E. (2018). Demographically diverse crowds are typi- cally not much wiser than homogeneous crowds. Proceedings of the National Academy of Sciences USA, 115(9), 2066-2071. https://doi.org/10.1073/ pnas.1717632115 · DOI
  34. Del Grande, C., & Kaczorowski, J. (2023). Rating versus ranking in a Delphi survey: A randomized controlled trial. Trials, 24(1), 543. https://doi. org/10.1186/s13063-023-07442-6 · DOI
  35. Dezecache, G., Dockendorff, M., Ferreiro, D. N., Deroy, O., & Bahrami, B. (2022). Democratic forecast: Small groups predict the future better than individuals and crowds. Journal of Experimental Psychology: Applied, 28(3), 525-537. https://doi.org/10.1037/xap0000424 · DOI
  36. Diamond, I. R., Grant, R. C., Feldman, B. M., Pencharz, P. B., Ling, S. C., Moore, A. M., & Wales, P. W. (2014). Defining consensus: A systematic review recommends methodologic criteria for reporting of Delphi studies. Journal of 116 REFERENCES Clinical Epidemiology, 67(4), 401-409. https://doi.org/10.1016/j. jclinepi.2013.12.002 · DOI · Paper page
  37. Drennan, J. (2003). Cognitive interviewing: Verbal data in the design and pretest- ing of questionnaires. Journal of Advanced Nursing, 42(1), 57-63. https:// doi.org/10.1046/j.1365-2648.2003.02579.x · DOI · Paper page
  38. Dubbeldeman, E. M., van der Kleij, R. M. J. J., Brakema, E. A., & Crone, M. R. (2024). Expert consensus on multilevel implementation hypotheses to promote the uptake of youth care guidelines: A Delphi study. Health Research Policy and Systems, 22(1), 89. https://doi.org/10.1186/s12961-024-01167-x · DOI
  39. Duijzer, R., Bernts, L. H. P., Geerts, A., van Hoek, B., Coenraad, M. J., Rovers, C., Alvaro, D., Kuijper, E. J., Nevens, F., Halbritter, J., Colmenero, J., Kupcinskas, J., Salih, M., Hogan, M. C., Ronot, M., Vilgrain, V., Hanemaaijer, N. M., Kamath, P. S., Strnad, P., et al. (2024). Clinical management of liver cyst infections: An international, modified Delphi-based clinical decision frame- work. The Lancet Gastroenterology & Hepatology, 9(9), 884-894. https://doi. org/10.1016/S2468-1253(24)00094-3 · DOI
  40. Eccles, M. P., & Mittman, B. S. (2006). Welcome to Implementation Science. Implementation Science, 1(1), 1. https://doi.org/10.1186/1748-5908-1-1 · DOI · Paper page
  41. Fan, T., Zhu, S., Wang, H., Dong, Y., Zhou, Y., Song, Y., Pan, S., Wu, Q., Smith, G. D., Li, Y., & Han, Y. (2024). Development and validation of the self-report symptom inventory of immune-related adverse events in patients with lung cancer. Asia-Pacific Journal of Oncology Nursing, 11(12), 100603. https://doi. org/10.1016/j.apjon.2024.100603 · DOI
  42. Ferguson, J. H. (1996). The NIH Consensus Development Program. The evolu- tion of guidelines. International Journal of Technology Assessment in Health Care, 12(3), 460-474. https://www.ncbi.nlm.nih.gov/pubmed/8840666 REFERENCES
  43. Ferri, C. P., Prince, M., Brayne, C., Brodaty, H., Fratiglioni, L., Ganguli, M., Hall, K., Hasegawa, K., Hendrie, H., Huang, Y., Jorm, A., Mathers, C., Menezes, P. R., Rimmer, E., & Scazufca, M. (2005). Global prevalence of dementia: A Delphi consensus study. Lancet, 366(9503), 2112-2117. https://doi. org/10.1016/s0140-6736(05)67889-0 · DOI
  44. Fischer, J. A., Kelly, C. M., Kitchener, B. A., & Jorm, A. F. (2013). Development of guidelines for adults on how to communicate with adolescents about mental health problems and other sensitive topics: A Delphi study. SAGE Open, 3(4), 2158244013516769. https://doi.org/10.1177/2158244013516769 · DOI
  45. Foth, T., Efstathiou, N., Vanderspank-Wright, B., Ufholz, L. A., Dütthorn, N., Zimansky, M., & Humphrey-Murto, S. (2016). The use of Delphi and Nominal Group Technique in nursing education: A review. International Journal of Nursing Studies, 60, 112-120. https://doi.org/10.1016/j. ijnurstu.2016.04.015 · DOI
  46. Frank, H. E., Albanese, A., Sun, S., Saadeh, F., Johnson, B. T., Elwy, A. R., & Loucks, E. B. (2024). Mindfulness-based stress reduction health insurance cov- erage: If, how, and when? An integrated knowledge translation (iKT) Delphi key informant analysis. Mindfulness, 15(5), 1220-1233. https://doi. org/10.1007/s12671-024-02366-x · DOI
  47. Frey, V., & Van de Rijt, A. (2021). Social influence undermines the wisdom of crowds in sequential decision making. Management Science, 67, 4273-4286. https://doi.org/10.1287/mnsc.2020.3713 · DOI
  48. Gallegos, D., Booth, S., Pollard, C. M., Chilton, M., & Kleve, S. (2023). Food security definition, measures and advocacy priorities in high-income countries: A Delphi consensus study. Public Health Nutrition, 26(10), 1986-1996. https://doi.org/10.1017/s1368980023000915 · DOI
  49. Galton, F. (1907). Vox populi. Nature, 75(1949), 450-451. https://doi. org/10.1038/075450a0 · DOI · Paper page
  50. Gargon, E., Crew, R., Burnside, G., & Williamson, P. R. (2019). Higher number of items associated with significantly lower response rates in COS Delphi sur- veys. Journal of Clinical Epidemiology, 108, 110-120. https://doi. org/10.1016/j.jclinepi.2018.12.010 · DOI
  51. Gattrell, W. T., Logullo, P., van Zuuren, E. J., Price, A., Hughes, E. L., Blazey, P., Winchester, C. C., Tovey, D., Goldman, K., Hungin, A. P., & Harrison, N. (2024). ACCORD (ACcurate COnsensus Reporting Document): A report- ing guideline for consensus methods in biomedicine developed via a modified 118 REFERENCES Delphi. PLoS Medicine, 21(1), e1004326. https://doi.org/10.1371/journal. pmed.1004326 · DOI · Paper page
  52. Geyle, H. M., Tingley, R., Amey, A. P., Cogger, H., Couper, P. J., Cowan, M., Craig, M. D., Doughty, P., Driscoll, D. A., Ellis, R. J., Emery, J. P., Fenner, A., Gardner, M. G., Garnett, S. T., Gillespie, G. R., Greenlees, M. J., Hoskin, C. J., Keogh, J. S., Lloyd, R., et al. (2021). Reptiles on the brink: Identifying the Australian terrestrial snake and lizard species most at risk of extinction. Pacific Conservation Biology, 27(1), 3-12. https://doi.org/10.1071/PC20033 · DOI
  53. Gibbins, A. K., Wood, P. J., & Spark, M. J. (2017). Managing inappropriate use of non-prescription combination analgesics containing codeine: A modified Delphi study. Research in Social & Administrative Pharmacy, 13(2), 369-377. https://doi.org/10.1016/j.sapharm.2016.02.015 · DOI
  54. Gnatzy, T., Warth, J., von der Gracht, H. A., & Darkow, I. (2011). Validating an innovative real-time Delphi approach -A methodological comparison between real-time and conventional Delphi studies. Technological Forecasting and Social Change, 78(9), 1681-1694. https://doi.org/10.1016/j. techfore.2011.04.006 · DOI
  55. Gordon, T., & Pease, A. (2006). RT Delphi: An efficient, "round-less" almost real time Delphi method. Technological Forecasting and Social Change, 73(4), 321-333. https://doi.org/10.1016/j.techfore.2005.09.005 · DOI
  56. Graefe, A., & Armstrong, J. S. (2011). Comparing face-to-face meetings, nominal groups, Delphi and prediction markets on an estimation task. International Journal of Forecasting, 27(1), 183-195. https://doi.org/10.1016/j. ijforecast.2010.05.004 · DOI
  57. Granovskiy, B., Gold, J. M., Sumpter, D. J., & Goldstone, R. L. (2015). Integration of social information by human groups. Topics in Cognitive Science, 7(3), 469-493. https://doi.org/10.1111/tops.12150 · DOI
  58. Grant, S., Booth, M., & Khodyakov, D. (2018). Lack of preregistered analysis plans allows unacceptable data mining for and selective reporting of consensus in Delphi studies. Journal of Clinical Epidemiology, 99, 96-105. https://doi. org/10.1016/j.jclinepi.2018.03.007 · DOI
  59. Gürçay, B., Mellers, B. A., & Baron, J. (2015). The power of social influence on estimation accuracy. Journal of Behavioral Decision Making, 28, 250-261. https://doi.org/10.1002/bdm.1843 · DOI
  60. Hart, L. M., Jorm, A. F., Kanowski, L. G., Kelly, C. M., & Langlands, R. L. (2009). Mental health first aid for Indigenous Australians: Using Delphi consensus studies to develop guidelines for culturally appropriate responses to mental health problems. BMC Psychiatry, 9, 47. https://doi.org/10.118 6/1471-244x-9-47 · DOI
  61. Hart, L. M., Mason, R. J., Kelly, C. M., Cvetkovski, S., & Jorm, A. F. (2016). 'teen Mental Health First Aid': A description of the program and an initial evaluation. International Journal of Mental Health Systems, 10, 3. https://doi. org/10.1186/s13033-016-0034-1 · DOI · Paper page
  62. Hart, L. M., Morgan, A. J., Rossetto, A., Kelly, C. M., Gregg, K., Gross, M., Johnson, C., & Jorm, A. F. (2022). teen Mental Health First Aid: 12-month outcomes from a cluster crossover randomized controlled trial evaluation of a universal program to help adolescents better support peers with a mental health problem. BMC Public Health, 22(1), 1159. https://doi.org/10.1186/ s12889-022-13554-6 · DOI
  63. Hasson, F., Keeney, S., & McKenna, H. (2000). Research guidelines for the Delphi survey technique. Journal of Advanced Nursing, 32(4), 1008-1015.
  64. Healy, D. (2006). Manufacturing consensus. Culture, Medicine and Psychiatry, 30(2), 135-156. https://doi.org/10.1007/s11013-006-9013-3 · DOI
  65. Hemming, V., Burgman, M. A., Hanea, A. M., McBride, M. F., & Wintle, B. C. (2018a). A practical guide to structured expert elicitation using the IDEA protocol. Methods in Ecology and Evolution, 9(1), 169-180. https://doi.org/1 0.1111/2041-210X.12857 · DOI
  66. Hemming, V., Walshe, T. V., Hanea, A. M., Fidler, F., & Burgman, M. A. (2018b). Eliciting improved quantitative judgements using the IDEA protocol: A case study in natural resource management. PLoS One, 13(6), e0198468. https:// doi.org/10.1371/journal.pone.0198468 · DOI
  67. Humphrey-Murto, S., & de Wit, M. (2019). The Delphi method-more research please. Journal of Clinical Epidemiology, 106, 136-139. https://doi. org/10.1016/j.jclinepi.2018.10.011 · DOI
  68. Humphrey-Murto, S., Varpio, L., Wood, T. J., Gonsalves, C., Ufholz, L. A., Mascioli, K., Wang, C., & Foth, T. (2017). The use of the Delphi and other consensus goup methods in medical education research: A review. Academic Medicine, 92(10), 1491-1498. https://doi.org/10.1097/ acm.0000000000001812 · DOI
  69. Jorm, A. (2025). Expert consensus in science. Palgrave Macmillan. https://doi. org/10.1007/978-981-97-9222-1 · DOI
  70. Jorm, A. F., Kitchener, B. A., & Reavley, N. J. (2019). Mental Health First Aid train- ing: Lessons learned from the global spread of a community education program. World Psychiatry, 18(2), 142-143. https://doi.org/10.1002/wps.20621 · DOI
  71. Jorm, A. F., & Ross, A. M. (2018). Guidelines for the public on how to provide mental health first aid: Narrative review. BJPsych Open, 4(6), 427-440. https:// doi.org/10.1192/bjo.2018.58 · DOI
  72. Jünger, S., Payne, S. A., Brine, J., Radbruch, L., & Brearley, S. G. (2017). Guidance on Conducting and REporting DElphi Studies (CREDES) in palliative care: Recommendations based on a methodological systematic review. Palliative Medicine, 31(8), 684-706. https://doi.org/10.1177/0269216317690685 · DOI · Paper page
  73. Kearney, A., Williamson, P., Young, B., Bagley, H., Gamble, C., Denegri, S., Muir, D., Simon, N. A., Thomas, S., Elliot, J. T., Bulbeck, H., Crocker, J. C., Planner, C., Vale, C., Clarke, M., Sprosen, T., & Woolfall, K. (2017). Priorities for methodological research on patient and public involvement in clinical trials: A modified Delphi process. Health Expectations, 20(6), 1401-1410. https://doi. org/10.1111/hex.12583 · DOI
  74. Keck, S., & Tang, W. (2020). Enhancing the wisdom of the crowd with cognitive- process diversity: The benefits of aggregating intuitive and analytical judg- ments. Psychological Science, 31(10), 1272-1282. https://doi. org/10.1177/0956797620941840 · DOI
  75. Kepp, K. P., Aavitsland, P., Ballin, M., Balloux, F., Baral, S., Bardosh, K., Bauchner, H., Bendavid, E., Bhopal, R., Blumstein, D. T., Boffetta, P., Bourgeois, F., Brufsky, A., Collignon, P. J., Cripps, S., Cristea, I. A., Curtis, N., Djulbegovic, B., Faude, O., et al. (2024). Panel stacking is a threat to consensus statement validity. Journal of Clinical Epidemiology, 173, Article 111428. https://doi. org/10.1016/j.jclinepi.2024.111428 REFERENCES · DOI
  76. Kitchener, B. A., Jorm, A. F., & Kelly, C. M. (2010). Mental health first aid man- ual (2nd ed.). Orygen Youth Health Research Centre.
  77. Koppold, D. A., Breinlinger, C., Hanslian, E., Kessler, C., Cramer, H., Khokhar, A. R., Peterson, C. M., Tinsley, G., Vernieri, C., Bloomer, R. J., Boschmann, M., Bragazzi, N. L., Brandhorst, S., Gabel, K., Goldhamer, A. C., Grajower, M. M., Harvie, M., Heilbronn, L., Horne, B. D., et al. (2024). International consensus on fasting terminology. Cell Metabolism, 36(8), 1779-1794.e1774. https://doi.org/10.1016/j.cmet.2024.06.013 · DOI
  78. Lakens, D., Mesquida, C., Rasti, S., & Ditroilo, M. (2024). The benefits of pre- registration and Registered Reports. Evidence-Based Toxicology, 2(1), 2376046. https://doi.org/10.1080/2833373X.2024.2376046 · DOI
  79. Lambert, S. D., Ould Brahim, L., Morrison, M., Girgis, A., Yaffe, M., Belzile, E., Clayberg, K., Robinson, J., Thorne, S., Bottorff, J. L., Duggleby, W., Campbell- Enns, H., Kim, Y., & Loiselle, C. G. (2019). Priorities for caregiver research in cancer care: An international Delphi survey of caregivers, clinicians, managers, and researchers. Supportive Care in Cancer, 27(3), 805-817. https://doi. org/10.1007/s00520-018-4314-y Lancet. (2022). Information for authors. www.thelancet.com · DOI
  80. Landeta, J., & Lertxundi, A. (2024). Quality indicators for Delphi studies. FUTURES & FORESIGHT SCIENCE, 6(1), e172. https://doi. org/10.1002/ffo2.172 · DOI
  81. Lazarus, J. V., Romero, D., Kopka, C. J., Karim, S. A., Abu-Raddad, L. J., Almeida, G., Baptista-Leite, R., Barocas, J. A., Barreto, M. L., Bar-Yam, Y., Bassat, Q., Batista, C., Bazilian, M., Chiou, S. T., Del Rio, C., Dore, G. J., Gao, G. F., Gostin, L. O., Hellard, M., et al. (2022). A multinational Delphi consensus to end the COVID-19 public health threat. Nature, 611(7935), 332-345. https://doi.org/10.1038/s41586-022-05398-2 · DOI · Paper page
  82. Lecours, A. (2020). Scientific, professional and experiential validation of the model of preventive behaviours at work: Protocol of a modified Delphi Study. BMJ Open, 10(9), e035606. https://doi.org/10.1136/bmjopen-2019-035606 · DOI
  83. Lee, C., Song, H., & Mjelde, J. W. (2008). The forecasting of International Expo tourism using quantitative and qualitative techniques. Tourism Management, 29(6), 1084-1098. https://doi.org/10.1016/j.tourman.2008.02.007 · DOI
  84. Lei, F. (2024). Online recruitment for an online survey study: Our experience of dealing with fraudsters. Applied Nursing Research, 80, 151854. https://doi. org/10.1016/j.apnr.2024.151854 · DOI
  85. Linstone, H. A., & Turoff, M. (2002). The Delphi method: Techniques and applica- tions. Addison-Wesley Publishing Company, Advanced Book Program. https:// books.google.com.au/books?id=uZ0RkAEACAAJ
  86. Liu, X., Cruz Rivera, S., Moher, D., Calvert, M. J., Denniston, A. K., Spirit, A. I., & Group, C.-A. W. (2020). Reporting guidelines for clinical trial reports for interventions involving artificial intelligence: The CONSORT-AI extension. Lancet Digital Health, 2(10), e537-e548. https://doi.org/10.1016/ S2589-7500(20)30218-1 · DOI · Paper page
  87. Locke, C. C., & Anderson, C. (2015). The downside of looking like a leader: Power, nonverbal confidence, and participative decision-making. Journal of Experimental Social Psychology, 58, 42-47. https://doi.org/10.1016/j. jesp.2014.12.004 · DOI
  88. MacLennan, S., Kirkham, J., Lam, T. B. L., & Williamson, P. R. (2018). A ran- domized trial comparing three Delphi feedback strategies found no evidence of a difference in a setting with high initial agreement. Journal of Clinical Epidemiology, 93, 1-8. https://doi.org/10.1016/j.jclinepi.2017.09.024 · DOI
  89. Mannes, A. E., Soll, J. B., & Larrick, R. P. (2014). The wisdom of select crowds. Journal of Personality and Social Psychology, 107, 276-299. https://doi. org/10.1037/a0036677 · DOI
  90. Manyara, A. M., Purvis, A., Ciani, O., Collins, G. S., & Taylor, R. S. (2024). Sample size in multistakeholder Delphi surveys: At what minimum sample size REFERENCES do replicability of results stabilize? Journal of Clinical Epidemiology, 174, Article 111485. https://doi.org/10.1016/j.jclinepi.2024.111485 · DOI · Paper page
  91. Martel, C., Allen, J., Pennycook, G., & Rand, D. G. (2024). Crowds can effec- tively identify misinformation at scale. Perspectives in Psychological Science, 19, 477-488. https://doi.org/10.1177/17456916231190388 · DOI · Paper page
  92. Mazariego, C., Jefford, M., Chan, R. J., Roberts, N., Millar, L., Anazodo, A., Hayes, S., Brown, B., Saunders, C., Webber, K., Vardy, J., Girgis, A., & Koczwara, B. (2022). Priority recommendations for the implementation of patient-reported outcomes in clinical cancer care: A Delphi study. Journal of Cancer Survivorship, 16(1), 33-43. https://doi.org/10.1007/s11764- 021-01135-2 · DOI
  93. McAdam, M. K., Baldessarini, R. J., Murphy, A. L., & Gardner, D. M. (2023). Second international consensus study of antipsychotic dosing (ICSAD-2). Journal of Psychopharmacology, 37(10), 982-991. https://doi. org/10.1177/02698811231205688 · DOI
  94. Meijering, J. V., & Tobi, H. (2016). The effect of controlled opinion feedback on Delphi features: Mixed messages from a real-world Delphi experiment. Technological Forecasting and Social Change, 103, 166-173. https://doi. org/10.1016/j.techfore.2015.11.008 · DOI
  95. Meijering, J. V., & Tobi, H. (2018). The effects of feeding back experts' own initial ratings in Delphi studies: A randomized trial. International Journal of Forecasting, 34(2), 216-224. https://doi.org/10.1016/j.ijforecast.2017.11.010 · DOI
  96. Mellers, B., Ungar, L., Baron, J., Ramos, J., Gurcay, B., Fincher, K., Scott, S. E., Moore, D., Atanasov, P., Swift, S. A., Murray, T., Stone, E., & Tetlock, P. E. (2014). Psychological strategies for winning a geopolitical forecast- ing tournament. Psychological Science, 25(5), 1106-1115. https://doi. org/10.1177/0956797614524255 · DOI · Paper page
  97. Mercier, H., & Claidière, N. (2022). Does discussion make crowds any wiser? Cognition, 222, 104912. https://doi.org/10.1016/j.cognition.2021.104912 · DOI
  98. Minas, H., & Jorm, A. F. (2010). Where there is no evidence: Use of expert con- sensus methods to fill the evidence gap in low-income countries and cultural minorities. International Journal of Mental Health Systems, 4, 33. https://doi. org/10.1186/1752-4458-4-33 · DOI
  99. Morgan, T. L., Fortier, M. S., Jain, R., Lane, K. N., Maclaren, K., McFadden, T., Prorok, J., Robison, J., Weston, Z. J., & Tomasone, J. R. (2025). Development of the Whole Day Matters Toolkit for Primary Care: A consensus-building study to mobilize national public health guidelines in practice. Health Promotion and Chronic Disease Prevention in Canada, 45(1), 1-19. https://doi. org/10.24095/hpcdp.45.1.01 · DOI
  100. Muchnik, L., Aral, S., & Taylor, S. J. (2013). Social influence bias: A randomized experiment. Science, 341, 647-651. https://doi.org/10.1126/ science.1240466 · DOI · Paper page
  101. Navajas, J., Niella, T., Garbulsky, G., Bahrami, B., & Sigman, M. (2018). Aggregated knowledge from a small number of debates outperforms the wis- dom of large crowds. Nature Human Behaviour, 2, 126-132. https://doi. org/10.1038/s41562-017-0273-4 · DOI
  102. Needham, D. M., Sepulveda, K. A., Dinglas, V. D., Chessare, C. M., Friedman, L. A., Bingham, C. O., III, & Turnbull, A. E. (2017). Core outcome measures for clinical research in acute respiratory failure survivors. An international mod- ified Delphi consensus study. American Journal of Respiratory and Critical Care Medicine, 196(9), 1122-1130. https://doi.org/10.1164/ rccm.201702-0372OC · DOI
  103. Niederberger, M., & Homberg, A. (2023). Argument-based QUalitative Analysis strategy (AQUA) for analyzing free-text responses in health sciences Delphi studies. MethodsX, 10, 102156. https://doi.org/10.1016/j.mex.2023.102156 · DOI
  104. Niederberger, M., Schifano, J., Deckert, S., Hirt, J., Homberg, A., Köberich, S., Kuhn, R., Rommel, A., Sonnberger, M., Alam, M., Backman, C., Banno, M., Bartoszko, J., Bloomfield, F., Bober, M. B., Chalkoo, M., Chan, T. M., Chen, Y., Coscia, C., et al. (2024). Delphi studies in social and health sciences- Recommendations for an interdisciplinary standardized reporting (DELPHISTAR). Results of a Delphi study. PLoS One, 19(8), Article e0304651. https://doi.org/10.1371/journal.pone.0304651 · DOI
  105. Niederberger, M., & Spranger, J. (2020). Delphi technique in health sciences: A map. Frontiers in Public Health, 8, 457. https://doi.org/10.3389/ fpubh.2020.00457 · DOI · Paper page
  106. Nilsson, K., Andersson, G., Johansson, P., & Lundgren, J. (2023). Developing and designing an internet-based support and education program for patients awaiting kidney transplantation with deceased donors through: A Delphi study. BMC Nephrology, 24(1), 311. https://doi.org/10.1186/s12882-023-03364-2 REFERENCES · DOI
  107. Nosek, B. A., Ebersole, C. R., DeHaven, A. C., & Mellor, D. T. (2018). The pre- registration revolution. Proceedings of the National Academy of Sciences USA, 115(11), 2600-2606. https://doi.org/10.1073/pnas.1708274114 · DOI · Paper page
  108. O'Leary, D. E. (2017). Crowd performance in prediction of the World Cup 2014. European Journal of Operational Research, 260, 715-724. https://doi. org/10.1016/j.ejor.2016.12.043 · DOI
  109. Page, S. E. (2007). The difference: How the power of diversity creates better groups, firms, schools, and societies. Princeton University Press.
  110. Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., Chou, R., Glanville, J., Grimshaw, J. M., Hrobjartsson, A., Lalu, M. M., Li, T., Loder, E. W., Mayo-Wilson, E., McDonald, S., et al. (2021). The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. BMJ, 372, n71. https://doi.org/10.1136/bmj.n71 · DOI · Paper page
  111. Päivärinta, T., Pekkola, S., & Moe, C. (2011). Grounding theory from Delphi studies. In International conference on information systems 2011, ICIS 2011.
  112. Paul, C. L., Sanson-Fisher, R., Douglas, H. E., Clinton-McHarg, T., Williamson, A., & Barker, D. (2011). Cutting the research pie: A value-weighting approach to explore perceptions about psychosocial research priorities for adults with haematological cancers. European Journal of Cancer Care, 20(3), 345-353. https://doi.org/10.1111/j.1365-2354.2010.01188.x · DOI
  113. Quirke, F. A., Battin, M. R., Bernard, C., Biesty, L., Bloomfield, F. H., Daly, M., Finucane, E., Haas, D. M., Healy, P., Hurley, T., Koskei, S., Meher, S., Molloy, E. J., Niaz, M., Bhraonáin, E. N., Okaronon, C. O., Tabassum, F., Walker, K., Webbe, J. R. H., et al. (2023). Multi-Round versus Real-Time Delphi survey approach for achieving consensus in the COHESION core outcome set: A randomised trial. Trials, 24(1), 461. https://doi.org/10.1186/ s13063-023-07388-9 · DOI
  114. Radomski, T. R., Decker, A., Khodyakov, D., Thorpe, C. T., Hanlon, J. T., Roberts, M. S., Fine, M. J., & Gellad, W. F. (2022). Development of a metric to detect and decrease low-value prescribing in older adults. JAMA Network Open, 5(2), e2148599. https://doi.org/10.1001/jamanetworkopen.2021.48599 · DOI
  115. Ramke, J., Evans, J. R., Habtamu, E., Mwangi, N., Silva, J. C., Swenor, B. K., Congdon, N., Faal, H. B., Foster, A., Friedman, D. S., Gichuhi, S., Jonas, J. B., Khaw, P. T., Kyari, F., Murthy, G. V. S., Wang, N., Wong, T. Y., Wormald, R., Yusufu, M., et al. (2022). Grand challenges in global eye health: A global pri- oritisation process using Delphi method. Lancet Healthy Longevity, 3(1), e31- e41. https://doi.org/10.1016/s2666-7568(21)00302-0 · DOI
  116. Rauch, W. (1979). The decision Delphi. Technological Forecasting and Social Change, 15(3), 159-169. https://doi.org/10.1016/0040-1625(79)90011-8 · DOI
  117. Robinson, J., Thorn, P., McKay, S., Hemming, L., Battersby-Coulter, R., Cooper, C., Veresova, M., Li, A., Reavley, N., Rice, S., Lamblin, M., Pirkis, J., Reidenberg, D., Harrison, V., Skehan, J., & La Sala, L. (2023). #chatsafe 2.0. Updated guidelines to support young people to communicate safely online about self-harm and suicide: A Delphi expert consensus study. PLoS One, 18(8), e0289494. https://doi.org/10.1371/journal.pone.0289494 · DOI
  118. Ross, A. M., Hart, L. M., Jorm, A. F., Kelly, C. M., & Kitchener, B. A. (2012). Development of key messages for adolescents on providing basic mental health first aid to peers: A Delphi consensus study. Early Intervention in Psychiatry, 6(3), 229-238. https://doi.org/10.1111/j.1751-7893.2011.00331.x · DOI
  119. Ross, A. M., Kelly, C. M., & Jorm, A. F. (2014). Re-development of mental health first aid guidelines for suicidal ideation and behaviour: A Delphi study. BMC Psychiatry, 14, 241. https://doi.org/10.1186/s12888-014-0241-8 · DOI
  120. Rowe, G., & Wright, G. (1996). The impact of task characteristics on the perfor- mance of structured group forecasting techniques. International Journal of Forecasting, 12(1), 73-89. https://doi.org/10.1016/0169-2070(95)00658-3 · DOI
  121. Rowe, G., Wright, G., & McColl, A. (2005). Judgment change during Delphi-like procedures: The role of majority influence, expertise, and confidence. Technological Forecasting and Social Change, 72(4), 377-399. https://doi. org/10.1016/j.techfore.2004.03.004 · DOI
  122. Ryan, J. M., Devane, D., Simiceva, A., Eppich, W., Kavanagh, D. O., Cullen, C., Hogan, A. M., & McNamara, D. A. (2024). Surgical Handover Core Outcome Measures (SH-CORE): A protocol for the development of a core outcome set for trials in surgical handover [Article]. Trials, 25(1), Article 373. https://doi. org/10.1186/s13063-024-08201-x · DOI
  123. Sackman, H. (1975). Summary evaluation of Delphi. Policy Analysis, 1(4), 693-718. http://www.jstor.org/stable/42784280
  124. Sahle, B. W., Reavley, N. J., Morgan, A. J., Yap, M. B. H., Reupert, A., & Jorm, A. F. (2022). A Delphi study to identify intervention priorities to prevent the occurrence and reduce the impact of adverse childhood experiences. The Australian and New Zealand Journal of Psychiatry, 56(6), 686-694. https:// doi.org/10.1177/00048674211025717 · DOI
  125. Saunders, H., Anderson, C., Feldman, F., Holroyd-Leduc, J., Jain, R., Liu, B., Macaulay, S., Marr, S., Silvius, J., Weldon, J., Bayoumi, A. M., Straus, S. E., Tricco, A. C., & Isaranuwatchai, W. (2023). Developing a fall prevention inter- vention economic model. PLoS One, 18(1 January), Article e0280572. https:// doi.org/10.1371/journal.pone.0280572 · DOI
  126. Schifano, J., & Niederberger, M. (2025). How Delphi studies in the health sci- ences find consensus: A scoping review. Systematic Reviews, 14(1), 14. https:// doi.org/10.1186/s13643-024-02738-3 · DOI
  127. Sii, S., Barton, K., Pasquale, L. R., Yamamoto, T., King, A. J., & Azuara-Blanco, A. (2018). Reporting harm in glaucoma surgical trials: Systematic review and a consensus-derived new classification system. American Journal of Ophthalmology, 194, 153-162. https://doi.org/10.1016/j.ajo.2018.07.014 · DOI
  128. Simoiu, C., Sumanth, C., Mysore, A., & Goel, S. (2019). Studying the "wisdom of crowds" at scale. In Seventh AAAI conference on human computation and crowdsourcing (HCOMP-19).
  129. Sinha, I. P., Smyth, R. L., & Williamson, P. R. (2011). Using the Delphi technique to determine which outcomes to measure in clinical trials: Recommendations for the future based on a systematic review of existing studies. PLoS Medicine, 8(1), e1000393. https://doi.org/10.1371/journal.pmed.1000393 · DOI · Paper page
  130. Smelt, A. F., Louter, M. A., Kies, D. A., Blom, J. W., Terwindt, G. M., van der Heijden, G. J., De Gucht, V., Ferrari, M. D., & Assendelft, W. J. (2014). What do patients consider to be the most important outcomes for effectiveness stud- ies on migraine treatment? Results of a Delphi study. PLoS One, 9(6), e98933. https://doi.org/10.1371/journal.pone.0098933 · DOI
  131. Spranger, J., Homberg, A., Sonnberger, M., & Niederberger, M. (2022). Reporting guidelines for Delphi techniques in health sciences: A methodologi- cal review. Zeitschrift für Evidenz, Fortbildung und Qualität im Gesundheitswesen, 172, 1-11. https://doi.org/10.1016/j.zefq.2022.04.025 · DOI · Paper page
  132. Sri, A., Bailey, K. E., Scarborough, R., Gilkerson, J. R., Thursky, K., Browning, G. F., & Hardefeldt, L. Y. (2024). Reaching consensus amongst international experts on the use of high importance-rated antimicrobials in animals -A Delphi study. One Health, 19, Article 100883. https://doi.org/10.1016/j. onehlt.2024.100883 · DOI
  133. Steinert, M. (2009). A dissensus based online Delphi approach: An explorative research tool. Technological Forecasting and Social Change, 76(3), 291-300. https://doi.org/10.1016/j.techfore.2008.10.006 · DOI
  134. Swedo, S. E., Baguley, D. M., Denys, D., Dixon, L. J., Erfanian, M., Fioretti, A., Jastreboff, P. J., Kumar, S., Rosenthal, M. Z., Rouw, R., Schiller, D., Simner, J., Storch, E. A., Taylor, S., Werff, K. R. V., Altimus, C. M., & Raver, S. M. (2022). Consensus definition of misophonia: A Delphi study. Frontiers in Neuroscience, 16, 841816. https://doi.org/10.3389/fnins.2022.841816 · DOI · Paper page
  135. Tapio, P. (2003). Disaggregative policy Delphi: Using cluster analysis as a tool for systematic scenario formation. Technological Forecasting and Social Change, 70(1), 83-101. https://doi.org/10.1016/S0040-1625(01)00177-9 · DOI
  136. Ter Veer, E., van Rijssen, L. B., Besselink, M. G., Mali, R. M. A., Berlin, J. D., Boeck, S., Bonnetain, F., Chau, I., Conroy, T., Van Cutsem, E., Deplanque, G., Friess, H., Glimelius, B., Goldstein, D., Herrmann, R., Labianca, R., Van Laethem, J. L., Macarulla, T., van der Meer, J. H. M., et al. (2018). Consensus statement on mandatory measurements in pancreatic cancer trials (COMM- PACT) for systemic treatment of unresectable disease. Lancet Oncology, 19(3), e151-e160. https://doi.org/10.1016/s1470-2045(18)30098-6 · DOI
  137. Thoomes, E., Falla, D., Cleland, J. A., Fernández-de-Las-Peñas, C., Gallina, A., & de Graaf, M. (2023). Conservative management for lumbar radiculopathy based on the stage of the disorder: A Delphi study. Disability and Rehabilitation, 45(21), 3539-3548. https://doi.org/10.1080/09638288.2022.2130448 · DOI
  138. Toulany, A., Khodyakov, D., Mooney, S., Stromquist, L., Bailey, K., Barber, C. E., Batthish, M., Cleverley, K., Dimitropoulos, G., Gorter, J. W., Grahovac, D., REFERENCES Grimes, R., Guttman, B., Hébert, M. L., John, T., Lo, L., Luong, D., MacGregor, L., Mukerji, G., et al. (2024). Quality indicators for transition from pediatric to adult care for youth with chronic conditions: Proposal for an online modified Delphi study. JMIR Research Protocols, 13, e60860. https:// doi.org/10.2196/60860 · DOI
  139. Tournier, A. L., Bonamin, L. V., Buchheim-Schmidt, S., Cartwright, S., Dombrowsky, C., Doesburg, P., Holandino, C., Kokornaczyk, M. O., van de Kraats, E. B., López-Carvallo, J. A., Nandy, P., Mazón-Suástegui, J. M., Mirzajani, F., Poitevin, B., Scherr, C., Thieves, K., Würtenberger, S., & Baumgartner, S. (2024). Scientific guidelines for preclinical research on poten- tised preparations manufactured according to current pharmacopoeias-The PrePoP guidelines. Journal of Integrative Medicine, 22(5), 533-544. https:// doi.org/10.1016/j.joim.2024.06.005 · DOI
  140. Turnbull, A. E., Dinglas, V. D., Friedman, L. A., Chessare, C. M., Sepúlveda, K. A., Bingham, C. O., III, & Needham, D. M. (2018). A survey of Delphi panelists after core outcome set development revealed positive feedback and methods to facilitate panel member participation. Journal of Clinical Epidemiology, 102, 99-106. https://doi.org/10.1016/j.jclinepi.2018.06.007 · DOI
  141. Waggoner, J., Carline, J. D., & Durning, S. J. (2016). Is there a consensus on consensus methodology? Descriptions and recommendations for future con- sensus research. Academic Medicine, 91(5), 663-668. https://doi. org/10.1097/acm.0000000000001092 · DOI
  142. Yap, M. B. H., & Jorm, A. F. (2015). Parental factors associated with childhood anxiety, depression, and internalizing problems: A systematic review and meta- analysis. Journal of Affective Disorders, 175, 424-440. https://doi. org/10.1016/j.jad.2015.01.050 · DOI
  143. Yap, M. B., Pilkington, P. D., Ryan, S. M., Kelly, C. M., & Jorm, A. F. (2014). Parenting strategies for reducing the risk of adolescent depression and anxiety disorders: A Delphi consensus study. Journal of Affective Disorders, 156, 67-75. https://doi.org/10.1016/j.jad.2013.11.017 · DOI
  144. Yu, M., Keel, S., Mariotti, S., Mills, J., & Müller, A. (2023). Development of the WHO eye care competency framework. Human Resources for Health, 21(1), 46. https://doi.org/10.1186/s12960-023-00834-4 · DOI
  145. Zickafoose, A., Lu, P., & Baker, M. (2022). Forecasting food innovations with a Delphi study. Food, 11(22). https://doi.org/10.3390/foods11223723 · DOI

Cited by

Sorted by citations

Related

Authors
Jorm, A.
Affiliations
The University of Melbourne (AU)
Corresponding author
Jorm, Anthony
ORCID
Jorm, Anthony
Venue
Advancing Methods for Interdisciplinarity in the Social Sciences
Publisher
Springer Nature Singapore
Published
2025
ISSN
  • 3059-3506
  • 3059-3514
DOI
10.1007/978-981-96-8357-4
OpenAlex
W4412732349
Cited by
22
References
184
Citations by year
2026: 22
FWCI
40.85
Type
book
Language
en
Open access
Licence
Locations
Springer eBooks (publishedVersion, cc-by)
Funders
  • Medical Research Council
  • National Health and Medical Research Council (1172889)
Topics
  • Social Sciences › Sociology and Political Science › Delphi Technique in Research
  • Social Sciences › Decision Sciences › Management Science and Operations Research › Multi-Criteria Decision Making
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement
Keywords
Delphi method, expert consensus, consensus building

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review