The DOI matches its Crossref record; 54 of the 55 references it cites were found in a registry or an index.
Using the Delphi expert consensus method in mental health research
Open accessAbstract
Objective: The article gives an introductory overview of the use of the Delphi expert consensus method in mental health research. It explains the rationale for using the method, examines the range of uses to which it has been put in mental health research, and describes the stages of carrying out a Delphi study using examples from the literature. Method: To ascertain the range of uses, a systematic search was carried out in PubMed. The article also examines the implications of ‘wisdom of crowds’ research for how to conduct Delphi studies. Results: The Delphi method is a systematic way of determining expert consensus that is useful for answering questions that are not amenable to experimental and epidemiological methods. The validity of the approach is supported by ‘wisdom of crowds’ research showing that groups can make good judgements under certain conditions. In mental health research, the Delphi method has been used for making estimations where there is incomplete evidence (e.g. What is the global prevalence of dementia?), making predictions (e.g. What types of interactions with a person who is suicidal will reduce their chance of suicide?), determining collective values (e.g. What areas of research should be given greatest priority?) and defining foundational concepts (e.g. How should we define ‘relapse’?). A range of experts have been used in Delphi research, including clinicians, researchers, consumers and caregivers. Conclusion: The Delphi method has a wide range of potential uses in mental health research.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Publications suggest that the results of single stakeholder Delphi studies stabilise [44], and achieve moderate replicability, with 20–30 participants [46].”
From Essential Information About Chronobiology and Chronotherapy for the Optimal Care of People With Bipolar Disorders: An Expert Consensus (2026) · cites this paper as [44]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“The study followed the key elements of the Delphi method [44].”
From Essential Information About Chronobiology and Chronotherapy for the Optimal Care of People With Bipolar Disorders: An Expert Consensus (2026) · cites this paper as [44]
The cited source says what the claim attributes to it.
In the citing paper
“There is no gold standard for defining consensus in a Delphi study [44, 47].”
From Essential Information About Chronobiology and Chronotherapy for the Optimal Care of People With Bipolar Disorders: An Expert Consensus (2026) · cites this paper as [44]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Preferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement.
(2009) · PubMed Central · Cited by 44,859
Cochrane Handbook for Systematic Reviews of Interventions: Cochrane Book Series
(2008) · doi:10.1002/9780470712184 · Cited by 25,048
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement
Moher, D., Liberati, A., Tetzlaff, J., & Altman, D. G. (2009) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2009.06.005 · Cited by 19,575
Hoffmann, T. C., Glasziou, P. P., Boutron, I., Milne, R., Perera, R., Moher, D., Altman, D. G., Barbour, V., Macdonald, H., Johnston, M., et al. (2014) · BMJ · doi:10.1136/bmj.g1687 · Cited by 10,548 · cited 2×
“…criteria for quality assessment of randomized clinical trials (Verhagen et al., 1998), guidelines for clinical trial protocol content (Tetzlaff et al., 2012), standards for reporting interventions used in trials (Hoffmann et al., 2014) and methods used in systematic reviews (Pincus et al., 2011).”
MethodGlobal prevalence of dementia: a Delphi consensus study
Ferri, C. P., Prince, M. J., Brayne, C., Brodaty, H., Fratiglioni, L., Ganguli, M., Hall, K. S., Hasegawa, K., Hendrie, H. C., Huang, Y., et al. (2005) · The Lancet · doi:10.1016/s0140-6736(05)67889-0 · Cited by 5,336
Self-citationImproving the Quality of Reporting of Randomized Controlled Trials: The CONSORT Statement
Begg, C. (1996) · JAMA · doi:10.1001/jama.1996.03540080059030 · Cited by 3,127
Verhagen, A. P., de Vet, H. C. W., de Bie, R. A., Kessels, A. G. H., Boers, M., Bouter, L. M., & Knipschild, P. G. (1998) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(98)00131-0 · Cited by 2,470
The difference: how the power of diversity creates better groups, firms, schools, and societies
(2007) · Choice Reviews Online · doi:10.5860/choice.45-1534 · Cited by 2,280
The Delphi Method: An Experimental Study of Group Opinion
Dalkey, N. C. (1969) · RAND Corporation eBooks · doi:10.7249/rm5888 · Cited by 1,673
International Consensus Study of Antipsychotic Dosing
Gardner, D. M., Murphy, A. L., O'Donnell, H., Centorrino, F., & Baldessarini, R. J. (2010) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2009.09060802 · Cited by 1,328
How social influence can undermine the wisdom of crowd effect
Lorenz, J., Rauhut, H., Schweitzer, F., & Helbing, D. (2011) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1008636108 · Cited by 1,017
Stability of response characteristics of a Delphi panel: application of bootstrap data expansion
Akins, R. B., Tolson, H., & Cole, B. R. (2005) · BMC Medical Research Methodology · doi:10.1186/1471-2288-5-37 · Cited by 994
Pacchiarotti, I., Bond, D. J., Baldessarini, R. J., Nolen, W. A., Grunze, H., Licht, R. W., Post, R. M., Berk, M., Goodwin, G. M., Sachs, G. S., et al. (2013) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2013.13020185 · Cited by 770
The wisdom of crowds : why the many are smarter than the few
Surowiecki, J. (2005) · Cited by 760
Cha, M., Eastwaad, S., Hartan, R., Maher, D., Olkin, I., Pitkin, R., Rennie, D., Schulz, K. F., Simel, D. L., & Straup, D. F. (1996) · Cited by 736
The Need for Evidence-Based Medicine
Sackett, D. L., & Rosenberg, W. M. (1995) · Journal of the Royal Society of Medicine · doi:10.1177/014107689508801105 · Cited by 694
Hadlaczky, G., Hökby, S., Mkrtchian, A., Carli, V., & Wasserman, D. (2014) · International Review of Psychiatry · doi:10.3109/09540261.2014.924910 · Cited by 433
Conducting Qualitative Research in Mental Health: Thematic and Content Analyses
Crowe, M., Inder, M., & Porter, R. (2015) · Australian & New Zealand Journal of Psychiatry · doi:10.1177/0004867415582053 · Cited by 398
TENTS guidelines: development of post-disaster psychosocial care guidelines through a Delphi process
Bisson, J. I., Tavakoly, B., Witteveen, A. B., Ajduković, D., Jehel, L., Johansen, V. A., Nordanger, D. Ø., García, F. O., Punamäki, R., Schnyder, U., et al. (2009) · The British Journal of Psychiatry · doi:10.1192/bjp.bp.109.066266 · Cited by 212
Davis‐Stober, C. P., Budescu, D. V., Dana, J. D., & Broomell, S. B. (2014) · Decision · doi:10.1037/dec0000004 · Cited by 202
Administration and policy in mental health
Menninger, R. W. (1993) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/bf00706991 · Cited by 185
Boccardi, M., Bocchetta, M., Apostolova, L. G., Barnes, J., Bartzokis, G., Corbetta, G., DeCarli, C. S., deToledo‐Morrell, L., Firbank, M., Ganzola, R., et al. (2014) · Alzheimer s & Dementia · doi:10.1016/j.jalz.2014.02.009 · Cited by 166
Pincus, T., Miles, C., Froud, R., Underwood, M., Carnes, D., & Taylor, S. J. (2011) · BMC Medical Research Methodology · doi:10.1186/1471-2288-11-14 · Cited by 161
Ross, A., Kelly, C., & Jorm, A. F. (2014) · BMC Psychiatry · doi:10.1186/s12888-014-0241-8 · Cited by 146
Self-citationRecovery in Psychosis: A Delphi Study With Experts by Experience
Law, H., & Morrison, A. P. (2014) · Schizophrenia Bulletin · doi:10.1093/schbul/sbu047 · Cited by 144
Research priorities for public mental health in Europe: recommendations of the ROAMER project
Forsman, A. K., Wahlbeck, K., Aaro, L. E., Alonso, J., Barry, M. M., Brunn, M., Cardoso, G., Cattan, M., de Girolamo, G., Eberhard-Gran, M., et al. (2015) · The European Journal of Public Health · doi:10.1093/eurpub/cku232 · Cited by 139
A comparison of formal consensus methods used for developing clinical guidelines
Hutchings, A., Raine, R., Sanderson, C. F. B., & Black, N. A. (2006) · Journal of Health Services Research & Policy · doi:10.1258/135581906778476553 · Cited by 125
Yap, M. B. H., Pilkington, P. D., Ryan, S., Kelly, C., & Jorm, A. F. (2013) · Journal of Affective Disorders · doi:10.1016/j.jad.2013.11.017 · Cited by 116
Self-citationEssential Evidence-Based Components of First-Episode Psychosis Services
Addington, D. E., McKenzie, E., Norman, R. M. G., Wang, J., & Bond, G. R. (2013) · Psychiatric Services · doi:10.1176/appi.ps.201200156 · Cited by 112
Noting a Landmark Achievement: Mental Health First Aid Training Reaches 1% of Australian Adults
Jorm, A. F., & Kitchener, B. A. (2011) · Australian & New Zealand Journal of Psychiatry · doi:10.3109/00048674.2011.594785 · Cited by 111
Self-citationGoodyear, M. J., Hill, T., Allchin, B., McCormick, F., Hine, R. H., Cuff, R., & O’Hanlon, B. (2015) · International Journal of Mental Health Nursing · doi:10.1111/inm.12120 · Cited by 101
Self-help strategies that are helpful for sub-threshold depression: A Delphi consensus study
Morgan, A. J., & Jorm, A. F. (2008) · Journal of Affective Disorders · doi:10.1016/j.jad.2008.08.004 · Cited by 91
Self-citationHart, L., Jorm, A. F., Kanowski, L. G., Kelly, C. M., & Langlands, R. L. (2009) · BMC Psychiatry · doi:10.1186/1471-244x-9-47 · Cited by 85
Self-citationMorgan, A. J., Jorm, A. F., & Mackinnon, A. J. (2012) · The British Journal of Psychiatry · doi:10.1192/bjp.bp.111.101394 · Cited by 79
Self-citationDeveloping a guideline for clinical trial protocol content: Delphi consensus survey
Tetzlaff, J., Moher, D., & Chan, A. (2012) · Trials · doi:10.1186/1745-6215-13-176 · Cited by 74
Berk, L., Berk, M., Dodd, S., Kelly, C., Cvetkovski, S., & Jorm, A. F. (2013) · BMC Medicine · doi:10.1186/1741-7015-11-162 · Cited by 59
Self-citationRoss, A., Hart, L., Jorm, A. F., Kelly, C. M., & Kitchener, B. A. (2012) · Early Intervention in Psychiatry · doi:10.1111/j.1751-7893.2011.00331.x · Cited by 57
Self-citationBerk, L., Jorm, A. F., Kelly, C. M., Dodd, S., & Berk, M. (2011) · Bipolar Disorders · doi:10.1111/j.1399-5618.2011.00942.x · Cited by 55
Self-citationService Users’ Priorities and Preferences for Treatment of Psychosis: A User-Led Delphi Study
Byrne, R. E., & Morrison, A. P. (2014) · Psychiatric Services · doi:10.1176/appi.ps.201300289 · Cited by 47
Reavley, N. J., Ross, A., Killackey, E. J., & Jorm, A. F. (2012) · BMC Psychiatry · doi:10.1186/1471-244x-12-135 · Cited by 46
Self-citationHart, L., Damiano, S. R., Chittleborough, P., Paxton, S. J., & Jorm, A. F. (2014) · Body Image · doi:10.1016/j.bodyim.2014.06.010 · Cited by 45
Self-citationRe-development of mental health first aid guidelines for non-suicidal self-injury: a Delphi study
Ross, A., Kelly, C. M., & Jorm, A. F. (2014) · BMC Psychiatry · doi:10.1186/s12888-014-0236-5 · Cited by 42
Self-citationChalmers, K., Bond, K. S., Jorm, A. F., Kelly, C. M., Kitchener, B. A., & Williams-Tchen, A. (2014) · International Journal of Mental Health Systems · doi:10.1186/1752-4458-8-6 · Cited by 42
Self-citationThe kind of group you want to belong to: Effects of group structure on group accuracy
Jönsson, M. L., Hahn, U., & Olsson, E. J. (2015) · Cognition · doi:10.1016/j.cognition.2015.04.013 · Cited by 40
Development and validation of the geriatric depression inventory in Chinese culture
Xie, Z., Lv, X., Hu, Y., Ma, W., Xie, H., Lin, K., Yu, X., & Wang, H. (2015) · International Psychogeriatrics · doi:10.1017/s1041610215000162 · Cited by 31
Fischer, J. A., Kelly, C. M., Kitchener, B. A., & Jorm, A. F. (2013) · SAGE Open · doi:10.1177/2158244013516769 · Cited by 31
Self-citationCairns, K., Yap, M. B. H., Reavley, N. J., & Jorm, A. F. (2015) · Journal of Affective Disorders · doi:10.1016/j.jad.2015.05.019 · Cited by 30
Self-citationFrisoni, G. B., & Jack, C. R. (2015) · Alzheimer s & Dementia · doi:10.1016/j.jalz.2014.05.1761 · Cited by 28
San, L., Serrano, M., Cañas, F., Romero, S. L., Sánchez-Cabezudo, Á., & Villar, M. (2014) · International Journal of Psychiatry in Clinical Practice · doi:10.3109/13651501.2014.1002501 · Cited by 27
Reavley, N. J., Ross, A., Killackey, E. J., & Jorm, A. F. (2013) · PeerJ · doi:10.7717/peerj.43 · Cited by 27
Self-citationLauriks, S., de Wit, M. A. S., Buster, M. C. A., Arah, O. A., & Klazinga, N. (2013) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-013-0506-4 · Cited by 18
Lenox‐Smith, A., Greenstreet, L., Burslem, K., & Knight, C. (2009) · Clinical Drug Investigation · doi:10.2165/00044011-200929030-00004 · Cited by 15
Clinical subtypes of core premenstrual disorders: a Delphi survey
ISMAIL, K. M. K., Nevatte, T. M., O’Brien, S., Paschetta, E., Bäckström, T. C., Dennerstein, L., Eriksson, E., Freeman, E. W., Panay, N., Pearlstein, T. B., et al. (2013) · Archives of Women s Mental Health · doi:10.1007/s00737-012-0326-7 · Cited by 8
Mental health care training for practitioners in remote and rural areas
de Mello, G., Fraser, F., Nicoll, P., Ker, J., Green, G., & Laird, C. (2013) · The Clinical Teacher · doi:10.1111/tct.12054 · Cited by 5
Flagged first, then by citations
1 of the 55 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Delphi Technique in Health Sciences: A Map
Niederberger, M., & Spranger, J. (2020) · Frontiers in Public Health · doi:10.3389/fpubh.2020.00457 · Cited by 1,072
Use of Delphi in health sciences research: A narrative review
Shang, Z. (2023) · Medicine · doi:10.1097/md.0000000000032829 · Cited by 739
Michie, S., Carey, R. N., Johnston, M., Rothman, A. J., de Bruin, M., Kelly, M. P., & Connell, L. E. (2018) · Annals of Behavioral Medicine · doi:10.1007/s12160-016-9816-6 · Cited by 475
Ahmadi, A., Noetel, M., Parker, P. D., Ryan, R. M., Ntoumanis, N., Reeve, J., Beauchamp, M. R., Dicke, T., Yeung, A. S., Ahmadi, M., et al. (2023) · Journal of Educational Psychology · doi:10.1037/edu0000783 · Cited by 615
Gouttebarge, V., Bindra, A., Blauwet, C. A., Campriani, N., Currie, A., Engebretsen, L., Hainline, B. W., Kroshus, E., Mcduff, D. R., Mountjoy, M. L., et al. (2020) · British Journal of Sports Medicine · doi:10.1136/bjsports-2020-102411 · Cited by 393
Metadata
- Authors
- Jorm, A. F.
- Affiliations
- The University of Melbourne (AU)
- Corresponding author
- Jorm, Anthony F
- ORCID
- Jorm, Anthony F
- Venue
- Australian & New Zealand Journal of Psychiatry
- Publisher
- SAGE Publications
- Published
- 2015-08-21
- Volume
- 49
- Issue
- 10
- Pages
- 887-897
- ISSN
- 0004-8674 (ISSN-L)
- 1440-1614
- OpenAlex
- W2111751736
- Cited by
- 765
- References
- 55
- Citations by year
- 2026: 81 · 2025: 109 · 2024: 115 · 2023: 82 · 2022: 93 · 2021: 80 · 2020: 67 · 2019: 58 · 2018: 40 · 2017: 20 · 2016: 18 · 2015: 1
- FWCI
- 10.69
- Type
- journal article
- Language
- en
- Open access
- green
- hdl.handle.net
- Locations
- Australian & New Zealand Journal of Psychiatry (publishedVersion)
- PubMed (publishedVersion)
- Minerva Access (University of Melbourne) (submittedVersion)
- Funders
- National Health and Medical Research Council
- Topics
- Social Sciences › Sociology and Political Science › Delphi Technique in Research
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement
- Health Sciences › Nursing › Research and Theory › Health Education and Validation
- Keywords
- Delphi method, expert consensus, mental health research, wisdom of crowds, suicide prevention, research prioritization, relapse definition
- MeSH terms
- Consensus Development Conferences as Topic, Consensus Statements as Topic, Delphi Technique, Humans, Mental Health, Consensus, Biomedical Research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
