A comparison of formal consensus methods used for developing clinical guidelines
The DOI matches its Crossref record.
Abstract
Objectives: To compare two consensus development methods commonly used for developing clinical guidelines in terms of the judgments produced, closeness of consensus, amount of change between rounds, concordance with research evidence and reliability. Methods: In all, 213 general practitioners and mental health professionals from England participated in four Delphi and four nominal groups. They rated the appropriateness of four treatments (cognitive behavioural therapy [CBT], behavioural therapy [BT], brief psychodynamic interpersonal therapy [BPIT] and antidepressants) for three conditions. First, participants rated the appropriateness of interventions independently, using a postal questionnaire. For nominal groups, the ratings were fed back and discussed at a meeting, and then group members privately completed the questionnaire again. For Delphi groups, there was feedback but no discussion, and the entire process was conducted by postal questionnaire. Results: The effect of consensus method on final ratings varied with therapeutic intervention, with nominal groups rating CBT and antidepressants more favourably than Delphi groups. Consensus was closer in the nominal than in the Delphi groups in both rounds. There was no overall difference between groups in their concordance with research evidence (odds ratio 1.13, 95% confidence interval 0.79–1.61). In this study, the Delphi method was more reliable ( κ coefficients 0.88 and 0.89 compared with 0.41 and 0.65 for nominal groups). Conclusions: The advantages of nominal groups (more consensus; greater understanding of reasons for disagreement) could be combined with the greater reliability of the Delphi approach by developing a hybrid method.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cited by
Guidelines for Reporting Reliability and Agreement Studies (GRRAS) were proposed
Kottner, J., Audigé, L., Brorson, S., Donner, A., Gajewski, B. J., Hróbjartsson, A., Roberts, C., Shoukri, M., & Streiner, D. L. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.03.002 · Cited by 2,174
Guidelines for Reporting Reliability and Agreement Studies (GRRAS) were proposed
Kottner, J., Audige, L., Brorson, S., Donner, A., Gajewski, B. J., Hróbjartsson, A., Roberts, C., Shoukri, M., & Streiner, D. L. (2011) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2011.01.016 · Cited by 997
Using the Delphi expert consensus method in mental health research
Jorm, A. F. (2015) · Australian & New Zealand Journal of Psychiatry · doi:10.1177/0004867415600891 · Cited by 765
Dickinson, L., Ahmed, H. U., Allen, C., Barentsz, J. O., Carey, B., Futterer, J. J., Heijmink, S. W., Hoskin, P. J., Kirkham, A., Padhani, A. R., et al. (2011) · European Urology · doi:10.1016/j.eururo.2010.12.009 · Cited by 737
The in-between world of knowledge brokering
Lomas, J. (2007) · BMJ · doi:10.1136/bmj.39038.593380.ae · Cited by 542
Metadata
- Authors
- Hutchings, A., Raine, R., Sanderson, C., & Black, N.
- Venue
- Journal of Health Services Research & Policy
- Publisher
- SAGE Publications
- Published
- 2006-10-01
- Volume
- 11
- Issue
- 4
- Pages
- 218-224
- ISSN
- 1355-8196 (ISSN-L)
- 1758-1060
- OpenAlex
- W2113904988
- Cited by
- 125
- References
- 0
- FWCI
- 12.68
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: A comparison of formal consensus methods used for developing clinical guidelines. https://app.cochat.ai/papers/10.1258/135581906778476553
- The paper
- Hutchings, A., Raine, R., Sanderson, C., & Black, N. (2006). A comparison of formal consensus methods used for developing clinical guidelines. Journal of Health Services Research & Policy. https://doi.org/10.1258/135581906778476553
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.