The DOI matches its Crossref record; 7 of the 7 references it cites were found in a registry or an index.
The use of qualitative methods to inform Delphi surveys in core outcome set development
Open accessAbstract
BACKGROUND: Core outcome sets (COS) help to minimise bias in trials and facilitate evidence synthesis. Delphi surveys are increasingly being used as part of a wider process to reach consensus about what outcomes should be included in a COS. Qualitative research can be used to inform the development of Delphi surveys. This is an advance in the field of COS development and one which is potentially valuable; however, little guidance exists for COS developers on how best to use qualitative methods and what the challenges are. This paper aims to provide early guidance on the potential role and contribution of qualitative research in this area. We hope the ideas we present will be challenged, critiqued and built upon by others exploring the role of qualitative research in COS development. This paper draws upon the experiences of using qualitative methods in the pre-Delphi stage of the development of three different COS. Using these studies as examples, we identify some of the ways that qualitative research might contribute to COS development, the challenges in using such methods and areas where future research is required. RESULTS: Qualitative research can help to identify what outcomes are important to stakeholders; facilitate understanding of why some outcomes may be more important than others, determine the scope of outcomes; identify appropriate language for use in the Delphi survey and inform comparisons between stakeholder data and other sources, such as systematic reviews. Developers need to consider a number of methodological points when using qualitative research: specifically, which stakeholders to involve, how to sample participants, which data collection methods are most appropriate, how to consider outcomes with stakeholders and how to analyse these data. A number of areas for future research are identified. CONCLUSIONS: Qualitative research has the potential to increase the research community's confidence in COS, although this will be dependent upon using rigorous and appropriate methodology. We have begun to identify some issues for COS developers to consider in using qualitative methods to inform the development of Delphi surveys in this article.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The Discovery of Grounded Theory; Strategies for Qualitative Research
Glaser, B. G., Strauss, A. L., & Strutzel, E. (1968) · Nursing Research · doi:10.1097/00006199-196807000-00014 · Cited by 39,449
Research guidelines for the Delphi survey technique
Hasson, F. (2000) · Journal of Advanced Nursing · doi:10.1046/j.1365-2648.2000.01567.x · Cited by 5,951
Qualitative Research: Consensus methods for medical and health services research
Jones, J., & Hunter, D. (1995) · BMJ · doi:10.1136/bmj.311.7001.376 · Cited by 3,471
Self-citationDeveloping core outcome sets for clinical trials: issues to consider
Williamson, P. R., Altman, D. G., Blazeby, J. M., Clarke, M., Devane, D., Gargon, E., & Tugwell, P. (2012) · Trials · doi:10.1186/1745-6215-13-132 · Cited by 1,661
Self-citationSystematic Review of the Empirical Evidence of Study Publication Bias and Outcome Reporting Bias
Dwan, K., Altman, D. G., Arnaiz, J. A., Bloom, J., Chan, A.-W., Cronin, E., Decullier, E., Easterbrook, P. J., Von Elm, E., Gamble, C., et al. (2008) · PLoS ONE · doi:10.1371/journal.pone.0003081 · Cited by 1,546
Self-citationKirkham, J. J., Dwan, K. M., Altman, D. G., Gamble, C., Dodd, S., Smyth, R., & Williamson, P. R. (2010) · BMJ · doi:10.1136/bmj.c365 · Cited by 1,133 · cited 1×
“based on the significance of the findings [7–9].”
BackgroundSelf-citationBruce, I., Harman, N., Williamson, P., Tierney, S., Callery, P., Mohiuddin, S., Payne, K., Fenwick, E., Kirkham, J., & O’Brien, K. (2015) · Health Technology Assessment · doi:10.3310/hta19680 · Cited by 65
Self-citation
Cited by
The COMET Handbook: version 1.0
Williamson, P. R., Altman, D. G., Bagley, H., Barnes, K. L., Blazeby, J. M., Brookes, S. T., Clarke, M., Gargon, E., Gorst, S., Harman, N., et al. (2017) · Trials · doi:10.1186/s13063-017-1978-4 · Cited by 2,011
Self-citationOutcomes That Define Successful Advance Care Planning: A Delphi Panel Consensus
Sudore, R. L., Heyland, D. K., Lum, H. D., Rietjens, J. A. C., Korfage, I. J., Ritchie, C. S., Hanson, L. C., Meier, D. E., Pantilat, S. Z., Lorenz, K., et al. (2018) · Journal of Pain and Symptom Management · doi:10.1016/j.jpainsymman.2017.08.025 · Cited by 354
Orbai, A.-M., de Wit, M., Mease, P., Shea, J. A., Gossec, L., Leung, Y. Y., Tillett, W., Elmamoun, M., Callis Duffin, K., Campbell, W., et al. (2017) · Annals of the Rheumatic Diseases · doi:10.1136/annrheumdis-2016-210242 · Cited by 268
Kottner, J., Jacobi, L., Hahnel, E., Alam, M., Balzer, K., Beeckman, D., Busard, C., Chalmers, J., Deckert, S., Eleftheriadou, V., et al. (2018) · British Journal of Dermatology · doi:10.1111/bjd.16324 · Cited by 40
Page, M. J., Huang, H., Verhagen, A. P., Gagnier, J. J., & Buchbinder, R. (2018) · Arthritis Care & Research · doi:10.1002/acr.23254 · Cited by 38
“Other domains that have never been measured in past trials may exist, and could be elicited from patients using qualitative methods, such as focus groups and interviews (35).”
Background
Metadata
- Authors
- Keeley, T., Williamson, P., Callery, P., Jones, L. L., Mathers, J., Jones, J., Young, B., & Calvert, M.
- Venue
- Trials
- Published
- 2016-05-04
- Volume
- 17
- Issue
- 1
- Pages
- 230-230
- ISSN
- 1745-6215 (ISSN-L)
- OpenAlex
- W2346057086
- Cited by
- 174
- References
- 7
- FWCI
- 39.06
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: The use of qualitative methods to inform Delphi surveys in core outcome set development. https://app.cochat.ai/papers/10.1186/s13063-016-1356-7
- The paper
- Keeley, T., Williamson, P., Callery, P., Jones, L. L., Mathers, J., Jones, J., Young, B., & Calvert, M. (2016). The use of qualitative methods to inform Delphi surveys in core outcome set development. Trials. https://doi.org/10.1186/s13063-016-1356-7
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.