The DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index.
The Use of the Delphi and Other Consensus Group Methods in Medical Education Research: A Review
Open accessAbstract
Purpose: Consensus group methods, such as the Delphi method and nominal group technique (NGT), are used to synthesize expert opinions when evidence is lacking. Despite their extensive use, these methods are inconsistently applied. Their use in medical education research has not been well studied. The authors set out to describe the use of consensus methods in medical education research and to assess the reporting quality of these methods and results. Method: Using scoping review methods, the authors searched the Medline, Embase, PsycInfo, PubMed, Scopus, and ERIC databases for 2009–2016. Full-text articles that focused on medical education and the keywords Delphi, RAND, NGT, or other consensus group methods were included. A standardized extraction form was used to collect article demographic data and features reflecting methodological rigor. Results: Of the articles reviewed, 257 met the inclusion criteria. The Modified Delphi (105/257; 40.8%), Delphi (91/257; 35.4%), and NGT (23/257; 8.9%) methods were most often used. The most common study purpose was curriculum development or reform (68/257; 26.5%), assessment tool development (55/257; 21.4%), and defining competencies (43/257; 16.7%). The reporting quality varied, with 70.0% (180/257) of articles reporting a literature review, 27.2% (70/257) reporting what background information was provided to participants, 66.1% (170/257) describing the number of participants, 40.1% (103/257) reporting if private decisions were collected, 37.7% (97/257) reporting if formal feedback of group ratings was shared, and 43.2% (111/257) defining consensus a priori. Conclusions: Consensus methods are poorly standardized and inconsistently used in medical education research. Improved criteria for reporting are needed.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Scoping studies: towards a methodological framework
Arksey, H., & O'Malley, L. (2005) · International Journal of Social Research Methodology · doi:10.1080/1364557032000119616 · Cited by 36,742
Scoping studies: advancing the methodology
Levac, D., Colquhoun, H., & O'Brien, K. K. (2010) · Implementation Science · doi:10.1186/1748-5908-5-69 · Cited by 15,268
Diamond, I. R., Grant, R. C., Feldman, B. M., Pencharz, P. B., Ling, S. C., Moore, A. M., & Wales, P. W. (2014) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2013.12.002 · Cited by 3,740
Boulkedid, R., Abdoul, H., Loustau, M., Sibony, O., & Alberti, C. (2011) · PLoS ONE · doi:10.1371/journal.pone.0020476 · Cited by 2,045
Sinha, I. P., Smyth, R. L., & Williamson, P. R. (2011) · PLoS Medicine · doi:10.1371/journal.pmed.1000393 · Cited by 891
Cited by
Malaisrie, S. C., Szeto, W. Y., Halas, M., Girardi, L. N., Coselli, J. S., Sundt, T. M., Chen, E. P., Fischbein, M. P., Gleason, T. G., Okita, Y., et al. (2021) · The Journal of Thoracic and Cardiovascular Surgery · doi:10.1016/j.jtcvs.2021.04.053 · Cited by 348
Artificial Intelligence in Undergraduate Medical Education: A Scoping Review
Lee, J., Wu, A. S., Li, D., & Kulasegaram, K. (. (2021) · Academic Medicine · doi:10.1097/acm.0000000000004291 · Cited by 324
Reporting guidelines for Delphi techniques in health sciences: A methodological review
Spranger, J., Homberg, A., Sonnberger, M., & Niederberger, M. (2022) · Zeitschrift für Evidenz, Fortbildung und Qualität im Gesundheitswesen · doi:10.1016/j.zefq.2022.04.025 · Cited by 262
How to use the Delphi method to aid in decision making and build consensus in pharmacy education
Olsen, A. A., Wolcott, M. D., Haines, S. T., Janke, K. K., & McLaughlin, J. E. (2021) · Currents in Pharmacy Teaching and Learning · doi:10.1016/j.cptl.2021.07.018 · Cited by 115
Vadiveloo, M., Lichtenstein, A. H., Anderson, C., Aspry, K., Foraker, R., Griggs, S., Hayman, L. L., Johnston, E., Stone, N. J., Thorndike, A. N., et al. (2020) · Circulation: Cardiovascular Quality and Outcomes · doi:10.1161/hcq.0000000000000094 · Cited by 112
Potential of large language models in health care: Delphi study
Denecke, K., May, R., LLMHealthGroup, & Rivera Romero, O. (2024) · Journal of Medical Internet Research · doi:10.2196/52399 · Cited by 111
Lee, Y.-M., Kim, S., Lee, Y.-H., Kim, H.-S., Seo, S. W., Kim, H., & Kim, K. J. (2024) · Academic Medicine · doi:10.1097/acm.0000000000005618 · Cited by 60
Using the Delphi Method to Establish Expert Consensus: A Practical Guide
Jorm, A. (2025) · Advancing Methods for Interdisciplinarity in the Social Sciences · doi:10.1007/978-981-96-8357-4 · Cited by 22
Arribas, C. M., Arcos, R., & Gertrudix, M. (2025) · Cogent Social Sciences · doi:10.1080/23311886.2025.2501759 · Cited by 9
Expert Consensus Guidelines for Assessing Students on the Social Determinants of Health
Mangold, K. A., Williams, A.-L., Ngongo, W., Liveris, M., Caruso Brown, A. E., Adler, M. D., & Campbell, M. (2023) · Teaching and Learning in Medicine · doi:10.1080/10401334.2022.2045490 · Cited by 9
Metadata
- Authors
- Humphrey-Murto, S., Varpio, L., Wood, T. J., Gonsalves, C., Ufholz, L.-A., Mascioli, K., Wang, C., & Foth, T.
- Venue
- Academic Medicine
- Publisher
- Oxford University Press (OUP)
- Published
- 2017-10
- Volume
- 92
- Issue
- 10
- Pages
- 1491-1498
- ISSN
- 1040-2446 (ISSN-L)
- OpenAlex
- W2733328613
- Cited by
- 613
- References
- 5
- FWCI
- 67.09
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
