Preregistering Qualitative Research: A Delphi Study
Open accessThe DOI matches its Crossref record; 20 of the 20 references it cites were found in a registry or an index.
Abstract
Preregistrations—records made a priori about study designs and analysis plans and placed in open repositories—are thought to strengthen the credibility and transparency of research. Different authors have put forth arguments in favor of introducing this practice in qualitative research and made suggestions for what to include in a qualitative preregistration form. The goal of this study was to gauge and understand what parts of preregistration templates qualitative researchers would find helpful and informative. We used an online Delphi study design consisting of two rounds with feedback reports in between. In total, 48 researchers participated (response rate: 16%). In round 1, panelists considered 14 proposed items relevant to include in the preregistration form, but two items had relevance scores just below our predefined criterion (68%) with mixed argument and were put forth again. We combined items where possible, leading to 11 revised items. In round 2, panelists agreed on including the two remaining items. Panelists also converged on suggested terminology and elaborations, except for two terms for which they provided clear arguments. The result is an agreement-based form for the preregistration of qualitative studies that consists of 13 items. The form will be made available as a registration option on Open Science Framework (osf.io). We believe it is important to assure that the strength of qualitative research, which is its flexibility to adapt, adjust and respond, is not lost in preregistration. The preregistration should provide a systematic starting point.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Lincoln, Y. S., Guba, E. G., & Pilotta, J. J. (1985) · International Journal of Intercultural Relations · doi:10.1016/0147-1767(85)90062-8 · Cited by 45,787
Tong, A., Sainsbury, P., & Craig, J. (2007) · International Journal for Quality in Health Care · doi:10.1093/intqhc/mzm042 · Cited by 42,509
The qualitative content analysis process
Elo, S., & Kyngäs, H. (2008) · Journal of Advanced Nursing · doi:10.1111/j.1365-2648.2007.04569.x · Cited by 22,617
Standards for Reporting Qualitative Research: A Synthesis of Recommendations
O’Brien, B. C., Harris, I. B., Beckman, T. J., Reed, D. A., & Cook, D. A. (2014) · Academic Medicine · doi:10.1097/acm.0000000000000388 · Cited by 12,211
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505
Research guidelines for the Delphi survey technique
Hasson, F. (2000) · Journal of Advanced Nursing · doi:10.1046/j.1365-2648.2000.01567.x · Cited by 5,951 · cited 1×
“However, we only planned two rounds and it is known that response rates tend to decline after more than two rounds (Hasson et al., 2000).”
BackgroundQualitative research: standards, challenges, and guidelines
Malterud, K. (2001) · The Lancet · doi:10.1016/s0140-6736(01)05627-6 · Cited by 5,014
The Delphi method as a research tool: an example, design considerations and applications
Okoli, C., & Pawlowski, S. D. (2004) · Information & Management · doi:10.1016/j.im.2003.11.002 · Cited by 4,771
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · CHANCE · doi:10.1080/09332480.2005.10722754 · Cited by 4,702 · cited 1×
“…funders attention toward the topic (Arnold Foundation, 2020; Nosek et al., (Bonniaud et al., 2018; Glasziou et al., 2014; Humphreys et al., 2013; Ioannidis, 2005; Miguel et al., 2014; Munaf`o et al., 2017; Nosek et al., 2018; Picciotto, 2018) and developed registries based on the type of study…”
BackgroundAre We There Yet? Data Saturation in Qualitative Research
Fusch, P., & Ness, L. (2015) · The Qualitative Report · doi:10.46743/2160-3715/2015.2281 · Cited by 4,460
Diamond, I. R., Grant, R. C., Feldman, B. M., Pencharz, P. B., Ling, S. C., Moore, A. M., & Wales, P. W. (2014) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2013.12.002 · Cited by 3,740
A manifesto for reproducible science
Munafò, M. R., Nosek, B. A., Bishop, D. V. M., Button, K. S., Chambers, C. D., Percie du Sert, N., Simonsohn, U., Wagenmakers, E.-J., Ware, J. J., & Ioannidis, J. P. A. (2017) · Nature Human Behaviour · doi:10.1038/s41562-016-0021 · Cited by 3,397
Promoting an open research culture
Nosek, B. A., Alter, G., Banks, G. C., Borsboom, D., Bowman, S. D., Breckler, S. J., Buck, S., Chambers, C. D., Chin, G., Christensen, G., et al. (2015) · Science · doi:10.1126/science.aab2374 · Cited by 2,708 · cited 1×
“This is reflected in increasing journals and funders attention toward the topic (Arnold Foundation, 2020; Nosek et al., 2015; Open Science Framework, 2020).”
BackgroundLevitt, H. M., Bamberg, M., Creswell, J. W., Frost, D. M., Josselson, R., & Suárez-Orozco, C. (2018) · American Psychologist · doi:10.1037/amp0000151 · Cited by 2,286 · cited 4×
“psychology and social sciences and screened the resulting Journal Article Reporting Guidelines for Qualitative research (Levitt et al., 2018) as well as guidelines for qualitative research submitted to the Journal of the Society for Social Work and Research (Wu et al., 2016).”
MethodInfluentialCollier, D. (2011) · PS: Political Science & Politics · doi:10.1017/s1049096511001429 · Cited by 2,238 · cited 1×
“Panelists argued that these proposed items were only relevant for a subset of qualitative research that explicitly investigates hypotheses such as process tracing (Collier, 2011).”
BackgroundJünger, S., Payne, S. A., Brine, J., Radbruch, L., & Brearley, S. G. (2017) · Palliative Medicine · doi:10.1177/0269216317690685 · Cited by 2,210
Boulkedid, R., Abdoul, H., Loustau, M., Sibony, O., & Alberti, C. (2011) · PLoS ONE · doi:10.1371/journal.pone.0020476 · Cited by 2,045 · cited 2×
“Our response rate of 16% might seem too low to base any broad agreement-related conclusions on; however, Boulkedid and colleagues found that the median number of Delphi panelists was 17 (Boulkedid et al., 2011), whereas ours was nearly double.”
BackgroundThe preregistration revolution
Nosek, B. A., Ebersole, C. R., DeHaven, A. C., & Mellor, D. T. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1708274114 · Cited by 1,998 · cited 3×
“…Nosek et al., (Bonniaud et al., 2018; Glasziou et al., 2014; Humphreys et al., 2013; Ioannidis, 2005; Miguel et al., 2014; Munaf`o et al., 2017; Nosek et al., 2018; Picciotto, 2018) and developed registries based on the type of study (e.g., clinicaltrials.gov for randomized clinical trials,…”
BackgroundReducing waste from incomplete or unusable reports of biomedical research
Glasziou, P., Altman, D. G., Bossuyt, P., Boutron, I., Clarke, M., Julious, S., Michie, S., Moher, D., & Wager, E. (2014) · The Lancet · doi:10.1016/s0140-6736(13)62228-x · Cited by 1,319
Author guidelines for manuscripts reporting on qualitative research
Wu, S., Wyant, D. C., & Fraser, M. W. (2016) · Journal of the Society for Social Work and Research · doi:10.1086/685816 · Cited by 59 · cited 2×
“psychology and social sciences and screened the resulting Journal Article Reporting Guidelines for Qualitative research (Levitt et al., 2018) as well as guidelines for qualitative research submitted to the Journal of the Society for Social Work and Research (Wu et al., 2016).”
Method
Cited by
Criteria for Good Qualitative Research: A Comprehensive Review
Yadav, D. (2022) · The Asia-Pacific Education Researcher · doi:10.1007/s40299-021-00619-0 · Cited by 217
“Guided by the science”: (De)politicising the UK government’s response to the coronavirus crisis
Kettell, S., & Kerr, P. (2022) · The British Journal of Politics and International Relations · doi:10.1177/13691481211054957 · Cited by 50
Do randomised clinical trials on dental caries adopt Open Science practices?
Acosta, C. D. P., Cenci, J., Brondani, B., Elagami, R. A., Pereira-Cenci, T., Cenci, M. S., Huysmans, M.-C. D. N. J. M., Raggio, D. P., Braga, M. M., & Mendes, F. M. (2024) · BMC Oral Health · doi:10.1186/s12903-024-05218-1 · Cited by 4
“This action may initially restrict some publications, mainly considering some study designs, such as qualitative research [36].”
Preregistration of Studies with Existing Data
Mertens, G., & Krypotos, A.-M. (2022) · Integrity of Scientific Research · doi:10.1007/978-3-030-99680-2_36
Metadata
- Authors
- Haven, T. L., Errington, T. M., Gleditsch, K. S., van Grootel, L., Jacobs, A. M., Kern, F. G., Piñeiro, R., Rosenblatt, F., & Mokkink, L. B.
- Venue
- International Journal of Qualitative Methods
- Publisher
- SAGE Publications
- Published
- 2020-01-01
- Volume
- 19
- ISSN
- 1609-4069 (ISSN-L)
- OpenAlex
- W3008814053
- Cited by
- 211
- References
- 20
- FWCI
- 56.76
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.