Publication bias in qualitative research: what becomes of qualitative research presented at conferences?
The DOI matches its Crossref record; 10 of the 10 references it cites were found in a registry or an index.
Abstract
Background: Less than half of studies presented at conferences remain unpublished two years later, and these studies differ systematically from those that are published. In particular, the unpublished studies are less likely to report statistically significant findings, and this introduces publication bias. This has been well documented for quantitative studies, but has never been explored in relation to qualitative research. Methods: We reviewed the abstracts of qualitative research presented at the 1998 (n = 110) and 1999 (n = 114) British Sociological Association (BSA) Medical Sociology meetings, and attempted to locate those studies in databases or by contacting authors. We also appraised the quality of reporting in each abstract. Results: We found an overall publication rate for these qualitative studies of 44.2%. This is nearly identical to the publication rate for quantitative research. The quality of reporting of study methods and findings in the abstract was positively related to the likelihood of publication. Conclusion: Qualitative research is as likely to remain unpublished as quantitative research. Moreover, non-publication appears to be related to the quality of reporting of methodological information in the original abstract, perhaps because this is a proxy for a study with clear objectives and clear findings. This suggests a mechanism by which “qualitative publication bias” might work: qualitative studies that do not show clear, or striking, or easily described findings may simply disappear from view. One implication of this is that, as with quantitative research, systematic reviews of qualitative studies may be biased if they rely only on published papers.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Systematic Reviews in the Social Sciences: A Practical Guide
Petticrew, M., & Roberts, H. (2006) · doi:10.1002/9780470754887 · Cited by 3,586
Self-citationPublication Bias in Meta‐Analysis: Prevention, Assessment and Adjustments
(2005) · doi:10.1002/0470870168 · Cited by 2,368
Dixon-Woods, M., Cavers, D., Agarwal, S., Annandale, E., Arthur, A., Harvey, J., Hsu, R., Katbamna, S., Olsen, R., Smith, L., et al. (2006) · BMC Medical Research Methodology · doi:10.1186/1471-2288-6-35 · Cited by 2,289
Investigating and dealing with publication and other biases in meta-analysis
Sterne, J. A. C., Egger, M., & Smith, G. D. (2001) · BMJ · doi:10.1136/bmj.323.7304.101 · Cited by 2,225
Clinical Trial Registration: A Statement from the International Committee of Medical Journal Editors
De Angelis, C., Drazen, J. M., Frizelle, F. A., Haug, C., Hoey, J., Horton, R., Kotzin, S., Laine, C., Marusic, A., Overbeke, A. J. P. M., et al. (2004) · New England Journal of Medicine · doi:10.1056/nejme048225 · Cited by 1,241
Full publication of results initially presented in abstracts.
Scherer, R. W., Langenberg, P., & von Elm, E. (2007) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000005.pub3 · Cited by 997
Resisting medicines: a synthesis of qualitative studies of medicine taking
Pound, P., Britten, N., Morgan, M., Yardley, L., Pope, C., Daker-White, G., & Campbell, R. (2005) · Social Science & Medicine · doi:10.1016/j.socscimed.2004.11.063 · Cited by 868
Publication Bias in Meta‐Analysis
Rothstein, H. R., Sutton, A. J., & Borenstein, M. (2005) · Publication Bias in Meta‐Analysis · doi:10.1002/0470870168.ch1 · Cited by 603
Integrating qualitative research with trials in systematic reviews
Thomas, J., Harden, A., Oakley, A., Oliver, S., Sutcliffe, K., Rees, R., Brunton, G., & Kavanagh, J. (2004) · BMJ · doi:10.1136/bmj.328.7446.1010 · Cited by 515
Publication Bias in Editorial Decision Making
Olson, C. M. (2002) · JAMA · doi:10.1001/jama.287.21.2825 · Cited by 395
Cited by
Full publication of results initially presented in abstracts.
Scherer, R. W., Langenberg, P., & von Elm, E. (2007) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000005.pub3 · Cited by 997
Lewin, S., Glenton, C., Munthe-Kaas, H., Carlsen, B., Colvin, C. J., Gülmezoglu, M., Noyes, J., Booth, A., Garside, R., & Rashidian, A. (2015) · PLOS Medicine · doi:10.1371/journal.pmed.1001895 · Cited by 866
The past, present and future of Registered Reports
Chambers, C. D., & Tzavella, L. (2021) · Nature Human Behaviour · doi:10.1038/s41562-021-01193-7 · Cited by 499
Full publication of results initially presented in abstracts
Scherer, R. W., Meerpohl, J. J., Pfeifer, N., Schmucker, C., Schwarzer, G., & von Elm, E. (2018) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000005.pub4 · Cited by 307
Page, M. J., Sterne, J. A. C., Higgins, J. P. T., & Egger, M. (2021) · Research Synthesis Methods · doi:10.1002/jrsm.1468 · Cited by 294
Smith, J., & Noble, H. (2014) · Evidence Based Nursing · doi:10.1136/eb-2014-101946 · Cited by 293
Hughes-Morley, A., Young, B., Waheed, W., Small, N., & Bower, P. (2015) · Journal of Affective Disorders · doi:10.1016/j.jad.2014.10.005 · Cited by 132
“Publication bias also exists in qualitative research (Petticrew et al., 2008), so our exclusion of grey literature may have resulted in bias.”
BackgroundShuval, K., Harker, K., Roudsari, B., Groce, N. E., Mills, B., Siddiqi, Z., & Shachak, A. (2011) · PLoS ONE · doi:10.1371/journal.pone.0016937 · Cited by 110
Metadata
- Authors
- Petticrew, M., Egan, M., Thomson, H., Hamilton, V, Kunkler, R., & Roberts, H.
- Venue
- Journal of Epidemiology and Community Health
- Publisher
- BMJ
- Published
- 2008-05-13
- Volume
- 62
- Issue
- 6
- Pages
- 552-554
- ISSN
- 0141-7681 (ISSN-L)
- 0143-005X
- 1470-2738
- OpenAlex
- W2098734159
- Cited by
- 75
- References
- 10
- FWCI
- 0.37
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.