Reporting Bias in Diagnostic and Prognostic Studies: Time for Action
Open accessThe DOI matches its Crossref record; 8 of the 8 references it cites were found in a registry or an index.
Abstract
Truth is not only violated by falsehood; it may be equally outraged by silence. —Henri-Frédéric Amiel (1812–1881) Publication bias and selective reporting are recognized problems in the biomedical literature (1)(2)(3). Positive or favorable findings are more likely to be reported than negative or inconclusive ones. Authors are often disappointed by negative findings and use multiple comparisons and subgroup analyses to generate positive results, while giving scant attention to analyses of negative results (4). Contrary to common beliefs, studies with positive results and those with negative results have been shown to be equivalent with respect to the quality of study design and execution (5), and authors and sponsors, rather than journal editors, appear to be primarily responsible for the reporting bias (5)(6)(7). Publication of only selected results distorts the research evidence base that underpins the practice of evidence-based medicine. This practice diminishes the value of systematic reviews and metaanalyses, leads to unnecessary duplication of efforts, and prevents the derivation of unbiased estimates for the risk–benefit ratio of a drug therapy or a diagnostic evaluation. Furthermore, such dubious publication practices are inconsistent with researchers’ moral responsibilities (8). Concerns over selective reporting of clinical trial results have been growing over the past several years (9)(10)(11). Comprehensive registration of clinical trials and public disclosure of study results have been suggested as the optimal solutions for this problem. Efforts to address this issue were intensified after the New York State Attorney General filed a lawsuit against GlaxoSmithKline on June 2, 2004, alleging the withholding of information regarding the efficacy and safety of paroxetine, a selective serotonin reuptake inhibitor, in children with depression (12). Shortly thereafter, the House of Delegates of the American Medical Association called on the US Department of Health and Human Services to establish a comprehensive national registry. Furthermore, the International Committee of Medical Journal Editors (ICMJE)1 published a statement in major medical journals requiring that starting July 1, 2005, all clinical trials must be registered as a condition to be considered for publication (13). Of all available registries at the time, the ICMJE recognized onlyhttp://www.clinicaltrials.org , which was authorized by the FDA Modernization Act of 1997 and sponsored by the US National Library of Medicine, as a registry that fulfilled the required criteria. Although the ICMJE establishes policy only for its 12 member journals (see http://www.icmje.org for a detailed description of the ICMJE and its purpose), the number of registered clinical trials increased suddenly by 67% after implementation of the ICMJE policy, from 15 000 to 25 000 (14). As of March 2008, approximately 52 000 clinical trials have been registered on this site. Although attention has been focused on clinical trials, selective publishing has also been reported as a problem in observational and laboratory-based investigations (4)(5)(6). In fact, reporting bias has been greater in observational and laboratory-based experimental studies than in randomized clinical trials (5). Unfortunately, no means are currently available to determine the real extent of this problem in diagnostic and prognostic studies, and such assessment is urgently needed (3)(15). While the problem of reporting bias was being addressed by medical organizations and journals, the clinical laboratory community was silently watching. We believe that it is time to act. It is imperative to create whatever measures are necessary to achieve the transparency in reporting of diagnostic and prognostic studies that the scientific community, the patients we serve, and the general public expect of our profession. The road map devised by our colleagues to tackle this issue for clinical trials can be used as a guide. We believe that only with the collaborative involvement of authors, sponsors, reviewers, journal editors, scientific professional organizations, and federal agencies can this issue be adequately addressed. We propose the following actions: (1) The scientific and medical communities should move away from the harmful notion that studies can usefully be classified as “positive” or “negative” in relation to the statistical significance of the results. Such terms cause much confusion, and indeed harm, through this bias that is introduced into the published literature. (2) Authors should recognize that “negative” results from well-conducted studies are as important to report as “positive” or “favorable” ones. A retrospective survey of 487 research projects approved by the Central Oxford Research Ethics Committee and conducted over a 4-year period showed that the quality of design and execution for positive and negative studies were equivalent (5). Furthermore, both positive and negative findi
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Publication bias in clinical research
Easterbrook, P. J., Gopalan, R., Berlin, J. A., & Matthews, D. R. (1991) · The Lancet · doi:10.1016/0140-6736(91)90201-y · Cited by 3,109 · cited 1×
“(5) Once a registry is identified, the editors of the major journals in laboratory medicine should follow the lead of the ICMJE and require that diagnostic and prognostic studies be registered ab initio as a condition for consideration for publication.”
BackgroundDeeks, J. J., Macaskill, P., & Irwig, L. (2005) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2005.01.016 · Cited by 2,910
Selective publication of antidepressant trials and its influence on apparent efficacy
Turner, E. H., Matthews, A. M., Linardatos, E., Tell, R. A., & Rosenthal, R. (2008) · New England Journal of Medicine · doi:10.1056/nejmsa065779 · Cited by 2,435
Chan, A.-W., Hróbjartsson, A., Haahr, M. T., Gøtzsche, P. C., & Altman, D. G. (2004) · JAMA · doi:10.1001/jama.291.20.2457 · Cited by 1,845
Self-citationClinical Trial Registration: A Statement from the International Committee of Medical Journal Editors
De Angelis, C., Drazen, J. M., Frizelle, F. A., Haug, C., Hoey, J., Horton, R., Kotzin, S., Laine, C., Marusic, A., Overbeke, A. J. P. M., et al. (2004) · New England Journal of Medicine · doi:10.1056/nejme048225 · Cited by 1,241
Constraints on Publication Rights in Industry-Initiated Clinical Trials
Gøtzsche, P. C., Hróbjartsson, A., Johansen, H. K., Haahr, M. T., Altman, D. G., & Chan, A.-W. (2006) · JAMA · doi:10.1001/jama.295.14.1645 · Cited by 81
Self-citationReporting research results: a moral obligation for all researchers.
Moher, D. (2007) · Canadian Journal of Anesthesia/Journal canadien d anesthésie · doi:10.1007/bf03022653 · Cited by 60
Assessing the quality of glucose monitor studies: a critical evaluation of published reports
Mahoney, J., & Ellison, J. (2007) · Clinical Chemistry · doi:10.1373/clinchem.2006.083493 · Cited by 48
Cited by
STARD 2015 guidelines for reporting diagnostic accuracy studies: explanation and elaboration
Cohen, J. F., Korevaar, D. A., Altman, D. G., Bruns, D. E., Gatsonis, C. A., Hooft, L., Irwig, L., Levine, D., Reitsma, J. B., de Vet, H. C. W., et al. (2016) · BMJ Open · doi:10.1136/bmjopen-2016-012799 · Cited by 2,570
Self-citationReporting Recommendations for Tumor Marker Prognostic Studies (REMARK): Explanation and Elaboration
Altman, D. G., McShane, L. M., Sauerbrei, W., & Taube, S. E. (2012) · PLoS Medicine · doi:10.1371/journal.pmed.1001216 · Cited by 1,013
Self-citationPrognosis Research Strategy (PROGRESS) 2: Prognostic Factor Research
Riley, R. D., Hayden, J. A., Steyerberg, E. W., Moons, K. G. M., Abrams, K., Kyzas, P. A., Malats, N., Briggs, A., Schroter, S., Altman, D. G., et al. (2013) · PLoS Medicine · doi:10.1371/journal.pmed.1001380 · Cited by 823
“Further, an increasing body of evidence has highlighted severe limitations [3,10–12].”
BackgroundSelf-citationSalameh, J.-P., Bossuyt, P. M., McGrath, T. A., Thombs, B. D., Hyde, C. J., Macaskill, P., Deeks, J. J., Leeflang, M., Korevaar, D. A., Whiting, P., et al. (2020) · BMJ · doi:10.1136/bmj.m2632 · Cited by 719
Self-citationSandroni, C., Cariou, A., Cavallaro, F., Cronberg, T., Friberg, H., Hoedemaekers, C., Horn, J., Nolan, J. P., Rossetti, A. O., & Soar, J. (2014) · Intensive Care Medicine · doi:10.1007/s00134-014-3470-x · Cited by 609
Metadata
- Authors
- Rifai, N., Altman, D. G., & Bossuyt, P. M.
- Venue
- Clinical Chemistry
- Publisher
- Oxford University Press (OUP)
- Published
- 2008-07-01
- Volume
- 54
- Issue
- 7
- Pages
- 1101-1103
- ISSN
- 0009-9147 (ISSN-L)
- 1530-8561
- OpenAlex
- W2053537559
- Cited by
- 57
- References
- 8
- FWCI
- 3.75
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.