Misrepresentation and distortion of research in biomedical literature
Open accessThe DOI matches its Crossref record; 91 of the 91 references it cites were found in a registry or an index; 7 of the 96 sources its claims cite were checked for support.
Abstract
Publication in peer-reviewed journals is an essential step in the scientific process. However, publication is not simply the reporting of facts arising from a straightforward analysis thereof. Authors have broad latitude when writing their reports and may be tempted to consciously or unconsciously “spin” their study findings. Spin has been defined as a specific intentional or unintentional reporting that fails to faithfully reflect the nature and range of findings and that could affect the impression the results produce in readers. This article, based on a literature review, reports the various practices of spin from misreporting by “beautification” of methods to misreporting by misinterpreting the results. It provides data on the prevalence of some forms of spin in specific fields and the possible effects of some types of spin on readers’ interpretation and research dissemination. We also discuss why researchers would spin their reports and possible ways to avoid it.
Overview
CoChat Summary
TL;DR
This review/perspective describes “spin” in biomedical literature—misreporting methods, results, interpretation, and rhetoric—and summarizes evidence on its prevalence and impact. It argues spin is a detrimental research practice and proposes protocol registration, reporting guidelines, editorial/peer-review changes, and reward-system reform to reduce it.
Key findings
- Spin is defined as reporting that fails to faithfully reflect the nature and range of findings and can affect readers’ impressions; practices include misreporting methods, misreporting results, misinterpretation, and rhetoric. For example, a survey of 2,235 reports of RCTs in Chinese medical journals found only about 7% met methodological criteria for authentic randomization, while 93% were falsely reported as RCTs.
- Prevalence varies by field: in two-arm parallel-group RCTs with nonstatistically significant primary outcomes, more than half of abstract conclusions contained spin; in observational studies evaluating an intervention, spin was identified in more than 80% of abstract conclusions, most frequently causal language; diagnosis-accuracy studies showed spin in one-third of articles in high-impact journals; HIV noninferiority trials showed spin in two-thirds of inconclusive studies; and more than half of molecular diagnostic test reports overinterpreted clinical applicability.
- Impact evidence includes an RCT of 300 corresponding authors/investigators: after reading an abstract with spin, readers were more likely to believe the treatment would be beneficial (mean difference 0.71, 95% CI 0.07–1.35, P = 0.030). Peer reviewers failed to identify spin in the abstract’s conclusion in 76% of reports, and media coverage of the statin controversy was associated with an 11% and 12% increase in the likelihood of existing users stopping treatment for primary and secondary prevention, respectively, potentially more than 2,000 extra cardiovascular events across the United Kingdom over 10 y.
- Other reported numbers: positive words in PubMed abstracts increased 880% from 2% in 1974–1980 to 17.5% in 2014; an assessment of 20,642 published articles in 40 journals over 20 y found 3.8% contained questionable images, with the number increasing; editors estimated about 20% of accepted papers contained at least one figure not complying with accepted practices.
Methods
- Design: literature review/perspective. The authors systematically searched MEDLINE via PubMed for articles on spin with an abstract, searching the entire database from 1966, with search date May 23, 2017; they also searched Google Scholar for articles citing key articles in the field.
- Sample: the search retrieved 592 citations, of which 49 were relevant; the authors also relied on a personal collection of articles meeting eligibility criteria. One researcher screened titles and abstracts, retrieved full texts when appropriate, and extracted information on type of spin, prevalence, associated factors, impact on readers’ interpretation, and possible ways to reduce spin.
- Analysis: narrative synthesis and classification of spin practices under misreporting methods, misreporting results, misinterpretation, and other types of spin, with reported prevalence and impact from included studies; scope was limited to biomedicine.
Limitations
- The paper states that only a single researcher screened citations, abstracts, and full texts, so the authors cannot rule out missing some relevant reports.
- The paper states that it reflects the authors’ knowledge and opinion informed by a literature review, and that its scope is limited to biomedicine; it also notes no systematic assessment of spin in systematic reviews and meta-analyses was identified, and no data exist on the prevalence of researchers using spin (though the authors suspect it is common).
- Evident from the text: the search strategy relied on title terms and “has abstract,” plus Google Scholar citation searching, and the paper does not report a formal quantitative synthesis or risk-of-bias assessment.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
6 Valid · 2 Not valid · 61 Not checked · 28 Own results and interpretations
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“For example, the reporting and interpretation of the risk of all-cause mortality in the DAPT (dual antiplatelet therapy) study randomizing 9,961 patients to continue DAPT beyond 1 y after stent placement or receive a placebo for 30 mo, raised some concerns (24, 25).”
What the sources say
- Not checked yetcites [24] Mauri L, et al. DAPT Study Investigators Twelve or 30 months of dual antiplatelet therapy after drug-eluting stents. N Engl J Med. 2014;371:2155–2166. doi: 10.1056/NEJMoa1409312. [DOI] [PMC free article] [PubMed] [Google Scholar] source
- Supportedcites [25] Sharp M, Haneef R, Ravaud P, Boutron I. Dissemination of 2014 dual antiplatelet therapy (DAPT) trial results: A systematic review of scholarly and media attention over 7 months. BMJ Open. 2017;7:e014503. doi: 10.1136/bmjopen-2016-014503. [DOI] [PMC free article] [PubMed] [Google Scholar] source
The Introduction says the DAPT trial’s reporting raised concerns because its abstract omitted the increased mortality risk and its discussion used questionable post hoc analyses to interpret it. The same section identifies the trial as randomizing 9,961 patients to continued treatment or placebo for 30 months after the first year. (LLM, conf=0.94)
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“A comparison of outcomes reported in protocols of RCTs submitted to ethics committees or registered in trial registries showed that scientists selectively report statistically significant outcomes (16–18).”
What the sources say
- Not checked yetcites [16] Chan AW, Hróbjartsson A, Haahr MT, Gøtzsche PC, Altman DG. Empirical evidence for selective reporting of outcomes in randomized trials: Comparison of protocols to published articles. JAMA. 2004;291:2457–2465. doi: 10.1001/jama.291.20.2457. [DOI] [PubMed] [Google Scholar] source
- Not checked yetcites [17] Mathieu S, Boutron I, Moher D, Altman DG, Ravaud P. Comparison of registered and published primary outcomes in randomized controlled trials. JAMA. 2009;302:977–984. doi: 10.1001/jama.2009.1242. [DOI] [PubMed] [Google Scholar] source
- Partly supportedcites [18] Dwan K, et al. Evidence for the selective reporting of analyses and discrepancies in clinical trials: A systematic review of cohort studies of clinical trials. PLoS Med. 2014;11:e1001666. doi: 10.1371/journal.pmed.1001666. [DOI] [PMC free article] [PubMed] [Google Scholar] sourceExperimental — accuracy not yet measured
Our copy of Dwan et al. (2014) does contain the substance of the sentence only as background in the Introduction, where 'A recent review showed that statistically significant outcomes were more likely to be fully reported than non-significant outcomes (range of odds ratios: 2.2 to 4.7)' is credited to reference [2] (Dwan, Gamble, Williamson, Kirkham 2013, PLoS ONE), i.e. an earlier review of publication/outcome reporting bias, not to this paper's own results. This paper's own scope is selective reporting of ANALYSES and discrepancies across source documents (Table 1 lists included cohorts whose protocols came from ethics committees, e.g. Chan 2008 Copenhagen and Frederiksberg, Hahn 2002, or trial registry entries), and its Discussion/Conclusions state the opposite of a demonstrated selective reporting of significant outcomes: 'There are to date no readily accessible data on selective reporting bias of analyses in cohorts of RCTs' and 'there is insufficient information to distinguish between bias, error, and legitimate changes'. So the protocol/registry-comparison framing and the ethics-committee cohorts fit this source, but the causal conclusion that scientists selectively report statistically significant outcomes is not this paper's finding and is hedged here as possibly bias, error or legitimate protocol departure. (LLM, conf=0.72)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Every source the paper cites for this claim was checked and supports it.
In the paper
“For example, in the DAPT study, the authors proposed a post hoc explanation for the increased rate of death in the experimental group based on a post hoc analysis to mitigate the role of prolonged treatment on this increased risk of mortality (25).”
What the sources say
- Supportedcites [25] Sharp M, Haneef R, Ravaud P, Boutron I. Dissemination of 2014 dual antiplatelet therapy (DAPT) trial results: A systematic review of scholarly and media attention over 7 months. BMJ Open. 2017;7:e014503. doi: 10.1136/bmjopen-2016-014503. [DOI] [PMC free article] [PubMed] [Google Scholar] source
The Introduction reports increased all-cause mortality with continued treatment and says the DAPT study authors used post hoc analyses to explain that increase and clear prolonged treatment of responsibility. It specifically describes a post hoc analysis excluding deaths potentially related to cancer diagnosed before enrolment. (LLM, conf=0.94)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Other methodological systematic reviews focusing on two-arm parallel-group RCTs with nonstatistically significant primary outcomes in specific medical fields found consistent results (48–54).”
What the sources say
- Not checked yetcites [48] Vera-Badillo FE, Shapiro R, Ocana A, Amir E, Tannock IF. Bias in reporting of end points of efficacy and toxicity in randomized, clinical trials for women with breast cancer. Ann Oncol. 2013;24:1238–1244. doi: 10.1093/annonc/mds636. [DOI] [PubMed] [Google Scholar] source
- Not checked yetcites [49] Amos AJ. A review of spin and bias use in the early intervention in psychosis literature. Prim Care Companion CNS Disord. 2014;16:PCC.13r01586. doi: 10.4088/PCC.13r01586. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Not checked yetcites [50] Arunachalam L, Hunter IA, Killeen S. Reporting of randomized controlled trials with statistically nonsignificant primary outcomes published in high-impact surgical journals. Ann Surg. 2017;265:1141–1145. doi: 10.1097/SLA.0000000000001795. [DOI] [PubMed] [Google Scholar] source
- Not checked yetcites [51] Gewandter JS, et al. Data interpretation in analgesic clinical trials with statistically nonsignificant primary analyses: An ACTTION systematic review. J Pain. 2015;16:3–10. doi: 10.1016/j.jpain.2014.10.003. [DOI] [PubMed] [Google Scholar] source
- Partly supportedcites [52] Le Fourn E, Giraudeau B, Chosidow O, Doutre MS, Lorette G. Study design and quality of reporting of randomized controlled trials of chronic idiopathic or autoimmune urticaria: Review. PLoS One. 2013;8:e70717. doi: 10.1371/journal.pone.0070717. [DOI] [PMC free article] [PubMed] [Google Scholar] sourceExperimental — accuracy not yet measured
The Results (Spin Strategy in Interpretation of Results; page not provided) report spin in 10 of 12 trials with nonsignificant primary outcomes, consistent with the claim's reference to similar findings in a specific medical field. But the review did not focus exclusively on two-arm parallel-group trials or nonsignificant trials: the Results (Design of the Studies; page not provided) include three cross-over trials and 12 trials with three or four arms, and the review assessed 52 trials overall. (LLM, conf=0.88)
- Supportedcites [53] Lockyer S, Hodgson R, Dumville JC, Cullum N. “Spin” in wound care research: The reporting and interpretation of randomized controlled trials with statistically non-significant primary outcome results or unspecified primary outcomes. Trials. 2013;14:371. doi: 10.1186/1745-6215-14-371. [DOI] [PMC free article] [PubMed] [Google Scholar] source
The Eligibility section describes a review of parallel-group wound-care RCTs, including a cohort with statistically non-significant primary outcomes. The Results report spin in 20 of 28 trials in that cohort, and the Cohort A key findings say that spin was common, as in general medical journals. The Cohort A characteristics table lists one comparator for each of its 28 trials, consistent with two-arm trials; page numbers are unavailable in our copy. (LLM, conf=0.86)
- Not checked yetcites [54] Patel SV, Van Koughnett JA, Howe B, Wexner SD. Spin is common in studies assessing robotic colorectal surgery: An assessment of reporting and interpretation of study results. Dis Colon Rectum. 2015;58:878–884. doi: 10.1097/DCR.0000000000000425. [DOI] [PubMed] [Google Scholar] source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“A systematic methodologic review of 51 RCTs assessing complex interventions with statistically significant small effects showed that authors exercised no caution in their interpretation of results in about half of the reports (33).”
What the sources say
- Not checked yetcites [33] Siontis GC, Ioannidis JP. Risk factors and interventions with statistically significant tiny effects. Int J Epidemiol. 2011;40:1292–1307. doi: 10.1093/ije/dyr099. [DOI] [PubMed] [Google Scholar] source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
43 more claims
Create a free account to see all 97 claims and to verify the 89 references still open.
- Probably not a substantial claim. 36
- It may state two separate claims as one. 28
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Studies of academic writing use the term 'spin' to describe authors' attempt to present a more positive description of a technique or drug in health sciences than the data actually support (Boutron et al., 2014; Boutron & Ravaud, 2018).”
From Tortured phrases: A dubious writing style emerging in science. Evidence of critical issues affecting established journals (2021) · cites this paper as Boutron & Ravaud, 2018
Experimental — accuracy not yet measured
Citations
Cites
Cook, R. J., Dickens, B. M., & Fathalla, M. F. (2003) · Reproductive Health and Human Rights · doi:10.1093/acprof:oso/9780199241323.003.0025 · Cited by 10,769
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505
Moher, D., Hopewell, S., Schulz, K. F., Montori, V., Gotzsche, P. C., Devereaux, P. J., Elbourne, D., Egger, M., & Altman, D. G. (2010) · BMJ · doi:10.1136/bmj.c869 · Cited by 8,883
Improving Bioscience Research Reporting: The ARRIVE Guidelines for Reporting Animal Research
Kilkenny, C., Browne, W. J., Cuthill, I. C., Emerson, M., & Altman, D. G. (2010) · PLoS Biology · doi:10.1371/journal.pbio.1000412 · Cited by 7,300
The ASA Statement on p -Values: Context, Process, and Purpose
Wasserstein, R. L., & Lazar, N. A. (2016) · The American Statistician · doi:10.1080/00031305.2016.1154108 · Cited by 6,306 · cited 1×
“In 2016, the American Statistical Association released a statement on statistical significance and the P value with six principles underlying the proper use and interpretation of this statistical tool (77).”
BackgroundImproving bioscience research reporting: The ARRIVE guidelines for reporting animal research
Kilkenny, C., Browne, W. J., Cuthill, I. C., Emerson, M., & Altman, D. G. (2010) · Journal of Pharmacology and Pharmacotherapeutics · doi:10.4103/0976-500x.72351 · Cited by 6,277
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · CHANCE · doi:10.1080/09332480.2005.10722754 · Cited by 4,702
A manifesto for reproducible science
Munafò, M. R., Nosek, B. A., Bishop, D. V. M., Button, K. S., Chambers, C. D., Percie du Sert, N., Simonsohn, U., Wagenmakers, E.-J., Ware, J. J., & Ioannidis, J. P. A. (2017) · Nature Human Behaviour · doi:10.1038/s41562-016-0021 · Cited by 3,397
Selective publication of antidepressant trials and its influence on apparent efficacy
Turner, E. H., Matthews, A. M., Linardatos, E., Tell, R. A., & Rosenthal, R. (2008) · New England Journal of Medicine · doi:10.1056/nejmsa065779 · Cited by 2,435
Measuring the Prevalence of Questionable Research Practices With Incentives for Truth Telling
John, L. K., Loewenstein, G., & Prelec, D. (2012) · Psychological Science · doi:10.1177/0956797611430953 · Cited by 2,239
HARKing: Hypothesizing after the results are known
Kerr, N. L. (1998) · Personality and Social Psychology Review · doi:10.1207/s15327957pspr0203_4 · Cited by 1,989
Twelve or 30 Months of Dual Antiplatelet Therapy after Drug-Eluting Stents
Mauri, L., Kereiakes, D. J., Yeh, R. W., Driscoll-Shempp, P., Cutlip, D. E., Steg, P. G., Normand, S.-L. T., Braunwald, E., Wiviott, S. D., Cohen, D. J., et al. (2014) · New England Journal of Medicine · doi:10.1056/nejmoa1409312 · Cited by 1,928
Chan, A.-W., Hróbjartsson, A., Haahr, M. T., Gøtzsche, P. C., & Altman, D. G. (2004) · JAMA · doi:10.1001/jama.291.20.2457 · Cited by 1,845
The Extent and Consequences of P-Hacking in Science
Head, M. L., Holman, L., Lanfear, R., Kahn, A. T., & Jennions, M. D. (2015) · PLOS Biology · doi:10.1371/journal.pbio.1002106 · Cited by 1,388 · cited 2×
“P-hacking, a detrimental practice, is defined as the misreporting of true effect sizes in published reports after researchers perform several statistical analyses and selectively choose to report or focus on those that produce statistically significant results (20).”
BackgroundPepe, M. S. (2004) · American Journal of Epidemiology · doi:10.1093/aje/kwh101 · Cited by 1,276
Negative results are disappearing from most disciplines and countries
Fanelli, D. (2012) · Scientometrics · doi:10.1007/s11192-011-0494-7 · Cited by 1,248
Boutron, I. (2010) · JAMA · doi:10.1001/jama.2010.651 · Cited by 842 · cited 4×
“In this study, spin was defined as “specific reporting strategies, whatever their motive, to highlight that the experimental treatment is beneficial, despite a statistically nonsignificant difference for the primary outcome, or to distract the reader from statistically nonsignificant results” (4).”
BackgroundInfluentialSelf-citationRescuing US biomedical research from its systemic flaws
Alberts, B., Kirschner, M. W., Tilghman, S., & Varmus, H. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1404402111 · Cited by 817 · cited 1×
“They are particularly questioning the expectation that this enterprise should continue expanding (83, 84).”
BackgroundBeswick, A. D., Rees, K., Dieppe, P., Ayis, S., Gooberman-Hill, R., Horwood, J., & Ebrahim, S. (2008) · The Lancet · doi:10.1016/s0140-6736(08)60342-6 · Cited by 780
Comparison of Registered and Published Primary Outcomes in Randomized Controlled Trials
Mathieu, S. (2009) · JAMA · doi:10.1001/jama.2009.1242 · Cited by 730
Kidwell, M. C., Lazarević, L. B., Baranski, E., Hardwicke, T. E., Piechowski, S., Falkenberg, L.-S., Kennett, C., Slowik, A., Sonnleitner, C., Hess-Holden, C., et al. (2016) · PLOS Biology · doi:10.1371/journal.pbio.1002456 · Cited by 700 · cited 1×
“The use of a badge to acknowledge open practices has been effective in changing researchers’ behavior (86).”
BackgroundEdwards, M. A., & Roy, S. (2017) · Environmental Engineering Science · doi:10.1089/ees.2016.0223 · Cited by 688
Lithium delays progression of amyotrophic lateral sclerosis
Fornai, F., Longone, P., Cafaro, L., Kastsiuchenka, O., Ferrucci, M., Manca, M. L., Lazzeri, G., Spalloni, A., Bellio, N., Lenzi, P., et al. (2008) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0708022105 · Cited by 550
Grant, R. L. (2014) · BMJ · doi:10.1136/bmj.f7450 · Cited by 515 · cited 1×
“Similarly, reporting odds ratios (ORs) instead of risk ratios (RRs) when the baseline risk is high can easily be misinterpreted (26).”
BackgroundIncreasing value and reducing waste in biomedical research: who's listening?
Moher, D., Glasziou, P., Chalmers, I., Nasser, M., Bossuyt, P. M. M., Korevaar, D. A., Graham, I. D., Ravaud, P., & Boutron, I. (2016) · The Lancet · doi:10.1016/s0140-6736(15)00307-4 · Cited by 462
Self-citationWhat's in a picture? The temptation of image manipulation
Rossner, M., & Yamada, K. M. (2004) · The Journal of Cell Biology · doi:10.1083/jcb.200406019 · Cited by 412
Cordoba, G., Schwartz, L., Woloshin, S., Bae, H., & Gotzsche, P. C. (2010) · BMJ · doi:10.1136/bmj.c3920 · Cited by 400
Importance of the Lay Press in the Transmission of Medical Knowledge to the Scientific Community
Phillips, D. P., Kanter, E. J., Bednarczyk, B., & Tastad, P. L. (1991) · New England Journal of Medicine · doi:10.1056/nejm199110173251620 · Cited by 399
Statistical learning and selective inference
Taylor, J., & Tibshirani, R. J. (2015) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1507583112 · Cited by 398 · cited 1×
“Cherry-picking can be particularly problematic in this era of massive observational data (23).”
BackgroundEvolution of Reporting P Values in the Biomedical Literature, 1990-2015
Chavalarias, D., Wallach, J. D., Li, A. H. T., & Ioannidis, J. P. A. (2016) · JAMA · doi:10.1001/jama.2016.1952 · Cited by 370
Boutron, I., Altman, D. G., Hopewell, S., Vera-Badillo, F., Tannock, I., & Ravaud, P. (2014) · Journal of Clinical Oncology · doi:10.1200/jco.2014.56.7503 · Cited by 368
Self-citationMultiplicity in randomised trials II: subgroup and interim analyses
Schulz, K. F., & Grimes, D. A. (2005) · The Lancet · doi:10.1016/s0140-6736(05)66516-6 · Cited by 357
How scientists fool themselves – and how they can stop
Nuzzo, R. (2015) · Nature · doi:10.1038/526182a · Cited by 296 · cited 1×
“HARKing, or hypothesizing after results are known (37), or JARKing, justifying after results are known (38), are also inappropriate practices.”
ResultYavchitz, A., Boutron, I., Bafeta, A., Marroun, I., Charles, P., Mantz, J., & Ravaud, P. (2012) · PLoS Medicine · doi:10.1371/journal.pmed.1001308 · Cited by 293 · cited 12×
“Biomedical spin was first systematically investigated with a representative sample of two-arm parallel-group RCTs with nonstatistically significant primary outcomes indexed in PubMed in 2006 (4).”
BackgroundInfluentialSelf-citationThe Prevalence of Inappropriate Image Duplication in Biomedical Research Publications
Bik, E. M., Casadevall, A., & Fang, F. C. (2016) · mBio · doi:10.1128/mbio.00809-16 · Cited by 287
Testing for the Presence of Positive-Outcome Bias in Peer Review: A Randomized Controlled Trial
Emerson, G. B., Warme, W. J., Wolf, F. M., Heckman, J. D., Brand, R. A., & Leopold, S. S. (2010) · Archives of Internal Medicine · doi:10.1001/archinternmed.2010.406 · Cited by 245 · cited 1×
“They were also more likely to detect errors in and award a low score to the methods of the negative version of the same manuscript, even though the Methods sections in both versions were identical (65).”
BackgroundVinkers, C. H., Tijdink, J. K., & Otte, W. M. (2015) · BMJ · doi:10.1136/bmj.h6467 · Cited by 237
Randomized trials published in some Chinese journals: how many are randomized?
Wu, T., Li, Y., Bian, Z., Liu, G., & Moher, D. (2009) · Trials · doi:10.1186/1745-6215-10-46 · Cited by 229
Yavchitz, A., Ravaud, P., Altman, D. G., Moher, D., Hrobjartsson, A., Lasserson, T., & Boutron, I. (2016) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2016.01.020 · Cited by 224
Self-citationMatthews, A., Herrett, E., Gasparrini, A., Van Staa, T., Goldacre, B., Smeeth, L., & Bhaskaran, K. (2016) · BMJ · doi:10.1136/bmj.i3283 · Cited by 221
Should people at low risk of cardiovascular disease take a statin?
Abramson, J. D., Rosenberg, H. G., Jewell, N., & Wright, J. M. (2013) · BMJ · doi:10.1136/bmj.f6123 · Cited by 208
Dwan, K., Altman, D. G., Clarke, M., Gamble, C., Higgins, J. P. T., Sterne, J. A. C., Williamson, P. R., & Kirkham, J. J. (2014) · PLoS Medicine · doi:10.1371/journal.pmed.1001666 · Cited by 187
Schulz, K. F., Grimes, D. A., Altman, D. G., & Hayes, R. J. (1996) · BMJ · doi:10.1136/bmj.312.7033.742 · Cited by 163
Overinterpretation and Misreporting of Diagnostic Accuracy Studies: Evidence of “Spin”
Ochodo, E. A., de Haan, M. C., Reitsma, J. B., Hooft, L., Bossuyt, P. M., & Leeflang, M. M. G. (2013) · Radiology · doi:10.1148/radiol.12120527 · Cited by 158 · cited 2×
“In diagnosis-accuracy studies, spin was identified in one-third of the articles published in high-impact factor journals (8), and in the field of molecular diagnostic tests, more than half of the reports overinterpreted the clinical applicability of the findings (39).”
BackgroundBrown, A. W., Bohan Brown, M. M., & Allison, D. B. (2013) · The American Journal of Clinical Nutrition · doi:10.3945/ajcn.113.064410 · Cited by 155
Promoting Research Integrity in a Global Environment
Mayer, T., & Steneck, N. (2011) · WORLD SCIENTIFIC eBooks · doi:10.1142/8102 · Cited by 152
Data Visualization in the Neurosciences: Overcoming the Curse of Dimensionality
Allen, E. A., Erhardt, E. B., & Calhoun, V. D. (2012) · Neuron · doi:10.1016/j.neuron.2012.05.001 · Cited by 146
Self-correction in science at work
Alberts, B., Cicerone, R. J., Fienberg, S. E., Kamb, A., McNutt, M., Nerem, R. M., Schekman, R., Shiffrin, R., Stodden, V., Suresh, S., et al. (2015) · Science · doi:10.1126/science.aab3847 · Cited by 143
Lazarus, C., Haneef, R., Ravaud, P., & Boutron, I. (2015) · BMC Medical Research Methodology · doi:10.1186/s12874-015-0079-x · Cited by 137 · cited 2×
“In observational studies evaluating an intervention, spin was identified in the abstract’s conclusions in more than 80% of reports, the most frequent type of spin being the use of causal language (10).”
BackgroundSelf-citationArrison, T., & Nerem, R. M. (2017) · Special Publications · doi:10.1002/9781119067825.ch2 · Cited by 135
Horton, R. (1995) · BMJ · doi:10.1136/bmj.310.6985.985 · Cited by 132
Policy: Twenty tips for interpreting scientific claims
Sutherland, W. J., Spiegelhalter, D., & Burgman, M. (2013) · Nature · doi:10.1038/503335a · Cited by 129 · cited 1×
“A user’s guide to detect misleading claims in clinical research reports (79) and tips for interpreting claims (6) are available, but should be more widely used.”
BackgroundSaturated fat is not the major issue
Malhotra, A. (2013) · BMJ · doi:10.1136/bmj.f6340 · Cited by 124
Vera-Badillo, F. E., Shapiro, R., Ocana, A., Amir, E., & Tannock, I. F. (2013) · Annals of Oncology · doi:10.1093/annonc/mds636 · Cited by 117
Users' guide to detecting misleading claims in clinical research reports
Montori, V. M., Jaeschke, R., Schünemann, H. J., Bhandari, M., Brozek, J. L., Devereaux, P. J., & Guyatt, G. H. (2004) · BMJ · doi:10.1136/bmj.329.7474.1093 · Cited by 106 · cited 1×
“A user’s guide to detect misleading claims in clinical research reports (79) and tips for interpreting claims (6) are available, but should be more widely used.”
Background"Spin" in scientific writing: scientific mischief and legal jeopardy.
Fletcher, R. H., & Black, B. (2007) · PubMed · Cited by 100
Assessing Value in Biomedical Research: The PQRST of Appraisal and Reward
Ioannidis, J. P. A., & Khoury, M. J. (2014) · JAMA · doi:10.1001/jama.2014.6932 · Cited by 92
A Review of Spin and Bias Use in the Early Intervention in Psychosis Literature
Amos, A. J. (2014) · The Primary Care Companion For CNS Disorders · doi:10.4088/pcc.13r01586 · Cited by 89
Risk factors and interventions with statistically significant tiny effects
Siontis, G. C., & Ioannidis, J. P. (2011) · International Journal of Epidemiology · doi:10.1093/ije/dyr099 · Cited by 80
Why do we think we know what we know? A metaknowledge analysis of the salt controversy
Trinquart, L., Johns, D. M., & Galea, S. (2016) · International Journal of Epidemiology · doi:10.1093/ije/dyv184 · Cited by 77 · cited 1×
“For example, an analysis of the patterns of knowledge generation surrounding the controversy between proponents and opponents of a population-wide reduction in salt consumption showed that reports were more likely to cite studies that had conclusions similar to rather than different from those of the author doing the…”
ResultThe effect of scientific misconduct on the results of clinical trials: A Delphi survey
Al-Marzouki, S., Roberts, I., Marshall, T., & Evans, S. (2005) · Contemporary Clinical Trials · doi:10.1016/j.cct.2005.01.011 · Cited by 70 · cited 1×
“A Delphi consensus survey of expert opinion identified some types of spin (overinterpretation of significant findings in small trials, elective reporting based on P values, and selective reporting of outcomes in the abstract) found among the questionable research practices most likely to occur (47).”
BackgroundTranslating statistical findings into plain English
Pocock, S. J., & Ware, J. H. (2009) · The Lancet · doi:10.1016/s0140-6736(09)60499-2 · Cited by 67
Pearson, H. (2005) · Nature · doi:10.1038/434952a · Cited by 67
Lockyer, S., Hodgson, R., Dumville, J. C., & Cullum, N. (2013) · Trials · doi:10.1186/1745-6215-14-371 · Cited by 66
Overinterpretation of Clinical Applicability in Molecular Diagnostic Research
Lumbreras, B., Parker, L. A., Porta, M., Pollán, M., Ioannidis, J. P. A., & Hernández-Aguado, I. (2009) · Clinical Chemistry · doi:10.1373/clinchem.2008.121517 · Cited by 65 · cited 2×
“In diagnosis-accuracy studies, spin was identified in one-third of the articles published in high-impact factor journals (8), and in the field of molecular diagnostic tests, more than half of the reports overinterpreted the clinical applicability of the findings (39).”
BackgroundLazarus, C., Haneef, R., Ravaud, P., Hopewell, S., Altman, D. G., & Boutron, I. (2016) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2016.04.012 · Cited by 58 · cited 2×
“For example, spin is frequently not detected, even by readers with a high level of expertise and awareness, such as peer-reviewers (78).”
BackgroundSelf-citationThe problems of taste in placebo matching: An evaluation of zinc gluconate for the common cold
Farr, B. M., & Gwaltney, J. M. (1987) · Journal of Chronic Diseases · doi:10.1016/0021-9681(87)90187-1 · Cited by 56 · cited 1×
“For example, some studies report “double-blind” methods, but the blinding is not credible (11, 12), or report an intent-to-treat analysis, but some patients are excluded from the analysis (13, 14).”
BackgroundArunachalam, L., Hunter, I. A., & Killeen, S. (2017) · Annals of Surgery · doi:10.1097/sla.0000000000001795 · Cited by 56
Attia, J., Ioannidis, J. P. A., Thakkinstian, A., McEvoy, M., Scott, R. J., Minelli, C., Thompson, J., Infante-Rivard, C., & Guyatt, G. (2009) · JAMA · doi:10.1001/jama.2008.993 · Cited by 55
Addressing systemic problems in the biomedical research enterprise
Alberts, B., Kirschner, M. W., Tilghman, S. M., & VARMUS, H. E. (2015) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1500969112 · Cited by 52
Translating statistical findings into plain English
Stang, A. (2009) · The Lancet · doi:10.1016/s0140-6736(09)61703-7 · Cited by 48
Gewandter, J. S., McKeown, A., McDermott, M. P., Dworkin, J. D., Smith, S. M., Gross, R. A., Hunsinger, M., Lin, A. H., Rappaport, B. A., Rice, A. S. C., et al. (2015) · The Journal of Pain · doi:10.1016/j.jpain.2014.10.003 · Cited by 47
Altwairgi, A. K., Booth, C. M., Hopman, W. M., & Baetz, T. D. (2012) · Journal of Clinical Oncology · doi:10.1200/jco.2012.41.8319 · Cited by 43
A list of highly influential biomedical researchers, 1996–2011
Boyack, K. W., Klavans, R., Sorensen, A. A., & Ioannidis, J. P. A. (2013) · European Journal of Clinical Investigation · doi:10.1111/eci.12171 · Cited by 42 · cited 1×
“Publication metrics, such as the number of publications, number of citations, journal impact factor, and h-index are used to measure academic productivity and scientists’ influence (64).”
MethodPatel, S. V., Van Koughnett, J. A. M., Howe, B., & Wexner, S. D. (2015) · Diseases of the Colon & Rectum · doi:10.1097/dcr.0000000000000425 · Cited by 39
Mathieu, S., Giraudeau, B., Soubrier, M., & Ravaud, P. (2012) · Joint Bone Spine · doi:10.1016/j.jbspin.2011.05.008 · Cited by 37
Self-citationHernandez, A. V., Pasupuleti, V., Deshpande, A., Thota, P., Collins, J. A., & Vidal, J. E. (2013) · PLoS ONE · doi:10.1371/journal.pone.0063272 · Cited by 29 · cited 1×
“One study in the field of HIV assessed the interpretation of noninferiority trial results and showed spin in two-thirds of the studies with inconclusive results (55).”
Randomized Trials Stopped Early for Benefit: A Systematic Review
Montori, V. M., Devereaux, P. J., Adhikari, N. K. J., Burns, K. E. A., Eggert, C. H., Briel, M., Lacchetti, C., Leung, T. W., Darling, E., Bryant, D. M., et al. (2005) · JAMA · doi:10.1001/jama.294.17.2203 · Cited by 26 · cited 1×
“Practices that can lead to P-hacking include an interim analysis to decide whether an experiment or a study should be stopped prematurely (21), as well as post hoc excluding of outliers from the analysis, deciding to combine or split groups, adjusting covariates, performing subgroup analysis (22), or choosing the thre…”
Desbiens, N. A. (2000) · Annals of Internal Medicine · doi:10.7326/0003-4819-133-4-200008150-00016 · Cited by 23
Commentary: Scientific heads are not turned by rhetoric
Greenhalgh, T. (1995) · BMJ · doi:10.1136/bmj.310.6985.987 · Cited by 23
Scientific rigor through videogames
Treuille, A., & Das, R. (2014) · Trends in Biochemical Sciences · doi:10.1016/j.tibs.2014.08.005 · Cited by 22 · cited 1×
“Such research imposes rigorous adherence to scientific methods, with a clear statement of hypothesis systematically preceding experiments; hence, cherry-picking would be caught easily because the data and hypothesis are open to all and fully searchable (87).”
BackgroundScreening for Hepatitis C Virus Infection in Adults
Chou, R., Cottrell, E. B., Wasson, N., Rahman, B., & Guise, J. (2012) · Europe PMC (PubMed Central) · Cited by 21
Le Fourn, E., Giraudeau, B., Chosidow, O., Doutre, M.-S., & Lorette, G. (2013) · PLoS ONE · doi:10.1371/journal.pone.0070717 · Cited by 19
Spin and Boasting in Research Articles
Cummings, P., & Rivara, F. P. (2012) · Archives of Pediatrics & Adolescent Medicine · doi:10.1001/archpediatrics.2012.1461 · Cited by 19 · cited 2×
“Any author can exaggerate the importance of the topic, unfairly dismiss previous work on it, or use persuasive words to convince the reader of a specific point of view (40, 41).”
BackgroundResultAddressing the Limitations of Structured Abstracts
Falagas, M., & Vergidis, P. I. (2004) · Annals of Internal Medicine · doi:10.7326/0003-4819-141-7-200410050-00023 · Cited by 16
Yavchitz, A., Ravaud, P., Hopewell, S., Baron, G., & Boutron, I. (2014) · BMC Medical Research Methodology · doi:10.1186/1471-2288-14-123 · Cited by 11 · cited 1×
“For example, the Annals of Internal Medicine requires the reporting of a limitation in the abstract (75, 76).”
BackgroundSelf-citationCepeda, M. S., Berlin, J. A., Glasser, S. C., Battisti, W. P., & Schuemie, M. J. (2015) · Drugs in R&D · doi:10.1007/s40268-015-0085-9 · Cited by 11
Methodological issues in trials assessing primary prophylaxis of venous thrombo-embolism
Thabut, G., Estellat, C., Boutron, I., Samama, C. M., & Ravaud, P. (2006) · European Heart Journal · doi:10.1093/eurheartj/ehi587 · Cited by 11 · cited 1×
“For example, some studies report “double-blind” methods, but the blinding is not credible (11, 12), or report an intent-to-treat analysis, but some patients are excluded from the analysis (13, 14).”
BackgroundSelf-citationOutcomes in the trial registry should match those in the protocol
Drysdale, H., Slade, E., Goldacre, B., & Heneghan, C. (2016) · The Lancet · doi:10.1016/s0140-6736(16)31128-x · Cited by 3
Sharp, M. K., Haneef, R., Ravaud, P., & Boutron, I. (2017) · BMJ Open · doi:10.1136/bmjopen-2016-014503 · Cited by 1
Self-citation
Research works printed but not linked: 1
- JC Bailar, How to distort the scientific record without actually lying: Truth, and arts of science. Eur J Oncol 11, 217–224 (2006).
Cited by
Fusaroli, M., Salvo, F., Begaud, B., AlShammari, T. M., Bate, A., Battini, V., Brueckner, A., Candore, G., Carnovale, C., Crisafulli, S., et al. (2024) · Drug Safety · doi:10.1007/s40264-024-01423-7 · Cited by 351
Delivering transformative action in paediatric pain: a Lancet Child & Adolescent Health Commission
Eccleston, C., Fisher, E., Howard, R. F., Slater, R., Forgeron, P., Palermo, T. M., Birnie, K. A., Anderson, B. J., Chambers, C. T., Crombez, G., et al. (2021) · The Lancet Child & Adolescent Health · doi:10.1016/s2352-4642(20)30277-7 · Cited by 328
Fusaroli, M., Salvo, F., Begaud, B., AlShammari, T. M., Bate, A., Battini, V., Brueckner, A., Candore, G., Carnovale, C., Crisafulli, S., et al. (2024) · Drug Safety · doi:10.1007/s40264-024-01421-9 · Cited by 270
Cabanac, G., Labbé, C., & Magazinov, A. (2021) · arXiv · doi:10.48550/arxiv.2107.06751 · Cited by 224
“Studies of academic writing use the term ‘spin’ to describe authors’ attempt to present a more positive description of a technique or drug in health sciences than the data actually support (Boutron et al., 2014; Boutron & Ravaud, 2018).”
MethodAcademic Misconduct, Misrepresentation and Gaming: A ReassessmentPreprint
Biagioli, M., Kenney, M. F., Martin, B. R., & Walsh, J. P. (2018) · SSRN Electronic Journal · doi:10.1016/j.respol.2018.10.025 · Cited by 146
Metadata
- Authors
- Boutron, I., & Ravaud, P.
- Affiliations
- Inserm (FR)
- Venue
- Proceedings of the National Academy of Sciences
- Publisher
- National Academy of Sciences
- Published
- 2018-03-12
- Volume
- 115
- Issue
- 11
- Pages
- 2613-2619
- ISSN
- 0027-8424 (ISSN-L)
- 1091-6490
- PMC
- PMC5856510
- OpenAlex
- W2793285640
- Semantic Scholar
- 13e1a17c9c9d1a99d05eed30291a4d26ad51fb3e
- Cited by
- 273
- References
- 91
- Citations by year
- 2026: 30 · 2025: 24 · 2024: 28 · 2023: 41 · 2022: 35 · 2021: 32 · 2020: 39 · 2019: 31 · 2018: 12
- FWCI
- 40.38
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Proceedings of the National Academy of Sciences (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
- Social Sciences › Education › Academic integrity and plagiarism
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Keywords
- biomedical literature, publication bias, research integrity, misinterpretation of results, research dissemination, peer review
- MeSH terms
- Humans, Literature, Publications, Peer Review, Research, Biomedical Research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-04, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-04 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol and qwen3.8-omni-flash).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.