Causal language and strength of inference in academic and media articles shared in social media (CLAIMS): A systematic review
Open accessThe DOI matches its Crossref record; 45 of the 45 references it cites were found in a registry or an index; 20 of the 33 sources its claims cite were checked for support.
Abstract
BACKGROUND: The pathway from evidence generation to consumption contains many steps which can lead to overstatement or misinformation. The proliferation of internet-based health news may encourage selection of media and academic research articles that overstate strength of causal inference. We investigated the state of causal inference in health research as it appears at the end of the pathway, at the point of social media consumption. METHODS: We screened the NewsWhip Insights database for the most shared media articles on Facebook and Twitter reporting about peer-reviewed academic studies associating an exposure with a health outcome in 2015, extracting the 50 most-shared academic articles and media articles covering them. We designed and utilized a review tool to systematically assess and summarize studies' strength of causal inference, including generalizability, potential confounders, and methods used. These were then compared with the strength of causal language used to describe results in both academic and media articles. Two randomly assigned independent reviewers and one arbitrating reviewer from a pool of 21 reviewers assessed each article. RESULTS: We accepted the most shared 64 media articles pertaining to 50 academic articles for review, representing 68% of Facebook and 45% of Twitter shares in 2015. Thirty-four percent of academic studies and 48% of media articles used language that reviewers considered too strong for their strength of causal inference. Seventy percent of academic studies were considered low or very low strength of inference, with only 6% considered high or very high strength of causal inference. The most severe issues with academic studies' causal inference were reported to be omitted confounding variables and generalizability. Fifty-eight percent of media articles were found to have inaccurately reported the question, results, intervention, or population of the academic study. CONCLUSIONS: We find a large disparity between the strength of language as presented to the research consumer and the underlying strength of causal inference among the studies most widely shared on social media. However, because this sample was designed to be representative of the articles selected and shared on social media, it is unlikely to be representative of all academic and media work. More research is needed to determine how academic institutions, media organizations, and social network sharing patterns impact causal inference and language as received by the research consumer.
Overview
CoChat Summary
TL;DR
In a systematic review of the 50 most-shared academic health studies and 64 corresponding media articles on Facebook/Twitter in 2015, most academic studies had low or very low strength of causal inference, and both academic and media language often overstated that inference. Media articles also frequently misreported key study characteristics.
Key findings
- 70% of academic studies were rated low or very low strength of causal inference (54% low, 16% very low), and only 6% were rated high or very high.
- 34% of academic studies and 48% of media articles used causal language reviewers considered too strong for the assessed causal inference; 58% of media articles inaccurately reported the question, results, intervention, or population of the academic study.
- The most severe academic issues were covariates/confounding (54% high or very high severity), generalizability (52%), and statistical methods (32%).
- Media language was stronger than academic language in 55% of cases, matched in 39%, and weaker in 6%; 83% of media articles linked to low/very low inference studies used moderate or strong causal language.
Methods
- Systematic review using NewsWhip Insights to identify the most-shared Facebook and Twitter media articles in 2015 reporting on peer-reviewed academic studies associating an exposure with a health outcome; extracted 50 academic articles and 64 associated media articles, representing 68% of Facebook shares and 45% of Twitter shares in 2015.
- Two randomly assigned independent reviewers plus one arbitrating reviewer from a pool of 21 assessed each article using a custom review tool; protocol followed PRISMA and was registered with PROSPERO (CRD42016045197).
- The tool rated strength of causal inference on a 5-point scale, causal language on a 3-point scale, and media accuracy; analysis used descriptive statistics in R and two-tailed t-tests.
Limitations
- Paper-stated: the sample is representative of articles selected and shared on social media, not all academic or media work; the study cannot isolate the roles of academic institutions, media, social media platforms, or the public; only published articles were reviewed, without replicating calculations, data, or code or assessing all error sources such as cherry-picking; only 50 article sets were reviewed; the review tool is subjective, resource-intensive, and exact reviewer agreement was imperfect; these limitations likely biased results toward finding higher strength of causal inference.
- Text-evident: the descriptive cross-sectional design cannot establish causal direction in the research-to-consumption pathway, and the media sample is limited to 64 articles linked to the 50 academic articles.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures



Verification
Claims the paper makes
13 Valid · 1 Not valid · 5 Not checked · 64 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Media outlets may spin [26–28] and encourage dissemination of eye-catching, potentially overstated, inaccurate [29], and/or misleading headlines in order to gain larger audience size [28, 30, 31].”
§ Causal language and strength of inference in academic and media articles shared in social media (CLAIMS): A systematic review > Introduction
What the sources say
- Not checked — no full text of the sourcecites [26] I Boutron, 2010, Reporting and interpretation of randomized controlled trials with statistically nonsignificant results for primary outcomes, JAMA source
- Partly supportedcites [27] S Lockyer, 2013, "Spin" in wound care research: the reporting and interpretation of randomized controlled trials with statistically non-significant primary outcome results or unspecified primary outcomes, Trials sourceExperimental — accuracy not yet measured
The Background describes spin as language and emphasis in trial reports that can mislead readers, and the Key findings report spin in wound-trial publications. The paper does not establish that media outlets spin headlines or do so to attract a larger audience; those are different actors and a motive the cited passages do not address. No page numbers are available in our copy. (LLM, conf=0.76)
- Partly supportedcites [28] R Haneef, 2015, Interpretation of Results of Studies Evaluating an Intervention Highlighted in Google Health News: A Cross-Sectional Study of News, PloS One sourceExperimental — accuracy not yet measured
The Results section (‘Prevalence of spin in news headlines’; page not supplied) reports spin in 58 of 130 headlines, including hype and misleading extrapolation. The paper does not establish that media outlets encouraged such headlines to gain a larger audience; the Discussion says it did not determine whether the spin originated in scientific articles or was added by journalists. (LLM, conf=0.89)
- Partly supportedcites [29] G Schwitzer, 2008, How do US journalists cover treatments, tests, products, and procedures? An evaluation of 500 stories, PLoS Med sourceExperimental — accuracy not yet measured
The Results and Impact section reports widespread shortcomings in health-news stories, and Next Steps says short health headlines can omit important context and mislead readers. Next Steps also discusses sensationalism as a way for journalists to attract attention. It does not establish that media outlets encourage dissemination of such headlines specifically to increase audience size. (LLM, conf=0.82)
- Partly supportedcites [30] R Moynihan, 2000, Coverage by the news media of the benefits and risks of medications, N Engl J Med sourceExperimental — accuracy not yet measured
The Conclusions (page unavailable) describe prior evidence of inaccurate coverage, overstatement, and sensationalism; the Results (page unavailable) show that medication stories often reported relative benefits without absolute benefits. These passages support concern about slanted coverage, but the paper does not establish that outlets disseminated misleading headlines to gain a larger audience. (LLM, conf=0.84)
- Partly supportedcites [31] A Yavchitz, 2012, Misrepresentation of randomized controlled trials in press releases and news coverage: a cohort study, PLoS Med sourceExperimental — accuracy not yet measured
The Abstract reports spin in 21 of 41 news items, and the Discussion says spin in press releases could increase journalists’ interest. The paper does not establish that media outlets promote misleading headlines to gain a larger audience; its proposed motive concerns interest from journalists, not audience size. (LLM, conf=0.84)
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: unclear — It is unclear whether it could be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Recent evidence suggests that exposure to health information in traditional and social media may impact health behaviors [2].”
§ Causal language and strength of inference in academic and media articles shared in social media (CLAIMS): A systematic review > Introduction
What the sources say
- Partly supportedcites [2] R Grilli, 2002, Mass media interventions: effects on health services utilisation, Cochrane Database Syst Rev sourceExperimental — accuracy not yet measured
The Background says mass media may influence individual health behaviours and gives examples involving dietary choices and contraceptive use. The review covers traditional mass-media channels, but it does not establish an effect of social media on health behaviours; it identifies the Internet as a channel for future research. (LLM, conf=0.88)
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources that could be checked couldn’t confirm or dispute this claim.
In the paper
“The authors of both publications later made statements retracting their conclusions, noting that they had overstated the strength of causal inference in light of the studies’ lack of methodological rigor [3].”
§ Causal language and strength of inference in academic and media articles shared in social media (CLAIMS): A systematic review > Introduction
What the sources say
- Could not verifycites [3] A Matthews, 2016, Impact of statin related media coverage on use of statins: interrupted time series analysis with UK primary care data, BMJ sourceExperimental — accuracy not yet measured
The Strengths and limitations section says the interrupted time series cannot confirm causality, but it does not report that the authors of two publications later retracted their conclusions or acknowledged a lack of methodological rigor. Searches for retractions, later statements, and overstated causal inference found no passage addressing those assertions. (LLM, conf=0.76)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Could not verify
Every source the paper cites for this claim was checked and supports it.
In the paper
“An estimated 200,000 patients temporarily ceased taking statins–drugs with strong evidence for efficacy and safety for treating hypertension [4, 5]–after press outlets reported on two studies suggesting that the medication was associated with high rates of adverse events.”
§ Causal language and strength of inference in academic and media articles shared in social media (CLAIMS): A systematic review > Introduction
What the sources say
- Supportedcites [3] A Matthews, 2016, Impact of statin related media coverage on use of statins: interrupted time series analysis with UK primary care data, BMJ source
The Impact on public health section estimates 218,971 excess UK patients stopped taking statins in the six months after media coverage, consistent with the rounded figure of 200,000. The Introduction links the coverage to two BMJ articles and discussion of purportedly high side-effect rates; Key findings says the increase in cessation was temporary. The paper does not establish that every individual who stopped later restarted. (LLM, conf=0.87)
hedgedstates a causen = 200,000- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Publication bias [20, 21], lack of replicability and replication [22], implication of a causal relationship even after stating that the evidence is insufficient to reach causal conclusions [23–25], and related phenomena may also introduce error into bodies of published literature.”
§ Causal language and strength of inference in academic and media articles shared in social media (CLAIMS): A systematic review > Introduction
What the sources say
- Supportedcites [23] SS Cofield, 2010, Use of causal language in observational studies of obesity and nutrition, Obes Facts source
The Introduction says causal language in observational studies can mislead readers. The Descriptive and Inferential Statistics section reports that 31 papers used causal language while also including a statement disavowing causation. Together, these passages support the cited concern that such reporting may introduce error into the literature. (LLM, conf=0.90)
- Not checked — no full text of the sourcecites [24] V Prasad, 2013, Observational studies often make clinical practice recommendations: an empirical evaluation of authors' attitudes, J Clin Epidemiol source
- Partly supportedcites [25] K Chiu, 2017, 'Spin' in published biomedical literature: A methodological systematic review, PLoS Biology sourceExperimental — accuracy not yet measured
The Results (Practices used to spin results) identify causal language in observational studies as spin because those designs do not permit causal inference. The Discussion likewise describes implying cause and effect as spin. Neither passage establishes the more specific sequence claimed here: authors first stating that the evidence is insufficient for a causal conclusion and then implying one. (LLM, conf=0.84)
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
49 more claims · 1 with their cited sources checked
Create a free account to see all 83 claims and to verify the 2 references still open.
- Probably not a substantial claim. 24
- It may state two separate claims as one. 16
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“in another study, roughly the same fraction of health-related papers widely shared on social media used inappropriately strong causal language (38).”
From Misinformation in and about science (2021) · cites this paper as [38]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
“Among the 35 academic articles rated as “Low” or “Very low” strength of causal inference, 45% (n = 16) were written with strong or moderate causal language.”
§ Causal language and strength of inference in academic and media articles shared in social media (CLAIMS): A systematic review > Results
Reported 45 · recomputed 45.714286
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Causal language and strength of inference in academic and media articles shared in social media (CLAIMS): A systematic review > Abstract > Results
“We accepted the most shared 64 media articles pertaining to 50 academic articles for review, representing 68% of Facebook and 45% of Twitter shares in 2015.”
§ Causal language and strength of inference in academic and media articles shared in social media (CLAIMS): A systematic review > Results
“The cumulative number of shares among the 231 screened articles screened represents 68% (938,596/1,375,152) of Facebook shares and 45% (189,777/423,996) of Twitter shares (S1 Fig)”
Citations
Cites
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: the PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & for the PRISMA Group (2009) · BMJ · doi:10.1136/bmj.b2535 · Cited by 83,823
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & The PRISMA Group (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000097 · Cited by 64,042 · cited 1×
“The final protocol for this project was developed and reported using PRISMA guidelines for systematic review [35], and registered with PROSPERO [36] as protocol CRD42016045197 on August 5, 2016.”
Preferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement.
(2009) · PubMed Central · Cited by 44,859
ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions
Sterne, J. A., Hernán, M. A., Reeves, B. C., Savović, J., Berkman, N. D., Viswanathan, M., Henry, D., Altman, D. G., Ansari, M. T., Boutron, I., et al. (2016) · BMJ · doi:10.1136/bmj.i4919 · Cited by 20,329 · cited 2×
“While similar tools and frameworks have recently been proposed, most notably the ROBINS-I [44] and related Risk of Bias 2.0 [45] tools, the CLAIMS tool differs in several key respects in order to be most useful for research consumers and reviewers.”
MethodInfluentialWang, G., Ma, R., Qiao, G., Wada, K., Aizawa, Y., & Satoh, T. (2015) · Integrative Medicine International · doi:10.1159/000368408 · Cited by 18,657
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505
Estimating the reproducibility of psychological science
Open Science Collaboration (2015) · Science · doi:10.1126/science.aac4716 · Cited by 8,515
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · CHANCE · doi:10.1080/09332480.2005.10722754 · Cited by 4,702 · cited 1×
“A host of factors can render studies inaccurate [15] or not useful in practice [16].”
BackgroundSystematic Review of the Empirical Evidence of Study Publication Bias and Outcome Reporting Bias
Dwan, K., Altman, D. G., Arnaiz, J. A., Bloom, J., Chan, A.-W., Cronin, E., Decullier, E., Easterbrook, P. J., Von Elm, E., Gamble, C., et al. (2008) · PLoS ONE · doi:10.1371/journal.pone.0003081 · Cited by 1,546 · cited 1×
“Publication bias [20, 21], lack of replicability and replication [22], implication of a causal relationship even after stating that the evidence is insufficient to reach causal conclusions [23–25], and related phenomena may also introduce error into bodies of published literature.”
BackgroundThe Extent and Consequences of P-Hacking in Science
Head, M. L., Holman, L., Lanfear, R., Kahn, A. T., & Jennions, M. D. (2015) · PLOS Biology · doi:10.1371/journal.pbio.1002106 · Cited by 1,388
Negative results are disappearing from most disciplines and countries
Fanelli, D. (2012) · Scientometrics · doi:10.1007/s11192-011-0494-7 · Cited by 1,248 · cited 1×
“Publication bias [20, 21], lack of replicability and replication [22], implication of a causal relationship even after stating that the evidence is insufficient to reach causal conclusions [23–25], and related phenomena may also introduce error into bodies of published literature.”
Background(2016) · Cochrane Database of Systematic Reviews · Cited by 985
Boutron, I. (2010) · JAMA · doi:10.1001/jama.2010.651 · Cited by 842
Hernán, M. A., Alonso, A., Logan, R., Grodstein, F., Michels, K. B., Willett, W. C., Manson, J. E., & Robins, J. M. (2008) · Epidemiology · doi:10.1097/ede.0b013e3181875e61 · Cited by 818 · cited 1×
“The summary assessments asked whether the study was likely to approximate hypothetical results from an “ideal” randomized controlled trial which would result in perfectly estimated and generalizable causal effects without practical and ethical constraints [40].”
BackgroundWhy Most Clinical Research Is Not Useful
Ioannidis, J. P. A. (2016) · PLOS Medicine · doi:10.1371/journal.pmed.1002049 · Cited by 655 · cited 1×
“A host of factors can render studies inaccurate [15] or not useful in practice [16].”
BackgroundMass media interventions: effects on health services utilisation
Grilli, R., Ramsay, C., & Minozzi, S. (2002) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd000389 · Cited by 533 · cited 1×
“Recent evidence suggests that exposure to health information in traditional and social media may impact health behaviors [2].”
BackgroundCoverage by the news media of the benefits and risks of medications
Moynihan, R., Bero, L., Ross-Degnan, D., Henry, D., Lee, K., Watkins, J., Mah, C., & Soumerai, S. B. (2000) · New England Journal of Medicine · doi:10.1056/nejm200006013422206 · Cited by 500 · cited 1×
“Media outlets may spin [26–28] and encourage dissemination of eye-catching, potentially overstated, inaccurate [29], and/or misleading headlines in order to gain larger audience size [28, 30, 31].”
BackgroundSumner, P., Vivian-Griffiths, S., Boivin, J., Williams, A., Venetis, C. A., Davies, A., Ogden, J., Whelan, L., Hughes, B., Dalton, B., et al. (2014) · BMJ · doi:10.1136/bmj.g7015 · Cited by 489
Boutron, I., Altman, D. G., Hopewell, S., Vera-Badillo, F., Tannock, I., & Ravaud, P. (2014) · Journal of Clinical Oncology · doi:10.1200/jco.2014.56.7503 · Cited by 368 · cited 1×
“Further contributing to the cycle, media coverage of specific conditions is associated with increased reporting of adverse events [6], public announcements of celebrity illnesses are associated with surges in screening and diagnostic tests [7, 8], and language “spin” may influence clinical decision makers’ assessment…”
Background‘Spin’ in published biomedical literature: A methodological systematic review
Chiu, K., Grundy, Q., & Bero, L. (2017) · PLOS Biology · doi:10.1371/journal.pbio.2002173 · Cited by 317 · cited 1×
“Previous studies examining overstatement of causal language utilize either language comparison and/or assumed strength of causal inference exclusively from very broad study design characteristics [12, 23, 25, 31].”
BackgroundYavchitz, A., Boutron, I., Bafeta, A., Marroun, I., Charles, P., Mantz, J., & Ravaud, P. (2012) · PLoS Medicine · doi:10.1371/journal.pmed.1001308 · Cited by 293 · cited 2×
“Previous studies examining overstatement of causal language utilize either language comparison and/or assumed strength of causal inference exclusively from very broad study design characteristics [12, 23, 25, 31].”
Background(2016) · doi:10.1002/14651858.cd201601 · Cited by 278
Generalizing Study Results: A Potential Outcomes Perspective
Lesko, C. R., Buchanan, A. L., Westreich, D., Edwards, J. K., Hudgens, M. G., & Cole, S. R. (2017) · Epidemiology · doi:10.1097/ede.0000000000000664 · Cited by 269
NIH Disease Funding Levels and Burden of Disease
Gillum, L. A., Gouveia, C., Dorsey, E. R., Pletcher, M., Mathers, C. D., McCulloch, C. E., & Johnston, S. C. (2011) · PLoS ONE · doi:10.1371/journal.pone.0016837 · Cited by 248 · cited 1×
“We speculate that the proliferation of social media and open access to research may increase ties from popular preferences to research production and selection, and that increased competition for stagnant research funding [46] is likely to exacerbate incentives to have research that stands out over that addressing are…”
BackgroundMatthews, A., Herrett, E., Gasparrini, A., Van Staa, T., Goldacre, B., Smeeth, L., & Bhaskaran, K. (2016) · BMJ · doi:10.1136/bmj.i3283 · Cited by 221 · cited 2×
“An example in the UK [3] highlights the dangers of weak and poorly-reported evidence: An estimated 200,000 patients temporarily ceased taking statins–drugs with strong evidence for efficacy and safety for treating hypertension [4, 5]–after press outlets reported on two studies suggesting that the medication was associ…”
BackgroundOffline: What is medicine's 5 sigma?
Horton, R. (2015) · The Lancet · doi:10.1016/s0140-6736(15)60696-1 · Cited by 219 · cited 1×
“These incentives apply across the academic spectrum, from individual researchers, to research intuitions, to press releases [10–12], and to peer reviewed publication [13, 14].”
BackgroundSchwitzer, G. (2008) · PLoS Medicine · doi:10.1371/journal.pmed.0050095 · Cited by 174 · cited 1×
“Media outlets may spin [26–28] and encourage dissemination of eye-catching, potentially overstated, inaccurate [29], and/or misleading headlines in order to gain larger audience size [28, 30, 31].”
Brown, A. W., Bohan Brown, M. M., & Allison, D. B. (2013) · The American Journal of Clinical Nutrition · doi:10.3945/ajcn.113.064410 · Cited by 155
Desai, S., & Jena, A. B. (2016) · BMJ · doi:10.1136/bmj.i6357 · Cited by 114 · cited 1×
“Further contributing to the cycle, media coverage of specific conditions is associated with increased reporting of adverse events [6], public announcements of celebrity illnesses are associated with surges in screening and diagnostic tests [7, 8], and language “spin” may influence clinical decision makers’ assessment…”
Press Releases by Academic Medical Centers: Not So Academic?
Woloshin, S., Schwartz, L. M., Casella, S. L., Kennedy, A. T., & Larson, R. J. (2009) · Annals of Internal Medicine · doi:10.7326/0003-4819-150-9-200905050-00007 · Cited by 108 · cited 1×
“These incentives apply across the academic spectrum, from individual researchers, to research intuitions, to press releases [10-12], and to peer reviewed publication [13, 14].”
BackgroundPrasad, V., Jorgenson, J., Ioannidis, J. P. A., & Cifu, A. (2013) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2012.11.005 · Cited by 90
Estimating the Reproducibility of Psychological Science
Nosek, B. A., Cohoon, J., Kidwell, M., & Spies, J. R. (2016) · doi:10.31219/osf.io/447b3 · Cited by 88
Faasse, K., Gamble, G., Cundy, T., & Petrie, K. J. (2012) · BMJ Open · doi:10.1136/bmjopen-2012-001607 · Cited by 83 · cited 1×
“Further contributing to the cycle, media coverage of specific conditions is associated with increased reporting of adverse events [6], public announcements of celebrity illnesses are associated with surges in screening and diagnostic tests [7, 8], and language “spin” may influence clinical decision makers’ assessment…”
BackgroundPoor replication validity of biomedical association studies reported by newspapers
Dumas-Mallet, E., Smith, A., Boraud, T., & Gonon, F. (2017) · PLOS ONE · doi:10.1371/journal.pone.0172650 · Cited by 76 · cited 1×
“The interaction between academia and media highlights some of these complexities, such as a recent study finding that the results of over half of studies on an association between exposure and outcome covered in the news are refuted by subsequent meta-analyses [33], and another showing that those articles publicly cri…”
BackgroundUse of causal language in observational studies of obesity and nutrition
Cofield, S. S., Corona, R. V., & Allison, D. B. (2010) · Obesity Facts · doi:10.1159/000322940 · Cited by 74 · cited 1×
“Previous studies examining overstatement of causal language utilize either language comparison and/or assumed strength of causal inference exclusively from very broad study design characteristics [12, 23, 25, 31].”
BackgroundHaneef, R., Lazarus, C., Ravaud, P., Yavchitz, A., & Boutron, I. (2015) · PLOS ONE · doi:10.1371/journal.pone.0140889 · Cited by 70 · cited 1×
“Media outlets may spin [26–28] and encourage dissemination of eye-catching, potentially overstated, inaccurate [29], and/or misleading headlines in order to gain larger audience size [28, 30, 31].”
BackgroundLockyer, S., Hodgson, R., Dumville, J. C., & Cullum, N. (2013) · Trials · doi:10.1186/1745-6215-14-371 · Cited by 66
Kelaher, M., Cawson, J., Miller, J., Kavanagh, A., Dunt, D., & Studdert, D. M. (2008) · International Journal of Epidemiology · doi:10.1093/ije/dyn090 · Cited by 59 · cited 1×
“Further contributing to the cycle, media coverage of specific conditions is associated with increased reporting of adverse events [6], public announcements of celebrity illnesses are associated with surges in screening and diagnostic tests [7, 8], and language “spin” may influence clinical decision makers’ assessment…”
Background(2016) · KETlib (University of Piraeus) · Cited by 57
Causal Impact: Epidemiological Approaches for a Public Health of Consequence
Westreich, D., Edwards, J. K., Rogawski, E. T., Hudgens, M. G., Stuart, E. A., & Cole, S. R. (2016) · American Journal of Public Health · doi:10.2105/ajph.2016.303226 · Cited by 51 · cited 1×
“External validity may be threatened by the limited generalizability of study populations to relevant populations or differences between the study environment and real-world interventions [18, 19].”
BackgroundEfficacy of Statin Therapy in Pulmonary Arterial Hypertension: A Systematic Review and Meta-Analysis
Rysz-Górzynska, M., Gluba-Brzózka, A., Sahebkar, A., Serban, M.-C., Mikhailidis, D. P., Ursoniu, S., Toth, P. P., Bittner, V., Watts, G. F., Lip, G. Y. H., et al. (2016) · Scientific Reports · doi:10.1038/srep30060 · Cited by 33 · cited 1×
“An example in the UK [3] highlights the dangers of weak and poorly-reported evidence: An estimated 200,000 patients temporarily ceased taking statins–drugs with strong evidence for efficacy and safety for treating hypertension [4, 5]–after press outlets reported on two studies suggesting that the medication was associ…”
BackgroundA systematic review and meta-analysis of trials using statins in pulmonary arterial hypertension
Anand, V., Garg, S., Duval, S., & Thenappan, T. (2016) · Pulmonary Circulation · doi:10.1086/687304 · Cited by 25 · cited 1×
“An example in the UK [3] highlights the dangers of weak and poorly-reported evidence: An estimated 200,000 patients temporarily ceased taking statins–drugs with strong evidence for efficacy and safety for treating hypertension [4, 5]–after press outlets reported on two studies suggesting that the medication was associ…”
BackgroundPress-released papers are more downloaded and cited: Table 1
Chapman, S., Nguyen, T. N., & White, C. (2007) · Tobacco Control · doi:10.1136/tc.2006.019034 · Cited by 24 · cited 1×
“These incentives apply across the academic spectrum, from individual researchers, to research intuitions, to press releases [10-12], and to peer reviewed publication [13, 14].”
BackgroundEditors, Publishers, Impact Factors, and Reprint Income
Marcovitch, H. (2010) · PLoS Medicine · doi:10.1371/journal.pmed.1000355 · Cited by 19 · cited 1×
“These incentives apply across the academic spectrum, from individual researchers, to research intuitions, to press releases [10–12], and to peer reviewed publication [13, 14].”
BackgroundBrookes, P. S. (2014) · PeerJ · doi:10.7717/peerj.313 · Cited by 16 · cited 1×
“The interaction between academia and media highlights some of these complexities, such as a recent study finding that the results of over half of studies on an association between exposure and outcome covered in the news are refuted by subsequent meta-analyses [33], and another showing that those articles publicly cri…”
Background
Research works printed but not linked: 2
- 2011, The Oxford 2011 Levels of Evidence
- Clarivate Analytics. Journal Citation Reports. 2015.
Cited by
Public health and online misinformation: challenges and recommendations
Swire-Thompson, B., & Lazer, D. (2020) · Annual Review of Public Health · doi:10.1146/annurev-publhealth-040119-094127 · Cited by 1,170
“In response to a rapid decline in human papillomavirus (HPV) vaccine uptake, Danish public health officials created c r i These technologies can be particularly useful for rural communities.”
BackgroundMisinformation in and about science
West, J. D., & Bergstrom, C. T. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1912444117 · Cited by 391
“One analysis found that nearly a third of 525 papers in top obesity or nutrition journals make inappropriate causal claims in their abstracts or titles (37); in another study, roughly the same fraction of health-related papers widely shared on social media used inappropriately strong causal language (38).”
BackgroundYeung, A. W. K., Tosevska, A., Klager, E., Eibensteiner, F., Tsagkaris, C., Parvanov, E. D., Nawaz, F. A., Völkl-Kernstock, S., Schaden, E., Kletecka-Pulker, M., et al. (2022) · Journal of Medical Internet Research · doi:10.2196/28152 · Cited by 203
“Often, media articles may report research findings with misinterpretation and overstatement that can lead to confusion, misinformation, and mistrust in scientific reporting [6].”
Guidance to best tools and practices for systematic reviews
Kolaski, K., Logan, L. R., & Ioannidis, J. P. A. (2023) · Systematic Reviews · doi:10.1186/s13643-023-02255-9 · Cited by 236
“This is indicated by recent reports of poor interpretability of systematic review results by readers [184–186] and misleading interpretations of the evidence related to the “spin” systematic review authors may put on their conclusions [50, 187].”
Prime, H., Andrews, K., Markwell, A., González, A., Janus, M., Tricco, A. C., Bennett, T. A., & Atkinson, L. R. (2023) · Clinical Child and Family Psychology Review · doi:10.1007/s10567-022-00423-2 · Cited by 157
Self-citation
Metadata
- Authors
- Haber, N., Smith, E. R., Moscoe, E., Andrews, K., Audy, R., Bell, W., Brennan, A. T., Breskin, A., Kane, J. C., Karra, M., McClure, E. S., Suarez, E. A., & on behalf of the CLAIMS research team
- Affiliations
- University of North Carolina at Chapel Hill (US)
- Boston Children's Hospital (US)
- Harvard University (US)
- Tufts University (US)
- Boston University (US)
- Johns Hopkins University (US)
- Corresponding author
- Haber, Noah
- ORCID
- Venue
- PLoS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2018-05-30
- Volume
- 13
- Issue
- 5
- Pages
- e0196346
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC5976147
- OpenAlex
- W2805549342
- Cited by
- 97
- References
- 45
- Citations by year
- 2026: 5 · 2025: 7 · 2024: 7 · 2023: 19 · 2022: 11 · 2021: 20 · 2020: 11 · 2019: 16 · 2018: 1
- FWCI
- 4.12
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- public-domain
- creativecommons.org/publicdomain/zero/1.0/ (vor)
- Locations
- PLoS ONE (publishedVersion, public-domain)
- Europe PMC (PubMed Central) (publishedVersion, public-domain)
- PubMed Central (publishedVersion, public-domain)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (acceptedVersion)
- OpenBU (Boston University) (acceptedVersion, cc-by)
- Digital Access to Scholarship at Harvard (DASH) (Harvard University) (submittedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Figshare (submittedVersion, cc-by)
- Funders
- Eunice Kennedy Shriver National Institute of Child Health and Human Development (P2CHD050924)
- Topics
- Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Health Sciences › Health Professions › General Health Professions › Health Literacy and Information Accessibility
- Keywords
- causal inference, quality of evidence, causal language, health misinformation, media coverage of research, social media sharing, Facebook, Twitter, research generalizability, confounding, health news
- MeSH terms
- Communication, Communications Media, Humans, Language, Causality, Internet, Biomedical Research, Social Media
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.