Causal language and strength of inference in academic and media articles shared in social media (CLAIMS): A systematic review
Open accessThe DOI matches its Crossref record; 45 of the 45 references it cites were found in a registry or an index; 20 of the 33 sources its claims cite were checked for support.
Abstract
BACKGROUND: The pathway from evidence generation to consumption contains many steps which can lead to overstatement or misinformation. The proliferation of internet-based health news may encourage selection of media and academic research articles that overstate strength of causal inference. We investigated the state of causal inference in health research as it appears at the end of the pathway, at the point of social media consumption. METHODS: We screened the NewsWhip Insights database for the most shared media articles on Facebook and Twitter reporting about peer-reviewed academic studies associating an exposure with a health outcome in 2015, extracting the 50 most-shared academic articles and media articles covering them. We designed and utilized a review tool to systematically assess and summarize studies' strength of causal inference, including generalizability, potential confounders, and methods used. These were then compared with the strength of causal language used to describe results in both academic and media articles. Two randomly assigned independent reviewers and one arbitrating reviewer from a pool of 21 reviewers assessed each article. RESULTS: We accepted the most shared 64 media articles pertaining to 50 academic articles for review, representing 68% of Facebook and 45% of Twitter shares in 2015. Thirty-four percent of academic studies and 48% of media articles used language that reviewers considered too strong for their strength of causal inference. Seventy percent of academic studies were considered low or very low strength of inference, with only 6% considered high or very high strength of causal inference. The most severe issues with academic studies' causal inference were reported to be omitted confounding variables and generalizability. Fifty-eight percent of media articles were found to have inaccurately reported the question, results, intervention, or population of the academic study. CONCLUSIONS: We find a large disparity between the strength of language as presented to the research consumer and the underlying strength of causal inference among the studies most widely shared on social media. However, because this sample was designed to be representative of the articles selected and shared on social media, it is unlikely to be representative of all academic and media work. More research is needed to determine how academic institutions, media organizations, and social network sharing patterns impact causal inference and language as received by the research consumer.
Overview
CoChat Summary
TL;DR
In a systematic review of the 50 most-shared academic health studies and 64 corresponding media articles on Facebook/Twitter in 2015, most academic studies had low or very low strength of causal inference, and both academic and media language often overstated that inference. Media articles also frequently misreported key study characteristics.
Key findings
- 70% of academic studies were rated low or very low strength of causal inference (54% low, 16% very low), and only 6% were rated high or very high.
- 34% of academic studies and 48% of media articles used causal language reviewers considered too strong for the assessed causal inference; 58% of media articles inaccurately reported the question, results, intervention, or population of the academic study.
- The most severe academic issues were covariates/confounding (54% high or very high severity), generalizability (52%), and statistical methods (32%).
- Media language was stronger than academic language in 55% of cases, matched in 39%, and weaker in 6%; 83% of media articles linked to low/very low inference studies used moderate or strong causal language.
Methods
- Systematic review using NewsWhip Insights to identify the most-shared Facebook and Twitter media articles in 2015 reporting on peer-reviewed academic studies associating an exposure with a health outcome; extracted 50 academic articles and 64 associated media articles, representing 68% of Facebook shares and 45% of Twitter shares in 2015.
- Two randomly assigned independent reviewers plus one arbitrating reviewer from a pool of 21 assessed each article using a custom review tool; protocol followed PRISMA and was registered with PROSPERO (CRD42016045197).
- The tool rated strength of causal inference on a 5-point scale, causal language on a 3-point scale, and media accuracy; analysis used descriptive statistics in R and two-tailed t-tests.
Limitations
- Paper-stated: the sample is representative of articles selected and shared on social media, not all academic or media work; the study cannot isolate the roles of academic institutions, media, social media platforms, or the public; only published articles were reviewed, without replicating calculations, data, or code or assessing all error sources such as cherry-picking; only 50 article sets were reviewed; the review tool is subjective, resource-intensive, and exact reviewer agreement was imperfect; these limitations likely biased results toward finding higher strength of causal inference.
- Text-evident: the descriptive cross-sectional design cannot establish causal direction in the research-to-consumption pathway, and the media sample is limited to 64 articles linked to the 50 academic articles.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures



Verification
Claims the paper makes
13 Valid · 1 Not valid · 5 Not checked · 64 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Media outlets may spin [26–28] and encourage dissemination of eye-catching, potentially overstated, inaccurate [29], and/or misleading headlines in order to gain larger audience size [28, 30, 31].”
§ Causal language and strength of inference in academic and media articles shared in social media (CLAIMS): A systematic review > Introduction
What the sources say
- Not checked — no full text of the sourcecites [26] I Boutron, 2010, Reporting and interpretation of randomized controlled trials with statistically nonsignificant results for primary outcomes, JAMA source
- Partly supportedcites [27] S Lockyer, 2013, "Spin" in wound care research: the reporting and interpretation of randomized controlled trials with statistically non-significant primary outcome results or unspecified primary outcomes, Trials sourceExperimental — accuracy not yet measured
The Background describes spin as language and emphasis in trial reports that can mislead readers, and the Key findings report spin in wound-trial publications. The paper does not establish that media outlets spin headlines or do so to attract a larger audience; those are different actors and a motive the cited passages do not address. No page numbers are available in our copy. (LLM, conf=0.76)
- Partly supportedcites [28] R Haneef, 2015, Interpretation of Results of Studies Evaluating an Intervention Highlighted in Google Health News: A Cross-Sectional Study of News, PloS One sourceExperimental — accuracy not yet measured
The Results section (‘Prevalence of spin in news headlines’; page not supplied) reports spin in 58 of 130 headlines, including hype and misleading extrapolation. The paper does not establish that media outlets encouraged such headlines to gain a larger audience; the Discussion says it did not determine whether the spin originated in scientific articles or was added by journalists. (LLM, conf=0.89)
- Partly supportedcites [29] G Schwitzer, 2008, How do US journalists cover treatments, tests, products, and procedures? An evaluation of 500 stories, PLoS Med sourceExperimental — accuracy not yet measured
The Results and Impact section reports widespread shortcomings in health-news stories, and Next Steps says short health headlines can omit important context and mislead readers. Next Steps also discusses sensationalism as a way for journalists to attract attention. It does not establish that media outlets encourage dissemination of such headlines specifically to increase audience size. (LLM, conf=0.82)
- Partly supportedcites [30] R Moynihan, 2000, Coverage by the news media of the benefits and risks of medications, N Engl J Med sourceExperimental — accuracy not yet measured
The Conclusions (page unavailable) describe prior evidence of inaccurate coverage, overstatement, and sensationalism; the Results (page unavailable) show that medication stories often reported relative benefits without absolute benefits. These passages support concern about slanted coverage, but the paper does not establish that outlets disseminated misleading headlines to gain a larger audience. (LLM, conf=0.84)
- Partly supportedcites [31] A Yavchitz, 2012, Misrepresentation of randomized controlled trials in press releases and news coverage: a cohort study, PLoS Med sourceExperimental — accuracy not yet measured
The Abstract reports spin in 21 of 41 news items, and the Discussion says spin in press releases could increase journalists’ interest. The paper does not establish that media outlets promote misleading headlines to gain a larger audience; its proposed motive concerns interest from journalists, not audience size. (LLM, conf=0.84)
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: unclear — It is unclear whether it could be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Recent evidence suggests that exposure to health information in traditional and social media may impact health behaviors [2].”
§ Causal language and strength of inference in academic and media articles shared in social media (CLAIMS): A systematic review > Introduction
What the sources say
- Partly supportedcites [2] R Grilli, 2002, Mass media interventions: effects on health services utilisation, Cochrane Database Syst Rev sourceExperimental — accuracy not yet measured
The Background says mass media may influence individual health behaviours and gives examples involving dietary choices and contraceptive use. The review covers traditional mass-media channels, but it does not establish an effect of social media on health behaviours; it identifies the Internet as a channel for future research. (LLM, conf=0.88)
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources that could be checked couldn’t confirm or dispute this claim.
In the paper
“The authors of both publications later made statements retracting their conclusions, noting that they had overstated the strength of causal inference in light of the studies’ lack of methodological rigor [3].”
§ Causal language and strength of inference in academic and media articles shared in social media (CLAIMS): A systematic review > Introduction
What the sources say
- Could not verifycites [3] A Matthews, 2016, Impact of statin related media coverage on use of statins: interrupted time series analysis with UK primary care data, BMJ sourceExperimental — accuracy not yet measured
The Strengths and limitations section says the interrupted time series cannot confirm causality, but it does not report that the authors of two publications later retracted their conclusions or acknowledged a lack of methodological rigor. Searches for retractions, later statements, and overstated causal inference found no passage addressing those assertions. (LLM, conf=0.76)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Could not verify
Every source the paper cites for this claim was checked and supports it.
In the paper
“An estimated 200,000 patients temporarily ceased taking statins–drugs with strong evidence for efficacy and safety for treating hypertension [4, 5]–after press outlets reported on two studies suggesting that the medication was associated with high rates of adverse events.”
§ Causal language and strength of inference in academic and media articles shared in social media (CLAIMS): A systematic review > Introduction
What the sources say
- Supportedcites [3] A Matthews, 2016, Impact of statin related media coverage on use of statins: interrupted time series analysis with UK primary care data, BMJ source
The Impact on public health section estimates 218,971 excess UK patients stopped taking statins in the six months after media coverage, consistent with the rounded figure of 200,000. The Introduction links the coverage to two BMJ articles and discussion of purportedly high side-effect rates; Key findings says the increase in cessation was temporary. The paper does not establish that every individual who stopped later restarted. (LLM, conf=0.87)
hedgedstates a causen = 200,000- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Publication bias [20, 21], lack of replicability and replication [22], implication of a causal relationship even after stating that the evidence is insufficient to reach causal conclusions [23–25], and related phenomena may also introduce error into bodies of published literature.”
§ Causal language and strength of inference in academic and media articles shared in social media (CLAIMS): A systematic review > Introduction
What the sources say
- Supportedcites [23] SS Cofield, 2010, Use of causal language in observational studies of obesity and nutrition, Obes Facts source
The Introduction says causal language in observational studies can mislead readers. The Descriptive and Inferential Statistics section reports that 31 papers used causal language while also including a statement disavowing causation. Together, these passages support the cited concern that such reporting may introduce error into the literature. (LLM, conf=0.90)
- Not checked — no full text of the sourcecites [24] V Prasad, 2013, Observational studies often make clinical practice recommendations: an empirical evaluation of authors' attitudes, J Clin Epidemiol source
- Partly supportedcites [25] K Chiu, 2017, 'Spin' in published biomedical literature: A methodological systematic review, PLoS Biology sourceExperimental — accuracy not yet measured
The Results (Practices used to spin results) identify causal language in observational studies as spin because those designs do not permit causal inference. The Discussion likewise describes implying cause and effect as spin. Neither passage establishes the more specific sequence claimed here: authors first stating that the evidence is insufficient for a causal conclusion and then implying one. (LLM, conf=0.84)
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
49 more claims · 1 with their cited sources checked
Create a free account to see all 83 claims and to verify the 2 references still open.
- Probably not a substantial claim. 24
- It may state two separate claims as one. 16
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“in another study, roughly the same fraction of health-related papers widely shared on social media used inappropriately strong causal language (38).”
From Misinformation in and about science (2021) · cites this paper as [38]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
“Among the 35 academic articles rated as “Low” or “Very low” strength of causal inference, 45% (n = 16) were written with strong or moderate causal language.”
§ Causal language and strength of inference in academic and media articles shared in social media (CLAIMS): A systematic review > Results
Reported 45 · recomputed 45.714286
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Causal language and strength of inference in academic and media articles shared in social media (CLAIMS): A systematic review > Abstract > Results
“We accepted the most shared 64 media articles pertaining to 50 academic articles for review, representing 68% of Facebook and 45% of Twitter shares in 2015.”
§ Causal language and strength of inference in academic and media articles shared in social media (CLAIMS): A systematic review > Results
“The cumulative number of shares among the 231 screened articles screened represents 68% (938,596/1,375,152) of Facebook shares and 45% (189,777/423,996) of Twitter shares (S1 Fig)”
Citations
Cites
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: the PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & for the PRISMA Group (2009) · BMJ · doi:10.1136/bmj.b2535 · Cited by 83,823
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & The PRISMA Group (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000097 · Cited by 64,042 · cited 1×
“The final protocol for this project was developed and reported using PRISMA guidelines for systematic review [35], and registered with PROSPERO [36] as protocol CRD42016045197 on August 5, 2016.”
Preferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement.
(2009) · PubMed Central · Cited by 44,859
ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions
Sterne, J. A., Hernán, M. A., Reeves, B. C., Savović, J., Berkman, N. D., Viswanathan, M., Henry, D., Altman, D. G., Ansari, M. T., Boutron, I., et al. (2016) · BMJ · doi:10.1136/bmj.i4919 · Cited by 20,329 · cited 2×
“While similar tools and frameworks have recently been proposed, most notably the ROBINS-I [44] and related Risk of Bias 2.0 [45] tools, the CLAIMS tool differs in several key respects in order to be most useful for research consumers and reviewers.”
MethodInfluentialWang, G., Ma, R., Qiao, G., Wada, K., Aizawa, Y., & Satoh, T. (2015) · Integrative Medicine International · doi:10.1159/000368408 · Cited by 18,657
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505
Estimating the reproducibility of psychological science
Open Science Collaboration (2015) · Science · doi:10.1126/science.aac4716 · Cited by 8,515
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · CHANCE · doi:10.1080/09332480.2005.10722754 · Cited by 4,702 · cited 1×
“A host of factors can render studies inaccurate [15] or not useful in practice [16].”
BackgroundSystematic Review of the Empirical Evidence of Study Publication Bias and Outcome Reporting Bias
Dwan, K., Altman, D. G., Arnaiz, J. A., Bloom, J., Chan, A.-W., Cronin, E., Decullier, E., Easterbrook, P. J., Von Elm, E., Gamble, C., et al. (2008) · PLoS ONE · doi:10.1371/journal.pone.0003081 · Cited by 1,546 · cited 1×
“Publication bias [20, 21], lack of replicability and replication [22], implication of a causal relationship even after stating that the evidence is insufficient to reach causal conclusions [23–25], and related phenomena may also introduce error into bodies of published literature.”
BackgroundThe Extent and Consequences of P-Hacking in Science
Head, M. L., Holman, L., Lanfear, R., Kahn, A. T., & Jennions, M. D. (2015) · PLOS Biology · doi:10.1371/journal.pbio.1002106 · Cited by 1,388
Research works printed but not linked: 2
- 2011, The Oxford 2011 Levels of Evidence
- Clarivate Analytics. Journal Citation Reports. 2015.
Cited by
Public health and online misinformation: challenges and recommendations
Swire-Thompson, B., & Lazer, D. (2020) · Annual Review of Public Health · doi:10.1146/annurev-publhealth-040119-094127 · Cited by 1,170
“In response to a rapid decline in human papillomavirus (HPV) vaccine uptake, Danish public health officials created c r i These technologies can be particularly useful for rural communities.”
BackgroundMisinformation in and about science
West, J. D., & Bergstrom, C. T. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1912444117 · Cited by 391
“One analysis found that nearly a third of 525 papers in top obesity or nutrition journals make inappropriate causal claims in their abstracts or titles (37); in another study, roughly the same fraction of health-related papers widely shared on social media used inappropriately strong causal language (38).”
BackgroundYeung, A. W. K., Tosevska, A., Klager, E., Eibensteiner, F., Tsagkaris, C., Parvanov, E. D., Nawaz, F. A., Völkl-Kernstock, S., Schaden, E., Kletecka-Pulker, M., et al. (2022) · Journal of Medical Internet Research · doi:10.2196/28152 · Cited by 203
“Often, media articles may report research findings with misinterpretation and overstatement that can lead to confusion, misinformation, and mistrust in scientific reporting [6].”
Guidance to best tools and practices for systematic reviews
Kolaski, K., Logan, L. R., & Ioannidis, J. P. A. (2023) · Systematic Reviews · doi:10.1186/s13643-023-02255-9 · Cited by 236
“This is indicated by recent reports of poor interpretability of systematic review results by readers [184–186] and misleading interpretations of the evidence related to the “spin” systematic review authors may put on their conclusions [50, 187].”
Prime, H., Andrews, K., Markwell, A., González, A., Janus, M., Tricco, A. C., Bennett, T. A., & Atkinson, L. R. (2023) · Clinical Child and Family Psychology Review · doi:10.1007/s10567-022-00423-2 · Cited by 157
Self-citationWhat Value Do Journal Whitelists and Blacklists Have in Academia?
Teixeira da Silva, J. A., & Tsigaris, P. (2018) · The Journal of Academic Librarianship · doi:10.1016/j.acalib.2018.09.017 · Cited by 89
Causal and Associational Language in Observational Health Research: A Systematic Evaluation
Haber, N., Wieten, S. E., Rohrer, J. M., Arah, O. A., Tennant, P. W. G., Stuart, E. A., Murray, E. J., Pilleron, S., Lam, S. T., Riederer, E., et al. (2022) · American Journal of Epidemiology · doi:10.1093/aje/kwac137 · Cited by 113
Self-citationIshizumi, A., Kolis, J., Abad, N., Prybylski, D., Brookmeyer, K. A., Voegeli, C., Wardle, C., & Chiou, H. (2024) · The Lancet Public Health · doi:10.1016/s2468-2667(24)00031-8 · Cited by 79
Claims of causality in health news: a randomised trial
Adams, R. C., Challenger, A., Bratton, L., Boivin, J., Bott, L., Powell, G., Williams, A., Chambers, C. D., & Sumner, P. (2019) · BMC Medicine · doi:10.1186/s12916-019-1324-7 · Cited by 69
“However, many are based on correlational evidence [16, 17], where causal conclusions often prove incorrect [18].”
BackgroundMecozzi, L., Mambrini, M., Ruscitti, F., Ferrini, E., Ciccimarra, R., Ravanetti, F., Sverzellati, N., Silva, M., Ruffini, L., Belenkov, S., et al. (2020) · Scientific Reports · doi:10.1038/s41598-020-71293-3 · Cited by 75
Ruscitti, F., Ravanetti, F., Bertani, V., Ragionieri, L., Mecozzi, L., Sverzellati, N., Silva, M., Ruffini, L., Menozzi, V., Civelli, M., et al. (2020) · Frontiers in Pharmacology · doi:10.3389/fphar.2020.01117 · Cited by 66
Online Medical Misinformation in Cancer: Distinguishing Fact From Fiction
Teplinsky, E., Ponce, S. B., Drake, E. K., Garcia, A. M., Loeb, S., Londen, G. J. V., Teoh, D. G. K., Thompson, M. A., & Schapira, L. (2022) · JCO Oncology Practice · doi:10.1200/op.21.00764 · Cited by 64
Brown, A. W., Aslibekyan, S. W., Bier, D. M., da Silva, R. F., Hoover, A. W., Klurfeld, D. M., Loken, E., Mayo‐Wilson, E., Menachemi, N., Pavela, G., et al. (2021) · Critical Reviews in Food Science and Nutrition · doi:10.1080/10408398.2021.1985427 · Cited by 65
Tomassi, A., Falegnami, A., & Romano, E. (2024) · PLoS ONE · doi:10.1371/journal.pone.0303183 · Cited by 107
Key concepts for making informed choices
Aronson, J. K., Barends, E. G. R., Boruch, R. F., Brennan, M. L., Chalmers, I., Chislett, J., Cunliffe‐Jones, P., Dahlgren, A., Gaarder, M. M., Haines, A., et al. (2019) · Nature · doi:10.1038/d41586-019-02407-9 · Cited by 64
Mentor and peer support for early career researchers sharing research with academia and beyond
Merga, M. K., & Mason, S. (2021) · Heliyon · doi:10.1016/j.heliyon.2021.e06172 · Cited by 67
Wali, N., Agho, K., & Renzaho, A. M. N. (2019) · Systematic Reviews · doi:10.1186/s13643-019-1112-7 · Cited by 53
Guidance to best tools and practices for systematic reviews
Kolaski, K., Logan, L. R., & Ioannidis, J. P. A. (2023) · JBI Evidence Synthesis · doi:10.11124/jbies-23-00139 · Cited by 63
Bougioukas, K. I., Bouras, E., Avgerinos, K. I., Dardavessis, T., & Haidich, A. (2020) · Health Information & Libraries Journal · doi:10.1111/hir.12318 · Cited by 57
From social media to artificial intelligence: improving research on digital harms in youth
Mansfield, K. L., Ghai, S., Hakman, T., Ballou, N., Vuorre, M., & Przybylski, A. K. (2025) · The Lancet Child & Adolescent Health · doi:10.1016/s2352-4642(24)00332-8 · Cited by 39
Being honest with causal language in writing for publication
Thapa, D. K., Visentin, D. C., Hunt, G. E., Watson, R. A., & Cleary, M. L. (2020) · Journal of Advanced Nursing · doi:10.1111/jan.14311 · Cited by 40
Reducing Health Misinformation in Science: A Call to Arms
Swire‐Thompson, B., & Lazer, D. M. J. (2022) · The Annals of the American Academy of Political and Social Science · doi:10.1177/00027162221087686 · Cited by 49
“For instance, the exaggeration of causal language can begin in the scientific article itself (Haber et al. 2018), or in the press release, which scientists themselves approve.”
BackgroundParra, C. O., Bertizzolo, L., Schroter, S., Dechartres, A., & Goetghebeur, E. (2021) · BMJ Open · doi:10.1136/bmjopen-2020-043339 · Cited by 29
“We then extracted the sentences summarising the results and conclusions to highlight any causal claims.”
MethodMerga, M. K., & Mason, S. (2020) · Higher Education Research & Development · doi:10.1080/07294360.2020.1815662 · Cited by 39
“Both media exposure and ability to maintain control of the message were challenges that emerged in relation to the media, with valid concerns about accuracy of representation of research (e.g., Haber et al., 2018).”
BackgroundCausal Language in Observational Orthopaedic Research
Varady, N. H., Feroe, A. G., Fontana, M. A., & Chen, A. F. (2021) · Journal of Bone and Joint Surgery · doi:10.2106/jbjs.20.01921 · Cited by 33
Causal criteria: time has come for a revision
Olsen, J., & Jensen, U. J. (2019) · European Journal of Epidemiology · doi:10.1007/s10654-018-00479-x · Cited by 33
Oxman, M., Larun, L., Pérez‐Gaxiola, G., Alsaid, D., Qasim, A., Rose, C. J., Bischoff, K., & Oxman, A. D. (2021) · F1000Research · doi:10.12688/f1000research.52894.1 · Cited by 56
“commentaries), and two duplicates: one doctoral thesis that is a composite of other studies that we assessed(22) and one working paper(23) superseded by the final publication.(24) The third eligibility criterion—that the researcher had used at least one explicit, prespecified and generic quality criterion—was least of…”
Daly, T. P., Mastroleo, I., Gorski, D. H., & Epelbaum, S. (2020) · Metamedicine · doi:10.1007/s11017-020-09536-7 · Cited by 28
Using claims in the media to teach essential concepts for evidence-based healthcare
Oxman, M., Habib, L., Jamtvedt, G., Kalsnes, B., & Molin, M. (2020) · BMJ Evidence-Based Medicine · doi:10.1136/bmjebm-2020-111390 · Cited by 22
Ahmad, D., Mohanty, I., Hazra, A., & Niyonsenga, T. (2021) · BMC Pregnancy and Childbirth · doi:10.1186/s12884-021-03563-5 · Cited by 33
Guidance to best tools and practices for systematic reviews1
Kolaski, K., Logan, L. R., & Ioannidis, J. P. A. (2023) · Journal of Pediatric Rehabilitation Medicine · doi:10.3233/prm-230019 · Cited by 48
“This is indicated by recent reports of poor interpretability of systematic review results by readers [184–186] and misleading interpretations of the evidence related to the “spin” systematic review authors may put on their conclusions [50, 187].”
Lee, J. J. R., Srinivasan, R., Ong, C. S., Alejo, D. E., Schena, S., Shpitser, I., Sussman, M. S., Whitman, G. J. R., & Malinsky, D. (2022) · Journal of Thoracic and Cardiovascular Surgery · doi:10.1016/j.jtcvs.2022.08.012 · Cited by 22
Guidance to best tools and practices for systematic reviews
Kolaski, K., Logan, L. R., & Ioannidis, J. P. A. (2023) · British Journal of Pharmacology · doi:10.1111/bph.16100 · Cited by 33
“Unfortunately, communication to the general public often reflects intentional or non-intentional misrepresentation or “ spin ” of the research findings (Alnemer et al., 2015; Haber et al., 2018; Nascimento et al., 2021; Swetland et al., 2021).”
BackgroundGuidance to best tools and practices for systematic reviews
Kolaski, K., Logan, L. R., & Ioannidis, J. P. A. (2023) · BMC Infectious Diseases · doi:10.1186/s12879-023-08304-x · Cited by 27
“This is indicated by recent reports of poor interpretability of systematic review results by readers [184–186] and misleading interpretations of the evidence related to the “spin” systematic review authors may put on their conclusions [50, 187].”
Merga, M. K., & Mason, S. (2020) · Learned Publishing · doi:10.1002/leap.1365 · Cited by 22
“However, whether using online media or traditional media to communicate research, there are numerous potential pitfalls (Haber et al., 2018).”
BackgroundDoctoral education and early career researcher preparedness for diverse research output production
Merga, M. K., & Mason, S. (2020) · Journal of Further and Higher Education · doi:10.1080/0309877x.2020.1807477 · Cited by 21
“Media articles reporting on academic studies may ‘overstate strength of causal inference and inaccurately report key study characteristics and/or results’ (Haber et al. 2018, 22).”
BackgroundMoukarzel, S., Caduff, A., Rehm, M., del Fresno, M., Pérez‐Escamilla, R., & Daly, A. J. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph18126181 · Cited by 22
“In fact, they are known to contribute to public skepticism, as well as slow and inaccurate public health communications [28,29].”
BackgroundGuidance to Best Tools and Practices for Systematic Reviews
Kolaski, K., Logan, L. R., & Ioannidis, J. P. A. (2023) · JBJS Reviews · doi:10.2106/jbjs.rvw.23.00077 · Cited by 28
Chang, A., Schulz, P. J., Jiao, W., & Liu, M. T. (2021) · JMIR Public Health and Surveillance · doi:10.2196/26660 · Cited by 19
Causal theory error in college students’ understanding of science studies
Seifert, C. M., Harrington, M. B., Michal, A. L., & Shah, P. (2022) · Cognitive Research Principles and Implications · doi:10.1186/s41235-021-00347-5 · Cited by 29
“Because media reports typically describe behavioral studies (e.g., Bleske-Rechek et al., 2018; Haber et al., 2018), psychology students may fare better (Hall & Seery, 2006).”
BackgroundBusam, B., & Solomon-Moore, E. (2021) · Health Communication · doi:10.1080/10410236.2021.1985859 · Cited by 16
Causal reasoning with causal graphs in educational technology research
Weidlich, J., Hicks, B., & Drachsler, H. (2023) · Educational Technology Research and Development · doi:10.1007/s11423-023-10241-0 · Cited by 25
“Systematic efforts to this effect can be found in Haber et al. (2018), where the causal language of academic and media articles is contrasted with the actual strength of inference.”
Mugisha, M., Nyirazinyoye, L., Simbi, C. M. C., Chesire, F., Ssenyonga, R., Oxman, M., Nsangi, A., Semakula, D., Rose, C. J., Moberg, J., et al. (2023) · Journal of Evidence-Based Medicine · doi:10.1111/jebm.12551 · Cited by 12
Bernardeau, C., Revol, B., Salvo, F., Fusaroli, M., Raschi, E., Cracowski, J., Roustit, M., & Khouri, C. (2025) · Drug Safety · doi:10.1007/s40264-025-01524-x · Cited by 18
“However, after retrieving all the citing studies, we decided to simplify the gradation to a four-level scale, given the difficulty of differentiating certain causal gradations and the low number of citations in some of them (Table 1) [30, 35, 36].”
Ito, C., Al‐Hassany, L., Kurth, T., & Glatz, T. (2025) · Neurology · doi:10.1212/wnl.0000000000210171 · Cited by 15
Introduction to the Symposium: Causal Inference and Public Health
Aiello, A. E., & Green, L. W. (2019) · Annual Review of Public Health · doi:10.1146/annurev-publhealth-111918-103312 · Cited by 11
“The “but” is usually a qualification, sometimes but not always, placed by science journalists (9), given their reading beyond the “breaking news” press releases.”
BackgroundElvsaas, I.-K. O., Garnweidner-Holme, L. M., Habib, L., & Molin, M. (2023) · JMIR Formative Research · doi:10.2196/44831 · Cited by 12
Winters, M., Larsson, A., Kowalski, J. H., & Sundberg, C. J. (2019) · PLoS ONE · doi:10.1371/journal.pone.0217295 · Cited by 13
Nordheim, L. V., Pettersen, K. S., Espehaug, B., Flottorp, S. A., & Guttersrud, Ø. (2019) · BMJ Open · doi:10.1136/bmjopen-2018-028781 · Cited by 12
Expert quotes and exaggeration in health news: a retrospective quantitative content analysis
Bossema, F. G., Burger, P., Bratton, L., Challenger, A., Adams, R. C., Sumner, P., Schat, J., Numans, M. E., & Smeets, I. (2019) · Wellcome Open Research · doi:10.12688/wellcomeopenres.15147.2 · Cited by 13
Løvlie, A. S., Waagstein, A., & Hyldgard, P. (2023) · Digital Journalism · doi:10.1080/21670811.2023.2193344 · Cited by 14
“Research within health communication has demonstrated that audiences struggle to assess unreliable health claims in the media (Boutron et al., 2019; Haber et al., 2018; WalshChilders et al., 2018).”
BackgroundCan ChatGPT Understand Causal Language in Science Claims?
Kim, Y., Guo, L., Yu, B., & Li, Y. (2023) · Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis (WASSA) · doi:10.18653/v1/2023.wassa-1.33 · Cited by 24
“Manual fact-checking of causal claims in academic publications, news and social media posts also demonstrated evidence of prevalent exaggeration when reporting causal findings (Cofield et al., 2010; Sumner et al., 2014; Haber et al., 2018).”
BackgroundAlvarez-Vargas, D., Braithwaite, D. W., Lortie‐Forgues, H., Moore, M. M., Castro, M., Wan, S., Martin, E. A., & Bailey, D. H. (2023) · PLoS ONE · doi:10.1371/journal.pone.0286403 · Cited by 18
“Such recommendations are often given in ways that imply stronger evidence for the relevant causal pathway than warranted by the method [16] consistent with the hypothesis that the use of causally ambiguous statistical language may be an example of “strategic ambiguity” [19, 20].”
BackgroundSun, Z., Cang, J., Ruan, Y., & Zhu, D. (2019) · The International Journal of Health Planning and Management · doi:10.1002/hpm.2894 · Cited by 8
DAG With Omitted Objects Displayed (DAGWOOD): a framework for revealing causal assumptions in DAGs
Haber, N., Wood, M. E., Wieten, S. E., & Breskin, A. (2022) · Annals of Epidemiology · doi:10.1016/j.annepidem.2022.01.001 · Cited by 12
“While strength of causal inference review inherently relies on assessments from the reviewers (16,17), DAGWOODs can help make those assessments more systematic and replicable.”
BackgroundSelf-citationOkonofua, E. S., Atikpo, E., Lasisi, K. H., Ajibade, F. O., & Idowu, T. E. (2021) · Environmental Technology · doi:10.1080/09593330.2021.1992508 · Cited by 9
Development and validation of a quality assessment tool to assess online nutrition information
Ellis, C. H., Moore, J. B., Ho, P., & Evans, C. (2023) · Digital Health · doi:10.1177/20552076231187249 · Cited by 10
Application of Micro-CT in Soft Tissue Specimen Imaging
Roque‐Torres, G. D. (2019) · Springer eBooks · doi:10.1007/978-3-030-16641-0_10 · Cited by 7
Mixed messages: most spinal pain and osteoarthritis observational research is unclear or misaligned
Gleadhill, C. P., Lee, H., Kamper, S. J., Cashin, A. G., Hansford, H. J., Traeger, A. C., da Silva, P. V., Nolan, E., Davidson, S. R. E., Wilczynska, M., et al. (2023) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2023.01.005 · Cited by 7
Timing, modifications, and tenor of firms' mandatory R&D disclosures: The role of competition
Rathee, C., Malik, S., & Salandra, R. (2024) · Research Policy · doi:10.1016/j.respol.2024.105112 · Cited by 9
Sánchez‐Iglesias, I., González-Castaño, M., & Fernández, A. J. M. (2021) · Religions · doi:10.3390/rel12121075 · Cited by 7
Exploring critical media health literacy (CMHL) in the online classroom
Squires, L. A., Peters, A. M. F., & Rohr, L. E. (2023) · Journal of Media Literacy Education · doi:10.23860/jmle-2023-15-1-5 · Cited by 10
Ríos, I. V., Sarmiento, A. I. P., & Castaño, A. F. (2021) · Floresta e Ambiente · doi:10.1590/2179-8087-floram-2020-0009 · Cited by 5
Guidance to best tools and practices for systematic reviews
Kolaski, K., Logan, L. R., & Ioannidis, J. P. A. (2023) · Acta Anaesthesiologica Scandinavica · doi:10.1111/aas.14295 · Cited by 11
“Unfortunately, communication to the general public often reflects intentional or non-intentional misrepresentation or “ spin ” of the research findings (Alnemer et al., 2015; Haber et al., 2018; Nascimento et al., 2021; Swetland et al., 2021).”
BackgroundThe influence of viewing a headline about ecstasy/Molly adulteration on future intentions to use
Palamar, J. J., Acosta, P., & Cleland, C. M. (2019) · Journal of Substance Use · doi:10.1080/14659891.2019.1664660 · Cited by 5
“Although research study results are not always accurately portrayed by media (Haber et al., 2018), some coverage leads to large effects on health-related behavior.”
BackgroundLenguaje Causal en Modelos de Ecuaciones Estructurales
Sánchez‐Iglesias, I., Aguayo‐Estremera, R., Miguel-Álvaro, A., & Paniagua, D. (2022) · Revista Iberoamericana de Diagnóstico y Evaluación - e Avaliação Psicológica · doi:10.21865/ridep66.5.03 · Cited by 5
Causal claims about correlations reduced in press releases following academic study of health news
Bratton, L., Adams, R. C., Challenger, A., Boivin, J., Bott, L. A., Chambers, C., & Sumner, P. (2020) · Wellcome Open Research · doi:10.12688/wellcomeopenres.15647.1 · Cited by 4
Panuganti, P. L., Bazzano, L., & Ley, S. H. (2021) · Elsevier eBooks · doi:10.1016/b978-0-12-815350-5.00009-7 · Cited by 7
Plant–Microbe Interactions in Bioremediation of Toxic Wastes in Tropical Environment
Udebuani, A. C., Ukachukwu, C. O., Onweremadu, E. U., & Okoli, I. C. (2021) · Springer eBooks · doi:10.1007/978-981-16-0602-1_9 · Cited by 4
Navigating and Leveraging Social Media
Chiang, A. L. (2021) · Gastrointestinal Endoscopy Clinics of North America · doi:10.1016/j.giec.2021.05.006 · Cited by 5
Why Science Succeeds, and Sometimes Doesn’t
Baron, J. A. (2019) · Cambridge University Press eBooks · doi:10.1017/9781108684354.004 · Cited by 3
Kamjan, S., de Groene, D., van den Hurk, Y., Zidarov, P., Elenski, N., Patterson, W. P., & Çakırlar, C. (2021) · Journal of Archaeological Science Reports · doi:10.1016/j.jasrep.2021.102789 · Cited by 5
Gerrits, R. G., van den Berg, M. J., Kunst, A. E., Klazinga, N., & Kringos, D. S. (2021) · PLoS ONE · doi:10.1371/journal.pone.0248753 · Cited by 4
“Such knowledge is currently inadequate [10, 14].”
Evidence‐based reasoning: Results from an intervention
Du, H., & List, A. (2024) · Applied Cognitive Psychology · doi:10.1002/acp.4238 · Cited by 2
Mugisha, M., Nyirazinyoye, L., Kayiranga, D., Simbi, C. M. C., Chesire, F., Ssenyonga, R., Oxman, M., Nsangi, A., Rose, C. J., Moberg, J., et al. (2025) · Trials · doi:10.1186/s13063-025-08779-w · Cited by 2
“This is in response to overwhelming misinformation that is available to the public through one-to-one communication, mass and social media, and other sources of communication [1–4].”
Sarmiento, A. I. P., Zárate, D. A., Peláez, J. J. Z., et al. (2019) · CERNE · doi:10.1590/01047760201925022596 · Cited by 2
Barakat, N., Novicoff, W. M., Werner, B. C., & Browne, J. A. (2022) · The Journal of Arthroplasty · doi:10.1016/j.arth.2022.12.008 · Cited by 2
Adetitun, D. O. (2020) · Springer eBooks · doi:10.1007/978-3-030-35296-7_12 · Cited by 2
Brown, N. (2019) · University of Groningen research database (University of Groningen / Centre for Information Technology) · doi:10.33612/diss.99196913 · Cited by 2
The Diffusion of Causal Language in Social Networks
Shi, Z., & Morstatter, F. (2024) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v18i1.31399 · Cited by 6
“In more recent literature, causal inference has emerged as a crucial research focus, where causal claims are presented and analyzed from different perspectives (Holland 1986; Marini and Singer 1988; Pearl 2009; Pearl and Mackenzie 2018; Haber et al. 2018; Wright and Augenstein 2021).”
BackgroundThe “bread and butter” of science: understanding how scientists communicate uncertainties
Roney, C., Egelhofer, J. L., & Lecheler, S. K. (2026) · Humanities and Social Sciences Communications · doi:10.1057/s41599-026-07026-0 · Cited by 4
“Initial research suggests that uncertainty is already underrepresented in scienti fi c articles (Haber et al. 2018) and later in press releases that are incorporated into news coverage (Sumner et al. 2014, 2016).”
Elliott-Sale, K. J., Burden, R. J., Ackerman, K. E., Burke, L. M., Constantini, N., Hackney, A. C., Harris, R., Impellizzeri, F. M., de Jonge, X. J., Lebrun, C. M., et al. (2026) · Sports Medicine · doi:10.1007/s40279-026-02442-3 · Cited by 3
“In contrast, much of the existing literature over-relies on associations that are misinterpreted as evidence of causation, often without employing the necessary methodological frameworks [54, 55, 65].”
Sánchez‐Iglesias, I., & Graña, D. F. (2023) · Schizophrenia Research · doi:10.1016/j.schres.2023.07.013 · Cited by 1
Ganderton, P. (2022) · Oxford University Press eBooks · doi:10.1093/obo/9780199830060-0238 · Cited by 1
The State of Sociology: Evidence from Dissertation AbstractsPreprint
Altman, M., & Cohen, P. N. (2024) · SocArXiv (OSF Preprints) · doi:10.31235/osf.io/a8uyp · Cited by 2
2020 vision? A retrospective study of time-bound curative claims in British and Irish newspapers
McKechnie, D. G., Rashid, M. A., & McCartney, M. (2022) · British Journal of General Practice · doi:10.3399/bjgp22x719261
Time Analysis for Sharing on Twitter Social Network by Using Neural Networks
Müngen, A. A., & Taş, İ. Ç. (2022) · International Informatics and Software Engineering Conference (IISEC) · doi:10.1109/iisec56263.2022.9998207
“Because Twitter is a microblog with many users, it is a widespread data source for data mining and social networking applications [6].”
BackgroundThe “Why” in Mental Health, Stigma, and Addictive Behaviors: Causal Inferences in Applied Settings
Sánchez‐Iglesias, I. (2023) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph20206915 · Cited by 1
“…the applied areas where incorrect causal inferences can yield adverse consequences in clinical practice and public health policies [13,14]. highlighting the applied areas where incorrect causal inferences can yield adverse consequences in clinical practice and public health policies [13,14].”
BackgroundBarakat, N., Werner, B. C., Arney, M. M., Novicoff, W. M., Browne, J. A., & Goodloe, J. B. (2024) · Journal of Orthopaedics · doi:10.1016/j.jor.2024.12.020
Santé de la femme, infodémie et mésinformation
Bertrand-Ferrandis, C. (2025) · Journal de l hypnose et de la santé intégrative · doi:10.3917/jhsi.031.0077
Dahmar, N., Moumene, R., Abdelkrim, R., et al. (2025) · South Florida Journal of Development · doi:10.46932/sfjdv6n8-001
A Nation of Curators: Educating Students to be Critical Consumers and Users of Online Information
Greene, J. A., Cartiff, B. M., Duke, R. F., & Deekens, V. M. (2019) · Emerald Group Publishing Limited eBooks · doi:10.1108/978-1-64113-853-620251013 · Cited by 9
Wang, K., Wei, C., & Labrecque, J. A. (2026) · BMJ Open · doi:10.1136/bmjopen-2025-109205 · Cited by 1
“There are also published studies in the field of health communication [12], media [13, 14], and computer science [15] that note this disconnect between the intention and expression of causation.”
Nazir, U., Kamarudin, N. S., & Majid, M. B. A. (2025) · International Conference on Interdisciplinary Computer Science and Engineering (ICICSE) · doi:10.1109/icicse67247.2025.11390793
“Particularly when randomised controlled trials (RCTs) are impractical or unethical, as is often the case in studies involving mental health and social media, causal inference is a vital method in research [37], [38].”
Tskhay, A., Longo, C., Moldakozhayev, A., Kang, N., Greenwood, C. M. T., Behruzi, R., Kubow, S. J., & Schuster, T. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.03.02.709039
Alzheimer’s disease : epistemology, ethics, innovation
Daly, T. P. (2021) · theses.fr (ABES) · doi:10.70675/6e99838az7dd4z402fz98fdzeca072d3ad13 · Cited by 2
Merga, M. K., Roni, S. M., & Dera, J. (2026) · Journal of Library Administration · doi:10.1080/01930826.2026.2695369
“Text simplification involves inevitable loss of nuance even when conducted by humans, such as when research is reported in the media (Haber et al., 2018).”
Metadata
- Authors
- Haber, N., Smith, E. R., Moscoe, E., Andrews, K., Audy, R., Bell, W., Brennan, A. T., Breskin, A., Kane, J. C., Karra, M., McClure, E. S., Suarez, E. A., & on behalf of the CLAIMS research team
- Affiliations
- University of North Carolina at Chapel Hill (US)
- Boston Children's Hospital (US)
- Harvard University (US)
- Tufts University (US)
- Boston University (US)
- Johns Hopkins University (US)
- Corresponding author
- Haber, Noah
- ORCID
- Venue
- PLoS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2018-05-30
- Volume
- 13
- Issue
- 5
- Pages
- e0196346
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC5976147
- OpenAlex
- W2805549342
- Cited by
- 97
- References
- 45
- Citations by year
- 2026: 5 · 2025: 7 · 2024: 7 · 2023: 19 · 2022: 11 · 2021: 20 · 2020: 11 · 2019: 16 · 2018: 1
- FWCI
- 4.12
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- public-domain
- creativecommons.org/publicdomain/zero/1.0/ (vor)
- Locations
- PLoS ONE (publishedVersion, public-domain)
- Europe PMC (PubMed Central) (publishedVersion, public-domain)
- PubMed Central (publishedVersion, public-domain)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (acceptedVersion)
- OpenBU (Boston University) (acceptedVersion, cc-by)
- Digital Access to Scholarship at Harvard (DASH) (Harvard University) (submittedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Figshare (submittedVersion, cc-by)
- Funders
- Eunice Kennedy Shriver National Institute of Child Health and Human Development (P2CHD050924)
- Topics
- Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Health Sciences › Health Professions › General Health Professions › Health Literacy and Information Accessibility
- Keywords
- causal inference, quality of evidence, causal language, health misinformation, media coverage of research, social media sharing, Facebook, Twitter, research generalizability, confounding, health news
- MeSH terms
- Communication, Communications Media, Humans, Language, Causality, Internet, Biomedical Research, Social Media
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.