CoChat
Sign Up
Cited by 4Academic Publishing and Open AccessJournalism and Media StudiesMedia Studies and Communication

Are media reports of published research an accurate representation of the research?

Open access
Yu, Jingqi, Yeung, Catherine, Soman, Dilip·2026·Behavioural Public Policy
DOI 10.1017/bpp.2025.5
Open this paper in CoChat — free accountOpen PDF

Abstract

It is perhaps uncontroversial to claim that behavioral science research is playing an increasingly important role in practice. However, practitioners largely rely on media reports rather than original research articles to learn about the science. Do these media reports contain all the information needed to understand the nuances of the research? To assess this question, we develop a set of rubrics to evaluate the fidelity of the media report to the original research article. As an illustration, we apply these rubrics to a sample of media reports based on several research articles published in one journal and identify common patterns, trends, and pitfalls in media presentations. We find preliminary evidence of low fidelity in presenting participant characteristics, contextual elements, and limitations of the original research. The media also appear to misreport correlational evidence as causal and sometimes miss acknowledging the hypothetical nature of evidence when hypothetical scenarios were used as the sole basis of conclusions. Furthermore, the media often present broad conclusions and personal opinions as directly backed by scientific evidence. To support more discerning consumption of behavioral insights from media sources, we propose a checklist to guide practitioners in evaluating and using information from media sources.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1017/bpp.2025.5

Cites

34 of 34 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • There is substantial opportunity to scale fidelity detection from prior work.

    “With recent advances in LLMs, there is substantial opportunity to scale fidelity detection from prior work 88”

    From Quantifying the prevalence and impact of overreaching causal claims in social science (2026) · cites this paper as [88]

    In “Fidelity issues in media report with illustrative examples” (page unavailable), the paper says its manual fidelity coding was resource-intensive and suggests that suitable AI systems might make it more scalable. It does not discuss recent LLM advances, and its suggestion is more tentative than the citing sentence’s claim of substantial opportunity. (LLM, conf=0.85)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

1 of the 35 references couldn’t be matched to a known paper.

Research works printed but not linked: 4

  1. Luca, M. and M. H. Bazerman (2021), The Power of Experiments: Decision Making in a Data-driven World, MIT Press
  2. Soman, D., and C. Yeung (Eds.) (2021), The Behaviorally Informed Organization, Toronto: University of Toronto Press
  3. Thaler, R. H., and C. R. Sunstein (2021), Nudge: The Final Edition, New York: Yale University Press
  4. Yu, J. and B. Feng (2024). Where do behavioural practitioners learn about the science. Toronto, Canada: Behavioral Economics in Action at Rotman (BEAR) Report series. Cite this article: Yu J, Yeung C and Soman D (2025), ' Are media reports of published research an accurate representation of the research?' , Behavioural Public Policy, 1-21. https://doi.org/10.1017/bpp.2025. · DOI

Cited by

Sorted by citations

Related

Authors
Yu, J., Yeung, C., & Soman, D.
Affiliations
  • University of Toronto (CA)
  • Chinese University of Hong Kong (HK)
Corresponding author
Yeung, Catherine
ORCID
Venue
Behavioural Public Policy
Publisher
Cambridge University Press (CUP)
Published
2025-04-30
Volume
10
Issue
3
Pages
481-501
ISSN
  • 2398-063X (ISSN-L)
  • 2398-0648
DOI
10.1017/bpp.2025.5
OpenAlex
W4409971529
Cited by
4
References
35
Citations by year
2026: 4
FWCI
4.02
Type
journal article
Language
en
Open access
Licence
Locations
Behavioural Public Policy (publishedVersion, cc-by)
Funders
Social Sciences and Humanities Research Council of Canada
Topics
  • Social Sciences › Decision Sciences › Information Systems and Management › Academic Publishing and Open Access
  • Social Sciences › Communication › Journalism and Media Studies
  • Social Sciences › Communication › Media Studies and Communication
Keywords
media coverage of research, research accuracy, behavioral science, behavioural insights, research limitations, hypothetical scenarios, assessment rubrics

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review