Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 14 of the 15 references it cites were found in a registry or an index.

A "take with a grain of salt" label for Holiday Review articles

Open access
Sibbald, B. · 2008 · Canadian Medical Association Journal

Abstract

[A CMAJ Deputy Editor responds:] We thank Christopher Naugler for bringing this to our attention. Normally, spoof science does not appear in PubMed searches. However, Naugler's findings show that the system we have been using is neither inclusive nor foolproof. In the future, we will include a

Overview

CoChat Summary

TL;DR

In a brief CMAJ commentary, Sibbald reports that a Google Scholar search found at least 4 instances where authors cited Holiday Review spoof articles as if they were real studies. The author suggests adding a disclaimer to evidence-free Holiday Review abstracts or titles if they remain indexed in MEDLINE.

Key findings

  • A Google Scholar search for citations of CMAJ Holiday Review articles published 1999–2006 found at least 4 instances where citing authors appeared to mistake evidence-free articles for real studies.
  • Examples included a citation of a case report whose patient was a cartoon character, a citation of the author’s own “completely evidence-free paper,” and 2 other citations of Holiday Review spoof studies.
  • The author proposes that if Holiday Review articles continue to be indexed in MEDLINE, each evidence-free abstract should include a note stating the article is for entertainment purposes only and is not a real study.
  • Additional information states that in future, CMAJ will include a disclaimer in the titles of its Holiday Review science articles.

Methods

  • Design: brief commentary/letter describing an informal citation search.
  • Data: Google Scholar citations of Holiday Review articles published in CMAJ from 1999–2006; these citations were then reviewed.
  • Analysis: qualitative review of whether citing authors understood the articles as tongue-in-cheek versus mistakenly treating them as evidence-based studies; no formal statistical analysis is reported.

Limitations

  • The paper states no formal limitations section.
  • Evident from the text: the search is described only briefly, with no reproducible search strategy, search date, number of citations screened, or inclusion criteria; the finding is reported as “at least 4 instances,” not a systematic count.
  • Evident from the text: the article is a short commentary/letter, so it does not provide a systematic assessment of MEDLINE indexing or citation practices.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1503/cmaj.1080031

Cites

14 of 15 references verified0 citation errors · 1 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

0 Valid · 0 Not valid · 1 Not checked · 5 Own results and interpretations

  • Probably not a substantial claim. 1
  • It may state two separate claims as one. 2

Create a free account to see these claims.

Citations

Cites

Research works printed but not linked: 1

  1. Mangion K. The stethoscope. Malta Med J 2007;19: 41-4.

Cited by

See all 1

Metadata

Authors
Sibbald, B.
Affiliations
Dai Nippon Printing (Japan) (JP)
Corresponding author
Sibbald, B.
Venue
Canadian Medical Association Journal
Published
2008-04-22
Volume
178
Issue
9
Pages
1186-1186
ISSN
  • 0820-3946 (ISSN-L)
  • 1488-2329
OpenAlex
W1968308010
Cited by
1
References
14
FWCI
1.25
Type
journal article
Language
en
Open access
Locations
  • Canadian Medical Association Journal (publishedVersion)
  • Europe PMC (PubMed Central) (publishedVersion)
  • PubMed Central (publishedVersion)
Funders
  • Pfizer
  • Ministry of Education, Science and Technology
Topics
  • Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
  • Social Sciences › Library and Information Sciences › Publishing and Scholarly Communication
  • Social Sciences › Library and Information Sciences › Academic Writing and Publishing
Keywords
medical journal publishing

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

A language model drew the claims from the paper on 2026-11-16, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-11-16 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.