CoChat

Lies, damn lies, and statistics*

Open access
Yentis, S. M. · 2012 · Anaesthesia
Cited by 26

The DOI matches its Crossref record; 3 of the 3 references it cites were found in a registry or an index.

Abstract

In April 2010, this journal carried an editorial entitled 'Fraud or flawed: adverse impact of fabricated or poor quality research' [1]. In it, Moore et al. described the difficulty in telling when trial data were fabricated, and how systematic reviews could help by demonstrating aberrations that appeared when data from different studies were presented together. Moore et al. used data from one researcher in Japan to illustrate their point, referring to a previous comment on this researcher's data that was published 10 years earlier, with the beautifully understated title 'Reported data on granisetron and postoperative nausea and vomiting by Fujii et al. are incredibly nice!' [2]. The Journal received a number of responses to Moore et al.'s editorial, amongst them one from a reader who bemoaned the fact that the evidence base remained distorted by that researcher's work, and challenging anaesthetic journal editors to do something about it. This all happened just a few months after an unprecedented international collaboration between anaesthetic journal editors and publishers, that had led to retraction of almost 90 published papers (including six from this journal) by Joachim Boldt [3]. With this in mind, I counter-challenged the correspondent to perform an analysis of Fujii's work, more in-depth than that of Kranke etal.'s. If suggestive that Fujii's data should be removed from the evidence base, this analysis might be used by the same group of editors to confront Fujii and/or his institution(s). So as the correspondent started his work, his letter remained unpublished, and the international group of editors started to discuss how the case might be handled. That correspondent was John Carlisle, and in this issue of the Journal he presents his analysis, some 19 months, 18 reiterations, two consultations with the Committee on Publication Ethics (COPE) and three statisticians later [4]. This is the piece of work that was going to be presented as evidence to Fujii's institutions – but in the event, it was never used for this purpose, for as we reached the final stages of the manuscript's preparation, there were rapid developments in Japan. In response to questions raised over a completely separate and coincidental concern that arose in late 2011, Fujii's institution (Toho University, Tokyo) set up an investigation that led to the dismissal of Fujii in February of this year for lack of ethical approval (see Anaesthesia homepage: http://onlinelibrary.wiley.com/journal/10.1111/%28ISSN%291365-2044). Carlisle's analysis [4] is an extraordinary piece of work. Necessarily based on complicated calculations, its message is nevertheless easily accessible to the reader through its every graph and p value, that clearly show the degree to which Fujii's data deviate from what would be expected. In an accompanying editorial [5], Pandit provides an explanation of the methods used and their context. Every submission to Anaesthesia is screened using specific software for overlap with other published work before it is reviewed, to detect plagiarism [6]; could Carlisle's methods similarly form the basis of a screening method for detecting fabrication? I have described in these pages not so long ago the murky world of research misconduct [6]. One question I didn't cover in any depth is whose responsibility it is to look into suspect cases. This is explored by Wager, in another accompanying editorial [7]. As she points out, institutions don't always do as they should when approached by editors with concerns – though the institutions in both the Boldt case and the Fujii case should be congratulated for their investigations. As Wager describes, we all have responsibility: from readers through editors, reviewers and publishers to co-investigators, institutions and even governments. From an editorial point of view, being able to access COPE for advice and support, as well as other groups such as the World Association of Medical Editors (see http://www.wame.org/), is enormously useful, and such organisations have a valuable role in keeping up the pressure on all parties, and at all levels, reminding them of their obligations. Medicine – and, it seems, anaesthesia in particular – has seen a flurry of research misconduct scandals recently, with potentially serious implications for the wellbeing of patients who might have received treatments that were ineffective or harmful, and/or been denied those that were beneficial. A further, and possibly longer-lasting, harm is the damage to public trust that results from such cases and the publicity around them. Our specialty is still reeling from the Reuben [8] and Boldt cases [3]; the last thing we needed was another, even bigger, case. We can only hope that the rule that bad things come in threes holds true. Such cases are deeply troubling, raising difficult questions about the framework within which research is done, the motivation and support of those who are prepared to risk so much personally

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1111/j.1365-2044.2012.07133.x

Cites

3 of 3 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 3 references

Research works printed but not linked: 5

  1. 10.1213/00000539-200004000-00053 · DOI · Paper page
  2. Editors‐in‐Chief Statement Regarding IRB Approval of Clinical Trials by Joachim Boldt
  3. 10.1111/j.1365-2044.2010.06557.x · DOI · Paper page
  4. 10.1111/j.1365-2044.2012.07132.x · DOI
  5. 10.1213/ane.0b013e3181a16846 · DOI · Paper page

Cited by

See all 26

Metadata

Authors
Yentis, S. M.
Venue
Anaesthesia
Publisher
Wiley
Published
2012-03-08
Volume
67
Issue
5
Pages
455-456
ISSN
  • 0003-2409 (ISSN-L)
  • 1365-2044
OpenAlex
W1789442286
Cited by
26
References
3
FWCI
11.46
Type
journal article
Language
en
Open access
bronze

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Lies, damn lies, and statistics*. https://app.cochat.ai/papers/10.1111/j.1365-2044.2012.07133.x
The paper
Yentis, S. M. (2012). Lies, damn lies, and statistics*. Anaesthesia. https://doi.org/10.1111/j.1365-2044.2012.07133.x

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.