The DOI matches its Crossref record; 11 of the 11 references it cites were found in a registry or an index.
Errors and Integrity in Seeking and Reporting Apparent Research Misconduct
Abstract
THE responsible conduct of research is the bedrock on which the scientific enterprise rests. Scientific integrity is indispensable for preserving the public trust and the trust of the scientific community, protecting those who participate in research (investigators and research subjects), and safeguarding the health and safety of patients who depend on scientific knowledge for their diagnosis and care. When the scientific content of Anesthesiology and the integrity of our authors are publically questioned, it is the journal’s responsibility to evaluate such claims and to inform our readers about the claims, the pertinent issues, and the results of our evaluation of the claims.Research misconduct has become an all too familiar stain on the research and clinical landscape, and it is occurring with increasing frequency and public awareness. Variably complicit in research misconduct, explicitly and/or implicitly, are investigators, authors, sponsors, academic institutions, and journals. Revelations of scientific misconduct have been punctuated by several well-known instances of gross fabrication and falsification in our specialty. Never tolerable, misconduct is perhaps even more egregious when occurring in randomized clinical trials, which are a foundational element informing clinical practice.The scientific community is ever watchful for fabrication and falsification—journal editors included. It is the bane of our existence. Were there only some valid algorithm or software that could detect fabrication and falsification (akin to that which detects duplication and plagiarism with remarkable alacrity and accuracy).A recent issue of the journal Anaesthesia contains an article by Dr. John Carlisle on “data fabrication and other reasons” for nonrandom sampling in randomized clinical trials in the specialty of anesthesia and in two general medical journals.1 It was accompanied by an editorial that celebrates the article, the statistical method used, and the application thereof to detecting scientific misconduct.2 We were particularly concerned because the Carlisle article identified 12 research investigations published in Anesthesiology with purportedly problematic results (at a P value cut-off less than 0.0001) that “might benefit from further investigation”...“to correct and if necessary retract,”1 and which might be “corrupted.”2In brief, the Carlisle Method3 evaluates the baseline variables (e.g., age, weight) in a manuscript and identifies those with more or less balance than would be expected by chance. Less balance is akin to data not being consistent with random sampling. The Carlisle Method therefore deems such data suspicious.1,4,5The Method assumes that every individual baseline demographic variable in a randomized controlled trial (i.e., what is typically presented in a manuscript’s “table 1”) is randomly assigned across treatment groups. Using this assumption, the Method then examines the observed distribution of individual P values that result from testing the null hypothesis that the randomized groups do not differ. The observed distributions are then compared to the expected distribution that should be observed when the null hypothesis is true (e.g., uniform distribution).With some degree of equipoise, the Carlisle article attributes observed deviations from these expected distributions to typographical error, unintentional error, correlation, stratified allocation, poor methodology, fabricated data, or fraud.1 It politely suggests that the articles in question “might benefit from further investigation,” but then asserts that “association of extreme distributions with trial retraction suggests that further investigation of uncorrected unretracted trials and their authors will result in most trials being corrected and some retracted.”1 Put more succinctly, a probability is extrapolated to a fact, which then becomes the basis for judgement and action (vide infra).The Carlisle Method has been touted as the long-sought “Holy Grail” for detecting fabrication and falsification in scientific publishing. Anaesthesia “can rightly claim with vicarious pride that one of its own, John Carlisle, is at the forefront of this effort,” and that Carlisle’s “statistical exposé…made the research world stand up and take notice” in an effort to prevent publication of fraudulent material.2Anaesthesia declared that it “has decided to screen all randomized trials submitted to the journal using the Carlisle Method” and “will (summarily) reject any that fall foul due to suspicious data that are not consistent with random sampling.”3 Furthermore, it “hopes that other journals will follow suit and also screen submissions.”3 Loadsman and McCulloch proclaim that “a strong argument could be made that every journal in the world now needs to apply Carlisle’s method to all the randomized clinical trials they’ve ever published.”2 All these emphatic exhortations are made while acknowledging that “clearly there is still a lot of work to
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The test makes the assumption that baseline variables are independent”
From Integrity of randomized clinical trials: Performance of integrity tests and checklists requires assessment (2023) · cites this paper as [68]
In the citing paper
“but they are usually correlated.68,71,72”
From Integrity of randomized clinical trials: Performance of integrity tests and checklists requires assessment (2023) · cites this paper as [68]
Citations
Cites
Testing for baseline balance in clinical trials
Senn, S. (1994) · Statistics in Medicine · doi:10.1002/sim.4780131703 · Cited by 503
Covariate Balance in Simple, Stratified and Clustered Comparative Studies
Hansen, B. B., & Bowers, J. (2008) · Statistical Science · doi:10.1214/08-sts254 · Cited by 414
Covariate imbalance and random allocation in clinical trials
Senn, S. J. (1989) · Statistics in Medicine · doi:10.1002/sim.4780080410 · Cited by 279
Reproducibility: A tragedy of errors
Allison, D. B., Brown, A. W., George, B. J., & Kaiser, K. A. (2016) · Nature · doi:10.1038/530027a · Cited by 255
Carlisle, J. B. (2017) · Anaesthesia · doi:10.1111/anae.13938 · Cited by 229
Carlisle, J. B., Dexter, F., Pandit, J. J., Shafer, S. L., & Yentis, S. M. (2015) · Anaesthesia · doi:10.1111/anae.13126 · Cited by 87
Simple randomization did not protect against bias in smaller trials
Nguyen, T., Collins, P. G. S., Lamy, A. L., Devereaux, P. J., Daurès, J., Landais, P., & Manach, Y. L. (2017) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.02.010 · Cited by 77
Evidence for non‐random sampling in randomised, controlled trials by Yuhji Saitoh
Carlisle, J. B., & Loadsman, J. A. (2017) · Anaesthesia · doi:10.1111/anae.13650 · Cited by 64
Klein, A. A. (2016) · Anaesthesia · doi:10.1111/anae.13731 · Cited by 21
Widening the search for suspect data – is the flood of retractions about to become a tsunami?
Loadsman, J. A., & McCulloch, T. J. (2017) · Anaesthesia · doi:10.1111/anae.13962 · Cited by 21
Cited by
Khan, K. S., Fawzy, M., & Chien, P. F. W. (2023) · International Journal of Gynecology & Obstetrics · doi:10.1002/ijgo.14837 · Cited by 20
Detection of data fabrication using statistical tools
Hartgerink, C. H. J., Voelkel, J. G., Wicherts, J. M., & van Assen, M. A. L. M. (2019) · doi:10.31234/osf.io/jkws4 · Cited by 16
Scientific Integrity and Misconduct—Yet Again
Kharasch, E. D. (2021) · Anesthesiology · doi:10.1097/aln.0000000000003916 · Cited by 19
Self-citationSeeking and reporting apparent research misconduct: errors and integrity
Kharasch, E. D., & Houle, T. T. (2018) · Anaesthesia · doi:10.1111/anae.14147 · Cited by 15
Self-citationHow a data detective exposed suspicious medical trials.
Adam, D. (2019) · Nature · doi:10.1038/d41586-019-02241-z · Cited by 12
Metadata
- Authors
- Kharasch, E. D., & Houle, T. T.
- Affiliations
- Harvard University (US)
- Venue
- Anesthesiology
- Published
- 2017-11
- Volume
- 127
- Issue
- 5
- Pages
- 733-737
- ISSN
- 0003-3022 (ISSN-L)
- OpenAlex
- W2747160812
- Semantic Scholar
- 05b315749c8bf67032c588db9e5c2eae6c7a28f4
- Cited by
- 15
- References
- 11
- Citations by year
- 2024: 1 · 2023: 2 · 2021: 2 · 2020: 1 · 2019: 4 · 2018: 3 · 2017: 2
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Anesthesiology (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Education › Academic integrity and plagiarism
- Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
- Physical Sciences › Mathematics › Statistics and Probability › Benford’s Law and Fraud Detection
- Keywords
- research misconduct, scientific integrity, data fabrication, falsification, non-probability sampling, randomized controlled trials, retraction, p-values, research funding conflicts of interest
- MeSH terms
- Humans, Scientific Misconduct, Biomedical Research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Errors and Integrity in Seeking and Reporting Apparent Research Misconduct. https://app.cochat.ai/papers/10.1097/aln.0000000000001875
- The paper
- Kharasch, E. D., & Houle, T. T. (2017). Errors and Integrity in Seeking and Reporting Apparent Research Misconduct. Anesthesiology. https://doi.org/10.1097/aln.0000000000001875
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.