History repeating: guidelines to address common problems in psychedelic science
Open accessAbstract
Research in the last decade has expressed considerable optimism about the clinical potential of psychedelics for the treatment of mental disorders. This optimism is reflected in an increase in research papers, investments by pharmaceutical companies, patents, media coverage, as well as political and legislative changes. However, psychedelic science is facing serious challenges that threaten the validity of core findings and raise doubt regarding clinical efficacy and safety. In this paper, we introduce the 10 most pressing challenges, grouped into easy, moderate, and hard problems. We show how these problems threaten internal validity (treatment effects are due to factors unrelated to the treatment), external validity (lack of generalizability), construct validity (unclear working mechanism), or statistical conclusion validity (conclusions do not follow from the data and methods). These problems tend to co-occur in psychedelic studies, limiting conclusions that can be drawn about the safety and efficacy of psychedelic therapy. We provide a roadmap for tackling these challenges and share a checklist that researchers, journalists, funders, policymakers, and other stakeholders can use to assess the quality of psychedelic science. Addressing today’s problems is necessary to find out whether the optimism regarding the therapeutic potential of psychedelics has been warranted and to avoid history repeating itself.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Blinding efficacy for participants and therapists and expectancy for participants will be assessed and reported.
“blinding efficacy (participants, therapists) and expectancy (participants) will be assessed and reported, as recommended in the literature [32–34].”
From Psilocybin-assisted therapy for severe alcohol use disorder: protocol for a double-blind, randomized, placebo-controlled, 7-month parallel-group phase II superiority trial (2024) · cites this paper as [32]
In “The breaking blind problem” (page unavailable), the paper recommends assessing and reporting blinding efficacy. In “Placebo effects” (page unavailable), it recommends assessing patients’ and therapists’ expectations, but does not explicitly recommend reporting expectancy; that part of the cited recommendation is not established by our copy. (LLM, conf=0.83)
Experimental — accuracy not yet measuredSystematic unblinding has been raised as a major threat to the validity of extant findings from double-blind psychedelic-assisted therapy trials due to high expectancy, presumably resulting in large placebo and nocebo effects in the experimental and control groups, respectively.
“systematic unblinding has been raised as a major threat to the validity of extant findings from ‘double blind’ PAT trials due to high expectancy, presumably resulting in large placebo and nocebo effects in the experimental group and the control group, respectively [32–34].”
There is a general call for more effective active placebo conditions and systematic reporting of blinding efficacy.
“leading to a general call for more effective active placebo conditions and a systematic reporting of blinding efficacy [32–34].”
The study proposes an active placebo condition aimed at maintaining blinding, while systematically assessing and reporting blinding efficacy in participants and therapists.
“proposing an active placebo condition aimed at maintaining blinding, while systematically assessing and reporting blinding efficacy in participants and therapists.”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
The citation lists haven’t been fetched yet
References 114 · Cited by 137
Research works printed but not linked: 5
- [17]Marks M Cohen IG. Patents on psychedelics: the next legal battlefront of drug development. Harv Law Rev Forum 2021; 135: 212.
- [68]Lindsay B. Footage of therapists spooning and pinning down patient in B.C. trial for MDMA therapy prompts review, https://www.cbc.ca/news/canada/british-columbia/bc-mdma-therapy-videos-1.6400256 (2022, accessed 18 September 2023).
- [113]Szigeti B Nutt D Carhart-Harris R, et al. On the fallibility of placebo control and how to address it: a case study in psychedelic microdosing. PsyArXiv, 2022.
- [120]Wampold BE Frost ND Yulish NE. Placebo effects in psychotherapy: a flawed concept and a contorted history. Psychol Conscious Theory Res Prac 2016; 3: 108– 120.
- [130]Schiavone SR Quinn KA Vazire S. A consensus-based tool for evaluating threats to the validity of empirical research. Psyarxiv, 2023.
- Authors
- van Elk, M., & Fried, E. I.
- Affiliations
- Cognitive Psychology Unit, Institute of Psychology, Leiden University, PO Box 9555, Leiden 2300 RB, The Netherlands
- Leiden University (NL)
- Venue
- Therapeutic Advances in Psychopharmacology
- Publisher
- SAGE Publications
- Published
- 2023-01
- Volume
- 13
- Pages
- 20451253231198466-20451253231198466
- ISSN
- 2045-1253 (ISSN-L)
- 2045-1261
- PMC
- PMC10521293
- OpenAlex
- W4387019277
- Cited by
- 137
- References
- 114
- Citations by year
- 2026: 35 · 2025: 60 · 2024: 39 · 2023: 3
- FWCI
- 6.95
- Type
- journal article
- Language
- en
- Update notices
- correction (2024-01-10)
- Open access
- gold
- doi.org
- Licence
- Locations
- Therapeutic Advances in Psychopharmacology (publishedVersion, cc-by-nc)
- PubMed Central (publishedVersion, cc-by-nc)
- PubMed (publishedVersion)
- Leiden Repository (Leiden University) (submittedVersion, cc-by-nc)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- NWO (VI.Vidi.191.107)
- Topics
- Social Sciences › Psychology › Clinical Psychology › Psychedelics and Drug Studies
- Social Sciences › Psychology › Clinical Psychology › Psychotherapy Techniques and Applications
- Life Sciences › Neuroscience › Cognitive Neuroscience › Pain Management and Placebo Effect
- Keywords
- psychedelics, psychedelic therapy, research validity, internal validity, generalizability, construct validity, statistical conclusion validity, mental illness, clinical efficacy and safety
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
