Why randomised controlled trials of psychological treatments are still essential
The DOI matches its Crossref record; 4 of the 4 references it cites were found in a registry or an index.
Abstract
The movement towards evidence-based practice in psychotherapy has recently provoked a backlash. In the past few years, some authors have raised questions concerning the validity of randomised controlled trials (RCTs), which have long been regarded as a crucial linchpin for evidence-based practice. These critics have argued that data from RCTs should not be prioritised above data derived from alternative evidentiary sources, such as observational studies, uncontrolled case studies, and clinical experience. Although many of the criticisms have been aimed at RCT methodology in general, some sceptics have extended them to raise questions concerning the role of RCTs in the evaluation of psychological treatments. For example, several scholars have recently contended that RCTs of psychotherapy are marked by limited external validity. Others have noted that RCTs can tell us whether, but not why, interventions work. In a 2017 blog post, Jonathan Shedler encouraged psychotherapists to ignore the new practice guidelines for post-traumatic stress disorder released by the American Psychological Association because they relied inordinately on RCTs when formulating recommendations. Similarly, the New Jersey Psychological Association recently maintained that these practice guidelines gave RCTs undue priority at the expense of other forms of evidence. The former Editor-in-Chief of Science News, Tom Siegfried, has also joined the fray. Drawing on arguments from a 2015 doctoral thesis by Christopher Blunt of the London School of Economics, Siegfried argued that methodological flaws, such as the frequent failure of random assignment to precisely equalise treatment and control groups on confounding variables, often affect RCTs. Echoing Blunt, Siegfried maintained that because RCTs are group designs, they do not permit practitioners to apply treatment recommendations to individual patients. These criticisms of RCTs are not entirely without merit. Nevertheless, they provide less than convincing grounds for downgrading the status of RCTs within the evidence-based practice hierarchy relevant to psychotherapy practice. Notably, many of the criticisms are largely ahistorical. Psychiatry’s past is replete with a long line of harmful interventions, including spinning, blistering, purging, leeching, prefrontal lobotomy, insulin coma therapy, and surgical removal of organs, all of which were justified on the basis of uncontrolled observations and clinical experience. More recently, several psychological treatments defended on these grounds, such as crisis debriefing for trauma victims or Scared Straight programmes for conduct disorder, were found to be ineffective or iatrogenic in subsequent controlled studies, including RCTs. Critics rightly observe that designs other than RCTs can provide useful evidence for psychological treatment outcomes, and that well conducted observational studies can sometimes be more informative than poorly conducted RCTs. Additionally, rigorous observational studies do not invariably overestimate the magnitudes of effects derived from RCTs. Nonetheless, observational studies cannot support strong causal inferences, and no responsible practitioner would administer a psychological treatment that does not produce causal effects. All else being equal, RCTs provide less biased estimates of psychotherapy outcomes than do other designs. By virtue of their minimisation of between-group differences via randomisation, RCTs afford enhanced control over various causes of spurious therapeutic effectiveness, which can fool observers into believing that ineffective psychotherapies are effective. Causes of spurious therapeutic effectiveness include regression to the mean, spontaneous remission, selection bias, maturation, patients’ concurrent use of adjunctive interventions, and other internal validity threats. Given their better control over causes of spurious therapeutic effectiveness, RCTs are probably superior to other designs with respect to a key scientific criterion: replicability. Indeed, an analysis of highly cited medical trials revealed that RCT designs generate substantially more replicable results than do non-randomised designs. RCT critics have at times fallen prey to logical errors. Some critics, such as the members of the UK Council of Psychotherapy, have conflated methodological limitations of specific RCTs with RCT methodology per se. For example, they and others have observed legitimately that RCTs have sometimes focused For more on the 2017 blog post by Jonathan Shedler see https:// www.psychologytoday.com/ blog/psychologically-minded/ 201711/selling-bad-therapytrauma-victims
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Randomized, Controlled Trials, Observational Studies, and the Hierarchy of Research Designs
Concato, J., Shah, N., & Horwitz, R. I. (2000) · New England Journal of Medicine · doi:10.1056/nejm200006223422507 · Cited by 3,563
Contradicted and Initially Stronger Effects in Highly Cited Clinical Research
Ioannidis, J. P. A. (2005) · JAMA · doi:10.1001/jama.294.2.218 · Cited by 1,465
Grove, W. M., & Meehl, P. E. (1996) · Psychology, Public Policy, and Law · doi:10.1037/1076-8971.2.2.293 · Cited by 1,457
Limitations of Randomized Control Designs in Psychotherapy Research
Shean, G. (2014) · Advances in Psychiatry · doi:10.1155/2014/561452 · Cited by 72
Cited by
Recent Developments in Group Psychotherapy Research
Rosendahl, J., Alldredge, C. T., Burlingame, G. M., & Strauss, B. (2021) · American Journal of Psychotherapy · doi:10.1176/appi.psychotherapy.20200031 · Cited by 117
Axelsson, E., & Hedman-Lagerlöf, E. (2019) · Expert Review of Pharmacoeconomics & Outcomes Research · doi:10.1080/14737167.2019.1703182 · Cited by 96
Byrne, E. M., Jensen, H., Thomsen, B. S., & Ramchandani, P. G. (2023) · Review of Education · doi:10.1002/rev3.3400 · Cited by 43
Wan Mohd Yunus, W. M. A., Matinolli, H.-M., Waris, O., Upadhyaya, S., Vuori, M., Korpilahti-Leino, T., Ristkari, T., Koffert, T., & Sourander, A. (2022) · Journal of Medical Internet Research · doi:10.2196/33337 · Cited by 26
Developing a brief acceptance and commitment therapy model for industrial psychologists
Van Lill, X., & Van Lill, R. (2022) · SA Journal of Industrial Psychology · doi:10.4102/sajip.v48i0.1897 · Cited by 7
Metadata
- Authors
- Lilienfeld, S. O., McKay, D., & Hollon, S. D.
- Venue
- The Lancet Psychiatry
- Publisher
- Elsevier BV
- Published
- 2018-07
- Volume
- 5
- Issue
- 7
- Pages
- 536-538
- ISSN
- 2215-0366 (ISSN-L)
- OpenAlex
- W2794485287
- Cited by
- 43
- References
- 4
- FWCI
- 6.91
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Why randomised controlled trials of psychological treatments are still essential. https://app.cochat.ai/papers/10.1016/s2215-0366(18)30045-2
- The paper
- Lilienfeld, S. O., McKay, D., & Hollon, S. D. (2018). Why randomised controlled trials of psychological treatments are still essential. The Lancet Psychiatry. https://doi.org/10.1016/s2215-0366(18)30045-2
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.