Real-world evidence: How pragmatic are randomized controlled trials labeled as pragmatic?
Open accessThe DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index.
Abstract
INTRODUCTION: Pragmatic randomized controlled trials (RCTs) mimic usual clinical practice and they are critical to inform decision-making by patients, clinicians and policy-makers in real-world settings. Pragmatic RCTs assess effectiveness of available medicines, while explanatory RCTs assess efficacy of investigational medicines. Explanatory and pragmatic are the extremes of a continuum. This debate article seeks to evaluate and provide recommendation on how to characterize pragmatic RCTs in light of the current landscape of RCTs. It is supported by findings from a PubMed search conducted in August 2017, which retrieved 615 RCTs self-labeled in their titles as "pragmatic" or "naturalistic". We focused on 89 of these trials that assessed medicines (drugs or biologics). DISCUSSION: 36% of these 89 trials were placebo-controlled, performed before licensing of the medicine, or done in a single-center. In our opinion, such RCTs overtly deviate from usual care and pragmatism. It follows, that the use of the term 'pragmatic' to describe them, conveys a misleading message to patients and clinicians. Furthermore, many other trials among the 615 coined as 'pragmatic' and assessing other types of intervention are plausibly not very pragmatic; however, this is impossible for a reader to tell without access to the full protocol and insider knowledge of the trial conduct. The degree of pragmatism should be evaluated by the trial investigators themselves using the PRECIS-2 tool, a tool that comprises 9 domains, each scored from 1 (very explanatory) to 5 (very pragmatic). CONCLUSIONS: To allow for a more appropriate characterization of the degree of pragmatism in clinical research, submissions of RCTs to funders, research ethics committees and to peer-reviewed journals should include a PRECIS-2 tool assessment done by the trial investigators. Clarity and accuracy on the extent to which a RCT is pragmatic will help understand how much it is relevant to real-world practice.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The PRECIS-2 tool: designing trials that are fit for purpose
Loudon, K., Treweek, S., Sullivan, F., Donnan, P., Thorpe, K. E., & Zwarenstein, M. (2015) · BMJ · doi:10.1136/bmj.h2147 · Cited by 1,846 · cited 5×
“We are dealing with a continuum and there are tools available to help investigators to assess the degree of pragmatism of their RCT [12, 18, 19].”
InfluentialImproving the reporting of pragmatic trials: An extension of the CONSORT statement
Zwarenstein, M., Treweek, S., Gagnier, J. J., Altman, D. G., Tunis, S., Haynes, B., Oxman, A. D., Moher, D., & groups, F. T. C. A. P. T. I. H. (. (2008) · BMJ · doi:10.1136/bmj.a2390 · Cited by 1,813
Ford, I., & Norrie, J. (2016) · New England Journal of Medicine · doi:10.1056/nejmra1510059 · Cited by 1,405
Explanatory and pragmatic attitudes in therapeutical trials
Schwartz, D., & Lellouch, J. (1967) · Journal of Chronic Diseases · doi:10.1016/0021-9681(67)90041-0 · Cited by 1,185 · cited 1×
“The term “pragmatic” for RCTs was introduced half a century ago [8].”
BackgroundTosh, G., Soares-Weiser, K., & Adams, C. E. (2011) · Dialogues in Clinical Neuroscience · doi:10.31887/dcns.2011.13.2/gtosh · Cited by 84
Cited by
Mc Cord, K. A., Ewald, H., Agarwal, A., Glinz, D., Aghlmandi, S., Ioannidis, J. P. A., & Hemkens, L. G. (2021) · BMJ · doi:10.1136/bmj.n450 · Cited by 61
Self-citationThe Current Status of EMDR Therapy, Specific Target Areas, and Goals for the Future
Matthijssen, S. J. M. A., Lee, C. W., de Roos, C., Barron, I. G., Jarero, I., Shapiro, E., Hurley, E. C., Schubert, S. J., Baptist, J., Amann, B. L., et al. (2020) · Journal of EMDR Practice and Research · doi:10.1891/emdr-d-20-00039 · Cited by 48
How do they add up? The interaction between the placebo and treatment effect: A systematic review
Boussageon, R., Howick, J., Baron, R., Naudet, F., Falissard, B., Harika‐Germaneau, G., Wassouf, I., Gueyffier, F., Jaafari, N., & Blanchard, C. (2022) · British Journal of Clinical Pharmacology · doi:10.1111/bcp.15345 · Cited by 36
Saunders, R., Cohen, Z. D., Ambler, G., DeRubeis, R. J., Wiles, N., Kessler, D., Gilbody, S., Hollon, S. D., Kendrick, T., Watkins, E., et al. (2021) · Journal of Personalized Medicine · doi:10.3390/jpm11121295 · Cited by 21
“The fact that all study participants were recruited in primary care also offers an improvement on much of the extant literature, where previously there has often been a lack of information regarding where participants were recruited from [61].”
BackgroundPragmatic Clinical Trials: A Study Design for Real-World Evidence
Capili, B., & Anastasi, J. K. (2025) · AJN American Journal of Nursing · doi:10.1097/ajn.0000000000000014 · Cited by 15
Metadata
- Authors
- Dal-Ré, R., Janiaud, P., & Ioannidis, J. P. A.
- Venue
- BMC Medicine
- Published
- 2018-04-03
- Volume
- 16
- Issue
- 1
- Pages
- 49-49
- ISSN
- 1741-7015 (ISSN-L)
- OpenAlex
- W2804017805
- Cited by
- 248
- References
- 5
- FWCI
- 20.96
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Real-world evidence: How pragmatic are randomized controlled trials labeled as pragmatic? https://app.cochat.ai/papers/10.1186/s12916-018-1038-2
- The paper
- Dal-Ré, R., Janiaud, P., & Ioannidis, J. P. A. (2018). Real-world evidence: How pragmatic are randomized controlled trials labeled as pragmatic? BMC Medicine. https://doi.org/10.1186/s12916-018-1038-2
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.