A pragmatic view on pragmatic trials
Open accessAbstract
Clinical trials have been the main tool used by the health sciences community to test and evaluate interventions, Trials can fall into two broad categories: pragmatic and explanatory. Pragmatic trials are designed to evaluate the effectiveness of interventions in real-life routine practice conditions, whereas explanatory trials aim to test whether an intervention works under optimal situations. Pragmatic trials produce results that can be generalized and applied in routine practice settings. Since most results from exploratory trials fail to be broadly generalizable, the "pragmatic design" has gained momentum. This review describes the concept of pragmatism, and explains in particular that there is a continuum between pragmatic and explanatory trials, rather than a dichotomy. Special focus is put on the limitations of the pragmatic trials, while recognizing the importance for and impact of this design on medical practice.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Pragmatic trials are prominent for examining the effectiveness of interventions in a real-world clinical setting and covering the full spectrum of the population to which the intervention will be applied.
“Pragmatic trials are prominent for examining the effectiveness of interventions in a real-world clinical setting and covering the full spectrum of the population to which the intervention will be applied (16).”
From Total Sleep Deprivation Followed by Bright Light Therapy as Rapid Relief for Depression: A Pragmatic Randomized Controlled Trial (2021) · cites this paper as [16]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
The ALLHAT Officers and Coordinators for the ALLHAT Collaborative Research Group (2002) · JAMA: The Journal of the American Medical Association · doi:10.1001/jama.288.23.2981 · Cited by 5,664 · see citing works
External validity of randomised controlled trials: “To whom do the results of this trial apply?”
Rothwell, P. M. (2005) · The Lancet · doi:10.1016/s0140-6736(04)17670-8 · Cited by 2,544 · see citing works
Wood, L., Egger, M., Gluud, L. L., Schulz, K. F., Jüni, P., Altman, D. G., Gluud, C., Martin, R. M., Wood, A. J. G., & Sterne, J. A. C. (2008) · BMJ · doi:10.1136/bmj.39465.451748.ad · Cited by 2,538 · see citing works
Combination of direct and indirect evidence in mixed treatment comparisons
Lu, G., & Ades, A. E. (2004) · Statistics in Medicine · doi:10.1002/sim.1875 · Cited by 2,149 · see citing works
Improving the reporting of pragmatic trials: An extension of the CONSORT statement
Zwarenstein, M., Treweek, S., Gagnier, J. J., Altman, D. G., Tunis, S., Haynes, B., Oxman, A. D., Moher, D., & groups, F. T. C. A. P. T. I. H. (. (2008) · BMJ · doi:10.1136/bmj.a2390 · Cited by 1,813 · see citing works
Tunis (2003) · doi:10.1001/jama.290.12.1624 · Cited by 1,724 · see citing works
Cipriani, A., Furukawa, T. A., Salanti, G., Geddes, J. R., Higgins, J. P., Churchill, R., Watanabe, N., Nakagawa, A., Omori, I. M., McGuire, H., et al. (2009) · The Lancet · doi:10.1016/s0140-6736(09)60046-5 · Cited by 1,654 · see citing works
Bias and causal associations in observational research
Grimes, D. A., & Schulz, K. F. (2002) · The Lancet · doi:10.1016/s0140-6736(02)07451-2 · Cited by 1,553 · see citing works
A pragmatic–explanatory continuum indicator summary (PRECIS): a tool to help trial designers
Thorpe, K. E., Zwarenstein, M., Oxman, A. D., Treweek, S., Furberg, C. D., Altman, D. G., Tunis, S., Bergel, E., Harvey, I., Magid, D. J., et al. (2009) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2008.12.011 · Cited by 1,389 · see citing works
Explanatory and pragmatic attitudes in therapeutical trials
Schwartz, D., & Lellouch, J. (1967) · Journal of Chronic Diseases · doi:10.1016/0021-9681(67)90041-0 · Cited by 1,185 · see citing works
Relton, C., Torgerson, D. J., O’Cathain, A., & Nicholl, J. P. (2010) · BMJ · doi:10.1136/bmj.c1066 · Cited by 519 · see citing works
Relative Citation Impact of Various Study Designs in the Health Sciences
Patsopoulos, N. A. (2005) · JAMA · doi:10.1001/jama.293.19.2362 · Cited by 498 · see citing works
Self-citationMaking trials matter: pragmatic and explanatory trials and the problem of applicability
Treweek, S. P., & Zwarenstein, M. (2009) · Trials · doi:10.1186/1745-6215-10-37 · Cited by 482 · see citing works
Impact of allocation concealment on conclusions drawn from meta-analyses of randomized trials
Pildal, J., Hrobjartsson, A., Jorgensen, K., Hilden, J., Altman, D., & Gotzsche, P. (2007) · International Journal of Epidemiology · doi:10.1093/ije/dym087 · Cited by 472 · see citing works
Randomisation to protect against selection bias in healthcare trials
Odgaard-Jensen, J., Vist, G. E., Timmer, A., Kunz, R., Akl, E. A., Schünemann, H., Briel, M., Nordmann, A. J., Pregno, S., & Oxman, A. D. (2011) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000012.pub3 · Cited by 439 · see citing works
Lawlor, D. A., Smith, G. D., Bruckdorfer, K. R., Kundu, D., & Ebrahim, S. (2004) · The Lancet · doi:10.1016/s0140-6736(04)16260-0 · Cited by 438 · see citing works
MacPherson, H. (2004) · Complementary Therapies in Medicine · doi:10.1016/j.ctim.2004.07.043 · Cited by 334 · see citing works
Issues in the reporting of epidemiological studies: a survey of recent practice.
Pocock, S. J., Collier, T. J., Dandreo, K. J., de Stavola, B. L., Goldman, M. B., Kalish, L. A., Kasten, L. E., & McCormack, V. A. (2004) · BMJ · doi:10.1136/bmj.38250.571088.55 · Cited by 313 · see citing works
Glasgow, R. E., Magid, D. J., Beck, A., Ritzwoller, D., & Estabrooks, P. A. (2005) · Medical Care · doi:10.1097/01.mlr.0000163645.41407.09 · Cited by 308 · see citing works
A simple and valid tool distinguished efficacy from effectiveness studies
Gartlehner, G., Hansen, R. A., Nissman, D., Lohr, K. N., & Carey, T. S. (2006) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2006.01.011 · Cited by 207 · see citing works
Industry sponsorship and selection of comparators in randomized clinical trials
Lathyris, D., Patsopoulos, N. A., Salanti, G., & Ioannidis, J. P. A. (2009) · European Journal of Clinical Investigation · doi:10.1111/j.1365-2362.2009.02240.x · Cited by 100 · see citing works
Self-citationIndirect comparisons: the mesh and mess of clinical trials
Ioannidis, J. P. A. (2006) · The Lancet · doi:10.1016/s0140-6736(06)69615-3 · Cited by 94 · see citing works
Generalizability of the Results of Randomized Trials
Weiss, N. S. (2008) · Archives of Internal Medicine · doi:10.1001/archinternmed.2007.30 · Cited by 84 · see citing works
Tosh, G., Soares-Weiser, K., & Adams, C. E. (2011) · Dialogues in Clinical Neuroscience · doi:10.31887/dcns.2011.13.2/gtosh · Cited by 84 · see citing works
Availability of Comparative Efficacy Data at the Time of Drug Approval in the United States
Goldberg, N. H. (2011) · JAMA · doi:10.1001/jama.2011.539 · Cited by 71 · see citing works
Why are so few randomized trials useful, and what can we do about it?
Zwarenstein, M., & Oxman, A. (2006) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2006.05.010 · Cited by 56 · see citing works
Koppenaal, T., Linmans, J., Knottnerus, J. A., & Spigt, M. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.11.020 · Cited by 56 · see citing works
Masuhr, F., Busch, M. A., & Einhäupl, K. M. (1998) · Stroke · doi:10.1161/01.str.29.2.339 · Cited by 41 · see citing works
What Have We Learned about Trial Design From NIMH-Funded Pragmatic Trials?
March, J. S., KRAEMER, H. C., Trivedi, M. H., Csernansky, J. G., Davis, J. M., Ketter, T. A., & Glick, I. D. (2010) · Neuropsychopharmacology · doi:10.1038/npp.2010.115 · Cited by 29 · see citing works
Explaining pragmatic trials to pragmatic policymakers
Maclure, M. (2009) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2008.06.021 · Cited by 21 · see citing works
Djulbegovic, M. (2011) · JAMA · doi:10.1001/jama.2010.2013 · Cited by 18 · see citing works
Understanding Randomized Controlled Trials: Explanatory or Pragmatic?
Charlton, B. G. (1994) · Family Practice · doi:10.1093/fampra/11.3.243 · Cited by 17 · see citing works
"Pragmatic" clinical trials: from whose perspective?
Karanicolas, P. J., Montori, V. M., Schünemann, H. J., et al. (2009) · BMJ Evidence-Based Medicine · doi:10.1136/ebm.14.5.130-b · Cited by 6 · see citing works
Flagged first, then by citations
Research works printed but not linked: 1
- [15]Prioritization. CoCR. Institute of Medicine Initial National Priorities for Compaiative Effectiveness Research. Washington, DC: National Academy Press, 2009
Cited by
Sorted by citations
Understanding and misunderstanding randomized controlled trials
Deaton, A., & Cartwright, N. (2018) · Social Science & Medicine · doi:10.1016/j.socscimed.2017.12.005 · Cited by 1,841 · see citing works
Kennedy-Martin, T., Curtis, S., Faries, D., Robinson, S., & Johnston, J. (2015) · Trials · doi:10.1186/s13063-015-1023-4 · Cited by 878 · see citing works
Khoruts, A., Rank, K. M., Newman, K. M., Viskocil, K., Vaughn, B. P., Hamilton, M. J., & Sadowsky, M. J. (2016) · Clinical Gastroenterology and Hepatology · doi:10.1016/j.cgh.2016.02.018 · Cited by 213 · see citing works
Richmond, H., Hall, A. M., Copsey, B., Hansen, Z., Williamson, E., Hoxey-Thomas, N., Cooper, Z., & Lamb, S. E. (2015) · PLOS ONE · doi:10.1371/journal.pone.0134192 · Cited by 205 · see citing works
Wahbeh, H., Svalina, M. N., & Oken, B. S. (2014) · Open Medicine Journal · doi:10.2174/1874220301401010066 · Cited by 116 · see citing works
Coelhoso, C. C., Tobo, P. R., Lacerda, S. S., Lima, A. H., Barrichello, C. R. C., Amaro Jr, E., & Kozasa, E. H. (2019) · Journal of Medical Internet Research · doi:10.2196/14269 · Cited by 95 · see citing works
Hinton, L., Tran, D., Nguyen, T.-N., Ho, J., & Gitlin, L. (2019) · BMJ Global Health · doi:10.1136/bmjgh-2019-001830 · Cited by 82 · see citing works
Optimization of home care nurses in Canada: A scoping review
Ganann, R., Weeres, A., Lam, A., Chung, H., & Valaitis, R. (2019) · Health & Social Care in the Community · doi:10.1111/hsc.12797 · Cited by 79 · see citing works
A pragmatic trial of a group intervention in senior housing communities to increase resilience
Treichler, E. B. H., Glorioso, D., Lee, E. E., Wu, T.-C., Tu, X. M., Daly, R., O’Brien, C., Smith, J. L., & Jeste, D. V. (2020) · International Psychogeriatrics · doi:10.1017/s1041610219002096 · Cited by 69 · see citing works
Meadows, G. N., Shawyer, F., Enticott, J. C., Graham, A. L., Judd, F., Martin, P. R., Piterman, L., & Segal, Z. (2014) · Australian & New Zealand Journal of Psychiatry · doi:10.1177/0004867414525841 · Cited by 65 · see citing works
Mc Cord, K. A., Ewald, H., Agarwal, A., Glinz, D., Aghlmandi, S., Ioannidis, J. P. A., & Hemkens, L. G. (2021) · BMJ · doi:10.1136/bmj.n450 · Cited by 61 · see citing works
Meadows, G., Brophy, L., Shawyer, F., Enticott, J. C., Fossey, E., Thornton, C. D., Weller, P. J., Wilson-Evered, E., Edan, V., & Slade, M. (2019) · The Lancet Psychiatry · doi:10.1016/s2215-0366(18)30429-2 · Cited by 59 · see citing works
on behalf of the Ethics of Pragmatic Trials project, Nicholls, S. G., Carroll, K., Zwarenstein, M., Brehaut, J. C., Weijer, C., Hey, S. P., Goldstein, C. E., Graham, I. D., Grimshaw, J. M., et al. (2019) · Trials · doi:10.1186/s13063-019-3899-x · Cited by 56 · see citing works
Does rumination mediate the relationship between mindfulness and depressive relapse?
Kearns, N. P., Shawyer, F., Brooker, J. E., Graham, A. L., Enticott, J. C., Martin, P. R., & Meadows, G. N. (2016) · Psychology and Psychotherapy: Theory, Research and Practice · doi:10.1111/papt.12064 · Cited by 39 · see citing works
Campbell, S. B., Erbes, C., Grubbs, K., & Fortney, J. (2020) · Journal of Traumatic Stress · doi:10.1002/jts.22542 · Cited by 27 · see citing works
Hadjistavropoulos, H. D., Peynenburg, V., Thiessen, D. L., Nugent, M., Adlam, K., Owens, K. M. B., Karin, E., Dear, B. F., & Titov, N. (2020) · Internet Interventions · doi:10.1016/j.invent.2020.100357 · Cited by 25 · see citing works
Merkouris, S. S., Rodda, S. N., Austin, D., Lubman, D. I., Harvey, P., Battersby, M., Cunningham, J., Lavis, T., Smith, D., & Dowling, N. A. (2017) · BMJ Open · doi:10.1136/bmjopen-2016-014226 · Cited by 21 · see citing works
Practice based research in dentistry: An alternative to deal with clinical questions
DEMARCO, F. F., CORREA, M. B., CENCI, M. S., BURKE, F. J. T., OPDAM, N. J. M., & FARIA-E-SILVA, A. L. (2020) · Brazilian Oral Research · doi:10.1590/1807-3107bor-2020.vol34.0071 · Cited by 19 · see citing works
Empathy-based supportive treatment in amyotrophic lateral sclerosis: A pragmatic study
Palmieri, A., Kleinbub, J. R., Pagnini, F., Sorarù, G., & Cipolletta, S. (2021) · American Journal of Clinical Hypnosis · doi:10.1080/00029157.2020.1797623 · Cited by 18 · see citing works
Practice-Based Research Networks and the Mandate for Real-World Evidence
Hall-Lipsy, E., Barraza, L., & Robertson, C. (2018) · American Journal of Law & Medicine · doi:10.1177/0098858818789428 · Cited by 16 · see citing works
Ioannou, M., Szabó, Z., Widmark-Jensen, M., Vyrinis, G., Karlsson, C., & Steingrimsson, S. (2021) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2021.705090 · Cited by 13 · see citing works
Oral rehabilitation outcomes Network-ORONet
Bassi, F., Carr, A. B., Chang, T.-L., Estafanous, E., Garrett, N. R., Happonen, R.-P., Koka, S., Laine, J., Osswald, M., Reintsema, H., et al. (2013) · The International Journal of Prosthodontics · doi:10.11607/ijp.3400 · Cited by 11 · see citing works
Kanzler, K. E., Robinson, P. J., McGeary, D. D., Mintz, J., Potter, J. S., Muñante, M., Lopez, E. J., Dougherty, D. M., Hale, W. J., & Velligan, D. I. (2018) · Contemporary Clinical Trials · doi:10.1016/j.cct.2018.01.004 · Cited by 7 · see citing works
Weight management programmes of extended duration
Brindal, E. (2017) · The Lancet · doi:10.1016/s0140-6736(17)31042-5 · Cited by 5 · see citing works
- Authors
- Patsopoulos, N. A.
- Affiliations
- Broad Institute (US)
- Corresponding author
- Patsopoulos, Nikolaos A.
- Venue
- Dialogues in Clinical Neuroscience
- Publisher
- Informa UK Limited
- Published
- 2011-06-30
- Volume
- 13
- Issue
- 2
- Pages
- 217-224
- ISSN
- 1294-8322 (ISSN-L)
- 1958-5969
- PMC
- PMC3181997
- OpenAlex
- W4295612440
- Cited by
- 844
- References
- 33
- Citations by year
- 2026: 31 · 2025: 53 · 2024: 84 · 2023: 89 · 2022: 71 · 2021: 107 · 2020: 66 · 2019: 77 · 2018: 62 · 2017: 59 · 2016: 58 · 2015: 34 · 2014: 24 · 2013: 20 · 2012: 3
- FWCI
- 13.17
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- Locations
- Dialogues in Clinical Neuroscience (publishedVersion, cc-by-nc-nd)
- Europe PMC (PubMed Central) (publishedVersion, cc-by-nc-nd)
- PubMed Central (publishedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- PubMed
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Keywords
- pragmatic trials, explanatory trials, generalizability, intervention effectiveness, routine clinical practice, trial design
- MeSH terms
- Clinical Trials as Topic, Humans, Research Design, Treatment Outcome, Evidence-Based Medicine
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
