The value of pragmatic and observational studies in health care and public health
Open accessThe DOI matches its Crossref record; 12 of the 12 references it cites were found in a registry or an index.
Abstract
Evidence-based practice is an important component of health care service delivery. However, there is a tendency, embodied in tools such as Grades of Recommendation, Assessment, Development, and Evaluation, to focus principally on the classification of study design, at the expense of a detailed assessment of the strengths and limitations of the individual study. Randomized controlled trials (RCTs), and in particular the classical "explanatory" RCT, have a privileged place in the hierarchy of evidence. However, classical RCTs have substantial limitations, most notably a lack of generalizability, which limit their direct applicability to clinical practice implementation. Pragmatic and observational studies can provide an invaluable perspective into real-world applicability. This evidence could be used more widely to complement ideal-condition results from classical RCTs, following the principle of triangulation. In this review article, we discuss several types of pragmatic and observational studies that could be used in this capacity. We discuss their particular strengths and how their limitations may be overcome and provide real-life examples by means of illustration.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Austin, P. C. (2011) · Multivariate Behavioral Research · doi:10.1080/00273171.2011.568786 · Cited by 12,242 · cited 1×
“As used in analyses from BSRBR-RA, propensity score matching techniques 39 can be used in longitudinal analysis of registry data in an effort to overcome confounding by baseline group differences, which is ordinarily a substantial disadvantage of nonrandomized studies of health care interventions.”
Randomized, Controlled Trials, Observational Studies, and the Hierarchy of Research Designs
Concato, J., Shah, N., & Horwitz, R. I. (2000) · New England Journal of Medicine · doi:10.1056/nejm200006223422507 · Cited by 3,563
What is ‘quality of evidence’ and why is it important to clinicians?
Guyatt, G. H., Oxman, A. D., Kunz, R., Vist, G. E., Falck-Ytter, Y., & Schünemann, H. J. (2008) · BMJ · doi:10.1136/bmj.39490.551019.be · Cited by 3,250
External validity of randomised controlled trials: “To whom do the results of this trial apply?”
Rothwell, P. M. (2005) · The Lancet · doi:10.1016/s0140-6736(04)17670-8 · Cited by 2,544 · cited 1×
“10,11 Indeed, a Cochrane review of a series of relatively small RCTs recommended nebulized saline for bronchiolitis in infants, 12 a recommendation that is not supported by any subsequent larger trials, such as that by Everard et al. 13 EBM, however conceptualized, is not perfect and relies on the…”
The PRECIS-2 tool: designing trials that are fit for purpose
Loudon, K., Treweek, S., Sullivan, F., Donnan, P., Thorpe, K. E., & Zwarenstein, M. (2015) · BMJ · doi:10.1136/bmj.h2147 · Cited by 1,846
Ford, I., & Norrie, J. (2016) · New England Journal of Medicine · doi:10.1056/nejmra1510059 · Cited by 1,405 · cited 11×
“15 It is important to note that there is evidence that well-designed observational studies do not systematically overestimate treatment effects compared to RCTs. 16 Methodological triangulation is an important concept in social science research, and especially in qualitative research.”
InfluentialExplanatory and pragmatic attitudes in therapeutical trials
Schwartz, D., & Lellouch, J. (1967) · Journal of Chronic Diseases · doi:10.1016/0021-9681(67)90041-0 · Cited by 1,185 · cited 1×
“Indeed, the distinction between classical RCTs and a more pragmatic RCT design was discussed back in the 1960s.(18) The concept of the pragmatic RCT arose out of a recognition of some of the issues with classical RCTs that we have discussed earlier, most notably a lack of generalizability and applicability to routine…”
Peer Review: A Flawed Process at the Heart of Science and Journals
Smith, R. (2006) · Journal of the Royal Society of Medicine · doi:10.1177/014107680609900414 · Cited by 880
A pragmatic view on pragmatic trials
Patsopoulos, N. A. (2011) · Dialogues in Clinical Neuroscience · doi:10.31887/dcns.2011.13.2/npatsopoulos · Cited by 844 · cited 2×
“As a challenger to the classical RCT paradigm, it is unsurprising that pragmatic trials have faced criticism in some quarters, as reviewed, for example, by Patsopoulos.”
Assessing the Gold Standard — Lessons from the History of RCTs
Bothwell, L. E., Greene, J. A., Podolsky, S. H., & Jones, D. S. (2016) · New England Journal of Medicine · doi:10.1056/nejmms1604593 · Cited by 664 · cited 10×
“15 It is important to note that there is evidence that well-designed observational studies do not systematically overestimate treatment effects compared to RCTs. 16 Methodological triangulation is an important concept in social science research, and especially in qualitative research.”
Influential
Cited by
Interpretation and Impact of Real-World Clinical Data for the Practicing Clinician
Blonde, L., Khunti, K., Harris, S. B., Meizinger, C., & Skolnik, N. S. (2018) · Advances in Therapy · doi:10.1007/s12325-018-0805-y · Cited by 714
Davis, A. C., Voelkel, J. L., Remmers, C. L., Adams, J. L., & McGlynn, E. A. (2023) · The Permanente Journal · doi:10.7812/tpp/22.172 · Cited by 402
European Resuscitation Council Guidelines 2021: Ethics of resuscitation and end of life decisions
Mentzelopoulos, S. D., Couper, K., Voorde, P. V. D., Druwé, P., Blom, M., Perkins, G. D., Lulic, I., Djakow, J., Raffay, V., Lilja, G., et al. (2021) · Resuscitation · doi:10.1016/j.resuscitation.2021.02.017 · Cited by 246
Volpi, S., Bult, C. J., Chisholm, R. L., Deverka, P. A., Ginsburg, G. S., Jacob, H. J., Kasapi, M., McLeod, H. L., Roden, D. M., Williams, M. S., et al. (2018) · Clinical Pharmacology & Therapeutics · doi:10.1002/cpt.1048 · Cited by 135
“Members of the IGNITE Network utilized a pragmatic study design (16) to examine outcomes with CYP2C19 genotype-guided antiplatelet therapy in patients undergoing percutaneous coronary intervention; this was performed while awaiting the results of an ongoing traditional randomized clinical trial with an estimated compl…”
Telemedicine Implementation for Safety Net Populations: A Systematic Review
Tierney, A. A., Mosqueda, M., Cesena, G., Frehn, J. L., Payán, D. D., & Rodriguez, H. P. (2024) · Telemedicine and e-Health · doi:10.1089/tmj.2023.0260 · Cited by 26
Metadata
- Authors
- Barnish, M., & Turner, S.
- Venue
- Pragmatic and Observational Research
- Publisher
- Informa UK Limited
- Published
- 2017-05
- Volume
- Volume 8
- Pages
- 49-55
- ISSN
- 1179-7266 (ISSN-L)
- OpenAlex
- W2613823791
- Cited by
- 118
- References
- 12
- FWCI
- 4.90
- Type
- journal article
- Language
- en
- Open access
- diamond
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: The value of pragmatic and observational studies in health care and public health. https://app.cochat.ai/papers/10.2147/por.s137701
- The paper
- Barnish, M., & Turner, S. (2017). The value of pragmatic and observational studies in health care and public health. Pragmatic and Observational Research. https://doi.org/10.2147/por.s137701
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.