CoChat
Sign Up
Cited by 191Ethics in Clinical ResearchBiomedical Ethics and RegulationStatistical Methods in Clinical Trials

Exploring the ethical and regulatory issues in pragmatic clinical trials

Open access
Califf, Robert M, Sugarman, Jeremy·2015·Clinical Trials
DOI 10.1177/1740774515598334PMC PMC4891194
Open this paper in CoChat — free account

Abstract

The need for high-quality evidence to support decision making about health and health care by patients, physicians, care providers, and policy-makers is well documented. However, serious shortcomings in evidence persist. Pragmatic clinical trials that use novel techniques including emerging information and communication technologies to explore important research questions rapidly and at a fraction of the cost incurred by more “traditional” research methods promise to help close this gap. Nevertheless, while pragmatic clinical trials can bridge clinical practice and research, they may also raise difficult ethical and regulatory challenges. In this article, the authors briefly survey the current state of evidence that is available to inform clinical care and other health-related decisions and discuss the potential for pragmatic clinical trials to improve this state of affairs. They then propose a new working definition for pragmatic research that centers upon fitness for informing decisions about health and health care. Finally, they introduce a project, jointly undertaken by the National Institutes of Health Health Care Systems Research Collaboratory and the National Patient-Centered Clinical Research Network (PCORnet), which addresses 11 key aspects of current systems for regulatory and ethical oversight of clinical research that pose challenges to conducting pragmatic clinical trials. In the series of articles commissioned on this topic published in this issue of Clinical Trials, each of these aspects is addressed in a dedicated article, with a special focus on the interplay between ethical and regulatory considerations and pragmatic clinical research aimed at informing “real-world” choices about health and health care.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1177/1740774515598334

Cites

2 of 2 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • There are already major legal differences at the national level about data collection, management, and security, not to mention international legislation.

    “There are already major legal differences at the national level about data collection, management, and security, not to mention international legislation (Califf and Sugarman 2015).”

    From Dental Research in the Digital Age: The Registry-Based Clinical Trial. (2026) · cites this paper as Califf and Sugarman 2015

    The “Ethical and regulatory challenges” section discusses oversight of pragmatic clinical trials, and the Collaboratory/PCORnet subsection identifies privacy as a concern (pages not provided). Neither passage addresses national legal differences in data collection, management, or security, or compares international legislation. Searches for those specific legal differences found no supporting passage in our copy. (LLM, conf=0.72)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 36

  1. [1]Tricoci P Allen JM Kramer JM Scientific evidence underlying the ACC/AHA clinical practice guidelines JAMA 2009 301 831 841 19244190
  2. [2]Wright JD Pawar N Gonzalez JS Scientific evidence underlying the American College of Obstetricians and Gynecologists' practice bulletins Obstet Gynecol 2011 118 505 512 21826038
  3. [3]Koh C Zhao X Samala N AASLD clinical practice guidelines: a critical review of scientific evidence and evolving recommendations Hepatology 2013 58 2142 2152 23775835 PMC4613804
  4. [5]Roos M Brodbeck J Sarkozy A A critical analysis of the scientific evidence behind international guidelines related to cardiac arrhythmias Circ Arrhythm Electrophysiol 2011 4 202 210 21372270
  5. [6]Feuerstein JD Akbari M Gifford AE Systematic analysis underlying the quality of the scientific evidence and conflicts of interest in interventional medicine subspecialty guidelines Mayo Clin Proc 2014 89 16 24 24388018
  6. [8]Institute of Medicine Roundtable on Evidence-Based Medicine Olsen LA Aisner D McGinnis JM Washington (DC) National Academies Press (US) 2007
  7. [9]Dewitt JG Hampton PM Development of a data warehouse at an academic health system: knowing a place for the first time Acad Med 2005 80 1019 1025 16249300
  8. [10]Chute CG Beck SA Fisk TB The Enterprise Data Trust at Mayo Clinic: a semantically integrated warehouse of biomedical data J Am Med Inform Assoc 2010 17 131 135 20190054 PMC3000789
  9. [11]Behrman RE Benner JS Brown JS Developing the Sentinel System--a national resource for evidence development N Engl J Med 2011 364 498 499 21226658
  10. [14]Nallamothu BK Hayward RA Bates ER Beyond the randomized clinical trial: the role of effectiveness studies in evaluating cardiovascular therapies Circulation 2008 118 1294 1303 18794402
  11. [15]Krumholz HM Wang Y Chen J Reduction in acute myocardial infarction mortality in the United States: risk-standardized mortality rates from 1995–2006 JAMA 2009 302 767 773 19690309 PMC3349070
  12. [17]O'Leary M Krailo M Anderson JR Progress in childhood cancer: 50 years of research collaboration, a report from the Children's Oncology Group Semin Oncol 2008 35 484 493 18929147 PMC2702720
  13. [18]Califf RM DeMets DL Principles from clinical trials relevant to clinical practice: Part I Circulation 2002 106 1015 1021 12186809
  14. [19]Califf RM DeMets DL Principles from clinical trials relevant to clinical practice: Part II Circulation 2002 106 1172 1175 12196347
  15. [20]DeMets DL Califf RM Lessons learned from recent cardiovascular clinical trials: Part I Circulation 2002 106 746 751 12163438
  16. [21]DeMets DL Califf RM Lessons learned from recent cardiovascular clinical trials: Part II Circulation 2002 106 880 886 12176964
  17. [23]Rosati K Evans B McGraw D for the Mini-Sentinel Privacy Panel HIPAA and Common Rule compliance in the Mini-Sentinel pilot http://www.mini-sentinel.org/work_products/About_Us/HIPAA_and_CommonRuleCompliance_in_the_Mini-SentinelPilot.pdf Accessed 30 December 2014
  18. [24]Schwartz D Lellouch J Explanatory and pragmatic attitudes in therapeutical trials J Chronic Dis 1967 20 637 648 4860352
  19. [25]Yusuf S Collins R Peto R Why do we need some large, simple randomized trials? Stat Med 1984 3 409 422 6528136
  20. [26]Tunis SR Stryer DB Clancy CM Practical clinical trials: increasing the value of clinical research for decision making in clinical and health policy JAMA 2003 290 1624 1632 14506122
  21. [27]Thorpe KE Zwarenstein M Oxman AD A pragmatic-explanatory continuum indicator summary (PRECIS): a tool to help trial designers J Clin Epidemiol 2009 62 464 475 19348971
  22. [29]Kass NE Faden RR Goodman SN The research-treatment distinction: a problematic approach for determining which activities should have ethical oversight Hastings Cent Rep 2013 Spec No S4 S15 23315895
  23. [33]Sugarman J Califf RM Ethics and regulatory complexities for pragmatic clinical trials JAMA 2014 311 2381 2382 24810723
  24. [34]Anderson ML Califf RM Sugarman J Ethical and regulatory issues of pragmatic cluster randomized trials in contemporary health systems Clin Trials 2015 12 276 286 25733677 PMC4498459
  25. [35]Whicher DM Miller JE Dunham KM Gatekeepers for pragmatic clinical trials Clin Trials 2015 in press PMC4592478 26374683
  26. [36]O’Rourke PP Carrithers J Patrick-Lake B Harmonization and streamlining of research oversight for pragmatic clinical trials Clin Trials 2015 in press PMC4592396 26374678
  27. [37]Finkelstein JA Brickman AL Capron A Oversight on the borderline: quality improvement and pragmatic research Clin Trials 2015 in press PMC4699562 26374685
  28. [38]Ali J Andrews JE Jr Somkin CP Harms, benefits, and the nature of interventions in pragmatic clinical trials Clin Trials 2015 in press PMC4592413 26374680
  29. [39]Smalley JB Merritt MW Al-Khatib SM Ethical responsibilities toward indirect and collateral participants in pragmatic clinical trials Clin Trials 2015 in press PMC4812163 26374687
  30. [40]Lantos JD Wendler D Septimus E Considerations in the evaluation and determination of minimal risk in pragmatic clinical trials Clin Trials 2015 in press PMC4703320 26374686
  31. [41]McKinney RE Jr Beskow LM Ford DE Use of altered informed consent in pragmatic clinical research Clin Trials 2015 in press PMC4688909 26374677
  32. [42]Welch MJ Lally R Miller JE The ethics and regulatory landscape of including vulnerable populations in pragmatic clinical trials Clin Trials 2015 in press PMC4662375 26374681
  33. [43]Anderson ML Griffin J Goldkind SF The Food and Drug Administration and pragmatic clinical trials of marketed medical products Clin Trial 2015 in press PMC4592418 26374684
  34. [44]McGraw D Greene SM Miner CS Privacy and confidentiality in pragmatic clinical trials Clin Trials 2015 in press PMC4702499 26374682
  35. [45]Ellenberg SS Culbertson R Gillen DL Data monitoring committees for pragmatic clinical trials Clin Trials 2015 in press PMC4592397 26374679
  36. [46]Weijer C Grimshaw JM Eccles MP The Ottawa Statement on the ethical design and conduct of cluster randomized trials PLoS Med 2012 9 e1001346 23185138 PMC3502500

Cited by

Sorted by citations

Authors
Califf, R. M., & Sugarman, J.
Affiliations
  • United States Food and Drug Administration (US)
  • Johns Hopkins University (US)
Corresponding author
Califf, Robert M
ORCID
Venue
Clinical Trials
Publisher
SAGE Publications
Published
2015-09-15
Volume
12
Issue
5
Pages
436-441
ISSN
  • 1740-7745 (ISSN-L)
  • 1740-7753
DOI
10.1177/1740774515598334
PMC
PMC4891194
OpenAlex
W2115742677
Cited by
191
References
2
Citations by year
2026: 15 · 2025: 14 · 2024: 14 · 2023: 13 · 2022: 17 · 2021: 20 · 2020: 27 · 2019: 21 · 2018: 14 · 2017: 12 · 2016: 14 · 2015: 10
FWCI
17.87
Type
journal article
Language
en
Open access
Licence
journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
Locations
  • Clinical Trials (publishedVersion)
  • PubMed (publishedVersion)
  • PubMed Central (acceptedVersion)
Funders
National Center for Complementary and Integrative Health (U54AT007748)
Topics
  • Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Ethics in Clinical Research
  • Health Sciences › Medicine › Physiology › Biomedical Ethics and Regulation
  • Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
Keywords
pragmatic trials, research ethics, regulatory oversight, evidence gaps, real-world evidence, information and communication technologies, PCORnet, clinical research regulation
MeSH terms
Clinical Trials as Topic, Decision Making, Humans, National Institutes of Health (U.S.), Surveys and Questionnaires, Research Design, United States, Biomedical Research

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review