The opportunities and challenges of pragmatic point-of-care randomised trials using routinely collected electronic records: evaluations of two exemplar trials
Open accessThe DOI matches its Crossref record; 37 of the 37 references it cites were found in a registry or an index.
Abstract
BACKGROUND: Pragmatic trials compare the effects of different decisions in usual clinical practice. OBJECTIVES: To develop and evaluate methods to implement simple pragmatic trials using routinely collected electronic health records (EHRs) and recruiting patients at the point of care; to identify the barriers and facilitators for general practitioners (GPs) and patients and the experiences of trial participants. DESIGN: Two exemplar randomised trials (Retropro and eLung) with qualitative evaluations. SETTING: Four hundred and fifty-nine English and Scottish general practices contributing EHRs to a research database, of which 17 participated in the trials. PARTICIPANTS: Retropro aimed to recruit 300 patients with hypercholesterolaemia and high cardiovascular risk and eLung aimed to recruit 150 patients with a chronic obstructive pulmonary disease exacerbation. INTERVENTIONS: Retropro randomised between simvastatin and atorvastatin and eLung between immediate antibiotics and deferred or non-use. eLung recruited during an unscheduled consultation using EHR flagging. MAIN OUTCOME MEASURE: Successful trial completion with implementation of information technology (IT) system for flagging and data processing and documentation of operational and scientific experiences. DATA SOURCES: EHR research database. RESULTS: The governance approval process took over 3 years. A total of 58.8% of the practices (n = 270) expressed interest in participating. The number of interested practices dropped substantially with each stage of the governance process. In Retropro, 6.5% of the practices (n = 30) were eventually approved and 3.7% (n = 17) recruited patients; in eLung, these numbers were 6.8% (n = 31) and 1.3% (n = 6) respectively. Retropro successfully completed recruitment (301 patients) whereas eLung recruited 31 patients. Retropro recruited 20.6% of all statin starters in recruiting practices and 1.1% in the EHR database; the comparable numbers for eLung were 32.3% and 0.9% respectively. The IT system allowed for complex eligibility criteria with central on and off control of recruitment and flagging at a practice. Good Clinical Practice guidelines, governance and consent procedures were found to have substantially affected the intended simple nature of the trials. One qualitative study of 13 clinicians found that clinicians were generally positive about the principle of computerised trial recruitment (flagging during consultation). However, trials which did not include patients with acute illness were favoured. The second qualitative process evaluation interviewed 27 GPs about their actual experiences, including declining, recruiting and non-recruiting GPs. Opportunistic patient recruitment during a routine GP consultation was found to be the most controversial element. The actual experiences of recruiting patients during unscheduled consultation were generally more positive than the hypothetical views of GPs. Several of the recruiting GPs reported the process took 5 minutes and was straightforward and feasible on most occasions. Almost all GPs expressed their strong support for the use of EHRs for trials. Ten eLung participants were interviewed, all of whom considered it acceptable to be recruited during a consultation and to use EHRs for trials. CONCLUSIONS: EHR point-of-care trials are feasible, although the recruitment of clinicians is a major challenge owing to the complexity of trial approvals. These trials will provide substantial evidence on clinical effectiveness only if trial interventions and participating clinicians and patients are typical of usual clinical care and trials are simple to initiate and conduct. Recommendations for research include the development of evidence and implementation of risk proportionality in trial governance and conduct. TRIAL REGISTRATION: Current Controlled Trials ISRCTN33113202 and ISRCTN72035428. FUNDING: This project was funded by the NIHR Health Technology Assessment programme and the Wellcome Trust and will be published in full in Health Technology Assessment; Vol. 18, No. 43. See the NIHR Journals Library website for further project information.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Methods for the Economic Evaluation of Health Care Programmes
Drummond, M. E., Sculpher, M. J., Torrance, G. W., O’Brien, B. J., & Stoddart, G. L. (2005) · doi:10.1093/oso/9780198529446.001.0001 · Cited by 13,648
Correspondence - Tranexamic acid for traumatic brain injury
(2005) · London School of Economics and Political Science Theses Online (London School of Economics and Political Science) · doi:10.1016/s0140-6736 · Cited by 11,894
Qualitative research in health care: Analysing qualitative data
Pope, C. (2000) · BMJ · doi:10.1136/bmj.320.7227.114 · Cited by 8,131
Qualitative data analysis for applied policy research
Ritchie, J., & Spencer, L. (2010) · Analyzing qualitative data · doi:10.4324/9780203413081_chapter_9 · Cited by 4,800
Black, D. M., Cummings, S. R., Karpf, D. B., Cauley, J. A., Thompson, D. E., Nevitt, M. C., Bauer, D. C., Genant, H. K., Haskell, W. L., Marcus, R., et al. (1996) · The Lancet · doi:10.1016/s0140-6736(96)07088-2 · Cited by 3,846
Decision Modelling For Health Economic Evaluation
Briggs, A., Claxton, K., & Sculpher, M. (2006) · doi:10.1093/oso/9780198526629.001.0001 · Cited by 3,153
Wood, L., Egger, M., Gluud, L. L., Schulz, K. F., Jüni, P., Altman, D. G., Gluud, C., Martin, R. M., Wood, A. J. G., & Sterne, J. A. C. (2008) · BMJ · doi:10.1136/bmj.39465.451748.ad · Cited by 2,538 · cited 1×
“A review of 146 meta-analyses found that effect estimates were exaggerated for subjective outcomes when there was inadequate or unclear allocation concealment or lack of blinding, whereas there was little evidence of bias in trials with objective outcomes.(143)”
Cardiovascular disease risk profiles
Anderson, K. M., Odell, P. M., Wilson, P. W. F., & Kannel, W. B. (1991) · American Heart Journal · doi:10.1016/0002-8703(91)90861-b · Cited by 2,163
Chan, A.-W., Hróbjartsson, A., Haahr, M. T., Gøtzsche, P. C., & Altman, D. G. (2004) · JAMA · doi:10.1001/jama.291.20.2457 · Cited by 1,845
Tunis (2003) · doi:10.1001/jama.290.12.1624 · Cited by 1,724
Predicting cardiovascular risk in England and Wales: prospective derivation and validation of QRISK2
Hippisley-Cox, J., Coupland, C., Vinogradova, Y., Robson, J., Minhas, R., Sheikh, A., & Brindle, P. (2008) · BMJ · doi:10.1136/bmj.39609.449676.25 · Cited by 1,459 · cited 1×
“The CVD risks as predicted by QRISK2 and Framingham risk score were then estimated.(33,34) The Framingham risk score uses an algorithm based on age, sex, ethnicity, smoking status, systolic blood pressure, ratio of cholesterol to high-density lipoprotein and history of diabetes mellitus and left ventricular hypertroph…”
MethodA pragmatic–explanatory continuum indicator summary (PRECIS): a tool to help trial designers
Thorpe, K. E., Zwarenstein, M., Oxman, A. D., Treweek, S., Furberg, C. D., Altman, D. G., Tunis, S., Bergel, E., Harvey, I., Magid, D. J., et al. (2009) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2008.12.011 · Cited by 1,389
Weiskopf, N. G., & Weng, C. (2013) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2011-000681 · Cited by 1,216 · cited 1×
“There is a clear research need to develop and adopt systematic, statistically based methods of data quality assessment.(176) Furthermore, the development of common data models and dictionaries could simplify research across different EHR research databases.”
BackgroundComponents of placebo effect: randomised controlled trial in patients with irritable bowel syndrome
Kaptchuk, T. J., Kelley, J. M., Conboy, L. A., Davis, R. B., Kerr, C. E., Jacobson, E. E., Kirsch, I., Schyner, R. N., Nam, B. H., Nguyen, L. T., et al. (2008) · BMJ · doi:10.1136/bmj.39524.439618.25 · Cited by 1,196 · cited 1×
“An example of this is a study that reported substantive differences in symptoms of inflammatory bowel disease between an attentive patient–clinician relationship compared with being on the waiting list.(140) Bias may also occur if the measurements and recording of the trial outcomes vary between allocated treatments.”
BackgroundExplanatory and pragmatic attitudes in therapeutical trials
Schwartz, D., & Lellouch, J. (1967) · Journal of Chronic Diseases · doi:10.1016/0021-9681(67)90041-0 · Cited by 1,185 · cited 2×
“A different type of trial (known as a pragmatic trial) is aimed at the decision about which of two clinical decisions should be preferred.(1) An important difference between explanatory and pragmatic trials concerns the handling of interventions with multiple components.”
BackgroundThe global burden of women's cancers: a grand challenge in global health
Ginsburg, O., Bray, F., Coleman, M. P., Vanderpuye, V., Eniu, A., Kotha, S. R., Sarker, M., Huong, T. T., Allemani, C., Dvaladze, A., et al. (2017) · The Lancet · doi:10.1016/s0140-6736(16)31392-7 · Cited by 1,127
Scientific evidence underlying the ACC/AHA clinical practice guidelines.
Tricoci, P. (2009) · JAMA · doi:10.1001/jama.2009.205 · Cited by 908 · cited 1×
“recommendations).(28) A recent analysis by John Ioannidis found that only 1 of the 24 blockbuster medicines (with sales > US$1B) had been studied in a trial with more than 10,000 participants.”
BackgroundBarriers to Participation in Randomised Controlled Trials
Ross, S., Grant, A., Counsell, C., Gillespie, W., Russell, I., & Prescott, R. (1999) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(99)00141-9 · Cited by 907
The placebo response in medicine: minimize, maximize or personalize?
Enck, P., Bingel, U., Schedlowski, M., & Rief, W. (2013) · Nature Reviews Drug Discovery · doi:10.1038/nrd3923 · Cited by 671 · cited 2×
“Enck et al.(142) discussed how the placebo effect should be managed.”
BackgroundBrookes, S. T., Whitley, E., Peters, T. J., Mulheran, P. A., Egger, M., & Davey Smith, G. (2001) · Health Technology Assessment · doi:10.3310/hta5330 · Cited by 580
Integrating qualitative research with trials in systematic reviews
Thomas, J., Harden, A., Oakley, A., Oliver, S., Sutcliffe, K., Rees, R., Brunton, G., & Kavanagh, J. (2004) · BMJ · doi:10.1136/bmj.328.7446.1010 · Cited by 515 · cited 1×
“In particular, it is more common to use qualitative methods to develop the design, conduct, outcomes and efficiency of clinical trials(86) or to enable health professionals, service users and other stakeholders to contribute their views and experiences to evaluation of treatments, interventions, policies or methodolog…”
MethodA rational framework for decision making by the National Institute for Clinical Excellence (NICE)
Claxton, K., Sculpher, M., & Drummond, M. (2002) · The Lancet · doi:10.1016/s0140-6736(02)09832-x · Cited by 456
A rational model for assessing and evaluating complex interventions in health care
May, C. (2006) · BMC Health Services Research · doi:10.1186/1472-6963-6-86 · Cited by 447
Long-term follow-up of the West of Scotland Coronary Prevention Study
Ford, I., Murray, H., Packard, C. J., Shepherd, J., Macfarlane, P. W., & Cobbe, S. M. (2007) · New England Journal of Medicine · doi:10.1056/nejmoa065994 · Cited by 416
Oliver, S., Clarke-Jones, L., Rees, R., Milne, R., Buchanan, P., Gabbay, J., Gyte, G., Oakley, A., & Stein, K. (2004) · Health Technology Assessment · doi:10.3310/hta8150 · Cited by 400 · cited 1×
“The need to include patients in developing research agendas is now well established.(164,165) Clinicians and patients should be involved in the development of the study protocol and review the need for any study-specific procedures.”
BackgroundFactors that limit the quality, number and progress of randomised controlled trials
Prescott, Counsell, Gillespie, Grant, Russell, Kiauka, Colthart, Ross, Shepherd, & Russell (1999) · Health Technology Assessment · doi:10.3310/hta3200 · Cited by 390 · cited 2×
“Concern for the doctor (or clinician)–patient relationship has frequently been cited as an important barrier.(52,53,61,62,67,69) In contrast, such concern was not demonstrated in this study and, in fact, it was thought that such a relationship might facilitate recruitment.”
BackgroundStrategies to improve recruitment to randomised controlled trials
Treweek, S., Mitchell, E., Pitkethly, M., Cook, J., Kjeldstrøm, M., Johansen, M., Taskila, T. K., Sullivan, F., Wilson, S., Jackson, C., et al. (2010) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000013.pub5 · Cited by 378 · cited 1×
“This is consistent with the literature as demonstrated by the Cochrane systematic review.(112) The 10 QUEAN study patient participants found recruitment at the point of care and use of EHRs to collect outcome data as acceptable including one patient who expressed a preference for receiving the patient study informatio…”
ResultLarge-scale randomized evidence: Large, simple trials and overviews of trials
Peto, R., Collins, R., & Gray, R. (1995) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(94)00150-o · Cited by 365
Bias in identifying and recruiting participants in cluster randomised trials: what can be done?
Eldridge, S., Kerry, S., & Torgerson, D. J. (2009) · BMJ · doi:10.1136/bmj.b4006 · Cited by 159
Self-citationPreventing Bias in Cluster Randomised Trials
Giraudeau, B., & Ravaud, P. (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000065 · Cited by 150 · cited 1×
“However, there are several limitations with cluster trials.(153,154) One limitation occurs if there is variability between practices.”
BackgroundRandomised clinical trials in general practice: lessons from a failure.
Tognoni, G., Alli, C., Avanzini, F., Bettelli, G., Colombo, F., Corso, R., Marchioli, R., & Zussino, A. (1991) · BMJ · doi:10.1136/bmj.303.6808.969 · Cited by 139
Salmon, P., Peters, S., Rogers, A., Gask, L., Clifford, R., Iredale, W., Dowrick, C., & Morriss, R. (2007) · Family Practice · doi:10.1093/fampra/cmm015 · Cited by 132 · cited 4×
“This is due to the strengths of in-depth interview techniques, which collect data grounded in the actual experience of the clinician’s recruitment decision.(89,98) Furthermore, Fletcher et al.”
BackgroundInfluentialIncentives and disincentives to participation by clinicians in randomised controlled trials
Rendell, J. M., Merritt, R. K., & Geddes, J. (2007) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000021.pub3 · Cited by 127 · cited 2×
“Concern for the doctor (or clinician)–patient relationship has frequently been cited as an important barrier.(52,53,61,62,67,69) In contrast, such concern was not demonstrated in this study and, in fact, it was thought that such a relationship might facilitate recruitment.”
BackgroundSystematic review to determine whether participation in a trial influences outcome
Vist, G. E., Hagen, K. B., Devereaux, P. J., Bryant, D., Kristoffersen, D. T., & Oxman, A. D. (2005) · BMJ · doi:10.1136/bmj.330.7501.1175 · Cited by 104
The unintended consequences of clinical trials regulations
McMahon, A. D., Conway, D. I., MacDonald, T. M., & McInnes, G. T. (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000131 · Cited by 73
Hetherton, J., Matheson, A., & Robson, M. (2004) · Primary Health Care Research and Development · doi:10.1191/1463423604pc168oa · Cited by 34 · cited 1×
“Thus, providing clinicians with information to demonstrate the need for a trial may aid recruitment.(64,68) The negative impact of a lack of clinical uncertainty was demonstrated by concerns expressed regarding eLung – participants were reluctant to randomise patients to no antibiotics even though no convincing eviden…”
Background
Cited by
Data Resource Profile: Clinical Practice Research Datalink (CPRD)
Herrett, E., Gallagher, A. M., Bhaskaran, K., Forbes, H., Mathur, R., van Staa, T., & Smeeth, L. (2015) · International Journal of Epidemiology · doi:10.1093/ije/dyv098 · Cited by 2,667
“The feasibility of this work has been tested: patients from the CPRD have been recruited to a pharmacogenetic study of statin-induced myopathy,(10,11) practices have been recruited to cluster randomised trials(12,13) and patients have been recruited to pragmatic point-of-care randomised trials.(14) The electronic heal…”
MethodSelf-citationTissot, H. C., Shah, A. D., Brealey, D., Harris, S., Agbakoba, R., Folarin, A., Romao, L., Roguski, L., Dobson, R., & Asselbergs, F. W. (2020) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2020.2977925 · Cited by 61
“Such trials may recruit a more representative population than conventional trials, and the results may better reflect effectiveness in routine clinical practice [25], [28].”
ResultRegistry-based randomised controlled trials: conduct, advantages and challenges—a systematic review
Shiely, F., O Shea, N., Murphy, E., & Eustace, J. (2024) · Trials · doi:10.1186/s13063-024-08209-3 · Cited by 24
Metadata
- Authors
- van Staa, T.-P., Dyson, L., McCann, G., Padmanabhan, S., Belatri, R., Goldacre, B., Cassell, J., Pirmohamed, M., Torgerson, D., Ronaldson, S., Adamson, J., Taweel, A., Delaney, B., Mahmood, S., Baracaia, S., Round, T., Fox, R., Hunter, T., Gulliford, M., & Smeeth, L.
- Venue
- Health Technology Assessment
- Published
- 2014-07
- Volume
- 18
- Issue
- 43
- Pages
- 1-146
- ISSN
- 1366-5278 (ISSN-L)
- 2046-4924
- OpenAlex
- W2018612904
- Cited by
- 155
- References
- 37
- FWCI
- 42.38
- Type
- journal article
- Language
- en
- Open access
- diamond
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.