CoChat

Clinical trial metadata: defining and extracting metadata on the design, conduct, results and costs of 125 randomised clinical trials funded by the National Institute for Health Research Health Technology Assessment programme

Open access
Raftery, James, Young, Amanda, Stanton, Louise, Milne, Ruairidh, Cook, Andrew, Turner, David, Davidson, Peter · 2015 · Health Technology Assessment
Cited by 74

The DOI matches its Crossref record; 27 of the 27 references it cites were found in a registry or an index.

Abstract

Background: By 2011, the Health Technology Assessment (HTA) programme had published the results of over 100 trials with another 220 in progress. The aim of the project was to develop and pilot ‘metadata’ on clinical trials funded by the HTA programme. Objectives: The aim of the project was to develop and pilot questions describing clinical trials funded by the HTA programme in terms of it meeting the needs of the NHS with scientifically robust studies. The objectives were to develop relevant classification systems and definitions for use in answering relevant questions and to assess their utility. Data sources: Published monographs and internal HTA documents. Review methods: A database was developed, ‘populated’ using retrospective data and used to answer questions under six prespecified themes. Questions were screened for feasibility in terms of data availability and/or ease of extraction. Answers were assessed by the authors in terms of completeness, success of the classification system used and resources required. Each question was scored to be retained, amended or dropped. Results: One hundred and twenty-five randomised trials were included in the database from 109 monographs. Neither the International Standard Randomised Controlled Trial Number nor the term ‘randomised trial’ in the title proved a reliable way of identifying randomised trials. Only limited data were available on how the trials aimed to meet the needs of the NHS. Most trials were shown to follow their protocols but updates were often necessary as hardly any trials recruited as planned. Details were often lacking on planned statistical analyses, but we did not have access to the relevant statistical plans. Almost all the trials reported on cost-effectiveness, often in terms of both the primary outcome and quality-adjusted life-years. The cost of trials was shown to depend on the number of centres and the duration of the trial. Of the 78 questions explored, 61 were well answered, 33 fully with 28 requiring amendment were the analysis updated. The other 17 could not be answered with readily available data. Limitations: The study was limited by being confined to 125 randomised trials by one funder. Conclusions: Metadata on randomised controlled trials can be expanded to include aspects of design, performance, results and costs. The HTA programme should continue and extend the work reported here. Funding: The National Institute for Health Research HTA programme.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.3310/hta19110

Cites

27 of 27 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Research works printed but not linked: 89

  1. Raftery, 2013, Health Technology Assessment in the UK, Lancet, 382, 1278, 10.1016/S0140-6736(13)61724-9 · DOI
  2. Hanney, 2007, An assessment of the impact of the NHS Health Technology Assessment Programme, Health Technol Assess, 11, 10.3310/hta11530 · DOI
  3. Buxton, 1996, How can payback from health services research be assessed?, J Health Serv Res Policy, 1, 35, 10.1177/135581969600100107 · DOI · Paper page
  4. Dent, 2011, Treatment success in pragmatic randomised controlled trials: a review of trials funded by the UK Health Technology Assessment programme, Trials, 12, 109, 10.1186/1745-6215-12-109 · DOI
  5. Djulbegovic, 2008, Treatment success in cancer: new cancer treatment successes identified in phase 3 randomized controlled trials conducted by the National Cancer Institute-sponsored cooperative oncology groups, 1955 to 2006, Arch Intern Med, 168, 632, 10.1001/archinte.168.6.632 · DOI · Paper page
  6. Jolly, 2007, The Birmingham Rehabilitation Uptake Maximisation Study (BRUM). Home-based compared with hospital-based cardiac rehabilitation in a multi-ethnic population: cost effectiveness and patient adherence, Health Technol Assess, 11, 10.3310/hta11350 · DOI · Paper page
  7. Ridyard, 2010, Methods for the collection of resource use data within clinical trials: a systematic review of studies funded by the UK Health Technology Assessment Program, Value Health, 13, 867, 10.1111/j.1524-4733.2010.00788.x · DOI
  8. Chase, 2000, What are the relative merits of the sources used to identify potential research priorities for the NHSHTA programme?, Int J Technol Assess Health Care, 16, 743, 10.1017/S0266462300102028 · doi:10.1017/s0266462300102028
  9. Drummond, 1996, Guidelines for authors and peer reviewers of economic submissions to the BMJ. The BMJ Economic Evaluation Working Party, BMJ, 313, 275, 10.1136/bmj.313.7052.275 · DOI · Paper page
  10. 2005, Attributing Revenue Costs of Externally-Funded Non-Commercial Research in the NHS (ARCO)
  11. Viergever, 2011, The quality of registration of clinical trials, PLOS ONE, 6, e14701, 10.1371/journal.pone.0014701 · DOI · Paper page
  12. International Committee of Medical Journal Editors (ICMJE). ICMJE Recommendations (‘The Uniform Requirements’). 2004. URL: www.icmje.org/about-icmje/faqs/icmje-recommendations/ (accessed 29 October 2014).
  13. Raftery, 2012, Registration of noncommercial randomised clinical trials: the feasibility of using trial registries to monitor the number of trials, Trials, 13, 140, 10.1186/1745-6215-13-140 · doi:10.1186/1745-6215-13-140
  14. Reveiz, 2010, Influence of trial registration on reporting quality of randomized trials: study from highest ranked journals, J Clin Epidemiol, 63, 1216, 10.1016/j.jclinepi.2010.01.013 · DOI
  15. Moja, 2009, Compliance of clinical trial registries with the World Health Organization minimum data set: a survey, Trials, 10, 56, 10.1186/1745-6215-10-56 · DOI · Paper page
  16. Ghersi, 2008, Reporting the findings of clinical trials: a discussion paper, Bull World Health Organ, 86, 492, 10.2471/BLT.08.053769 · DOI · Paper page
  17. Sekeres, 2008, Poor reporting of scientific leadership information in clinical trial registers, PLOS ONE, 3, e1610, 10.1371/journal.pone.0001610 · DOI · Paper page
  18. Mathieu, 2009, Comparison of registered and published primary outcomes in randomized controlled trials, JAMA, 302, 977, 10.1001/jama.2009.1242 · DOI · Paper page
  19. Liberati, 2009, The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration, PLOS Med, 6, e1000100, 10.1371/journal.pmed.1000100 · DOI · Paper page
  20. Thabane, 2010, A tutorial on pilot studies: the what, why and how, BMC Med Res Methodol, 10, 1, 10.1186/1471-2288-10-1 · DOI · Paper page
  21. Schwartz, 2009, Explanatory and pragmatic attitudes in therapeutical trials, J Clin Epidemiol, 62, 499, 10.1016/j.jclinepi.2009.01.012 · DOI · Paper page
  22. Cooksey, 2006, A Review of UK Health Research Funding
  23. Chalkidou, 2009, Comparative effectiveness research priorities: identifying critical gaps in evidence for clinical and health policy decision making, Int J Technol Assess Health Care, 25, 241, 10.1017/S0266462309990225 · doi:10.1017/s0266462309990225
  24. Jones, 1995, Setting priorities for research and development in the NHS: a case study on the interface between primary and secondary care, BMJ, 311, 1076, 10.1136/bmj.311.7012.1076 · DOI
  25. Noorani, 2007, Priority setting for health technology assessments: a systematic review of current practical approaches, Int J Technol Assess Health Care, 23, 310, 10.1017/S026646230707050X · DOI
  26. Tomlinson, 2009, Research priorities for health of people with disabilities: an expert opinion exercise, Lancet, 374, 1857, 10.1016/S0140-6736(09)61910-3 · DOI
  27. Sibbald, 2009, Priority setting: what constitutes success? A conceptual framework for successful priority setting, BMC Health Serv Res, 9, 43, 10.1186/1472-6963-9-43 · DOI · Paper page
  28. Gandhi, 2008, Patient-important outcomes in registered diabetes trials, JAMA, 299, 2543, 10.1001/jama.299.21.2543 · DOI · Paper page
  29. Montori, 2006, Systematic evaluation of the quality of randomized controlled trials in diabetes, Diabetes Care, 29, 1833, 10.2337/dc06-0077 · DOI · Paper page
  30. Rahimi, 2010, Outcome selection and role of patient reported outcomes in contemporary cardiovascular trials: systematic review, BMJ, 341, 10.1136/bmj.c5707 · DOI · Paper page
  31. Chalmers, 2009, Avoidable waste in the production and reporting of research evidence, Lancet, 374, 86, 10.1016/S0140-6736(09)60329-9 · DOI · Paper page
  32. Schulz, 1995, Empirical evidence of bias – dimensions of methodological quality associated with estimates of treatment effects in controlled trials, JAMA, 273, 408, 10.1001/jama.1995.03520290060030 · DOI
  33. Thorpe, 2009, A pragmatic-explanatory continuum indicator summary (PRECIS): a tool to help trial designers, J Clin Epidemiol, 62, 464, 10.1016/j.jclinepi.2008.12.011 · DOI · Paper page
  34. Begg, 1996, Improving the quality of reporting of randomized controlled trials: the CONSORT statement, JAMA, 276, 637, 10.1001/jama.1996.03540080059030 · DOI · Paper page
  35. Chalmers, 2003, Descriptive survey of non-commercial randomised controlled trials in the United Kingdom, 1980–2002, BMJ, 327, 1017, 10.1136/bmj.327.7422.1017 · DOI · Paper page
  36. Williams, 2006, What are the clinical outcome and cost effectiveness of endoscopy undertaken by nurses when compared with doctors? A Multi-Institution Nurse Endoscopy Trial (MINuET), Health Technol Assess, 10, 10.3310/hta10400 · DOI
  37. Walley, 2007, Health technology assessment in England: assessment and appraisal, Med J Aust, 187, 283, 10.5694/j.1326-5377.2007.tb01244.x · doi:10.5694/j.1326-5377.2007.tb01244.x
  38. 2011, A New Pathway for the Regulation and Governance of Health Research
  39. Gajewski, 2008, Predicting accrual in clinical trials with Bayesian posterior predictive distributions, Stat Med, 27, 2328, 10.1002/sim.3128 · DOI
  40. Williford, 1987, The ‘constant intake rate’ assumption in interim recruitment goal methodology for multicenter clinical trials, J Chronic Dis, 40, 297, 10.1016/0021-9681(87)90045-2 · doi:10.1016/0021-9681(87)90045-2
  41. Carter, 2005, Practical considerations for estimating clinical trial accrual periods: application to a multi-center effectiveness study, BMC Med Res Methodol, 5, 11, 10.1186/1471-2288-5-11 · DOI
  42. Moussa, 1984, Planning a clinical trial with allowance for cost and patient recruitment rate, Comput Programs Biomed, 18, 173, 10.1016/0010-468X(84)90049-7 · doi:10.1016/0010-468x(84)90049-7
  43. Buchan, 2010, Do recorded abstracts from scientific meetings concur with the research presented?, Eye, 24, 695, 10.1038/eye.2009.133 · DOI
  44. Watson, 2006, Increasing recruitment to randomised trials: a review of randomised controlled trials, BMC Med Res Methodol, 6, 34, 10.1186/1471-2288-6-34 · DOI · Paper page
  45. Fletcher, 2012, Improving the recruitment activity of clinicians in randomised controlled trials: a systematic review, BMJ Open, 2, e000496, 10.1136/bmjopen-2011-000496 · DOI · Paper page
  46. Booker, 2011, A systematic review of the effect of retention methods in population-based cohort studies, BMC Public Health, 11, 249, 10.1186/1471-2458-11-249 · DOI · Paper page
  47. Meyers, 2003, What does it take to retain substance-abusing adolescents in research protocols? Delineation of effort required, strategies undertaken, costs incurred, and 6-month post-treatment differences by retention difficulty, Drug Alcohol Depend, 69, 73, 10.1016/S0376-8716(02)00252-1 · DOI
  48. Fisher, 2012, AASAP: A program to increase recruitment and retention in clinical trials, Patient Educ Couns, 86, 372, 10.1016/j.pec.2011.07.002 · DOI · Paper page
  49. Donovan, 2003, Prostate Testing for Cancer and Treatment (ProtecT) feasibility study, Health Technol Assess, 7, 10.3310/hta7140 · DOI · Paper page
  50. Lane, 2011, A Peer Review Intervention for Monitoring and Evaluating sites (PRIME) that improved randomized controlled trial conduct and performance, J Clin Epidemiol, 64, 628, 10.1016/j.jclinepi.2010.10.003 · DOI
  51. Barnard, 2010, A systematic review of models to predict recruitment to multicentre clinical trials, BMC Med Res Methodol, 10, 63, 10.1186/1471-2288-10-63 · DOI
  52. 1997, Responsibilities for Meeting Patient Care Costs Associated with Research and Development in the NHS. HSG(97)32
  53. Ewart, 2009, Undisclosed changes in outcomes in randomized controlled trials: an observational study, Ann Fam Med, 7, 542, 10.1370/afm.1017 · DOI · Paper page
  54. Kavvoura, 2008, Evaluation of the potential excess of statistically significant findings in published genetic association studies: application to Alzheimer’s disease, Am J Epidemiol, 168, 855, 10.1093/aje/kwn206 · DOI · Paper page
  55. Pildal, 2005, Comparison of descriptions of allocation concealment in trial protocols and the published reports: cohort study, BMJ, 330, 1049, 10.1136/bmj.38414.422650.8F · DOI · Paper page
  56. Scharf, 2006, Adverse event reporting in publications compared with sponsor database for cancer clinical trials, J Clin Oncol, 24, 3933, 10.1200/JCO.2005.05.3959 · DOI · Paper page
  57. Vedula, 2009, Outcome reporting in industry-sponsored trials of gabapentin for off-label use, N Engl J Med, 361, 1963, 10.1056/NEJMsa0906126 · DOI · Paper page
  58. Bland, 2009, The tyranny of power: is there a better way to calculate sample size?, BMJ, 339, b3985, 10.1136/bmj.b3985 · DOI · Paper page
  59. Hernández, 2005, Subgroup analysis and covariate adjustment in randomized clinical trials of traumatic brain injury: a systematic review, Neurosurgery, 57, 1244, 10.1227/01.NEU.0000186039.57548.96 · DOI · Paper page
  60. Pocock, 2007, Figures in clinical trial reports: current practice and scope for improvement, Trials, 8, 36, 10.1186/1745-6215-8-36 · DOI
  61. Djulbegovic, 2012, New treatments compared to established treatments in randomized trials (review), Cochrane Database Syst Rev, 10, MR000024
  62. Anderson, 1986, Policy space areas and properties of benefit cost/utility analysis, JAMA, 255, 794, 10.1001/jama.1986.03370060108029 · doi:10.1001/jama.1986.03370060108029
  63. Black, 1990, The CE plane: a graphic representation of cost-effectiveness, Med Decis Mak, 10, 212, 10.1177/0272989X9001000308 · DOI · Paper page
  64. Drummond, 2005, Methods for the Economic Evaluation of Health Care Programmes, 10.1093/oso/9780198529446.001.0001 · DOI · Paper page
  65. Raftery, 2009, Should NICE’s threshold range for cost per QALY be raised? No, BMJ, 338, b185, 10.1136/bmj.b185 · DOI
  66. Towse, 2009, Should NICE’s threshold range for cost per QALY be raised? Yes, BMJ, 338, b181, 10.1136/bmj.b181 · DOI
  67. Petrou, 2011, Economic evaluation alongside randomised controlled trials: design, conduct, analysis, and reporting, BMJ, 342, 14, 10.1136/bmj.d1548 · DOI · Paper page
  68. Petrou, 2011, Economic evaluation using decision analytical modelling: design, conduct, analysis, and reporting, BMJ, 342, d1766, 10.1136/bmj.d1766 · DOI · Paper page
  69. Sculpher, 2006, Whither trial-based economic evaluation for health care decision making?, Health Econ, 15, 677, 10.1002/hec.1093 · DOI · Paper page
  70. Barber, 1998, Analysis and interpretation of cost data in randomised controlled trial: review of published studies, BMJ, 317, 1195, 10.1136/bmj.317.7167.1195 · DOI · Paper page
  71. Doshi, 2006, Analyses of cost data in economic evaluations conducted alongside randomized controlled trials, Value Health, 9, 334, 10.1111/j.1524-4733.2006.00122.x · DOI
  72. O’Sullivan, 2005, Collection of health economic data alongside clinical trials: is there a future for piggyback evaluations?, Value Health, 8, 67, 10.1111/j.1524-4733.2005.03065.x · DOI
  73. Glick, 2007, Economic Evaluation in Clinical Trials
  74. Evers, 2005, Criteria list for assessment of methodological quality of economic evaluations: consensus on health economic criteria, Int J Technol Assess Health Care, 21, 240, 10.1017/S0266462305050324 · DOI · Paper page
  75. Chiou, 2003, Development and validation of a grading system for the quality of cost-effectiveness studies, Med Care, 41, 32, 10.1097/00005650-200301000-00007 · DOI · Paper page
  76. 2005, Methods for the Economic Evaluation of Health Care Programmes, 27
  77. Briggs, 2001, The death of cost-minimization analysis?, Health Econ, 10, 179, 10.1002/hec.584 · DOI · Paper page
  78. Kaitin, 2011, Pharmaceutical innovation in the 21st century: new drug approvals in the first decade, 2000–2009, Clin Pharmacol Ther, 89, 183, 10.1038/clpt.2010.286 · DOI · Paper page
  79. Adams, 2010, Spending on new drug development, Health Econ, 19, 130, 10.1002/hec.1454 · DOI · Paper page
  80. Light, 2011, Demythologizing the high costs of pharmaceutical research, BioSocieties, 6, 34, 10.1057/biosoc.2010.40 · DOI · Paper page
  81. Hackshaw, 2008, Setting up non-commercial clinical trials takes too long in the UK: findings from a perspective study, J R Soc Med, 101, 299, 10.1258/jrsm.2008.070373 · doi:10.1258/jrsm.2008.070373
  82. Hutchings, 2011, A1 financial and time costs of R&D governance and regulation in England and Wales: evidence from the SAFER 2 trial, Emerg Med J, 28, e2, 10.1136/emermed-2011-200645.1 · doi:10.1136/emermed-2011-200645.1
  83. Snowdon, 2006, Financial considerations in the conduct of multi-centre randomised controlled trials: evidence from a qualitative study, Trials, 7, 34, 10.1186/1745-6215-7-34 · DOI
  84. Al-Shahi, 2007, Research governance impediments to clinical trials: a retrospective survey, J R Soc Med, 100, 101, 10.1258/jrsm.100.2.101 · doi:10.1258/jrsm.100.2.101
  85. Martin, 2010, Identifying and eliminating the roadblocks to comparative-effectiveness research, N Engl J Med, 363, 105, 10.1056/NEJMp1001201 · DOI
  86. 2012, Attributing the costs of health and social care Research & Development (AcoRD)
  87. Bailey, 1998, Research and Development for the NHS, 59
  88. Dent, 2011, ‘Flogging dead horses’: evaluating when have clinical trials achieved sufficiency and stability? A case study in cardiac rehabilitation, Trials, 12, 83, 10.1186/1745-6215-12-83 · DOI
  89. International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use (ICH). Statistical Principles For Clinical Trials. 1998. URL: www.ich.org (accessed 24 November 2014).

Cited by

See all 74

Metadata

Authors
Raftery, J., Young, A., Stanton, L., Milne, R., Cook, A., Turner, D., & Davidson, P.
Venue
Health Technology Assessment
Published
2015-02
Volume
19
Issue
11
Pages
1-138
ISSN
  • 1366-5278 (ISSN-L)
  • 2046-4924
OpenAlex
W2022238080
Cited by
74
References
27
FWCI
3.59
Type
journal article
Language
en
Open access
diamond
Funders
National Institute for Health Research (08/117/01)

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.