Assessing the Gold Standard — Lessons from the History of RCTs
The DOI matches its Crossref record; 56 of the 56 references it cites were found in a registry or an index.
Abstract
Randomized, controlled trials have become the gold standard of medical knowledge. Yet their scientific and political history offers lessons about the complexity of medicine and disease and the economic and political forces shaping the production and circulation of knowledge.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Randomized clinical trials (RCT) serve as the most valid study design to inform clinical practice according to evidence-based medicine (EBM) principles.1,2”
From Integrity of randomized clinical trials: Performance of integrity tests and checklists requires assessment (2023) · cites this paper as [1]
Citations
Cites
Optimal Medical Therapy with or without PCI for Stable Coronary Disease
Boden, W. E., O'Rourke, R. A., Teo, K. K., Hartigan, P. M., Maron, D. J., Kostuk, W. J., Knudtson, M., Dada, M., Casperson, P., Harris, C. L., et al. (2007) · New England Journal of Medicine · doi:10.1056/nejmoa070829 · Cited by 4,699
Elkin, I. (1989) · Archives of General Psychiatry · doi:10.1001/archpsyc.1989.01810110013002 · Cited by 2,300
Zika Virus Infection in Pregnant Women in Rio de Janeiro
Brasil, P., Pereira, J. P., Moreira, M. E., Ribeiro Nogueira, R. M., Damasceno, L., Wakimoto, M., Rabello, R. S., Valderramos, S. G., Halai, U.-A., Salles, T. S., et al. (2016) · New England Journal of Medicine · doi:10.1056/nejmoa1602412 · Cited by 2,196
A Controlled Trial of Arthroscopic Surgery for Osteoarthritis of the Knee
Moseley, J. B., O'Malley, K., Petersen, N. J., Menke, T. J., Brody, B. A., Kuykendall, D. H., Hollingsworth, J. C., Ashton, C. M., & Wray, N. P. (2002) · New England Journal of Medicine · doi:10.1056/nejmoa013259 · Cited by 2,149
Impure Science: AIDS, Activism, and the Politics of Knowledge
(1997) · Nature Medicine · doi:10.1038/nm0297-242 · Cited by 1,666
The effectiveness of psychotherapy: the Consumer Reports study
Seligman, M. E. P. (1995) · American Psychologist · doi:10.1037/0003-066x.50.12.965 · Cited by 1,608
Emergence of Zaire Ebola Virus Disease in Guinea
Baize, S., Pannetier, D., Oestereich, L., Rieger, T., Koivogui, L., Magassouba, N., Soropogui, B., Sow, M. S., Keïta, S., De Clerck, H., et al. (2014) · New England Journal of Medicine · doi:10.1056/nejmoa1404505 · Cited by 1,366
TRANSLUMINAL DILATATION OF CORONARY-ARTERY STENOSIS
Grüntzig, A. (1978) · The Lancet · doi:10.1016/s0140-6736(78)90500-7 · Cited by 1,343
Instruments, Randomization, and Learning about Development
Deaton, A. (2010) · Journal of Economic Literature · doi:10.1257/jel.48.2.424 · Cited by 1,286
A review of uses of health care utilization databases for epidemiologic research on therapeutics
Schneeweiss, S., & Avorn, J. (2005) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2004.10.012 · Cited by 1,277
The Oregon Experiment — Effects of Medicaid on Clinical Outcomes
Baicker, K., Taubman, S. L., Allen, H. L., Bernstein, M., Gruber, J. H., Newhouse, J. P., Schneider, E. C., Wright, B. J., Zaslavsky, A. M., & Finkelstein, A. N. (2013) · New England Journal of Medicine · doi:10.1056/nejmsa1212321 · Cited by 1,150
When Experiments Travel: Clinical Trials and the Global Search for Human Subjects
Petryna, A. (2009) · Princeton University Press eBooks · doi:10.2307/j.ctt7rzzt · Cited by 947
Lurie, P., & Wolfe, S. M. (1997) · New England Journal of Medicine · doi:10.1056/nejm199709183371212 · Cited by 882
The Ethics of Clinical Research in the Third World
Angell, M. (1997) · New England Journal of Medicine · doi:10.1056/nejm199709183371209 · Cited by 863
Reputation and Power: Organizational Image and Pharmaceutical Regulation at the FDA
Carpenter, D. (2014) · Princeton University Press eBooks · doi:10.1515/9781400835119 · Cited by 848
Uneasy alliance—clinical investigators and the pharmaceutical industry
Bodenheimer, T. (2000) · New England Journal of Medicine · doi:10.1056/nejm200005183422024 · Cited by 765
The gold standard : the challenge of evidence-based medicine and standardization in health care
Timmermans, S., & Berg, M. (2003) · Cited by 758
When Experiments Travel: Clinical Trials and the Global Search for Human Subjects
Petryna, A. (2009) · DigitalGeorgetown (Georgetown University Library) · doi:10.1515/9781400830824 · Cited by 755
Deacon, B. J. (2013) · Clinical Psychology Review · doi:10.1016/j.cpr.2012.09.007 · Cited by 643
An Evaluation of Internal-Mammary-Artery Ligation by a Double-Blind Technic
Cobb, L. A., Thomas, G. I., Dillard, D. H., Merendino, K. A., & Bruce, R. A. (1959) · New England Journal of Medicine · doi:10.1056/nejm195905282602204 · Cited by 534
Murphy, M. L., Hultgren, H. N., Detre, K., Thomsen, J., Takaro, T., & Study, P. O. T. V. A. C. (1977) · New England Journal of Medicine · doi:10.1056/nejm197709222971201 · Cited by 473
Persistence of Ebola Virus in Ocular Fluid during Convalescence
Varkey, J. B., Shantha, J. G., Crozier, I., Kraft, C. S., Lyon, G. M., Mehta, A. K., Kumar, G., Smith, J. R., Kainulainen, M. H., Whitmer, S., et al. (2015) · New England Journal of Medicine · doi:10.1056/nejmoa1500306 · Cited by 470
When experiments travel: clinical trials and the global search for human subjects
(2009) · Choice Reviews Online · doi:10.5860/choice.47-1460 · Cited by 449
The Progress of Experiment: Science and Therapeutic Reform in the United States, 1900-1990
Parascandola, J., & Marks, H. M. (1999) · The American Historical Review · doi:10.2307/2650255 · Cited by 419
Of Mice but Not Men: Problems of the Randomized Clinical Trial
Hellman, S., & Hellman, D. S. (1991) · New England Journal of Medicine · doi:10.1056/nejm199105303242208 · Cited by 339
Outcome Reporting Among Drug Trials Registered in ClinicalTrials.gov
Bourgeois, F. T., Murthy, S., & Mandl, K. D. (2010) · Annals of Internal Medicine · doi:10.7326/0003-4819-153-3-201008030-00006 · Cited by 332
Thorlund, J. B., Juhl, C. B., Roos, E. M., & Lohmander, L. (2015) · British Journal of Sports Medicine · doi:10.1136/bjsports-2015-h2747rep · Cited by 316
Ablon, J. S., & Jones, E. E. (2002) · American Journal of Psychiatry · doi:10.1176/appi.ajp.159.5.775 · Cited by 280
Prescribing by numbers: drugs and the definition of disease
(2007) · Choice Reviews Online · doi:10.5860/choice.44-6261 · Cited by 263
Medical Ethics and Controlled Trials
Hill, A. B. (1963) · BMJ · doi:10.1136/bmj.1.5337.1043 · Cited by 261
The history and fate of the gold standard
Jones, D. S., & Podolsky, S. H. (2015) · The Lancet · doi:10.1016/s0140-6736(15)60742-5 · Cited by 250
Self-citationIssues in the Registration of Clinical Trials
Zarin, D. A., Ide, N. C., Tse, T., Harlan, W. R., West, J. C., & Lindberg, D. A. B. (2007) · JAMA · doi:10.1001/jama.297.19.2112 · Cited by 222
Developments in Post-marketing Comparative Effectiveness Research
Schneeweiss, S. (2007) · Clinical Pharmacology & Therapeutics · doi:10.1038/sj.clpt.6100249 · Cited by 197
Powerful medicines: the benefits, risks, and costs of prescription drugs
(2005) · Choice Reviews Online · doi:10.5860/choice.42-5297a · Cited by 191
Evidence-Based Medicine and the Search for a Science of Clinical Care
Daly, J. (2005) · University of California Press eBooks · doi:10.1525/9780520931442 · Cited by 151
Reform, Regulation, and Pharmaceuticals — The Kefauver–Harris Amendments at 50
Greene, J. A., & Podolsky, S. H. (2012) · New England Journal of Medicine · doi:10.1056/nejmp1210007 · Cited by 148
Self-citationCritical Analysis of Coronary Artery Bypass Graft Surgery: A 30-Year Journey
Favaloro, R. G. (1998) · Journal of the American College of Cardiology · doi:10.1016/s0735-1097(97)00559-7 · Cited by 129
Coronary-Artery Surgery at the Crossroads
Braunwald, E. (1977) · New England Journal of Medicine · doi:10.1056/nejm197709222971209 · Cited by 116
Epstein, S. (1997) · Social Studies of Science · doi:10.1177/030631297027005001 · Cited by 114
The Truth and Consequences of the COURAGE Trial
Kereiakes, D. J., Teirstein, P. S., Sarembock, I. J., Holmes, D. R., Krucoff, M. W., O’Neill, W. W., Waksman, R., Williams, D. O., Popma, J. J., Buchbinder, M., et al. (2007) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2007.07.063 · Cited by 113
Are Randomized Trials Appropriate for Evaluating New Operations?
Bonchek, L. I. (1979) · New England Journal of Medicine · doi:10.1056/nejm197907053010110 · Cited by 101
Evidence-Based Medicine and the Search for a Science of Clinical Care
Godwin, M. S. (2005) · JAMA · doi:10.1001/jama.294.11.1429-a · Cited by 87
Jones, D. S. (2000) · Isis · doi:10.1086/384853 · Cited by 85
Self-citationFinancial Conflict-of-Interest Policies in Clinical Research: Issues for Clinical Investigators
Boyd, E. A., Cho, M. K., & Bero, L. A. (2003) · Academic Medicine · doi:10.1097/00001888-200308000-00002 · Cited by 81
The antibiotic era: reform, resistance, and the pursuit of a rational therapeutics
(2015) · Choice Reviews Online · doi:10.5860/choice.190487 · Cited by 69
PUBLICATION BIAS AND META-ANALYSES: A Practical Example
Burdett, S., Stewart, L. A., & Tierney, J. F. (2003) · International Journal of Technology Assessment in Health Care · doi:10.1017/s0266462303000126 · Cited by 67
Drugs and Operations: Some Important Differences
Love, J. W. (1975) · JAMA · doi:10.1001/jama.1975.03250010019016 · Cited by 58
Special Correspondence: A Debate on Coronary Bypass
(1977) · New England Journal of Medicine · doi:10.1056/nejm197712292972612 · Cited by 52
The World Bank is finally embracing science
Lancet, T. (2004) · The Lancet · doi:10.1016/s0140-6736(04)16945-6 · Cited by 45
United States Private-Sector Physicians and Pharmaceutical Contract Research: A Qualitative Study
Fisher, J. A., & Kalbaugh, C. A. (2012) · PLoS Medicine · doi:10.1371/journal.pmed.1001271 · Cited by 37
Opinel, A., Tröhler, U., Gluud, C. N., Gachelin, G., Smith, G. D., Podolsky, S. H., & Chalmers, I. (2011) · International Journal of Epidemiology · doi:10.1093/ije/dyr162 · Cited by 24
Self-citationCHABBERT, Y. A. (1956) · New England Journal of Medicine · doi:10.1056/nejm195611292552212 · Cited by 11
Drugs and operations. Some important differences
Love, J. W. (1975) · JAMA · doi:10.1001/jama.232.1.37 · Cited by 10
The Emergence of the Randomized Controlled Trial: Origins to 1980
Bothwell, L. E. (2014) · doi:10.7916/d8k072v0 · Cited by 5 · cited 8×
“Data are from Bothwell.3 Es tim at ed P er ce nt ag e of A ll Pu bl is he d R C Ts du ri ng E ac h 5- Ye ar P er io d 40 50 60 70 20 80 30 10 0 90 19 46 –1 95 0 19 51 –1 95 5 19 56 –1 96 0 19 61 –1 96 5 19 66 –1 97 0 19 71 –1 97 5 19 76 –1 98 0 19 81 –1 98 5 19 86 –1 99 0 19 91 –1 99 5 19 96 –2 00 0 20 01 –2 00 5 20 0…”
BackgroundInfluentialSelf-citationUllyot, D. J., Wisneski, J. A., Sullivan, R. W., & Gertz, E. W. (1976) · Journal of Thoracic and Cardiovascular Surgery · doi:10.1016/s0022-5223(19)40019-6 · Cited by 2
Therapeutic Evolution or Revolution? Metaphors and Their Consequences
Jones, D. S. (2016) · Digital Access to Scholarship at Harvard (DASH) (Harvard University) · Cited by 1
Self-citation
Research works printed but not linked: 3
- Podolsky SH, Jones DS, Kaptchuk TJ. From trials to trials: blinding, medicine, and honest adjudication. In: Robertson CT, Kesselheim AS, eds. Blinding as a solution to bias: strengthening biomedical science, forensic science, and law. Amsterdam: Elsevier, 2016:45-58.
- Kuo W-H. Japan and Taiwan in the wake of bio-globalization: drugs, race and standards (Ph.D. dissertation). Cambridge: Massachusetts Institute of Technology, 2005.
- Rothman DJ, 2000, New York Rev Books, 47, 60
Cited by
Evidence for Health Decision Making — Beyond Randomized, Controlled Trials
Frieden, T. R. (2017) · New England Journal of Medicine · doi:10.1056/nejmra1614394 · Cited by 770
Alipanah, N., Jarlsberg, L., Miller, C., Linh, N. N., Falzon, D., Jaramillo, E., & Nahid, P. (2018) · PLOS Medicine · doi:10.1371/journal.pmed.1002595 · Cited by 503
Econometric Methods for Program Evaluation
Abadie, A., & Cattaneo, M. D. (2018) · Annual Review of Economics · doi:10.1146/annurev-economics-080217-053402 · Cited by 431
Adaptive platform trials: definition, design, conduct and reporting considerations
Coalition, T. A. P. T. (2019) · Nature Reviews Drug Discovery · doi:10.1038/s41573-019-0034-3 · Cited by 412
Lewis, C., Roberts, N. P., Gibson, S., & Bisson, J. I. (2020) · European Journal of Psychotraumatology · doi:10.1080/20008198.2019.1709709 · Cited by 382
“The methodology may have excluded some potentially high-quality sources of evidence, such as large observational studies and nonrandomized controlled effectiveness studies (Bothwell et al., 2016), which could contribute to a more accurate overall assessment of dropout.”
Background
Metadata
- Authors
- Bothwell, L. E., Greene, J. A., Podolsky, S. H., & Jones, D. S.
- Affiliations
- Brigham and Women's Hospital (US)
- Venue
- New England Journal of Medicine
- Publisher
- Massachusetts Medical Society
- Published
- 2016-06-02
- Volume
- 374
- Issue
- 22
- Pages
- 2175-2181
- ISSN
- 0028-4793 (ISSN-L)
- 1533-4406
- OpenAlex
- W2413861456
- Cited by
- 664
- References
- 54
- Citations by year
- 2026: 40 · 2025: 101 · 2024: 59 · 2023: 53 · 2022: 49 · 2021: 76 · 2020: 93 · 2019: 71 · 2018: 74 · 2017: 43 · 2016: 5
- FWCI
- 337.69
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- New England Journal of Medicine (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Arts and Humanities › History and Philosophy of Science › History of Science and Medicine
- Health Sciences › Medicine › Pharmacology (medical) › Historical Medical Research and Treatments
- Social Sciences › Arts and Humanities › History and Philosophy of Science › Philosophy of Biology and Scientific Explanation
- Keywords
- randomized controlled trials, history of medicine, evidence-based medicine, scientific knowledge production, medical politics, health research funding
- MeSH terms
- Drug Industry, Female, Humans, Legislation, Drug, Pregnancy, Randomized Controlled Trials as Topic, Drug Approval, History, 20th Century, History, 21st Century
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.