Multiplicity in randomised trials I: endpoints and treatments
Abstract
Multiplicity problems emerge from investigators looking at many additional endpoints and treatment group comparisons. Thousands of potential comparisons can emanate from one trial. Investigators might only report the significant comparisons, an unscientific practice if unwitting, and fraudulent if intentional. Researchers must report all the endpoints analysed and treatments compared. Some statisticians propose statistical adjustments to account for multiplicity. Simply defined, they test for no effects in all the primary endpoints undertaken versus an effect in one or more of those endpoints. In general, statistical adjustments for multiplicity provide crude answers to an irrelevant question. However, investigators should use adjustments when the clinical decision-making argument rests solely on one or more of the primary endpoints being significant. In these cases, adjustments somewhat rescue scattershot analyses. Readers need to be aware of the potential for under-reporting of analyses.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Given that the approach here is of planned comparisons, where all analyses are pre-specified, it is not necessarily appropriate to apply corrections for multiple testing (Feise, 2002; Schulz and Grimes, 2005).
“However, given that the approach here is of planned comparisons, where all analyses are pre-specified, it is not necessarily appropriate to apply these corrections (Feise, 2002; Schulz and Grimes, 2005).”
From Sleep duration and psychotic experiences in patients at risk of psychosis: a secondary analysis of the EDIE-2 trial (2019) · cites this paper as Schulz and Grimes, 2005
Multiple-arm trials provide more information than two-arm trials can provide.
“Such trials provide more information than two-arm trials can provide [6].”
From Reporting of analyses from randomized controlled trials with multiple arms: a systematic review (2013) · cites this paper as [6]
In an RCT with three arms, there are seven theoretically possible comparisons.
“For instance, in an RCT with three arms, there are seven theoretically possible comparisons [6].”
From Reporting of analyses from randomized controlled trials with multiple arms: a systematic review (2013) · cites this paper as [6]
Cites
Practical Statistics for Medical Research.
Walter, S., & Altman, D. G. (1992) · Biometrics · doi:10.2307/2532320 · Cited by 11,665 · see citing works
Peto, R., Pike, M. C., Armitage, P., Breslow, N. E., Cox, D. R., Howard, S. V., Mantel, N., McPherson, K., Peto, J., & Smith, P. G. (1977) · British Journal of Cancer · doi:10.1038/bjc.1977.1 · Cited by 8,285 · see citing works
No adjustments are needed for multiple comparisons.
Rothman, K. J. (1990) · Epidemiology · doi:10.1097/00001648-199001000-00010 · Cited by 5,965 · see citing works
What's wrong with Bonferroni adjustments
Perneger, T. V. (1998) · BMJ · doi:10.1136/bmj.316.7139.1236 · Cited by 5,900 · see citing works
Moher, D., Schulz, K. F., & Altman, D. G. (2001) · The Lancet · doi:10.1016/s0140-6736(00)04337-3 · Cited by 5,373 · see citing works
Self-citationPractical Statistics for Medical Research.
Bartko, J. J., & Altman, D. G. (1992) · Journal of the American Statistical Association · doi:10.2307/2290246 · Cited by 4,834 · see citing works
The Revised CONSORT Statement for Reporting Randomized Trials: Explanation and Elaboration
Altman, D. G., Schulz, K. F., Moher, D., Egger, M., Davidoff, F., Elbourne, D., Gøtzsche, P. C., Lang, T., & for the CONSORT Group (2001) · Annals of Internal Medicine · doi:10.7326/0003-4819-134-8-200104170-00012 · Cited by 3,835 · see citing works
Self-citationClinical Trials: A Practical Approach.
Hill, I. D., & Pocock, S. (1984) · Biometrics · doi:10.2307/2531186 · Cited by 2,918 · see citing works
Peto, R., Pike, M. C., Armitage, P., Breslow, N. E., Cox, D. R., Howard, S. V., Mantel, N., McPherson, K., Peto, J., & Smith, P. G. (1976) · British Journal of Cancer · doi:10.1038/bjc.1976.220 · Cited by 2,294 · see citing works
Fundamentals of Clinical Trials
Friedman, L. M., Furberg, C. D., DeMets, D. L., Reboussin, D. M., & Granger, C. B. (2015) · doi:10.1007/978-3-319-18539-2 · Cited by 2,237 · see citing works
Chan, A.-W., Hróbjartsson, A., Haahr, M. T., Gøtzsche, P. C., & Altman, D. G. (2004) · JAMA · doi:10.1001/jama.291.20.2457 · Cited by 1,845 · see citing works
Sterne, J. A. C. (2001) · BMJ · doi:10.1136/bmj.322.7280.226 · Cited by 1,466 · see citing works
Multiple Comparisons: Theory and Methods
Christensen, R., & Hsu, J. C. (1997) · Technometrics · doi:10.2307/1270920 · Cited by 1,105 · see citing works
Composite Outcomes in Randomized Trials
Freemantle, N., Calvert, M., Wood, J., Eastaugh, J., & Griffin, C. (2003) · JAMA · doi:10.1001/jama.289.19.2554 · Cited by 785 · see citing works
Statistical Power, Sample Size, and Their Reporting in Randomized Controlled Trials
Moher, D. (1994) · JAMA: The Journal of the American Medical Association · doi:10.1001/jama.1994.03520020048013 · Cited by 608 · see citing works
Sample size calculations in randomised trials: mandatory and mystical
Schulz, K. F., & Grimes, D. A. (2005) · The Lancet · doi:10.1016/s0140-6736(05)61034-3 · Cited by 549 · see citing works
Self-citationMeinert, C. L. (1986) · DigitalGeorgetown (Georgetown University Library) · doi:10.1093/acprof:oso/9780195035681.001.0001 · Cited by 509 · see citing works
Multiple Comparisons and Related Issues in the Interpretation of Epidemiologic Data
Savitz, D. A., & Olshan, A. F. (1995) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a117737 · Cited by 463 · see citing works
Multiplicity in randomised trials II: subgroup and interim analyses
Schulz, K. F., & Grimes, D. A. (2005) · The Lancet · doi:10.1016/s0140-6736(05)66516-6 · Cited by 357 · see citing works
Self-citationStatistics in medical journals: some recent trends
Altman, D. G. (2000) · Statistics in Medicine · doi:10.1002/1097-0258(20001215)19:23 · Cited by 252 · see citing works
Clinical Trials: Design, Conduct, and Analysis
Baker, L. H. (1987) · JAMA · doi:10.1001/jama.1987.03390090119041 · Cited by 231 · see citing works
Statistical Issues in Drug Development
Ziegel, E. R., & Senn, S. J. (1999) · Technometrics · doi:10.2307/1271380 · Cited by 224 · see citing works
Pocock, S. J. (1997) · Controlled Clinical Trials · doi:10.1016/s0197-2456(97)00008-1 · Cited by 184 · see citing works
Sankoh, A. J., D’Agostino, R. B., & Huque, M. F. (2003) · Statistics in Medicine · doi:10.1002/sim.1557 · Cited by 110 · see citing works
Quantitative Evaluation of Multiplicity in Epidemiology and Public Health Research
Ottenbacher, K. J. (1998) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a009501 · Cited by 81 · see citing works
Statistical Issues in Drug Development
(1999) · Technometrics · doi:10.1080/00401706.1999.10485963 · Cited by 77 · see citing works
Fundamentals of Clinical Trials
Pocock, S. J., Friedman, L. M., Furberg, C. D., & Demets, D. L. (1982) · Biometrics · doi:10.2307/2530084 · Cited by 66 · see citing works
P-Value Interpretation and Alpha Allocation in Clinical Trials
Moyé, L. A. (1998) · Annals of Epidemiology · doi:10.1016/s1047-2797(98)00003-9 · Cited by 56 · see citing works
Bauer, P., Chi, G., Geller, N. L., Gould, A. L., Jordan, D., Mohanty, S., O’Neill, R. T., & Westfall, P. H. (2003) · Journal of Biopharmaceutical Statistics · doi:10.1081/bip-120024203 · Cited by 9 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Intensive blood glucose control and vascular outcomes in patients with type 2 diabetes
(2008) · New England Journal of Medicine · doi:10.1056/nejmoa0802987 · Cited by 7,397 · see citing works
SPIRIT 2013 explanation and elaboration: guidance for protocols of clinical trials
Chan, A.-W., Tetzlaff, J. M., Gotzsche, P. C., Altman, D. G., Mann, H., Berlin, J. A., Dickersin, K., Hrobjartsson, A., Schulz, K. F., Parulekar, W. R., et al. (2013) · BMJ · doi:10.1136/bmj.e7586 · Cited by 6,251 · see citing works
Self-citationWhen to use the <scp>B</scp> onferroni correction
Armstrong, R. A. (2014) · Ophthalmic and Physiological Optics · doi:10.1111/opo.12131 · Cited by 3,031 · see citing works
Glickman, M. E., Rao, S. R., & Schultz, M. R. (2014) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2014.03.012 · Cited by 1,561 · see citing works
Liberal or Restrictive Transfusion after Cardiac Surgery
Murphy, G. J., Pike, K., Rogers, C. A., Wordsworth, S., Stokes, E. A., Angelini, G. D., & Reeves, B. C. (2015) · New England Journal of Medicine · doi:10.1056/nejmoa1403612 · Cited by 787 · see citing works
Vitamin D supplementation and prevention of type 2 diabetes
Pittas, A. G., Dawson-Hughes, B., Sheehan, P., Ware, J. H., Knowler, W. C., Aroda, V. R., Brodsky, I., Ceglia, L., Chadha, C., Chatterjee, R., et al. (2019) · New England Journal of Medicine · doi:10.1056/nejmoa1900906 · Cited by 673 · see citing works
Follow-up of blood-pressure lowering and glucose control in type 2 diabetes
Zoungas, S., Chalmers, J., Neal, B., Billot, L., Li, Q., Hirakawa, Y., Arima, H., Monaghan, H., Joshi, R., Colagiuri, S., et al. (2014) · New England Journal of Medicine · doi:10.1056/nejmoa1407963 · Cited by 626 · see citing works
Correction for Multiple Testing
Streiner, D. L., & Norman, G. R. (2011) · CHEST Journal · doi:10.1378/chest.11-0523 · Cited by 573 · see citing works
Combination Antifungal Therapy for Cryptococcal Meningitis
Day, J., Chau, T. T., Wolbers, M., Mai, P. P., Dung, N. T., Mai, N. H., Phu, N. H., Nghia, H. D. T., Phong, N. D., Cao, Q. T., et al. (2013) · New England Journal of Medicine · doi:10.1056/nejmoa1110404 · Cited by 542 · see citing works
Effect of Online Hemodiafiltration on All-Cause Mortality and Cardiovascular Outcomes
Grooteman, M. P. C., van den Dorpel, M. A., Bots, M. L., Penne, E. L., van der Weerd, N. C., Mazairac, A. H. A., den Hoedt, C. H., van der Tweel, I., Levesque, R. L., Nubé, M. J., et al. (2012) · Journal of the American Society of Nephrology · doi:10.1681/asn.2011121140 · Cited by 527 · see citing works
Patil, S. R., Arnold, B. F., Salvatore, A. L., Briceno, B., Ganguly, S., Colford, J. M., & Gertler, P. J. (2014) · PLoS Medicine · doi:10.1371/journal.pmed.1001709 · Cited by 487 · see citing works
Beard, D. J., Rees, J. L., Cook, J. A., Rombach, I., Cooper, C., Merritt, N., Shirkey, B. A., Donovan, J., Gwilym, S., Savulescu, J., et al. (2017) · The Lancet · doi:10.1016/s0140-6736(17)32457-1 · Cited by 465 · see citing works
Bertelsen, M., Jeppesen, P., Petersen, L., Thorup, A. A. E., Øhlenschlæger, J., le Quach, P., Christensen, T. Ø., Krarup, G., Jørgensen, P., & Nordentoft, M. (2008) · Archives of General Psychiatry · doi:10.1001/archpsyc.65.7.762 · Cited by 457 · see citing works
Benger, J. R., Kirby, K., Black, S., Brett, S. J., Clout, M., Lazaroo, M. J., Nolan, J. P., Reeves, B. C., Robinson, M., Scott, L. J., et al. (2018) · JAMA · doi:10.1001/jama.2018.11597 · Cited by 450 · see citing works
High-Dose Rifapentine with Moxifloxacin for Pulmonary Tuberculosis
Jindani, A., Harrison, T. S., Nunn, A., Phillips, P. P. J., Churchyard, G., Charalambous, S., Hatherill, M., Geldenhuys, H. D., McIlleron, H., Zvada, S., et al. (2014) · New England Journal of Medicine · doi:10.1056/nejmoa1314210 · Cited by 430 · see citing works
Mitri, J., Dawson-Hughes, B., Hu, F., & Pittas, A. (2011) · American Journal of Clinical Nutrition · doi:10.3945/ajcn.111.011684 · Cited by 431 · see citing works
An introduction to multiplicity issues in clinical trials: the what, why, when and how
Li, G., Taljaard, M., Van den Heuvel, E. R., Levine, M. A., Cook, D. J., Wells, G. A., Devereaux, P. J., & Thabane, L. (2016) · International Journal of Epidemiology · doi:10.1093/ije/dyw320 · Cited by 381 · see citing works
Pickering, A. J., Djebbari, H., López, C., Coulibaly, M., & Alzúa, M. L. (2015) · The Lancet Global Health · doi:10.1016/s2214-109x(15)00144-8 · Cited by 393 · see citing works
Rubin, M. (2021) · Synthese · doi:10.1007/s11229-021-03276-4 · Cited by 371 · see citing works
Multiplicity in randomised trials II: subgroup and interim analyses
Schulz, K. F., & Grimes, D. A. (2005) · The Lancet · doi:10.1016/s0140-6736(05)66516-6 · Cited by 357 · see citing works
Self-citationWittert, G., Bracken, K., Robledo, K. P., Grossmann, M., Yeap, B. B., Handelsman, D. J., Stuckey, B., Conway, A., Inder, W., McLachlan, R., et al. (2021) · The Lancet Diabetes & Endocrinology · doi:10.1016/s2213-8587(20)30367-3 · Cited by 362 · see citing works
Villar, J., Abdel‐Aleem, H., Merialdi, M., Mathai, M., Ali, M. M., Zavaleta, N., Purwar, M., Hofmeyr, G. J., Ngoc, N. T. N., Campodónico, L., et al. (2006) · American Journal of Obstetrics and Gynecology · doi:10.1016/j.ajog.2006.01.068 · Cited by 348 · see citing works
Streiner, D. L. (2015) · American Journal of Clinical Nutrition · doi:10.3945/ajcn.115.113548 · Cited by 344 · see citing works
Morawski, K., Ghazinouri, R., Krumme, A., Lauffenburger, J. C., Lu, Z., Durfee, E., Oley, L., Lee, J., Mohta, N., Haff, N., et al. (2018) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2018.0447 · Cited by 333 · see citing works
Belenchia, A. M., Tosh, A. K., Hillman, L. S., & Peterson, C. A. (2013) · American Journal of Clinical Nutrition · doi:10.3945/ajcn.112.050013 · Cited by 328 · see citing works
Reporting of Multi-Arm Parallel-Group Randomized Trials: Extension of the CONSORT 2010 Statement
Juszczak, E., Altman, D. G., Hopewell, S., & Schulz, K. (2019) · JAMA · doi:10.1001/jama.2019.3087 · Cited by 324 · see citing works
Self-citationStudy design of ASPirin in Reducing Events in the Elderly (ASPREE): A randomized, controlled trial
(2013) · Contemporary Clinical Trials · doi:10.1016/j.cct.2013.09.014 · Cited by 310 · see citing works
Gray, K. M., Carpenter, M. J., Baker, N. L., DeSantis, S. M., Kryway, E., Hartwell, K. J., McRae-Clark, A. L., & Brady, K. T. (2012) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2012.12010055 · Cited by 306 · see citing works
Bobrow, K., Farmer, A. J., Springer, D., Shanyinde, M., Yu, L.-M., Brennan, T., Rayner, B., Namane, M., Steyn, K., Tarassenko, L., et al. (2016) · Circulation · doi:10.1161/circulationaha.115.017530 · Cited by 295 · see citing works
Turner, J. A., Mancl, L., & Aaron, L. A. (2006) · Pain · doi:10.1016/j.pain.2005.11.017 · Cited by 270 · see citing works
Ferrando, C., Soro, M., Unzueta, C., Suárez-Sipmann, F., Canet, J., Librero, J., Pozo, N., Peiró, S., Llombart, A., León, I., et al. (2018) · The Lancet Respiratory Medicine · doi:10.1016/s2213-2600(18)30024-9 · Cited by 258 · see citing works
Arnold, B. F., Null, C., Luby, S. P., UNICOMB, L. E., Stewart, C. P., Dewey, K. G., Ahmed, T., Ashraf, S., Christensen, G., Clasen, T., et al. (2013) · BMJ Open · doi:10.1136/bmjopen-2013-003476 · Cited by 250 · see citing works
Evidence at a glance: error matrix approach for overviewing available evidence
Keus, F., Wetterslev, J., Gluud, C. N., & van Laarhoven, C. J. H. M. (2010) · BMC Medical Research Methodology · doi:10.1186/1471-2288-10-90 · Cited by 218 · see citing works
Virtual Reality for Pediatric Needle Procedural Pain: Two Randomized Clinical Trials
Chan, E., Hovenden, M., Ramage, E., Ling, N., Pham, J. H., Rahim, A., Lam, C., Liu, L., Foster, S., Sambell, R., et al. (2019) · The Journal of Pediatrics · doi:10.1016/j.jpeds.2019.02.034 · Cited by 223 · see citing works
Rangan, A., Brealey, S., Keding, A., Corbacho, B., Northgraves, M. J., Kottam, L., Goodchild, L. M., Srikesavan, C. S., Rex, S., Charalambous, C. P., et al. (2020) · The Lancet · doi:10.1016/s0140-6736(20)31965-6 · Cited by 210 · see citing works
Rubin, M. (2017) · Review of General Psychology · doi:10.1037/gpr0000123 · Cited by 205 · see citing works
García-León, M. Á., Pérez-Mármol, J. M., Gonzalez-Pérez, R., García-Ríos, M. D. C., & Peralta-Ramírez, M. I. (2019) · Physiology & Behavior · doi:10.1016/j.physbeh.2019.02.001 · Cited by 197 · see citing works
Bhattacharya, S., Harrild, K., Mollison, J., Wordsworth, S., Tay, C. C. K., Harrold, A., McQueen, D., Lyall, H., Johnston, L., Burrage, J., et al. (2008) · BMJ · doi:10.1136/bmj.a716 · Cited by 197 · see citing works
Teraa, M., Sprengers, R. W., Schutgens, R. E. G., Slaper‐Cortenbach, I., van der Graaf, Y., Algra, A., van der Tweel, I., Doevendans, P. A., Mali, W. P. T. M., Moll, F. L., et al. (2015) · Circulation · doi:10.1161/circulationaha.114.012913 · Cited by 187 · see citing works
Martens, C. R., Rossman, M. J., Mazzo, M. R., Jankowski, L. R., Nagy, E. E., Denman, B. A., Richey, J. J., Johnson, S. A., Ziemba, B. P., Wang, Y., et al. (2020) · GeroScience · doi:10.1007/s11357-020-00156-6 · Cited by 182 · see citing works
Patel, A. A., Cass, A., Peiris, D., Usherwood, T. P., Brown, A. D. H., Jan, S., Neal, B. C., Hillis, G. S., Rafter, N., Tonkin, A., et al. (2014) · European Journal of Preventive Cardiology · doi:10.1177/2047487314530382 · Cited by 183 · see citing works
Wainwright, C. E., Vidmar, S., Armstrong, D., Byrnes, C. A., Carlin, J. B., Cheney, J. L., Cooper, P., Grimwood, K., Moodie, M., ROBERTSON, C. F., et al. (2011) · JAMA · doi:10.1001/jama.2011.954 · Cited by 180 · see citing works
Moon, R. J., Harvey, N. C., Robinson, S. M., Ntani, G., Davies, J. H., Inskip, H. M., Godfrey, K. M., Dennison, E. M., Calder, P. C., Cooper, C. C., et al. (2012) · The Journal of Clinical Endocrinology & Metabolism · doi:10.1210/jc.2012-2482 · Cited by 170 · see citing works
Walker, J. G., Batterham, P. J., Mackinnon, A. J., Jorm, A. F., Hickie, I., Fenech, M., Kljakovic, M., Crisp, D., & Christensen, H. (2012) · The American Journal of Clinical Nutrition · doi:10.3945/ajcn.110.007799 · Cited by 165 · see citing works
Subtle cognitive impairments in patients with long-term cure of Cushing’s disease
Tiemensma, J., Kokshoorn, N. E., Biermasz, N. R., Keijser, B.-J. S. A., Wassenaar, M. J. E., Middelkoop, H. A. M., Pereira, A. M., & Romijn, J. A. (2010) · The Journal of Clinical Endocrinology & Metabolism · doi:10.1210/jc.2009-2032 · Cited by 159 · see citing works
Coup de Grâce for a Tough Old Bull: “Statistically Significant” Expires
Hurlbert, S. H., Levine, R. A., & Utts, J. M. (2019) · The American Statistician · doi:10.1080/00031305.2018.1543616 · Cited by 159 · see citing works
The Definition and Measurement of Disease Modification in Inflammatory Rheumatic Diseases
Aletaha, D., & Smolen, J. S. (2006) · Rheumatic Disease Clinics of North America · doi:10.1016/j.rdc.2005.09.005 · Cited by 152 · see citing works
Stein, A., Netsi, E., Lawrence, P. J., Granger, C., Kempton, C., Craske, M. G., Nickless, A., Mollison, J., Stewart, D. A., Rapa, E., et al. (2018) · The Lancet Psychiatry · doi:10.1016/s2215-0366(18)30006-3 · Cited by 148 · see citing works
Peiris, D., Usherwood, T. P., Panaretto, K. S., Harris, M., Hunt, J., Redfern, J., Zwar, N. A., Colagiuri, S., Hayman, N. E., Lo, S., et al. (2015) · Circulation Cardiovascular Quality and Outcomes · doi:10.1161/circoutcomes.114.001235 · Cited by 139 · see citing works
Haidar, A., Tsoukas, M. A., Bernier-Twardy, S., Yale, J., Rutkowski, J., Bossy, A., Pytka, E., Fathi, A. E., Strauss, N., & Legault, L. (2020) · Diabetes Care · doi:10.2337/dc19-1922 · Cited by 138 · see citing works
Olsson, C. A., Foley, D. L., Parkinson-Bates, M., Byrnes, G., McKenzie, M., Patton, G. C., Morley, R., Anney, R. J. L., Craig, J. M., & Saffery, R. (2010) · Biological Psychology · doi:10.1016/j.biopsycho.2009.12.003 · Cited by 137 · see citing works
Kars, M., Delgado, V., Holman, E. R., Feelders, R. A., Smit, J. W. A., Romijn, J. A., Bax, J. J., & Pereira, A. M. (2008) · The Journal of Clinical Endocrinology & Metabolism · doi:10.1210/jc.2007-2658 · Cited by 135 · see citing works
Mayo-Wilson, E., Fusco, N., Li, T., Hong, H., Canner, J. K., & Dickersin, K. (2017) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.05.007 · Cited by 131 · see citing works
Key Issues in End Point Selection for Heart Failure Trials: Composite End Points
Neaton, J. D., Gray, G. W., Zuckerman, B. D., & Konstam, M. A. (2005) · Journal of Cardiac Failure · doi:10.1016/j.cardfail.2005.08.350 · Cited by 127 · see citing works
Management of Medication Overuse Headache: 1-Year Randomized Multicentre Open-Label Trial
Hagen, K., Albretsen, C., Vilming, S. T., Salvesen, R., Grøning, M., Helde, G., Gravdahl, G. B., Zwart, J.-A., & Stovner, L. (2008) · Cephalalgia · doi:10.1111/j.1468-2982.2008.01711.x · Cited by 126 · see citing works
Seidler, A. L., Aberoumand, M., Hunter, K. E., Barba, A., Libesman, S., Williams, J. G., Shrestha, N., Aagerup, J., Sotiropoulos, J. X., Montgomery, A. A., et al. (2023) · The Lancet · doi:10.1016/s0140-6736(23)02468-6 · Cited by 125 · see citing works
Mellor, R., Bennell, K. L., Grimaldi, A. M., Nicolson, P. J. A., Kasza, J., Hodges, P. W., Wajswelner, H., & Vicenzino, B. (2018) · BMJ · doi:10.1136/bmj.k1662 · Cited by 124 · see citing works
Efficacy of psychosocial interventions in cancer care: Evidence is weaker than it first looks
Coyne, J. C., Lepore, S. J., & Palmer, S. C. (2006) · Annals of Behavioral Medicine · doi:10.1207/s15324796abm3202_5 · Cited by 122 · see citing works
Visual evoked potential latency, amplitude and habituation in migraine: A longitudinal study
Sand, T., Zhitniy, N., White, L. R., & Stovner, L. J. (2008) · Clinical Neurophysiology · doi:10.1016/j.clinph.2008.01.009 · Cited by 119 · see citing works
Interictal quantitative EEG in migraine: a blinded controlled study
Bjørk, M., Stovner, L. J., Engstrøm, M., Stjern, M., Hagen, K., & Sand, T. (2009) · The Journal of Headache and Pain · doi:10.1007/s10194-009-0140-4 · Cited by 106 · see citing works
Cheatham, C. L., Goldman, B. D., Fischer, L. M., da Costa, K.-A., Reznick, J. S., & Zeisel, S. H. (2012) · American Journal of Clinical Nutrition · doi:10.3945/ajcn.112.037184 · Cited by 105 · see citing works
for the Regensburg QoL Study Group, Klinkhammer‐Schalke, M., Koller, M. T., Steinger, B., Ehret, C., Ernst, B., Wyatt, J. C., Hofstädter, F., & Lorenz, W. (2012) · British Journal of Cancer · doi:10.1038/bjc.2012.4 · Cited by 103 · see citing works
Eyles, H. C., & Mhurchú, C. N. (2009) · Nutrition Reviews · doi:10.1111/j.1753-4887.2009.00219.x · Cited by 102 · see citing works
Murray, E., Sweeting, M. J., Dack, C., Pal, K., Modrow, K., Hudda, M. T., Li, J., Ross, J., Alkhaldi, G., Barnard, M., et al. (2017) · BMJ Open · doi:10.1136/bmjopen-2017-016009 · Cited by 98 · see citing works
A randomized placebo-controlled trial of an omega-3 fatty acid and vitamins E+C in schizophrenia
Bentsen, H., Osnes, K., Refsum, H., Solberg, D. K., & Bøhmer, T. (2013) · Translational Psychiatry · doi:10.1038/tp.2013.110 · Cited by 97 · see citing works
van Zuilen, A. D., Bots, M. L., Dülger, A., van der Tweel, I., van Buren, M., ten Dam, M. A. G. J., Kaasjager, K. A. H., Ligtenberg, G., Sijpkens, Y. W. J., Sluiter, H. E., et al. (2012) · Kidney International · doi:10.1038/ki.2012.137 · Cited by 96 · see citing works
Pickering, A. J., Crider, Y., Sultana, S., Swarthout, J., Goddard, F. G., Anjerul Islam, S., Sen, S., Ayyagari, R., & Luby, S. P. (2019) · The Lancet Global Health · doi:10.1016/s2214-109x(19)30315-8 · Cited by 95 · see citing works
Pickering, A. J., Njenga, S. M., Steinbaum, L., Swarthout, J., Lin, A., Arnold, B. F., Stewart, C. P., Dentz, H. N., Mureithi, M., Chieng, B., et al. (2019) · PLOS Medicine · doi:10.1371/journal.pmed.1002841 · Cited by 95 · see citing works
Habauzit, V., Verny, M., Milenković, D., Barber‐Chamoux, N., Mazur, A., Dubray, C., & Morand, C. (2015) · American Journal of Clinical Nutrition · doi:10.3945/ajcn.114.104646 · Cited by 95 · see citing works
Beck, E., Bo, S., Jørgensen, M. S., Gondan, M., Poulsen, S., Storebø, O. J., Fjellerad Andersen, C., Folmo, E., Sharp, C., Pedersen, J., et al. (2020) · Journal of Child Psychology and Psychiatry · doi:10.1111/jcpp.13152 · Cited by 93 · see citing works
Resolve conflicting rankings of outcomes in network meta‐analysis: Partial ordering of treatments
Rücker, G., & Schwarzer, G. (2017) · Research Synthesis Methods · doi:10.1002/jrsm.1270 · Cited by 94 · see citing works
Engstrøm, M., Hagen, K., Bjørk, M., Stovner, L. J., Gravdahl, G. B., Stjern, M., & Sand, T. (2013) · The Journal of Headache and Pain · doi:10.1186/1129-2377-14-12 · Cited by 91 · see citing works
Ace and Aro: Understanding Differences in Romantic Attractions Among Persons Identifying as Asexual
Antonsen, A. N., Zdaniuk, B., Yule, M. A., & Brotto, L. A. (2020) · Archives of Sexual Behavior · doi:10.1007/s10508-019-01600-1 · Cited by 90 · see citing works
Ensuring transparency and minimization of methodologic bias in preclinical pain research
Andrews, N. A., Latrémolière, A., Basbaum, A. I., Mogil, J. S., Porreca, F., Rice, A. S. C., Woolf, C. J., Currie, G. L., Dworkin, R. H., Eisenach, J. C., et al. (2015) · Pain · doi:10.1097/j.pain.0000000000000458 · Cited by 92 · see citing works
Mellor, R., Bennell, K. L., Grimaldi, A. M., Nicolson, P. J. A., Kasza, J., Hodges, P. W., Wajswelner, H., & Vicenzino, B. (2018) · British Journal of Sports Medicine · doi:10.1136/bjsports-2018-k1662rep · Cited by 89 · see citing works
da Mota, L. M. H., Cruz, B. A., Brenol, C. V., Pereira, I. A., Fronza, L. S. R., Bértolo, M. B., Freitas, M. V. C., da Silva, N. A., Louzada‐Júnior, P., Giorgi, R. D. N., et al. (2011) · Revista Brasileira de Reumatologia · doi:10.1590/s0482-50042011000300002 · Cited by 88 · see citing works
Sherman, B. J., McRae‐Clark, A. L., Baker, N. L., Sonne, S. C., Killeen, T. K., Cloud, K., & Gray, K. M. (2017) · American Journal on Addictions · doi:10.1111/ajad.12503 · Cited by 87 · see citing works
Risk factors for childhood enteric infection in urban Maputo, Mozambique: A cross-sectional study
Knee, J., Sumner, T., Adriano, Z., Berendes, D. M., de Bruijn, E., Schmidt, W., Nalá, R., Cumming, O., & Brown, J. M. (2018) · PLoS neglected tropical diseases · doi:10.1371/journal.pntd.0006956 · Cited by 87 · see citing works
Polychlorinated biphenyls and breast cancer: A congener-specific meta-analysis
Leng, L., Li, J., Luo, X.-M., Kim, J.-Y., Li, Y.-M., Guo, X., Chen, X., Yang, Q., Li, G., & Tang, N. (2015) · Environment International · doi:10.1016/j.envint.2015.12.022 · Cited by 84 · see citing works
Cluver, L. D., Meinck, F., Yakubovich, A. R., Doubt, J., Redfern, A., Ward, C. L., Salah, N., Stone, S. D., Petersen, T., Mpimpilashe, P., et al. (2016) · BMC Public Health · doi:10.1186/s12889-016-3262-z · Cited by 84 · see citing works
Parker, R. A., & Weir, C. J. (2020) · Clinical Trials · doi:10.1177/1740774520941419 · Cited by 82 · see citing works
Salisbury, C., Montgomery, A. A., Hollinghurst, S. P., Hopper, C. D., Bishop, A., Franchini, A., Kaur, S., Coast, J., Hall, J., Grove, S., et al. (2013) · BMJ · doi:10.1136/bmj.f43 · Cited by 82 · see citing works
Kyle, S. D., Hurry, M. E. D., Emsley, R., Marsden, A., Omlin, X., Juss, A., Spiegelhalder, K., Bisdounis, L., Luik, A. I., Espie, C. A., et al. (2020) · SLEEP · doi:10.1093/sleep/zsaa034 · Cited by 85 · see citing works
Thermal pain thresholds are decreased in the migraine preattack phase
Sand, T., Zhitniy, N., Nilsen, K. B., Helde, G., Hagen, K., & Stovner, L. J. (2008) · European Journal of Neurology · doi:10.1111/j.1468-1331.2008.02276.x · Cited by 79 · see citing works
Haag, G., Diener, H.-C., May, A., Meyer, C., Morck, H., Straube, A., Wessely, P., & Evers, S. (2011) · The Journal of Headache and Pain · doi:10.1007/s10194-010-0266-4 · Cited by 78 · see citing works
Pickard, R. S., Starr, K., MacLennan, G., Kilonzo, M. M., Lam, T. B. L., Thomas, R. E., Burr, J. M., Norrie, J. D., McPherson, G. C., McDonald, A. M., et al. (2015) · Health Technology Assessment · doi:10.3310/hta19630 · Cited by 78 · see citing works
Page, M. J., McKenzie, J. E., & Forbes, A. B. (2013) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2012.10.010 · Cited by 75 · see citing works
How to deal with multiple endpoints in clinical trials
Neuhäuser, M. (2006) · Fundamental and Clinical Pharmacology · doi:10.1111/j.1472-8206.2006.00437.x · Cited by 77 · see citing works
Lonsdale, C., Hall, A. M., Murray, A., Williams, G. C., McDonough, S., Ntoumanis, N., Owen, K. B., Schwarzer, R., Parker, P. D., Kolt, G. S., et al. (2017) · Archives of Physical Medicine and Rehabilitation · doi:10.1016/j.apmr.2017.02.025 · Cited by 77 · see citing works
Segobin, S. H., Lanièpce, A., Ritz, L., Lannuzel, C., Boudehent, C., Cabé, N., Urso, L., Vabret, F., Eustache, F., Beaunieux, H., et al. (2019) · Brain · doi:10.1093/brain/awz056 · Cited by 82 · see citing works
Lower Low-Density Lipoprotein Cholesterol Levels Are Associated with Severe Dengue Outcome
Biswas, H. H., Gordon, A. L., Nuñez, A., Pérez, M. Á., Balmaseda, Á., & Harris, E. (2015) · PLoS neglected tropical diseases · doi:10.1371/journal.pntd.0003904 · Cited by 75 · see citing works
Crowther, C. A., Ashwood, P., McPhee, A. J., Flenady, V., Tran, T. S., Dodd, J. M., Robinson, J. S., & for the PROGRESS Study Group (2017) · PLoS Medicine · doi:10.1371/journal.pmed.1002390 · Cited by 74 · see citing works
Role of proteinuria in defining pre‐eclampsia: Clinical outcomes for women and babies
Thornton, C. E., Makris, A., Ogle, R. F., Tooher, J., & Hennessy, A. (2009) · Clinical and Experimental Pharmacology and Physiology · doi:10.1111/j.1440-1681.2009.05334.x · Cited by 73 · see citing works
Nguyen, T. N., Harris, K., Woodward, M., Chalmers, J., Cooper, M., Hamet, P., Harrap, S., Heller, S., MacMahon, S., Mancia, G., et al. (2021) · Diabetes Care · doi:10.2337/dc20-2664 · Cited by 72 · see citing works
Sleep quality and arousal in migraine and tension-type headache: the headache-sleep study
Engstrøm, M., Hagen, K., Bjørk, M., Stovner, L. J., & Sand, T. (2014) · Acta Neurologica Scandinavica · doi:10.1111/ane.12237 · Cited by 71 · see citing works
Pennington, L., Thomson, K., James, P., Martin, L., & McNally, R. (2009) · Journal of Speech Language and Hearing Research · doi:10.1044/1092-4388(2009/07-0187 · Cited by 71 · see citing works
Moon, S. J., Kim, K., Lee, W. J., Lee, M. Y., Vigersky, R., & Park, C. (2023) · Diabetes, Obesity and Metabolism · doi:10.1111/dom.14852 · Cited by 69 · see citing works
Sleep-related and non-sleep-related migraine: interictal sleep quality, arousals and pain thresholds
Engstrøm, M., Hagen, K., Bjørk, M., Gravdahl, G. B., & Sand, T. (2013) · The Journal of Headache and Pain · doi:10.1186/1129-2377-14-68 · Cited by 69 · see citing works
Ercümen, A., Benjamin‐Chung, J., Arnold, B. F., Lin, A., Hubbard, A., Stewart, C. P., Rahman, Z., Parvez, S. M., UNICOMB, L. E., Rahman, M. M., et al. (2019) · PLoS neglected tropical diseases · doi:10.1371/journal.pntd.0007323 · Cited by 70 · see citing works
- Authors
- Schulz, K. F., & Grimes, D. A.
- Affiliations
- Family Health International 360 (US)
- Corresponding author
- Schulz, Kenneth F
- Venue
- The Lancet
- Publisher
- Elsevier BV
- Published
- 2005-04
- Volume
- 365
- Issue
- 9470
- Pages
- 1591-1595
- ISSN
- 0099-5355 (ISSN-L)
- 0140-6736
- OpenAlex
- W2034698532
- Cited by
- 366
- References
- 29
- Citations by year
- 2026: 8 · 2025: 21 · 2024: 15 · 2023: 14 · 2022: 19 · 2021: 20 · 2020: 28 · 2019: 23 · 2018: 27 · 2017: 21 · 2016: 17 · 2015: 30 · 2014: 16 · 2013: 17 · 2012: 19
- FWCI
- 13.12
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- The Lancet (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Keywords
- multiplicity, multiple endpoints, network meta-analysis, statistical adjustment, primary endpoint, selective reporting, randomised trials, clinical decision-making
- MeSH terms
- Data Interpretation, Statistical, Research Design, Randomized Controlled Trials as Topic, Endpoint Determination
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
