Do multiple outcome measures require p-value adjustment?
Open accessAbstract
BACKGROUND: Readers may question the interpretation of findings in clinical trials when multiple outcome measures are used without adjustment of the p-value. This question arises because of the increased risk of Type I errors (findings of false "significance") when multiple simultaneous hypotheses are tested at set p-values. The primary aim of this study was to estimate the need to make appropriate p-value adjustments in clinical trials to compensate for a possible increased risk in committing Type I errors when multiple outcome measures are used. DISCUSSION: The classicists believe that the chance of finding at least one test statistically significant due to chance and incorrectly declaring a difference increases as the number of comparisons increases. The rationalists have the following objections to that theory: 1) P-value adjustments are calculated based on how many tests are to be considered, and that number has been defined arbitrarily and variably; 2) P-value adjustments reduce the chance of making type I errors, but they increase the chance of making type II errors or needing to increase the sample size. SUMMARY: Readers should balance a study's statistical significance with the magnitude of effect, the quality of the study and with findings from other studies. Researchers facing multiple outcome measures might want to either select a primary outcome measure or use a global assessment measure, rather than adjusting the p-value.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The significance criterion was set at p ≤ 0.01 to decrease the chances of making a Type 1 error.
“Due to this statistical approach and to decrease the chances of making a Type 1 error, the criterion for significance was set at p ≤ 0.01 [32].”
From The Effects of an Eight over Cricket Bowling Spell upon Pace Bowling Biomechanics and Performance within Different Delivery Lengths (2019) · cites this paper as [32]
Given that the approach here is of planned comparisons, where all analyses are pre-specified, it is not necessarily appropriate to apply corrections for multiple testing (Feise, 2002; Schulz and Grimes, 2005).
“However, given that the approach here is of planned comparisons, where all analyses are pre-specified, it is not necessarily appropriate to apply these corrections (Feise, 2002; Schulz and Grimes, 2005).”
From Sleep duration and psychotic experiences in patients at risk of psychosis: a secondary analysis of the EDIE-2 trial (2019) · cites this paper as Feise, 2002
The overall α level was set at 0.05, using Bonferroni's criterion for the primary outcome but not for secondary outcomes.
“The overall α level was set at 0.05, using Bonferroni's criterion for the primary outcome but not for the secondary outcomes, to balance between type I and type II error (Feise, 2002).”
From Efficacy of 8- and 4-Session Mindfulness-Based Interventions in a Non-clinical Population: A Controlled Study (2017) · cites this paper as Feise, 2002
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
A Simple Sequentially Rejective Multiple Test Procedure
(1979) · Scandinavian Journal of Statistics · doi:10.2307/4615733 · Cited by 21,526 · see citing works
No adjustments are needed for multiple comparisons.
Rothman, K. J. (1990) · Epidemiology · doi:10.1097/00001648-199001000-00010 · Cited by 5,965 · see citing works
What's wrong with Bonferroni adjustments
Perneger, T. V. (1998) · BMJ · doi:10.1136/bmj.316.7139.1236 · Cited by 5,900 · see citing works
Multiple significance tests: the Bonferroni method
Bland, M., & Altman, D. G. (1995) · BMJ · doi:10.1136/bmj.310.6973.170 · Cited by 3,898 · see citing works
Rectangular Confidence Regions for the Means of Multivariate Normal Distributions
Šidák, Z. (1967) · Journal of the American Statistical Association · doi:10.1080/01621459.1967.10482935 · Cited by 2,904 · see citing works
An improved Bonferroni procedure for multiple tests of significance
SIMES, R. J. (1986) · Biometrika · doi:10.1093/biomet/73.3.751 · Cited by 2,396 · see citing works
Multiple Comparison Procedures
Hahs-Vaughn, D., & Lomax, R. (2013) · Routledge eBooks · doi:10.4324/9780203137819-17 · Cited by 2,132 · see citing works
A Stagewise Rejective Multiple Test Procedure Based on a Modified Bonferroni Test
Hommel, G. (1988) · Biometrika · doi:10.1093/biomet/75.2.383 · Cited by 1,367 · see citing works
Adjusting for multiple testing when reporting research results: the Bonferroni vs Holm methods.
Aickin, M., & Gensler, H. (1996) · American Journal of Public Health · doi:10.2105/ajph.86.5.726 · Cited by 1,346 · see citing works
Procedures for Comparing Samples with Multiple Endpoints
O’Brien, P. C. (1984) · Biometrics · doi:10.2307/2531158 · Cited by 1,163 · see citing works
ON THE USE AND INTERPRETATION OF CERTAIN TEST CRITERIA FOR PURPOSES OF STATISTICAL INFERENCE
Neyman, J., & Pearson, E. S. (1928) · Biometrika · doi:10.1093/biomet/20a.3-4.263 · Cited by 1,081 · see citing works
Williams, D. A. (1971) · Biometrics · doi:10.2307/2528930 · Cited by 780 · see citing works
Modified Sequentially Rejective Multiple Test Procedures
Shaffer, J. P. (1986) · Journal of the American Statistical Association · doi:10.1080/01621459.1986.10478341 · Cited by 774 · see citing works
Comparing the Means of Several Groups
Godfrey, K. (1985) · New England Journal of Medicine · doi:10.1056/nejm198512053132305 · Cited by 740 · see citing works
MULTIPLE COMPARISON PROCEDURES UPDATED
Ludbrook, J. (1998) · Clinical and Experimental Pharmacology and Physiology · doi:10.1111/j.1440-1681.1998.tb02179.x · Cited by 613 · see citing works
Testing the Statistical Certainty of a Response to Increasing Doses of a Drug
Tukey, J. W., Ciminera, J. L., & Heyse, J. F. (1985) · Biometrics · doi:10.2307/2530666 · Cited by 488 · see citing works
Some Thoughts on Clinical Trials, Especially Problems of Multiplicity
Tukey, J. W. (1977) · Science · doi:10.1126/science.333584 · Cited by 480 · see citing works
Multiple Comparisons and Related Issues in the Interpretation of Epidemiologic Data
Savitz, D. A., & Olshan, A. F. (1995) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a117737 · Cited by 463 · see citing works
Moore, R. H., Gore, S. M., & Altman, D. G. (1983) · Journal of the American Statistical Association · doi:10.2307/2288688 · Cited by 303 · see citing works
Stoline, M. R. (1981) · The American Statistician · doi:10.1080/00031305.1981.10479331 · Cited by 280 · see citing works
THE PROBLEM OF MULTIPLE INFERENCE IN STUDIES DESIGNED TO GENERATE HYPOTHESES
Thomas, D. C., Siemiatycki, J., Dewar, R. A. D., Robins, J. M., Goldberg, M. S., & Armstrong, B. (1985) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a114189 · Cited by 190 · see citing works
p Value Adjustments for Multiple Tests in Multivariate Binomial Models
Westfall, P. H., & Young, S. S. (1989) · Journal of the American Statistical Association · doi:10.1080/01621459.1989.10478837 · Cited by 187 · see citing works
The evolving case-control study
Cole, P. T. (1979) · Journal of Chronic Diseases · doi:10.1016/0021-9681(79)90006-7 · Cited by 173 · see citing works
Some statistical methods for multiple endpoints in clinical trials
Zhang, J., Quan, H., Ng, J., & Stepanavage, M. E. (1997) · Controlled Clinical Trials · doi:10.1016/s0197-2456(96)00129-8 · Cited by 170 · see citing works
A Modified Bonferroni Method for Discrete Data
Tarone, R. E. (1990) · Biometrics · doi:10.2307/2531456 · Cited by 153 · see citing works
Multiple Comparisons, Explained
Goodman, S. N. (1998) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a009531 · Cited by 100 · see citing works
Greenhalgh, T. (1997) · BMJ · doi:10.1136/bmj.315.7104.364 · Cited by 99 · see citing works
A Biometrics Invited Paper. Assessing Laboratory Evidence for Neoplastic Activity
Mantel, N. (1980) · Biometrics · doi:10.2307/2530209 · Cited by 97 · see citing works
Interpretation and power of a pooled index.
Goldsmith, C. H., SMYTHE, H. A., & Helewa, A. (1993) · PubMed · Cited by 96 · see citing works
Reporting the results of epidemiologic studies.
Walker, A. M. (1986) · American Journal of Public Health · doi:10.2105/ajph.76.5.556 · Cited by 93 · see citing works
Thompson, J. R. (1998) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a009530 · Cited by 89 · see citing works
Biostatistics for Epidemiologists
Ziegel, E. R., & Ahlbom, A. (1995) · Technometrics · doi:10.2307/1269943 · Cited by 84 · see citing works
Voss, S., & George, S. (1995) · BMJ · doi:10.1136/bmj.310.6986.1073 · Cited by 63 · see citing works
Empirical Bayes adjustments for multiple results in hypothesis-generating or surveillance studies.
Steenland, K., Bray, I., Greenland, S., & Boffetta, P. (2000) · PubMed · Cited by 63 · see citing works
Feise, R. J. (2001) · Journal of Manipulative and Physiological Therapeutics · doi:10.1067/mmt.2001.112007 · Cited by 23 · see citing works
Self-citationRE: "MULTIPLE COMPARISONS AND RELATED ISSUES IN THE INTERPRETATION OF EPIDEMIOLOGIC DATA"
Manor, O., & Peritz, E. (1997) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a009041 · Cited by 14 · see citing works
Turkheimer, F., Pettigrew, K. D., Sokoloff, L., Smith, C. B., & Schmidt, K. (2000) · NeuroImage · doi:10.1006/nimg.2000.0608 · Cited by 8 · see citing works
The interpretation of multiple P-values
Hart, A. A. M. (1994) · Radiotherapy and Oncology · doi:10.1016/0167-8140(94)90073-6 · Cited by 4 · see citing works
Bayesian and Non-Bayesian Methods of Inference
SMALL, R. D., & SCHOR, S. S. (1983) · Annals of Internal Medicine · doi:10.7326/0003-4819-99-6-857 · Cited by 4 · see citing works
Reply from Troyanovich regarding letter to theeditor from Lewit
Ormos, G. (2001) · Journal of Manipulative and Physiological Therapeutics · doi:10.1067/mmt.2001.112006 · Cited by 1 · see citing works
Significance of multiple inferential tests.
Wh, R., & Eicke, F. J. (1990) · PubMed · Cited by 1 · see citing works
Ostelo, R., & de Vet, H. C. W. (2001) · Journal of Manipulative and Physiological Therapeutics · doi:10.1067/mmt.2001.112008 · Cited by 1 · see citing works
Flagged first, then by citations
Research works printed but not linked: 1
- Feise RJ Behavioral-graded activity compared with usual care after first-time disk surgery: Considerations of the design of a randomized clinical trial (Letter). J Manipulative Physiol Ther 2001 24 67 68 11174699 10.1067/mmt.2001.112007 · DOI
Cited by
Sorted by citations
Adjust for multiple comparisons? It’s not that simple
Althouse, A. D. (2016) · The Annals of Thoracic Surgery · doi:10.1016/j.athoracsur.2015.11.024 · Cited by 825 · see citing works
Lonsdorf, T. B., Menz, M. M., Andreatta, M., Fullana, M. A., Golkar, A., Haaker, J., Heitland, I., Hermann, A., Kuhn, M., Kruse, O., et al. (2017) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2017.02.026 · Cited by 837 · see citing works
COVID-19 and vaccine hesitancy: A longitudinal study
Fridman, A., Gershon, R., & Gneezy, A. (2021) · PLoS ONE · doi:10.1371/journal.pone.0250123 · Cited by 677 · see citing works
Differences in Rural and Urban Health Information Access and Use
Chen, X., Orom, H., Hay, J. L., Waters, E. A., Schofield, E., Li, Y., & Kiviniemi, M. T. (2018) · The Journal of Rural Health · doi:10.1111/jrh.12335 · Cited by 602 · see citing works
Escaping the Bonferroni iron claw in ecological studies
García, L. V. (2004) · Oikos · doi:10.1111/j.0030-1299.2004.13046.x · Cited by 552 · see citing works
Meta-Analysis of MR Imaging in the Diagnosis of Breast Lesions
Peters, N. H. G. M., Rinkes, I. H. M. B., Zuithoff, N. P. A., Mali, W. P. T. M., Moons, K. G. M., & Peeters, P. H. M. (2007) · Radiology · doi:10.1148/radiol.2461061298 · Cited by 529 · see citing works
Rhea: a transparent and modular R pipeline for microbial profiling based on 16S rRNA gene amplicons
Lagkouvardos, I., Fischer, S. E., Kumar, N., & Clavel, T. (2017) · PeerJ · doi:10.7717/peerj.2836 · Cited by 434 · see citing works
An introduction to multiplicity issues in clinical trials: the what, why, when and how
Li, G., Taljaard, M., Van den Heuvel, E. R., Levine, M. A., Cook, D. J., Wells, G. A., Devereaux, P. J., & Thabane, L. (2016) · International Journal of Epidemiology · doi:10.1093/ije/dyw320 · Cited by 381 · see citing works
Green and blue spaces and behavioral development in Barcelona schoolchildren: The BREATHE project
Amoly, E., Dadvand, P., Forns, J., López-Vicente, M., Basagaña, X., Julvez, J., Alvarez-Pedrerol, M., Nieuwenhuijsen, M. J., & Sunyer, J. (2014) · Environmental Health Perspectives · doi:10.1289/ehp.1408215 · Cited by 390 · see citing works
Determinants of Disease Presentation and Outcome during Cryptococcosis: The CryptoA/D Study
Dromer, F., Mathoulin-Pélissier, S., Launay, O., Lortholary, O., & the French Cryptococcosis Study Group (2007) · PLoS Medicine · doi:10.1371/journal.pmed.0040021 · Cited by 374 · see citing works
Rubin, M. (2021) · Synthese · doi:10.1007/s11229-021-03276-4 · Cited by 371 · see citing works
Prevalence of Disorders Recorded in Dogs Attending Primary-Care Veterinary Practices in England
O'Neill, D. G., Church, D. B., McGreevy, P. D., Thomson, P. C., & Brodbelt, D. C. (2014) · PLoS ONE · doi:10.1371/journal.pone.0090501 · Cited by 376 · see citing works
Morawski, K., Ghazinouri, R., Krumme, A., Lauffenburger, J. C., Lu, Z., Durfee, E., Oley, L., Lee, J., Mohta, N., Haff, N., et al. (2018) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2018.0447 · Cited by 333 · see citing works
Wheelock, Å. M., & Wheelock, C. E. (2013) · Molecular BioSystems · doi:10.1039/c3mb70194h · Cited by 338 · see citing works
Effectiveness of the AUDIT-C as a Screening Test for Alcohol Misuse in Three Race/Ethnic Groups
Frank, D., DeBenedetti, A. F., Volk, R. J., Williams, E. C., Kivlahan, D. R., & Bradley, K. A. (2008) · Journal of General Internal Medicine · doi:10.1007/s11606-008-0594-0 · Cited by 336 · see citing works
Neville, H. J., Stevens, C., Pakulak, E., Bell, T. A., Fanning, J. L., Klein, S., & Isbell, E. (2013) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1304437110 · Cited by 334 · see citing works
Collaborative care for bipolar disorder, Part II: impact on clinical outcome, function, and costs.
Bauer, M. S., McBride, L., Williford, W. O., Glick, H., Kinosian, B., Altshuler, L., Beresford, T., Kilbourne, A. M., & Sajatovic, M. (2006) · Psychiatric Services · doi:10.1176/ps.2006.57.7.937 · Cited by 312 · see citing works
Gray, K. M., Carpenter, M. J., Baker, N. L., DeSantis, S. M., Kryway, E., Hartwell, K. J., McRae-Clark, A. L., & Brady, K. T. (2012) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2012.12010055 · Cited by 306 · see citing works
Johansson, M. E., Cameron, I. G. M., van der Kolk, N. M., de Vries, N. M., Klimars, E., Toni, I., Bloem, B. R., & Helmich, R. C. G. (2021) · Annals of Neurology · doi:10.1002/ana.26291 · Cited by 311 · see citing works
Possin, K. L., Merrilees, J. J., Dulaney, S., Bonasera, S. J., Chiong, W., Lee, K., Hooper, S. M., Allen, I. E., Braley, T., Bernstein, A., et al. (2019) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2019.4101 · Cited by 272 · see citing works
Modular cognitive-behavioral therapy for body dysmorphic disorder: A randomized controlled trial
Wilhelm, S., Phillips, K. A., Didie, E., Buhlmann, U., Greenberg, J. L., Fama, J. M., Keshaviah, A., & Steketee, G. (2014) · Behavior Therapy · doi:10.1016/j.beth.2013.12.007 · Cited by 238 · see citing works
Savini, G. (2005) · British Journal of Ophthalmology · doi:10.1136/bjo.2004.052498 · Cited by 218 · see citing works
Jones, A., Hardman, C. A., Lawrence, N., & Field, M. (2017) · Appetite · doi:10.1016/j.appet.2017.05.032 · Cited by 215 · see citing works
Common pitfalls in statistical analysis: The perils of multiple testing
Ranganathan, P., Pramesh, C. S., & Buyse, M. E. (2016) · Perspectives in Clinical Research · doi:10.4103/2229-3485.179436 · Cited by 206 · see citing works
Rubin, M. (2017) · Review of General Psychology · doi:10.1037/gpr0000123 · Cited by 205 · see citing works
Navarro-Haro, M. V., Modrego-Alarcón, M., Hoffman, H. G., López-Montoyo, A., Navarro-Gil, M., Montero-Marin, J., García-Palacios, A., Borao, L., & García-Campayo, J. (2019) · Frontiers in Psychology · doi:10.3389/fpsyg.2019.00055 · Cited by 189 · see citing works
Tropical mangrove species discrimination using hyperspectral data: A laboratory study
Vaiphasa, C., Ongsomwang, S., Vaiphasa, T., & Skidmore, A. K. (2005) · Estuarine Coastal and Shelf Science · doi:10.1016/j.ecss.2005.06.014 · Cited by 196 · see citing works
Mesnage, R., Renney, G., Séralini, G., Ward, M. A., & Antoniou, M. N. (2017) · Scientific Reports · doi:10.1038/srep39328 · Cited by 193 · see citing works
Neurocognitive profile in 22q11 deletion syndrome and schizophrenia
Chow, E. W. C., Watson, M., Young, D. A., & Bassett, A. S. (2006) · Schizophrenia Research · doi:10.1016/j.schres.2006.04.007 · Cited by 186 · see citing works
Tournadre, A., Pereira, B., Dutheil, F., Giraud, C., Courteix, D., Sapin, V., Frayssac, T., Mathieu, S., Malochet‐Guinamand, S., & Soubrier, M. (2017) · Journal of Cachexia Sarcopenia and Muscle · doi:10.1002/jcsm.12189 · Cited by 185 · see citing works
Correcting for multiple-testing in multi-arm trials: is it necessary and is it done?
Wason, J. M. S., Stecher, L., & Mander, A. (2014) · Trials · doi:10.1186/1745-6215-15-364 · Cited by 180 · see citing works
Martin, K., Wiesner, G., Breuer, T., Lange, R., & Tassani, P. (2008) · Anesthesia & Analgesia · doi:10.1213/ane.0b013e318184bc20 · Cited by 178 · see citing works
Electrical stimulation enhances sensory recovery: A randomized controlled trial
Wong, J. N., Olson, J. L., Morhart, M. J., & Chan, K. M. (2015) · Annals of Neurology · doi:10.1002/ana.24397 · Cited by 177 · see citing works
Wu, C., Huang, P.-C., Chen, Y., Lin, K., & Yang, H.-W. (2013) · Archives of Physical Medicine and Rehabilitation · doi:10.1016/j.apmr.2013.02.007 · Cited by 173 · see citing works
An Updated Recommendation for Multiple Comparisons
Sauder, D., & DeMars, C. E. (2019) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245918808784 · Cited by 181 · see citing works
Ross, S. M., Brewin, C. R., Curran, H. V., Furlong, C. E., Abraham-Smith, K., & Harrison, V. (2010) · Open Research Online - ORO (The Open University) · Cited by 164 · see citing works
Haagsma, J. A., Scholten, A. C., Andriessen, T. M. J. C., Vos, P. E., van Beeck, E. F., & Polinder, S. (2014) · Journal of Neurotrauma · doi:10.1089/neu.2013.3283 · Cited by 163 · see citing works
Novotny, J. A., Baer, D. J., Khoo, C., Gebauer, S. K., & Charron, C. S. (2015) · Journal of Nutrition · doi:10.3945/jn.114.203190 · Cited by 159 · see citing works
Ambient Air Pollution and Preeclampsia: A Spatiotemporal Analysis
Dadvand, P., Figueras, F., Basagaña, X., Beelen, R., Martinez, D., Cirach, M., Schembari, A., Hoek, G., Brunekreef, B., & Nieuwenhuijsen, M. J. (2013) · Environmental Health Perspectives · doi:10.1289/ehp.1206430 · Cited by 145 · see citing works
Coutinho, A. M., Sousa, I., Martins, M., Correia, C., Morgadinho, T., Bento, C., Marques, C., Ataíde, A., Miguel, T. S., Moore, J. H., et al. (2007) · Human Genetics · doi:10.1007/s00439-006-0301-3 · Cited by 153 · see citing works
Atchison, C., Bowman, L. R., Vrinten, C., Redd, R., Pristerà, P., Eaton, J., & Ward, H. (2021) · BMJ Open · doi:10.1136/bmjopen-2020-043577 · Cited by 379 · see citing works
Controlling the false discovery rate and increasing statistical power in ecological studies
Waite, T. A., & Campbell, L. G. (2006) · Ecoscience · doi:10.2980/1195-6860(2006)13[439:ctfdra]2.0.co;2 · Cited by 149 · see citing works
Edwards, R. R., Mensing, G., Cahalan, C., Greenbaum, S., Narang, S., Belfer, I., Schreiber, K. L., Campbell, C., Wasan, A. D., & Jamison, R. N. (2013) · Journal of Pain and Symptom Management · doi:10.1016/j.jpainsymman.2012.06.016 · Cited by 150 · see citing works
Colonization of second-trimester placenta parenchyma
Onderdonk, A. B., Hecht, J. L., McElrath, T. F., Delaney, M. L., Allred, E. N., & Leviton, A. (2008) · American Journal of Obstetrics and Gynecology · doi:10.1016/j.ajog.2007.11.068 · Cited by 146 · see citing works
Corpus callosum volume in children with autism
Hardan, A. Y., Pabalan, M., Gupta, N., Bansal, R., Melhem, N., Fedorov, S., Keshavan, M. S., & Minshew, N. J. (2009) · Psychiatry Research Neuroimaging · doi:10.1016/j.pscychresns.2009.03.005 · Cited by 146 · see citing works
Gleibs, I. H., Haslam, C., Jones, J. M., Alexander Haslam, S., McNeill, J., & Connolly, H. (2011) · Aging & Mental Health · doi:10.1080/13607863.2010.536137 · Cited by 140 · see citing works
Pham, J. C., Kirsch, T. D., Hill, P. M., DeRuggerio, K., & Hoffmann, B. (2011) · Academic Emergency Medicine · doi:10.1111/j.1553-2712.2011.01042.x · Cited by 138 · see citing works
Knols, R. H., de Bruin, E. D., Uebelhart, D., Aufdemkampe, G., Schanz, U., Stenner, F., Hitz, F., Taverna, C., & Aaronson, N. K. (2010) · Bone Marrow Transplantation · doi:10.1038/bmt.2010.288 · Cited by 139 · see citing works
Grucza, R. A., Bucholz, K. K., Rice, J. P., & Bierut, L. J. (2008) · Alcoholism Clinical and Experimental Research · doi:10.1111/j.1530-0277.2008.00635.x · Cited by 135 · see citing works
Parental cognitive impairment and child maltreatment in Canada
McConnell, D., Feldman, M. A., Aunos, M., & Prasad, N. (2011) · Child Abuse & Neglect · doi:10.1016/j.chiabu.2011.04.005 · Cited by 134 · see citing works
Agboada, D., Mosayebi-Samani, M., Jamil, A., Kuo, M., & Nitsche, M. A. (2019) · Scientific Reports · doi:10.1038/s41598-019-54621-0 · Cited by 133 · see citing works
Multiple comparisons: To compare or not to compare, that is the question
Barnett, M. J., Doroudgar, S., Khosraviani, V., & Ip, E. J. (2021) · Research in Social and Administrative Pharmacy · doi:10.1016/j.sapharm.2021.07.006 · Cited by 133 · see citing works
Cognitive deficits in long-term anabolic-androgenic steroid users
Kanayama, G., Kean, J., Hudson, J. I., & Pope, H. G. (2012) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2012.11.008 · Cited by 132 · see citing works
Fat Consumption and Its Association With Age-Related Macular Degeneration
Chong, E. W. T. (2009) · Archives of Ophthalmology · doi:10.1001/archophthalmol.2009.60 · Cited by 130 · see citing works
Consumer Neuroscience-Based Metrics Predict Recall, Liking and Viewing Rates in Online Advertising
Guixeres, J., Alcañiz, J. E. B., Azofra, J. M. A., Alcañíz, M., Colomer, A., Fuentes-Hurtado, F., & Ornedo, V. N. (2017) · Frontiers in Psychology · doi:10.3389/fpsyg.2017.01808 · Cited by 130 · see citing works
Yoga and meditation for menopausal symptoms in breast cancer survivors—a randomized controlled trial
Cramer, H., Rabsilber, S., Lauche, R., Kümmel, S., & Dobos, G. (2015) · Cancer · doi:10.1002/cncr.29330 · Cited by 131 · see citing works
Penetrance of Polygenic Obesity Susceptibility Loci across the Body Mass Index Distribution
Abadi, A., Alyass, A., Robiou-du-Pont, S., Bolker, B. M., Singh, P., Mohan, V., Díaz, R., Engert, J. C., Yusuf, S., Gerstein, H. C., et al. (2017) · The American Journal of Human Genetics · doi:10.1016/j.ajhg.2017.10.007 · Cited by 121 · see citing works
Rondonotti, E., Hassan, C., Tamanini, G., Antonelli, G., Andrisani, G., Leonetti, G., Paggi, S., Amato, A., Scardino, G., Di Paolo, D., et al. (2023) · Endoscopy · doi:10.1055/a-1852-0330 · Cited by 128 · see citing works
Effect of an Internet-Based Program on Weight Loss for Low-Income Postpartum Women
Phelan, S., Hagobian, T. A., Brannen, A., Hatley, K. E., Schaffner, A. A., Muñoz‐Christian, K., & Tate, D. F. (2017) · JAMA · doi:10.1001/jama.2017.7119 · Cited by 126 · see citing works
On‐Road Driving with Moderate Visual Field Loss
Bowers, A. R., Eli Peli, A., Elgin, J. M., McGWIN, G., & Owsley, C. J. (2005) · Optometry and Vision Science · doi:10.1097/01.opx.0000175558.33268.b5 · Cited by 127 · see citing works
Mindfulness-based stress reduction for residents: A randomized controlled trial
Verweij, H., van Ravesteijn, H., van Hooff, M. L. M., Lagro-Janssen, A. L. M., & Speckens, A. E. M. (2018) · Journal of General Internal Medicine · doi:10.1007/s11606-017-4249-x · Cited by 122 · see citing works
Demarzo, M., Montero-Marin, J., Puebla-Guedea, M., Navarro-Gil, M., Herrera-Mercadal, P., Moreno-González, S., Calvo-Carrión, S., Bafaluy-Franch, L., & Garcia-Campayo, J. (2017) · Frontiers in Psychology · doi:10.3389/fpsyg.2017.01343 · Cited by 121 · see citing works
Lagkouvardos, I., Kläring, K., Heinzmann, S. S., Platz, S., Scholz, B., Engel, K., Schmitt‐Kopplin, P., Haller, D., Rohn, S., Skurk, T., et al. (2015) · Molecular Nutrition & Food Research · doi:10.1002/mnfr.201500125 · Cited by 122 · see citing works
Sheikh, M. A., Abelsen, B., & Olsen, J. A. (2016) · Frontiers in Psychology · doi:10.3389/fpsyg.2016.00727 · Cited by 122 · see citing works
Lo, K., Stephenson, M., & Lockwood, C. S. (2017) · The JBI Database of Systematic Reviews and Implementation Reports · doi:10.11124/jbisrir-2017-003456 · Cited by 122 · see citing works
Kennedy, P. J., Cryan, J. F., Quigley, E. M. M., Dinan, T. G., & Clarke, G. (2014) · Psychological Medicine · doi:10.1017/s003329171400052x · Cited by 120 · see citing works
Youth Empowerment Programs for Improving Adolescents’ Self-Efficacy and Self-Esteem
Morton, M., & Montgomery, P. (2012) · Research on Social Work Practice · doi:10.1177/1049731512459967 · Cited by 120 · see citing works
O’Leary, C., Nassar, N., Zubrick, S. R., Kurinczuk, J. J., Stanley, F. J., & Bower, C. I. (2009) · Addiction · doi:10.1111/j.1360-0443.2009.02756.x · Cited by 117 · see citing works
A Survey of American Psychiatrists' Attitudes Toward Classic Hallucinogens
Barnett, B. S., Siu, W. O., & Pope, H. G. (2018) · Journal of Nervous & Mental Disease · doi:10.1097/nmd.0000000000000828 · Cited by 116 · see citing works
Docking, S., & Cook, J. L. (2015) · Scandinavian Journal of Medicine and Science in Sports · doi:10.1111/sms.12491 · Cited by 117 · see citing works
Residual neurocognitive features of long‐term ecstasy users with minimal exposure to other drugs
Halpern, J. H., Sherwood, A. R., Hudson, J. I., Gruber, S. A., Kozin, D., & Pope, H. G. (2010) · Addiction · doi:10.1111/j.1360-0443.2010.03252.x · Cited by 118 · see citing works
Figueroa, R. (2010) · Journal of Developmental & Behavioral Pediatrics · doi:10.1097/dbp.0b013e3181e5ac93 · Cited by 116 · see citing works
Nevicka, B., Vianen, A. E. M. V., Hoogh, A. H. B. D., & Voorn, B. (2018) · Journal of Applied Psychology · doi:10.1037/apl0000298 · Cited by 122 · see citing works
An MRI and proton spectroscopy study of the thalamus in children with autism
Hardan, A. Y., Minshew, N. J., Melhem, N., Srihari, S., Jo, B., Bansal, R., Keshavan, M. S., & Stanley, J. A. (2008) · Psychiatry Research Neuroimaging · doi:10.1016/j.pscychresns.2007.12.002 · Cited by 115 · see citing works
Ivanov, A. V., Dabiesingh, D. S., Bhumireddy, G. P., Mohamed, A., Asfour, A. M. S., Briggs, W. M., Ho, J., Khan, S., Grossman, A., Klem, I., et al. (2017) · Circulation Cardiovascular Imaging · doi:10.1161/circimaging.117.006174 · Cited by 114 · see citing works
Association between urban environment and mental health in Brussels, Belgium
Pelgrims, I., Devleesschauwer, B., Guyot, M., Keune, H., Nawrot, T. S., Remmen, R., Saenen, N. D., Trabelsi, S., Thomas, I., Aerts, R., et al. (2021) · BMC Public Health · doi:10.1186/s12889-021-10557-7 · Cited by 115 · see citing works
Corpus Callosum Volume and Neurocognition in Autism
Keary, C. J., Minshew, N. J., Bansal, R., Goradia, D. D., Fedorov, S., Keshavan, M. S., & Hardan, A. Y. (2009) · Journal of Autism and Developmental Disorders · doi:10.1007/s10803-009-0689-4 · Cited by 115 · see citing works
Dzakpasu, S., Fahey, J., Kirby, R. S., Tough, S., CHALMERS, B. E., Heaman, M. I., Bartholomew, S., Biringer, A. B., Darling, E., Lee, L. S., et al. (2015) · BMC Pregnancy and Childbirth · doi:10.1186/s12884-015-0452-0 · Cited by 113 · see citing works
MetaboLyzer: A Novel Statistical Workflow for Analyzing Postprocessed LC–MS Metabolomics Data
Mak, T. D., Laiakis, E. C., Goudarzi, M., & Fornace, A. J. (2013) · Analytical Chemistry · doi:10.1021/ac402477z · Cited by 110 · see citing works
Consideration of smoothing techniques for hyperspectral remote sensing
Vaiphasa, C. (2005) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2005.11.002 · Cited by 109 · see citing works
Treatment motivation in drug users: A theory-based analysis
Longshore, D. L., & Teruya, C. (2005) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2005.06.011 · Cited by 109 · see citing works
The Effects of Thoracic Manipulation on Heart Rate Variability: A Controlled Crossover Trial
Budgell, B., & Polus, B. (2006) · Journal of Manipulative and Physiological Therapeutics · doi:10.1016/j.jmpt.2006.08.011 · Cited by 108 · see citing works
Loneliness in Adults on the Autism Spectrum
Ee, D., Hwang, Y. I., Reppermund, S., Srasuebkul, P., Trollor, J. N., Foley, K., & Arnold, S. R. C. (2019) · Autism in Adulthood · doi:10.1089/aut.2018.0038 · Cited by 107 · see citing works
Kennedy, P. J., Clarke, G., O‘Neill, A., Groeger, J. A., Quigley, E. M. M., Shanahan, F., Cryan, J. F., & Dinan, T. G. (2014) · Psychological Medicine · doi:10.1017/s0033291713002171 · Cited by 106 · see citing works
Influence of Bolus Consistency on Lingual Behaviors in Sequential Swallowing
Steele, C. M., & Lieshout, P. H. H. M. V. (2004) · Dysphagia · doi:10.1007/s00455-004-0006-5 · Cited by 106 · see citing works
Interictal quantitative EEG in migraine: a blinded controlled study
Bjørk, M., Stovner, L. J., Engstrøm, M., Stjern, M., Hagen, K., & Sand, T. (2009) · The Journal of Headache and Pain · doi:10.1007/s10194-009-0140-4 · Cited by 106 · see citing works
Martinez, A., Proupim, N., & Sánchez, M. (2008) · British Journal of Ophthalmology · doi:10.1136/bjo.2008.137471 · Cited by 105 · see citing works
Cox, C. M., Kenardy, J. A., & Hendrikz, J. K. (2009) · Journal of Pediatric Psychology · doi:10.1093/jpepsy/jsp095 · Cited by 103 · see citing works
Kiehna, E. N., Mulhern, R. K., Li, C., Xiong, X., & Merchant, T. E. (2006) · Journal of Clinical Oncology · doi:10.1200/jco.2005.03.8547 · Cited by 103 · see citing works
Volumetric alterations of the orbitofrontal cortex in autism
Girgis, R. R., Minshew, N. J., Melhem, N., Nutche, J. J., Keshavan, M. S., & Hardan, A. Y. (2006) · Progress in Neuro-Psychopharmacology and Biological Psychiatry · doi:10.1016/j.pnpbp.2006.06.007 · Cited by 102 · see citing works
Mezger, M., Steffens, M., Beyer, M., Manger, C., Eberle, J., Toliat, M. R., Wienker, T. F., Ljungman, P. T., Hebart, H. F., Dornbusch, H. J., et al. (2007) · Blood · doi:10.1182/blood-2007-05-090928 · Cited by 101 · see citing works
Pope, H. G., Kean, J., Nash, A., Kanayama, G., Samuel, D. B., Bickel, W. K., & Hudson, J. I. (2010) · Experimental and Clinical Psychopharmacology · doi:10.1037/a0019370 · Cited by 100 · see citing works
Grass, D. S., Ross, J., Family, F., Barbour, J. R., Simpson, H. J., Coulibaly, D. M., Hernandez, J., Chen, Y., Slavkovich, V. N., Li, Y., et al. (2009) · Environmental Research · doi:10.1016/j.envres.2009.10.006 · Cited by 99 · see citing works
Murphy, A. W., Cupples, M. E., Smith, S. M., Byrne, M., Byrne, M. C., Newell, J., & for the SPHERE study team (2009) · BMJ · doi:10.1136/bmj.b4220 · Cited by 99 · see citing works
Faasse, K., Cundy, T., Gamble, G. D., & Petrie, K. J. (2012) · Psychosomatic Medicine · doi:10.1097/psy.0b013e3182738826 · Cited by 95 · see citing works
Isernia, S., Tella, S. D., Pagliari, C., Jonsdottir, J., Castiglioni, C., Gindri, P., Salza, M., Gramigna, C., Palumbo, G., Molteni, F., et al. (2020) · Frontiers in Neurology · doi:10.3389/fneur.2020.00846 · Cited by 98 · see citing works
McLean, S. A., Diatchenko, L. B., Lee, Y. M., Swor, R. A., Domeier, R. M., Jones, J. S., Jones, C. W., Reed, C., Harris, R. E., Maixner, W. D., et al. (2010) · Journal of Pain · doi:10.1016/j.jpain.2010.05.008 · Cited by 96 · see citing works
Dwulit, A. D., & Rzymski, P. (2019) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph16101861 · Cited by 95 · see citing works
AP-1 Transcription Factors Mediate BDNF-Positive Feedback Loop in Cortical Neurons
Tuvikene, J., Pruunsild, P., Orav, E., Esvald, E., & Timmusk, T. (2016) · Journal of Neuroscience · doi:10.1523/jneurosci.3360-15.2016 · Cited by 94 · see citing works
- Authors
- Feise, R. J.
- Corresponding author
- Feise, Ronald J
- ORCID
- Feise, Ronald J
- Venue
- BMC Medical Research Methodology
- Published
- 2002-06-17
- Volume
- 2
- Issue
- 1
- Pages
- 8-8
- ISSN
- 1471-2288 (ISSN-L)
- PMC
- PMC117123
- OpenAlex
- W2139498473
- Cited by
- 1,299
- References
- 42
- Citations by year
- 2026: 55 · 2025: 67 · 2024: 75 · 2023: 101 · 2022: 79 · 2021: 133 · 2020: 95 · 2019: 115 · 2018: 83 · 2017: 98 · 2016: 72 · 2015: 45 · 2014: 47 · 2013: 33 · 2012: 32
- FWCI
- 5.75
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- BMC Medical Research Methodology (publishedVersion)
- Europe PMC (PubMed Central) (publishedVersion, other-oa)
- PubMed Central (publishedVersion, other-oa)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Keywords
- p-value adjustment, type I error, multiple comparisons, clinical trials, primary outcomes, statistical significance, sample size
- MeSH terms
- Clinical Trials as Topic, Data Interpretation, Statistical, Humans, Confidence Intervals, Treatment Outcome
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
