The DOI matches its Crossref record; 6 of the 7 references it cites were found in a registry or an index.
Small studies: strengths and limitations
Open accessAbstract
A large number of clinical research studies are conducted, including audits of patient data, observational studies, clinical trials and those based on laboratory analyses. While small studies can be published over a short time-frame, there needs to be a balance between those that can be performed quickly and those that should be based on more subjects and hence may take several years to complete. The present article provides an overview of the main considerations associated with small studies. The definition of “small” depends on the main study objective. When simply describing the characteristics of a single group of subjects, for example the prevalence of smoking, the larger the study the more reliable the results. The main results should have 95% confidence intervals (CI), and the width of these depend directly on the sample size: large studies produce narrow intervals and, therefore, more precise results. A study of 20 subjects, for example, is likely to be too small for most investigations. For example, imagine that the proportion of smokers among a particular group of 20 individuals is 25%. The associated 95% CI is 9–49. This means that the true prevalence in these subjects generally is anywhere between a low or high value, which is not a useful result. When comparing characteristics between two or more groups of subjects ( e.g. examining risk factors or treatments for disease), the size of the study depends on the magnitude of the expected effect size, which is usually quantified by a relative risk, odds ratio, absolute risk difference, hazard ratio, or difference between two means or medians. The smaller the true-effect size, the larger the study needs to be 1, 2. This is because it is more difficult to distinguish between a real effect and random variation. Consider mortality as the end-point in a …
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Studies of intervention and control group population sample sizes (N) < 10 were excluded to reduce the likelihood of small study effects, for example over-estimation of effect size [33].”
From Estimating the Effect of Aerobic Exercise Training on Novel Lipid Biomarkers: A Systematic Review and Multivariate Meta-Analysis of Randomized Controlled Trials (2023) · cites this paper as [33]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“This fact was confirmed in 2021 by the European Institute for Gender Equality [67], which shows only 30.7% of women on Ibex-35 companies' Board of Directors.”
From Pilot study of an online brief mindfulness-based intervention to improve the well-being of managers and reduce their stress at work. (2024) · cites this paper as [67]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Clinical Trials: A Practical Approach.
Hill, I. D., & Pocock, S. (1984) · Biometrics · doi:10.2307/2531186 · Cited by 2,918
Statistics notes: Absence of evidence is not evidence of absence
Altman, D. G., & Bland, J. M. (1995) · BMJ · doi:10.1136/bmj.311.7003.485 · Cited by 1,946
Sample Size Tables for Clinical Studies
Machin, D., Campbell, M. J., Tan, S. B., & Tan, S. H. (2008) · Wiley eBooks · doi:10.1002/9781444300710 · Cited by 1,336
Clinical Trials a Practical Approach
Pocock, S. J. (2013) · Thammasat University Digital Collections · Cited by 213
Lee, S. M., James, L. E., Büchler, T., Snee, M. P., Ellis, P., & Hackshaw, A. K. (2007) · Lung Cancer · doi:10.1016/j.lungcan.2007.08.032 · Cited by 36
Self-citationLee, S. M., Rudd, R., Woll, P. J., Ottensmeier, C. H. H., James, L. E., Gower, N. H., Spiro, S. G., Jitlal, M., & Hackshaw, A. K. (2008) · Journal of Clinical Oncology · doi:10.1200/jco.2008.26.15_suppl.8045 · Cited by 12
Self-citation
Cited by
Sorted by citations
The global prevalence of rheumatoid arthritis: a meta-analysis based on a systematic review
Almutairi, K. B., Nossent, J. C., Preen, D. B., Keen, H. I., & Inderjeeth, C. A. (2020) · Rheumatology International · doi:10.1007/s00296-020-04731-0 · Cited by 810
Wang, L., Hong, P. J., May, C., Rehman, Y., Oparin, Y., Hong, C. J. H., Hong, B. Y., AminiLari, M., Gallo, L., Kaushal, A., et al. (2021) · BMJ · doi:10.1136/bmj.n1034 · Cited by 318
Wallis, C. J. D., Butaney, M., Satkunasivam, R., Freedland, S. J., Patel, S. P., Hamid, O., Pal, S. K., & Klaassen, Z. W. A. (2019) · JAMA Oncology · doi:10.1001/jamaoncol.2018.5904 · Cited by 294
Ahmad, R., Riaz, M., Khan, A., Aljamea, A., Algheryafi, M., Sewaket, D., & Alqathama, A. A. (2021) · Phytotherapy Research · doi:10.1002/ptr.7215 · Cited by 183
“Yes, indeed, such kinds of studies (enrolling patients, reviewing patient records, performing biochemical analyses/asking subjects to complete study questionnaires) are help- ful, and data produced thereof may be utilised to design confirmatory studies on a large scale (Hackshaw, 2008).”
BackgroundCortez, J. M., Gupta, M., Amaram, A., Pizzino, J., Sawhney, M., & Sood, B. G. (2010) · The Journal of Maternal-Fetal & Neonatal Medicine · doi:10.3109/14767058.2010.511335 · Cited by 164
“Such studies are often performed in a small number of subjects initially [26].”
BackgroundDescribing Populations and Samples in Doctoral Student Research
Casteel, A., & Bridier, N. (2021) · International journal of doctoral studies · doi:10.28945/4766 · Cited by 173
“If the sampling frame is not large enough, the researcher runs the risk of not obtaining sufficient data, resulting in imprecise measures of effect size for quantitative studies (Hackshaw, 2008) or not reaching data saturation for qualitative studies (Fusch & Ness, 2015).”
BackgroundMetabolomic Profiling of Bile Acids in Clinical and Experimental Samples of Alzheimer’s Disease
Pan, X., Elliott, C. T., McGuinness, B., Passmore, P., Kehoe, P. G., Hölscher, C., McClean, P. L., Graham, S. F., & Green, B. D. (2017) · Metabolites · doi:10.3390/metabo7020028 · Cited by 153
“Thirdly, the relatively small sample size and student’s T-test that were used in this pilot study may result in false-positives or in over-estimating the magnitude of changes [29].”
BackgroundThe effectiveness of dyadic interventions for people with dementia and their caregivers
Moon, H. E., & Adams, K. B. (2012) · Dementia · doi:10.1177/1471301212447026 · Cited by 153
“The small studies may be unable to achieve statistical power, and results may be misinterpreted or fail to produce reliable or precise estimates (resulting, for example, in false positive results or overestimation) (Hackshaw, 2008).”
BackgroundAlansare, A. B., Alford, K., Lee, S., Church, T., & Jung, H. C. (2018) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph15071508 · Cited by 137
“It is believed that the adequate number of subjects are required in clinical interventions to provide precise conclusions [36].”
BackgroundGreco, E., Simić, T., Ringash, J. G., TOMLINSON, G. A., Inamoto, Y., & Martino, R. (2018) · International Journal of Radiation Oncology*Biology*Physics · doi:10.1016/j.ijrobp.2018.01.097 · Cited by 125
Obesity negatively impacts lung function in children and adolescents
Davidson, W. J., Mackenzie‐Rife, K. A., Witmans, M., Montgomery, M. D., Ball, G. D. C., Egbogah, S., & Eves, N. D. (2013) · Pediatric Pulmonology · doi:10.1002/ppul.22915 · Cited by 129
Daling, L. M., & Schlittmeier, S. J. (2022) · Human Factors The Journal of the Human Factors and Ergonomics Society · doi:10.1177/00187208221105135 · Cited by 117
“In order to include studies with reliable and precise estimates, it was also determined that only studies with sample sizes of n ≥ 10 should be considered in the fi nal review, ensuring the statistical foundation of the conclusions (Hackshaw, 2008; Aguinis & Harden, 2008).”
BackgroundSzefler, S. J., Murphy, K. R., Harper, T. B., Boner, A. L., Laki, I., Engel, M., Azzi, G. E., Moroni-Zentgraf, P., Finnigan, H., & Hamelmann, E. H. (2017) · Journal of Allergy and Clinical Immunology · doi:10.1016/j.jaci.2017.01.014 · Cited by 115
“Furthermore, although smaller, short-term studies can provide valuable information over a short time frame, they may overestimate the magnitude of treatment effects and can require validation from larger confirmatory studies.(61) Nevertheless, the relatively large participant population in this trial increases the rel…”
BackgroundLukyanenko, R., Parsons, J., Wiersma, Y. F., & Maddah, M. (2019) · MIS Quarterly · doi:10.25300/misq/2019/14439 · Cited by 123
Research Population and Sampling in Quantitative Study
Hossan, D., Mansor, Z. D., & Jaharuddin, N. S. (2023) · International Journal of Business and Technopreneurship (IJBT) · doi:10.58915/ijbt.v13i3.263 · Cited by 111
Thalidomide for managing cancer cachexia
Reid, J., Mills, M. E., Cantwell, M. M., Cardwell, C. R., Murray, L., & Donnelly, M. (2012) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd008664.pub2 · Cited by 109
Association Between Sex and Immune-Related Adverse Events During Immune Checkpoint Inhibitor Therapy
Jing, Y., Zhang, Y., Wang, J., Li, K., Chen, X., Heng, J., Gao, Q., Ye, Y., Zhang, Z., Liu, Y., et al. (2021) · JNCI Journal of the National Cancer Institute · doi:10.1093/jnci/djab035 · Cited by 103
Effectiveness of Mobile Apps for Smoking Cessation: Α Review
Regmi, K., Kassim, N., Ahmad, N., & Tuah, N. A. (2017) · Tobacco Prevention & Cessation · doi:10.18332/tpc/70088 · Cited by 97
Sealing versus partial caries removal in primary molars: a randomized clinical trial
Hesse, D., Bonifácio, C. C., Mendes, F. M., Braga, M. M., Imparato, J. C. P., & Raggio, D. P. (2014) · BMC Oral Health · doi:10.1186/1472-6831-14-58 · Cited by 98
“Nevertheless, we acknowledge the limits of our study as having a small sample size and a limited time of evaluation; however according to Hackshaw [40] there is nothing wrong with conducting well-designed small studies; they just need to be interpreted carefully.”
MethodSpace Use and Habitat Selection by Resident and Transient Coyotes (Canis latrans)
Hinton, J. W., van Manen, F. T., & Chamberlain, M. J. (2015) · PLoS ONE · doi:10.1371/journal.pone.0132203 · Cited by 98
“Because estimates of density, dispersal, and survival may be biased within small study areas [21,22], we define a minimum geographic extent as 2,500 km(2).”
BackgroundLiu, X., Guo, C., Tou, F., Wen, X., Kuang, Y., Zhu, Q., & Hu, H. (2019) · International Journal of Cancer · doi:10.1002/ijc.32744 · Cited by 83
“meta-analysis, which may result in an elevated false discovery rate(46) or even false-positive results.(47) Second, several subgroups, for which only two trials were considered, might have increased the probability of selection bias.”
BackgroundJiménez-Redondo, S., de Miguel, B. B., Banegas, J. G., Mercedes, L. G., Gómez‐Pavón, J., & Vives, C. C. (2013) · The journal of nutrition health & aging · doi:10.1007/s12603-013-0416-x · Cited by 79
Effect of Elevated Ambient Temperature on Maternal, Foetal, and Neonatal Outcomes: A Scoping Review
Dalugoda, Y., Kuppa, J., Phung, H. N., Rutherford, S., & Phung, D. T. (2022) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph19031771 · Cited by 81
“Caution must be applied when interpreting findings from studies with a small sample size, as the findings might not be reliable or precise [99].”
BackgroundThe growth potential for dairy probiotics
Reid, G. (2015) · International Dairy Journal · doi:10.1016/j.idairyj.2015.04.004 · Cited by 81
“While a larger sample size is needed to prove cause and effect, these small studies can provide useful information to identify potential responders, as long as conclusions are not over-drawn (Hackshaw, 2008).”
BackgroundSmall Samples in Instructed Second Language Acquisition Research
Loewen, S., & Hui, B. (2021) · Modern Language Journal · doi:10.1111/modl.12700 · Cited by 72
Gao, J., Choo, K. T. W., Cao, J., Lee, R. K.-W., & Perrault, S. T. (2023) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3617362 · Cited by 70
“However, no significant differences were discerned within Despite the modest size of the statistically significant differences observed in our study, it is worth noting that even such small results can have meaningful implications [5, 41].”
BackgroundCritical Success Factors to Improve the Game Development Process from a Developer's Perspective
Aleem, S., Capretz, L. F., & Ahmed, F. (2016) · Scholarship@Western (Western University) · Cited by 70
Johnson, S., Cowell, F., Gillespie, S., & Göebel, A. (2022) · European Journal of Pain · doi:10.1002/ejp.1953 · Cited by 68
“…identifies long-term predictors, and treatments that can foster good functional and occupational recovery. size of ≤20 (such samples were considered too small to draw yield reliable or precise estimates (Hackshaw, 2008), assessed response to treatment, or were not published in English or French.”
BackgroundMcKee, G., Codd, M., Dempsey, O., Gallagher, P., & Comiskey, C. M. (2017) · BMC Nursing · doi:10.1186/s12912-017-0214-6 · Cited by 68
COPD: misuse of inhaler devices in clinical practice
Araújo, A., Teixeira, P., Hespanhol, V. P., & de Sousa, J. C. (2019) · International Journal of COPD · doi:10.2147/copd.s178040 · Cited by 67
“This may limit the generalization of results to other populations.(24) However, we have studied a significant number of patients using a wide range of IDs.”
ResultEarly versus late awake prone positioning in non-intubated patients with COVID-19
Kaur, R., Vines, D. L., Mirza, S. H., Elshafei, A. A., Jackson, J. A., Harnois, L. J., Weiss, T. T., Scott, J. B., Trump, M. W., Mogri, I., et al. (2021) · Critical Care · doi:10.1186/s13054-021-03761-9 · Cited by 66
“The sample size in this study is very small which may have impacted the magnitude of an association between the study groups and outcomes [36].”
Pope, C. J., Mazmanian, D. S., Bédard, M., & Sharma, V. (2016) · Journal of Affective Disorders · doi:10.1016/j.jad.2016.04.014 · Cited by 66
Stress and Chronic Illness: the Case of Diabetes
Morris, T., Moore, M., & Morris, F. A. (2011) · Journal of Adult Development · doi:10.1007/s10804-010-9118-3 · Cited by 72
“In quantitative research, a large sample size ensures adequate power, thus increasing the credibility and generalisability of the results; whereas findings based on small samples need to be interpreted with caution, because it is unlikely that the results reflect reliable or precise estimates due to limited power (e.g…”
ResultChataway, M. L., Hart, T. C., Coomber, R., & Bond, C. (2017) · Applied Geography · doi:10.1016/j.apgeog.2017.06.010 · Cited by 62
“However, we have made it clear that our findings have been produced from a hypothesis-generating study and that a larger confirmatory study is still needed (Hackshaw, 2008).”
ResultHam, K., Chin, S., Suh, Y. J., Rhee, M., Yu, E.-S., Lee, H. J., Kim, J.-H., Kim, S. W., Koh, S.-J., & Chung, K.-M. (2019) · Frontiers in Psychology · doi:10.3389/fpsyg.2019.01592 · Cited by 61
COPD: understanding patients’ adherence to inhaled medications
Araújo, A., Teixeira, P., Hespanhol, V. P., & de Sousa, J. C. (2018) · International Journal of COPD · doi:10.2147/copd.s160982 · Cited by 58
“This may limit the generalization of results to other populations.(35) As patients were recruited sequentially, we cannot exclude selection bias.”
ResultSearching for a Consensus Among Inflammatory Bowel Disease Studies: A Systematic Meta-Analysis
Abdel-Rahman, L. I. H., & Morgan, X. C. (2022) · Inflammatory Bowel Diseases · doi:10.1093/ibd/izac194 · Cited by 54
Lu, L., Gong, P., Cong, N., Li, Z., Zhao, Y., & Chen, Y. (2019) · BMC Public Health · doi:10.1186/s12889-019-7022-8 · Cited by 56
“One recommendation for comparing characteristics between groups of subjects in small studies is looking at the degree of difference [37].”
BackgroundVoorberg, A. N., Kamphuis, E., Christoffers, W. A., & Schuttelaar, M. L. A. (2023) · British Journal of Dermatology · doi:10.1093/bjd/ljad156 · Cited by 50
“29 Furthermore, in a randomized controlled study, it was found that patients with AD show a higher placebo effect and maintain this placebo effect longer than persons with healthy skin.”
Genomic and genetic studies of systemic sclerosis: A systematic review
Chairta, P. P., Nicolaou, P., & Christodoulou, K. (2016) · Human Immunology · doi:10.1016/j.humimm.2016.10.017 · Cited by 51
Belina, D., Hailu, Y., Gobena, T., Hald, T., & Njage, P. M. K. (2021) · One Health Outlook · doi:10.1186/s42522-021-00048-5 · Cited by 58
Effects of exercise on electroencephalography-recorded neural oscillations: a systematic review
Hosang, L., Mouchlianitis, E. D., Guérin, S. M. R., & Karageorghis, C. I. (2022) · International Review of Sport and Exercise Psychology · doi:10.1080/1750984x.2022.2103841 · Cited by 50
The Role of Depression in Short-Term Mood and Fatigue Responses to Acute Exercise
Weinstein, A. A., Deuster, P. A., Francis, J., Beadling, C. W., & Kop, W. J. (2009) · International Journal of Behavioral Medicine · doi:10.1007/s12529-009-9046-4 · Cited by 48
“A major limitation of studies with small sample sizes is that they can produce falsepositive results or overestimate the magnitude of an association [25].”
BackgroundHockey, A., Milojev, P., Sibley, C. G., Donovan, C. L., & Barlow, F. K. (2021) · Body Image · doi:10.1016/j.bodyim.2021.06.007 · Cited by 63
Association Mapping Analysis for Fruit Quality Traits in Prunus persica Using SNP Markers
Forcada, C. F. I., Guajardo, V., Reyes-Chin-Wo, S., & Moreno, M. Á. (2019) · Frontiers in Plant Science · doi:10.3389/fpls.2018.02005 · Cited by 45
“Although the small sample size can cause a loss of power or provide associations that are not of general applicability (Hackshaw, 2008), they are valid for the set of genotypes used in the present study.”
BackgroundRuscitti, P., Ursini, F., Cipriani, P., Greco, M., Alvaro, S., Vasiliki, L., Benedetto, P. D., Carubbi, F., Berardicurti, O., Gulletta, E., et al. (2019) · Medicine · doi:10.1097/md.0000000000014587 · Cited by 47
Vitale, R., & Kim, Y. (2020) · Metabolic Syndrome and Related Disorders · doi:10.1089/met.2020.0048 · Cited by 43
“71 Second, only one study 32 measured both macronutrient and caloric changes from participant documented food diaries.”
Böge, K., Hahne, I., Bergmann, N., Wingenfeld, K., Zierhut, M., Thomas, N., Ta, T. M. T., Bajbouj, M., & Hahn, E. (2021) · Schizophrenia Research · doi:10.1016/j.schres.2020.12.008 · Cited by 44
Smith, C., Doma, K., Heilbronn, B., & Leicht, A. S. (2022) · Military Medicine · doi:10.1093/milmed/usac040 · Cited by 42
Current understanding of the metabolism of micronutrients in chronic alcoholic liver disease
Wu, J., & Meng, Q. (2020) · World Journal of Gastroenterology · doi:10.3748/wjg.v26.i31.4567 · Cited by 44
Atlaw, D., Sahiledengle, B., & Tariku, Z. (2021) · Environmental Health and Preventive Medicine · doi:10.1186/s12199-021-00983-9 · Cited by 41
“This is supported by the fact that a small sample size will have a wider confidence interval and overestimate the magnitude of the association [87].”
BackgroundHoechter, D. J., Becker-Pennrich, A., Langrehr, J., Bruegel, M., Zwißler, B., Schaefer, S., Spannagl, M., Hinske, L. C., & Zöller, M. (2020) · Thrombosis Research · doi:10.1016/j.thromres.2020.08.030 · Cited by 40
“The smaller the true-effect size, the larger the study needs to be to distinguish between real effects and random variations [26].”
Hassan, J., Shannon, S., Tully, M. A., McCartan, C. J., Davidson, G., Bunn, R., & Breslin, G. (2022) · Journal of Psychiatric and Mental Health Nursing · doi:10.1111/jpm.12832 · Cited by 36
“Sample sizes can affect the reliability, as a wide 95% confidence intervals and large standard errors will occur within small samples, i.e., 20 participants may not produce precise results for the cohort (Hackshaw, 2008).”
Empirical investigation of key business factors for digital game performance
Aleem, S., Capretz, L. F., & Ahmed, F. (2015) · Entertainment Computing · doi:10.1016/j.entcom.2015.09.001 · Cited by 39
Ganguly, A., Talukdar, A., & Chatterjee, D. (2019) · Knowledge and Process Management · doi:10.1002/kpm.1614 · Cited by 39
Neufeld, C. B., Palma, P. D. C., Caetano, K. A. D. S., Brust-Renck, P. G., Curtiss, J. E., & Hofmann, S. G. (2019) · International Journal of Clinical and Health Psychology · doi:10.1016/j.ijchp.2019.11.004 · Cited by 43
Wood, G., Taylor, E., Ng, V., Murrell, A., Patil, A., van der Touw, T., Sigal, R., Wolden, M., & Smart, N. (2022) · British Journal of Sports Medicine · doi:10.1136/bjsports-2021-103999 · Cited by 45
Cheng, W. H., Teo, R. H., Cheng, L. J., Lau, Y., & Lau, S. T. (2023) · Sleep Health · doi:10.1016/j.sleh.2023.04.004 · Cited by 41
Meta-analysis in finance research: Opportunities, challenges, and contemporary applications
Geyer‐Klingeberg, J., Hang, M., & Rathgeber, A. (2020) · International Review of Financial Analysis · doi:10.1016/j.irfa.2020.101524 · Cited by 41
Pan, H., Fokkema, T., Switsers, L., Dury, S., Hoens, S., & Donder, L. D. (2021) · European Journal of Ageing · doi:10.1007/s10433-021-00625-7 · Cited by 40
“A small sample size increases the probability of making a type II error (i.e. predictors are deemed insignificant while in fact they are important; Hackshaw 2008).”
BackgroundNon-invasive blood glucose monitoring technology in diabetes management: review
Moses, J. C., Adibi, S., Wickramasinghe, N. S., Nguyen, L., Angelova, M. N., & Islam, S. M. S. (2024) · mHealth · doi:10.21037/mhealth-23-9 · Cited by 41
“Studies with a small sample size could negatively influence the outcomes (40), hence we have considered studies with at least 20 diabetes participants.”
Sankarasubramanian, V., Harte, S. E., Chiravuri, S., Harris, R. E., Brummett, C. M., Patil, P. G., Clauw, D. J., & Lempka, S. F. (2018) · Neuromodulation Technology at the Neural Interface · doi:10.1111/ner.12804 · Cited by 37
“They are more likely to produce false-positive results or over-estimate the magnitude of associations between SCS and clinical outcomes (128).”
BackgroundLi, A. Y. C., & Liu, J. K. K. (2021) · Current Psychology · doi:10.1007/s12144-021-02006-w · Cited by 42
“Furthermore, this result drawn from relatively low sample size can create a false positive result (Hackshaw, 2008).”
BackgroundLoneliness inside of the brain: evidence from a large dataset of resting-state fMRI in young adult
Denilson Brilliant, T., Takeuchi, H., Nouchi, R., Yokoyama, R., Kotozaki, Y., Nakagawa, S., Hanawa, S., Sekiguchi, A., Ikeda, S., Sakaki, K., et al. (2022) · Scientific Reports · doi:10.1038/s41598-022-11724-5 · Cited by 38
Cerea, S., Bottesi, G., Grisham, J. R., & Ghisi, M. (2017) · International Journal of Psychiatry in Clinical Practice · doi:10.1080/13651501.2017.1393545 · Cited by 36
“Some statistical biases related to small sample sizes, as for example under-powered results, falsepositive results, or overestimation of the magnitude of associations (Friston, 2012; Hackshaw, 2008), are well known; therefore, present findings have to be interpreted with caution.”
BackgroundShore, N. D., Tammela, T. L. J., Massard, C., Bono, P., Aspegrén, J., Mustonen, M. V. J., & Fizazi, K. S. (2017) · European Urology Focus · doi:10.1016/j.euf.2017.01.015 · Cited by 33
“Nevertheless, the promising antitumour activity and favourable safety profile of ODM201 provide a basis for future confirmatory phase 3 trials [29].”
BackgroundA case–control study in Taiwanese cohort and meta-analysis of serum ferritin in pancreatic cancer
Park, J. M., Mau, C.-Z., Chen, Y., Su, Y., Chen, H. A., Huang, S., Chang, J., & Chiu, C. (2021) · Scientific Reports · doi:10.1038/s41598-021-00650-7 · Cited by 33
A comparison between hyaluronic acid and other single ingredient eye drops for dry eye, a review
Hynnekleiv, L., Magnø, M. S., Moschowits, E., Tønseth, K. A., Vehof, J., & Utheim, T. P. (2023) · Acta Ophthalmologica · doi:10.1111/aos.15675 · Cited by 32
“Smaller studies, especially studies with 20 participants or fewer, are likely to be un-derpowered for the magnitude of the expected effect size (Hackshaw, 2008).”
BackgroundGenetic variations in taste perception modify alcohol drinking behavior in Koreans
Choi, J., Lee, J., Yang, S. S., & Kim, J. (2017) · Appetite · doi:10.1016/j.appet.2017.02.022 · Cited by 32
Exploring Variation in Police Perceptions of De-Escalation: Do Officer Characteristics Matter?
White, M. D., Mora, V. J., & Orosco, C. (2019) · Policing: A Journal of Policy and Practice · doi:10.1093/police/paz062 · Cited by 30
Burden and Risk Factors of Melioidosis in Southeast Asia: A Scoping Review
Selvam, K., Ganapathy, T., Najib, M. A., Khalid, M. F., Abdullah, N. A., Harun, A. B., Mohammad, W. M. Z. W., & Aziah, I. (2022) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph192315475 · Cited by 32
McMullan, C. J., Hickson, D. A., Taylor, H. A., & Forman, J. P. (2015) · Journal of Hypertension · doi:10.1097/hjh.0000000000000638 · Cited by 32
“Although the association of 24-h ABPM characteristics with renal disease has been studied in patients with pre-existing CKD [8,23–25], studies among those with normal renal function are small and retrospective [9,26,27], and none have examined African Americans – a population at increased risk of having both abnormali…”
BackgroundWood, G., Taylor, E., Ng, V., Murrell, A., Patil, A., van der Touw, T., Wolden, M., Andronicos, N., & Smart, N. A. (2023) · Sports Medicine · doi:10.1007/s40279-023-01817-0 · Cited by 31
“Studies of intervention and control group population sample sizes (N) < 10 were excluded to reduce the likelihood of small study effects, for example over-estimation of effect size [33].”
BackgroundBioavailability of Orally Administered Drugs in Critically Ill Patients
Forsberg, J., Bedard, E. L. R., & Mahmoud, S. H. (2022) · Journal of Pharmacy Practice · doi:10.1177/08971900221100205 · Cited by 31
Collins, S. E., Duncan, M. H., Smart, B. F., Saxon, A. J., Malone, D. K., Jackson, T. R., & Ries, R. K. (2014) · Substance Abuse · doi:10.1080/08897077.2014.904838 · Cited by 33
“Second, this study had a small sample size, which can produce false-positive results or overestimate the effect size of an intervention.(101) On the other hand, small studies are often underpowered to find statistically significant differences where they exist.”
MethodState of the evidence: a survey of global disparities in clinical trials
Marshall, I. J., L’Esperance, V., Marshall, I. J., Thomas, J., Noel-Storr, A. H., Soboczenski, F., Nye, B., Nenkova, A., & Wallace, B. (2021) · BMJ Global Health · doi:10.1136/bmjgh-2020-004145 · Cited by 32
Clinical Spectroscopy: Lost in Translation?
Cameron, J. M., Rinaldi, C., Rutherford, S. H., Sala, A., Theakstone, A. G., & Baker, M. J. (2021) · Applied Spectroscopy · doi:10.1177/00037028211021846 · Cited by 32
“Another common issue is that many proof-of-concept studies involve relatively small patient cohorts, and this can over-estimate the significance of the results as they are not entirely representative of larger populations.(143) Sample numbers are often a limiting factor in the development of new technologies since the…”
BackgroundNardon, M., Ruzzante, F., O'Donnell, L., Adami, A., Dayanidhi, S., & Bertucco, M. (2021) · Gait & Posture · doi:10.1016/j.gaitpost.2021.09.190 · Cited by 31
Patel, T. K., Patel, P. B., Bhalla, H. L., Dwivedi, P., Bajpai, V., & Kishore, S. (2022) · European Journal of Clinical Pharmacology · doi:10.1007/s00228-022-03419-7 · Cited by 29
Gorman, E. A., Rynne, J., Gardiner, H. J., Rostron, A., Bannard‐Smith, J., Bentley, A. M., Brealey, D. A., Campbell, C., Curley, G. F., Clarke, M. J., et al. (2023) · American Journal of Respiratory and Critical Care Medicine · doi:10.1164/rccm.202302-0297oc · Cited by 30
Lung Cancer and Self-Management Interventions: A Systematic Review of Randomised Controlled Trials
Rowntree, R. A., & Hosseinzadeh, H. (2022) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph19010536 · Cited by 29
“Small sample sizes increase the risk of distorting the results [47].”
BackgroundGrindey, A., & Bradshaw, T. (2022) · International Journal of Mental Health Nursing · doi:10.1111/inm.12992 · Cited by 32
Incognito, O., Laura, S., & Pinto, G. (2021) · International Journal of Music Education · doi:10.1177/02557614211027247 · Cited by 29
“As suggested by Hackshaw (2008), in these cases it is necessary to interpret the results carefully.”
BackgroundKannan, P., Hsu, W. H., Suen, W. T., Chan, L. M., Assor, A., & Ho, C. M. (2021) · Complementary Therapies in Clinical Practice · doi:10.1016/j.ctcp.2021.101502 · Cited by 52
Xu, Y., Zhang, Y., Zhao, D., Tian, Y., & Yuan, T. (2023) · Nature Mental Health · doi:10.1038/s44220-023-00118-9 · Cited by 31
Lipid Oxidation Assessed by Indirect Calorimetry Predicts Metabolic Syndrome and Type 2 Diabetes
Pujia, A., Mazza, E., Ferro, Y., Gazzaruso, C., Coppola, A., Doldo, P., Grembiale, R. D., Pujia, R., Romeo, S., & Montalcini, T. (2019) · Frontiers in Endocrinology · doi:10.3389/fendo.2018.00806 · Cited by 28
Patole, S., Pawale, D., & Rath, C. P. (2024) · Healthcare · doi:10.3390/healthcare12020171 · Cited by 30
Almaraz, D., Galdós, J. S., Martín, F. M., Sánchez‐Iglesias, I., Fernández, A. J. M., & Goldsby, T. L. (2022) · Healthcare · doi:10.3390/healthcare10081447 · Cited by 31
“As stated by Hackshaw [22], studies with small samples may produce false-positive results or overestimate the magnitude of an association.”
BackgroundYip, A. L. K., Karatzias, T., & Chien, W. T. (2022) · Annals of Medicine · doi:10.1080/07853890.2022.2108551 · Cited by 24
“Sample sizes of under 20 were suggested as small in this study [24].”
BackgroundNugier, A., Roebroeck, É., Anier, N., Kleinlogel, E. P., Chatard, A., & Guimond, S. (2016) · International Review of Social Psychology · doi:10.5334/irsp.61 · Cited by 25
“However, because small studies need to be interpreted carefully, we recognize the necessity to replicate these results in the future (Hackshaw, 2008).”
BackgroundThe effects of meditation on individuals facing loneliness: a scoping review
Saini, G. K., Haseeb, S. B., Taghi-Zada, Z., & Ng, J. Y. (2021) · BMC Psychology · doi:10.1186/s40359-021-00585-8 · Cited by 28
“Of the studies with an RCT design, all contained small sample sizes, ranging from 22 to 323 participants, with the definition of small sample size depending upon the individual objectives of each study [34].”
BackgroundLeung, P., Orgeta, V., & Orrell, M. W. (2017) · International Journal of Geriatric Psychiatry · doi:10.1002/gps.4654 · Cited by 27
“The results may be misinterpreted or fail to produce reliable outcomes (Hackshaw, 2008).”
BackgroundDixon, D. L., Khaddage, S., Bhagat, S., Koenig, R. A., Salgado, T. M., & Baker, W. L. (2020) · Journal of clinical lipidology · doi:10.1016/j.jacl.2020.04.004 · Cited by 24
Nieuwmeijer, C., MARSHALL, N. A., & van Oers, B. (2021) · Research Papers in Education · doi:10.1080/02671522.2021.1998207 · Cited by 24
“for assigning significant levels of resources (Hackshaw 2008).”
BackgroundPerceptions of fishermen towards marine protected areas in Cambodia and the Philippines
Hamilton, M. (2012) · Bioscience Horizons The International Journal of Student Research · doi:10.1093/biohorizons/hzs007 · Cited by 24
Rash, J. A., Poulin, P. A., Shergill, Y., Romanow, H. C., Freeman, J., Taljaard, M., Hebert, G., Stiell, I. G., & Smyth, C. E. (2018) · Pain Research and Management · doi:10.1155/2018/1875967 · Cited by 28
“Small samples can produce false-positive results or overestimate the magnitude of eHect [79].”
BackgroundEmpirical Investigation of Key Factors for SaaS Architecture
Aleem, S., Ahmed, F., Batool, R., & Khattak, A. M. (2019) · IEEE Transactions on Cloud Computing · doi:10.1109/tcc.2019.2906299 · Cited by 26
BUILDING TRUST IN WAQF MANAGEMENT — IMPLICATIONS OF GOOD GOVERNANCE AND TRANSPARENT REPORTING
Hasan, R., Ahmad, A. U. F., & Siraj, S. A. (2020) · The Singapore Economic Review · doi:10.1142/s0217590820420059 · Cited by 23
“The sample size is sufficient in the context of the study as Hackshaw (2008) recommended testing a new research hypothesis in a small number of the subject.”
BackgroundBawa, P. (2020) · International Journal of Game-Based Learning · doi:10.4018/ijgbl.2020070102 · Cited by 29
Giordano, V., Boni, G., Godoy‐Santos, A. L., Pires, R. E. S., Fukuyama, J. M., Koch, H. A., & Giannoudis, P. V. (2020) · European Journal of Trauma and Emergency Surgery · doi:10.1007/s00068-020-01337-w · Cited by 23
Metadata
- Authors
- Hackshaw, A.
- Affiliations
- Cancer Research UK (GB)
- Corresponding author
- Hackshaw, A.
- ORCID
- Hackshaw, A.
- Venue
- European Respiratory Journal
- Publisher
- European Respiratory Society (ERS)
- Published
- 2008-10-31
- Volume
- 32
- Issue
- 5
- Pages
- 1141-1143
- ISSN
- 0903-1936 (ISSN-L)
- 1399-3003
- OpenAlex
- W2097412619
- Cited by
- 496
- References
- 7
- Citations by year
- 2026: 29 · 2025: 36 · 2024: 37 · 2023: 56 · 2022: 57 · 2021: 86 · 2020: 51 · 2019: 38 · 2018: 24 · 2017: 20 · 2016: 20 · 2015: 17 · 2014: 9 · 2013: 5 · 2012: 5
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- European Respiratory Journal (publishedVersion)
- PubMed (publishedVersion)
- UCL Discovery (University College London) (submittedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Keywords
- small sample size, sample size determination, confidence intervals, statistical power, effect size, clinical research design, research limitations, prevalence estimation
- MeSH terms
- Antineoplastic Agents, Clinical Trials as Topic, Data Interpretation, Statistical, Diet Therapy, Humans, Lung Neoplasms, Research Design, Risk Factors, Smoking, Thalidomide, Time Factors, Reproducibility of Results, Weight Loss, Sample Size, Small Cell Lung Carcinoma
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
