CoChat
Sign Up
Cited by 496Statistical Methods in EpidemiologyStatistical Methods in Clinical TrialsMeta-analysis and systematic reviews

Small studies: strengths and limitations

Open access
Hackshaw, A.·2008·European Respiratory Journal
DOI 10.1183/09031936.00136408
Open this paper in CoChat — free accountOpen PDF

Abstract

A large number of clinical research studies are conducted, including audits of patient data, observational studies, clinical trials and those based on laboratory analyses. While small studies can be published over a short time-frame, there needs to be a balance between those that can be performed quickly and those that should be based on more subjects and hence may take several years to complete. The present article provides an overview of the main considerations associated with small studies. The definition of “small” depends on the main study objective. When simply describing the characteristics of a single group of subjects, for example the prevalence of smoking, the larger the study the more reliable the results. The main results should have 95% confidence intervals (CI), and the width of these depend directly on the sample size: large studies produce narrow intervals and, therefore, more precise results. A study of 20 subjects, for example, is likely to be too small for most investigations. For example, imagine that the proportion of smokers among a particular group of 20 individuals is 25%. The associated 95% CI is 9–49. This means that the true prevalence in these subjects generally is anywhere between a low or high value, which is not a useful result. When comparing characteristics between two or more groups of subjects ( e.g. examining risk factors or treatments for disease), the size of the study depends on the magnitude of the expected effect size, which is usually quantified by a relative risk, odds ratio, absolute risk difference, hazard ratio, or difference between two means or medians. The smaller the true-effect size, the larger the study needs to be 1, 2. This is because it is more difficult to distinguish between a real effect and random variation. Consider mortality as the end-point in a …

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1183/09031936.00136408

Cites

6 of 6 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Studies with intervention and control group sample sizes less than 10 were excluded to reduce the likelihood of small study effects, such as over-estimation of effect size.

    “Studies of intervention and control group population sample sizes (N) < 10 were excluded to reduce the likelihood of small study effects, for example over-estimation of effect size [33].”

    From Estimating the Effect of Aerobic Exercise Training on Novel Lipid Biomarkers: A Systematic Review and Multivariate Meta-Analysis of Randomized Controlled Trials (2023) · cites this paper as [33]

    The LIMITATIONS section says small studies can overestimate the magnitude of an association, supporting the stated reason for exclusion. The HOW SMALL IS 'SMALL'? section says appropriate sample size depends on the expected effect; it does not establish fewer than 10 participants per group as a cutoff. (LLM, conf=0.87)

    Experimental — accuracy not yet measured

  • Only 30.7% of women are on Ibex-35 companies' Board of Directors.

    “This fact was confirmed in 2021 by the European Institute for Gender Equality [67], which shows only 30.7% of women on Ibex-35 companies' Board of Directors.”

    From Pilot study of an online brief mindfulness-based intervention to improve the well-being of managers and reduce their stress at work. (2024) · cites this paper as [67]

    The paper’s ‘How small is “small”?’ and ‘Limitations’ sections discuss sample size and interpretation of study results, not corporate boards. Searches for Ibex-35, the 30.7% figure, and the European Institute for Gender Equality found no relevant passage in our copy, so it cannot verify the claim. (LLM, conf=0.78)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

1 of the 7 references couldn’t be matched to a known paper.

Cited by

Sorted by citations

Related

Authors
Hackshaw, A.
Affiliations
Cancer Research UK (GB)
Corresponding author
Hackshaw, A.
ORCID
Hackshaw, A.
Venue
European Respiratory Journal
Publisher
European Respiratory Society (ERS)
Published
2008-10-31
Volume
32
Issue
5
Pages
1141-1143
ISSN
  • 0903-1936 (ISSN-L)
  • 1399-3003
DOI
10.1183/09031936.00136408
OpenAlex
W2097412619
Cited by
496
References
7
Citations by year
2026: 29 · 2025: 36 · 2024: 37 · 2023: 56 · 2022: 57 · 2021: 86 · 2020: 51 · 2019: 38 · 2018: 24 · 2017: 20 · 2016: 20 · 2015: 17 · 2014: 9 · 2013: 5 · 2012: 5
Type
journal article
Language
en
Open access
Locations
  • European Respiratory Journal (publishedVersion)
  • PubMed (publishedVersion)
  • UCL Discovery (University College London) (submittedVersion)
Topics
  • Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
  • Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
Keywords
small sample size, sample size determination, confidence intervals, statistical power, effect size, clinical research design, research limitations, prevalence estimation
MeSH terms
Antineoplastic Agents, Clinical Trials as Topic, Data Interpretation, Statistical, Diet Therapy, Humans, Lung Neoplasms, Research Design, Risk Factors, Smoking, Thalidomide, Time Factors, Reproducibility of Results, Weight Loss, Sample Size, Small Cell Lung Carcinoma

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review