Number needed to treat (NNT) in clinical literature: an appraisal
Open accessAbstract
BACKGROUND: The number needed to treat (NNT) is an absolute effect measure that has been used to assess beneficial and harmful effects of medical interventions. Several methods can be used to calculate NNTs, and they should be applied depending on the different study characteristics, such as the design and type of variable used to measure outcomes. Whether or not the most recommended methods have been applied to calculate NNTs in studies published in the medical literature is yet to be determined. The aim of this study is to assess whether the methods used to calculate NNTs in studies published in medical journals are in line with basic methodological recommendations. METHODS: The top 25 high-impact factor journals in the "General and/or Internal Medicine" category were screened to identify studies assessing pharmacological interventions and reporting NNTs. Studies were categorized according to their design and the type of variables. NNTs were assessed for completeness (baseline risk, time horizon, and confidence intervals [CIs]). The methods used for calculating NNTs in selected studies were compared to basic methodological recommendations published in the literature. Data were analyzed using descriptive statistics. RESULTS: The search returned 138 citations, of which 51 were selected. Most were meta-analyses (n = 23, 45.1%), followed by clinical trials (n = 17, 33.3%), cohort (n = 9, 17.6%), and case-control studies (n = 2, 3.9%). Binary variables were more common (n = 41, 80.4%) than time-to-event (n = 10, 19.6%) outcomes. Twenty-six studies (51.0%) reported only NNT to benefit (NNTB), 14 (27.5%) reported both NNTB and NNT to harm (NNTH), and 11 (21.6%) reported only NNTH. Baseline risk (n = 37, 72.5%), time horizon (n = 38, 74.5%), and CI (n = 32, 62.7%) for NNTs were not always reported. Basic methodological recommendations to calculate NNTs were not followed in 15 studies (29.4%). The proportion of studies applying non-recommended methods was particularly high for meta-analyses (n = 13, 56.5%). CONCLUSIONS: A considerable proportion of studies, particularly meta-analyses, applied methods that are not in line with basic methodological recommendations. Despite their usefulness in assisting clinical decisions, NNTs are uninterpretable if incompletely reported, and they may be misleading if calculating methods are inadequate to study designs and variables under evaluation. Further research is needed to confirm the present findings.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Moher, D., Hopewell, S., Schulz, K. F., Montori, V., Gotzsche, P. C., Devereaux, P. J., Elbourne, D., Egger, M., & Altman, D. G. (2010) · BMJ · doi:10.1136/bmj.c869 · Cited by 8,883 · see citing works
The Revised CONSORT Statement for Reporting Randomized Trials: Explanation and Elaboration
Altman, D. G., Schulz, K. F., Moher, D., Egger, M., Davidoff, F., Elbourne, D., Gøtzsche, P. C., Lang, T., & for the CONSORT Group (2001) · Annals of Internal Medicine · doi:10.7326/0003-4819-134-8-200104170-00012 · Cited by 3,835 · see citing works
Chapter 9: Analysing Data and Undertaking Meta-Analyses
Deeks, J. J., Higgins, J. P. T., & Altman, D. G. (2008) · Cited by 2,782 · see citing works
The number needed to treat: a clinically useful measure of treatment effect
Cook, R. J., & Sackett, D. L. (1995) · BMJ · doi:10.1136/bmj.310.6977.452 · Cited by 1,885 · see citing works
An Assessment of Clinically Useful Measures of the Consequences of Treatment
Laupacis, A., Sackett, D. L., & Roberts, R. S. (1988) · New England Journal of Medicine · doi:10.1056/nejm198806303182605 · Cited by 1,681 · see citing works
The Medical Dictionary for Regulatory Activities (MedDRA)
Brown, E. G., Wood, L., & Wood, S. (1999) · Drug Safety · doi:10.2165/00002018-199920020-00002 · Cited by 1,396 · see citing works
Evidence-based medicine : how to practice and teach EBM
Straus, S. E. (2005) · Cited by 1,378 · see citing works
Confidence intervals for the number needed to treat.
Altman, D. G. (1998) · BMJ · doi:10.1136/bmj.317.7168.1309 · Cited by 938 · see citing works
Calculating the number needed to treat for trials where the outcome is time to an event
Altman, D. G., & Andersen, P. K. (1999) · BMJ · doi:10.1136/bmj.319.7223.1492 · Cited by 710 · see citing works
Graham, D. J., Ouellet‐Hellstrom, R., MaCurdy, T. E., Ali, F. N., Sholley, C., Worrall, C. M., & Kelman, J. A. (2010) · JAMA · doi:10.1001/jama.2010.920 · Cited by 512 · see citing works
Using Numerical Results from Systematic Reviews in Clinical Practice
McQuay, H. J., & Moore, R. A. (1997) · Annals of Internal Medicine · doi:10.7326/0003-4819-126-9-199705010-00007 · Cited by 452 · see citing works
Armitage, P. 1.-2. (2005) · Wiley eBooks · doi:10.1002/0470011815 · Cited by 409 · see citing works
Furukawa, T. A., Guyatt, G. H., & Griffith, L. E. (2002) · International Journal of Epidemiology · doi:10.1093/ije/31.1.72 · Cited by 318 · see citing works
Numbers needed to treat derived from meta-analyses---sometimes informative, usually misleading
Smeeth, L. L., Haines, A., & Ebrahim, S. B. J. (1999) · BMJ · doi:10.1136/bmj.318.7197.1548 · Cited by 301 · see citing works
Simpson's paradox and calculation of number needed to treat from meta-analysis
Cates, C. J. (2002) · BMC Medical Research Methodology · doi:10.1186/1471-2288-2-1 · Cited by 251 · see citing works
Citrome, L., & Ketter, T. A. (2013) · International Journal of Clinical Practice · doi:10.1111/ijcp.12142 · Cited by 251 · see citing works
Reporting Number Needed to Treat and Absolute Risk Reduction in Randomized Controlled Trials
Nuovo, J. (2002) · JAMA · doi:10.1001/jama.287.21.2813 · Cited by 241 · see citing works
Schmid, C. H., Lau, J., McIntosh, M. W., & Cappelleri, J. C. (1998) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19980915)17:17 · Cited by 210 · see citing works
Meta-analysis, Simpson's paradox, and the number needed to treat
Altman, D. G., & Deeks, J. J. (2002) · BMC Medical Research Methodology · doi:10.1186/1471-2288-2-3 · Cited by 159 · see citing works
Austin, P. C. (2009) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2009.03.012 · Cited by 145 · see citing works
da Costa, B. R., Rutjes, A. W., Johnston, B. C., Reichenbach, S., Nüesch, E., Tonia, T., Gemperli, A., Guyatt, G. H., & Jüni, P. (2012) · International Journal of Epidemiology · doi:10.1093/ije/dys124 · Cited by 145 · see citing works
Bender, R., & Blettner, M. (2002) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(01)00510-8 · Cited by 130 · see citing works
Mt‐Isa, S., Hallgreen, C. E., Wang, N., Callréus, T., Genov, G., Hirsch, I., Hobbiger, S. F., Hockley, K. S., Luciani, D., Phillips, L. D., et al. (2014) · Pharmacoepidemiology and Drug Safety · doi:10.1002/pds.3636 · Cited by 130 · see citing works
Number Needed to Treat: Properties and Problems
Hutton, J. L. (2000) · Journal of the Royal Statistical Society Series A (Statistics in Society) · doi:10.1111/1467-985x.00175 · Cited by 112 · see citing works
The "number needed to treat" turns 20 -- and continues to be used and misused
McAlister, F. A. (2008) · Canadian Medical Association Journal · doi:10.1503/cmaj.080484 · Cited by 108 · see citing works
Austin, P. C., & Laupacis, A. (2011) · The International Journal of Biostatistics · doi:10.2202/1557-4679.1285 · Cited by 94 · see citing works
Common problems related to the use of number needed to treat
Stang, A., Poole, C., & Bender, R. (2009) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2009.08.006 · Cited by 80 · see citing works
Number needed to treat in COPD: exacerbations versus pneumonias
Suissa, S. (2013) · Thorax · doi:10.1136/thoraxjnl-2012-202709 · Cited by 77 · see citing works
Relative vs. absolute measures of benefit and risk: what’s the difference?
Citrome, L. (2009) · Acta Psychiatrica Scandinavica · doi:10.1111/j.1600-0447.2009.01449.x · Cited by 74 · see citing works
Number needed to treat is incorrect without proper time-related considerations
Suissa, D., Brassard, P., Smiechowski, B., & Suissa, S. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2011.04.009 · Cited by 73 · see citing works
Shepherd, J., Kastelein, J. J. P., Bittner, V., Carmena, R., DEEDWANIA, P. C., Breazna, A., Dobson, S., Wilson, D. J., Zuckerman, A. L., & WENGER, N. K. (2008) · Mayo Clinic Proceedings · doi:10.4065/83.8.870 · Cited by 71 · see citing works
Systematic reviews experience major limitations in reporting absolute effects
Alonso-Coello, P., Carrasco-Labra, A., Brignardello-Petersen, R., Neumann, I., Akl, E. A., Vernooij, R. W. M., Johnston, B. C., Sun, X., Briel, M., Busse, J. W., et al. (2016) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2015.11.002 · Cited by 70 · see citing works
Estimating adjusted NNT measures in logistic regression analysis
Bender, R., Kuß, O., Hildebrandt, M., & Gehrmann, U. (2007) · Statistics in Medicine · doi:10.1002/sim.3061 · Cited by 67 · see citing works
Lubsen, J. C., Hoes, A. W., & Grobbee, D. E. (2000) · The Lancet · doi:10.1016/s0140-6736(00)03215-3 · Cited by 53 · see citing works
Pooling data for Number Needed to Treat: no problems for apples
Moore, R. A., Gavaghan, D. J., Edwards, J. E., Wiffen, P. J., & McQuay, H. J. (2002) · BMC Medical Research Methodology · doi:10.1186/1471-2288-2-2 · Cited by 51 · see citing works
Communicating risks at the population level: application of population impact numbers
Heller, R. F., Buchan, I. E., Edwards, R., Lyratzopoulos, G., McElduff, P., & Leger, S. S. (2003) · BMJ · doi:10.1136/bmj.327.7424.1162 · Cited by 51 · see citing works
Heller, R. F., Dobson, A. J., Attia, J., & Page, J. (2002) · Journal of Epidemiology & Community Health · doi:10.1136/jech.56.8.606 · Cited by 51 · see citing works
Calculation of NNTs in RCTs with time-to-event outcomes: A literature review
Hildebrandt, M., Vervölgyi, E., & Bender, R. (2009) · BMC Medical Research Methodology · doi:10.1186/1471-2288-9-21 · Cited by 46 · see citing works
The present efficacy of multiple sclerosis therapeutics: Is the new 66% just the old 33%?
Klawiter, E. C., Cross, A. H., & Naismith, R. T. (2009) · PubMed · doi:10.1212/wnl.0b013e3181b9c8f7 · Cited by 41 · see citing works
Mayne, T. J., Whalen, E., & Vu, A. (2005) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2005.07.006 · Cited by 40 · see citing works
Calculation of Number Needed to Treat
Suissa, S. (2009) · New England Journal of Medicine · doi:10.1056/nejmc0903274 · Cited by 40 · see citing works
Implementing guidelines in primary care: can population impact measures help?
Heller, R. F., Edwards, R., & McElduff, P. (2003) · BMC Public Health · doi:10.1186/1471-2458-3-7 · Cited by 38 · see citing works
Laubender, R. P., & Bender, R. (2010) · Statistics in Medicine · doi:10.1002/sim.3793 · Cited by 32 · see citing works
The Number Needed to Treat: 25 Years of Trials and Tribulations in Clinical Research
Suissa, S. (2015) · Rambam Maimonides Medical Journal · doi:10.5041/rmmj.10218 · Cited by 28 · see citing works
Data sources on drug safety evaluation: a review of recent published meta‐analyses
Alves, C., Marques, F. B., & Macedo, A. F. (2011) · Pharmacoepidemiology and Drug Safety · doi:10.1002/pds.2260 · Cited by 25 · see citing works
Self-citationNumber Needed to Treat ( NNT )
Bender, R. (2005) · Encyclopedia of Biostatistics · doi:10.1002/0470011815.b2a04032 · Cited by 23 · see citing works
Mendes, D., Alves, C., & Marques, F. B. (2016) · CNS Drugs · doi:10.1007/s40263-016-0377-9 · Cited by 23 · see citing works
Self-citationImpact numbers in health policy decisions
Attia, J., Page, J., Heller, R. F., & Dobson, A. J. (2002) · Journal of Epidemiology & Community Health · doi:10.1136/jech.56.8.600 · Cited by 22 · see citing works
Mendes, D., Alves, C., & Marques, F. B. (2015) · Pharmacoepidemiology and Drug Safety · doi:10.1002/pds.3874 · Cited by 20 · see citing works
Self-citationNumbers needed to treat derived from meta-analysis: a word of caution.
Peterson, D. (2003) · BMJ Evidence-Based Medicine · doi:10.1136/ebm.8.4.104 · Cited by 18 · see citing works
Mendes, D., Alves, C., & Marques, F. B. (2016) · Expert Opinion on Drug Safety · doi:10.1080/14740338.2016.1217989 · Cited by 16 · see citing works
Self-citationNumbers needed to treat derived from meta-analysis: A word of caution
Arthur Marx, M., & Heiner C. Bucher, M. (2003) · ACP Journal Club · doi:10.7326/acpjc-2003-138-2-a11 · Cited by 16 · see citing works
Bender, R., Kromp, M., Kiefer, C., & Sturtz, S. (2013) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2013.01.011 · Cited by 12 · see citing works
Laubender, R. P., & Bender, R. (2013) · Statistics in Medicine · doi:10.1002/sim.5913 · Cited by 7 · see citing works
Girerd, N., Rabilloud, M., Duarte, K., et al. (2013) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2013.08.010 · Cited by 5 · see citing works
Estimation of numbers needed to treat should be based on absolute risks
Bender, R., Kromp, M., Kiefer, C., et al. (2013) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2013.08.009 · Cited by 2 · see citing works
Flagged first, then by citations
Research works printed but not linked: 2
- [11]BMJ. For Authors, Resources for Authors, Article Types, Research. bmj.com. http://www.bmj.com/about-bmj/resources-authors/article-types/research. Accessed 13 Oct 2016.
- [37]Thomson-Reuters. Journal citation reports. http://thomsonreuters.com/en/products-services/scholarly-scientific-research/research-management-and-evaluation/journal-citation-reports.html. Accessed 24 Aug 2016.
Cited by
Sorted by citations
Safety planning-type interventions for suicide prevention: meta-analysis
Nuij, C., van Ballegooijen, W., de Beurs, D., Juniar, D., Erlangsen, A., Portzky, G., O'Connor, R. C., Smit, J. H., Kerkhof, A., & Riper, H. (2021) · The British Journal of Psychiatry · doi:10.1192/bjp.2021.50 · Cited by 270 · see citing works
“21 To account for differences between study populations, interventions and control conditions, we performed a random-effects meta-analysis.”
MethodA literature review and meta-analysis on the effects of ADHD medications on functional outcomes
Boland, H., DiSalvo, M., Fried, R., Woodworth, K. Y., Wilens, T., Faraone, S. V., & Biederman, J. (2020) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2020.01.006 · Cited by 240 · see citing works
Vázquez, G. H., Bahji, A., Undurraga, J., Tondo, L., & Baldessarini, R. J. (2021) · Journal of Psychopharmacology · doi:10.1177/02698811211013579 · Cited by 95 · see citing works
Kadakia, A., Dembek, C., Heller, V., Singh, R., Uyei, J., Hagi, K., Nosaka, T., & Loebel, A. (2021) · BMC Psychiatry · doi:10.1186/s12888-021-03220-3 · Cited by 79 · see citing works
- Authors
- Mendes, D., Alves, C., & Batel-Marques, F.
- Affiliations
- Association for Innovation and Biomedical Research on Light and Image (PT)
- Corresponding author
- Mendes, Diogo
- Venue
- BMC Medicine
- Published
- 2017-06-01
- Volume
- 15
- Issue
- 1
- Pages
- 112-112
- ISSN
- 1741-7015 (ISSN-L)
- PMC
- PMC5455127
- OpenAlex
- W2621228814
- Cited by
- 169
- References
- 58
- Citations by year
- 2026: 9 · 2025: 31 · 2024: 21 · 2023: 27 · 2022: 15 · 2021: 24 · 2020: 19 · 2019: 19 · 2018: 3 · 2017: 1
- FWCI
- 10.27
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- BMC Medicine (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, other-oa)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Portuguese National Funding Agency for Science, Research and Technology (RCAAP Project by FCT) (submittedVersion, cc-by)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Keywords
- number needed to treat, reporting quality, methodological recommendations, meta-analysis, clinical trials, pharmacological interventions, baseline risk, time horizon, confidence intervals, medical journals, reproducibility, time-to-event outcomes, binary outcomes
- MeSH terms
- Humans, Periodicals as Topic, Journal Impact Factor, Numbers Needed To Treat
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
