The DOI matches its Crossref record; 15 of the 15 references it cites were found in a registry or an index.
The statistical significance of randomized controlled trial results is frequently fragile: a case for a Fragility Index
Open accessAbstract
OBJECTIVES: A P-value <0.05 is one metric used to evaluate the results of a randomized controlled trial (RCT). We wondered how often statistically significant results in RCTs may be lost with small changes in the numbers of outcomes. STUDY DESIGN AND SETTING: A review of RCTs in high-impact medical journals that reported a statistically significant result for at least one dichotomous or time-to-event outcome in the abstract. In the group with the smallest number of events, we changed the status of patients without an event to an event until the P-value exceeded 0.05. We labeled this number the Fragility Index; smaller numbers indicated a more fragile result. RESULTS: The 399 eligible trials had a median sample size of 682 patients (range: 15-112,604) and a median of 112 events (range: 8-5,142); 53% reported a P-value <0.01. The median Fragility Index was 8 (range: 0-109); 25% had a Fragility Index of 3 or less. In 53% of trials, the Fragility Index was less than the number of patients lost to follow-up. CONCLUSION: The statistically significant results of many RCTs hinge on small numbers of events. The Fragility Index complements the P-value and helps identify less robust results.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
On the psychology of prediction.
Kahneman, D., & Tversky, A. (1973) · Psychological Review · doi:10.1037/h0034747 · Cited by 6,202 · cited 1×
“Work in cognitive psychology demonstrated that even experts in probability are intuitively poor in determining the relevance of these issues in the experimental design [2].”
BackgroundBeta blockade during and after myocardial infarction: An overview of the randomized trials
Yusuf, S., Peto, R., Lewis, J., Collins, R., & Sleight, P. (1985) · Progress in Cardiovascular Diseases · doi:10.1016/s0033-0620(85)80003-7 · Cited by 3,205 · cited 1×
“[12] suggested trials of at least 650 events were required to be sufficiently confident that true effects were identified in cardiovascular trials, and this was reinforced by simulation data [13].”
BackgroundSterne, J. A. C. (2001) · BMJ · doi:10.1136/bmj.322.7280.226 · Cited by 1,466
Contradicted and Initially Stronger Effects in Highly Cited Clinical Research
Ioannidis, J. P. A. (2005) · JAMA · doi:10.1001/jama.294.2.218 · Cited by 1,465
Toward Evidence-Based Medical Statistics. 1: The P Value Fallacy
Goodman, S. N. (1999) · Annals of Internal Medicine · doi:10.7326/0003-4819-130-12-199906150-00008 · Cited by 1,185
Collins, R. T., Peto, R., Flather, M. D., Parish, S., Sleight, P., Conway, M. A., Pipilis, A., Baigent, C., Barnett, D., Boissel, J., et al. (1995) · Ghent University Academic Bibliography (Ghent University) · Cited by 1,115
Woods, K. L., Fletcher, S., Roffe, C., & Haider, Y. (1992) · The Lancet · doi:10.1016/0140-6736(92)91828-v · Cited by 537
Controversy in Counting and Attributing Events in Clinical Trials
Sackett, D. L., & Gent, M. (1979) · New England Journal of Medicine · doi:10.1056/nejm197912273012602 · Cited by 413
Statistical significance tests.
Cox, D. (1982) · British Journal of Clinical Pharmacology · doi:10.1111/j.1365-2125.1982.tb01987.x · Cited by 334
Cited by
Kristensen, S. L., Rørth, R., Jhund, P. S., Docherty, K. F., Sattar, N., Preiss, D., Køber, L., Petrie, M. C., & McMurray, J. J. V. (2019) · The Lancet Diabetes & Endocrinology · doi:10.1016/s2213-8587(19)30249-9 · Cited by 1,663
Homocysteine lowering interventions for preventing cardiovascular events
Martí-Carvajal, A. J., Solà, I., Lathyris, D., & Salanti, G. (2009) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd006612.pub2 · Cited by 520
Borges, F. K., Bhandari, M., Guerra-Farfan, E., Patel, A., Sigamani, A., Umer, M., Tiboni, M. E., Villar-Casares, M. D. M., Tandon, V., Tomas-Hernandez, J., et al. (2020) · The Lancet · doi:10.1016/s0140-6736(20)30058-1 · Cited by 370
Self-citationStrategies to improve retention in randomised trials
Brueton, V. C., Tierney, J., Stenning, S., Harding, S., Meredith, S., Nazareth, I., & Rait, G. (2013) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000032.pub2 · Cited by 279
Abd El Aziz, M., Cahyadi, O., Meier, J. J., Schmidt, W. E., & Nauck, M. A. (2020) · Diabetes, Obesity and Metabolism · doi:10.1111/dom.13924 · Cited by 160
Metadata
- Authors
- Walsh, M., Srinathan, S. K., McAuley, D. F., Mrkobrada, M., Levine, O., Ribic, C., Molnar, A. O., Dattani, N. D., Burke, A., Guyatt, G., Thabane, L., Walter, S. D., Pogue, J., & Devereaux, P. J.
- Venue
- Journal of Clinical Epidemiology
- Publisher
- Elsevier BV
- Published
- 2014-06
- Volume
- 67
- Issue
- 6
- Pages
- 622-628
- ISSN
- 0895-4356 (ISSN-L)
- OpenAlex
- W2164957370
- Cited by
- 762
- References
- 15
- FWCI
- 26.81
- Type
- journal article
- Language
- en
- Open access
- hybrid
- Licence
- www.elsevier.com/tdm/userlicense/1.0/ (tdm)
- creativecommons.org/licenses/by-nc-nd/3.0/ (vor)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: The statistical significance of randomized controlled trial results is frequently fragile: a case for a Fragility Index. https://app.cochat.ai/papers/10.1016/j.jclinepi.2013.10.019
- The paper
- Walsh, M., Srinathan, S. K., McAuley, D. F., Mrkobrada, M., Levine, O., Ribic, C., Molnar, A. O., Dattani, N. D., Burke, A., Guyatt, G., Thabane, L., Walter, S. D., Pogue, J., & Devereaux, P. J. (2014). The statistical significance of randomized controlled trial results is frequently fragile: a case for a Fragility Index. Journal of Clinical Epidemiology. https://doi.org/10.1016/j.jclinepi.2013.10.019
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.