The DOI matches its Crossref record; 11 of the 11 references it cites were found in a registry or an index.
Likelihood of Null Effects of Large NHLBI Clinical Trials Has Increased over Time
Open accessAbstract
BACKGROUND: We explore whether the number of null results in large National Heart Lung, and Blood Institute (NHLBI) funded trials has increased over time. METHODS: We identified all large NHLBI supported RCTs between 1970 and 2012 evaluating drugs or dietary supplements for the treatment or prevention of cardiovascular disease. Trials were included if direct costs >$500,000/year, participants were adult humans, and the primary outcome was cardiovascular risk, disease or death. The 55 trials meeting these criteria were coded for whether they were published prior to or after the year 2000, whether they registered in clinicaltrials.gov prior to publication, used active or placebo comparator, and whether or not the trial had industry co-sponsorship. We tabulated whether the study reported a positive, negative, or null result on the primary outcome variable and for total mortality. RESULTS: 17 of 30 studies (57%) published prior to 2000 showed a significant benefit of intervention on the primary outcome in comparison to only 2 among the 25 (8%) trials published after 2000 (χ2=12.2,df= 1, p=0.0005). There has been no change in the proportion of trials that compared treatment to placebo versus active comparator. Industry co-sponsorship was unrelated to the probability of reporting a significant benefit. Pre-registration in clinical trials.gov was strongly associated with the trend toward null findings. CONCLUSIONS: The number NHLBI trials reporting positive results declined after the year 2000. Prospective declaration of outcomes in RCTs, and the adoption of transparent reporting standards, as required by clinicaltrials.gov, may have contributed to the trend toward null findings.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures


Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Registering a protocol before the start of the experiment enables researchers to demonstrate that the hypothesis, approach, and analysis were planned in advance and not shaped by data as they emerged; it enhances scientific rigour and protects the researcher against concerns about selective reporting of results [239,240].”
From Reporting animal research: Explanation and elaboration for the ARRIVE guidelines 2.0 (2020) · cites this paper as [239]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Preregistration is required for all government-funded medical clinical trials in the United States (Food and Drug Administration, 2007; see also Kaplan & Irvin, 2015).”
From Do growth mindset interventions impact students’ academic achievement? A systematic review and meta-analysis with recommendations for best practices. (2023) · cites this paper as Kaplan & Irvin, 2015
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Effect estimates may vary according to whether or not trials are registered (Kaplan 2015).”
From Probiotics for treating acute infectious diarrhoea (2020) · cites this paper as Kaplan 2015
The cited source says what the claim attributes to it.
In the citing paper
“Registering a protocol before the start of the experiment enables researchers to demonstrate that the hypothesis, approach, and analysis were planned in advance and not shaped by data as they emerged; it enhances scientific rigour and protects the researcher against concerns about selective reporting of results [239,240].”
From Reporting animal research: Explanation and elaboration for the ARRIVE guidelines 2.0 (2020) · cites this paper as [239]
In the citing paper
“The proportion of large, National Heart, Lung, and Blood Institute studies reporting null findings over time has increased, which is suggestive of decreases in publication bias (68).”
From Issues with data and analyses: Errors, underlying themes, and potential solutions (2018) · cites this paper as [68]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Publication bias in clinical research
Easterbrook, P. J., Gopalan, R., Berlin, J. A., & Matthews, D. R. (1991) · The Lancet · doi:10.1016/0140-6736(91)90201-y · Cited by 3,109 · cited 1×
“It is widely noted that journals favor publication of statistically significant findings[2].”
BackgroundMoher, D. (2001) · JAMA · doi:10.1001/jama.285.15.1987 · Cited by 2,545
Alzheimer’s disease drug-development pipeline: Few candidates, frequent failures
Cummings, J. L., Morstorf, T., & Zhong, K. (2014) · Alzheimer's Research & Therapy · doi:10.1186/alzrt269 · Cited by 1,744
The ClinicalTrials.gov Results Database — Update and Key Issues
Zarin, D. A., Tse, T., Williams, R. J., Califf, R. M., & Ide, N. C. (2011) · New England Journal of Medicine · doi:10.1056/nejmsa1012065 · Cited by 838 · cited 1×
“Registered studies must provide: the study’s purpose, recruitment status, design, eligibility criteria, locations and pre-specified primary and secondary outcomes[11].”
BackgroundTrial Registration at ClinicalTrials.gov between May and October 2005
Zarin, D. A., Tse, T., & Ide, N. C. (2005) · New England Journal of Medicine · doi:10.1056/nejmsa053234 · Cited by 291
Cardiovascular Drug Development
Fordyce, C. B., Roe, M. T., Ahmad, T., Libby, P., Borer, J. S., Hiatt, W. R., Bristow, M. R., Packer, M., Wasserman, S. M., Braunstein, N., et al. (2015) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2015.03.016 · Cited by 202 · cited 1×
“Since about 1998, there has been a systematic decline in the number approvals for new cardiovascular drugs[6].”
BackgroundMoving Toward Transparency of Clinical Trials
Zarin, D. A., & Tse, T. (2008) · Science · doi:10.1126/science.1153632 · Cited by 182
Publication of Trials Funded by the National Heart, Lung, and Blood Institute
Gordon, D., Taddei‐Peters, W. C., Mascette, A. M., Antman, M., Kaufmann, P. G., & Lauer, M. S. (2013) · New England Journal of Medicine · doi:10.1056/nejmsa1300237 · Cited by 128
Fisher, J. A., Cottingham, M. D., & Kalbaugh, C. A. (2014) · Social Science & Medicine · doi:10.1016/j.socscimed.2014.08.023 · Cited by 82 · cited 1×
“It has been argued that there have been few efficacious drugs in the pipeline[4,5].”
BackgroundRoth, J. A., Etzioni, R., Waters, T. M., Pettinger, M. B., Rossouw, J. E., Anderson, G. L., Chlebowski, R. T., Manson, J. E., Hlatky, M. A., Johnson, K., et al. (2014) · Annals of Internal Medicine · doi:10.7326/m13-2348 · Cited by 64 · cited 1×
“The economic analysis estimated that there was about $140 returned for each dollar invested in the study[13].”
Background
Research works printed but not linked: 1
- D Gordon, 2013, Publication of trials funded by the National Heart, Lung, and Blood Institute, N Engl J Med, 369, 1226, 10.1056/nejmsa1300237 · DOI
Cited by
Reporting animal research: Explanation and elaboration for the ARRIVE guidelines 2.0
Percie du Sert, N., Ahluwalia, A., Alam, S., Avey, M. T., Baker, M., Browne, W. J., Clark, A., Cuthill, I. C., Dirnagl, U., Emerson, M., et al. (2020) · PLOS Biology · doi:10.1371/journal.pbio.3000411 · Cited by 2,677
The preregistration revolution
Nosek, B. A., Ebersole, C. R., DeHaven, A. C., & Mellor, D. T. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1708274114 · Cited by 1,998
“For example, Kaplan and Irvin (43) observed a dramatic drop in the rate of positive results following the requirement to preregister primary outcomes in a sample of clinical trials.”
BackgroundDo “brain-training” programs work?
Simons, D. J., Boot, W. R., Charness, N., Gathercole, S. E., Chabris, C. F., Hambrick, D. Z., & Stine-Morrow, E. A. L. (2016) · Psychological Science in the Public Interest · doi:10.1177/1529100616661983 · Cited by 1,183
How Open Science Helps Researchers Succeed
McKiernan, E. C., Bourne, P. E., Brown, C. T., Buck, S., Kenall, A., Lin, J., McDougall, D., Nosek, B. A., Ram, K., Soderberg, C. K., et al. (2016) · eLife · doi:10.7554/elife.16800 · Cited by 798
Video Improves Learning in Higher Education: A Systematic Review
Noetel, M., Griffith, S., Delaney, O., Sanders, T., Parker, P., del Pozo Cruz, B., & Lonsdale, C. (2021) · Review of Educational Research · doi:10.3102/0034654321990713 · Cited by 378
“For example, studies that prospectively registered their methods and primary outcome are much less likely to find significant results (Kaplan & Irvin, 2015).”
Result
Metadata
- Authors
- Kaplan, R. M., & Irvin, V. L.
- Affiliations
- Agency for Healthcare Research and Quality (US)
- Oregon State University (US)
- Corresponding author
- Kaplan, Robert M.
- Venue
- PLOS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2015-08-05
- Volume
- 10
- Issue
- 8
- Pages
- e0132382
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC4526697
- OpenAlex
- W1944584781
- Cited by
- 356
- References
- 11
- Citations by year
- 2026: 13 · 2025: 17 · 2024: 22 · 2023: 44 · 2022: 28 · 2021: 50 · 2020: 58 · 2019: 33 · 2018: 37 · 2017: 24 · 2016: 21 · 2015: 9
- FWCI
- 35.39
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- public-domain
- creativecommons.org/publicdomain/zero/1.0/ (vor)
- Locations
- PLoS ONE (publishedVersion, public-domain)
- Europe PMC (PubMed Central) (publishedVersion, public-domain)
- PubMed Central (publishedVersion, public-domain)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Figshare (submittedVersion, cc-by)
- Funders
- Oregon State University
- National Institutes of Health
- Agency for Healthcare Research and Quality
- National Heart, Lung, and Blood Institute
- U.S. National Library of Medicine
- Office of Behavioral and Social Sciences Research
- Topics
- Health Sciences › Nursing › Issues, ethics and legal aspects › Pharmaceutical industry and healthcare
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Keywords
- negative results, clinical trial results, cardiovascular disease prevention, reporting guidelines, all-cause mortality, publication bias
- MeSH terms
- Cardiovascular Diseases, Humans, Publishing, Registries, Research Design, United States, Randomized Controlled Trials as Topic, Treatment Outcome, National Heart, Lung, and Blood Institute (U.S.)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Likelihood of Null Effects of Large NHLBI Clinical Trials Has Increased over Time. https://app.cochat.ai/papers/10.1371/journal.pone.0132382
- The paper
- Kaplan, R. M., & Irvin, V. L. (2015). Likelihood of Null Effects of Large NHLBI Clinical Trials Has Increased over Time. PLOS ONE. https://doi.org/10.1371/journal.pone.0132382
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.