The DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.
Relationships fade with time: a meta-analysis of temporal trends in publication in ecology and evolution
Open accessAbstract
Both significant positive and negative relationships between the magnitude of research findings (their 'effect size') and their year of publication have been reported in a few areas of biology. These trends have been attributed to Kuhnian paradigm shifts, scientific fads and bias in the choice of study systems. Here we test whether or not these isolated cases reflect a more general trend. We examined the relationship using effect sizes extracted from 44 peer-reviewed meta-analyses covering a wide range of topics in ecological and evolutionary biology. On average, there was a small but significant decline in effect size with year of publication. For the original empirical studies there was also a significant decrease in effect size as sample size increased. However, the effect of year of publication remained even after we controlled for sampling effort. Although these results have several possible explanations, it is suggested that a publication bias against non-significant or weaker findings offers the most parsimonious explanation. As in the medical sciences, non-significant results may take longer to publish and studies with both small sample sizes and non-significant results may be less likely to be published.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Statistical power analysis for the behavioral sciences
(1990) · Computers, Environment and Urban Systems · doi:10.1016/0198-9715(90)90050-4 · Cited by 65,269
The Structure of Scientific Revolutions
Kuhn, T. S., & Schlegel, R. (1963) · Physics Today · doi:10.1063/1.3050879 · Cited by 35,538
Operating characteristics of a rank correlation test for publication bias
Begg, C. B., & Mazumdar, M. (1994) · Biometrics · doi:10.2307/2533446 · Cited by 16,955
The Structure of Scientific Revolutions
Kuhn, T. S. (1996) · doi:10.7208/chicago/9780226458106.001.0001 · Cited by 7,281
Hempel, C. G. (1966) · Medical Entomology and Zoology · Cited by 2,922
Parametric measures of effect size.
Rosenthal, R. (1994) · Cited by 1,847
What is this thing called science?
(2014) · Choice Reviews Online · doi:10.5860/choice.51-6134 · Cited by 1,568
Summing up: The Science of Reviewing Research
Wilkinson, I., Light, R. J., & Pillemer, D. B. (1987) · Journal of Educational Statistics · doi:10.2307/1164691 · Cited by 1,471
Summing Up: The Science of Reviewing Research
Light, R. J., & Pillemer, D. B. (1984) · Harvard University Press eBooks · doi:10.4159/9780674040243 · Cited by 1,427
Publication bias: evidence of delayed publication in a cohort study of clinical research projects
Stern, J. M., & Simes, R. J. (1997) · BMJ · doi:10.1136/bmj.315.7109.640 · Cited by 818 · cited 1×
“Our results mirror those in medicine and the social sciences (Song et al. 2000) where it takes longer to publish non-signi® cant results (Stern & Simes 1997; Ioannidis 1998).”
Result
Cited by
A farewell to Bonferroni: the problems of low statistical power and publication bias
Nakagawa, S. (2004) · Behavioral Ecology · doi:10.1093/beheco/arh107 · Cited by 2,097
Why Most Discovered True Associations Are Inflated
Ioannidis, J. P. A. (2008) · Epidemiology · doi:10.1097/ede.0b013e31818131e7 · Cited by 1,495
Rosenberg, M. S. (2005) · Evolution · doi:10.1111/j.0014-3820.2005.tb01004.x · Cited by 1,285
Forstmeier, W., & Schielzeth, H. (2011) · Behavioral Ecology and Sociobiology · doi:10.1007/s00265-010-1038-5 · Cited by 1,120
Methodological issues and advances in biological meta-analysis
Nakagawa, S., & Santos, E. S. A. (2012) · Evolutionary Ecology · doi:10.1007/s10682-012-9555-5 · Cited by 1,032
Land‐use intensity and the effects of organic farming on biodiversity: a hierarchical meta‐analysis
Tuck, S. L., Winqvist, C., Mota, F., Ahnström, J., Turnbull, L. A., & Bengtsson, J. (2014) · Journal of Applied Ecology · doi:10.1111/1365-2664.12219 · Cited by 846
Hadfield, J. D., & Nakagawa, S. (2010) · Journal of Evolutionary Biology · doi:10.1111/j.1420-9101.2009.01915.x · Cited by 822
A meta-analysis of procedures to change implicit measures
Forscher, P. S., Lai, C. K., Axt, J. R., Ebersole, C. R., Herman, M., Devine, P. G., & Nosek, B. A. (2019) · Journal of Personality and Social Psychology · doi:10.1037/pspa0000160 · Cited by 700
O'Dea, R. E., Lagisz, M., Jennions, M. D., Koricheva, J., Noble, D. W. A., Parker, T. H., Gurevitch, J., Page, M. J., Stewart, G., Moher, D., et al. (2021) · Biological Reviews · doi:10.1111/brv.12721 · Cited by 697
Self-citationMeta-evaluation of meta-analysis: ten appraisal questions for biologists
Nakagawa, S., Noble, D. W. A., Senior, A. M., & Lagisz, M. (2017) · BMC Biology · doi:10.1186/s12915-017-0357-7 · Cited by 579
How much variance can be explained by ecologists and evolutionary biologists?
Møller, A., & Jennions, M. D. (2002) · Oecologia · doi:10.1007/s00442-002-0952-2 · Cited by 553
Self-citationUses and misuses of meta‐analysis in plant ecology
Koricheva, J., & Gurevitch, J. (2014) · Journal of Ecology · doi:10.1111/1365-2745.12224 · Cited by 460
Egg size and offspring quality: a meta‐analysis in birds
Krist, M. (2011) · Biological Reviews · doi:10.1111/j.1469-185x.2010.00166.x · Cited by 472
Detecting and avoiding likely false‐positive findings – a practical guide
Forstmeier, W., Wagenmakers, E., & Parker, T. H. (2017) · Biological Reviews · doi:10.1111/brv.12315 · Cited by 436
The effects of sex hormones on immune function: a meta-analysis: Sex hormones and immune function
Foo, Y. Z., Nakagawa, S., Rhodes, G., & Simmons, L. W. (2017) · Biological Reviews · doi:10.1111/brv.12243 · Cited by 458
Methods for testing publication bias in ecological and evolutionary meta‐analyses
Nakagawa, S., Lagisz, M., Jennions, M. D., Koricheva, J., Noble, D. W. A., Parker, T. H., Sánchez‐Tójar, A., Yang, Y., & O'Dea, R. E. (2022) · Methods in Ecology and Evolution · doi:10.1111/2041-210x.13724 · Cited by 399
Self-citationSupport for major hypotheses in invasion biology is uneven and declining
Jeschke, J., Gómez Aparicio, L., Haider, S., Heger, T., Lortie, C., Pyšek, P., & Strayer, D. (2012) · NeoBiota · doi:10.3897/neobiota.14.3435 · Cited by 374
Melby-Lervåg, M., & Lervåg, A. (2014) · Psychological Bulletin · doi:10.1037/a0033890 · Cited by 363
Phenological tracking enables positive species responses to climate change
Cleland, E. E., Allen, J. M., Crimmins, T. M., Dunne, J. A., Pau, S., Travers, S. E., Zavaleta, E. S., & Wolkovich, E. M. (2012) · Ecology · doi:10.1890/11-1912.1 · Cited by 354
Nykänen, H., & Koricheva, J. (2004) · Oikos · doi:10.1111/j.0030-1299.2004.12768.x · Cited by 350
Sexual selection and sperm quantity: meta-analyses of strategic ejaculation
Kelly, C. D., & Jennions, M. D. (2011) · Biological Reviews · doi:10.1111/j.1469-185x.2011.00175.x · Cited by 318
Self-citationAttention bias modification for social anxiety: A systematic review and meta-analysis
Heeren, A., Mogoașe, C., Philippot, P., & McNally, R. J. (2015) · Clinical Psychology Review · doi:10.1016/j.cpr.2015.06.001 · Cited by 312
Bodey, T. W., Cleasby, I. R., Bell, F., Parr, N., Schultz, A., Votier, S. C., & Bearhop, S. (2018) · Methods in Ecology and Evolution · doi:10.1111/2041-210x.12934 · Cited by 312
The earth is flat ( p > 0.05): significance thresholds and the crisis of unreplicable research
Amrhein, V., Korner-Nievergelt, F., & Roth, T. (2017) · PeerJ · doi:10.7717/peerj.3544 · Cited by 308
Model systems in ecology: dissecting the endophyte–grass literature
Saikkonen, K., Lehtonen, P., Helander, M., Koricheva, J., & Faeth, S. H. (2006) · Trends in Plant Science · doi:10.1016/j.tplants.2006.07.001 · Cited by 304
Evidence of an Edge Effect on Avian Nest Success
BATÁRY, P., & BÁLDI, A. (2004) · Conservation Biology · doi:10.1111/j.1523-1739.2004.00184.x · Cited by 296
A survey of the statistical power of research in behavioral ecology and animal behavior
Jennions, M. D. (2003) · Behavioral Ecology · doi:10.1093/beheco/14.3.438 · Cited by 278
Self-citationIs bilingualism related to a cognitive advantage in children? A systematic review and meta-analysis.
Gunnerud, H. L., ten Braak, D., Reikerås, E. K. L., Donolato, E., & Melby-Lervåg, M. (2020) · Psychological Bulletin · doi:10.1037/bul0000301 · Cited by 281
Publication bias and the canonization of false facts
Nissen, S. B., Magidson, T., Gross, K., & Bergstrom, C. T. (2016) · eLife · doi:10.7554/elife.21451 · Cited by 266
Evidence of experimental bias in the life sciences: why we need blind data recording
Holman, L., Head, M. L., Lanfear, R., & Jennions, M. D. (2015) · PLOS Biology · doi:10.1371/journal.pbio.1002190 · Cited by 263
Self-citationPublication bias in ecology and evolution: an empirical assessment using the ‘trim and fill’ method
JENNIONS, M. D., & MØLLER, A. P. (2002) · Biological Reviews · doi:10.1017/s1464793101005875 · Cited by 230
Self-citationPereira, T. V., & Ioannidis, J. P. A. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.12.012 · Cited by 160
Leimu, R., & Koricheva, J. (2004) · Proceedings of the Royal Society of London. Series B: Biological Sciences · doi:10.1098/rspb.2004.2828 · Cited by 156
Evolution and Translation of Research Findings: From Bench to Where
Ioannidis, J. P. A. (2006) · PLoS Clinical Trials · doi:10.1371/journal.pctr.0010036 · Cited by 129
Nakagawa, S., & Parker, T. H. (2015) · BMC Biology · doi:10.1186/s12915-015-0196-3 · Cited by 110
Non‐significant results in ecology: a burden or a blessing in disguise?
(2003) · Oikos · doi:10.1034/j.1600-0579.2003.12353.x · Cited by 75
“Contrary to predictions by some ecologists (Csada et al. 1996, Jennions and Møller 2002) and evidence coming from medicine (reviewed in Song et al. 2000), the mean proportion of non-significant results was significantly higher in published (49”
BackgroundResultHow to critically read ecological meta-analyses
Lortie, C. J., Stewart, G., Rothstein, H., & Lau, J. (2015) · Research Synthesis Methods · doi:10.1002/jrsm.1109 · Cited by 39
Meta‐analysis can “fail”: reply to Kotiaho and Tomkins
Jennions, M. D., Møller, A. P., & Hunt, J. (2004) · Oikos · doi:10.1111/j.0030-1299.2004.12265.x · Cited by 16
Self-citationTransgenerational effects of obesogenic diets in rodents: A meta‐analysis
Anwer, H., Morris, M. J., Noble, D. W. A., Nakagawa, S., & Lagisz, M. (2022) · Obesity Reviews · doi:10.1111/obr.13342 · Cited by 14
A meta-analysis of the effects in alpha acoustic indices
Wang, Y., Zhang, Y., Xia, C., Pape Møller, A., & 1. Ministry of Education Key Laboratory for Biodiversity Science and Ecological Engineering, C. O. L. S. (2023) · Biodiversity Science · doi:10.17520/biods.2022369 · Cited by 7
Metadata
- Authors
- Jennions, M. D., & Møller, A. P.
- Venue
- Proceedings of the Royal Society of London. Series B: Biological Sciences
- Publisher
- The Royal Society
- Published
- 2002-01-07
- Volume
- 269
- Issue
- 1486
- Pages
- 43-48
- ISSN
- 0080-4649 (ISSN-L)
- 0962-8452
- 1471-2954
- OpenAlex
- W2169737291
- Cited by
- 255
- References
- 32
- FWCI
- 3.16
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.