Rising complexity and falling explanatory power in ecology
The DOI matches its Crossref record; 44 of the 44 references it cites were found in a registry or an index.
Abstract
Analyses of published research can provide a realistic perspective on the progress of science. By analyzing more than 18 000 articles published by the preeminent ecological societies, we found that (1) ecological research is becoming increasingly statistically complex, reporting a growing number of P values per article and (2) the value of reported coefficient of determination ( R 2 ) has been falling steadily, suggesting a decrease in the marginal explanatory power of ecology. These trends may be due to changes in the way ecology is studied or in the way the findings of investigations are reported. Determining the reason for increasing complexity and declining marginal explanatory power would require a critical review of the scientific process in ecology, from research design to dissemination, and could influence the public interpretation and policy implications of ecological findings.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“assessments in various disciplines suggest that, far from becoming increasingly short and trivial, published studies are getting longer, more complex, and richer in data (e.g., refs. 32 -34).”
From Is science really facing a reproducibility crisis, and do we need it to? (2018) · cites this paper as [34]
Citations
Cites
Statistical Methods for Research Workers
(1945) · American Journal of Clinical Pathology · doi:10.1093/ajcp/15.11.552b · Cited by 10,880
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505
Griffiths, T. L., & Steyvers, M. (2004) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0307752101 · Cited by 6,049 · cited 2×
“…output has provided several insights into the trajectory of scientific disciplines, including a better understanding of shifts in topic selection (Griffiths and Steyvers 2004; Perc 2013), shifts in patterns of scientific collaboration (Wuchty et al. 2007), and the consequences of selective…”
BackgroundWhy Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · CHANCE · doi:10.1080/09332480.2005.10722754 · Cited by 4,702
The Increasing Dominance of Teams in Production of Knowledge
Wuchty, S., Jones, B. F., & Uzzi, B. (2007) · Science · doi:10.1126/science.1136099 · Cited by 3,615
Systematic Review of the Empirical Evidence of Study Publication Bias and Outcome Reporting Bias
Dwan, K., Altman, D. G., Arnaiz, J. A., Bloom, J., Chan, A.-W., Cronin, E., Decullier, E., Easterbrook, P. J., Von Elm, E., Gamble, C., et al. (2008) · PLoS ONE · doi:10.1371/journal.pone.0003081 · Cited by 1,546
Statistical Methods for Research Workers
(1926) · Journal of Comparative Pathology and Therapeutics · doi:10.1016/s0368-1742(26)80018-9 · Cited by 1,283
A variance explanation paradox: When a little is a lot.
Abelson, R. P. (1985) · Psychological Bulletin · doi:10.1037/0033-2909.97.1.129 · Cited by 1,045
Ecological Forecasts: An Emerging Imperative
Clark, J. S., Carpenter, S. R., Barber, M., Collins, S., Dobson, A., Foley, J. A., Lodge, D. M., Pascual, M., Pielke, R., Pizer, W., et al. (2001) · Science · doi:10.1126/science.293.5530.657 · Cited by 913
Rescuing US biomedical research from its systemic flaws
Alberts, B., Kirschner, M. W., Tilghman, S., & Varmus, H. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1404402111 · Cited by 817 · cited 2×
“Among these are the relative strengths of the incentives offered to researchers for undertaking marginal improvements on existing studies rather than risk-taking and original research (Alberts et al. 2014), and for their capacity to produce statistically significant results rather than for their ability to explain and…”
BackgroundMulti-University Research Teams: Shifting Impact, Geography, and Stratification in Science
Jones, B. F., Wuchty, S., & Uzzi, B. (2008) · Science · doi:10.1126/science.1158357 · Cited by 812
Do Pressures to Publish Increase Scientists' Bias? An Empirical Support from US States Data
Fanelli, D. (2010) · PLoS ONE · doi:10.1371/journal.pone.0010271 · Cited by 805
How much variance can be explained by ecologists and evolutionary biologists?
Møller, A., & Jennions, M. D. (2002) · Oecologia · doi:10.1007/s00442-002-0952-2 · Cited by 553
Palmer, A. R. (2000) · Annual Review of Ecology and Systematics · doi:10.1146/annurev.ecolsys.31.1.441 · Cited by 385
Alternatives To Statistical Hypothesis Testing In Ecology: A Guide To Self Teaching
Hobbs, N. T., & Hilborn, R. (2006) · Ecological Applications · doi:10.1890/04-0645 · Cited by 384
Electronic Publication and the Narrowing of Science and Scholarship
Evans, J. A. (2008) · Science · doi:10.1126/science.1150473 · Cited by 364
Use, Overuse, and Misuse of Significance Tests in Evolutionary Biology and Ecology
(1991) · The Bulletin of the Ecological Society of America · doi:10.2307/20167258 · Cited by 281
Slash and Burn Impacts on a Costa Rican Wet Forest Site
Ewel, J., Berish, C., Brown, B., Price, N., & Raich, J. (1981) · Ecology · doi:10.2307/1937748 · Cited by 273
Inference in ecology and evolution
Stephens, P. A., Buskirk, S. W., & del Rio, C. M. (2007) · Trends in Ecology & Evolution · doi:10.1016/j.tree.2006.12.003 · Cited by 268
Evans, J. A., & Foster, J. G. (2011) · Science · doi:10.1126/science.1201765 · Cited by 252
A Note on Percent Variance Explained as A Measure of the Importance of Effects
Rosenthal, R., & Rubin, D. B. (1979) · Journal of Applied Social Psychology · doi:10.1111/j.1559-1816.1979.tb02713.x · Cited by 235
On Probability as a Basis for Action
Deming, W. E. (1975) · The American Statistician · doi:10.1080/00031305.1975.10477402 · Cited by 202
Odenbaugh, J. (2005) · Biology & Philosophy · doi:10.1007/s10539-004-0478-6 · Cited by 172
Rules of thumb for judging ecological theories
Ginzburg, L. R., & Jensen, C. X. J. (2004) · Trends in Ecology & Evolution · doi:10.1016/j.tree.2003.11.004 · Cited by 167
How Well Do Criminologists Explain Crime? Statistical Modeling in Published Studies
Weisburd, D., & Piquero, A. R. (2008) · Crime and Justice · doi:10.1086/524284 · Cited by 158
Leimu, R., & Koricheva, J. (2004) · Proceedings of the Royal Society of London. Series B: Biological Sciences · doi:10.1098/rspb.2004.2828 · Cited by 156 · cited 3×
“Within some subdisciplines, however, journal impact factor sometimes influenced effect size (Murtaugh 2002) and studies with disconfirming evidence were published in lower impact journals (Leimu and Koricheva 2004).”
BackgroundFlows of Research Manuscripts Among Scientific Journals Reveal Hidden Submission Patterns
Calcagno, V., Demoinet, E., Gollner, K., Guidi, L., Ruths, D., & de Mazancourt, C. (2012) · Science · doi:10.1126/science.1227833 · Cited by 155
A possible declining trend for worldwide innovation
Huebner, J. (2005) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2005.01.003 · Cited by 142
JOURNAL QUALITY, EFFECT SIZE, AND PUBLICATION BIAS IN META-ANALYSIS
Murtaugh, P. A. (2002) · Ecology · doi:10.1890/0012-9658(2002)083[1162:jqesap]2.0.co;2 · Cited by 126 · cited 1×
“Murtaugh PA. 2002.”
Academia's obsession with quantity
Fischer, J., Ritchie, E. G., & Hanspach, J. (2012) · Trends in Ecology & Evolution · doi:10.1016/j.tree.2012.05.010 · Cited by 123
ON THE EVOLUTION OF ECOLOGICAL IDEAS: PARADIGMS AND SCIENTIFIC PROGRESS
Graham, M. H., & Dayton, P. K. (2002) · Ecology · doi:10.1890/0012-9658(2002)083[1481:oteoei]2.0.co;2 · Cited by 118
Publication bias and merit in ecology
Lortie, C. J., Aarssen, L. W., Budden, A. E., Koricheva, J. K., Leimu, R., & Tregenza, T. (2007) · Oikos · doi:10.1111/j.0030-1299.2007.15686.x · Cited by 103
Statistical reform in medicine, psychology and ecology
Fidler, F. M., Cumming, G., Burgman, M. A., & Thomason, N. R. (2004) · The Journal of Socio-Economics · doi:10.1016/j.socec.2004.09.035 · Cited by 102
Trends in Ecological Research during the Last Three Decades – A Systematic Review
Carmel, Y., Kent, R., Bar‐Massada, A., Blank, L., Liberzon, J., Nezer, O., Sapir, G., & Federman, R. (2013) · PLoS ONE · doi:10.1371/journal.pone.0059813 · Cited by 98
Self-organization of progress across the century of physics
Perc, M. (2013) · Scientific Reports · doi:10.1038/srep01720 · Cited by 91
High Impact = High Statistical Standards? Not Necessarily So
Tressoldi, P. E., Giofré, D., Sella, F., & Cumming, G. (2013) · PLoS ONE · doi:10.1371/journal.pone.0056180 · Cited by 79
Complexity and the productivity of innovation
Strumsky, D., Lobo, J., & Tainter, J. A. (2010) · Systems Research and Behavioral Science · doi:10.1002/sres.1057 · Cited by 76
Self-organization of progress across the century of physics
Perc, M. (2017) · University of Maribor digital library (University of Maribor) · Cited by 63
On Probability as a Basis for Action
Deming, W. E. (1975) · The American Statistician · doi:10.2307/2683482 · Cited by 59
The economics of ecology journals
Bergstrom, C. T., & Bergstrom, T. C. (2006) · Frontiers in Ecology and the Environment · doi:10.1890/1540-9295(2006)4[488:teoej]2.0.co;2 · Cited by 45
How Not to Lie with Statistics: Avoiding Common Mistakes in Quantitative Political Science
King, G. (1986) · American Journal of Political Science · doi:10.2307/2111095 · Cited by 41
Obsession with quantity: a view from the south.
Loyola, R., Diniz-Filho, J. A., & Bini, L. (2012) · Trends in Ecology & Evolution · doi:10.1016/j.tree.2012.07.016 · Cited by 26
Quantifying the ease of scientific discovery
Samuel Arbesman, C. (2010) · Scientometrics · doi:10.1007/s11192-010-0232-6 · Cited by 23
Lortie, C. J., Aarssen, L. W., Budden, A. E., & Leimu, R. (2013) · Scientometrics · doi:10.1007/s11192-012-0822-6 · Cited by 22
Research works printed but not linked: 1
- Connor RO., 2000, Why ecology lags behind biology, The Scientist, 14, 35
Cited by
A brief introduction to mixed effects modelling and multi-model inference in ecology
Harrison, X. A., Donaldson, L., Correa-Cano, M. E., Evans, J., Fisher, D. N., Goodwin, C. E. D., Robinson, B. S., Hodgson, D. J., & Inger, R. (2018) · PeerJ · doi:10.7717/peerj.4794 · Cited by 2,094
Is science really facing a reproducibility crisis, and do we need it to?
Fanelli, D. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1708272114 · Cited by 416
Addressing context dependence in ecology
Catford, J. A., Wilson, J. R. U., Pyšek, P., Hulme, P. E., & Duncan, R. P. (2022) · Trends in Ecology & Evolution · doi:10.1016/j.tree.2021.09.007 · Cited by 344
Evidence of experimental bias in the life sciences: why we need blind data recording
Holman, L., Head, M. L., Lanfear, R., & Jennions, M. D. (2015) · PLOS Biology · doi:10.1371/journal.pbio.1002190 · Cited by 263
Heino, J., Melo, A. S., Bini, L. M., Altermatt, F., Al‐Shami, S. A., Angeler, D. G., Bonada, N., Brand, C., Callisto, M., Cottenie, K., et al. (2015) · Ecology and Evolution · doi:10.1002/ece3.1439 · Cited by 196
Metadata
- Authors
- Low-Décarie, E., Chivers, C., & Granados, M.
- Affiliations
- McGill University (CA)
- ORCID
- Chivers, Corey
- Venue
- Frontiers in Ecology and the Environment
- Publisher
- Wiley
- Published
- 2014-08-20
- Volume
- 12
- Issue
- 7
- Pages
- 412-418
- ISSN
- 1540-9295 (ISSN-L)
- 1540-9309
- OpenAlex
- W2108866660
- Cited by
- 111
- References
- 43
- Citations by year
- 2026: 10 · 2025: 7 · 2024: 9 · 2023: 6 · 2022: 14 · 2021: 2 · 2020: 7 · 2019: 7 · 2018: 23 · 2017: 7 · 2016: 5 · 2015: 12 · 2014: 2
- FWCI
- 8.48
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Frontiers in Ecology and the Environment (publishedVersion)
- Open Access at Essex (University of Essex) (submittedVersion)
- Topics
- Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Keywords
- statistical complexity, explanatory power, R-squared, p-values, publication analysis, ecological society, research design, environmental policy
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.