Repeatability and transparency in ecological research
Open accessThe DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index.
Abstract
A fundamental tenet of science is that results must be reproducible by other scientists before they are accepted as factual. However, because ecological phenomena are context-dependent, and because that context changes through time and space, it is virtually impossible to reproduce precisely or quantitatively any single experimental or observational field study in ecology. Yet many ecological studies can be repeated. In particular, ecological synthesis—the assembly of derived data sets and their subsequent analysis, reanalysis, and metaanalysis—should be easy to repeat and reproduce. Such syntheses also demonstrate qualitative and quantitative consistency among many ecological studies (Gurevitch et al. 1992, Warwick and Clarke 1993, Jonsen et al. 2003, Walker et al. 2006, Cardinale et al. 2006, Marczak et al. 2007, Vander Zanden and Fetzer 2007) and provide strong support for general ecological theories. It should come as no surprise that meta-analysis by Mittelbach et al. (2001) of the effect of productivity on species richness has led to the development of a cottage industry focused on empirical testing of this relationship (post-2001 examples abound in Appendix A of Whittaker 2010). But it is much more surprising that continual reanalyses of the same data sets (Whittaker and Heegaard 2003, Gillman and Wright 2006, Partel et al. 2007) have yielded such disparate results that Whittaker (2010) has suggested abandoning the effort to obtain consistent results from the available data. He goes even further, suggesting that ecology may not yet be ready for meta-analysis and data synthesis. For two reasons, I respectfully suggest that Whittaker’s critique is misplaced. First, of all the studies critiqued by Whittaker (2010), only Mittelbach et al. (2001) actually conducted a formal meta-analysis. The others, as pointed out by Whittaker (2010), undertook extensive primary analyses, but the authors did not conduct formal meta-analyses (Gurevitch and Hedges 1999). Second, and more importantly, if ecological synthesis is transparent—data, models, and analytical tools are available freely to the research community—then it should yield consistent, repeatable results. We may then disagree on the interpretation of the resulting synthesis, but at least we will be able to agree on the reproducibility of the results themselves.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Effects of biodiversity on the functioning of trophic groups and ecosystems
Cardinale, B. J., Srivastava, D. S., Emmett Duffy, J., Wright, J. P., Downing, A. L., Sankaran, M., & Jouseau, C. (2006) · Nature · doi:10.1038/nature05202 · Cited by 1,871
WHAT IS THE OBSERVED RELATIONSHIP BETWEEN SPECIES RICHNESS AND PRODUCTIVITY?
Mittelbach, G. G., Steiner, C. F., Scheiner, S. M., Gross, K. L., Reynolds, H. L., Waide, R. B., Willig, M. R., Dodson, S. I., & Gough, L. (2001) · Ecology · doi:10.1890/0012-9658(2001)082[2381:witorb]2.0.co;2 · Cited by 1,650
Plant community responses to experimental warming across the tundra biome
Walker, M. D., Wahren, C. H., Hollister, R. D., Henry, G. H. R., Ahlquist, L. E., Alatalo, J. M., Bret-Harte, M. S., Calef, M. P., Callaghan, T. V., Carroll, A. B., et al. (2006) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0503198103 · Cited by 1,310
THE INFLUENCE OF PRODUCTIVITY ON THE SPECIES RICHNESS OF PLANTS: A CRITICAL ASSESSMENT
Gillman, L. N., & Wright, S. D. (2006) · Ecology · doi:10.1890/0012-9658(2006)87[1234:tiopot]2.0.co;2 · Cited by 305
Whittaker, R. J. (2010) · Ecology · doi:10.1890/08-0968.1 · Cited by 223 · cited 1×
“…qualitative and quantitative consistency among many ecological studies (Gurevitch et al. 1992, Warwick and Clarke 1993, Jonsen et al. 2003, Walker et al. 2006, Cardinale et al. 2006, Marczak et al. 2007, Vander Zanden and Fetzer 2007) and provide strong support for general ecological theories .”
Background
Cited by
Big data and the future of ecology
Hampton, S. E., Strasser, C. A., Tewksbury, J. J., Gram, W. K., Budden, A. E., Batcheller, A. L., Duke, C. S., & Porter, J. H. (2013) · Frontiers in Ecology and the Environment · doi:10.1890/120103 · Cited by 924
Meta-evaluation of meta-analysis: ten appraisal questions for biologists
Nakagawa, S., Noble, D. W. A., Senior, A. M., & Lagisz, M. (2017) · BMC Biology · doi:10.1186/s12915-017-0357-7 · Cited by 579
Environmental performance evaluation with big data: theories and methods
Song, M.-L., Fisher, R., Wang, J.-L., & Cui, L.-B. (2018) · Annals of Operations Research · doi:10.1007/s10479-016-2158-8 · Cited by 295
Smart Earth: A meta-review and implications for environmental governance
Bakker, K., & Ritts, M. (2018) · Global Environmental Change · doi:10.1016/j.gloenvcha.2018.07.011 · Cited by 244
“Collaboration and data sharing as such remain considerably removed from the actual opportunities enabled by Smart Earth (Ellison, 2010; Reichman et al., 2011; Hampton et al., 2013; Volk et al., 2014).”
Whittaker, R. J. (2010) · Ecology · doi:10.1890/08-0968.1 · Cited by 223
Metadata
- Authors
- Ellison, A. M.
- Venue
- Ecology
- Publisher
- Wiley
- Published
- 2010-09
- Volume
- 91
- Issue
- 9
- Pages
- 2536-2539
- ISSN
- 0012-9658 (ISSN-L)
- 1939-9170
- OpenAlex
- W2089391420
- Cited by
- 80
- References
- 5
- FWCI
- 16.26
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.