The DOI matches its Crossref record; 57 of the 57 references it cites were found in a registry or an index.
Nonindependence and sensitivity analyses in ecological and evolutionary meta‐analyses
Abstract
Meta‐analysis is an important tool for synthesizing research on a variety of topics in ecology and evolution, including molecular ecology, but can be susceptible to nonindependence. Nonindependence can affect two major interrelated components of a meta‐analysis: (i) the calculation of effect size statistics and (ii) the estimation of overall meta‐analytic estimates and their uncertainty. While some solutions to nonindependence exist at the statistical analysis stages, there is little advice on what to do when complex analyses are not possible, or when studies with nonindependent experimental designs exist in the data. Here we argue that exploring the effects of procedural decisions in a meta‐analysis (e.g. inclusion of different quality data, choice of effect size) and statistical assumptions (e.g. assuming no phylogenetic covariance) using sensitivity analyses are extremely important in assessing the impact of nonindependence. Sensitivity analyses can provide greater confidence in results and highlight important limitations of empirical work (e.g. impact of study design on overall effects). Despite their importance, sensitivity analyses are seldom applied to problems of nonindependence. To encourage better practice for dealing with nonindependence in meta‐analytic studies, we present accessible examples demonstrating the impact that ignoring nonindependence can have on meta‐analytic estimates. We also provide pragmatic solutions for dealing with nonindependent study designs, and for analysing dependent effect sizes. Additionally, we offer reporting guidelines that will facilitate disclosure of the sources of nonindependence in meta‐analyses, leading to greater transparency and more robust conclusions.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000100 · Cited by 28,164
Cochrane Handbook for Systematic Reviews of Interventions: Cochrane Book Series
(2008) · doi:10.1002/9780470712184 · Cited by 25,048
Conducting Meta-Analyses in R with the metafor Package
Viechtbauer, W. (2010) · Journal of Statistical Software · doi:10.18637/jss.v036.i03 · Cited by 20,464
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2009) · doi:10.1002/9780470743386 · Cited by 14,225
Mixed-Effects Models in Sand S-PLUS
Pinheiro, J. C., & Bates, D. M. (2000) · Statistics and Computing · doi:10.1007/978-1-4419-0318-1 · Cited by 11,187
Generalized linear mixed models: a practical guide for ecology and evolution
Bolker, B. M., Brooks, M. E., Clark, C. J., Geange, S. W., Poulsen, J. R., Stevens, M. H. H., & White, J.-S. S. (2009) · Trends in Ecology & Evolution · doi:10.1016/j.tree.2008.10.008 · Cited by 8,910
Pseudoreplication and the Design of Ecological Field Experiments
Hurlbert, S. H. (1984) · Ecological Monographs · doi:10.2307/1942661 · Cited by 8,377
MCMC Methods for Multi-Response Generalized Linear Mixed Models: The MCMCglmmR Package
Hadfield, J. D. (2010) · Journal of Statistical Software · doi:10.18637/jss.v033.i02 · Cited by 4,799
The meta-analysis of response ratios in experimental ecology
Hedges, L. V., Gurevitch, J., & Curtis, P. S. (1999) · Ecology · doi:10.1890/0012-9658(1999)080[1150:tmaorr]2.0.co;2 · Cited by 4,704
The Handbook of Research Synthesis and Meta-Analysis
(2019) · doi:10.7758/9781610448864 · Cited by 4,553
Effect size, confidence interval and statistical significance: a practical guide for biologists
Nakagawa, S., & Cuthill, I. C. (2007) · Biological Reviews · doi:10.1111/j.1469-185x.2007.00027.x · Cited by 3,835
Self-citationHow should meta‐regression analyses be undertaken and interpreted?
Thompson, S. G., & Higgins, J. P. T. (2002) · Statistics in Medicine · doi:10.1002/sim.1187 · Cited by 3,026
Repeatability for Gaussian and non-Gaussian data: a practical guide for biologists.
Nakagawa, S., & Schielzeth, H. (2010) · Biological Reviews · doi:10.1111/j.1469-185x.2010.00141.x · Cited by 2,584
Publication Bias in Meta‐Analysis: Prevention, Assessment and Adjustments
(2005) · doi:10.1002/0470870168 · Cited by 2,368
Meta-analysis of experiments with matched groups or repeated measures designs
Dunlap, W. P., Cortina, J. M., Vaslow, J. B., & Burke, M. J. (1996) · Psychological Methods · doi:10.1037/1082-989x.1.2.170 · Cited by 1,693 · cited 1×
“Formulas for effect size and corresponding sampling error variance have been derived to ameliorate the problems of nonindependence when correlations exist between groups for specific study designs (e.g. for matchedpairs designs; See Dunlap et al. 1996; Lajeunesse 2011; Borenstein et al. 2009).”
BackgroundCheung, M. W.-L. (2014) · Psychological Methods · doi:10.1037/a0032968 · Cited by 1,342
Handbook of Meta-analysis in Ecology and Evolution
(2013) · doi:10.1515/9781400846184 · Cited by 1,183
STATISTICAL ISSUES IN ECOLOGICAL META-ANALYSES
Gurevitch, J., & Hedges, L. V. (1999) · Ecology · doi:10.1890/0012-9658(1999)080[1142:siiema]2.0.co;2 · Cited by 1,081
Methodological issues and advances in biological meta-analysis
Nakagawa, S., & Santos, E. S. A. (2012) · Evolutionary Ecology · doi:10.1007/s10682-012-9555-5 · Cited by 1,032
Self-citationA simple method for distinguishing within- versus between-subject effects using mixed models
van de Pol, M., & Wright, J. (2009) · Animal Behaviour · doi:10.1016/j.anbehav.2008.11.006 · Cited by 951
AGUILAR, R., QUESADA, M., ASHWORTH, L., HERRERIAS‐DIEGO, Y., & LOBO, J. (2008) · Molecular Ecology · doi:10.1111/j.1365-294x.2008.03971.x · Cited by 878
Hadfield, J. D., & Nakagawa, S. (2010) · Journal of Evolutionary Biology · doi:10.1111/j.1420-9101.2009.01915.x · Cited by 822
Self-citationMeta-evaluation of meta-analysis: ten appraisal questions for biologists
Nakagawa, S., Noble, D. W. A., Senior, A. M., & Lagisz, M. (2017) · BMC Biology · doi:10.1186/s12915-017-0357-7 · Cited by 579
Self-citationEffect Sizes in Cluster-Randomized Designs
Hedges, L. V. (2007) · Journal of Educational and Behavioral Statistics · doi:10.3102/1076998606298043 · Cited by 567
META-ANALYSIS OF COD–SHRIMP INTERACTIONS REVEALS TOP-DOWN CONTROL IN OCEANIC FOOD WEBS
Worm, B., & Myers, R. A. (2003) · Ecology · doi:10.1890/0012-9658(2003)084[0162:maocsi]2.0.co;2 · Cited by 482
A quantitative review of heterozygosity–fitness correlations in animal populations
CHAPMAN, J. R., NAKAGAWA, S., COLTMAN, D. W., SLATE, J., & SHELDON, B. C. (2009) · Molecular Ecology · doi:10.1111/j.1365-294x.2009.04247.x · Cited by 465
Self-citationUses and misuses of meta‐analysis in plant ecology
Koricheva, J., & Gurevitch, J. (2014) · Journal of Ecology · doi:10.1111/1365-2745.12224 · Cited by 460
On the meta-analysis of response ratios for studies with correlated and multi-group designs
Lajeunesse, M. J. (2011) · Ecology · doi:10.1890/11-0423.1 · Cited by 432
Open MEE : Intuitive, open‐source software for meta‐analysis in ecology and evolutionary biology
Wallace, B. C., Lajeunesse, M. J., Dietz, G., Dahabreh, I. J., Trikalinos, T. A., Schmid, C. H., & Gurevitch, J. (2017) · Methods in Ecology and Evolution · doi:10.1111/2041-210x.12708 · Cited by 421
EARLY EMERGENCE ENHANCES PLANT FITNESS: A PHYLOGENETICALLY CONTROLLED META-ANALYSIS
Verdú, M., & Traveset, A. (2005) · Ecology · doi:10.1890/04-1647 · Cited by 403
Weak evidence for anticipatory parental effects in plants and animals
Uller, T., Nakagawa, S., & English, S. (2013) · Journal of Evolutionary Biology · doi:10.1111/jeb.12212 · Cited by 374
Self-citationNext-generation sequencing data interpretation: enhancing reproducibility and accessibility
Nekrutenko, A., & Taylor, J. (2012) · Nature Reviews Genetics · doi:10.1038/nrg3305 · Cited by 318
Heterogeneity in ecological and evolutionary meta‐analyses: its magnitude and implications
Senior, A. M., Grueber, C. E., Kamiya, T., Lagisz, M., O'Dwyer, K., Santos, E. S. A., & Nakagawa, S. (2016) · Ecology · doi:10.1002/ecy.1591 · Cited by 299
Self-citationGilbert, K. J., Andrew, R. L., Bock, D. G., Franklin, M. T., Kane, N. C., Moore, J., Moyers, B. T., Renaut, S., Rennison, D. J., Veen, T., et al. (2012) · Molecular Ecology · doi:10.1111/j.1365-294x.2012.05754.x · Cited by 290
Effect Size in Ecological Experiments: The Application of Biological Models in Meta‐Analysis
Osenberg, C. W., Sarnelle, O., & Cooper, S. D. (1997) · The American Naturalist · doi:10.1086/286095 · Cited by 278
Review of methodology of quantitative reviews using meta‐analysis in ecology
Gates, S. (2002) · Journal of Animal Ecology · doi:10.1046/j.1365-2656.2002.00634.x · Cited by 235
Defining behavioural syndromes and the role of ‘syndrome deviation’ in understanding their evolution
Dingemanse, N. J., Dochtermann, N. A., & Nakagawa, S. (2012) · Behavioral Ecology and Sociobiology · doi:10.1007/s00265-012-1416-2 · Cited by 209
Self-citationManipulation of host behaviour by parasites: a weakening paradigm?
Poulin, R. (2000) · Proceedings of the Royal Society of London. Series B: Biological Sciences · doi:10.1098/rspb.2000.1072 · Cited by 185
Pollution and Its Impact on Wild Animals: A Meta-Analysis on Oxidative Stress
Isaksson, C. (2010) · EcoHealth · doi:10.1007/s10393-010-0345-7 · Cited by 177
Does phylogeny matter? Assessing the impact of phylogenetic information in ecological meta‐analysis
Chamberlain, S. A., Hovick, S. M., Dibble, C. J., Rasmussen, N. L., Van Allen, B. G., Maitner, B. S., Ahern, J. R., Bell‐Dereske, L. P., Roy, C. L., Meza‐Lopez, M., et al. (2012) · Ecology Letters · doi:10.1111/j.1461-0248.2012.01776.x · Cited by 176
HELLER, R., & SIEGISMUND, H. R. (2009) · Molecular Ecology · doi:10.1111/j.1365-294x.2009.04185.x · Cited by 164
Meta‐Analysis and the Comparative Phylogenetic Method
Lajeunesse, M. J. (2009) · The American Naturalist · doi:10.1086/603628 · Cited by 160
Meta‐analysis of magnitudes, differences and variation in evolutionary parameters
Morrissey, M. B. (2016) · Journal of Evolutionary Biology · doi:10.1111/jeb.12950 · Cited by 149
Meta‐analysis: A need for well‐defined usage in ecology and conservation biology
Vetter, D., Rücker, G., & Storch, I. (2013) · Ecosphere · doi:10.1890/es13-00062.1 · Cited by 124
Troubleshooting Public Data Archiving: Suggestions to Increase Participation
Roche, D. G., Lanfear, R., Binning, S. A., Haff, T. M., Schwanz, L. E., Cain, K. E., Kokko, H., Jennions, M. D., & Kruuk, L. E. B. (2014) · PLoS Biology · doi:10.1371/journal.pbio.1001779 · Cited by 123
Adams, D. C. (2008) · Evolution · doi:10.1111/j.1558-5646.2007.00314.x · Cited by 122
Booksmythe, I., Mautz, B. S., Davis, J., Nakagawa, S., & Jennions, M. D. (2015) · Biological reviews/Biological reviews of the Cambridge Philosophical Society · doi:10.1111/brv.12220 · Cited by 104
Self-citationAveraging Dependent Effect Sizes in Meta-Analysis: a Cautionary Note about Procedures
Marín‐Martínez, F., & Sánchez‐Meca, J. (1999) · The Spanish Journal of Psychology · doi:10.1017/s1138741600005436 · Cited by 99
Multivariate meta‐analysis using individual participant data
Riley, R. D., Price, M. J., Jackson, D., Wardle, M., Gueyffier, F., Wang, J., Staessen, J. A., & White, I. R. (2014) · Research Synthesis Methods · doi:10.1002/jrsm.1129 · Cited by 98
Inaccuracy of regression results in replacing bivariate correlations
Aloe, A. M. (2014) · Research Synthesis Methods · doi:10.1002/jrsm.1126 · Cited by 79
Meta-analytic insights into evolutionary ecology: an introduction and synthesis
Nakagawa, S., & Poulin, R. (2012) · Evolutionary Ecology · doi:10.1007/s10682-012-9593-z · Cited by 77
Self-citationReproducible research in the study of biological coloration
White, T. E., Dalrymple, R. L., Noble, D. W. A., O’Hanlon, J. C., Zurek, D. B., & Umbers, K. D. L. (2015) · Animal Behaviour · doi:10.1016/j.anbehav.2015.05.007 · Cited by 56
Self-citationAll eggs are made equal: meta‐analysis of egg sexual size dimorphism in birds
Rutkowska, J., Dubiec, A., & Nakagawa, S. (2013) · Journal of Evolutionary Biology · doi:10.1111/jeb.12282 · Cited by 49
Self-citationEffect of maternal diet on offspring coping styles in rodents: a systematic review and meta‐analysis
Besson, A. A., Lagisz, M., Senior, A. M., Hector, K. L., & Nakagawa, S. (2015) · Biological reviews/Biological reviews of the Cambridge Philosophical Society · doi:10.1111/brv.12210 · Cited by 29
Self-citation16. Statistical Models for the Meta-analysis of Nonindependent Data
Mengersen, K. L., Jennions, M. D., & Schmid, C. H. (2013) · Princeton University Press eBooks · doi:10.1515/9781400846184-018 · Cited by 26
Formalizing the definition of meta‐analysis in Molecular Ecology
ArchMiller, A. A., Bauer, E. F., Koch, R. E., Wijayawardena, B. K., Anil, A., Kottwitz, J. J., Munsterman, A. S., & Wilson, A. E. (2015) · Molecular Ecology · doi:10.1111/mec.13264 · Cited by 17
Patterns of linkage disequilibrium in mitochondrial DNA of 16 ruminant populations
Slate, J., & Phua, S. H. (2003) · Molecular Ecology · doi:10.1046/j.1365-294x.2003.01765.x · Cited by 13
Cited by
Meta-evaluation of meta-analysis: ten appraisal questions for biologists
Nakagawa, S., Noble, D. W. A., Senior, A. M., & Lagisz, M. (2017) · BMC Biology · doi:10.1186/s12915-017-0357-7 · Cited by 579
Self-citationMethods for testing publication bias in ecological and evolutionary meta‐analyses
Nakagawa, S., Lagisz, M., Jennions, M. D., Koricheva, J., Noble, D. W. A., Parker, T. H., Sánchez‐Tójar, A., Yang, Y., & O'Dea, R. E. (2022) · Methods in Ecology and Evolution · doi:10.1111/2041-210x.13724 · Cited by 399
Self-citationGlobal effects of non‐native tree species on multiple ecosystem services
Castro‐Díez, P., Vaz, A. S., Silva, J. S., van Loo, M., Alonso, Á., Aponte, C., Bayón, Á., Bellingham, P. J., Chiuffo, M. C., DiManno, N., et al. (2019) · Biological Reviews · doi:10.1111/brv.12511 · Cited by 281
Gender differences in individual variation in academic grades fail to fit expected patterns for STEM
O’Dea, R. E., Lagisz, M., Jennions, M. D., & Nakagawa, S. (2018) · Nature Communications · doi:10.1038/s41467-018-06292-0 · Cited by 253
Self-citationAn assessment of statistical methods for nonindependent data in ecological meta‐analyses
Song, C., Peacor, S. D., Osenberg, C. W., & Bence, J. R. (2020) · Ecology · doi:10.1002/ecy.3184 · Cited by 71
Metadata
- Authors
- Noble, D. W. A., Lagisz, M., O'dea, R. E., & Nakagawa, S.
- Venue
- Molecular Ecology
- Publisher
- Wiley
- Published
- 2017-03-09
- Volume
- 26
- Issue
- 9
- Pages
- 2410-2425
- ISSN
- 0962-1083 (ISSN-L)
- 1365-294X
- OpenAlex
- W2582721760
- Cited by
- 267
- References
- 57
- FWCI
- 14.92
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.