Meta-analysis and the science of research synthesis
Open accessThe DOI matches its Crossref record; 102 of the 102 references it cites were found in a registry or an index; 9 of the 69 sources its claims cite were checked for support.
Abstract
Meta-analysis is the quantitative, scientific synthesis of research results. Since the term and modern approaches to research synthesis were first introduced in the 1970s, meta-analysis has had a revolutionary effect in many scientific fields, helping to establish evidence-based practice and to resolve seemingly contradictory research outcomes. At the same time, its implementation has engendered criticism and controversy, in some cases general and others specific to particular disciplines. Here we take the opportunity provided by the recent fortieth anniversary of meta-analysis to reflect on the accomplishments, limitations, recent advances and directions for future developments in the field of research synthesis.
Overview
CoChat Summary
TL;DR
This review reflects on meta-analysis as quantitative research synthesis since the 1970s, its contributions to evidence-based practice, and its methodological development across medicine, social sciences, and ecology/evolution/conservation. It argues that meta-analysis has resolved contradictory evidence and enabled broad generalizations, but faces a “midlife crisis” from poor-quality applications, publication/research bias, and misuse of the term.
Key findings
- Meta-analysis was first introduced in the 1970s; the term was coined by Gene Glass in 1976, and the paper marks its 40th anniversary. It has helped establish evidence-based practice, resolve seemingly contradictory results, and spread across many scientific fields.
- In medicine, a meta-analysis of 20 clinical studies found maternal obesity associated with increased neural tube defect risk: a 1.7-fold rise for moderately obese women and over a 3-fold risk for severely obese women. In social science, a meta-analysis of 35 studies found no significant differences between multisystemic therapy and conventional social services for serious juvenile offenders.
- In ecology, evolution and conservation, meta-analysis was adopted in the early 1990s. Parmesan and Yohe used data on over 1,700 species to show significant range shifts and advancement of spring events; Benayas and colleagues found ecological restoration increased biodiversity and ecosystem services, though values remained lower in restored versus intact reference ecosystems.
- Methodological advances include fixed- and random-effects models, meta-regression, heterogeneity quantification (τ and I²), publication-bias tools, network meta-analysis, phylogenetic meta-analysis, and software such as MetaWin and OpenMEE. PRISMA includes a 27-item checklist for systematic review reporting.
Methods
- Design: narrative review/commentary on meta-analysis and research synthesis, not a new meta-analysis. The authors state they highlight principles, limitations, utility, and future developments.
- Data or sample: draws on published literature and illustrative examples across medicine, social sciences, and ecology/evolution/conservation; includes Box 1 on meta-analytic models and Table 1 on EEC milestones.
- Main analysis: conceptual and methodological synthesis; no primary data collection, systematic search protocol, or quantitative pooling is reported in the paper.
Limitations
- The paper states that meta-analysis and systematic reviews can highlight evidence deficiencies but cannot overcome them; publication bias and research bias may be suspected and sometimes estimated but cannot truly be corrected; selective or incomplete reporting can constrain synthesis.
- The paper states that too many published meta-analyses are of low quality, the term “meta-analysis” is frequently misapplied, and vote-counting persists despite being discredited; it also notes concerns about academic inequality and the “research parasites” controversy.
- The text makes evident that, as a narrative review, it does not provide a systematic search strategy, formal inclusion criteria, or a quantitative synthesis; its examples are illustrative rather than exhaustive.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
4 Valid · 4 Not valid · 37 Not checked · 33 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Similarly, a synthesis may be constrained by either selective or incomplete reporting 66,67 .”
What the sources say
- Not checked yetcites [66] Stewart, L. A . et al. Preferred reporting items for a systematic review and meta-analysis of individual participant data: the PRISMA-IPD statement. J. Am. Med. Assoc. 313, 1657–1665 (2015) source
- Partly supportedcites [67] Saldanha, I. J . et al. Evaluating Data Abstraction Assistant, a novel software application for data abstraction during systematic reviews: protocol for a randomized controlled trial. Syst. Rev. 5, 196 (2016) sourceExperimental — accuracy not yet measured
The Background (page unavailable) reports that omissions and errors during data abstraction changed summary meta-analytic results, supporting the general point that a synthesis can be affected by missing information. It describes information present in articles but omitted by abstractors, not selective or incomplete reporting by the studies themselves; that specific mechanism is unsupported. (LLM, conf=0.78)
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“When the number of studies is an order of magnitude greater than that, the inability of narrative processes to adequately summarize the outcomes increases exponentially 2,3 .”
What the sources say
- Not checked yetcites [2] Roberts, P. D ., Stewart, G. B. & Pullin, A. S. Are review articles a reliable source of evidence to support conservation and environmental management? A comparison with medicine. Biol. Conserv. 132, 409–423 (2006) source
- Partly supportedcites [3] Bastian, H ., Glasziou, P . & Chalmers, I. Seventy-five trials and eleven systematic reviews a day: how will we ever keep up? PLoS Med. 7, e1000326 (2010) sourceExperimental — accuracy not yet measured
Our copy supports the qualitative half of the claim: the 'How Close Are We to Archie Cochrane's Goal?' section cites Goldschmidt and Mulrow for 'how great the potential is for error in reviews of health literature that were not conducted systematically' and states that 'the staple of medical literature synthesis remains the non-systematic narrative review', while the same section (Figure 4 discussion) and 'Where to Now?' document the growth from ~14 trials a day in 1979 to 75 trials and 11 systematic reviews a day, with 'the task keeps increasing in size and complexity' ('Where Are We Now?'). What the source does NOT support is the quantitative assertion attributed to it: nowhere does our copy say the inability of narrative processes to summarise outcomes 'increases exponentially', nor tie it to study counts an order of magnitude beyond a dozen — the only dozen-scale figure in the paper is the median number of trials per systematic review (six to 16, Cochrane reviews averaging over 12), which is offered as a measure of reviewing workload and coverage, not of narrative-review failure. The exponential scaling is therefore the citing authors' characterisation, not this paper's. (LLM, conf=0.72)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“However, as the number of publications and research results has mushroomed across scientific fields, the difficulties encountered in carrying out narrative reviews to identify and summarize evidence in a transparent manner has become increasingly apparent 1 .”
What the sources say
- Not checked yetcites [1] Jennions, M. D ., Lortie, C. J. & Koricheva, J. in The Handbook of Meta-analysis in Ecology and Evolution (eds Koricheva, J . et al.) Ch. 23, 364–380 (Princeton Univ. Press, 2013) source
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, in many cases of the resolution of scientific problems, when the results of an individual study are combined with those of other studies, the resulting synthesis provides a more general and complete picture of the questions being addressed.”
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A statement the paper makes itself — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Sometimes, the striking results of initial studies are not confirmed by those of subsequent studies or by syntheses of a body of research.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
15 more claims
Create a free account to see all 78 claims and to verify the 60 references still open.
- It may state two separate claims as one. 32
- Probably not a substantial claim. 48
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The trade-off with our method is that it does not allow for determining the magnitude of the effect of the practices33–36 but only the proportion of impacts33,37 as positive, negative or neutral.”
From Farming practices to enhance biodiversity across biomes: a systematic review (2024) · cites this paper as [36]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Besides, publication biases that have been documented in several scientific fields90,91 can also affect ML-based science.92,93”
From Leakage and the reproducibility crisis in machine-learning-based science (2023) · cites this paper as [91]
Experimental — accuracy not yet measuredIn the citing paper
“Moreover, since a meta-analysis can be conducted only when sufficient and appropriate quantitative data from original studies are available, the number of published meta-analyses about a certain topic is an indicator of the overall understanding of the subject, allowing us to reveal research gaps (25, 26).”
From Agricultural diversification promotes multiple ecosystem services without compromising yield (2020) · cites this paper as [25]
In the citing paper
“Meta-analysis optimises statistical power for detecting treatment effects by combining data from multiple similar studies [22, 23].”
From Does the growing of Bt maize change abundance or ecological function of non-target animals compared to the growing of non-GM maize? A systematic review (2022) · cites this paper as [23]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Box 1: Meta-analytic models and heterogeneity
“𝜀 is the within-study sampling error (solid black lines), the deviation from the study-specific mean.”
§ Box 1: Meta-analytic models and heterogeneity
“the solid black lines are deviations from the overall mean (panels a,b) or slope (panels c,d)”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Box 1: Meta-analytic models and heterogeneity
“The assumption of the fixed-effect meta-regression is that a moderator(s) can explain all between-study heterogeneity”
§ Box 1: Meta-analytic models and heterogeneity
“the blue distributions the between-study variance (panels a-d).”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Box 1: Meta-analytic models and heterogeneity
“Two basic models of meta-analysis are the fixed-effect model (Fig 1a) and the random-effects model (Fig 1b) 6 .”
§ Box 1: Meta-analytic models and heterogeneity
“Figure 1. A variety of charts and plots common in meta-analysis. a. PRISMA diagram 20 , b. a forest plot showing overall means based on a fixed-effect model and random-effects model”
Citations
Cites
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: the PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & for the PRISMA Group (2009) · BMJ · doi:10.1136/bmj.b2535 · Cited by 83,823
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & The PRISMA Group (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000097 · Cited by 64,042 · cited 1×
“Procedural protocols for systematic reviews are well established especially in the medical and social sciences (Preferred Reporting Items for Systematic Reviews, or PRISMA; www.prisma-statement.org) 20 .”
Bias in meta-analysis detected by a simple, graphical test
Egger, M., Smith, G. D., Schneider, M., & Minder, C. (1997) · BMJ · doi:10.1136/bmj.315.7109.629 · Cited by 56,722
Preferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement.
(2009) · PubMed Central · Cited by 44,859
Meta-analysis in clinical trials
DerSimonian, R., & Laird, N. (1986) · Controlled Clinical Trials · doi:10.1016/0197-2456(86)90046-2 · Cited by 39,552
Quantifying heterogeneity in a meta‐analysis
Higgins, J. P. T., & Thompson, S. G. (2002) · Statistics in Medicine · doi:10.1002/sim.1186 · Cited by 36,763
PRISMA-P Group, Moher, D., Shamseer, L., Clarke, M., Ghersi, D., Liberati, A., Petticrew, M., Shekelle, P., & Stewart, L. A. (2015) · Systematic Reviews · doi:10.1186/2046-4053-4-1 · Cited by 26,223
Conducting Meta-Analyses in R with the metafor Package
Viechtbauer, W. (2010) · Journal of Statistical Software · doi:10.18637/jss.v036.i03 · Cited by 20,464 · cited 1×
“…the use of machine learning to screen studies 82 for inclusion in systematic reviews and meta-analyses for large reviews, increasingly sophisticated software and models for complex meta-regression 53,83 , and integration of meta-analysis with decision support in medicine and other domains 84 .”
Wang, G., Ma, R., Qiao, G., Wada, K., Aizawa, Y., & Satoh, T. (2015) · Integrative Medicine International · doi:10.1159/000368408 · Cited by 18,657
Duval, S., & Tweedie, R. (2000) · Biometrics · doi:10.1111/j.0006-341x.2000.00455.x · Cited by 14,099
Research works printed but not linked: 3
- Hedges, L. & Olkin, I. Statistical Methods for Meta-analysis (Academic Press, 1985)
- Higgins, J. P. T . & Green, S. (eds) Cochrane Handbook for Systematic Reviews of Interventions: Version 5.1.0 (Wiley, 2011).This large collaborative work provides definitive guidance for the production of systematic reviews in medicine and is of broad interest for methods development outside the medical field.
- Cooper, H. M ., Hedges, L. V . & Valentine, J. C. (eds) The Handbook of Research Synthesis and Meta-analysis 2nd edn (Russell Sage Foundation, 2009).This book is an important compilation that builds on the ground-breaking first edition to set the standard for best practice in meta-analysis, primarily in the social sciences but with applications to medicine and other fields.
Cited by
Items 501–600 of 1,000
Hu, S., 童楷程, Chen, W., Wang, Y., Wang, Y., & Yang, L. (2022) · Field Crops Research · doi:10.1016/j.fcr.2022.108562 · Cited by 28
Zhang, H., & Xie, P. (2023) · The Science of The Total Environment · doi:10.1016/j.scitotenv.2023.167018 · Cited by 28
Mupepele, A., Bruelheide, H., Dauber, J., Kruess, A., Potthast, T., Wägele, W. J., & Klein, A. (2019) · Basic and Applied Ecology · doi:10.1016/j.baae.2019.04.001 · Cited by 28
Social incentive factors in interventions promoting sustainable behaviors: A meta-analysisPreprint
Nguyen-Van, P., Stenger, A., & Tiet, T. T. (2021) · RePEc: Research Papers in Economics · Cited by 29
Yin, M., Gao, X., Kuang, W., & Tenuta, M. (2023) · Geoderma · doi:10.1016/j.geoderma.2023.116538 · Cited by 33
GHG emissions and energy consumption of residential buildings—a systematic review and meta-analysis
Fan, Y., & Fang, C. (2023) · Environmental Monitoring and Assessment · doi:10.1007/s10661-023-11515-z · Cited by 27
“It is an effective tool to comprehend the overall situation and fundamental trends of research topics (Castrejon-Campos, 2022; Gurevitch et al., 2018).”
MethodFacilitating comparable research in seedling functional ecology
Winkler, D. E., Garbowski, M., Kožić, K., Ladouceur, E. R., Larson, J. E., Martin, S., Rosche, C., Roscher, C., Slate, M. L., & Korell, L. (2024) · Methods in Ecology and Evolution · doi:10.1111/2041-210x.14288 · Cited by 30
“However, meta-analyses or syntheses are only possible if appropriate amounts of quantitative data are available employing similar methods with standardised measurements (Gerstner et al., 2017; Gurevitch et al., 2018; Koricheva et al., 2013).”
MethodMeta-analysis reveals weak associations between reef fishes and corals
Muruga, P., Siqueira, A. C., & Bellwood, D. R. (2024) · Nature Ecology & Evolution · doi:10.1038/s41559-024-02334-7 · Cited by 33
“We adopted a rigorous meta-analytic approach 29 , first by systematically screening 4,653 unique articles (Extended Data Fig.”
InfluentialA Meta-Analysis of Mathematics Interventions: Examining the Impacts of Intervention Characteristics
Rojo, M., Gersib, J. A., Powell, S. R., Shen, Z., King, S. G., Akther, S. S., Arsenault, T. L., Bos, S. E., Lariviere, D. O., & Lin, X. (2024) · Educational Psychology Review · doi:10.1007/s10648-023-09843-0 · Cited by 48
“Rather, such findings often indicate inconclusive results, attributable to limitations such as a smaller number of studies or methodological differences (Gurevitch et al., 2018).”
Wan, L., Liu, G., Sun, J. L., Ma, J., Cheng, H. L., Yu, S., Du, C., & Su, X. (2024) · The Science of The Total Environment · doi:10.1016/j.scitotenv.2024.172006 · Cited by 67
Tariq, M., Liu, Y., Ali, R., Shoukat, C. A., Aftab, Q., Lu, J., & Zhang, Y. (2024) · The Science of The Total Environment · doi:10.1016/j.scitotenv.2024.175354 · Cited by 33
Fifteen Years of Research on Customer Loyalty Formation: A Meta-Analytic Structural Equation Model
So, K. K. F., Yang, Y., & Li, X. R. (2024) · Cornell Hospitality Quarterly · doi:10.1177/19389655241276506 · Cited by 84
“In an article published in Nature , Gurevitch et al. (2018) noted that, since the term meta-analysis and modern approaches to research synthesis were introduced in the 1970s, this statistical method has revolutionized research in many scientific disciplines, providing a way to establish…”
Neidhardt, H. (2020) · Plants People Planet · doi:10.1002/ppp3.10122 · Cited by 27
Liver Transplantation for Propionic Acidemia: Evidence From a Systematic Review and Meta-analysis
Zhou, G., Jiang, Y., Wu, S., Kong, Y., Sun, L., & Zhu, Z. (2020) · Transplantation · doi:10.1097/tp.0000000000003501 · Cited by 28
“Nonetheless, in a therapeutic context, especially for ultra-orphan diseases in which most studies were clinical reports, excluding them from the systematic review would increase the risk of bias and reduce efficiency.(47) Second, the interpretation of endpoints is limited by the different lengths of the follow-up peri…”
BackgroundFeng, C., Gu, R., Li, T., Wang, L., Zhang, Z., Luo, W., & Eickhoff, S. B. (2021) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2021.09.041 · Cited by 34
Gong, C., Tan, Q., Liu, G., & Xu, M. (2022) · CATENA · doi:10.1016/j.catena.2022.106232 · Cited by 29
Ras, J., Kengne, A. P., Smith, D. L., Soteriades, E. S., & Leach, L. L. (2023) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph20042816 · Cited by 44
Meta‐analysis: microbial inactivation in milk using pulsed electric field
Nabilah, U. U., Sitanggang, A. B., Dewanti‐Hariyadi, R., Sugiarto, A. T., & Purnomo, E. H. (2022) · International Journal of Food Science & Technology · doi:10.1111/ijfs.15942 · Cited by 26
The growth‐promoting effects of exercise in finfish: A systematic review and meta‐analysis
Rodgers, E. M., & Isaza, D. F. G. (2023) · Reviews in Aquaculture · doi:10.1111/raq.12879 · Cited by 30
Zhang, L., Meng, F., Zhang, X., Gao, Q., & Li, Y. (2024) · Agricultural Water Management · doi:10.1016/j.agwat.2024.109043 · Cited by 26
Kakos, N. J., Lufler, R. S., Cyr, B., Zwirner, C., Hurley, E., Heinrich, C., & Wilson, A. B. (2024) · Advances in Health Sciences Education · doi:10.1007/s10459-024-10373-9 · Cited by 37
“Typical limitations of meta-analyses include publication bias, research bias (e.g., sampling or interpretation bias), and reliance on empirical data that may be fraught with selective or incomplete data reporting (Gurevitch et al., 2018).”
Teng, X., Huang, D., Zhi, Y., Li, Y., Dong, D., Wu, X., Wang, Y., Jiang, Z., Huang, H., Tang, Y., et al. (2025) · Scientific Reports · doi:10.1038/s41598-025-18170-z · Cited by 39
“As a quantitative review, a meta-analysis could address this issue by calculating the overall effect sizes and summarizing the results of the main articles to reveal sources of variation across studies 22 .”
Rodgers, E. M., Poletto, J. B., Isaza, D. F. G., Eenennaam, J. P. V., Connon, R. E., Todgham, A. E., Seesholtz, A. M., Heublein, J. C., Cech, J. J., Kelly, J. T., et al. (2019) · Conservation Physiology · doi:10.1093/conphys/coz035 · Cited by 30
“By combining data from numerous sources, meta-analyses also generate greater explanatory power and may reveal effects not detected by individual studies (Mulrow, 1994; Gurevitch et al., 2018).”
BackgroundSampayo-Cordero, M., Miguel-Huguet, B., Pardo-Mateos, A., Malfettone, A., Pérez-García, J. M., Llombart‐Cussac, A., Cortés, J., Moltó‐Abad, M., Muñoz-Delgado, C., Perez-Quintana, M., et al. (2019) · Orphanet Journal of Rare Diseases · doi:10.1186/s13023-019-1202-6 · Cited by 31
Introduced non-native mangroves express better growth performance than co-occurring native mangroves
Fazlioglu, F., & Chen, L. (2020) · Scientific Reports · doi:10.1038/s41598-020-60454-z · Cited by 33
Negative impacts of plant diversity loss on carbon sequestration exacerbate over time in grasslands
Wang, C., Tang, Y., Li, X., Zhang, W., Zhao, C., & Li, C. (2020) · Environmental Research Letters · doi:10.1088/1748-9326/abaf88 · Cited by 28
Saqr, M., & López‐Pernas, S. (2021) · International Journal of Computer-Supported Collaborative Learning · doi:10.1007/s11412-021-09356-4 · Cited by 36
“Meta-analysis provides the means for assessing the consistency of results through estimation of heterogeneity (i.e., variability within results) as well as predictive intervals (i.e., the extent to which future results are expected to be similar) (Gurevitch et al., 2018).”
BackgroundOladipupo, S., Wilson, A. E., Hu, X. P., & Appel, A. (2022) · Insects · doi:10.3390/insects13020117 · Cited by 23
“, effect size) from each study and converts this estimate to a standardized metric [25,26].”
BackgroundGuang, X., Jian, C., Qu, M., Zhang, J., Liu, H., Zhao, Y., & Huang, B. (2022) · The Science of The Total Environment · doi:10.1016/j.scitotenv.2022.153562 · Cited by 24
A Global Synthesis of the Effects of Livestock Activity on Hydrological Processes
Eldridge, D. J., Ding, J., & Travers, S. K. (2022) · Ecosystems · doi:10.1007/s10021-022-00746-9 · Cited by 38
The impacts of agroforestry on soil multi-functionality depending on practices and duration
Li, S., Gong, S., Hou, Y., Li, X., & Wang, C. (2022) · The Science of The Total Environment · doi:10.1016/j.scitotenv.2022.157438 · Cited by 33
Ecological non-equilibrium and biological conservation
Oró, D., & Martínez‐Abraín, A. (2023) · Biological Conservation · doi:10.1016/j.biocon.2023.110258 · Cited by 35
Agra, J. U. M., Cornelissen, T., Junior, A. B. V., & Callisto, M. (2023) · Oikos · doi:10.1111/oik.10186 · Cited by 28
An assessment of the quality of current clinical meta-analyses
Hameed, I., Demetres, M. R., Tam, D. Y., Rahouma, M., Khan, F., Wright, D., Mages, K. C., DeRosa, A. P., Nelson, B. B., Pain, K. J., et al. (2020) · BMC Medical Research Methodology · doi:10.1186/s12874-020-00999-9 · Cited by 26
The role of nitric oxide in portal hypertensive systemic and portal vascular pathology.
Hartleb, M., Michielsen, P. P., & Dziurkowska-Marek, A. (1997) · PubMed · Cited by 26
The Cooperation Databank: Machine-Readable Science Accelerates Research SynthesisPreprint
Spadaro, G., Tiddi, I., Columbus, S., Jin, S., ten Teije, A., & Balliet, D. P. (2020) · doi:10.31234/osf.io/rveh3 · Cited by 26
The Future of Predictive Ecology
Elliott‐Graves, A. (2019) · Philosophical Topics · doi:10.5840/philtopics20194714 · Cited by 26
“In fact, the high frequency of predictive failure has led some ecologists to question the status of their discipline as truly scientific (Hayes and Barry 2007; Valéry et al. 2013).”
BackgroundSimões, L. H. P., Guillemot, J., Ronquim, C. C., Weidlich, E. W. A., Muys, B., Fuza, M. S., de Lima, R. A. F., & Brancalion, P. H. S. (2024) · Global Change Biology · doi:10.1111/gcb.17269 · Cited by 33
“As no estimates of within-study variance were reported for most of the studies, we used the total sampling area surveyed in each study as a metric of sampling size to weigh individual effect size in the model (Gurevitch et al., 2018 ).”
MethodXie, Y., Zeng, F., & Yang, Y. (2024) · Acta Psychologica · doi:10.1016/j.actpsy.2024.104486 · Cited by 57
Investigation of agricultural nutrient removal by ecological ditches using meta-analysis
Bai, J., Qin, Y., Zhao, J., & Song, Y. (2024) · Agriculture Ecosystems & Environment · doi:10.1016/j.agee.2024.109401 · Cited by 30
Arsenic toxicity exacerbates China’s groundwater and health crisis
Liu, X., Yue, F., Wong, W. W., Lin, S.-C., Guo, T.-L., & Li, S. (2025) · Environment International · doi:10.1016/j.envint.2025.109435 · Cited by 25
Drivers of piscivory in a globally distributed aquatic predator (brown trout): a meta-analysis
Sánchez‐Hernández, J. (2020) · Scientific Reports · doi:10.1038/s41598-020-68207-8 · Cited by 28
Leclercq, V., Hiligsmann, M., Parisi, G., Beaudart, C., Tirelli, E., & Bruyère, O. (2020) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2020.08.011 · Cited by 24
“To provide reliable conclusions, a robust methodology for conducting SRs/ MAs is therefore essential [3].”
MethodKonno, K., Cheng, S. H., Eales, J. F., Frampton, G. K., Kohl, C., Livoreil, B., Macura, B., O’Leary, B. C., Randall, N., Taylor, J. J., et al. (2020) · Environmental Science & Policy · doi:10.1016/j.envsci.2020.08.021 · Cited by 29
“While one of the major justifications for conducting evidence reviews is to collate primary studies for evidence users, as more and more are published, the problem of large volumes of literature extends to evidence reviews themselves (Gurevitch et al., 2018; Haddaway et al., 2015).”
BackgroundMethodMeta-analysis reveals materiomic relationships in major ampullate silk across the spider phylogeny
Craig, H. C., Piorkowski, D., Nakagawa, S., Kasumovic, M. M., & Blamires, S. J. (2020) · Journal of The Royal Society Interface · doi:10.1098/rsif.2020.0471 · Cited by 29
“Accordingly, meta-analyses have increasingly been used to understand the nature of the relationships among covarying biological traits across species [35,36].”
BackgroundSelf-citationDevesa-Peiró, A., Sebastián-León, P., Pellicer, A., & Díaz-Gimeno, P. (2021) · Molecular Human Reproduction · doi:10.1093/molehr/gaab011 · Cited by 30
“…arise from low sample sizes, sample heterogeneity, undetected confounding variables in gene expression experimental procedures or differing experimental designs and data analysis protocols (Gurevitch et al. , 2018; Suhorutshenko et al. , 2018; Craciunas et al. , 2019; Devesa-Peiro et al. , 2020).”
Burgess, B. J., Purves, D. W., Mace, G. M., & Murrell, D. J. (2021) · Global Change Biology · doi:10.1111/gcb.15630 · Cited by 28
Allen, W. J., Sapsford, S. J., & Dickie, I. A. (2021) · New Phytologist · doi:10.1111/nph.17455 · Cited by 25
“…study), predicting plant–soil feedback effects (De Long et al., 2019; Wandrag et al., 2020), translating findings from the laboratory to the field (Heinze et al., 2016; Forero et al., 2019), unravelling context-dependency along abiotic and biotic gradients (Smith-Ramesh & Reynolds, 2017; Beals…”
BackgroundPour, M. J., Mesrābādi, J., & Asarian, M. (2021) · Online Information Review · doi:10.1108/oir-01-2021-0011 · Cited by 43
Effects of elevated CO2 concentration on CH4 and N2O emissions from paddy fields: A meta-analysis
Yu, H., Wang, T., Huang, Q., Song, K., Zhang, G., Ma, J., & Xu, H. (2021) · Science China Earth Sciences · doi:10.1007/s11430-021-9848-2 · Cited by 31
“We excluded macroscopic evaluation or review papers (Gurevitch et al., 2018).”
BackgroundBieber, B. V., Vyas, D. K., Koltz, A. M., Burkle, L. A., Bey, K. S., Guzinski, C., Murphy, S. M., & Vidal, M. C. (2022) · Functional Ecology · doi:10.1111/1365-2435.14197 · Cited by 28
Meta-QTL Analysis for Yield Components in Common Bean (Phaseolus vulgaris L.)
Arriagada, O., Arévalo, B., Cabeza, R. A., Carrasco, B., & Schwember, A. R. (2022) · Plants · doi:10.3390/plants12010117 · Cited by 25
“MQTL analysis is a genetic technique that allows the integration of the information available from linkage analysis and identification of QTL to refine the QTL locations by narrowing the confidence intervals [85].”
MethodGlobal climate impacts of agriculture: A meta-regression analysis of food production
Gaugler, T., Stöckl, S., & Rathgeber, A. (2020) · Journal of Cleaner Production · doi:10.1016/j.jclepro.2020.122575 · Cited by 28
Adoption of protocols to improve quality of medical research
Montagna, E., Zaia, V., & Laporta, G. Z. (2019) · Einstein (São Paulo) · doi:10.31744/einstein_journal/2020ed5316 · Cited by 36
“Since they are able to provide precise responses, they are considered the studies with the highest level of evidence.(3) The base of the naturally will have a large amount of data.”
BackgroundLi, L., Liu, T., Li, J., Yang, Y., Liu, H., & Zhang, P. (2024) · Aquaculture Reports · doi:10.1016/j.aqrep.2024.102121 · Cited by 34
Soil moisture estimation with microwave remote sensing: a systematic review and meta-analysis
Xu, N., Daccache, A., Ahmadi, A., Houtz, D., & Puig, F. (2025) · International Journal of Digital Earth · doi:10.1080/17538947.2025.2468413 · Cited by 31
“Therefore, we applied a rigorous protocol for systematic bibliographic review (Ahmadi et al. 2022, 2021; Gurevitch et al. 2018) with the following research question: ‘ How accurate microwave remote sensing is in estimating SM content?’.”
Zhu, P., Zheng, X., ., G. X., Chen, G., Lü, Z., & Gurr, G. M. (2019) · Pest Management Science · doi:10.1002/ps.5719 · Cited by 30
Mangrove research in Colombia: Temporal trends, geographical coverage and research gaps
Castellanos-Galindo, G. A., Kluger, L. C., Camargo, M. A., Kintz, J. R. C., Pineda, J. E. M., Blanco, J. F., & Wolff, M. (2020) · Estuarine Coastal and Shelf Science · doi:10.1016/j.ecss.2020.106799 · Cited by 27
Yang, J., Kang, Y., Zhao, W., Jiang, J., Jiang, Y., Zhao, B. Q., Jiao, M., Yuan, B., Zhao, J., & Ma, B. (2021) · Bioactive Materials · doi:10.1016/j.bioactmat.2021.08.016 · Cited by 31
The effect of natural infrastructure on water erosion mitigation in the Andes
Vanacker, V., Molina, A., Rosas, M. A., Bonnesoeur, V., Román‐Dañobeytia, F., Ochoa-Tocachi, B. F., & Buytaert, W. (2022) · SOIL · doi:10.5194/soil-8-133-2022 · Cited by 35
Lunny, C., Thirugnanasampanthar, S. S., Kanji, S., Ferri, N., Pieper, D., Whitelaw, S., Tasnim, S., Nelson, H. H., Reid, E. K., Zhang, J. H., et al. (2022) · BMC Medical Research Methodology · doi:10.1186/s12874-022-01750-2 · Cited by 27
“Statistical advances of current interest include multiple imputations to model missing data, meta-regression and model selection, living systematic reviews, and network meta-analyses [45, 46], to name a few.”
BackgroundMethodCarrillo-Reche, J., Noc, T. L., van Apeldoorn, D. F., Juventia, S. D., Westhoek, A., Shanmugam, S., Kristensen, H. L., Hondebrink, M., Himanen, S., Kivijärvi, P., et al. (2023) · Agriculture Ecosystems & Environment · doi:10.1016/j.agee.2023.108564 · Cited by 32
Zhao, Z., Hui, C., Peng, S., Yi, S., Li, Z., Reddy, G. V. P., & van Kleunen, M. (2023) · Journal of Applied Ecology · doi:10.1111/1365-2664.14485 · Cited by 39
Doing Meta-Analysis with R - A Hands-On Guide
Lortie, C. J. (2022) · Journal of Statistical Software · doi:10.18637/jss.v102.b02 · Cited by 23
Wei, X., Wu, F., Hědenec, P., Yue, K., Peng, Y., Yang, J., Zhang, X., & Ni, X. (2022) · Fundamental Research · doi:10.1016/j.fmre.2022.01.029 · Cited by 26
“(4) ) were calculated by random mixed effect model that fitted ln 𝑅 as the response variable and included study identity as a random effect factor to include both the within-study variance and an extra variance component to account for between-study variance [36] .”
MethodOptimizing feeding frequencies in fish: A meta-analysis and machine learning approach
Huang, M. H., Zhou, Y., Yang, X., Gao, Q., Chen, Y.-N., Ren, Y.-C., & Dong, S. (2024) · Aquaculture · doi:10.1016/j.aquaculture.2024.741678 · Cited by 23
Khan, A., Azeem, I., Jing, H., Chen, Y., Yuan, Y., Shah, T., Adeel, M., Shakoor, N., Asghar, R. M. A., Cao, W., et al. (2024) · Soil and Tillage Research · doi:10.1016/j.still.2024.106430 · Cited by 28
Assessing risk of bias in toxicological studies in the era of artificial intelligence
Härtung, T., Hoffmann, S., & Whaley, P. (2025) · Archives of Toxicology · doi:10.1007/s00204-025-03978-5 · Cited by 33
“• Meta-analysis of publication bias: Gurevitch et al. (2018) discuss the importance of considering bias in meta-analyses and systematic reviews, which is relevant to the section's discussion on how bias affects evidence synthesis and decision-making.”
Vanlinthout, L. E., Geniets, B., Driessen, J. J., Saldien, V., Lapré, R., Berghmans, J. M., Uwimpuhwe, G., & Hens, N. (2019) · Pediatric Anesthesia · doi:10.1111/pan.13806 · Cited by 29
How does tillage intensity affect chemical soil health indicators? A United States meta‐analysis
Nunes, M. R., Karlen, D. L., Moorman, T. B., & Cambardella, C. A. (2020) · Agrosystems Geosciences & Environment · doi:10.1002/agg2.20083 · Cited by 29
The effects of adolescent prosocial behavior interventions: a meta-analytic review
Shin, J., & Lee, B. (2021) · Asia Pacific Education Review · doi:10.1007/s12564-021-09691-z · Cited by 32
Xin, C., & Hsu, Y. (2022) · Interactive Learning Environments · doi:10.1080/10494820.2022.2072898 · Cited by 31
“And the results were not standardised across studies for direct comparison as recommended by Lipsey and Wilson (2001), which makes the sample size for the meta-analysis so poor that it is more susceptible to the possibility of publication bias(Gurevitch et al., 2018).”
Effects of nitrogen and phosphorus on chlorophyll a in lakes of China: a meta-analysis
Yu, G., Zhang, S., Qin, W., Guo, Y., Zhao, R., Liu, C., Wang, C., Li, D., & Wang, Y. (2022) · Environmental Research Letters · doi:10.1088/1748-9326/ac7d64 · Cited by 25
Xun, C., Deng, H., Zhao, J., Ge, L., & Hu, Z. (2022) · Frontiers in Immunology · doi:10.3389/fimmu.2022.972247 · Cited by 26
“A systematic review is a type of literature review that aims to address a specific research question by gathering, selecting, analyzing, and synthesizing all the relevant evidence (11, 12).”
BackgroundTowards Nuts and Bolts of Conducting Literature Review: A Typology of Literature Review
Taherdoost, H. (2023) · Electronics · doi:10.3390/electronics12040800 · Cited by 56
Xu, G., & Tao, M. (2023) · Frontiers in Environmental Science · doi:10.3389/fenvs.2022.1051636 · Cited by 27
Research Methods in Learning Design and Technology
Romero‐Hall, E. (2020) · Routledge eBooks · doi:10.4324/9780429260919 · Cited by 25
Lewis, M. L., Mathur, M. B., VanderWeele, T. J., & Frank, M. C. (2022) · Royal Society Open Science · doi:10.1098/rsos.211499 · Cited by 30
“Statistical meta-analysis [3,4] has long been considered a gold standard methodology for estimating effect sizes.”
MethodYitbarek, S., Chen, K., Celestin, M., & McCary, M. A. (2023) · Ecological Applications · doi:10.1002/eap.2869 · Cited by 37
Jinmin, Z., & Qi, F. (2023) · Frontiers in Psychology · doi:10.3389/fpsyg.2023.1270642 · Cited by 37
“Then, quantitative synthesis was chosen to examine the relationship between learning flow and academic performance in a series of tasks, environments and populations (Gurevitch et al., 2018).”
MethodWastewater treatment in lagoons: A systematic review and a meta-analysis
Calabrò, P. S., Pangallo, D., & Zema, D. A. (2024) · Journal of Environmental Management · doi:10.1016/j.jenvman.2024.120974 · Cited by 34
Xiang, Y., Rillig, M. C., Peñuelas, J., Nizzetto, L., Sardans, J., Jian, L., Zhang, J., Li, R. Q., Liu, Y., Luo, Y., et al. (2025) · Environmental Science & Technology · doi:10.1021/acs.est.5c12883 · Cited by 24
Norberg, J., Blenckner, T., Cornell, S., Petchey, O. L., & Hillebrand, H. (2022) · Ecology Letters · doi:10.1111/ele.13984 · Cited by 27
“If the theory (or the prediction derived from it) allows quantitative tests, the systematic review can be extended towards a quantitative synthesis in the form of meta- analysis to explicitly test how general the prediction is (Hillebrand & Gurevitch, 2016; Gurevitch et al., 2018).”
BackgroundMethodInfluentialWillmott, T. J., Mathew, A., Saleme, P., & Rundle-Thiele, S. R. (2022) · Trauma Violence & Abuse · doi:10.1177/15248380221078891 · Cited by 29
“Data were inductively analysed by the lead reviewer and cross-checked by another member of the review team using a narrative evidence synthesis approach permitting a more nuanced and fine-grained examination of included studies (Gurevitch et al., 2018).”
MethodDecision Tree for Key Comparisons
Possolo, A. M., Koepke, A., Newton, D., & Winchester, M. R. (2021) · Journal of Research of the National Institute of Standards and Technology · doi:10.6028/jres.126.007 · Cited by 34
Effects of Mobile Learning in English Language Learning: A Meta-Analysis and Research Synthesis
Garzón, J., Λαμπρόπουλος, Γ., & Burgos, D. (2023) · Electronics · doi:10.3390/electronics12071595 · Cited by 63
“Concerning the study conducted by Chen [27], the inclusion of non-peerreviewed articles did not allow the author to guarantee the quality of the conclusions of the meta-analysis [29].”
BackgroundA meta-analysis of disturbance caused by drones on nesting birds
de Leija, A. C., Mirzadi, R., Randall, J., Portmann, M., Mueller, E., & Gawlik, D. E. (2023) · Journal of Field Ornithology · doi:10.5751/jfo-00259-940203 · Cited by 22
Yin, P., Zhang, C., Du, T., Yi, S., Yu, L., Tian, F., Chen, W., & Zhai, Q. (2023) · Food Research International · doi:10.1016/j.foodres.2023.112981 · Cited by 23
Priyadarshana, T. S., & Slade, E. M. (2023) · Journal of Animal Ecology · doi:10.1111/1365-2656.13965 · Cited by 38
The power of data synthesis to shape the future of the restoration community and capacity
Ladouceur, E. R., & Shackelford, N. (2020) · Restoration Ecology · doi:10.1111/rec.13251 · Cited by 25
Meta‐Analysis of Urban Non‐Point Source Pollution From Road and Roof Runoff Across China
Wang, Y., Li, C., Qiao, J., Hu, Y., Zhang, Q., Yin, J., & Slater, L. (2025) · Earth's Future · doi:10.1029/2024ef005296 · Cited by 22
“Meta‐analyses have evolved as a key tool to facilitate rapid scientific progress by quantifying what is known and identifying what is unknown (Gurevitch et al., 2018).”
Ricolfi, L., Vendl, C., Bräunig, J., Taylor, M. D., Hesselson, D., Neely, G. G., Lagisz, M., & Nakagawa, S. (2024) · Environment International · doi:10.1016/j.envint.2024.108860 · Cited by 31
Self-citationYang, Y., Lagisz, M., Williams, C., Noble, D. W. A., Pan, J., & Nakagawa, S. (2024) · Methods in Ecology and Evolution · doi:10.1111/2041-210x.14377 · Cited by 36
“Quantitative synthesis of research findings has become increasingly important for guiding scientific research and informing evidence-based decision-making (Gurevitch et al., 2018).”
Self-citationThe psychology of rock climbing: A systematic review
Mangan, K., Andrews, K., Miles, B., & Draper, N. (2024) · Psychology of sport and exercise · doi:10.1016/j.psychsport.2024.102763 · Cited by 21
Recent Advances in Machine Learning Models for Predicting Toxicity of Inorganic Nanoparticles
Li, M., Li, Q.-Z., Zhao, Y., & Gao, X. (2025) · Chem & Bio Engineering · doi:10.1021/cbe.5c00048 · Cited by 28
Mustafa, A. T., & Alyozbaky, O. S. (2025) · Franklin Open · doi:10.1016/j.fraope.2025.100350 · Cited by 41
Root biomass allocation in southern temperate forests
Easdale, T. A., Richardson, S. J., Marden, M., England, J. R., Gayoso-Aguilar, J. A., Guerra-Carcamo, J., McCarthy, J. K., Paul, K. I., Schwendenmann, L., & Brandon, A. M. (2019) · Forest Ecology and Management · doi:10.1016/j.foreco.2019.117542 · Cited by 21
An assessment of statistical methods for non‐independent data in ecological meta‐analyses: Reply
Song, C., Peacor, S. D., Osenberg, C. W., & Bence, J. R. (2021) · Ecology · doi:10.1002/ecy.3578 · Cited by 22
Improved estimation of nitrogen dynamics in paddy surface water in China
Ruan, S., Zhuang, Y., Zhang, L., Li, S., Chen, J., Wen, W., Zhai, L., Liu, H., & Du, Y. (2022) · Journal of Environmental Management · doi:10.1016/j.jenvman.2022.114932 · Cited by 21
Papadimitropoulou, K., Riley, R. D., Dekkers, O. M., Stijnen, T., & le Cessie, S. (2022) · Research Synthesis Methods · doi:10.1002/jrsm.1592 · Cited by 23
“evidence has led to the development of the metaanalysis framework and its numerous methodological advances over the years.(5) The widespread adoption of meta-analysis by many research fields has increased the need for robust statistical tools for analysis.”
Background
Metadata
- Authors
- Gurevitch, J., Koricheva, J., Nakagawa, S., & Stewart, G.
- Affiliations
- Stony Brook University (US)
- Royal Holloway University of London (GB)
- Garvan Institute of Medical Research (AU)
- Newcastle University (GB)
- Corresponding author
- Gurevitch, Jessica
- Koricheva, Julia
- Nakagawa, Shinichi
- Stewart, Gavin
- Venue
- Nature
- Published
- 2018-03
- Volume
- 555
- Issue
- 7695
- Pages
- 175-182
- ISSN
- 0028-0836 (ISSN-L)
- 1476-4687
- OpenAlex
- W2791681873
- Semantic Scholar
- ac5ad1932e31c3ea141366e8e073d6d95869c4ef
- Cited by
- 1,709
- References
- 101
- Citations by year
- 2026: 177 · 2025: 230 · 2024: 231 · 2023: 199 · 2022: 234 · 2021: 235 · 2020: 204 · 2019: 130 · 2018: 64
- FWCI
- 31.20
- Type
- journal article
- Language
- en
- Open access
- green
- hdl.handle.net
- Licence
- cc-by-nc-nd
- www.springer.com/tdm (tdm)
- Locations
- Nature (publishedVersion)
- PubMed (publishedVersion)
- UNSWorks (UNSW Sydney) (submittedVersion, cc-by-nc-nd)
- RePEc: Research Papers in Economics (submittedVersion)
- Newcastle University ePrints (Newcastle Univesity)
- Funders
- National Science Foundation (1262402)
- Laura and John Arnold Foundation
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Keywords
- meta-analysis, evidence synthesis, history of meta-analysis, evolutionary biology, conservation biology, artificial intelligence, evidence-based practice
- MeSH terms
- Animals, Biological Science Disciplines, Ecology, Biological Evolution, Research, Meta-Analysis as Topic, History, 20th Century, History, 21st Century
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-04, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-04 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol and qwen3.8-omni-flash).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.