CoChat
Sign Up
Cited by 881Meta-analysis and systematic reviewsscientometrics and bibliometrics researchPsychology Research and Bibliometrics

Introduction, comparison, and validation of Meta‐Essentials : A free and simple tool for meta‐analysis

Open access
Suurmond, Robert, van Rhee, Henk, Hak, Tony·2017·Research Synthesis Methods
DOI 10.1002/jrsm.1260PMC PMC5725669

Published version of a preprint

Open this paper in CoChat — free accountOpen PDF

Abstract

We present a new tool for meta‐analysis, Meta‐Essentials , which is free of charge and easy to use. In this paper, we introduce the tool and compare its features to other tools for meta‐analysis. We also provide detailed information on the validation of the tool. Although free of charge and simple, Meta‐Essentials automatically calculates effect sizes from a wide range of statistics and can be used for a wide range of meta‐analysis applications, including subgroup analysis, moderator analysis, and publication bias analyses. The confidence interval of the overall effect is automatically based on the Knapp‐Hartung adjustment of the DerSimonian‐Laird estimator. However, more advanced meta‐analysis methods such as meta‐analytical structural equation modelling and meta‐regression with multiple covariates are not available. In summary, Meta‐Essentials may prove a valuable resource for meta‐analysts, including researchers, teachers, and students.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1002/jrsm.1260

Cites

16 of 16 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Comprehensive Meta-Analysis v.3 is considered highly reliable and valid in various systematic assessments.

    “To perform our analyses, we used Comprehensive Meta-Analysis v.3 [84], a program considered highly reliable and valid in various systematic assessments [85, 86].”

    From Does time management work? A meta-analysis (2021) · cites this paper as [86]

    The Meta-analysis tools section includes CMA among programs frequently used by scholars across disciplines. The Validation section reports agreement between Meta-Essentials and CMA v.2.041 on tested analyses, but it does not establish that CMA v.3 is highly reliable and valid across various systematic assessments. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 55

  1. [5]Ellis PD. The Essential Guide to Effect Sizes. Cambridge, UK: Cambridge University Press; 2010.
  2. [6]Aloe AM. An empirical investigation of partial effect sizes in meta‐analysis of correlational data. J Gen Psychol. 2014; 141: 47‐ 64. 24838020
  3. [7]Aloe AM, Becker BJ. An effect size for regression predictors in meta‐analysis. J Educ Behav Stat. 2012; 37: 278‐ 297.
  4. [10]Schmid CH, Stewart GB, Rothstein HR, Lajeunesse MJ, Gurevitch J. Software for Statistical Meta‐Analysis In: Koricheva J, Gurevitch J, Mengersen K, eds. Handbook of Meta‐Analysis in Ecology and Evolution. Princeton and Oxford: Princeton University Press; 2013.
  5. [11]Palmer TM, Sterne JAC. Meta‐analysis in Stata: an Updated Collection From the Stata Journal. Stata Press; 2016.
  6. [14]Review Manager (RevMan). [Computer Program]. Version 5.3. Copenhagen: The Nordic Cochrane Centre, The Cochrane Collaboration; 2014.
  7. [15]Schwarzer G. Meta: an R package for meta‐analysis. R News. 2007; 7: 40‐ 45.
  8. [16]Viechtbauer W. Conducting meta‐analyses in R with the metafor package. J Stat Softw. 2010; 36: 1‐ 48.
  9. [19]Higgins JPT, Green S (eds.), 2011 Cochrane handbook for systematic reviews of interventions [Updates March 2011], The Cochrane Collaboration http://handbook.cochrane.org.
  10. [20]DerSimonian R, Laird N, 1986 Meta‐analysis in clinical trials. Controlled clinical trials. 3802833
  11. [21]Chung Y, Rabe‐Hesketh S, Choi IH. Avoiding zero between‐study variance estimates in random‐effects meta‐analysis. Stat Med. 2013; 32: 4071‐ 4089. 23670939
  12. [24]Hardy RJ, Thompson SG. A likelihood approach to meta‐analysis with random effects. Stat Med. 1996; 15: 619‐ 629. 8731004
  13. [25]Thompson SG, Sharp SJ. Explaining heterogeneity in meta‐analysis: a comparison of methods. Stat Med. 1999; 18: 2693‐ 2708. 10521860
  14. [26]Deeks JJ. Issues in the selection of a summary statistic for meta‐analysis of clinical trials with binary outcomes. Stat Med. 2002; 21: 1575‐ 1600. 12111921
  15. [28]Cummings P. The relative merits of risk ratios and odds ratios. Arch Pediatr Adolesc Med. 2009; 163: 438‐ 445. 19414690
  16. [29]Sinclair JC, Bracken MB. Clinically useful measures of effect in binary analyses of randomized trials. J Clin Epidemiol. 1994; 47: 881‐ 889. 7730891
  17. [30]Zhang J, Yu KF. What's the relative risk? A method of correcting the odds ratio in cohort studies of common outcomes. JAMA. 1998; 280: 1690‐ 1691. 9832001
  18. [31]Localio AR, Margolis DJ, Berlin JA. Relative risks and confidence intervals were easily computed indirectly from multivariable logistic regression. J Clin Epidemiol. 2007; 60: 874‐ 882. 17689803
  19. [32]Daly LE. Confidence limits made easy: interval estimation using a substitution method. Am J Epidemiol. 1998; 147: 783‐ 790. 9554420
  20. [33]McNutt LA. Estimating the relative risk in cohort studies and clinical trials of common outcomes. Am J Epidemiol. 2003; 157: 940‐ 943. 12746247
  21. [35]Sanchez‐Meca J, Marin‐Martinez F, Sánchez‐Meca J, Marín‐Martínez F. Confidence intervals for the overall effect size in random‐effects meta‐analysis. Psychol Methods. 2008; 13: 31‐ 48. 18331152
  22. [37]Kiran A, Crespillo AP, Rahimi K. Graphics and statistics for cardiology: data visualisation for meta‐analysis. Heart. 2017; 103: 19‐ 23. 27810865
  23. [44]Hyndman RJ, Fan Y. Sample quantiles in statistical packages. Am Stat. 1996; 50: 361
  24. [45]Aboodarda SJ, Page PA, Behm DG. Muscle activation comparisons between elastic and isoinertial resistance: a meta‐analysis. Clin Biomech. 2016; 39: 52‐ 61. 27681867
  25. [46]Barreto NB, Hogg MA. Evaluation of and support for group prototypical leaders: a meta‐analysis of twenty years of empirical research. Soc Influ. 2017; 12: 41‐ 55.
  26. [47]Boot N, Nevicka B, Baas M. Subclinical symptoms of attention‐deficit/hyperactivity disorder (ADHD) are associated with specific creative processes. Personal Individ Differ. 2017; 114: 73‐ 81.
  27. [48]Burmester B, Leathem J, Merrick P. Subjective cognitive complaints and objective cognitive function in aging: a systematic review and meta‐analysis of recent cross‐sectional findings. Neuropsychol Rev. 2016; 26: 376‐ 393. 27714573
  28. [49]Dickinson T, Becerra R, Coombes J. Executive functioning deficits among adults with bipolar disorder (types I and II): a systematic review and meta‐analysis. J Affect Disord. 2017; 28501741
  29. [50]Graziani F, Gennai S, Lillo L, Petrini M, 2017 A systematic review and meta‐analysis of epidemiologic observational evidence on glycaemic control and risk of developing diabetes in healthy patietns. In: 18th International Congress SIdP. Italian Society of Periodontology and Implantology, Rimini, Italy.
  30. [51]Jones MK, Dickter B, Beard C, Perales R, Bunge EL. Meta‐analysis on cognitive behavioral treatment and behavioral intervention technologies for anxious youth: more than a BIT effective. Contemp Behav Health Care. 2016; 1: 1‐ 9.
  31. [52]Kavé G, Goral M. Word retrieval in connected speech in Alzheimer's disease: a review with meta‐analyses. Aphasiology. 2017; 1‐ 23.
  32. [53]Moery E, Calin‐Jageman RJ. Direct and conceptual replications of Eskine (2013): organic food exposure has little to no effect on moral judgments and prosocial behavior. Soc Psychol Personal Sci. 2016a; 7: 312‐ 319.
  33. [54]Moery E, Calin‐Jageman RJ. Direct and conceptual replications of Eskine (2013): organic food exposure has little to no effect on moral judgments and prosocial behavior. Soc Psychol Personal Sci. 2016b; 7: 1‐ 8.
  34. [56]Mummareddy N, Dewan MC, Mercier MR, Naftel RP, Iii JCW, Bonfield CM, Scoliosis in myelomeningocele: epidemiology, management, and functional outcome. 2017; 1– 10. 28452655
  35. [57]Richardson M, McEwan K, Maratos F, Sheffield D. Joy and calm: how an evolutionary functional model of affect regulation informs positive emotions in nature. Evol Psychol Sci. 2016; 2: 308‐ 320.
  36. [58]Schreglmann SR, Krauss JK, Chang JW, Bhatia KP, Kaegi G. Functional lesional neurosurgery for tremor—a protocol for a systematic review and meta‐analysis. BMJ Open. 2017; 7. PMC5623440 28487460
  37. [59]Sridharan K, Sivaramakrishnan G. Tranexamic acid in total hip arthroplasty: a recursive cumulative meta‐analysis of randomized controlled trials and assessment of publication bias. J Orthop. 2017; 14: 323‐ 328. 28559649 PMC5440691
  38. [61]van Halewijn G, Deckers J, Tay HY, van Domburg R, Kotseva K, Wood D. Lessons from contemporary trials of cardiovascular prevention and rehabilitation: a systematic review and meta‐analysis. Int J Cardiol. 2017; 232: 294‐ 303. 28094128
  39. [63]Vuong HG, Altibi AMA, Duong UNP, Hassell L. Prognostic implication of BRAF and TERT promoter mutation combination in papillary thyroid carcinoma––a meta‐analysis. Clin Endocrinol (Oxf). 2017a; 38: 42‐ 49. 28666074
  40. [64]Vuong HG, Altibi AMA, Duong UNP, et al. BRAF mutation is associated with an improved survival in glioma—a systematic review and meta‐analysis. Mol Neurobiol. 2017b. 28534272
  41. [65]Vuong HG, Duong UNP, Altibi AMA, et al. A meta‐analysis of prognostic roles of molecular markers in papillary thyroid carcinoma. Endocrine Connections. 2017c. PMC5424840 28219937
  42. [67]Rothstein H, Sutton A, Borenstein M. Publication Bias in Meta‐analysis: Prevention, Assessment and Adjustments. Chichester, UK: Wiley; 2006.
  43. [68]Lau J, Ioannidis JPA, Terrin N, Schmid CH, Olkin I. The case of the misleading funnel plot. BMJ. 2006; 333: 597‐ 600. 16974018 PMC1570006
  44. [70]Cumming G, Calin‐Jageman R. Introduction to the New Statistics: Estimation, Open Science, and Beyond. New York: Routledge; 2016.
  45. [71]Li T, Saldanha IJ, Vedula SS, et al. Learning by doing‐teaching systematic review methods in 8 weeks. Res Synth Methods. 2014; 5: 254‐ 263. 26052850
  46. [73]Yan T, Kull TJ. Supplier opportunism in buyer‐supplier new product development: a China–U.S. study of antecedents, consequences, and cultural/institutional contexts. Decisi Sci. 2015; 46: 403‐ 445.
  47. [74]Tessarolo P. Is integration enough for fast product development? An empirical investigation of the contextual effects of product vision. J Prod Innov Manag. 2007; 24: 69‐ 82.
  48. [75]Parker DB, Zsidisin GA, Ragatz GL. Timing and extent of supplier integration in new product development: a contingency approach. J Supply Chain Manag. 2008; 44: 71‐ 83.
  49. [76]Lin R‐J. Supplier integration and time‐based capabilities: an empirical study. J Chin Inst Ind Eng. 2009; 26: 215‐ 227.
  50. [77]Koufteros X, Rawski GE, Rupak R. Organizational integration for product development: the effects on glitches, on‐time execution of engineering change orders, and market success. Decis Sci. 2010; 41: 49‐ 80.
  51. [78]Perols J, Zimmermann C, Kortmann S. On the relationship between supplier integration and time‐to‐market. J Oper Manag. 2013; 31: 153‐ 167.
  52. [79]Yan T, Dooley KJ. Communication intensity, goal congruence, and uncertainty in buyer‐supplier new product development. J Oper Manag. 2013; 31: 523‐ 542.
  53. [80]Lau AKW, Tang E, Yam RCM. Effects of supplier and customer integration on product innovation and performance: empirical evidence in Hong Kong manufacturers. J Prod Innov Manag. 2010; 27: 761‐ 777.
  54. [83]Laseter TM, Ramdas K. Product types and supplier roles in product development: an exploratory analysis. IEEE Trans Eng Manag. 2002; 49: 107‐ 118.
  55. [84]Lipsey MW, Wilson D. Practical Meta‐analysis. Thousand Oaks, London, New Delhi: SAGE Publications; 2001.

Cited by

Sorted by citations

Authors
Suurmond, R., van Rhee, H., & Hak, T.
Affiliations
  • Erasmus University Rotterdam (NL)
  • Rotterdam School of Management Erasmus University The Netherlands
Corresponding author
Suurmond, Robert
ORCID
Venue
Research Synthesis Methods
Publisher
Wiley
Published
2017-09-29
Volume
8
Issue
4
Pages
537-553
ISSN
  • 1759-2879 (ISSN-L)
  • 1759-2887
DOI
10.1002/jrsm.1260
PMC
PMC5725669
OpenAlex
W2742272122
Cited by
881
References
16
Citations by year
2026: 70 · 2025: 119 · 2024: 118 · 2023: 133 · 2022: 138 · 2021: 136 · 2020: 104 · 2019: 47 · 2018: 16
FWCI
40.86
Type
journal article
Language
en
Open access
Licence
Locations
  • Research Synthesis Methods (publishedVersion, cc-by-nc-nd)
  • Europe PMC (PubMed Central) (publishedVersion, cc-by-nc-nd)
  • PubMed Central (publishedVersion, cc-by-nc-nd)
  • PubMed (publishedVersion)
  • Research Publications (Maastricht University) (publishedVersion, cc-by-nc-nd)
  • EUR Research Repository (Erasmus University Rotterdam) (publishedVersion, other-oa)
  • EUR Research Repository (Erasmus University Rotterdam) (submittedVersion, other-oa)
  • RePub (Erasmus University Rotterdam) (submittedVersion, other-oa)
  • Research Publications (Maastricht University) (submittedVersion, other-oa)
  • ERIM Top-Core Articles
Topics
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › scientometrics and bibliometrics research
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
Keywords
meta-analysis software, free software, effect size estimation, confidence intervals, subgroup analysis, moderator analysis, publication bias, software validation, software comparison
MeSH terms
Algorithms, Computers, Humans, Programming Languages, Regression Analysis, Software, Meta-Analysis as Topic, Models, Statistical, Publication Bias, Validation Studies as Topic

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review