The DOI matches its Crossref record; 18 of the 27 references it cites were found in a registry or an index.
Closing the Gap between Methodologists and End-Users: R as a Computational Back-End
Open accessAbstract
The R environment provides a natural platform for developing new statistical methods due to the mathematical expressiveness of the language, the large number of existing libraries, and the active developer community. One drawback to R, however, is the learning curve; programming is a deterrent to non-technical users, who typically prefer graphical user interfaces (GUIs) to command line environments. Thus, while statisticians develop new methods in R, practitioners are often behind in terms of the statistical techniques they use as they rely on GUI applications. Meta-analysis is an instructive example; cutting-edge meta-analysis methods are often ignored by the overwhelming majority of practitioners, in part because they have no easy way of applying them. This paper proposes a strategy to close the gap between the statistical state-of-the-science and what is applied in practice. We present open-source meta-analysis software that uses R as the underlying statistical engine, and Python for the GUI. We present a framework that allows methodologists to implement new methods in R that are then automatically integrated into the GUI for use by end-users, so long as the programmer conforms to our interface. Such an approach allows an intuitive interface for non-technical users while leveraging the latest advanced statistical methods implemented by methodologists.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Meta-analysis in clinical trials
DerSimonian, R., & Laird, N. (1986) · Controlled Clinical Trials · doi:10.1016/0197-2456(86)90046-2 · Cited by 39,552 · cited 1×
“A striking example is the almost universal use of the DerSimonian-Laird method for random effects univariate meta-analysis (DerSimonian and Laird 1986) (the default or only method in the aforementioned programs), irrespective of good data that alternative random effect methods have better statistical…”
MethodMeta-analysis of Observational Studies in Epidemiology<SUBTITLE>A Proposal for Reporting</SUBTITLE>
Stroup, D. F. (2000) · JAMA · doi:10.1001/jama.283.15.2008 · Cited by 20,623
Conducting Meta-Analyses in R with the metafor Package
Viechtbauer, W. (2010) · Journal of Statistical Software · doi:10.18637/jss.v036.i03 · Cited by 20,464 · cited 4×
“Line 2 is the call out to the random effects as implemented by the metafor package (Viechtbauer 2010).”
MethodInfluentialStatistical Methods for Research Workers
Kendall, M. (1937) · Nature · doi:10.1038/139737d0 · Cited by 9,676 · cited 1×
“…disciplines, including the natural sciences, such as physics, astronomy (Airy 1875) and ecology (Osenberg, Sarnelle, and Goldberg 1999), as well as medicine (Simpson and Pearson 1904; Fisher 1925), epidemiology (Stroup et al. 2000), psychology (Glass 1976), and education (Kulik and Kulik 1989).”
BackgroundPrimary, Secondary, and Meta-Analysis of Research
GLASS, G. V. (1976) · Educational Researcher · doi:10.3102/0013189x005010003 · Cited by 4,890 · cited 1×
“…disciplines, including the natural sciences, such as physics, astronomy (Airy 1875) and ecology (Osenberg, Sarnelle, and Goldberg 1999), as well as medicine (Simpson and Pearson 1904; Fisher 1925), epidemiology (Stroup et al. 2000), psychology (Glass 1976), and education (Kulik and Kulik 1989).”
BackgroundBias and Efficiency of Meta-Analytic Variance Estimators in the Random-Effects Model
Viechtbauer, W. (2005) · Journal of Educational and Behavioral Statistics · doi:10.3102/10769986030003261 · Cited by 1,699 · cited 2×
“…random effects univariate meta-analysis (DerSimonian and Laird 1986) (the default or only method in the aforementioned programs), irrespective of good data that alternative random effect methods have better statistical properties (Turner, Omar, Yang, Goldstein, and Thompson 2000; Viechtbauer 2005).”
MethodThe R Commander: A Basic-Statistics Graphical User Interface to R
Fox, J. (2005) · Journal of Statistical Software · doi:10.18637/jss.v014.i09 · Cited by 1,098 · cited 1×
“Our approach is similar in spirit to the R Commander package ( Rcmdr ), which provides an “easy-to-use, cross-platform, extensible GUI” with enough functionality to span a basic statistics course (Fox 2005).”
MethodBax, L., Yu, L.-M., Ikeda, N., Tsuruta, H., & Moons, K. G. (2006) · BMC Medical Research Methodology · doi:10.1186/1471-2288-6-50 · Cited by 514 · cited 2×
“Currently, meta-analysis methods are implemented in many statistical packages, including Stata (Sterne 2009), SPSS (Lipsey and Wilson 2001), SAS (Arthur, Bennett, Huffcutt, Arthur, and Bennett 2001) and R (Schwarzer 2012; Lumley 2009; Viechtbauer 2010), in addition to Microsoft Excel plug-ins, such as MIX (Bax, Yu, Ike…”
MethodGurevitch, J., Curtis, P. S., & Jones, M. H. (2001) · Advances in Ecological Research · doi:10.1016/s0065-2504(01)32013-5 · Cited by 421
Report on Certain Enteric Fever Inoculation Statistics
Simpson, L. (1904) · British medical journal · doi:10.1136/bmj.2.2288.1243 · Cited by 283 · cited 1×
“…disciplines, including the natural sciences, such as physics, astronomy (Airy 1875) and ecology (Osenberg, Sarnelle, and Goldberg 1999), as well as medicine (Simpson and Pearson 1904; Fisher 1925), epidemiology (Stroup et al. 2000), psychology (Glass 1976), and education (Kulik and Kulik 1989).”
BackgroundData Extraction Errors in Meta-analyses That Use Standardized Mean Differences
Gøtzsche, P. C., Hróbjartsson, A., Marić, K., & Tendal, B. (2007) · JAMA · doi:10.1001/jama.298.4.430 · Cited by 238 · cited 1×
“Automating such calculations can reduce the reviewers’ work burden and minimize data extraction errors (Gotzsche, Hrobjartsson, Maric, and Tendal 2007).”
BackgroundConducting Meta-Analysis Using SAS
Arthur, W., Bennett, W., & Huffcutt, A. I. (2001) · Psychology Press eBooks · doi:10.4324/9781410600028 · Cited by 199 · cited 3×
“Currently, meta-analysis methods are implemented in many statistical packages, including Stata (Sterne 2009), SPSS (Lipsey and Wilson 2001), SAS (Arthur, Bennett, Huffcutt, Arthur, and Bennett 2001) and R (Schwarzer 2012; Lumley 2009; Viechtbauer 2010), in addition to Microsoft Excel plug-ins, such…”
MethodSmolen, J. S., Schoels, M., Nishimoto, N., Breedveld, F. C., Burmester, G. R., Dougados, M., Emery, P., Ferraccioli, G., Gabay, C., Gibofsky, A., et al. (2012) · Annals of the Rheumatic Diseases · doi:10.1136/annrheumdis-2012-202469 · Cited by 122 · cited 1×
“…disciplines, including the natural sciences, such as physics, astronomy (Airy 1875) and ecology (Osenberg, Sarnelle, and Goldberg 1999), as well as medicine (Simpson and Pearson 1904; Fisher 1925), epidemiology (Stroup et al. 2000), psychology (Glass 1976), and education (Kulik and Kulik 1989).”
BackgroundKulik, J., & Kulik, C. (1989) · The SAGE Glossary of the Social and Behavioral Sciences · doi:10.4135/9781412972024.n1600 · Cited by 107 · cited 1×
“…disciplines, including the natural sciences, such as physics, astronomy (Airy 1875) and ecology (Osenberg, Sarnelle, and Goldberg 1999), as well as medicine (Simpson and Pearson 1904; Fisher 1925), epidemiology (Stroup et al. 2000), psychology (Glass 1976), and education (Kulik and Kulik 1989).”
BackgroundMeta‐analysis in Ecology: Concepts, Statistics, and Applications
Osenberg, C., Sarnelle, O., & Goldberg, D. (1999) · Ecology · doi:10.1890/0012-9658(1999)080[1103:maiecs]2.0.co;2 · Cited by 70 · cited 1×
“Meta-analysis has a long tradition in many scientific disciplines, including the natural sciences, such as physics, astronomy (Airy 1875) and ecology (Osenberg, Sarnelle, and Goldberg 1999), as well as medicine (Simpson and Pearson 1904; Fisher 1925), epidemiology (Stroup et al. 2000), psychology…”
BackgroundMetaEasy: A Meta-Analysis Add-In for MicrosoftExcel
Kontopantelis, E., & Reeves, D. (2009) · Journal of Statistical Software · doi:10.18637/jss.v030.i07 · Cited by 57 · cited 2×
“Currently, meta-analysis methods are implemented in many statistical packages, including Stata (Sterne 2009), SPSS (Lipsey and Wilson 2001), SAS (Arthur, Bennett, Huffcutt, Arthur, and Bennett 2001) and R (Schwarzer 2012; Lumley 2009; Viechtbauer 2010), in addition to Microsoft Excel plug-ins, such as MIX (Bax, Yu, Ike…”
MethodHillebrand, H. (2008) · Wiley eBooks · doi:10.1002/9780470015902.a0003272 · Cited by 24
Hahn, S., Williamson, P. R., Hutton, J. L., Garner, P., & Flynn, E. V. (2000) · Statistics in Medicine · doi:10.1002/1097-0258(20001230)19:24
Cited by
Sorted by citations
Clinical, laboratory and imaging features of COVID-19: A systematic review and meta-analysis
Rodriguez-Morales, A. J., Cardona-Ospina, J. A., Gutiérrez-Ocampo, E., Villamizar-Peña, R., Holguin-Rivera, Y., Escalera-Antezana, J. P., Alvarado-Arnez, L. E., Bonilla-Aldana, D. K., Franco-Paredes, C., Henao-Martinez, A. F., et al. (2020) · Travel Medicine and Infectious Disease · doi:10.1016/j.tmaid.2020.101623 · Cited by 2,542
“The meta-analyses were performed using Stata, and the software OpenMeta[Analyst] [37] and Comprehensive Meta Analysis ve.3.3® licensed for Universidad Tecnológica de Pereira, Colombia.”
MethodInfluentialMethods to estimate the between‐study variance and its uncertainty in meta‐analysis
Veroniki, A. A., Jackson, D., Viechtbauer, W., Bender, R., Bowden, J., Knapp, G., Kuss, O., Higgins, J. P., Langan, D., & Salanti, G. (2016) · Research Synthesis Methods · doi:10.1002/jrsm.1164 · Cited by 1,685
Peri‐implant health and disease. A systematic review of current epidemiology
Derks, J., & Tomasi, C. (2015) · Journal of Clinical Periodontology · doi:10.1111/jcpe.12334 · Cited by 1,302
“Meta-analyses were performed to estimate overall prevalence of peri-implant mucositis and periimplantitis (OpenMeta[Analyst]) (Wallace et al. 2012).”
MethodBhimraj, A., Morgan, R. L., Shumaker, A. H., Baden, L. R., Cheng, V. C., Edwards, K. M., Gallagher, J. C., Gandhi, R. T., Muller, W. J., Nakamura, M., et al. (2022) · Clinical Infectious Diseases · doi:10.1093/cid/ciac724 · Cited by 240
“Where applicable, data were pooled using random effects model (fixed effects model for 2 or less trials or pooling of rates) using RevMan or OpenMeta [7].”
MethodHollocks, M. J., Lerh, J. W., Magiati, I., Meiser-Stedman, R., & Brugha, T. S. (2019) · Psychological Medicine · doi:10.1017/s0033291718002283 · Cited by 988
“OpenMeta, a tool for running metafor package in R (Viechtbauer, 2010), was used to conduct the meta-analysis (Wallace et al., 2012).”
Metadata
- Authors
- Wallace, B. C., Dahabreh, I. J., Trikalinos, T. A., Lau, J., Trow, P., & Schmid, C. H.
- Affiliations
- Tufts University (US)
- Venue
- Journal of Statistical Software
- Publisher
- Foundation for Open Access Statistic
- Published
- 2012
- Volume
- 49
- Issue
- 5
- ISSN
- 1548-7660 (ISSN-L)
- OpenAlex
- W2120771911
- Cited by
- 1,125
- References
- 27
- Citations by year
- 2026: 50 · 2025: 89 · 2024: 115 · 2023: 105 · 2022: 127 · 2021: 143 · 2020: 133 · 2019: 94 · 2018: 76 · 2017: 68 · 2016: 65 · 2015: 37 · 2014: 17 · 2013: 4 · 2012: 1
- FWCI
- 18.44
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- cc-by
- Locations
- Journal of Statistical Software (publishedVersion, cc-by)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Data Analysis with R
- Social Sciences › Decision Sciences › Information Systems and Management › Scientific Computing and Data Management
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Keywords
- R, meta-analysis software, graphical user interface, Python, open source software, software integration, statistical software development, non-technical users
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
