The DOI matches its Crossref record; 66 of the 67 references it cites were found in a registry or an index.
Neither fixed nor random: weighted least squares meta‐regression
Open accessAbstract
Our study revisits and challenges two core conventional meta-regression estimators: the prevalent use of 'mixed-effects' or random-effects meta-regression analysis and the correction of standard errors that defines fixed-effects meta-regression analysis (FE-MRA). We show how and explain why an unrestricted weighted least squares MRA (WLS-MRA) estimator is superior to conventional random-effects (or mixed-effects) meta-regression when there is publication (or small-sample) bias that is as good as FE-MRA in all cases and better than fixed effects in most practical applications. Simulations and statistical theory show that WLS-MRA provides satisfactory estimates of meta-regression coefficients that are practically equivalent to mixed effects or random effects when there is no publication bias. When there is publication selection bias, WLS-MRA always has smaller bias than mixed effects or random effects. In practical applications, an unrestricted WLS meta-regression is likely to give practically equivalent or superior estimates to fixed-effects, random-effects, and mixed-effects meta-regression approaches. However, random-effects meta-regression remains viable and perhaps somewhat preferable if selection for statistical significance (publication bias) can be ruled out and when random, additive normal heterogeneity is known to directly affect the 'true' regression coefficient. Copyright © 2016 John Wiley & Sons, Ltd.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“However, the simulations reported by Stanley & Doucouliagos, 2017) show that there is little difference between random effects models and the UWLS‐MRA, which appears to be even superior when there is a sizeable.”
From Public sector reforms and their impact on the level of corruption: A systematic review (2021) · cites this paper as Stanley & Doucouliagos, 2017
In the citing paper
“Conventional meta‐analytic models assume that the sampling variances are known. Conversely, the UWLS‐MRA assumes that the sampling variances are known only up to a proportionality constant σe2, which is “automatically estimated by the mean squared error, MSE” (Stanley & Doucouliagos, 2017, p. 22).”
From Public sector reforms and their impact on the level of corruption: A systematic review (2021) · cites this paper as Stanley & Doucouliagos, 2017
In the citing paper
“Recently, Stanley & Doucouliagos, 2017) challenged the conventional meta (regression) models by proposing a third alternative: the Unconditional Weighted Least Square Meta‐Regression Analysis (henceforth UWLS‐MRA).”
From Public sector reforms and their impact on the level of corruption: A systematic review (2021) · cites this paper as Stanley & Doucouliagos, 2017
Citations
Cites
Bias in meta-analysis detected by a simple, graphical test
Egger, M., Smith, G. D., Schneider, M., & Minder, C. (1997) · BMJ · doi:10.1136/bmj.315.7109.629 · Cited by 56,722
Quantifying heterogeneity in a meta‐analysis
Higgins, J. P. T., & Thompson, S. G. (2002) · Statistics in Medicine · doi:10.1002/sim.1186 · Cited by 36,763
(2001) · RePEc: Research Papers in Economics · Cited by 27,577
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2009) · doi:10.1002/9780470743386 · Cited by 14,225
Wasserman, S. S., Hedges, L. V., & Olkin, I. (1988) · Journal of Educational Statistics · doi:10.2307/1164953 · Cited by 10,377
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2021) · doi:10.1002/9781119558378 · Cited by 9,799
(2000) · Cited by 9,198
UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
Service, I., & Shirazi, M. A. (2005) · Proceedings of the Water Environment Federation · doi:10.2175/193864705783867675 · Cited by 6,679
Statistical Methods for Meta-Analysis.
Freeman, P. R., Hedges, L. V., & Olkin, I. (1986) · Biometrics · doi:10.2307/2531069 · Cited by 4,840
The Handbook of Research Synthesis.
Howel, D., Cooper, H., & Hedges, L. V. (1995) · Biometrics · doi:10.2307/2533295 · Cited by 3,657
Kmenta, J. (1997) · University of Michigan Press eBooks · doi:10.3998/mpub.15701 · Cited by 3,240
Fixed-and random-effects models in meta-analysis.
Hedges, L. V., & Vevea, J. L. (1998) · Psychological Methods · doi:10.1037/1082-989x.3.4.486 · Cited by 2,977
Introduction to the Theory and Practice of Econometrics
Ziegel, E. R., Judge, G. G., Hill, P., Griffiths, W. E., Lütkepohl, H., Lee, T. C. M., & Lütkepohl, H. (1989) · Technometrics · doi:10.2307/1270023 · Cited by 2,251
The value of a statistical life: A critical review of market estimates throughout the world
Viscusi, W. K., & Aldy, J. E. (2003) · Journal of Risk and Uncertainty · doi:10.1023/a:1025598106257 · Cited by 2,135
Improved tests for a random effects meta‐regression with a single covariate
Knapp, G., & Hartung, J. (2003) · Statistics in Medicine · doi:10.1002/sim.1482 · Cited by 2,075
Econometric Theory and Methods
Davidson, R., & MacKinnon, J. G. (2003) · Cited by 1,768
Wickens, M. R., & Kmenta, J. (1972) · Economica · doi:10.2307/2552662 · Cited by 1,405
Controlling the risk of spurious findings from meta-regression
Higgins, J. P. T., & Thompson, S. G. (2004) · Statistics in Medicine · doi:10.1002/sim.1752 · Cited by 1,267
IV.—On Least Squares and Linear Combination of Observations
Aitken, A. C. (1936) · Proceedings of the Royal Society of Edinburgh · doi:10.1017/s0370164600014346 · Cited by 950
Stanley, T. D., & Doucouliagos, H. (2014) · Research Synthesis Methods · doi:10.1002/jrsm.1095 · Cited by 916
Self-citationTurner, R. M., Davey, J., Clarke, M. J., Thompson, S. G., & Higgins, J. P. (2012) · International Journal of Epidemiology · doi:10.1093/ije/dys041 · Cited by 837
META‐REGRESSION ANALYSIS: A QUANTITATIVE METHOD OF LITERATURE SURVEYS
Stanley, T. D., & Jarrell, S. B. (2005) · Journal of Economic Surveys · doi:10.1111/j.0950-0804.2005.00249.x · Cited by 805
Self-citationMeta-Regression Analysis in Economics and Business
Stanley, T. D., & Doucouliagos, H. (2012) · Routledge eBooks · doi:10.4324/9780203111710 · Cited by 788
Self-citationThe Power of Bias in Economics Research
Ioannidis, J. P. A., Stanley, T. D., & Doucouliagos, H. (2017) · The Economic Journal · doi:10.1111/ecoj.12461 · Cited by 634
Self-citationStanley, T. D. (2008) · Oxford Bulletin of Economics and Statistics · doi:10.1111/j.1468-0084.2007.00487.x · Cited by 605
Self-citationRecent developments in meta‐analysis
Sutton, A. J., & Higgins, J. P. T. (2007) · Statistics in Medicine · doi:10.1002/sim.2934 · Cited by 601
Publication Selection Bias in Minimum‐Wage Research? A Meta‐Regression Analysis
Doucouliagos, H., & Stanley, T. D. (2009) · British Journal of Industrial Relations · doi:10.1111/j.1467-8543.2009.00723.x · Cited by 507
Self-citationIntroduction to the Theory and Practice of Econometrics.
Trader, R. L., Judge, G. G., Hill, R. C., Griffiths, W. E., Lütkepohl, H., & Lee, T.-C. (1983) · Journal of the American Statistical Association · doi:10.2307/2288235 · Cited by 499
Understanding heterogeneity in meta-analysis: the role of meta-regression
Baker, W. L., Michael White, C., Cappelleri, J. C., Kluger, J., Coleman, C. I., & From the Health Outcomes, P. (2009) · International Journal of Clinical Practice · doi:10.1111/j.1742-1241.2009.02168.x · Cited by 492
META‐ANALYSIS OF ECONOMICS RESEARCH REPORTING GUIDELINES
Stanley, T. D., Doucouliagos, H., Giles, M. J., Heckemeyer, J. H., Johnston, R. J., Laroche, P., Nelson, J. P., Paldam, M., Poot, J., Pugh, G., et al. (2013) · Journal of Economic Surveys · doi:10.1111/joes.12008 · Cited by 463
Self-citation(1994) · Cited by 463
Sterling, T. D., Rosenbaum, W. L., & Weinkam, J. J. (1995) · The American Statistician · doi:10.1080/00031305.1995.10476125 · Cited by 453
Multivariate Random-effects Meta-regression: Updates to Mvmeta
White, I. R. (2011) · The Stata Journal: Promoting communications on statistics and Stata · doi:10.1177/1536867x1101100206 · Cited by 400
Hedges, L. V. (1994) · Cited by 397
Meta-analysis in Stata : an updated collection from the Stata journal
Palmer, T., & Sterne, J. A. C. (2015) · Medical Entomology and Zoology · Cited by 388
A comparison of heterogeneity variance estimators in combining results of studies
Sidik, K., & Jonkman, J. N. (2007) · Statistics in Medicine · doi:10.1002/sim.2688 · Cited by 380
Moreno, S. G., Sutton, A. J., Ades, A., Stanley, T. D., Abrams, K. R., Peters, J. L., & Cooper, N. J. (2009) · BMC Medical Research Methodology · doi:10.1186/1471-2288-9-2 · Cited by 373
Self-citationRandom-Effects Meta-Analyses Are Not Always Conservative
Poole, C., & Greenland, S. (1999) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a010035 · Cited by 338
Misunderstandings about Q and ‘Cochran's Q test' in meta‐analysis
Hoaglin, D. C. (2015) · Statistics in Medicine · doi:10.1002/sim.6632 · Cited by 313
Comparing meta-analytic moderator estimation techniques under realistic conditions.
Steel, P. D., & Kammeyer-Mueller, J. D. (2002) · Journal of Applied Psychology · doi:10.1037/0021-9010.87.1.96 · Cited by 309 · cited 1×
“…mixed-effects MRA—employ WLS.1 WLS has long been used by meta-analysts: Stanley and Jarrell (1989), Raudenbush (1994), Thompson and Sharp (1999), Higgins and Thompson (2002), Steel and Kammeyer-Mueller (2002), Baker et al. (2009), Copas and Lozada (2009), and Moreno et al. (2009), to cite a few.”
MethodARE ALL ECONOMIC FACTS GREATLY EXAGGERATED? THEORY COMPETITION AND SELECTIVITY
Doucouliagos, H., & Stanley, T. D. (2011) · Journal of Economic Surveys · doi:10.1111/j.1467-6419.2011.00706.x · Cited by 302
Self-citationModelling publication bias in meta-analysis: a review
Sutton, A. J., Song, F., Gilbody, S. M., & Abrams, K. R. (2000) · Statistical Methods in Medical Research · doi:10.1177/096228020000900503 · Cited by 288
Neither fixed nor random: weighted least squares meta‐analysis
Stanley, T. D., & Doucouliagos, H. (2015) · Statistics in Medicine · doi:10.1002/sim.6481 · Cited by 258
Self-citationTheoria combinationis observationum erroribus minimis obnoxiae
Gauß, C. F. (1823) · doi:10.3931/e-rara-2857 · Cited by 234
Publication Bias in Empirical Sociological Research
Gerber, A. S., & Malhotra, N. (2008) · Sociological Methods & Research · doi:10.1177/0049124108318973 · Cited by 175
Could It Be Better to Discard 90% of the Data? A Statistical Paradox
Stanley, T. D., Jarrell, S. B., & Doucouliagos, H. (2010) · The American Statistician · doi:10.1198/tast.2009.08205 · Cited by 170
Self-citationSterling, T. D., Rosenbaum, W. L., & Jj, W. (1995) · The American Statistician · doi:10.2307/2684823 · Cited by 168
Confidence intervals for random effects meta-analysis and robustness to publication bias
Henmi, M., & Copas, J. B. (2010) · Statistics in Medicine · doi:10.1002/sim.4029 · Cited by 167
Testing for Publication Bias in Political Science
Gerber, A. S., GREEN, D. P., & Nickerson, D. W. (2001) · Political Analysis · doi:10.1093/oxfordjournals.pan.a004877 · Cited by 162
Are estimates of the value of a statistical life exaggerated?
Doucouliagos, H., Stanley, T. D., & Giles, M. J. (2011) · Journal of Health Economics · doi:10.1016/j.jhealeco.2011.10.001 · Cited by 143
Self-citationDoes the UK Minimum Wage Reduce Employment? A Meta‐Regression Analysis
Leonard, M. D. L., Stanley, T. D., & Doucouliagos, H. (2013) · British Journal of Industrial Relations · doi:10.1111/bjir.12031 · Cited by 94
Self-citationPublication selection and the income elasticity of the value of a statistical life
Doucouliagos, H., Stanley, T. D., & Viscusi, W. K. (2013) · Journal of Health Economics · doi:10.1016/j.jhealeco.2013.10.010 · Cited by 93
Self-citationEfficiency Wages, Productivity and Simultaneity: A Meta-Regression Analysis
Peach, E. K., & Stanley, T. D. (2009) · Journal of Labor Research · doi:10.1007/s12122-009-9066-5 · Cited by 74
Self-citationJacquez, J. A., Mather, F. J., & Crawford, C. R. (1968) · Biometrics · doi:10.2307/2528321 · Cited by 61
Policy Challenges of the Heterogeneity of the Value of Statistical Life
Viscusi, W. K. (2011) · Foundations and Trends® in Microeconomics · doi:10.1561/0700000011 · Cited by 54
Depicting estimates using the intercept in meta‐regression models: The moving constant technique
Johnson, B. T., & Huedo-Medina, T. B. (2011) · Research Synthesis Methods · doi:10.1002/jrsm.49 · Cited by 49
Pay for Performance and Corporate Governance Reform
Doucouliagos, H., Haman, J., & Stanley, T. D. (2010) · Industrial Relations A Journal of Economy and Society · doi:10.1111/j.1468-232x.2012.00695.x · Cited by 43
Self-citationTheory Competition and Selectivity: Are All Economic Facts Greatly Exaggerated?Preprint
Doucouliagos, H., & Stanley, T. D. (2008) · RePEc: Research Papers in Economics · Cited by 42
Self-citationPublication selection in health policy research: The winner's curse hypothesis
Costa‐Font, J., McGuire, A. J., & Stanley, T. D. (2012) · Health Policy · doi:10.1016/j.healthpol.2012.10.015 · Cited by 41
Self-citationMeta‐analysis inside and outside particle physics: two traditions that should converge?
Baker, R., & Jackson, D. (2012) · Research Synthesis Methods · doi:10.1002/jrsm.1065 · Cited by 39
Koren, J. (2016) · University of Michigan Library Repository · Cited by 37
Where is scientific evidence supporting EU policy on BSE and pharmaceuticals?
Earl‐Slater, A. (1998) · BMJ · doi:10.1136/bmj.316.7131.629 · Cited by 31
Consequences of effect size heterogeneity for meta-analysis: a Monte Carlo study
Koetse, M. J., Florax, R. J. G. M., & de Groot, H. L. F. (2009) · Statistical Methods & Applications · doi:10.1007/s10260-009-0125-0 · Cited by 27
The Radial Plot in Meta-Analysis: Approximations and Applications
Copas, J. B., & Lozada-Can, C. (2009) · Journal of the Royal Statistical Society Series C (Applied Statistics) · doi:10.1111/j.1467-9876.2008.00650.x · Cited by 20
(2011) · RePEc: Research Papers in Economics
Self-citationExplaining heterogeneity in meta-analysis: a comparison of methods
Thompson, S. G., & Sharp, S. J. (1999) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19991030)18:20
Research works printed but not linked: 7
- Greene WE, 1990, Econometric Analysis
- Hopewell S, 2009, Publication bias in clinical trials due to statistical significance or direction of trial result, Cochrane Review
- Konstantopoulos S, 2004, Quantitative Methodology for the Social Sciences, 281
- Raudenbush SW, 1994, The Handbook of Research Synthesis, 301
- Sharp S, 1998, sbe23: meta‐analysis regression, Stata Technical Bulletin, 42, 16
- 10.1111/j.1467-6419.1989.tb00064.x · DOI · Paper page
- U.S. Dept. of Transportation, 2011, Office of the Assistant Secretary for Transportation Policy, Memorandum: Treatment of the Economic Value of Statistical Life in Departmental Analyses—2011 Interim Adjustment
Cited by
Sorted by citations
Meta-analysis of action video game impact on perceptual, attentional, and cognitive skills.
Bediou, B., Adams, D. M., Mayer, R. E., Tipton, E., Green, C. S., & Bavelier, D. (2018) · Psychological Bulletin · doi:10.1037/bul0000130 · Cited by 807
Correcting for Bias in Psychology: A Comparison of Meta-Analytic Methods
Carter, E. C., Schönbrodt, F. D., Gervais, W. M., & Hilgard, J. (2019) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245919847196 · Cited by 588
What meta-analyses reveal about the replicability of psychological research.
Stanley, T. D., Carter, E. C., & Doucouliagos, H. (2018) · Psychological Bulletin · doi:10.1037/bul0000169 · Cited by 307
Self-citationReporting guidelines for meta-analysis in economics
Havránek, T., Stanley, T. D., Doucouliagos, H., Bom, P., Geyer‐Klingeberg, J., Iwasaki, I., Reed, W. R., Rost, K., & van Aert, R. C. M. (2020) · Journal of Economic Surveys · doi:10.1111/joes.12363 · Cited by 271
Self-citationLimitations of PET-PEESE and Other Meta-Analysis Methods
Stanley, T. D. (2017) · Social Psychological and Personality Science · doi:10.1177/1948550617693062 · Cited by 229
Self-citation
Metadata
- Authors
- Stanley, T. D., & Doucouliagos, H.
- Affiliations
- Hendrix College (US)
- Deakin University (AU)
- Corresponding author
- Stanley, T. D.
- Venue
- Research Synthesis Methods
- Publisher
- Wiley
- Published
- 2016-06-20
- Volume
- 8
- Issue
- 1
- Pages
- 19-42
- ISSN
- 1759-2879 (ISSN-L)
- 1759-2887
- OpenAlex
- W2462813397
- Cited by
- 205
- References
- 67
- Citations by year
- 2026: 7 · 2025: 17 · 2024: 17 · 2023: 28 · 2022: 24 · 2021: 31 · 2020: 26 · 2019: 23 · 2018: 15 · 2017: 13 · 2016: 2
- FWCI
- 16.50
- Type
- journal article
- Language
- en
- Open access
- green
- figshare.com
- Locations
- Research Synthesis Methods (publishedVersion)
- PubMed (publishedVersion)
- Figshare (submittedVersion, cc-by)
- Zenodo (CERN European Organization for Nuclear Research) (submittedVersion, other-oa)
- Funders
- Grantová Agentura České Republiky (15‐02411S)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Keywords
- publication bias, small sample bias, standard error correction, heterogeneity
- MeSH terms
- Algorithms, Computer Simulation, Humans, Markov Chains, Publishing, Regression Analysis, Research Design, Models, Statistical, Least-Squares Analysis, Publication Bias, Sample Size
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
