The DOI matches its Crossref record; 92 of the 92 references it cites were found in a registry or an index.
Obtaining Unbiased Results in Meta-Analysis: The Importance of Correcting for Statistical Artifacts
Abstract
Most published meta-analyses address only artifactual variance due to sampling error and ignore the role of other statistical and psychometric artifacts, such as measurement error variance (due to factors including unreliability of measurements, group misclassification, and variable treatment strength) and selection effects (including range restriction or enhancement and collider biases). These artifacts can have severe biasing effects on the results of individual studies and meta-analyses. Failing to account for these artifacts can lead to inaccurate conclusions about the mean effect size and between-studies effect-size heterogeneity, and can influence the results of meta-regression, publication-bias, and sensitivity analyses. In this article, we provide a brief introduction to the biasing effects of measurement error variance and selection effects and their relevance to a variety of research designs. We describe how to estimate the effects of these artifacts in different research designs and correct for their impacts in primary studies and meta-analyses. We consider meta-analyses of correlations, observational group differences, and experimental effects. We provide R code to implement the corrections described.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Because the key test we aimed to perform was differences in performance across time, correcting for test-retest reliability was necessary to avoid issues of bias in effect sizes.”
From Menstrual cycle effects on cognitive performance: A meta-analysis (2025) · cites this paper as [68]
Citations
Cites
Sample Selection Bias as a Specification Error
Heckman, J. J. (1979) · Econometrica · doi:10.2307/1912352 · Cited by 28,779 · cited 1×
“…selection models can also be applied to estimate relationships in the unselected population (see, e.g., Fife, Hunter, & Mendoza, 2016; Gross & McGanney, 1987; Heckman, 1976, 1979; Olson & Becker, 1983; Pfaffel, Kollmayer, Schober, & Spiel, 2016; Puhani, 2000; Yang, Sackett, & Nho, 2004).”
Conducting Meta-Analyses in R with the metafor Package
Viechtbauer, W. (2010) · Journal of Statistical Software · doi:10.18637/jss.v036.i03 · Cited by 20,464 · cited 1×
“…effect sizes, corrected variances, and sample and moderator information for use with the meta-analysis models provided in the metafor package (Viechtbauer, 2010): es_data - get_metafor(ma_results_d, analyses = list(construct_y = "Y"), ma_method = "ic", correction_type = "ts") In this code,…”
MethodThe proof and measurement of association between two things
Spearman, C. (1904) · The American Journal of Psychology · doi:10.2307/1412159 · Cited by 8,039
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567
Methods of Meta-Analysis: Correcting Error and Bias in Research Findings
Schmidt, F. L., & Hunter, J. E. (2015) · doi:10.4135/9781483398105 · Cited by 5,542
The Handbook of Research Synthesis and Meta-Analysis
(2019) · doi:10.7758/9781610448864 · Cited by 4,553
Rammstedt, B., & John, O. P. (2007) · Journal of Research in Personality · doi:10.1016/j.jrp.2006.02.001 · Cited by 4,244 · cited 1×
“Many short-form measures of constructs include items chosen to tap a construct broadly and with limited redundancy (Rammstedt & John, 2007; Yarkoni, 2010).”
BackgroundIdentification of Causal Effects Using Instrumental Variables
Angrist, J. D., Imbens, G. W., & Rubin, D. B. (1996) · Journal of the American Statistical Association · doi:10.1080/01621459.1996.10476902 · Cited by 4,230
A general approach to causal mediation analysis.
Imai, K., Keele, L., & Tingley, D. (2010) · Psychological Methods · doi:10.1037/a0020761 · Cited by 4,143 · cited 2×
“If these assumptions are unreasonable, other estimators with different assumptions can be used (Imai et al., 2010), but these are less readily applied in meta-analysis.”
BackgroundMethodFixed-and random-effects models in meta-analysis.
Hedges, L. V., & Vevea, J. L. (1998) · Psychological Methods · doi:10.1037/1082-989x.3.4.486 · Cited by 2,977 · cited 5×
“When d values are corrected only for measurement error in the dependent variable and ryypooled′ is used, the sampling-error adjustment follows the same process as Equation 26: SE SE d dd d cc 2 2 2= × obs obs ( / ) (27) For corrections involving converting d to r and back, the sampling error is adjusted by converting…”
BackgroundMethodInfluential
Research works printed but not linked: 5
- 10.1037/0021-9010.74.3.478 · doi:10.1037/0021-9010.74.3.478 · Paper page
- Carter E., 2017, PsyArXiv
- 10.1037/0021-9010.79.5.767 · doi:10.1037/0021-9010.79.5.767 · Paper page
- Kulinskaya E., 2018, Paper presented at the Society for Research Synthesis Methodology conference
- 10.1037/0021-9010.85.1.112 · doi:10.1037/0021-9010.85.1.112 · Paper page
Cited by
Harari, M. B., Parola, H. R., Hartwell, C. J., & Riegelman, A. (2020) · Journal of Vocational Behavior · doi:10.1016/j.jvb.2020.103377 · Cited by 291
Factors Associated with Teacher Wellbeing: A Meta-Analysis
Zhou, S., Slemp, G. R., & Vella‐Brodrick, D. (2024) · Educational Psychology Review · doi:10.1007/s10648-024-09886-x · Cited by 302
“Thus, to the extent that existing estimates of associations of teacher wellbeing and its key antecedents, consequences and correlates exist, they are systematically biased by such artefacts and are not reflective of their true value (Schmidt & Hunter, 2015; Wiernik & Dahlke, 2020).”
Albulescu, P., Macsinga, I., Rusu, A., Sulea, C., Bodnaru, A., & Tulbure, B. T. (2022) · PLOS ONE · doi:10.1371/journal.pone.0272460 · Cited by 168
Slemp, G. R., Field, J. G., Ryan, R. M., Forner, V. W., Van den Broeck, A., & Lewis, K. J. (2024) · Journal of Personality and Social Psychology · doi:10.1037/pspi0000459 · Cited by 182
An Aberrant Abundance of Cronbach’s Alpha Values at .70
Hussey, I., Alsalti, T., Bosco, F. A., Elson, M., & Arslan, R. C. (2025) · Advances in Methods and Practices in Psychological Science · doi:10.1177/25152459241287123 · Cited by 225
“Several types of meta-analysis, such as psychometric meta-analyses (Schmidt & Hunter, 2015; Wiernik & Dahlke, 2020), adjust effect sizes for the reliability of their measures (i.e., disattenuate for reliability).”
Metadata
- Authors
- Wiernik, B. M., & Dahlke, J. A.
- Affiliations
- University of South Florida (US)
- Human Resources Research Organization (US)
- Corresponding author
- Wiernik, Brenton M.
- Venue
- Advances in Methods and Practices in Psychological Science
- Publisher
- SAGE Publications
- Published
- 2020-01-22
- Volume
- 3
- Issue
- 1
- Pages
- 94-123
- ISSN
- 2515-2459 (ISSN-L)
- 2515-2467
- OpenAlex
- W3002826573
- Cited by
- 135
- References
- 91
- Citations by year
- 2026: 20 · 2025: 19 · 2024: 28 · 2023: 21 · 2022: 19 · 2021: 20 · 2020: 7
- FWCI
- 10.22
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
- Locations
- Advances in Methods and Practices in Psychological Science (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement
- Keywords
- meta-analysis bias, measurement error variance, selection effects, range restriction, collider bias, effect size heterogeneity, meta-regression, publication bias, sampling error, measurement reliability, experimental effects
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Obtaining Unbiased Results in Meta-Analysis: The Importance of Correcting for Statistical Artifacts. https://app.cochat.ai/papers/10.1177/2515245919885611
- The paper
- Wiernik, B. M., & Dahlke, J. A. (2020). Obtaining Unbiased Results in Meta-Analysis: The Importance of Correcting for Statistical Artifacts. Advances in Methods and Practices in Psychological Science. https://doi.org/10.1177/2515245919885611
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.