Revealed or Concealed? Transparency of Procedures, Decisions, and Judgment Calls in Meta-Analyses
The DOI matches its Crossref record; 61 of the 61 references it cites were found in a registry or an index.
Abstract
The authors examined the degree to which meta-analyses in the organizational sciences transparently report procedures, decisions, and judgment calls by systematically reviewing all (198) meta-analyses published between 1995 and 2008 in 11 top journals that publish meta-analyses in industrial and organizational psychology and organizational behavior. The authors extracted information on 54 features of each meta-analysis. On average, the meta-analyses in the sample provided 52.8% of the information needed to replicate the meta-analysis or to assess its validity and 67.6% of the information considered to be most important according to expert meta-analysts. More recently published meta-analyses exhibited somewhat more transparent reporting practices than older ones did. Overall transparency of reporting (but not reporting of the most important items) was associated with higher ranked journals; transparency was not significantly related to number of citations. The authors discuss the implications of inadequate reporting of meta-analyses for development of cumulative knowledge and effective practice and make suggestions for improving the current state of affairs.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Perceived organizational support: a review of the literature.
Rhoades, L., & Eisenberger, R. (2002) · Journal of Applied Psychology · doi:10.1037/0021-9010.87.4.698 · Cited by 7,592
Justice at the millennium: A meta-analytic review of 25 years of organizational justice research.
Colquitt, J. A., Conlon, D. E., Wesson, M. J., Porter, C. O. L. H., & Ng, K. Y. (2001) · Journal of Applied Psychology · doi:10.1037/0021-9010.86.3.425 · Cited by 5,942
Moher, D., Cook, D. J., Eastwood, S., Olkin, I., Rennie, D., & Stroup, D. F. (1999) · The Lancet · doi:10.1016/s0140-6736(99)04149-5 · Cited by 4,826
Harter, J. K., Schmidt, F. L., & Hayes, T. L. (2002) · Journal of Applied Psychology · doi:10.1037/0021-9010.87.2.268 · Cited by 4,623
Trust in leadership: Meta-analytic findings and implications for research and practice.
Dirks, K. T., & Ferrin, D. L. (2002) · Journal of Applied Psychology · doi:10.1037/0021-9010.87.4.611 · Cited by 3,550
A meta-analytic examination of the correlates of the three dimensions of job burnout
Lee, R. T., & Ashforth, B. E. (1996) · Journal of Applied Psychology · doi:10.1037/0021-9010.81.2.123 · Cited by 3,262
Personality and leadership: A qualitative and quantitative review
Judge, T. A., Bono, J. E., Ilies, R., & Gerhardt, M. W. (2002) · Journal of Applied Psychology · doi:10.1037/0021-9010.87.4.765 · Cited by 3,190
Task versus relationship conflict, team performance, and team member satisfaction: A meta-analysis.
De Dreu, C. K. W., & Weingart, L. R. (2003) · Journal of Applied Psychology · doi:10.1037/0021-9010.88.4.741 · Cited by 3,071
Meta-Analytic review of leader–member exchange theory: Correlates and construct issues.
Gerstner, C. R., & Day, D. V. (1997) · Journal of Applied Psychology · doi:10.1037/0021-9010.82.6.827 · Cited by 3,005
Colquitt, J. A., Scott, B. A., & LePine, J. A. (2007) · Journal of Applied Psychology · doi:10.1037/0021-9010.92.4.909 · Cited by 2,606
Cited by
Best-practice recommendations for defining, identifying, and handling outliers
Aguinis, H., Gottfredson, R. K., & Joo, H. (2013) · Organizational Research Methods · doi:10.1177/1094428112470848 · Cited by 1,341
Aguinis, H., Gottfredson, R. K., & Culpepper, S. A. (2013) · Journal of Management · doi:10.1177/0149206313478188 · Cited by 1,090
Abusive supervision: A meta-analysis and empirical review
Mackey, J. D., Frieder, R. E., Brees, J. R., & Martinko, M. J. (2017) · Journal of Management · doi:10.1177/0149206315573997 · Cited by 706
Sample Selection in Systematic Literature Reviews of Management Research
Hiebl, M. R. W. (2023) · Organizational Research Methods · doi:10.1177/1094428120986851 · Cited by 635
Digital transformation: A review and research agenda
Plekhanov, D., Franke, H., & Netland, T. H. (2023) · European Management Journal · doi:10.1016/j.emj.2022.09.007 · Cited by 606
Metadata
- Authors
- Aytug, Z. G., Rothstein, H. R., Zhou, W., & Kern, M. C.
- Venue
- Organizational Research Methods
- Publisher
- SAGE Publications
- Published
- 2011-04-05
- Volume
- 15
- Issue
- 1
- Pages
- 103-133
- ISSN
- 1094-4281 (ISSN-L)
- 1552-7425
- OpenAlex
- W2164715050
- Cited by
- 147
- References
- 61
- FWCI
- 6.42
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Revealed or Concealed? Transparency of Procedures, Decisions, and Judgment Calls in Meta-Analyses. https://app.cochat.ai/papers/10.1177/1094428111403495
- The paper
- Aytug, Z. G., Rothstein, H. R., Zhou, W., & Kern, M. C. (2012). Revealed or Concealed? Transparency of Procedures, Decisions, and Judgment Calls in Meta-Analyses. Organizational Research Methods. https://doi.org/10.1177/1094428111403495
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.