Guidelines for Meta-Analyses of Counseling Psychology Research

Quintana, Stephen M.; Minami, Takuya

doi:10.1177/0011000006286991

Cited by 83Meta-analysis and systematic reviewsPsychology Research and BibliometricsCounseling Practices and Supervision

Guidelines for Meta-Analyses of Counseling Psychology Research

Quintana, Stephen M., Minami, Takuya·2006·The Counseling Psychologist

DOI 10.1177/0011000006286991

Open this paper in CoChat — free account

Abstract

This article conceptually describes the steps in conducting quantitative meta-analyses of counseling psychology research with minimal reliance on statistical formulas. The authors identify sources that describe necessary statistical formula for various meta-analytic calculations and describe recent developments in meta-analytic techniques. The authors illustrate meta-analytic procedures with recently published meta-analyses in counseling psychology, and last, they provide a guide for writing up meta-analytic studies for publication.

Integrity signals

Registry

Crossref · 10.1177/0011000006286991

Cites

110 of 110 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Statistical Power Analysis for the Behavioral Sciences
Muller, K. E., & Cohen, J. (1989) · Technometrics · doi:10.2307/1270020 · Cited by 99,588 · see citing works
Statistical power analysis for the behavioral sciences
Cohen, J. (1990) · Computers Environment and Urban Systems · doi:10.1016/0198-9715(90)90050-4 · Cited by 65,406 · see citing works
Statistical Power Analysis for the Behavioral Sciences (2nd ed.).
Lachenbruch, P. A., & Cohen, J. (1989) · Journal of the American Statistical Association · doi:10.2307/2290095 · Cited by 37,385 · see citing works
Untitled work
(2002) · Cited by 13,211 · see citing works
Book Reviews: Statistical Methods for Meta-Analysis Larry V. Hedges and Ingram Olkin New York: Academic Press, 1985. xxii + 369 pp
Wasserman, S. S., Hedges, L. V., & Olkin, I. (1988) · Journal of Educational Statistics · doi:10.2307/1164953 · Cited by 10,347 · see citing works
Untitled work
(2000) · Cited by 9,198 · see citing works
The file drawer problem and tolerance for null results.
Rosenthal, R. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.3.638 · Cited by 9,068 · see citing works
Methods of Meta-Analysis: Correcting Error and Bias in Research Findings
Schmidt, F. L., & Hunter, J. E. (2015) · doi:10.4135/9781483398105 · Cited by 5,542 · see citing works
Making Vocational Choices: A Theory of Vocational Personalities and Work Environments
Holland, J. L. (1973) · Cited by 4,990 · see citing works
Statistical Methods for Meta-Analysis.
Freeman, P. R., Hedges, L. V., & Olkin, I. (1986) · Biometrics · doi:10.2307/2531069 · Cited by 4,832 · see citing works
Meta-Analytic Procedures for Social Research
Rosenthal, R. (1991) · doi:10.4135/9781412984997 · Cited by 4,635 · see citing works
Scale Development Research: A Content Analysis and Recommendations for Best Practices
Worthington, R. L., & Whittaker, T. A. (2006) · The Counseling Psychologist · doi:10.1177/0011000006288127 · Cited by 4,087 · see citing works
The earth is round (p < .05).
Cohen, J. (1994) · American Psychologist · doi:10.1037/0003-066x.49.12.997 · Cited by 4,003 · see citing works
The Handbook of Research Synthesis.
Howel, D., Cooper, H., & Hedges, L. V. (1995) · Biometrics · doi:10.2307/2533295 · Cited by 3,657 · see citing works
Fixed-and random-effects models in meta-analysis.
Hedges, L. V., & Vevea, J. L. (1998) · Psychological Methods · doi:10.1037/1082-989x.3.4.486 · Cited by 2,977 · see citing works
A Brief Guide to Structural Equation Modeling
Weston, R., & Gore, P. A. (2006) · The Counseling Psychologist · doi:10.1177/0011000006286345 · Cited by 2,718 · see citing works
Handbook of multicultural counseling
(1995) · Choice Reviews Online · doi:10.5860/choice.33-2419 · Cited by 2,635 · see citing works
Methods of Meta-Analysis: Correcting Error and Bias in Research Findings.
Raudenbush, S. W., Hunter, J. E., & Schmidt, F. L. (1991) · Journal of the American Statistical Association · doi:10.2307/2289738 · Cited by 2,363 · see citing works
Relation of self-efficacy beliefs to academic outcomes: A meta-analytic investigation.
Multon, K. D., Brown, S. D., & Lent, R. W. (1991) · Journal of Counseling Psychology · doi:10.1037/0022-0167.38.1.30 · Cited by 2,309 · see citing works
Meta-analysis of psychotherapy outcome studies.
Smith, M. L., & Glass, G. V. (1977) · American Psychologist · doi:10.1037/0003-066x.32.9.752 · Cited by 2,154 · see citing works
Black and white racial identity: theory, research, and practice
(1990) · Choice Reviews Online · doi:10.5860/choice.28-0596 · Cited by 2,055 · see citing works
The efficacy of psychological, educational, and behavioral treatment: Confirmation from meta-analysis.
Lipsey, M. W., & Wilson, D. B. (1993) · American Psychologist · doi:10.1037/0003-066x.48.12.1181 · Cited by 2,033 · see citing works
The Handbook of Research Synthesis.
Light, R. J., Cooper, H., & Hedges, L. V. (1994) · Journal of the American Statistical Association · doi:10.2307/2291021 · Cited by 1,880 · see citing works
Parametric measures of effect size.
Rosenthal, R. (1994) · Cited by 1,856 · see citing works
The effects of psychotherapy: an evaluation.
Eysenck, H. J. (1952) · Journal of Consulting Psychology · doi:10.1037/h0063633 · Cited by 1,734 · see citing works
The Great Psychotherapy Debate: Models Methods and Findings
Horvath, A. O. (2002) · Psychotherapy Research · doi:10.1093/ptr/12.1.108 · Cited by 1,705 · see citing works
On the Use of Beta Coefficients in Meta-Analysis.
Peterson, R. A., & Brown, S. P. (2005) · Journal of Applied Psychology · doi:10.1037/0021-9010.90.1.175 · Cited by 1,643 · see citing works
Generalized Eta and Omega Squared Statistics: Measures of Effect Size for Some Common Research Designs.
Olejnik, S., & Algina, J. (2003) · Psychological Methods · doi:10.1037/1082-989x.8.4.434 · Cited by 1,622 · see citing works
A simple, general purpose display of magnitude of experimental effect.
Rosenthal, R., & Rubin, D. B. (1982) · Journal of Educational Psychology · doi:10.1037//0022-0663.74.2.166 · Cited by 1,233 · see citing works
Writing meta-analytic reviews.
Rosenthal, R. (1995) · Psychological Bulletin · doi:10.1037/0033-2909.118.2.183 · Cited by 1,108 · see citing works
A meta-analysis of outcome studies comparing bona fide psychotherapies: Empiricially, "all must have prizes."
Wampold, B. E., Mondin, G. W., Moody, M., Stich, F., Benson, K., & Ahn, H.-N. (1997) · Psychological Bulletin · doi:10.1037/0033-2909.122.3.203 · Cited by 1,091 · see citing works
Effects of involvement on persuasion: A meta-analysis.
Johnson, B. T., & Eagly, A. H. (1989) · Psychological Bulletin · doi:10.1037/0033-2909.106.2.290 · Cited by 986 · see citing works
Comprehensive Meta-Analysis
Espinosa, J. C. (2004) · Cited by 939 · see citing works
A simple, general purpose display of magnitude of experimental effect.
Rosenthal, R., & Rubin, D. B. (1982) · Journal of Educational Psychology · doi:10.1037/0022-0663.74.2.166 · Cited by 920 · see citing works
The power of statistical tests in meta-analysis
Hedges, L. V., & Pigott, T. D. (2001) · Psychological Methods · doi:10.1037/1082-989x.6.3.203 · Cited by 716 · cited 4× · see citing works
“Using Hedges and Pigott (2001), first, the researchers must specify the magnitude of effect size that would be scientifically meaningful or would reflect practical significance.”
BackgroundMethodInfluential
Factor Analysis in Counseling Psychology Research, Training, and Practice
Kahn, J. H. (2006) · The Counseling Psychologist · doi:10.1177/0011000006286347 · Cited by 694 · see citing works
The power of statistical tests for moderators in meta-analysis
Hedges, L. V., & Pigott, T. D. (2004) · Psychological Methods · doi:10.1037/1082-989x.9.4.426 · Cited by 672 · see citing works
Effect-Size Indices for Dichotomized Outcomes in Meta-Analysis.
Sánchez‐Meca, J., Marín‐Martínez, F., & Moscoso, S. C. (2003) · Psychological Methods · doi:10.1037/1082-989x.8.4.448 · Cited by 635 · see citing works
Where oh where are the specific ingredients? A meta-analysis of component studies in counseling and psychotherapy.
Ahn, H.-N., & Wampold, B. E. (2001) · Journal of Counseling Psychology · doi:10.1037/0022-0167.48.3.251 · Cited by 571 · see citing works
Measurement error in psychological research: Lessons from 26 research scenarios.
Schmidt, F. L., & Hunter, J. E. (1996) · Psychological Methods · doi:10.1037/1082-989x.1.2.199 · Cited by 567 · see citing works
How to Estimate and Interpret Various Effect Sizes.
Vacha‐Haase, T., & Thompson, B. (2004) · Journal of Counseling Psychology · doi:10.1037/0022-0167.51.4.473 · Cited by 538 · see citing works
File drawer problem
Brand, J. L. (1997) · Digital Commons - Andrews University (Andrews University) · Cited by 535 · see citing works
Sex of researchers and sex-typed communications as determinants of sex differences in influenceability: A meta-analysis of social influence studies.
Eagly, A. H., & Carli, L. L. (1981) · Psychological Bulletin · doi:10.1037/0033-2909.90.1.1 · Cited by 525 · see citing works
Contrasts and Correlations in Effect-Size Estimation
Rosnow, R. L., Rosenthal, R., & Rubin, D. B. (2000) · Psychological Science · doi:10.1111/1467-9280.00287 · Cited by 513 · see citing works
Vote-counting methods in research synthesis.
Hedges, L., & Olkin, I. (1980) · Psychological Bulletin · doi:10.1037/0033-2909.88.2.359 · Cited by 511 · see citing works
How are we doing in soft psychology?
Rosenthal, R. (1990) · American Psychologist · doi:10.1037/0003-066x.45.6.775 · Cited by 507 · see citing works
How meta-analysis increases statistical power.
Cohn, L. D., & Becker, B. J. (2003) · Psychological Methods · doi:10.1037/1082-989x.8.3.243 · Cited by 500 · see citing works
Stochastically dependent effect sizes.
Gleser, L. J., & Olkin, I. (1994) · Cited by 485 · see citing works
An exercise in mega-silliness.
Eysenck, H. J. (1978) · American Psychologist · doi:10.1037/0003-066x.33.5.517.a · Cited by 466 · see citing works
Untitled work
(1994) · Cited by 463 · see citing works
How hard is hard science, how soft is soft science? The empirical cumulativeness of research.
Hedges, L. V. (1987) · American Psychologist · doi:10.1037/0003-066x.42.5.443 · Cited by 435 · see citing works
EVALUATING CODING DECISIONS
Vevea, J. L., Zelinsky, N. A. M., & Orwin, R. G. (2019) · Russell Sage Foundation eBooks · doi:10.7758/9781610448864.13 · Cited by 417 · see citing works
Effect sizes for experimenting psychologists.
Rosnow, R. L., & Rosenthal, R. (2003) · Canadian Journal of Experimental Psychology/Revue canadienne de psychologie expérimentale · doi:10.1037/h0087427 · Cited by 416 · see citing works
Grey Literature in Meta-Analyses
Conn, V. S., Valentine, J. C., Cooper, H. M., & Rantz, M. J. (2003) · Nursing Research · doi:10.1097/00006199-200307000-00008 · Cited by 400 · see citing works
Selection Models and the File Drawer Problem
Iyengar, S., & Greenhouse, J. B. (1988) · Statistical Science · doi:10.1214/ss/1177013012 · Cited by 399 · see citing works
Fixed effects models.
Hedges, L. V. (1994) · Cited by 397 · see citing works
requivalent: A simple effect size indicator.
Rosenthal, R., & Rubin, D. B. (2003) · Psychological Methods · doi:10.1037/1082-989x.8.4.492 · Cited by 380 · see citing works
Statistical Methods for the Social and Behavioral Sciences.
Stallings, W. M., Marascuilo, L. A., & Serlin, R. C. (1989) · Journal of the American Statistical Association · doi:10.2307/2289907 · Cited by 371 · see citing works
Establishing Specificity in Psychotherapy: A Meta-Analysis of Structural Equivalence of Placebo Controls.
Baskin, T. W., Tierney, S. C., Minami, T., & Wampold, B. E. (2003) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.71.6.973 · Cited by 351 · see citing works
Self-citation
Handbook of multicultural counseling, 2nd ed.
Ponterotto, J. G., Casas, J., Suzuki, L. A., & Alexander, C. M. (2001) · Cited by 337 · see citing works
Of apples and oranges, file drawers and garbage: Why validity issues in meta-analysis will not go away
Sharpe, D. (1997) · Clinical Psychology Review · doi:10.1016/s0272-7358(97)00056-1 · Cited by 334 · see citing works
Career-intervention outcome: A replication and extension of Oliver and Spokane (1988).
Whiston, S. C., Sexton, T. L., & Lasoff, D. L. (1998) · Journal of Counseling Psychology · doi:10.1037/0022-0167.45.2.150 · Cited by 322 · see citing works
Magnitude of experimental effect and a table for its rapid estimation.
Friedman, H. (1968) · Psychological Bulletin · doi:10.1037/h0026258 · Cited by 317 · see citing works
A meta-(re) analysis of the effects of cognitive therapy versus ‘other therapies’ for depression
Wampold, B. E., Minami, T., Baskin, T. W., & Callen Tierney, S. (2002) · Journal of Affective Disorders · doi:10.1016/s0165-0327(00)00287-1 · Cited by 305 · see citing works
Self-citation
Treating Cronbach’s Alpha Reliability Coefficients as Data in Counseling Research
Helms, J. E., Henze, K. T., Sass, T. L., & Mifsud, V. A. (2006) · The Counseling Psychologist · doi:10.1177/0011000006288308 · Cited by 276 · see citing works
Book Review: Making Vocational Choices: A Theory of Vocational Personalities & Work Environments (2nd ed.). John L. Holland
Wallman, G. H. (1985) · NASPA Journal · doi:10.1080/00220973.1985.11071939 · Cited by 273 · see citing works
Mediators and moderators in meta-analysis: There's a reason we don't let dodo birds tell us which psychotherapies should have prizes.
Shadish, W. R., & Sweeney, R. B. (1991) · Journal of Consulting and Clinical Psychology · doi:10.1037//0022-006x.59.6.883 · Cited by 253 · see citing works
Measures of effect size for categorical data.
Fleiss, J. L. (1994) · Cited by 248 · see citing works
Cross-cultural structural equivalence of RIASEC models and measures.
Rounds, J., & Tracey, T. J. G. (1996) · Journal of Counseling Psychology · doi:10.1037/0022-0167.43.3.310 · Cited by 247 · see citing works
Advanced Applications of Structural Equation Modeling in Counseling Psychology Research
Martens, M. P., & Haase, R. F. (2006) · The Counseling Psychologist · doi:10.1177/0011000005283395 · Cited by 246 · see citing works
Psychotherapy outcome: Smith and Glass' conclusions stand up under scrutiny.
Landman, J., & Dawes, R. M. (1982) · American Psychologist · doi:10.1037//0003-066x.37.5.504 · Cited by 226 · see citing works
SCIENTIFIC COMMUNICATION AND LITERATURE RETRIEVAL
White, H. D. (2019) · Russell Sage Foundation eBooks · doi:10.7758/9781610448864.7 · Cited by 221 · see citing works
All Must Have Prizes
Phillips, M. (1996) · Medical Entomology and Zoology · Cited by 216 · see citing works
Using the normal quantile plot to explore meta-analytic data sets.
Wang, M. C., & Bushman, B. J. (1998) · Psychological Methods · doi:10.1037/1082-989x.3.1.46 · Cited by 216 · see citing works
Effect-Size Measures and Meta-Analytic Thinking in Counseling Psychology Research
Henson, R. K. (2006) · The Counseling Psychologist · doi:10.1177/0011000005283558 · Cited by 200 · see citing works
The consequence of ignoring a nested factor on measures of effect size in analysis of variance.
Wampold, B. E., & Serlin, R. C. (2000) · Psychological Methods · doi:10.1037/1082-989x.5.4.425 · Cited by 186 · see citing works
Effectiveness of grief therapy: A meta-analysis.
Allumbaugh, D. L., & Hoyt, W. T. (1999) · Journal of Counseling Psychology · doi:10.1037/0022-0167.46.3.370 · Cited by 185 · see citing works
Mediators and moderators in meta-analysis: There's a reason we don't let dodo birds tell us which psychotherapies should have prizes.
Shadish, W. R., & Sweeney, R. B. (1991) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.59.6.883 · Cited by 177 · see citing works
Psychotherapy outcome: Smith and Glass' conclusions stand up under scrutiny.
Landman, J., & Dawes, R. M. (1982) · American Psychologist · doi:10.1037/0003-066x.37.5.504 · Cited by 170 · see citing works
Judging research quality.
Wortman, P. M. (1994) · Cited by 154 · see citing works
MetaWin: Statistical Software for Meta-analysis with Resampling Tests
Adams, D. C., Gurevitch, J., & Rosenburg, M. S. (1997) · Cited by 153 · see citing works
Advantages of excluding underpowered studies in meta-analysis: Inclusionist versus exclusionist viewpoints.
Kraemer, H. C., Gardner, C., Brooks, J. O., & Yesavage, J. A. (1998) · Psychological Methods · doi:10.1037/1082-989x.3.1.23 · Cited by 141 · see citing works
Ethnic minorities' ratings of ethnically similar and European American counselors: A meta-analysis.
Coleman, H. L. K., Wampold, B. E., & Casali, S. L. (1995) · Journal of Counseling Psychology · doi:10.1037/0022-0167.42.1.55 · Cited by 136 · see citing works
Research on religion-accomodative counseling: Review and meta-analysis.
McCullough, M. E. (1999) · Journal of Counseling Psychology · doi:10.1037/0022-0167.46.1.92 · Cited by 136 · see citing works
Dependent Effect Sizes in Meta-Analysis: Incorporating the Degree of Interdependence.
Cheung, S. F., & Chan, D. K. ‐. (2004) · Journal of Applied Psychology · doi:10.1037/0021-9010.89.5.780 · Cited by 118 · see citing works
Discriminant Analysis in Counseling Psychology Research
Sherry, A. R. (2006) · The Counseling Psychologist · doi:10.1177/0011000006287103 · Cited by 116 · see citing works
Construct Validation in Counseling Psychology Research
Hoyt, W. T., Warbasse, R. E., & Chu, E. Y. (2006) · The Counseling Psychologist · doi:10.1177/0011000006287389 · Cited by 115 · see citing works
A Scientist-Practitioner Model of Psychological Assessment
Spengler, P. M., Strohmer, D. C., Dixon, D. N., & Shivy, V. A. (1995) · The Counseling Psychologist · doi:10.1177/0011000095233009 · Cited by 101 · see citing works
Relation between conceptual level and counseling-related tasks: A meta-analysis.
Holloway, E. L., & Wampold, B. E. (1986) · Journal of Counseling Psychology · doi:10.1037/0022-0167.33.3.310 · Cited by 100 · see citing works
IDENTIFYING POTENTIALLY INTERESTING VARIABLES AND ANALYSIS OPPORTUNITIES
Lipsey, M. W. (2019) · Russell Sage Foundation eBooks · doi:10.7758/9781610448864.11 · Cited by 100 · see citing works
Hypotheses and problems in research synthesis.
Hall, J. A., Tickle‐Degnen, L., Rosenthal, R., & Mosteller, F. C. (1994) · Cited by 98 · see citing works
A Meta-Analytic Review of Racial-Ethnic Matching for African American and Caucasian American Clients and Clinicians.
Shin, S., Chow, C. M., Camacho-Gonsalves, T., Levy, R. J., Allen, I. E., & Leff, H. S. (2005) · Journal of Counseling Psychology · doi:10.1037/0022-0167.52.1.45 · Cited by 90 · see citing works
Does the White Racial Identity Attitude Scale measure racial identity?
Behrens, J. T. (1997) · Journal of Counseling Psychology · doi:10.1037/0022-0167.44.1.3 · Cited by 81 · see citing works
How significant is a significant difference? Average effect size of research in counseling psychology.
Haase, R. F., Waechter, D. M., & Solomon, G. S. (1982) · Journal of Counseling Psychology · doi:10.1037/0022-0167.29.1.58 · Cited by 79 · see citing works
The meta‐analysis of partial effect sizes
Keef, S. P., & Roberts, L. (2004) · British Journal of Mathematical and Statistical Psychology · doi:10.1348/000711004849303 · Cited by 77 · see citing works
Multiple Criteria for Evaluating the Magnitude of Experimental Effects.
Haase, R., Ellis, M. V., & Ladany, N. (1989) · Journal of Counseling Psychology · doi:10.1037/0022-0167.36.4.511 · Cited by 66 · see citing works
Should Providers of Treatment Be Regarded as a Random Factor? If It Ain't Broke, Don't "Fix" It: A Comment on Siemer and Joormann (2003).
Serlin, R. C., Wampold, B. E., & Levin, J. R. (2003) · Psychological Methods · doi:10.1037/1082-989x.8.4.524 · Cited by 58 · see citing works
Applied statistical power analysis and the interpretation of nonsignificant results by research consumers.
Fagley, N. (1985) · Journal of Counseling Psychology · doi:10.1037/0022-0167.32.3.391 · Cited by 57 · see citing works
Cultural validity and inherent challenges in quantitative methods for multicultural research.
Quintana, S. M., Troyano, N., & Taylor, G. M. (2001) · Cited by 56 · see citing works
Self-citation
Psychotherapy length of stay and outcome: A meta-analysis of the effect of therapist sex.
Bowman, D., Scogin, F. R., Floyd, M. R., & McKendree-Smith, N. L. (2001) · Psychotherapy · doi:10.1037/0033-3204.38.2.142 · Cited by 48 · see citing works
Client-Case Manager Racial Matching in a Program for Homeless Persons With Serious Mental Illness
Chinman, M., Rosenheck, R. A., & Lam, J. A. (2000) · Psychiatric Services · doi:10.1176/appi.ps.51.10.1265 · Cited by 40 · see citing works
Antecedents and effects of perceived therapist credibility: A meta-analysis.
Hoyt, W. T. (1996) · Journal of Counseling Psychology · doi:10.1037/0022-0167.43.4.430 · Cited by 34 · see citing works
Antecedents and effects of perceived therapist credibility: A meta-analysis.
Hoyt, W. T. (1996) · Journal of Counseling Psychology · doi:10.1037//0022-0167.43.4.430 · Cited by 33 · see citing works
Meta-Analysis and Power: Some Suggestions for the Use of Power in Research Synthesis
Muncer, S., Craigie, M. A., & Holmes, J. (2003) · Understanding Statistics · doi:10.1207/s15328031us0201_01 · Cited by 29 · see citing works
The Metric Comparability of Meta-Analytic Effect-Size Estimators From Factorial Designs.
Gillett, R. (2003) · Psychological Methods · doi:10.1037/1082-989x.8.4.419 · Cited by 27 · see citing works
Some comments on the use of magnitude-of-effect estimates.
Strube, M. (1988) · Journal of Counseling Psychology · doi:10.1037/0022-0167.35.3.342 · Cited by 26 · see citing works
Fail-Safe Ns for One- Versus Two-Tailed Tests Lead to Different Conclusions About Publication Bias
Hsu, L. (2002) · Understanding Statistics · doi:10.1207/s15328031us0102_02 · Cited by 24 · see citing works
Beyond significant yet meaningless results in psychotherapy research.
Paquin, M. J. R. (1983) · Psychotherapy · doi:10.1037/h0088476 · Cited by 15 · see citing works
Meta-Analyses of Cluster Randomization Trials
Donner, A. P., Piaggio, G. G., & Villar, J. (2003) · Evaluation & the Health Professions · doi:10.1177/0163278703255234 · Cited by 12 · see citing works
Pitfalls of Meta-Analyses
Barber, J. P., & Milrod, B. L. (2004) · American Journal of Psychiatry · doi:10.1176/appi.ajp.161.6.1131 · Cited by 5 · see citing works

Flagged first, then by citations

Sorted by citations

Scale Development Research: A Content Analysis and Recommendations for Best Practices
Worthington, R. L., & Whittaker, T. A. (2006) · The Counseling Psychologist · doi:10.1177/0011000006288127 · Cited by 4,087 · see citing works
A Brief Guide to Structural Equation Modeling
Weston, R., & Gore, P. A. (2006) · The Counseling Psychologist · doi:10.1177/0011000006286345 · Cited by 2,718 · see citing works
Racism as a Determinant of Health: A Systematic Review and Meta-Analysis
Paradies, Y., Ben, J., Denson, N., Elias, A., Priest, N., Pieterse, A., Gupta, A., Kelaher, M., & Gee, G. (2015) · PLOS ONE · doi:10.1371/journal.pone.0138511 · Cited by 2,610 · see citing works
“Third, we calculated a fail-safe N to estimate the number of un-located studies with an average zero effect size required to change the results substantively [65].”
Method
Correlates and consequences of internalized stigma for people living with mental illness: A systematic review and meta-analysis
Livingston, J. D., & Boyd, J. E. (2010) · Social Science & Medicine · doi:10.1016/j.socscimed.2010.09.030 · Cited by 1,677 · see citing works
Instructional animation versus static pictures: A meta-analysis
Höffler, T. N., & Leutner, D. (2007) · Learning and Instruction · doi:10.1016/j.learninstruc.2007.09.013 · Cited by 992 · see citing works
A systematic review of studies examining the relationship between reported racism and health and wellbeing for children and young people
Priest, N., Paradies, Y., Trenerry, B., Truong, M., Karlsen, S., & Kelly, Y. (2013) · Social Science & Medicine · doi:10.1016/j.socscimed.2012.11.031 · Cited by 877 · see citing works
Factor Analysis in Counseling Psychology Research, Training, and Practice
Kahn, J. H. (2006) · The Counseling Psychologist · doi:10.1177/0011000006286347 · Cited by 694 · see citing works
Racial/ethnic matching of clients and therapists in mental health services: A meta-analytic review of preferences, perceptions, and outcomes.
Cabral, R. R., & Smith, T. B. (2011) · Journal of Counseling Psychology · doi:10.1037/a0025266 · Cited by 651 · see citing works
Perceived Racism and Mental Health Among Black American Adults: A Meta-Analytic Review
Pieterse, A. L., Todd, N. R., Neville, H. A., & Carter, R. T. (2012) · Journal of Counseling Psychology · doi:10.1037/a0026208 · Cited by 649 · see citing works
Meta-analyses of the relationship between conformity to masculine norms and mental health-related outcomes.
Wong, Y. J., Ho, M.-H. R., Wang, S.-Y., & Miller, I. S. K. (2017) · Journal of Counseling Psychology · doi:10.1037/cou0000176 · Cited by 507 · see citing works
Relationship Between School Suspension and Student Outcomes: A Meta-Analysis
Noltemeyer, A. L., Ward, R. M., & Mcloughlin, C. S. (2015) · School Psychology Review · doi:10.17105/spr-14-0008.1 · Cited by 455 · see citing works
Qualitative Research in Counseling Psychology
Morrow, S. L. (2007) · The Counseling Psychologist · doi:10.1177/0011000006286990 · Cited by 450 · see citing works
Racism and health service utilisation: A systematic review and meta-analysis
Ben, J., Cormack, D. M., Harris, R. B., & Paradies, Y. (2017) · PLoS ONE · doi:10.1371/journal.pone.0189900 · Cited by 368 · see citing works
The relative efficacy of bona fide psychotherapies for treating post-traumatic stress disorder: A meta-analysis of direct comparisons
Benish, S. G., Imel, Z. E., & Wampold, B. E. (2007) · Clinical Psychology Review · doi:10.1016/j.cpr.2007.10.005 · Cited by 335 · see citing works
Spatial Ability: Its Influence on Learning with Visualizations—a Meta-Analytic Review
Höffler, T. N. (2010) · Educational Psychology Review · doi:10.1007/s10648-010-9126-7 · Cited by 334 · see citing works
Long-term psychodynamic psychotherapy in complex mental disorders: update of a meta-analysis
Leichsenring, F., & Rabung, S. (2011) · The British Journal of Psychiatry · doi:10.1192/bjp.bp.110.082776 · Cited by 300 · see citing works
Racial Discrimination and Health Outcomes Among Racial/Ethnic Minorities: A Meta‐Analytic Review
Carter, R. T., Lau, M. Y., Johnson, V. E., & Kirkinis, K. (2017) · Journal of Multicultural Counseling and Development · doi:10.1002/jmcd.12076 · Cited by 289 · see citing works
Examining the relationship between organizational commitment and turnover intention via a meta-analysis
Güzeller, C. O., & Çeliker, N. (2019) · International Journal of Culture Tourism and Hospitality Research · doi:10.1108/ijcthr-05-2019-0094 · Cited by 288 · see citing works
Neuropsychological consequences of chronic opioid use: a quantitative review and meta-analysis.
Baldacchino, A., Balfour, D. J. K., Passetti, F., Humphris, G., & Matthews, K. (2012) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2012.06.006 · Cited by 263 · see citing works
Research on religion/spirituality and forgiveness: A meta-analytic review.
Davis, D. E., Worthington, E. L., Hook, J. N., & Hill, P. C. (2013) · Psychology of Religion and Spirituality · doi:10.1037/a0033637 · Cited by 247 · see citing works
Advanced Applications of Structural Equation Modeling in Counseling Psychology Research
Martens, M. P., & Haase, R. F. (2006) · The Counseling Psychologist · doi:10.1177/0011000005283395 · Cited by 246 · see citing works
Racial Discrimination and Asian Mental Health: A Meta-Analysis
Lee, D. L., & Ahn, S. (2011) · The Counseling Psychologist · doi:10.1177/0011000010381791 · Cited by 227 · see citing works
The relation of racial identity, ethnic identity, and racial socialization to discrimination–distress: A meta-analysis of Black Americans
Lee, D. L., & Ahn, S. (2013) · Journal of Counseling Psychology · doi:10.1037/a0031275 · Cited by 222 · see citing works
Discrimination Against Latina/os: A Meta-Analysis of Individual-Level Resources and Outcomes
Lee, D. L., & Ahn, S. (2012) · The Counseling Psychologist · doi:10.1177/0011000011403326 · Cited by 208 · see citing works
Content analysis of acculturation research in counseling and counseling psychology: A 22-year review.
Yoon, E., Langrehr, K. J., & Ong, L. Z. (2010) · Journal of Counseling Psychology · doi:10.1037/a0021128 · Cited by 203 · see citing works
Unique effects and moderators of effects of sources on self-efficacy: A model-based meta-analysis.
Byars‐Winston, A. M., Diestelmann, J., Savoy, J. N., & Hoyt, W. T. (2017) · Journal of Counseling Psychology · doi:10.1037/cou0000219 · Cited by 200 · see citing works
The contributions of parental attachment bonds to college student development and adjustment: A meta-analytic review.
Mattanah, J. F., Lopez, F. G., & Govern, J. M. (2011) · Journal of Counseling Psychology · doi:10.1037/a0024635 · Cited by 192 · see citing works
Paradigms, Purpose, and the Role of the Literature
Haverkamp, B. E., & Young, R. A. (2007) · The Counseling Psychologist · doi:10.1177/0011000006292597 · Cited by 181 · see citing works
Distinctions without a difference: Direct comparisons of psychotherapies for alcohol use disorders.
Imel, Z. E., Wampold, B. E., Miller, S. D., & Fleming, R. (2008) · Psychology of Addictive Behaviors · doi:10.1037/a0013171 · Cited by 174 · see citing works
Neuropsychological basic deficits in preschoolers at risk for ADHD: A meta-analysis
Pauli-Pott, U., & Becker, K. (2011) · Clinical Psychology Review · doi:10.1016/j.cpr.2011.02.005 · Cited by 164 · see citing works
Neurocognitive consequences of chronic cannabis use: a systematic review and meta-analysis
Figueiredo, P. R., Tolomeo, S., Steele, J. D., & Baldacchino, A. (2019) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2019.10.014 · Cited by 141 · see citing works
Neurobehavioral consequences of chronic intrauterine opioid exposure in infants and preschool children: a systematic review and meta-analysis
Baldacchino, A., Arbuckle, K., Petrie, D., & McCowan, C. (2014) · BMC Psychiatry · doi:10.1186/1471-244x-14-104 · Cited by 130 · see citing works
Construct Validation in Counseling Psychology Research
Hoyt, W. T., Warbasse, R. E., & Chu, E. Y. (2006) · The Counseling Psychologist · doi:10.1177/0011000006287389 · Cited by 115 · see citing works
Benchmarks for psychotherapy efficacy in adult major depression.
Minami, T., Wampold, B. E., Serlin, R. C., Kircher, J. C., & Brown, G. S. (. (2007) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.75.2.232 · Cited by 113 · see citing works
Self-citation
Board Attributes and Corporate Social Responsibility Disclosure: A Meta-Analysis
Guerrero‐Villegas, J., Pérez‐Calero, L., González, J. M. H., & Giráldez, P. (2018) · Sustainability · doi:10.3390/su10124808 · Cited by 106 · see citing works
Resource-Based Perspective on ICT Use and Firm Performance: A Meta-analysis Investigating the Moderating Role of Cross-Country ICT Development Status
Karim, M. S., Nahar, S., & Demirbağ, M. (2022) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2022.121626 · Cited by 96 · see citing works
A Meta-Analysis of the Relationship Between Internalized Racial Oppression and Health-Related Outcomes
Gale, M. M., Pieterse, A. L., Lee, D. L., Huynh, K., Powell, S., & Kirkinis, K. (2020) · The Counseling Psychologist · doi:10.1177/0011000020904454 · Cited by 94 · see citing works
Student evaluations of teaching: combining the meta-analyses and demonstrating further evidence for effective use
Wright, S. L., & Jenkins‐Guarnieri, M. A. (2011) · Assessment & Evaluation in Higher Education · doi:10.1080/02602938.2011.563279 · Cited by 90 · see citing works
Clinical utility of categorical and dimensional perspectives on personality pathology: A meta-analytic review.
Bornstein, R. F., & Natoli, A. P. (2019) · Personality Disorders Theory Research and Treatment · doi:10.1037/per0000365 · Cited by 86 · see citing works
Effectiveness of Long-term Psychodynamic Psychotherapy: A Meta-analysis
Leichsenring, F. (2008) · JAMA · doi:10.1001/jama.300.13.1551 · Cited by 65 · see citing works
School-Wide Positive Behavioral Interventions and Supports (SWPBIS): A synthesis of existing research
Noltemeyer, A. L., Palmer, K., James, A. G., & Wiechman, S. (2018) · International Journal of School & Educational Psychology · doi:10.1080/21683603.2018.1425169 · Cited by 61 · see citing works
Efficacy of Counseling and Psychotherapy in Schools: A Meta-Analytic Review of Treatment Outcome Studies 1Ψ7
Baskin, T. W., Slaten, C. D., Crosby, N. R., Pufahl, T., Schneller, C. L., & Ladell, M. (2010) · The Counseling Psychologist · doi:10.1177/0011000010369497 · Cited by 56 · see citing works
Workplace Waste Recycling Behaviour: A Meta-Analytical Review
Oke, A. (2015) · Sustainability · doi:10.3390/su7067175 · Cited by 39 · see citing works
Evidence‐based practice in psychology: Implications for research and research training
Bauer, R. M. (2007) · Journal of Clinical Psychology · doi:10.1002/jclp.20374 · Cited by 39 · see citing works
Initial evidence for the criterion‐related and structural validity of the long versions of the Coach–Athlete Relationship Questionnaire
Rhind, D. J. A., & Jowett, S. (2010) · European Journal of Sport Science · doi:10.1080/17461391003699047 · Cited by 38 · see citing works
Social Participation Interventions for Adults with Mental Health Problems: A Review and Narrative Synthesis
Fendt-Newlin, M., Webber, M., Morris, D., & Howarth, S. (2015) · Social Work Research · doi:10.1093/swr/svv015 · Cited by 38 · see citing works
Do the institutional environment and types of owners influence the relationship between ownership concentration and board of director independence? An international meta-analysis
Pérez‐Calero, L., González, J. M. H., & López‐Iturriaga, F. J. (2018) · International Review of Financial Analysis · doi:10.1016/j.irfa.2018.11.014 · Cited by 34 · see citing works
Unpacking stigma: meta-analyses of correlates and moderators of personal stigma in psychosis
Eliasson, E. T., McNamee, L., Swanson, L., Lawrie, S. M., & Schwannauer, M. (2021) · Clinical Psychology Review · doi:10.1016/j.cpr.2021.102077 · Cited by 33 · see citing works
Meta‐Analysis: A Systematic Method for Synthesizing Counseling Research
Whiston, S. C., & Li, P. (2011) · Journal of Counseling & Development · doi:10.1002/j.1556-6678.2011.tb00089.x · Cited by 27 · see citing works
A statistical view of research synthesis of patterns of species richness along productivity gradients: devils, forests and trees
Gurevitch, J., & Mengersen, K. L. (2010) · Ecology · doi:10.1890/09-1039.1 · Cited by 25 · see citing works
The value for innovation of inter-firm networks and forming alliances: A meta-analytic model of indirect effects
Salazar, A. J., González, J. M. H., Duysters, G. M., Sabidussi, A., & Allen, M. R. (2016) · Computers in Human Behavior · doi:10.1016/j.chb.2016.05.073 · Cited by 21 · see citing works
A Meta-Analysis of School Belonging and Academic Success and Persistence
Moallem, I. (2013) · Loyola eCommons (Loyola University of Chicago) · Cited by 20 · see citing works
Systematic review of meta-analysis studies in the tourism and hospitality literature
Ustunel, M. F., Çeliker, N., & Güzeller, C. O. (2021) · European Journal of Tourism Research · doi:10.54055/ejtr.v27i.2145 · Cited by 18 · see citing works
Cognitive-Behavioral Therapy for Mania: A Meta-Analysis of Randomized Controlled Trials
Gregory, V. L. (2010) · Social Work in Mental Health · doi:10.1080/15332981003744388 · Cited by 18 · see citing works
Racism in Australia: a protocol for a systematic review and meta-analysis
Ben, J., Elias, A., Issaka, A., Truong, M., Dunn, K. M., Sharples, R., McGarty, C., Walton, J., Mansouri, F., Denson, N., et al. (2022) · Systematic Reviews · doi:10.1186/s13643-022-01919-2 · Cited by 17 · see citing works
Comment: Why Meta-Analyses Rarely Resolve Ideological Debates
Ferguson, C. J. (2014) · Emotion Review · doi:10.1177/1754073914523046 · Cited by 15 · see citing works
A meta‐analysis of support variables and well‐being in ethnic minority populations
Vera, E. M., Yoon, E., Chander, N., Kim, D., Liu, H., Kolas, D., Norgord, R., Gomez, M., Daniels, R., Matthews, K., et al. (2020) · Journal of Applied Social Psychology · doi:10.1111/jasp.12703 · Cited by 14 · see citing works
Key Challenges and Some Guidance on Using Strong Quantitative Methodology in Education Research
Henson, R. K., Stewart, G., & Bedford, L. A. (2020) · Journal of Urban Mathematics Education · doi:10.21423/jume-v13i2a382 · Cited by 14 · see citing works
A tale of two RCTs: Using Randomized Controlled Trials to benchmark Routine Clinical (psychological) Treatments for chronic pain
Fenton, G., & Morley, S. (2013) · Pain · doi:10.1016/j.pain.2013.06.033 · Cited by 14 · see citing works
Metaanalyse–praktische Schritte und Entscheidungen im Umsetzungsprozess
Schewe, A. F., Hulsheger, U. R., & Maier, G. W. (2014) · Zeitschrift für Arbeits- und Organisationspsychologie A&O · doi:10.1026/0932-4089/a000165 · Cited by 13 · see citing works
Storying away self-doubt: Can narratives dispel threats to the self?
Jennings, L. E., & McLean, K. C. (2013) · Journal of Research in Personality · doi:10.1016/j.jrp.2013.02.006 · Cited by 13 · see citing works
Self-determination and cultural considerations: An Asian perspective.
Lee, E., Lam, C. S., & Ditchman, N. (2015) · American Psychological Association eBooks · doi:10.1037/14644-012 · Cited by 12 · see citing works
Unveiling the Hidden Layers of Employees’ Job Satisfaction and Organizational Commitment: A Meta-analysis
Gupta, R., & Agrawal, R. (2023) · Business Perspectives and Research · doi:10.1177/22785337221148885 · Cited by 8 · see citing works
Effectiveness of Long-Term Psychodynamic Psychotherapy: First Meta-Analytic Evidence and Its Discussion
Rabung, S., & Leichsenring, F. (2012) · Humana Press eBooks · doi:10.1007/978-1-60761-792-1_3 · Cited by 4 · see citing works
A Meta-Analysis of the Correlates of Educational and Vocational Goals for Latina/o Students
Risco, C. M., O’Brien, K. M., Grivel, M. M., & Castro, J. E. (2017) · Journal of Career Assessment · doi:10.1177/1069072717723091 · Cited by 5 · see citing works
Effectiveness of knowledge translation of social interventions across economic boundaries: a systematic review
Fendt-Newlin, M., & Webber, M. (2015) · European Journal of Social Work · doi:10.1080/13691457.2015.1025710 · Cited by 5 · see citing works
Socio-structural influences on the work participation of refugees: an exploratory systematic mixed studies review
Mpofu, E., Stevens, C., Biggs, H. C., & Johnson, E. T. (2012) · Vulnerable Groups & Inclusion · doi:10.3402/vgi.v3i0.16066 · Cited by 3 · see citing works
Long-Term Psychodynamic Psychotherapy in Complex Mental Disorders: Update of a Meta-Analysis
Leichsenring, F., & Rabung, S. (2014) · FOCUS The Journal of Lifelong Learning in Psychiatry · doi:10.1176/appi.focus.12.3.336 · Cited by 3 · see citing works
Meta-Analysis as a Choice to Improve Research in Career and Technical Education
Gordon, H. R. D., McClain, C. R., Kim, Y.-S., & Maldonado, C. (2010) · Career and Technical Education Research · doi:10.5328/cter35.208 · Cited by 2 · see citing works
Quantitative Research Synthesis: The Use of Meta-Analysis in Career Guidance and Vocational Psychology
Gore, P. A., & Minami, T. (2008) · Springer eBooks · doi:10.1007/978-1-4020-6230-8_31 · Cited by 2 · see citing works
Self-citation
The effect of STEM practices on students' attitudes and achievements: A meta-analysis study
Kurt, A., & Akıncı, M. (2025) · International Journal of Assessment Tools in Education · doi:10.21449/ijate.1459076 · Cited by 1 · see citing works
Looking Back Across the Bridge From the Future to the Past
Carter, R. T. (2007) · The Counseling Psychologist · doi:10.1177/0011000007306852 · Cited by 1 · see citing works
Depression, Alcohol Abuse, and Alcoholism in One versus Two Parents and the Implications for Child Attachment and Self-Regulation in Infancy through Adolescence
Ridgeway, B. (2015) · International Scholarly Research Notices · doi:10.1155/2015/275649 · Cited by 1 · see citing works
Enhancing mental health practice in Sierra Leone : a social intervention development study
Fendt-Newlin, M. (2018) · White Rose eTheses Online (University of Leeds, The University of Sheffield, University of York) · Cited by 1 · see citing works
An Empirical Study of Bonett's (2009) Meta-Analytic Model
Xie, A. (2012) · University of Minnesota Digital Conservancy (University of Minnesota)
Darstellung der Wirksamkeit von kognitiv-behavioraler Therapie und Antidepressiva-Therapie bei der Behandlung der Generalisierten Angststörung
Staudacher, K. (2012) · doi:10.53846/goediss-1185
Benchmarking outcomes for psychological treatments of chronic pain
Fenton, G. (2010) · White Rose eTheses Online (University of Leeds, The University of Sheffield, University of York)
Marital Selflessness Scale (MSS): An Exploratory Factor Analysis
Castronova, M. (2014) · The Scholars Repository - LLU (Loma Linda University)
Current Status of Cultural Competence Advocacy in Clinical, School, and Counseling Psychology
Frisby, C. L., & Perez, O. R. (2018) · Springer eBooks · doi:10.1007/978-3-319-78997-2_2
A Comparative Study of Extended Meta-Ethnography and Meta-Analysis Based on the Fundamental Micro-Purposes of a Literature Review
Booker, R.-A. R. (2008) · ScholarWorks - WMU (Western Michigan University)

Meta‐Analysis: A Systematic Method for Synthesizing Counseling Research
Whiston, S. C., & Li, P. (2011) · Journal of Counseling & Development · doi:10.1002/j.1556-6678.2011.tb00089.x · Cited by 27 · see citing works
Conducting a Meta-Analysis of Counseling Outcome Research
Erford, B. T., Savin‐Murphy, J., & Butler, C. (2010) · Counseling Outcome Research and Evaluation · doi:10.1177/2150137809356682 · Cited by 54 · see citing works
Central questions about meta-analyses in psychological research: An annotated reading list
Buecker, S., Stricker, J. H., & Schneider, M. (2021) · Current Psychology · doi:10.1007/s12144-021-01957-4 · Cited by 5 · see citing works
A Practical Tutorial on Conducting Meta-Analysis in R
Re, A. C. D. (2015) · The Quantitative Methods for Psychology · doi:10.20982/tqmp.11.1.p037 · Cited by 113 · see citing works
Guidelines for reporting systematic reviews and meta-analyses
Rubio-Aparicio, M., Sánchez-Meca, J., Marín-Martínez, F., & López-López, J. A. (2018) · Anales de Psicología · doi:10.6018/analesps.34.2.320131 · Cited by 56 · see citing works

Authors: Quintana, S. M., & Minami, T.
Affiliations: University of Wisconsin–Madison (US)
University of Utah (US)
Corresponding author: Quintana, Stephen M.
ORCID: Minami, Takuya
Venue: The Counseling Psychologist
Publisher: SAGE Publications
Published: 2006-11
Volume: 34
Issue: 6
Pages: 839-877
ISSN: 0011-0000 (ISSN-L)
1552-3861
DOI: 10.1177/0011000006286991
OpenAlex: W2103089424
Cited by: 83
References: 109
Citations by year: 2025: 1 · 2023: 1 · 2022: 2 · 2021: 3 · 2020: 3 · 2019: 3 · 2018: 5 · 2017: 4 · 2016: 2 · 2015: 7 · 2014: 5 · 2013: 6 · 2012: 5
FWCI: 6.46
Type: journal article
Language: en
Open access: closed
Licence: journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
Locations: The Counseling Psychologist (publishedVersion)
Topics: Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
Social Sciences › Psychology › Social Psychology › Counseling Practices and Supervision
Keywords: meta-analysis, counseling psychology research, statistical formulas, publication guidelines

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Guidelines for Meta-Analyses of Counseling Psychology Research

Abstract

Integrity signals

Claims the paper makes

Cites

Cited by

Related

Record

About these checks