When small effects are impressive

Prentice, Deborah A.; Miller, Dale T.

doi:10.1037/0033-2909.112.1.160

Cited by 1,008Meta-analysis and systematic reviewsPsychology Research and BibliometricsPsychometric Methodologies and Testing

When small effects are impressive

Prentice, Deborah A., Miller, Dale T.·1992·Psychological Bulletin

DOI 10.1037/0033-2909.112.1.160

Open this paper in CoChat — free account

Abstract

Effect size is becoming an increasingly popular measure of the importance of an effect, both in individual studies and in meta-analyses. However, a large effect size is not the only way to demonstrate that an effect is important. This article describes 2 alternative methodological strategies, in which importance is a function of how minimal a manipulation of the independent variable or how difficult-to-influence a dependent variable will still produce an effect. These methodologies demonstrate the importance of an independent variable or psychological process, even though they often yield effects that are small in statistical terms

Integrity signals

Registry

Crossref · 10.1037/0033-2909.112.1.160

Cites

27 of 27 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Small effects are impressive when the dependent variable is not expected to be easily influenced by the independent variable.
“Prentice and Miller (1992) argue that small effects are impressive when the dependent variable is not expected to be easily influenced by the independent variable.”
From Psychosocial functioning and the cortisol awakening response: meta-analysis, P-curve analysis, and evaluation of the evidential value in existing studies (2017) · cites this paper as Prentice and Miller (1992)

Statistical power analysis for the behavioral sciences
Cohen, J. (1990) · Computers Environment and Urban Systems · doi:10.1016/0198-9715(90)90050-4 · Cited by 65,406 · see citing works
Attitudinal Effects of Mere Exposure
Zajonc, R. B. (1968) · Journal of Personality and Social Psychology · doi:10.1037/h0025848 · Cited by 6,745 · see citing works
Social categorization and intergroup behaviour
Tajfel, H., Billig, M. G., Bundy, R. P., & Flament, C. (1971) · European Journal of Social Psychology · doi:10.1002/ejsp.2420010202 · Cited by 5,562 · see citing works
Effects of Group Pressure Upon the Modification and Distortion of Judgments
Asch, S. E. (1961) · Documents of Gestalt Psychology · doi:10.1525/9780520313514-017 · Cited by 2,644 · see citing works
Things I have learned (so far).
Cohen, J. (1990) · American Psychologist · doi:10.1037/0003-066x.45.12.1304 · Cited by 1,781 · see citing works
"From Jerusalem to Jericho": A study of situational and dispositional variables in helping behavior.
Darley, J. M., & Batson, C. D. (1973) · Journal of Personality and Social Psychology · doi:10.1037/h0034449 · Cited by 1,665 · see citing works
Groups, Leadership, and Men
Besanceney, P. H., Guetzkow, H. S., & White, W. R. (1952) · The American Catholic Sociological Review · doi:10.2307/3707197 · Cited by 1,450 · see citing works
Social categorization and similarity in intergroup behaviour
Billig, M., & Tajfel, H. (1973) · European Journal of Social Psychology · doi:10.1002/ejsp.2420030103 · Cited by 1,283 · see citing works
In defense of external invalidity.
Mook, D. G. (1983) · American Psychologist · doi:10.1037/0003-066x.38.4.379 · Cited by 1,221 · see citing works
Affective Discrimination of Stimuli That Cannot Be Recognized
Kunst-Wilson, W. R., & Zajonc, R. B. (1980) · Science · doi:10.1126/science.7352271 · Cited by 1,213 · see citing works
Effect of feeling good on helping: cookies and kindness.
Isen, A., & Levin, P. F. (1972) · Journal of Personality and Social Psychology · doi:10.1037/h0032317 · Cited by 1,130 · see citing works
The Psychopathology of Everyday Life
(2013) · Pluto Press eBooks · doi:10.2307/j.ctt183p0xg.6 · Cited by 1,047 · see citing works
A variance explanation paradox: When a little is a lot.
Abelson, R. P. (1985) · Psychological Bulletin · doi:10.1037/0033-2909.97.1.129 · Cited by 1,045 · see citing works
Statistical procedures and the justification of knowledge in psychological science.
Rosnow, R. L., & Rosenthal, R. (1992) · American Psychological Association eBooks · doi:10.1037/10109-027 · Cited by 653 · see citing works
Physicians' use of probabilistic information in a real clinical setting.
Christensen-Szalanski, J. J. J., & Bushyhead, J. B. (1981) · Journal of Experimental Psychology Human Perception & Performance · doi:10.1037//0096-1523.7.4.928 · Cited by 443 · see citing works
Sexism and beautyism in personnel consultant decision making.
Cash, T. F., Gillen, B., & Burns, D. S. (1977) · Journal of Applied Psychology · doi:10.1037/0021-9010.62.3.301 · Cited by 367 · see citing works
Beautiful but dangerous: Effects of offender attractiveness and nature of the crime on juridic judgment.
Sigall, H., & Ostrove, N. (1975) · Journal of Personality and Social Psychology · doi:10.1037/h0076472 · Cited by 364 · see citing works
The effect of physical appearance on the judgment of guilt, interpersonal attraction, and severity of recommended punishment in a simulated jury task
Efran, M. G. (1974) · Journal of Research in Personality · doi:10.1016/0092-6566(74)90044-0 · Cited by 333 · see citing works
Review of developments in meta-analytic method.
Bangert‐Drowns, R. L. (1986) · Psychological Bulletin · doi:10.1037/0033-2909.99.3.388 · Cited by 327 · see citing works
A Contextualist Theory of Knowledge: Its Implications for Innovation and Reform in Psychological Research
McGuire, W. J. (1983) · Advances in experimental social psychology · doi:10.1016/s0065-2601(08)60393-7 · Cited by 252 · see citing works
A Note on Percent Variance Explained as A Measure of the Importance of Effects
Rosenthal, R., & Rubin, D. B. (1979) · Journal of Applied Social Psychology · doi:10.1111/j.1559-1816.1979.tb02713.x · Cited by 238 · see citing works
Feeling more than we can know: Exposure effects without learning.
Wilson, W. R. (1979) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.37.6.811 · Cited by 228 · see citing works
Social categorization and discriminatory behavior: Extinguishing the minimal intergroup discrimination effect.
Locksley, A., Ortiz, V., & Hepburn, C. (1980) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.39.5.773 · Cited by 197 · see citing works
Exposure and affect: A field experiment
Zajonc, R. B., & Rajecki, D. W. (1969) · Psychonomic Science · doi:10.3758/bf03329178 · Cited by 81 · see citing works
Dead on arrival
Sudnow, D. (1967) · Society · doi:10.1007/bf03180093 · Cited by 74 · see citing works
Meaningful measures of effect.
Yeaton, W. H., & SECHREST, L. B. (1981) · Journal of Consulting and Clinical Psychology · doi:10.1037//0022-006x.49.5.766 · Cited by 38 · see citing works
Statistical considerations in analyzing the results of a collection of experiments
Mayo, R. J. (1978) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00075683 · Cited by 11 · see citing works

Flagged first, then by citations

Sorted by citations

Experimental evidence of massive-scale emotional contagion through social networks
Kramer, A. D. I., Guillory, J. E., & Hancock, J. T. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1320040111 · Cited by 3,687 · see citing works
Evaluating Effect Size in Psychological Research: Sense and Nonsense
Funder, D. C., & Ozer, D. J. (2019) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245919847202 · Cited by 3,295 · see citing works
Toward an integrative social identity model of collective action: A quantitative research synthesis of three socio-psychological perspectives.
van Zomeren, M., Postmes, T., & Spears, R. (2008) · Psychological Bulletin · doi:10.1037/0033-2909.134.4.504 · Cited by 2,714 · see citing works
Sensitivity and attachment: A meta-analysis on parental antecedents of infant attachment
De Wolff, M. S., & van Ijzendoorn, M. H. (1997) · Child Development · doi:10.1111/j.1467-8624.1997.tb04218.x · Cited by 2,526 · see citing works
The Essential Guide to Effect Sizes: Statistical Power, Meta-Analysis, and the Interpretation of Research Results
Ellis, P. D. (2010) · doi:10.1017/cbo9780511761676 · Cited by 2,461 · see citing works
Facebook Use Predicts Declines in Subjective Well-Being in Young Adults
Kross, E., Verduyn, P., Demiralp, E., Park, J., Lee, D. S., Lin, N., Shablack, H., Jonides, J., & Ybarra, O. (2013) · PLoS ONE · doi:10.1371/journal.pone.0069841 · Cited by 1,596 · see citing works
Experimental disclosure and its moderators: A meta-analysis.
Frattaroli, J. (2006) · Psychological Bulletin · doi:10.1037/0033-2909.132.6.823 · Cited by 1,502 · see citing works
On the use of college students in social science research: Insights from a second-order meta-analysis
Peterson, R. A. (2001) · Journal of Consumer Research · doi:10.1086/323732 · Cited by 1,258 · see citing works
Religiousness and depression: Evidence for a main effect and the moderating influence of stressful life events.
Smith, T. B., McCullough, M. E., & Poll, J. (2003) · Psychological Bulletin · doi:10.1037/0033-2909.129.4.614 · Cited by 1,150 · see citing works
Predicting voice behavior in work groups.
LePine, J. A., & Van Dyne, L. (1998) · Journal of Applied Psychology · doi:10.1037/0021-9010.83.6.853 · Cited by 1,192 · see citing works
Mini Meta‐Analysis of Your Own Studies: Some Arguments on Why and a Primer on How
Goh, J. X., Hall, J. A., & Rosenthal, R. (2016) · Social and Personality Psychology Compass · doi:10.1111/spc3.12267 · Cited by 994 · see citing works
A little thanks goes a long way: explaining why gratitude expressions motivate prosocial behavior
Grant, A. M., & Gino, F. (2010) · Journal of Personality and Social Psychology · doi:10.1037/a0017935 · Cited by 870 · see citing works
The relationship between professional burnout and quality and safety in healthcare: a meta-analysis
Salyers, M. P., Bonfils, K. A., Luther, L., Firmin, R. L., White, D. A., Adams, E. L., & Rollins, A. L. (2017) · Journal of General Internal Medicine · doi:10.1007/s11606-016-3886-9 · Cited by 843 · see citing works
Becoming happier takes both a will and a proper way: An experimental longitudinal intervention to boost well-being.
Lyubomirsky, S., Dickerhoof, R., Boehm, J. K., & Sheldon, K. M. (2011) · Emotion · doi:10.1037/a0022575 · Cited by 720 · see citing works
Do inclusive leaders help to reduce turnover in diverse groups? The moderating role of leader–member exchange in the diversity to turnover relationship.
Nishii, L. H., & Mayer, D. M. (2009) · Journal of Applied Psychology · doi:10.1037/a0017190 · Cited by 689 · see citing works
Sample Size Planning for Statistical Power and Accuracy in Parameter Estimation
Maxwell, S. E., Kelley, K., & Rausch, J. R. (2008) · Annual Review of Psychology · doi:10.1146/annurev.psych.59.103006.093735 · Cited by 542 · see citing works
Personality-relationship transaction in young adulthood.
Neyer, F. J., & Asendorpf, J. B. (2001) · Journal of Personality and Social Psychology · doi:10.1037//0022-3514.81.6.1190 · Cited by 518 · see citing works
The N-Pact Factor: Evaluating the Quality of Empirical Journals with Respect to Sample Size and Statistical Power
Fraley, R. C., & Vazire, S. (2014) · PLoS ONE · doi:10.1371/journal.pone.0109019 · Cited by 409 · see citing works
A meta-analytic review of the Penn Resiliency Program’s effect on depressive symptoms.
Brunwasser, S. M., Gillham, J. E., & Kim, E. S. (2009) · Journal of Consulting and Clinical Psychology · doi:10.1037/a0017671 · Cited by 428 · see citing works
The impact of non-response bias due to sampling in public health studies: A comparison of voluntary versus mandatory recruitment in a Dutch national survey on adolescent health
Cheung, K. L., ten Klooster, P. M., Smit, C., de Vries, H., & Pieterse, M. E. (2017) · BMC Public Health · doi:10.1186/s12889-017-4189-8 · Cited by 421 · see citing works
Making the break count: an episodic examination of recovery activities, emotional experiences, and positive affective displays
Trougakos, J. P., Beal, D. J., Green, S. G., & Weiss, H. M. (2008) · Academy of Management Journal · doi:10.5465/amj.2008.30764063 · Cited by 400 · see citing works
Designing experiments that generalize
Highhouse, S. (2009) · Organizational Research Methods · doi:10.1177/1094428107300396 · Cited by 353 · see citing works
"Is there nothing more practical than a good theory?": Why innovations and advances in health behavior change will arise if interventions are used to test and refine theory
Rothman, A. J. (2004) · International Journal of Behavioral Nutrition and Physical Activity · doi:10.1186/1479-5868-1-11 · Cited by 322 · see citing works
Beyond small, medium, or large: points of consideration when interpreting effect sizes
Bakker, A., Cai, J., English, L., Kaiser, G., Mesa, V., & Van Dooren, W. (2019) · Educational Studies in Mathematics · doi:10.1007/s10649-019-09908-4 · Cited by 302 · see citing works
The gateway belief model: a large-scale replication
van der Linden, S., Leiserowitz, A., & Maibach, E. (2019) · Journal of Environmental Psychology · doi:10.1016/j.jenvp.2019.01.009 · Cited by 262 · see citing works
Quantitative Research Methods, Study Quality, and Outcomes: The Case of Interaction Research
Plonsky, L., & Gass, S. (2011) · Language Learning · doi:10.1111/j.1467-9922.2011.00640.x · Cited by 267 · see citing works
Psychometric Properties of the Screen for Child Anxiety Related Emotional Disorders (SCARED) in the General Adolescent Population
HALE, W. W., RAAIJMAKERS, Q., MURIS, P., & MEEUS, W. (2005) · Journal of the American Academy of Child & Adolescent Psychiatry · doi:10.1097/00004583-200503000-00013 · Cited by 253 · see citing works
It’s Not All About Me: Motivating Hand Hygiene Among Health Care Professionals by Focusing on Patients
Grant, A. M., & Hofmann, D. A. (2011) · Psychological Science · doi:10.1177/0956797611419172 · Cited by 239 · see citing works
Meta-analysis in Second Language Research: Choices and Challenges
Oswald, F. L., & Plonsky, L. (2010) · Annual Review of Applied Linguistics · doi:10.1017/s0267190510000115 · Cited by 237 · see citing works
Statistical considerations in the design and analysis of community intervention trials
Donner, A., & Klar, N. (1996) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(95)00511-0 · Cited by 230 · see citing works
Parenting stress mediates between maternal maltreatment history and maternal sensitivity in a community sample
Pereira, J., Vickers, K., Atkinson, L., Gonzalez, A., Wekerle, C., & Levitan, R. (2012) · Child Abuse & Neglect · doi:10.1016/j.chiabu.2012.01.006 · Cited by 222 · see citing works
Digitalizing procurement: The impact of data analytics on supply chain performance
Hallikas, J., Immonen, M., & Brax, S. (2021) · Supply Chain Management: An International Journal · doi:10.1108/scm-05-2020-0201 · Cited by 218 · see citing works
Metaphors for the War (or Race) against Climate Change
Flusberg, S. J., Matlock, T., & Thibodeau, P. H. (2017) · Environmental Communication · doi:10.1080/17524032.2017.1289111 · Cited by 203 · see citing works
On the Overuse and Misuse of Mediation Analysis: It May Be a Matter of Timing
Tate, C. U. (2015) · Basic and Applied Social Psychology · doi:10.1080/01973533.2015.1062380 · Cited by 153 · see citing works
Distal factors in risk perception
Sjöberg, L. (2003) · Journal of Risk Research · doi:10.1080/1366987032000088847 · Cited by 144 · see citing works
From ideation to action: Differentiating between those who think about suicide and those who attempt suicide in a national study of young adults
Wetherall, K., Cleare, S., Eschle, S., Ferguson, E., O'Connor, D. B., O'Carroll, R. E., & O'Connor, R. C. (2018) · Journal of Affective Disorders · doi:10.1016/j.jad.2018.07.074 · Cited by 131 · see citing works
When bad things happen to good feedback: Exacerbating the need for self‐justification with self‐affirmations
Blanton, H., Cooper, J., Slkurnik, I., & Aronson, J. (1997) · Personality and Social Psychology Bulletin · doi:10.1177/0146167297237002 · Cited by 127 · see citing works
Sleep, impulse control, and sensation-seeking predict delinquent behavior in adolescents, emerging adults, and adults
Peach, H. D., & Gaultney, J. F. (2013) · Journal of Adolescent Health · doi:10.1016/j.jadohealth.2013.03.012 · Cited by 117 · see citing works
The association of neighborhood poverty with personality change in childhood
Hart, D., Atkins, R., & Matsuba, M. K. (2008) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.94.6.1048 · Cited by 110 · see citing works
Acknowledging uncertainty impacts public acceptance of climate scientists’ predictions
Howe, L. C., MacInnis, B., Krosnick, J. A., Markowitz, E. M., & Socolow, R. (2019) · Nature Climate Change · doi:10.1038/s41558-019-0587-5 · Cited by 100 · see citing works
Psychosocial functioning and the cortisol awakening response: meta-analysis, P-curve analysis, and evaluation of the evidential value in existing studies
Boggero, I. A., Hostinar, C. E., Haak, E. A., Murphy, M. L. M., & Segerstrom, S. C. (2017) · Biological Psychology · doi:10.1016/j.biopsycho.2017.08.058 · Cited by 97 · see citing works
“Prentice and Miller (1992) argue that small effects are impressive when the dependent variable is not expected to be easily influenced by the independent variable.”
Background
It Could Have Been True: How Counterfactual Thoughts Reduce Condemnation of Falsehoods and Increase Political Polarization
Effron, D. A. (2018) · Personality and Social Psychology Bulletin · doi:10.1177/0146167217746152 · Cited by 79 · see citing works
Intelligence and interpersonal sensitivity: A meta-analysis
Murphy, N. A., & Hall, J. A. (2011) · Intelligence · doi:10.1016/j.intell.2010.10.001 · Cited by 73 · see citing works
Childhood Physical and Emotional Abuse by a Parent: Transference Effects in Adult Interpersonal Relations
Berenson, K. R., & Andersen, S. M. (2006) · Personality and Social Psychology Bulletin · doi:10.1177/0146167206291671 · Cited by 72 · see citing works
Negative body talk as an outcome of friends’ fitness posts on social networking sites: body surveillance and social comparison as potential moderators
Arroyo, A., & Brunner, S. R. (2016) · Journal of Applied Communication Research · doi:10.1080/00909882.2016.1192293 · Cited by 67 · see citing works
The Paradox of Viral Outrage
Sawaoka, T., & Monin, B. (2018) · Psychological Science · doi:10.1177/0956797618780658 · Cited by 54 · see citing works
Blurred Lines: How the Collectivism Norm Operates Through Perceived Group Diversity to Boost or Harm Group Performance in Himalayan Mountain Climbing
Chatman, J. A., Greer, L. L., Sherman, E., & Doerr, B. (2019) · Organization Science · doi:10.1287/orsc.2018.1268 · Cited by 32 · see citing works
On the correspondence between dream content and target material under laboratory conditions: A meta-analysis of dream-ESP studies, 1966-2016
Storm, L., Sherwood, S. J., Roe, C. A., Tressoldi, P. E., Rock, A. J., & Di Risio, L. (2017) · International Journal of Dream Research · doi:10.11588/ijodr.2017.2.34888 · Cited by 29 · see citing works
“The correlation coefficient r is one of the most commonly used effect size measures (Prentice & Miller, 1992).”
Method
Internalized weight stigma in women with class III obesity: A randomized controlled trial of a virtual lifestyle modification intervention followed by a mindful self-compassion intervention
Braun, T. D., Olson, K., Panza, E., Lillis, J., Schumacher, L., Abrantes, A. M., Kunicki, Z., & Unick, J. L. (2022) · Obesity Science & Practice · doi:10.1002/osp4.616 · Cited by 18 · see citing works
Testing a Self-Compassion Intervention Among Job Seekers: Self-Compassion Beneficially Impacts Affect Through Reduced Self-Criticism
Kreemers, L. M., van Hooft, E. A. J., van Vianen, A. E. M., & Sisouw de Zilwa, S. C. M. (2020) · Frontiers in Psychology · doi:10.3389/fpsyg.2020.01371 · Cited by 14 · see citing works
“Given that our intervention was small and performed in a field setting, which carries more noise than a laboratory setting, having a modest effect is understandable and even promising (Prentice and Miller, 1992).”
Background

Susquehanna Chorale Spring Concert "Roots and Wings"
Hawkins, D. (2017)
Personality correlates and utilitarian judgments in the everyday context: Psychopathic traits and differential effects of empathy, social dominance orientation, and dehumanization beliefs
Takamatsu, R. (2019) · Personality and Individual Differences · doi:10.1016/j.paid.2019.03.029 · Cited by 22 · see citing works
Amount of altruistic punishment accounts for subsequent emotional gratification in participants with primary psychopathy
Masui, K., Iriguchi, S., Nomura, M., & Ura, M. (2011) · Personality and Individual Differences · doi:10.1016/j.paid.2011.07.006 · Cited by 14 · see citing works
The Information Used to Judge Supportiveness Depends on Whether the Judgment Reflects the Personality of Perceivers, the Objective Characteristics of Targets, or their Unique Relationships
Lakey, B., Lutz, C. J., & Scoboria, A. (2004) · Journal of Social and Clinical Psychology · doi:10.1521/jscp.23.6.817.54806 · Cited by 14 · see citing works
The Mental Representations Of High And Low Entitativity Groups
Johnson, A. L., & Queller, S. (2003) · Social Cognition · doi:10.1521/soco.21.2.101.21316 · Cited by 13 · see citing works

Authors: Prentice, D. A., & Miller, D. T.
Affiliations: Princeton University (US)
ORCID: Prentice, Deborah A.
Miller, Dale T.
Venue: Psychological Bulletin
Publisher: American Psychological Association (APA)
Published: 1992-07
Volume: 112
Issue: 1
Pages: 160-164
ISSN: 0033-2909 (ISSN-L)
1939-1455
DOI: 10.1037/0033-2909.112.1.160
OpenAlex: W2011756420
Cited by: 1,008
References: 27
Citations by year: 2026: 32 · 2025: 34 · 2024: 21 · 2023: 33 · 2022: 38 · 2021: 37 · 2020: 56 · 2019: 48 · 2018: 50 · 2017: 45 · 2016: 45 · 2015: 45 · 2014: 41 · 2013: 40 · 2012: 49
FWCI: 12.85
Type: journal article
Language: en
Open access: closed
Locations: Psychological Bulletin (publishedVersion)
Topics: Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
Keywords: effect size, meta-analysis, cognitive processes

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

When small effects are impressive

Abstract

Integrity signals

Claims the paper makes

Claims other papers make about this one

Cites

Cited by

Related

Record

About these checks