The DOI matches its Crossref record; 47 of the 47 references it cites were found in a registry or an index.
Consequences of prejudice against the null hypothesis.
Abstract
The consequences of prejudice against accepting the null hypothesis were examined through (a) a mathematical model intended to stimulate the research-publication process and (b) case studies of apparent erroneous rejec-tions of the null hypothesis in published psychological research. The input parameters for the model characterize investigators ' probabilities of selecting a problem for which the null hypothesis is true, of reporting, following up on, or abandoning research when data do or do not reject the null hypothesis, and they characterize editors ' probabilities of publishing manuscripts concluding in favor of or against the null hypothesis. With estimates of the input parameters based on a questionnaire survey of a sample of social psychologists, the model output indicates a dysfunctional research-publication system. Particularly, the model indicates that there may be relatively few publications on problems for which the null hypothesis is (at least to a reasonable approximation) true, and of these, a high proportion will erroneously reject the null hypothesis. The case studies provide additional support for this conclusion. Accordingly, it is
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The bias against negative results appears to be a consequence of both editors and reviewers finding them less compelling for publication and from authors themselves not trying to publish them 8-12”
From Initial evidence of research quality of registered reports compared with the standard publishing model (2021) · cites this paper as [11]
In the citing paper
“Positive results -- finding that a treatment is effective or that there is a relationship between two variables -- are more likely to be published than negative results 6-12 .”
From Initial evidence of research quality of registered reports compared with the standard publishing model (2021) · cites this paper as [11]
In the citing paper
“Theoretical analyses point to a system of incentives that prioritizes innovation over verification, leading to infrequent efforts to replicate findings and to behaviors that could reduce the replicability of published findings such as selective reporting, presenting exploratory findings as confirmatory tests, and failures of documentation, transparency, and sharing”
From Investigating the replicability of preclinical cancer biology (2021) · cites this paper as Greenwald, 1975
In the citing paper
“Research that produces novel results, statistically significant results (that is, typically p < 0.05) and seemingly 'clean' results is more likely to be published 2,3 .”
From Power failure: why small sample size undermines the reliability of neuroscience (2013) · cites this paper as [3]
In the citing paper
“An important qualification about this result is the possibility that the original studies have inflated effect sizes due to publication, selection, reporting, or other biases ( 9, 1223 ).”
From Estimating the reproducibility of psychological science (2015) · cites this paper as [16]
In the citing paper
“Potentially problematic practices include selective reporting, selective analysis, and insufficient specification of the conditions necessary or sufficient to obtain the results ( 1223 ).”
From Estimating the reproducibility of psychological science (2015) · cites this paper as [16]
In the citing paper
“There is good evidence that low power of research designs combined with publication bias favoring positive results over negative results produces a literature with upwardly biased effect sizes ( 14, 16, 30, 31 ).”
From Estimating the reproducibility of psychological science (2015) · cites this paper as [16]
In the citing paper
“Concerns about reproducibility are widespread across disciplines ( 921 ).”
From Estimating the reproducibility of psychological science (2015) · cites this paper as [16]
Citations
Cites
Nonparametric Statistics for the Behavioral Sciences
Guitton, H., & Siegel, S. (1958) · Revue économique · doi:10.2307/3498751 · Cited by 35,279
The Influence of Source Credibility on Communication Effectiveness
Hovland, C. I., & Weiss, W. (1951) · Public Opinion Quarterly · doi:10.1086/266350 · Cited by 3,543
Boneau, C. A., & Hays, W. L. (1966) · The American Journal of Psychology · doi:10.2307/1420739 · Cited by 1,887
The statistical power of abnormal-social psychological research: A review.
Cohen, J. (1962) · The Journal of Abnormal and Social Psychology · doi:10.1037/h0045186 · Cited by 1,824
Bayesian statistical inference for psychological research.
Edwards, W., Lindman, H., & Savage, L. J. (1963) · Psychological Review · doi:10.1037/h0044139 · Cited by 1,734
Experimental and quasi-experimental designs for research on teaching
Campbell, D. T. (1963) · Medical Entomology and Zoology · Cited by 1,541
Statistical significance in psychological research.
Lykken, D. T. (1968) · Psychological bulletin · doi:10.1037/h0026141 · Cited by 1,363
Artifact in behavioral research
Rosenthal, R., & Rosnow, R. L. (1969) · Medical Entomology and Zoology · Cited by 1,186
Sterling, T. D. (1959) · Journal of the American Statistical Association · doi:10.1080/01621459.1959.10501497 · Cited by 964
The test of significance in psychological research.
Bakan, D. (1966) · Psychological Bulletin · doi:10.1037/h0020412 · Cited by 957
Cited by
Sorted by citations
Items 201–300 of 1,001
Lessons From Pinocchio: Cues to Deception May Be Highly Exaggerated
Luke, T. J. (2019) · Perspectives on Psychological Science · doi:10.1177/1745691619838258 · Cited by 174
Public Administration Research and Practice: A Methodological Manifesto
Gill, J., & Meier, K. (2000) · Journal of Public Administration Research and Theory · doi:10.1093/oxfordjournals.jpart.a024262 · Cited by 152
“This is called the "illusion of theory confirmation" (Greenwald 1975, Lindsay 1995) because the NHST is presented as evidence of the exclusivity of explanation of this single research hypothesis.”
BackgroundDalton, D. R., Aguinis, H., Dalton, C. M., Bosco, F. A., & Pierce, C. A. (2012) · Personnel Psychology · doi:10.1111/j.1744-6570.2012.01243.x · Cited by 157
“Similarly, in a widely cited meta-analysis of antecedents and consequents of organizational commitment (OC), Mathieu and Zajac (1990) noted that “an article-by-article search of the Journal of Applied Psychology, the Academy of Management Journal, Administrative Science Quarterly, Human Relations, Organizational Behav…”
BackgroundFate of biomedical research protocols and publication bias in France: retrospective cohort study
Decullier, É., Lhéritier, V., & Chapuis, F. R. (2005) · BMJ · doi:10.1136/bmj.38488.385995.8f · Cited by 153
Three Threats to Validity of Choice‐based and Matched‐Sample Studies in Accounting Research*
Cram, D. P., Karan, V., & Stuart, I. (2009) · Contemporary Accounting Research · doi:10.1506/car.26.2.7 · Cited by 153
Uneven progress: Social psychology and the study of attitudes.
Eagly, A. H. (1992) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.63.5.693 · Cited by 157
Effectiveness of Multiple Request Strategies: A Synthesis of Research Results
Fern, E. F., Monroe, K. B., & Avila, R. A. (1986) · Journal of Marketing Research · doi:10.1177/002224378602300206 · Cited by 151
“It should be noted that published reports tend to be biased toward reporting significant findings and larger mean effect sizes (Cooper 1984; Glass, McGaw, and Smith 1981; Greenwald 1975; Rosenthal 1984).”
ResultUnger, R. K. (1983) · Psychology of Women Quarterly · doi:10.1111/j.1471-6402.1983.tb00614.x · Cited by 151
“Studies on sex differences probably also suffer from prejudice against the null hypothesis (Greenwald, 1975).”
BackgroundMild head injury in children and adolescents: A review of studies (1970–1995).
Satz, P., Zaucha, K., McCleary, C., Light, R., Asarnow, R., & Becker, D. (1997) · Psychological Bulletin · doi:10.1037/0033-2909.122.2.107 · Cited by 245
On the probability of making Type I errors.
Pollard, P., & Richardson, J. T. E. (1987) · Psychological Bulletin · doi:10.1037/0033-2909.102.1.159 · Cited by 155
What's in a norm? Sources and processes of norm change.
Paluck, E. L. (2009) · Journal of Personality and Social Psychology · doi:10.1037/a0014688 · Cited by 147
“If forces idiosyncratic to the reconciliation program were responsible for the 10 null and negative results, why was the same pattern of no change in beliefs (but change in perceived norms) also observed for health listeners? As Greenwald (1975) asserted, in some cases null results can be informative.”
ResultAttitude Toward Self, Social Factors, and Achievement in Mathematics: A Meta-Analytic Review
Ma, X., & Kishor, N. (1997) · Educational Psychology Review · doi:10.1023/a:1024785812050 · Cited by 147
Researchers’ Intuitions About Power in Psychological Research
Bakker, M., Hartgerink, C. H. J., Wicherts, J. M., & van der Maas, H. L. J. (2016) · Psychological Science · doi:10.1177/0956797616647519 · Cited by 158
“In addition, when Greenwald (1975) asked social psychologists what the acceptable Type II error rate was, the average response was around .27, which means that an acceptable level of power would be .73, which again is markedly higher than the overall power estimates for published studies, as…”
Finding the missing science: The fate of studies submitted for review by a human subjects committee.
Cooper, H., DeNeve, K., & Charlton, K. (1997) · Psychological Methods · doi:10.1037/1082-989x.2.4.447 · Cited by 144
“Greenwald (1975) demonstrated that prejudice against the null hypothesis is not restricted to journal reviewers and editors only.”
BackgroundDickersin, K., & Chalmers, I. (2011) · Journal of the Royal Society of Medicine · doi:10.1258/jrsm.2011.11k042 · Cited by 149
PUBLICATION BIAS: A CASE STUDY OF FOUR TEST VENDORS
McDaniel, M. A., Rothstein, H. R., & Whetzel, D. L. (2006) · Personnel Psychology · doi:10.1111/j.1744-6570.2006.00059.x · Cited by 147
“There is also some evidence that editorial policy, at least in some journals, favors the publication of significant results (Dickersin, 2005; Greenwald, 1975).”
ResultEmpathic concern and helping behavior: Egoism or altruism?
Schroeder, D. A., Dovidio, J. F., Sibicky, M. E., Matthews, L. L., & Allen, J. L. (1988) · Journal of Experimental Social Psychology · doi:10.1016/0022-1031(88)90024-8 · Cited by 142
The impact of attitudes on memory: An affair to remember.
Eagly, A. H., Chen, S., Chaiken, S., & Shaw-Barnes, K. (1999) · Psychological Bulletin · doi:10.1037/0033-2909.125.1.64 · Cited by 151
“When viewing the results of this meta-analysis solely in terms of the magnitude of the attitude congeniality effect it obtained, psychologists might make the mistake of concluding, in line with informal characterizations of this literature (e.g., Fishbein & Ajzen, 1975; Greenwald, 1975), that attitudes exert little, i…”
BackgroundMethodResultInfluentialSignificance tests as sorcery: Science is empirical—significance tests are not
Lambdin, C. (2012) · Theory & Psychology · doi:10.1177/0959354311429854 · Cited by 143
“…Thomason, Cumming, Finch, & Leeman, 2004; Fisher, 1955, 1956; Gigerenzer, 1987, 1993, 2004; Gigerenzer et al., 1989; Gill, 1999; Granaas, 2002; Greenwald, 1975; Hubbard & Armstrong, 2006; Hubbard & Lindsay, 2008; Hubbard & Ryan, 2000; Jones & Tukey, 2000; Kirk, 1996, 2003; Lindsay, 1995;…”
BackgroundBösch, H., Steinkamp, F., & Boller, E. (2006) · Psychological Bulletin · doi:10.1037/0033-2909.132.4.497 · Cited by 141
Bamber, L. S., Christensen, T. E., & Gaver, K. M. (2000) · Accounting Organizations and Society · doi:10.1016/s0361-3682(99)00027-6 · Cited by 129
The Meta-Analysis of Clinical Judgment Project
Spengler, P. M., White, M. J., Ægisdóttir, S., Maugherman, A. S., Anderson, L. A., Cook, R. S., Nichols, C. N., Lampropoulos, G. K., Walker, B. S., Cohen, G. R., et al. (2007) · The Counseling Psychologist · doi:10.1177/0011000006295149 · Cited by 148
On the Tyranny of Hypothesis Testing in the Social Sciences
Loftus, G. R. (1991) · Contemporary Psychology · doi:10.1037/029395 · Cited by 136
“There is a profound asymmetry between conclusions issuing from rejecting the null hypothesis on the one hand, and failing to reject it on the other (see Greenwald, 1975 for a lengthy discussion of this bias).”
Podsakoff, P. M. (1995) · Journal of Management · doi:10.1016/0149-2063(95)90015-2 · Cited by 135
A Critical Appraisal of Demand Artifacts in Consumer Research
Shimp, T. A., Hyatt, E. M., & Snyder, D. J. (1991) · Journal of Consumer Research · doi:10.1086/209259 · Cited by 131
That Still, Small Voice: Commitment to Cooperate as an Internalized Versus a Social Norm
Kerr, N. L., Garst, J., Lewandowski, D. A., & Harris, S. E. (1997) · Personality and Social Psychology Bulletin · doi:10.1177/01461672972312007 · Cited by 134
Threats to the validity of research syntheses
Matt, G. E., & Cook, T. D. (1994) · Cited by 133
(2020) · Cambridge University Press eBooks · doi:10.1017/9781108762519 · Cited by 154
Shadish, W. R., Zelinsky, N. A. M., Vevea, J. L., & Kratochwill, T. R. (2016) · Journal of Applied Behavior Analysis · doi:10.1002/jaba.308 · Cited by 130
Meta-analysis of ERP investigations of pain empathy underlines methodological issues in ERP research
Coll, M. (2018) · Social Cognitive and Affective Neuroscience · doi:10.1093/scan/nsy072 · Cited by 142
“Putting aside the fact that negative results can sometimes be informative if statistical power is high enough (Greenwald, 1975), the main consequence of this practice is that any attempt to meta-analytically summarize the results of such studies will be difficult, inevitably biased and of…”
BackgroundViolent pornography, antiwoman thoughts, and antiwoman acts: In search of reliable effects
Fisher, W. A., & Grenier, G. (1994) · The Journal of Sex Research · doi:10.1080/00224499409551727 · Cited by 138
A Model of Sex‐Role Transcendence
Rebecca, M., Hefner, R., & Oleshansky, B. (1976) · Journal of Social Issues · doi:10.1111/j.1540-4560.1976.tb02607.x · Cited by 131
Schwab, A., Abrahamson, E., Starbuck, W. H., & Fidler, F. M. (2010) · Organization Science · doi:10.1287/orsc.1100.0557 · Cited by 129
“However, probably because he integrated NHSTs into his very popular textbook, Fisher was able to persuade many to adopt NHSTs. Complaints about NHSTs have multiplied over time (Cohen, 1994; Greenwald, 1975; Schmidt and Hunter, 1997; Schwab and Starbuck, 2009; Seth et al., 2009; Thompson, 1999b).”
Promoting Open Science to Increase the Trustworthiness of Evidence in Special Education
Adelson, J. L., Barton, E., Bradshaw, C., Nosek, B. A., Bryant, D., Cook, B. G., Coyne, M., DeBettencourt, L., DeHaven, A. C., Dymond, S., et al. (2018) · Exceptional Children · doi:10.1177/0014402918793138 · Cited by 136
“…of effective practices (Cook & Therrien, 2017; Therrien & Cook, 2018), researchers often fail to submit them for publication (i.e., publication bias; Franco, Malhotra, & Simonovits, 2014; Greenwald, 1975), likely due to perceptions that such research is not valued and is unlikely to be published.”
BackgroundKrull, D. S., Loy, M. H.-M., Lin, J. S., Wang, C.-F., Chen, S., & Zhao, X. (1999) · Personality and Social Psychology Bulletin · doi:10.1177/0146167299258003 · Cited by 128
“As Greenwald (1975) suggests, confidence in a null effect also is strengthened when differences are obtained where they should be found.”
BackgroundMethodInfluentialThe Value of Preregistration for Psychological Science: A Conceptual AnalysisPreprint
Lakens, D. (2019) · doi:10.31234/osf.io/jbh4w · Cited by 130
Höge, T. (2008) · Stress and Health · doi:10.1002/smi.1226 · Cited by 138
Advocacy and Objectivity in Science
Armstrong, J. S. (1979) · Management Science · doi:10.1287/mnsc.25.5.423 · Cited by 129
“Although advocacy improves efficiency by directing the scientist's efforts towards the testing of a single hypothesis, it also contributes to a loss of objectivity for the individual scientist [13].”
Agreeableness and Its Consequences: A Quantitative Review of Meta-Analytic Findings
Wilmot, M. P., & Ones, D. S. (2022) · Personality and Social Psychology Review · doi:10.1177/10888683211073007 · Cited by 130
Improving what is published: A model in search of an editor.
Kupfersmid, J. H. (1988) · American Psychologist · doi:10.1037/0003-066x.43.8.635 · Cited by 126
How Trustworthy Is the Scientific Literature in Industrial and Organizational Psychology?
Kepes, S., & McDaniel, M. A. (2013) · Industrial and Organizational Psychology · doi:10.1111/iops.12045 · Cited by 136
“For decades, we have known that studies with statistically significant results are more likely to get published in our journals than studies with nonsignificant results (Greenwald, 1975; Orlitzky, 2012; Porter, 1992; Rosenthal, 1979; Sterling, 1959).”
BackgroundResultAutomatic Stereotyping: Category, Trait, and Behavioral Activations
Kawakami, K., Young, H., & Dovidio, J. F. (2002) · Personality and Social Psychology Bulletin · doi:10.1177/0146167202281001 · Cited by 125
“As Greenwald (1975) notes, however, there may be a disciplinary “prejudice against the null hypothesis” (p. 1).”
BackgroundWeisz, J. R., & Zigler, E. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.4.831 · Cited by 125
Brennan, E. M., Bradley, J. R., Allen, M. D., & Perry, D. F. (2008) · Early Education and Development · doi:10.1080/10409280801975834 · Cited by 129
“Additionally, we hoped to temper the bias that can result from research syntheses focused solely on peer-reviewed articles, given the tendency not to publish null findings (Greenwald, 1975; Sadish, 1992).”
BackgroundWhen is statistical significance not significant?
Filho, D. B. F., Paranhos, R., da Rocha, E. C., Batista, M., Silva, J. A. D., Santos, M. L., & Marino, J. G. (2013) · Brazilian Political Science Review · doi:10.1590/s1981-38212013000100002 · Cited by 125
Open exchange and epistemic progress.
Mahoney, M. J. (1985) · American Psychologist · doi:10.1037/0003-066x.40.1.29 · Cited by 121
Harm and Offence in Media Content: A Review of the Evidence
Hargrave, A. M., & Livingstone, S. (2006) · London School of Economics and Political Science Research Online (London School of Economics and Political Science) · Cited by 117
A meta‐analysis of the clinical manifestations of older‐onset systemic lupus erythematosus
Ward, M. M., & Polisson, R. P. (1989) · Arthritis & Rheumatism · doi:10.1002/anr.1780321007 · Cited by 118
Cashen, L. H., & Geiger, S. W. (2004) · Organizational Research Methods · doi:10.1177/1094428104263676 · Cited by 126
“Instead, the focus should be directed at determining whether the effect size is negligible or trivial (Cohen, 1977, 1990; Greenwald, 1993).”
BackgroundInfluentialThompson, B. (1999) · Theory & Psychology · doi:10.1177/095935439992006 · Cited by 138
“Greenwald (1975) cited relevant actual examples.”
Avoiding Bias in Publication Bias Research: The Value of “Null” Findings
Kepes, S., Banks, G. C., & Oh, I. (2012) · Journal of Business and Psychology · doi:10.1007/s10869-012-9279-0 · Cited by 132
Kelley, K., & Rausch, J. R. (2006) · Psychological Methods · doi:10.1037/1082-989x.11.4.363 · Cited by 116
“The accuracy of parameter estimates is also important in another context when one wishes to show support for the null hypothesis (e.g., Greenwald, 1975) or in the context of equivalence testing (e.g., Steiger, 2004; Tryon, 2001).”
BackgroundMethodological notes on the accuracy-confidence relation in eyewitness identifications.
Wells, G. L., & Lindsay, R. C. L. (1985) · Journal of Applied Psychology · doi:10.1037/0021-9010.70.2.413 · Cited by 116
The effect of horizontal eye movements on free recall: a preregistered adversarial collaboration.
Matzke, D., Nieuwenhuis, S., van Rijn, H., Slagter, H., van der Molen, M. W., & Wagenmakers, E. (2015) · Journal of experimental psychology. General · doi:10.1037/xge0000038 · Cited by 148
“That is, the beneficial effect of horizontal eye movements on free recall may seem more established than it actually is, due to publication bias and the file-drawer problem (Greenwald, 1975; Rosenthal, 1979).”
Multiple discovery and invention: Zeitgeist, genius, or chance?
Simonton, D. K. (1979) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.37.9.1603 · Cited by 113
Psychology of the scientist: An analysis of problem-solving bias
Mahoney, M. J., & DeMonbreun, B. G. (1977) · Cognitive Therapy and Research · doi:10.1007/bf01186796 · Cited by 113
“However, there are more than a few epistemological and practical problems with null hypothesis testing (Greenwald, 1975), some of which are illustrated in the present study.”
BackgroundQuantifying Support for the Null Hypothesis in Psychology: An Empirical Investigation
Aczél, B., Pálfi, B., Szollosi, A., Kovács, M., Szászi, B., Szécsi, P., Zrubka, M., Gronau, Q. F., van den Bergh, D., & Wagenmakers, E. (2018) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245918773742 · Cited by 122
Wells, G. L., & Lüüs, C. A. E. (1990) · Personality and Social Psychology Bulletin · doi:10.1177/0146167290161008 · Cited by 111
Meta-analysis: Techniques, applications, and functions.
Strube, M., & Hartmann, D. P. (1983) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.51.1.14 · Cited by 113
Bishop, D. (2019) · Quarterly Journal of Experimental Psychology · doi:10.1177/1747021819886519 · Cited by 115
“As Hahn and Harris (2014) noted, a range of definitions of “confirmation bias” exist—here, I will define it as the tendency to seek out evidence that supports our position.”
BackgroundInfluentialDoes Sunshine Prime Loyal? Affective Priming in the Naming Task
Klauer, K. C., & Musch, J. (2001) · The Quarterly Journal of Experimental Psychology Section A · doi:10.1080/713755986 · Cited by 108
“If only those studies are published in which the null hypothesis is rejected and the effect is found, the resulting selection severely distorts the data base on which such reviews rely, thereby compromising the conclusions that can be drawn from the research literature (Greenwald, 1975).”
BackgroundThe Quest for α: Developments in Multiple Comparison Procedures in the Quarter Century Since
Hancock, G. R., & Klockars, A. J. (1996) · Review of Educational Research · doi:10.3102/00346543066003269 · Cited by 108
“The first such method was formally proposed by Holm (1979), although anticipated by Hartley (1955) many years earlier, and is generally referred to as a sequentially rejective Bonferroni procedure.”
BackgroundMethodScientific Utopia: II. Restructuring incentives and practices to promote truth over publishabilityPreprint
Nosek, B. A., Spies, J. R., & Motyl, M. S. (2016) · doi:10.31219/osf.io/k4bgq · Cited by 114
Many Labs 5: Testing Pre-Data-Collection Peer Review as an Intervention to Increase Replicability
Ebersole, C. R., Mathur, M. B., Baranski, E., Bart-Plange, D.-J., Buttrick, N. R., Chartier, C. R., Corker, K. S., Corley, M., Hartshorne, J. K., IJzerman, H., et al. (2020) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245920958687 · Cited by 105
“The initial finding might have been a false positive, reflecting either a “normal” Type I error or one made more likely by selectively reporting positive results and ignoring null results (Greenwald, 1975; Rosenthal, 1979; Sterling, 1959) or by employing flexibility in analytic decisions and reporting (Gelman & Loken,…”
BackgroundPredicting replication outcomes in the Many Labs 2 study
Forsell, E., Viganola, D., Pfeiffer, T., Almenberg, J., Wilson, B., Chen, Y., Nosek, B. A., Johannesson, M., & Dreber, A. (2018) · Journal of Economic Psychology · doi:10.1016/j.joep.2018.10.009 · Cited by 101
Accessibility and fit as determinants of the salience of social categorizations
Blanz, M. (1999) · European Journal of Social Psychology · doi:10.1002/(sici)1099-0992(199902)29:1 · Cited by 105
Barak, A., Fisher, W. A., Belfry, S., & Lashambe, D. R. (1999) · Journal of Psychology & Human Sexuality · doi:10.1300/j056v11n01_04 · Cited by 101
“Further, special reference should be made to the fact that this research represents a ‘‘classic’’ null findings report, in that it presents null findings in the context of significant findings (for an extended discussion of issues relating to null findings, see Greenwald, 1975 ).”
Keselman, H. J., Cribbie, R. A., & Holland, B. S. (1999) · Psychological Methods · doi:10.1037/1082-989x.4.1.58 · Cited by 101
History of the sleeper effect: Some logical pitfalls in accepting the null hypothesis.
Cook, T. D., Gruder, C. L., Hennigan, K. M., & Flay, B. R. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.4.662 · Cited by 101
Perry, D. F., Allen, M. D., Brennan, E. M., & Bradley, J. R. (2010) · Early Education and Development · doi:10.1080/10409280903475444 · Cited by 105
“Because our review team decided to include both published peer-reviewed articles and unpublished studies with comparison groups, we aimed at avoiding the bias resulting from the tendency to leave null results unpublished (Greenwald, 1975; Sadish, 1992).”
MethodMaking 'null effects' informative: statistical techniques and inferential frameworks
Harms, C., & Lakens, D. (2018) · Journal of Clinical and Translational Research · doi:10.18053/jctres.03.2017s2.007 · Cited by 128
Making 'Null Effects' Informative: Statistical Techniques and Inferential Frameworks
Harms, C., & Lakens, D. (2018) · doi:10.31234/osf.io/48zca · Cited by 99
Social Psychology, Science and History
Gergen, K. J. (1976) · Personality and Social Psychology Bulletin · doi:10.1177/014616727600200409 · Cited by 94
Active Versus Passive Processing of Spatial Information in a Computer-Simulated Environment
Wilson, P. N., Foreman, N., Gillett, R., & Fraser, D. S. (1997) · Ecological Psychology · doi:10.1207/s15326969eco0903_3 · Cited by 101
Bar‐Anan, Y., Houwer, J. D., & Nosek, B. A. (2010) · Quarterly Journal of Experimental Psychology · doi:10.1080/17470211003802442 · Cited by 94
Précis of Statistical significance: Rationale, validity, and utility
Chow, S. L. (1998) · Behavioral and Brain Sciences · doi:10.1017/s0140525x98001162 · Cited by 94
Sturman, M. C. (1999) · Educational and Psychological Measurement · doi:10.1177/00131649921969956 · Cited by 93
Tricco, A. C., Tetzlaff, J., Pham, B., Brehaut, J., & Moher, D. (2009) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2008.08.008 · Cited by 91
The person‐situation debate: A critical multiplist perspective
Houts, A. C., Cook, T. D., & Shadish, W. R. (1986) · Journal of Personality · doi:10.1111/j.1467-6494.1986.tb00390.x · Cited by 90
Experimental methods and outcome evaluation.
Mahoney, M. J. (1978) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.46.4.660 · Cited by 93
Publication Bias in Two Political Behavior Literatures
Gerber, A. S., Malhotra, N., Dowling, C. M., & Doherty, D. (2010) · American Politics Research · doi:10.1177/1532673x09350979 · Cited by 67
“A number of studies have identified publication bias in the fields of psychology (e.g., Coursol & Wagner, 1986; Greenwald, 1975; Sterling, 1959), public health and medicine (e.g., Gotzsche, 2006), economics (e.g., Ashenfelter, Harmon, & Oosterbeek, 1999; Card & Krueger, 1995; De Long & Lang, 1992;…”
BackgroundHyatt, C. S., Owens, M. M., Crowe, M. L., Carter, N. T., Lynam, D. R., & Miller, J. D. (2019) · NeuroImage · doi:10.1016/j.neuroimage.2019.116225 · Cited by 94
“We believe a fi eld-wide prejudice against the null (Greenwald, 1975) has been a source of undue pressure on researchers to present signi fi cant fi ndings; conducting analyses with adequate statistical power and pre-registration of intended covariate use greatly mitigates these concerns.”
Publication Bias: A call for improved meta‐analytic practice in the organizational sciences
Banks, G. C., Kepes, S., & McDaniel, M. A. (2012) · International Journal of Selection and Assessment · doi:10.1111/j.1468-2389.2012.00591.x · Cited by 96
“In these instances, studies are rejected in the editorial review process because the sample size was small, the results were not statistically significant (Greenwald, 1975), the results were contrary to theory, contrary to the position of the editor/reviewers, or contrary to purportedly well-established knowledge (Ban…”
ResultThe Effect of Nutritional Information Disclosure in Advertising: An Information Processing Approach
Brucks, M. L., Mitchell, A. A., & Staelin, R. (1984) · Journal of Public Policy & Marketing · doi:10.1177/074391568400300101 · Cited by 92
American psychology in the eighties: Who will buy?
Fishman, D. B., & Neigher, W. D. (1982) · American Psychologist · doi:10.1037/0003-066x.37.5.533 · Cited by 94
Systematic Reviews With Meta-Analysis
Crocetti, E. (2015) · Emerging Adulthood · doi:10.1177/2167696815617076 · Cited by 117
“…and in line with hypotheses are more likely to be submitted for publication and to be accepted than study results which are nonsignificant or significant but in contrast with hypotheses (Cooper, DeNeve, & Charlton, 1997; Greenwald, 1975; Krzyzanowska, Pintilie, &Tannock, 2003; Mahoney, 1977).”
ResultCunningham, M. R., & Baumeister, R. F. (2016) · Frontiers in Psychology · doi:10.3389/fpsyg.2016.01639 · Cited by 93
“But, the preference for significant results in the publication process can exert a bias in a meta-analytic sample (Greenwald, 1975; Rosenthal, 1979).”
BackgroundMcCabe, J. E. (2013) · Psychological Bulletin · doi:10.1037/a0034835 · Cited by 103
Uniformity versus Flexibility: Evidence from Pricing of the Pension Obligation
Hann, R. N., Lu, Y. Y., & Subramanyam, K. R. (2007) · The Accounting Review · doi:10.2308/accr.2007.82.1.107 · Cited by 89
“Following Chung and Kallapur (2003) and Greenwald (1975), we examine the confidence intervals of the coefficient differences to ascertain whether the null is likely to be true.”
A Validity-Based Framework for Understanding Replication in Psychology
Fabrigar, L. R., Wegener, D. T., & Petty, R. E. (2020) · Personality and Social Psychology Review · doi:10.1177/1088868320931366 · Cited by 114
Why most psychological research findings are not even wrong
Scheel, A. M. (2022) · Infant and Child Development · doi:10.1002/icd.2295 · Cited by 111
Kepes, S., Bennett, A. A., & McDaniel, M. A. (2014) · Academy of Management Learning and Education · doi:10.5465/amle.2013.0193 · Cited by 99
An introduction to using Bayesian linear regression with clinical data
Baldwin, S. A., & Larson, M. J. (2017) · Behaviour Research and Therapy · doi:10.1016/j.brat.2016.12.016 · Cited by 87
“…(c) authors, reviewers, and editors privileging statistically significant results over non-significant results may distort the published literature (Greenwald, 1975) as well as create incentives that can lead to poor data analysis practices (cf. Rosenthal, 1994); (d) a focus on p-values leads to a…”
BackgroundPhilosophical eminence, beliefs, and zeitgeist: An individual-generational analysis.
Simonton, D. K. (1976) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.34.4.630 · Cited by 88
A Practical Guide to Modern Methods of Meta-Analysis.
Hedges, L. V. (1989) · Cited by 86
Null hypothesis disrespect in neuropsychology: dangers of alpha and beta errors.
Soper, H., Cicchetti, D. V., Satz, P., Light, R., & Orsini, D. L. (1988) · Journal of Clinical and Experimental Neuropsychology · doi:10.1080/01688638808408240 · Cited by 81
Mood-specific effects on appraisal and emotion judgements
Siemer, M. (2001) · Cognition & Emotion · doi:10.1080/02699930126083 · Cited by 84
Dance, K. A., & Neufeld, R. W. J. (1988) · Psychological Bulletin · doi:10.1037/0033-2909.104.2.192 · Cited by 84
Malignant side effects of null-hypothesis significance testing
Branch, M. (2014) · Theory & Psychology · doi:10.1177/0959354314525282 · Cited by 81
“…probabilistic information about the reliability of research findings (e.g., Bakan, 1966; Carver, 1978; Cohen, 1994; Falk & Greenbaum, 1995; Gelman, Carlin, Stern, & Rubin, 1995; Gigerenzer, 1993; Greenwald, 1975; Kline, 2004; Lambdin, 2012; Nickerson, 2000; Oakes, 1986; Sohn, 1998; Thompson, 1999).”
BackgroundResearch Design in Clinical Psychology
Kazdin, A. E. (2021) · doi:10.1017/9781108993647 · Cited by 240
Metadata
- Authors
- Greenwald, A. G.
- Affiliations
- The Ohio State University (US)
- Corresponding author
- Greenwald, Anthony G.
- Venue
- Psychological Bulletin
- Published
- 1975-01
- Volume
- 82
- Issue
- 1
- Pages
- 1-20
- ISSN
- 0033-2909 (ISSN-L)
- 1939-1455
- OpenAlex
- W2116618538
- Cited by
- 1,022
- References
- 47
- Citations by year
- 2026: 12 · 2025: 17 · 2024: 18 · 2023: 19 · 2022: 22 · 2021: 54 · 2020: 49 · 2019: 28 · 2018: 43 · 2017: 39 · 2016: 39 · 2015: 28 · 2014: 28 · 2013: 35 · 2012: 32
- FWCI
- 20.20
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Psychological Bulletin (publishedVersion)
- Funders
- National Science Foundation
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
- Keywords
- null hypothesis, publication bias, editorial decision-making, social psychology
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
