Theory-Testing in Psychology and Physics: A Methodological Paradox
The DOI matches its Crossref record; 14 of the 14 references it cites were found in a registry or an index.
Abstract
Because physical theories typically predict numerical values, an improvement in experimental precision reduces the tolerance range and hence increases corroborability. In most psychological research, improved power of a statistical design leads to a prior probability approaching ½ of finding a significant difference in the theoretically predicted direction. Hence the corroboration yielded by “success” is very weak, and becomes weaker with increased precision. “Statistical significance” plays a logical role in psychology precisely the reverse of its role in physics. This problem is worsened by certain unhealthy tendencies prevalent among psychologists, such as a premium placed on experimental “cuteness” and a free reliance upon ad hoc explanations to avoid refutation.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“According to Meehl (1967) and Peter (1981), correlations between constructs from the same sample are highly likely, if the sample is large and reliability is given (at least to some degree).”
From Development and Validation of a Basic Psychological Needs Scale for Technology Use (2024) · cites this paper as Meehl (1967)
In the citing paper
“Why? Such studies are cheap and can be farmed for significant results, especially when hypotheses only predict differences from the null, rather than precise quantitative differences and trends [3].”
From The natural selection of bad science (2016) · cites this paper as [3]
Citations
Cites
(2020) · Radical Treatment · doi:10.2307/j.ctvw1d7dg.9 · Cited by 3,373
ALEXANDER, P. (1963) · Philosophical Books · doi:10.1111/j.1468-0149.1963.tb00798.x · Cited by 2,854
The logic of scientific discovery
(1959) · Journal of the Franklin Institute · doi:10.1016/s0016-0032(59)90407-7 · Cited by 1,915
Boneau, C. A., & Hays, W. L. (1966) · The American Journal of Psychology · doi:10.2307/1420739 · Cited by 1,881
The statistical power of abnormal-social psychological research: A review.
Cohen, J. (1962) · The Journal of Abnormal and Social Psychology · doi:10.1037/h0045186 · Cited by 1,824
Logic of Scientific Discovery, The
(2010) · Encyclopedia of Research Design · doi:10.4135/9781412961288.n223 · Cited by 1,545
Statistics for Psychologists: An Intermediate Course
Everitt, B. S. (2001) · Psychology Press eBooks · doi:10.4324/9781410605542 · Cited by 1,045
The test of significance in psychological research.
Bakan, D. (1966) · Psychological Bulletin · doi:10.1037/h0020412 · Cited by 911
The fallacy of the null-hypothesis significance test.
Rozeboom, W. W. (1960) · Psychological Bulletin · doi:10.1037/h0042040 · Cited by 729
Directional statistical decisions.
Kaiser, H. F. (1960) · Psychological Review · doi:10.1037/h0047595 · Cited by 212
Research works printed but not linked: 1
- Lykken, 1966, Reports from the Research Laboratories of the Department of Psychiatry, University of Minnesota. Report No. PR-66-9
Cited by
Items 301–400 of 1,001
Moeller, J. (2022) · Journal for Person-Oriented Research · doi:10.17505/jpor.2021.23795 · Cited by 46
Ecology, statistics, and the art of misdiagnosis: The need for a paradigm shift
Germano, J. D. (1999) · Environmental Reviews · doi:10.1139/a99-014 · Cited by 46
Rethinking ‘quantitative’ methods and the development of new researchers
Gorard, S. A. C. (2015) · Review of Education · doi:10.1002/rev3.3041 · Cited by 45
Leenen, I. (2012) · Investigación en Educación Médica · doi:10.22201/fm.20075057e.2012.04.00010 · Cited by 45
Campbell, L., Loving, T. J., & LeBel, E. P. (2014) · Personal Relationships · doi:10.1111/pere.12053 · Cited by 45
Thomas, J. G., & Sharp, P. B. (2019) · Clinical Psychological Science · doi:10.1177/2167702618810223 · Cited by 45
VanDerHeyden, A. M., & Burns, M. K. (2018) · School Psychology Review · doi:10.17105/spr-2018-0042.v47-4 · Cited by 45
Knowles, E. S. (1983) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.45.6.1263 · Cited by 45
Bootstrapping to test for nonzero population correlation coefficients using univariate sampling.
Beasley, W. H., DeShea, L., Toothaker, L. E., Mendoza, J. L., Bard, D., & Rodgers, J. L. (2007) · Psychological Methods · doi:10.1037/1082-989x.12.4.414 · Cited by 45
The Motivational Effects of Pay Fairness: A Longitudinal Study in Chinese Star-Level Hotels
Wu, X., Sturman, M. C., & Wang, C. (2013) · Cornell Hospitality Quarterly · doi:10.1177/1938965512471891 · Cited by 45
The "Politics" of Redistribution: Toward a Model of the Policy-Making Process in the American States
Uslaner, E. M., & Weber, R. E. (1975) · American Politics Research · doi:10.1177/1532673x7500300202 · Cited by 45
Approaching a fair deal for significance and other concerns
Giner‐Sorolla, R. (2016) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2016.01.010 · Cited by 45
Editorial Commentary: Addressing Confusion in the Diffusion of Archival Data Research
Miller, J. W., Davis‐Sramek, B., Fugate, B. S., Pagell, M. D., & Flynn, B. B. (2020) · Journal of Supply Chain Management · doi:10.1111/jscm.12236 · Cited by 45
Bayesian data analysis for newcomersPreprint
Kruschke, J. K., & Liddell, T. M. (2017) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/nqfr5 · Cited by 44
Moore, J. (2010) · The Psychological Record · doi:10.1007/bf03395698 · Cited by 43
Holtz, P., & Odağ, Ö. (2018) · Qualitative Research in Psychology · doi:10.1080/14780887.2018.1447622 · Cited by 43
The widespread misinterpretation of p -values as error probabilities
Hubbard, R. (2011) · Journal of Applied Statistics · doi:10.1080/02664763.2011.567245 · Cited by 43
Statistical Power: Derivation, Development, and Data-Analytic Implications
Chase, L. J., & Tucker, R. K. (1976) · The Psychological Record · doi:10.1007/bf03394413 · Cited by 43
Further discussion of Fiedler's contingency model of leadership effectiveness
Ashour, A. S. (1973) · Organizational Behavior and Human Performance · doi:10.1016/0030-5073(73)90059-7 · Cited by 43
Law and the fireside inductions (with postscript): Some reflections of a clinical psychologist
Meehl, P. E. (1989) · Behavioral Sciences & the Law · doi:10.1002/bsl.2370070408 · Cited by 42
Self-citationDamaging Real Lives through Obstinacy: Re-Emphasising Why Significance Testing is Wrong
Gorard, S. A. C. (2016) · Sociological Research Online · doi:10.5153/sro.3857 · Cited by 42
Dombrowski, S. C., McGill, R. J., Farmer, R. L., Kranzler, J. H., & Canivez, G. L. (2021) · School Psychology Review · doi:10.1080/2372966x.2021.1960126 · Cited by 42
Mixed-effects design analysis for experimental phonetics
Kirby, J., & Sonderegger, M. (2018) · Journal of Phonetics · doi:10.1016/j.wocn.2018.05.005 · Cited by 42
Past and Future American Psychological Association Guidelines for Statistical Practice
Finch, S., Thomason, N., & Cumming, G. (2002) · Theory & Psychology · doi:10.1177/0959354302126005 · Cited by 40
Error Probabilities in Educational and Psychological Research
Westermann, R., & Hager, W. (1986) · Journal of Educational Statistics · doi:10.2307/1164973 · Cited by 36
Mecklenbr�uker, S., & Hager, W. (1984) · Psychological Research · doi:10.1007/bf00309069 · Cited by 42
Development and Validation of a Basic Psychological Needs Scale for Technology Use
Moradbakhti, L., Leichtmann, B., & Mara, M. (2024) · Psychological Test Adaptation and Development · doi:10.1027/2698-1866/a000062 · Cited by 41
Revising Beliefs about the Merit of Unconscious Thought: Evidence in Favor of the Null Hypothesis
Newell, B. R., & Rakow, T. (2011) · Social Cognition · doi:10.1521/soco.2011.29.6.711 · Cited by 41
Adaptive Measurement and Assessment
Barney, M., & Fisher, W. P. (2016) · Annual Review of Organizational Psychology and Organizational Behavior · doi:10.1146/annurev-orgpsych-041015-062329 · Cited by 41
Making inferential leaps: Manipulation checks and the road towards strong inference
Gruijters, S. L. K. (2021) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2021.104251 · Cited by 42
McManus, I. C. (2022) · Symmetry · doi:10.3390/sym14040814 · Cited by 41
Quantitative Approaches to Treatment Process, Change Process, and Process-Outcome Research
Gelo, O. C. G., & Manzo, S. (2014) · Springer eBooks · doi:10.1007/978-3-7091-1382-0_13 · Cited by 41
The Verification of Independent Variables in Teaching Methods Research
Shaver, J. P. (1983) · Educational Researcher · doi:10.3102/0013189x012008003 · Cited by 41
The Replication Problem in Sociology: A Report and a Suggestion*
Wilson, F. D., SMOKE, G. L., & Martin, J. D. (1973) · Sociological Inquiry · doi:10.1111/j.1475-682x.1973.tb00711.x · Cited by 41
Brenes, E. R., Ciravegna, L., & Woodside, A. G. (2016) · Industrial Marketing Management · doi:10.1016/j.indmarman.2016.12.001 · Cited by 40
Basic statistical considerations for physiology: The journal Temperature toolbox
Caldwell, A. R., & Cheuvront, S. N. (2019) · Temperature · doi:10.1080/23328940.2019.1624131 · Cited by 40
New Guidelines for Null Hypothesis Significance Testing in Hypothetico-Deductive IS Research
scientist, B. S.-E. D., Mertens, W., & Recker, J. (2020) · Journal of the Association for Information Systems · doi:10.17705/1jais.00629 · Cited by 40
Onwuegbuzie, A. J., & Levin, J. R. (2003) · Journal of Modern Applied Statistical Methods · doi:10.22237/jmasm/1051747920 · Cited by 40
Czachowski, S., Seed, P. T., Schofield, P., & Ashworth, M. J. (2011) · PLoS ONE · doi:10.1371/journal.pone.0027378 · Cited by 40
Gorard, S. A. C., Hordósy, R., & See, B. H. (2013) · Journal of School Choice · doi:10.1080/15582159.2013.791182 · Cited by 40
Quantifying Archaeological Research
Clark, G. A. (1982) · Elsevier eBooks · doi:10.1016/b978-0-12-003105-4.50011-x · Cited by 40
Gorard, S. A. C., & Gorard, J. (2015) · International Journal of Social Research Methodology · doi:10.1080/13645579.2015.1091235 · Cited by 39
The empirical march: Making science better at self-correction.
Makel, M. C. (2014) · Psychology of Aesthetics Creativity and the Arts · doi:10.1037/a0035803 · Cited by 39
Fitts, D. A. (2010) · Behavior Research Methods · doi:10.3758/brm.42.1.3 · Cited by 39
Blind Analysis as a Correction for Confirmatory Bias in Physics and in Psychology
MacCoun, R. J., & Perlmutter, S. (2017) · Wiley eBooks · doi:10.1002/9781119095910.ch15 · Cited by 38
Understanding Replication in a Way That Is True to Science
Haig, B. D. (2021) · Review of General Psychology · doi:10.1177/10892680211046514 · Cited by 37
Hilbig, B. E., Moshagen, M., & Zettler, I. (2016) · European Journal of Personality · doi:10.1002/per.2085 · Cited by 37
Nonparametric Combination (NPC): A Framework for Testing Elaborate Theories
Caughey, D., Dafoe, A., & Seawright, J. (2017) · The Journal of Politics · doi:10.1086/689287 · Cited by 37
From Discovery to Justification: Outline of an Ideal Research Program in Empirical Psychology
Witte, E. H., & Zenker, F. (2017) · Frontiers in Psychology · doi:10.3389/fpsyg.2017.01847 · Cited by 38
Positive psychology and romantic scientism.
Brown, N. J. L., Sokal, A. D., & Friedman, H. L. (2014) · American Psychologist · doi:10.1037/a0037390 · Cited by 37
Problems in the Theory of Developmental Sequences
Campbell, R. L., & Richie, M. (2009) · Human Development · doi:10.1159/000272879 · Cited by 37
Predicting Worker Performance In Inequitable Settings
Vecchio, R. P. (1982) · Academy of Management Review · doi:10.5465/amr.1982.4285489 · Cited by 37
Bridging Functional MR Images and Scientific Inference: Reproducibility Maps
Liou, M., Su, H.-R., Lee, J.-D., Cheng, P. E., Huang, C.-C., & Tsai, C.-H. (2003) · Journal of Cognitive Neuroscience · doi:10.1162/089892903770007326 · Cited by 35
New Guidelines for Null Hypothesis Significance Testing in Hypothetico-Deductive IS Research
Mertens, W., & Recker, J. (2023) · Technology, work and globalization · doi:10.1007/978-3-031-38719-7_13 · Cited by 36
Do we know what we test and do we test what we want to know?
Klugkist, I. G., van Wesel, F., & Bullens, J. (2011) · International Journal of Behavioral Development · doi:10.1177/0165025411425873 · Cited by 36
Magnetic vestibular stimulation modulates default mode network fluctuations
Boegle, R., Stephan, T., Ertl, M., Glasauer, S., & Dieterich, M. (2015) · NeuroImage · doi:10.1016/j.neuroimage.2015.11.065 · Cited by 36
Robust Standards in Cognitive Science
Crüwell, S., Stefan, A. M., & Evans, N. J. (2019) · Computational Brain & Behavior · doi:10.1007/s42113-019-00049-8 · Cited by 37
The affective consequences of artistic and scientific problem solving
Feist, G. J. (1994) · Cognition & Emotion · doi:10.1080/02699939408408955 · Cited by 36
Analysis of No-Difference Findings in Evaluation Research
Julnes, G., & Mohr, L. B. (1989) · Evaluation Review · doi:10.1177/0193841x8901300604 · Cited by 36
Meehl, P. E. (1983) · University of Minnesota Digital Conservancy (University of Minnesota) · Cited by 36
Self-citationConfidence Intervals and the Scientific Method: A Case for Holm on the Range
Serlin, R. C. (1993) · The Journal of Experimental Education · doi:10.1080/00220973.1993.10806595 · Cited by 36
Erratum: How to do things with historical texts
Biernacki, R. (2015) · American Journal of Cultural Sociology · doi:10.1057/ajcs.2015.8 · Cited by 36
What the replication crisis means for intervention science
Hillary, F. G., & Medaglia, J. D. (2019) · International Journal of Psychophysiology · doi:10.1016/j.ijpsycho.2019.05.006 · Cited by 36
Meaningful change definitions: sample size planning for experimental intervention research
Gruijters, S. L. K., & Peters, G.-J. (2020) · Psychology and Health · doi:10.1080/08870446.2020.1841762 · Cited by 35
Beyond p-values: Using Bayesian Data Analysis in Science Education Research
Kubsch, M., Stamer, I., Steiner, M., Neumann, K., & Parchmann, I. (2021) · Practical assessment, research & evaluation · doi:10.7275/vzpw-ng13 · Cited by 35
The Effect of Model Misspecification on Growth Mixture Model Class Enumeration
McNeish, D., & Harring, J. R. (2017) · Journal of Classification · doi:10.1007/s00357-017-9233-y · Cited by 35
On Behaviorism, Knowledge, and Causal Explanation
Moore, J. (1984) · The Psychological Record · doi:10.1007/bf03394854 · Cited by 35
Individuality in a Developmental Context: Some Empirical and Theoretical Considerations
Mumford, M. D., & Owens, W. A. (2009) · Human Development · doi:10.1159/000272906 · Cited by 35
Batmaz, A. U., & Stuerzlinger, W. (2022) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2022.3203105 · Cited by 34
Improving the Meta-Analytic Assessment of Effect Size Variance With an Informed Bayesian Prior
Steel, P., Kammeyer‐Mueller, J. D., & Paterson, T. A. (2014) · Journal of Management · doi:10.1177/0149206314551964 · Cited by 34
Testing Point Null Hypothesis of a Normal Mean and the Truth: 21st Century Perspective
Rao, C. R., & Lovrić, M. (2016) · Journal of Modern Applied Statistical Methods · doi:10.22237/jmasm/1478001660 · Cited by 34
Career orientations of I.S. personnel
Ginzberg, M. J., & Baroudi, J. J. (1992) · ACM SIGCPR Conference on Computer Personnel Research (SIGCPR) · doi:10.1145/144001.144027 · Cited by 34
Theory, Experiment and the Shifting Publication Scene in Personality and Social Psychology
Kruglanski, A. W. (1975) · Personality and Social Psychology Bulletin · doi:10.1177/014616727500100305 · Cited by 34
Spatial confidence sets for raw effect size images
Bowring, A., Telschow, F. J. E., Schwartzman, A., & Nichols, T. E. (2019) · NeuroImage · doi:10.1016/j.neuroimage.2019.116187 · Cited by 33
Confidence intervals, precision and confounding
Trafimow, D. (2018) · New Ideas in Psychology · doi:10.1016/j.newideapsych.2018.04.005 · Cited by 33
Some methodological reflections on the difficulties of psychoanalytic research.
Meehl, P. E. (1973) · PubMed · Cited by 33
Self-citationHoltz, P. (2020) · Frontiers in Psychology · doi:10.3389/fpsyg.2020.545959 · Cited by 32
Statistical Conclusion Validity for Organizational Science Researchers: A Review
Austin, J. T., Boyle, K. A., & Lualhati, J. C. (1998) · Organizational Research Methods · doi:10.1177/109442819812002 · Cited by 32
The origins of ‘mainstream sociology’ and other issues in the history of American sociology
Turner, S. (1994) · Social Epistemology · doi:10.1080/02691729408578729 · Cited by 32
Mental Health Concepts of Evangelical Protestants
McLatchie, L. R., & Dragūns, J. G. (1984) · The Journal of Psychology · doi:10.1080/00223980.1984.10542857 · Cited by 32
Mathematical Statements in the Experimental Analysis of Behavior
Plaud, J. J., & O’Donohue, W. (1991) · The Psychological Record · doi:10.1007/bf03395120 · Cited by 32
The Epistemic Importance of Establishing the Absence of an Effect
Fidler, F. M., Thorn, F. S., Barnett, A., Kambouris, S., & Kruger, A. (2018) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245918770407 · Cited by 32
Kruschke, J. K., & Liddell, T. M. (2016) · OSF Preprints (OSF Preprints) · doi:10.31219/osf.io/ksfyr · Cited by 32
Tablets for teaching and learning: A systematic review and meta-analysis
Tamim, R., Borokhovski, E., Pickup, D., Bernard, R. M., & Saadi, L. E. (2015) · Commonwealth of Learning · doi:10.13140/rg.2.1.3670.5042 · Cited by 31
Integrating Media Selection and Media Effects Using Decision Theory
Fisher, J. T., & Hamilton, K. A. (2021) · Journal of Media Psychology Theories Methods and Applications · doi:10.1027/1864-1105/a000315 · Cited by 31
Empirical Tests of Scale Type for Individual Ratings
Westermann, R. (1985) · Applied Psychological Measurement · doi:10.1177/014662168500900304 · Cited by 31
Dawes, R. M. (2002) · Philosophy of Science · doi:10.1086/341844 · Cited by 31
The illusion of science in comparative cognitionPreprint
Farrar, B. G., & Ostojić, L. (2019) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/hduyx · Cited by 31
Pre-Registration in Social Psychology—a Discussion and Suggested TemplatePreprint
Veer, A. V. '., & Giner‐Sorolla, R. (2016) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/4frms · Cited by 32
Crüwell, S., & Evans, N. J. (2021) · Royal Society Open Science · doi:10.1098/rsos.210155 · Cited by 30
When numbers fail: do researchers agree on operationalization of published research?
Haucke, M. N., Hoekstra, R., & van Ravenzwaaij, D. (2021) · Royal Society Open Science · doi:10.1098/rsos.191354 · Cited by 30
Bayesian Variance Estimation for Meta-Analysis: Quantifying Our Uncertainty
Steel, P., & Kammeyer‐Mueller, J. D. (2007) · Organizational Research Methods · doi:10.1177/1094428107300339 · Cited by 30
Schmidt, F. M., & Hunter, J. E. (1995) · Evaluation & the Health Professions · doi:10.1177/016327879501800405 · Cited by 30
Behavioral Self-Control and Career Development
Thoresen, C. E., & Ewart, C. K. (1976) · The Counseling Psychologist · doi:10.1177/001100007600600307 · Cited by 30
Finding Practical Knowledge in Entrepreneurship
Kenworthy, T. P., & McMullan, W. E. (2013) · Entrepreneurship Theory and Practice · doi:10.1111/etap.12048 · Cited by 30
Introducing Advances in Methods and Practices in Psychological Science
Simons, D. J. (2018) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245918757424 · Cited by 30
Moeller, J., Dietrich, J., Neubauer, A. B., Brose, A., Kühnel, J., Dehne, M., Jaehne, M. F., Schmiedek, F., Bellhäuser, H., Malmberg, L., et al. (2022) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/5wsna · Cited by 30
Replication and the Establishment of Scientific Truth
Iso‐Ahola, S. E. (2020) · Frontiers in Psychology · doi:10.3389/fpsyg.2020.02183 · Cited by 29
Pfaff, H., & Schmitt, J. M. (2023) · Journal of Evaluation in Clinical Practice · doi:10.1111/jep.13890 · Cited by 29
Holtz, P., & Monnerjahn, P. (2017) · Journal for the Theory of Social Behaviour · doi:10.1111/jtsb.12134 · Cited by 29
Metadata
- Authors
- Meehl, P. E.
- Corresponding author
- Meehl, Paul E.
- Venue
- Philosophy of Science
- Publisher
- Cambridge University Press (CUP)
- Published
- 1967-06
- Volume
- 34
- Issue
- 2
- Pages
- 103-115
- ISSN
- 0031-8248 (ISSN-L)
- 1539-767X
- OpenAlex
- W1963603753
- Cited by
- 962
- References
- 14
- Citations by year
- 2026: 29 · 2025: 41 · 2024: 29 · 2023: 50 · 2022: 53 · 2021: 50 · 2020: 39 · 2019: 39 · 2018: 39 · 2017: 55 · 2016: 32 · 2015: 27 · 2014: 28 · 2013: 23 · 2012: 22
- FWCI
- 15.77
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Philosophy of Science (publishedVersion)
- Topics
- Social Sciences › Psychology › General Psychology › Academic and Historical Perspectives in Psychology
- Social Sciences › Arts and Humanities › History and Philosophy of Science › Philosophy of Biology and Scientific Explanation
- Social Sciences › Arts and Humanities › History and Philosophy of Science › Logic and Philosophy of Science
- Keywords
- theory testing, corroboration, statistical significance, experimental precision, statistical power, prior probability, direction of effect, experimental design, philosophy of psychology, philosophy of physics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.