The DOI matches its Crossref record; 34 of the 34 references it cites were found in a registry or an index.
Why Summaries of Research on Psychological Theories are Often Uninterpretable
Abstract
Null hypothesis testing of correlational predictions from weak substantive theories in soft psychology is subject to the influence of ten obfuscating factors whose effects are usually (1) sizeable, (2) opposed, (3) variable, and (4) unknown. The net epistemic effect of these ten obfuscating influences is that the usual research literature review is well-nigh uninterpretable. Major changes in graduate education, conduct of research, and editorial policy are proposed.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Convergent and discriminant validation by the multitrait-multimethod matrix.
Campbell, D. T., & Fiske, D. W. (1959) · Psychological Bulletin · doi:10.1037/h0046016 · Cited by 17,116
Construct validity in psychological tests.
Cronbach, L. J., & Meehl, P. E. (1955) · Psychological Bulletin · doi:10.1037/h0040957 · Cited by 11,195
Self-citationThe Logic of Scientific Discovery
(2013) · A Realist Theory of Science · doi:10.4324/9780203090732-9 · Cited by 7,731
Falsification and the Methodology of Scientific Research Programmes
Lakatos, I. (1970) · Criticism and the Growth of Knowledge · doi:10.1017/cbo9781139171434.009 · Cited by 4,649 · cited 2×
“…money goes down the drain in research that pseudotests theories in soft psychology and that it would be a material social advance as well as a reduction in what Lakatos has called "intellectual pollution" (Lakatos, 1970, fn. 1 on p. 176) if we would quit engaging in this feckless enterprise.”
BackgroundMethodFalsification and the Methodology of Scientific Research Programmes
Lakatos, I. (1976) · Can Theories be Refuted? · doi:10.1007/978-94-010-1863-0_14 · Cited by 3,915
Meta-analysis in social research
(1982) · Evaluation and Program Planning · doi:10.1016/0149-7189(82)90011-8 · Cited by 3,853
(2020) · Radical Treatment · doi:10.2307/j.ctvw1d7dg.9 · Cited by 3,373
Lehman, H. (1972) · Philosophy of Science · doi:10.1086/288419 · Cited by 2,998
ALEXANDER, P. (1963) · Philosophical Books · doi:10.1111/j.1468-0149.1963.tb00798.x · Cited by 2,854
Statistics for the social sciences
Hays, W. L. (1973) · Medical Entomology and Zoology · Cited by 2,081
Research works printed but not linked: 5
- Brunswik E., 1947, Systematic and representative design of psychological experiments. (University of California Syllabus Series, No. 304)
- Carnap R., 1956, Minnesota studies in the philosophy of science: I. The foundations of science and the concepts of psychology and psychoanalysis, 38
- Hunter J. E., 1982, Studying organizations: innovations in methodology, 4
- Meehl P. E., 1970, Minnesota studies in the philosophy of science: IV. Analyses of theories and methods of physics and psychology, 8, 403
- Meehl P. E., 1983, Minnesota studies in the philosophy of science: X. Testing scientific theories, 349
Cited by
Issues and opinion on structural equation modeling
Chin, W. W. (1998) · Journal of the Association for Information Systems · doi:10.5555/290231.290235 · Cited by 4,577
Cohen, J. (1994) · American Psychologist · doi:10.1037/0003-066x.49.12.997 · Cited by 4,003
Exploratory Factor Analysis: A Guide to Best Practice
Watkins, M. W. (2018) · Journal of Black Psychology · doi:10.1177/0095798418771807 · Cited by 2,218
Choosing Prediction Over Explanation in Psychology: Lessons From Machine Learning
Yarkoni, T., & Westfall, J. (2017) · Perspectives on Psychological Science · doi:10.1177/1745691617693393 · Cited by 1,916
Judge, T. A., Erez, A., Bono, J. E., & Thoresen, C. J. (2002) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.83.3.693 · Cited by 1,480
Grove, W. M., & Meehl, P. E. (1996) · Psychology, Public Policy, and Law · doi:10.1037/1076-8971.2.2.293 · Cited by 1,457
Self-citationFormative versus reflective measurement models: Two applications of formative measurement
Coltman, T., Devinney, T. M., Midgley, D. F., & Venaik, S. (2008) · Journal of Business Research · doi:10.1016/j.jbusres.2008.01.013 · Cited by 1,332
The Gaussian Graphical Model in Cross-Sectional and Time-Series Data
Epskamp, S., Waldorp, L. J., Mõttus, R., & Borsboom, D. (2018) · Multivariate Behavioral Research · doi:10.1080/00273171.2018.1454823 · Cited by 1,134
Null hypothesis significance testing: a review of an old and continuing controversy
Nickerson, R. S. (2000) · Psychological Methods · doi:10.1037/1082-989x.5.2.241 · Cited by 1,090
Being Inconsistent About Consistency: When Coefficient Alpha Does and Doesn't Matter
Streiner, D. L. (2003) · Journal of Personality Assessment · doi:10.1207/s15327752jpa8003_01 · Cited by 1,070
State of the aRt personality research: A tutorial on network analysis of personality data in R
Costantini, G., Epskamp, S., Borsboom, D., Perugini, M., Mõttus, R., Waldorp, L. J., & Cramer, A. O. J. (2015) · Journal of Research in Personality · doi:10.1016/j.jrp.2014.07.003 · Cited by 982
The Problem of Units and the Circumstance for POMP
Cohen, P., Cohen, J., Aiken, L. S., & West, S. G. (1999) · Multivariate Behavioral Research · doi:10.1207/s15327906mbr3403_2 · Cited by 900
Abandon Statistical Significance
McShane, B. B., Gal, D., Gelman, A., Robert, C., & Tackett, J. L. (2019) · The American Statistician · doi:10.1080/00031305.2018.1527253 · Cited by 791
Toward an Integrated Theory of Schizotaxia, Schizotypy, and Schizophrenia
Meehl, P. E. (1990) · Journal of Personality Disorders · doi:10.1521/pedi.1990.4.1.1 · Cited by 751
Self-citationBootstraps taxometrics: Solving the classification problem in psychopathology
Meehl, P. E. (1995) · American Psychologist · doi:10.1037/0003-066x.50.4.266 · Cited by 702
Self-citationMeehl, P. E. (1990) · Psychological Inquiry · doi:10.1207/s15327965pli0102_1 · Cited by 673
Self-citationCFI versus RMSEA: a comparison of two fit indexes for structural equation modeling
Rigdon, E. E. (1996) · Structural Equation Modeling: A Multidisciplinary Journal · doi:10.1080/10705519609540052 · Cited by 668
Understanding the Impact of Convergent Validity on Research Results
Carlson, K. D., & Herdman, A. O. (2012) · Organizational Research Methods · doi:10.1177/1094428110392383 · Cited by 642
Manipulating and Measuring Model Interpretability
Poursabzi-Sangdeh, F., Goldstein, D. G., Hofman, J. M., Wortman Vaughan, J. W., & Wallach, H. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445315 · Cited by 728
Measurement in Psychology: A Critical History of a Methodological Concept
Michell, J. (1999) · Cambridge University Press eBooks · doi:10.1017/cbo9780511490040 · Cited by 617
Finding Our Way: An Introduction to Path Analysis
Streiner, D. L. (2005) · The Canadian Journal of Psychiatry · doi:10.1177/070674370505000207 · Cited by 588
Yarkoni, T. (2022) · Behavioral and Brain Sciences · doi:10.1017/s0140525x20001685 · Cited by 701
SOCIAL CONTEXT OF PERFORMANCE EVALUATION DECISIONS.
Judge, T. A., & Ferris, G. R. (1993) · Academy of Management Journal · doi:10.2307/256513 · Cited by 544
Factors and Taxa, Traits and Types, Differences of Degree and Differences in Kind
Meehl, P. E. (1992) · Journal of Personality · doi:10.1111/j.1467-6494.1992.tb00269.x · Cited by 507
Self-citationDefault Bayes Factors for Model Selection in Regression
Rouder, J. N., & Morey, R. D. (2012) · Multivariate Behavioral Research · doi:10.1080/00273171.2012.734737 · Cited by 477
Replication, falsification, and the crisis of confidence in social psychology
Earp, B. D., & Trafimow, D. (2015) · Frontiers in Psychology · doi:10.3389/fpsyg.2015.00621 · Cited by 472
Small is beautiful: In defense of the small-N design
Smith, P. L., & Little, D. R. (2018) · Psychonomic Bulletin & Review · doi:10.3758/s13423-018-1451-8 · Cited by 456
The Theory Crisis in Psychology: How to Move Forward
Eronen, M. I., & Bringmann, L. F. (2021) · Perspectives on Psychological Science · doi:10.1177/1745691620970586 · Cited by 454
Pre-registration in social psychology—A discussion and suggested template
van 't Veer, A. E., & Giner-Sorolla, R. (2016) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2016.03.004 · Cited by 439
Amrhein, V., Trafimow, D., & Greenland, S. (2019) · The American Statistician · doi:10.1080/00031305.2018.1543137 · Cited by 425
Why Hypothesis Testers Should Spend Less Time Testing Hypotheses
Scheel, A. M., Tiokhin, L., Isager, P. M., & Lakens, D. (2021) · Perspectives on Psychological Science · doi:10.1177/1745691620966795 · Cited by 382
“Meehl (1990) termed this the “derivation chain,” a conjunction of theoretical and auxiliary premises that are necessary to predict observable outcomes.”
BackgroundMethodInfluentialMethodological Issues in Psychological Research on Culture
van de Vijver, F. J. R., & Leung, K. (2000) · Journal of Cross-Cultural Psychology · doi:10.1177/0022022100031001004 · Cited by 410
The Handbook of Behavior Change
(2020) · Cambridge University Press eBooks · doi:10.1017/9781108677318 · Cited by 407
Tryon, W. W. (2001) · Psychological Methods · doi:10.1037/1082-989x.6.4.371 · Cited by 391
Direct Fit to Nature: An Evolutionary Perspective on Biological and Artificial Neural Networks
Hasson, U., Nastase, S. A., & Goldstein, A. (2020) · Neuron · doi:10.1016/j.neuron.2019.12.002 · Cited by 387
Statistical Inference as Severe Testing: How to Get Beyond the Statistics Wars
Mayo, D. G. (2018) · doi:10.1017/9781107286184 · Cited by 376
Integrating explanation and prediction in computational social science
Hofman, J. M., Watts, D. J., Athey, S., Garip, F., Griffiths, T. L., Kleinberg, J., Margetts, H., Mullainathan, S., Salganik, M. J., Vazire, S., et al. (2021) · Nature · doi:10.1038/s41586-021-03659-0 · Cited by 371
Psychology Will Be a Much Better Science When We Change the Way We Analyze Data
Loftus, G. R. (1996) · Current Directions in Psychological Science · doi:10.1111/1467-8721.ep11512376 · Cited by 361
Method Effects, Measurement Error, and Substantive Conclusions
Lance, C. E., Dawson, B., Birkelbach, D., & Hoffman, B. J. (2010) · Organizational Research Methods · doi:10.1177/1094428109352528 · Cited by 355
Keep it real: rethinking the primacy of experimental control in cognitive neuroscience
Nastase, S. A., Goldstein, A., & Hasson, U. (2020) · NeuroImage · doi:10.1016/j.neuroimage.2020.117254 · Cited by 344
The Insignificance of Null Hypothesis Significance Testing
Gill, J. (1999) · Political Research Quarterly · doi:10.1177/106591299905200309 · Cited by 338
Boyd, R. L., & Schwartz, H. A. (2021) · Journal of Language and Social Psychology · doi:10.1177/0261927x20967028 · Cited by 317
Lack of theory building and testing impedes progress in the factor and network literature
Fried, E. I. (2020) · Psychological Inquiry · doi:10.1080/1047840x.2020.1853461 · Cited by 317
The earth is flat ( p > 0.05): significance thresholds and the crisis of unreplicable research
Amrhein, V., Korner-Nievergelt, F., & Roth, T. (2017) · PeerJ · doi:10.7717/peerj.3544 · Cited by 308
Identifying critical dimensions for discriminating among rapists.
Prentky, R. A., & Knight, R. A. (1991) · Journal of Consulting and Clinical Psychology · doi:10.1037//0022-006x.59.5.643 · Cited by 304
Should social science be more solution-oriented?
Watts, D. J. (2017) · Nature Human Behaviour · doi:10.1038/s41562-016-0015 · Cited by 301
The Relationship of Self-Efficacy Appraisals to Subsequent Health Related Outcomes: A Meta-Analysis
Holden, G. (1992) · Social Work in Health Care · doi:10.1300/j010v16n01_05 · Cited by 297
On Construct Validity: Issues of Method and Measurement.
Smith, G. T. (2005) · Psychological Assessment · doi:10.1037/1040-3590.17.4.396 · Cited by 290
When Null Hypothesis Significance Testing Is Unsuitable for Research: A Reassessment
Szucs, D., & Ioannidis, J. P. A. (2017) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2017.00390 · Cited by 298
Null hypothesis significance testing: On the survival of a flawed method.
Krueger, J. (2001) · American Psychologist · doi:10.1037//0003-066x.56.1.16 · Cited by 282
Measuring customer loyalty with multi‐item scales: A case for caution
Söderlund, M. (2006) · International Journal of Service Industry Management · doi:10.1108/09564230610651598 · Cited by 280
Understanding extraverts' enjoyment of social situations: The importance of pleasantness.
Lucas, R. E., & Diener, E. (2001) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.81.2.343 · Cited by 273
Meaningful Mediation Analysis: Plausible Causal Inference and Informative Communication
Pieters, R. (2017) · Journal of Consumer Research · doi:10.1093/jcr/ucx081 · Cited by 266
Changing Behavior Using the Theory of Planned Behavior
Ajzen, I., & Schmidt, P. (2020) · The Handbook of Behavior Change · doi:10.1017/9781108677318.002 · Cited by 266
On the Perception of Social Consensus
Krueger, J. (1998) · Advances in Experimental Social Psychology · doi:10.1016/s0065-2601(08)60384-6 · Cited by 254
Epistemology and the Psychology of Human Judgment
Bishop, M. A., & Trout, J. D. (2005) · doi:10.1093/0195162293.001.0001 · Cited by 253
The relationship of trait emotional intelligence with academic performance: A meta-analytic review
Perera, H. N., & DiGiacomo, M. (2013) · Learning and Individual Differences · doi:10.1016/j.lindif.2013.08.002 · Cited by 234
Leichtmann, B., Humer, C., Hinterreiter, A., Streit, M., & Mara, M. (2023) · Computers in Human Behavior · doi:10.1016/j.chb.2022.107539 · Cited by 200
Post-adoption behaviors of e-service customers: The interplay of cognition and emotion
Chea, S., & Luo, M. M. (2008) · International Journal of Electronic Commerce · doi:10.2753/jec1086-4415120303 · Cited by 188
Wang, Z. (2014) · Task-Based Language Teaching · doi:10.1075/tblt.5.02wan · Cited by 164
EREZ, A., BLOOM, M. C., & WELLS, M. T. (1996) · Personnel Psychology · doi:10.1111/j.1744-6570.1996.tb01801.x · Cited by 161
Changing Behavior Using the Model of Action Phases
Keller, L., Gollwitzer, P. M., & Sheeran, P. (2020) · The Handbook of Behavior Change · doi:10.1017/9781108677318.006 · Cited by 147
Borzekowski, D. L. G., Robinson, T. N., & Killen, J. D. (2000) · Journal of Adolescent Health · doi:10.1016/s1054-139x(99)00044-0 · Cited by 146
Schultebraucks, K., Yadav, V., Shalev, A. Y., Bonanno, G. A., & Galatzer-Levy, I. R. (2022) · Psychological Medicine · doi:10.1017/s0033291720002718 · Cited by 108
Making 'Null Effects' Informative: Statistical Techniques and Inferential Frameworks
Harms, C., & Lakens, D. (2018) · doi:10.31234/osf.io/48zca · Cited by 99
Malignant side effects of null-hypothesis significance testing
Branch, M. (2014) · Theory & Psychology · doi:10.1177/0959354314525282 · Cited by 81
Bishop, D. V. M., & Thompson, P. A. (2016) · PeerJ · doi:10.7717/peerj.1715 · Cited by 90
Leichtmann, B., Nitsch, V., & Mara, M. (2022) · Frontiers in Robotics and AI · doi:10.3389/frobt.2022.838116 · Cited by 66
Where Are the Self-Correcting Mechanisms in Science?
Vazire, S., & Holcombe, A. O. (2022) · Review of General Psychology · doi:10.1177/10892680211033912 · Cited by 81
Causal inference on human behaviour
Bailey, D. H., Jung, A. J., Beltz, A. M., Eronen, M. I., Gische, C., Hamaker, E. L., Kording, K. P., Lebel, C., Lindquist, M. A., Moeller, J., et al. (2024) · Nature Human Behaviour · doi:10.1038/s41562-024-01939-z · Cited by 56
Development and Validation of a Basic Psychological Needs Scale for Technology Use
Moradbakhti, L., Leichtmann, B., & Mara, M. (2024) · Psychological Test Adaptation and Development · doi:10.1027/2698-1866/a000062 · Cited by 41
Rigorous exploration in a model-centric science via epistemic iteration
Devezer, B., & Buzbas, E. O. (2023) · Journal of Applied Research in Memory and Cognition · doi:10.1037/mac0000121 · Cited by 27
Single Case Quantitative Methods for Practice‐Based Evidence
McMillan, D., & Morley, S. (2010) · Developing and Delivering Practice‐Based Evidence · doi:10.1002/9780470687994.ch5 · Cited by 18
(2009) · PubMed · Cited by 13
Pairing Machine Learning and Clinical Psychology: How You Evaluate Predictive Performance Matters
Jacobucci, R., Littlefield, A. K., Millner, A. J., Kleiman, E., & Steinley, D. (2020) · doi:10.31234/osf.io/2yber · Cited by 12
Metadata
- Authors
- Meehl, P. E.
- Venue
- Psychological Reports
- Publisher
- SAGE Publications
- Published
- 1990-02
- Volume
- 66
- Issue
- 1
- Pages
- 195-244
- ISSN
- 0033-2941 (ISSN-L)
- 1558-691X
- OpenAlex
- W1985898303
- Cited by
- 909
- References
- 34
- FWCI
- 29.10
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.