The DOI matches its Crossref record; 99 of the 99 references it cites were found in a registry or an index.
Why Hypothesis Testers Should Spend Less Time Testing Hypotheses
Open accessPublished version of a preprint
Abstract
For almost half a century, Paul Meehl educated psychologists about how the mindless use of null-hypothesis significance tests made research on theories in the social sciences basically uninterpretable. In response to the replication crisis, reforms in psychology have focused on formalizing procedures for testing hypotheses. These reforms were necessary and influential. However, as an unexpected consequence, psychological scientists have begun to realize that they may not be ready to test hypotheses. Forcing researchers to prematurely test hypotheses before they have established a sound "derivation chain" between test and theory is counterproductive. Instead, various nonconfirmatory research activities should be used to obtain the inputs necessary to make hypothesis tests informative. Before testing hypotheses, researchers should spend more time forming concepts, developing valid measures, establishing the causal relationships between concepts and the functional form of those relationships, and identifying boundary conditions and auxiliary assumptions. Providing these inputs should be recognized and incentivized as a crucial goal in itself. In this article, we discuss how shifting the focus to nonconfirmatory research can tie together many loose ends of psychology's reform movement and help us to develop strong, testable theories, as Paul Meehl urged.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Due to this lack of theory and robust data, we as AI researchers might simply not be ready to test specific hypotheses for certain contexts-a conclusion that is not unheard of in other research fields, such as psychology (Scheel et al., 2021) and human-robot interaction (Leichtmann et al., 2022).”
From Effects of Explainable Artificial Intelligence on trust and human behavior in a high-risk decision task (2023) · cites this paper as Scheel et al., 2021
The cited source says what the claim attributes to it.
In the citing paper
“Scheel et al. (2021) suggested that, before testing hypotheses confirmatorily, prior steps such as exploratory experimentation can help to discover relationships between variables and to explore boundaries on the basis of which more concrete theoretical assumptions can be made.”
From Effects of Explainable Artificial Intelligence on trust and human behavior in a high-risk decision task (2023) · cites this paper as Scheel et al., 2021
The cited source says what the claim attributes to it.
In the citing paper
“Thus, it becomes evident that more specific theorizing enables more precise operationalizations and more adequate empirical testing (Scheel et al., 2021).”
From Do Salient Social Norms Moderate Mortality Salience Effects? A (Challenging) Meta-Analysis of Terror Management Studies (2023) · cites this paper as Scheel et al., 2021
The cited source says what the claim attributes to it.
In the citing paper
“Even though the studies have confirmed the significance and influence of BPN Satisfaction for HCI research, the inconsistent use of measures is problematic in terms of comparability between studies and their replicability.”
From Development and Validation of a Basic Psychological Needs Scale for Technology Use (2024) · cites this paper as Scheel et al. (2021)
The cited source says what the claim attributes to it.
In the citing paper
“If a study fails to apply a valid measure, theoretical inferences cannot be made as the findings could equally measure random noise or a related but nevertheless distinct construct (Scheel et al., 2021).”
From Development and Validation of a Basic Psychological Needs Scale for Technology Use (2024) · cites this paper as Scheel et al., 2021
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The genetical evolution of social behaviour. I
Hamilton, W. D. (1964) · Journal of Theoretical Biology · doi:10.1016/0022-5193(64)90038-4 · Cited by 15,725 · cited 2×
“For example, natural selection favors strategies that maximize inclusive fitness (Hamilton, 1964), which is often not accomplished by self-preservation (e.”
Background(2002) · Cited by 13,211
Causality: Models, Reasoning, and Inference
Pearl, J. (2009) · doi:10.1017/cbo9780511803161 · Cited by 11,653
Estimating the reproducibility of psychological science
Open Science Collaboration (2015) · Science · doi:10.1126/science.aac4716 · Cited by 8,515
Mate selection—A selection for a handicap
Zahavi, A. (1975) · Journal of Theoretical Biology · doi:10.1016/0022-5193(75)90111-3 · Cited by 5,251 · cited 1×
“For example, when Zahavi (1975) proposed the idea that the costliness of signals ensures their reliability (i.e., the handicap principle), many biologists found the idea implausible.”
BackgroundThe genetical evolution of social behaviour. II
Hamilton, W. D. O. (1964) · Journal of Theoretical Biology · doi:10.1016/0022-5193(64)90039-6 · Cited by 4,158 · cited 1×
“For example, natural selection favors strategies that maximize inclusive fitness (Hamilton, 1964), which is often not accomplished by self-preservation (e.g., people investing less in their future health when extrinsic mortality risks are high; Nettle, 2010).”
BackgroundA Next Generation Connectivity Map: L1000 Platform and the First 1,000,000 Profiles
Subramanian, A., Narayan, R., Corsello, S. M., Peck, D. D., Natoli, T. E., Lu, X., Gould, J., Davis, J. F., Tubelli, A. A., Asiedu, J. K., et al. (2017) · Cell · doi:10.1016/j.cell.2017.10.049 · Cited by 3,515
The Methodology of Scientific Research Programmes: Philosophical Papers
Lakatos, I. (1978) · doi:10.1017/cbo9780511621123 · Cited by 3,045
Hempel, C. G. (1966) · Medical Entomology and Zoology · Cited by 2,518
The Causes and Consequences of a Need for Self-Esteem: A Terror Management Theory
Greenberg, J., Pyszczynski, T., & Solomon, S. (1986) · Public Self and Private Self · doi:10.1007/978-1-4613-9564-5_10 · Cited by 2,326 · cited 1×
“For example, terror-management theory (TMT) assumes that humans have an instinct for self-preservation that led to the evolution of an incapacitating fear of death with which humans cope via an anxiety-reducing “terrormanagement” system (Greenberg et al., 1986).”
Background
Research works printed but not linked: 7
- Dubin R., 1969, Theory building
- Hernán M., 2020, Causal inference: What if
- Nature Human Behaviour . (2020). Registered reports: Author and reviewer guidelines. https://web.archive.org/web/20200229230434/https://media.nature.com/original/nature-cms/uploads/ckeditor/attachments/4825/RegisteredReportsGuidelines_NatureHumanBehaviour.pdf
- Riesch H. (2008). Scientists’ views of the philosophy of science [Doctoral dissertation, University College London]. https://discovery.ucl.ac.uk/id/eprint/1446063
- Steinnes K. K. (2017). Too cute for words: Cuteness evokes the kama muta emotion and motivates communal sharing [Master’s thesis, University of Oslo]. http://urn.nb.no/URN:NBN:no-60030
- Tinbergen N., 1951, The study of instinct
- Zittermann A., 2014, Anticancer Research, 34, 4641
Cited by
Sorted by citations
Lakens, D. (2022) · Collabra Psychology · doi:10.1525/collabra.33267 · Cited by 1,962
Self-citationThat’s a Lot to Process! Pitfalls of Popular Path Models
Rohrer, J. M., Hünermund, P., Arslan, R. C., & Elson, M. (2022) · Advances in Methods and Practices in Psychological Science · doi:10.1177/25152459221095827 · Cited by 267
“For example, we may notice that open-ended exploration, description (Rozin, 2001), or prediction (Yarkoni & Westfall, 2017) are more suitable endeavors for the matter at hand or that more basic questions about measurement need to be settled first (Scheel et al., 2021).”
BackgroundBack to Basics: The Importance of Conceptual Clarification in Psychological Science
Bringmann, L. F., Elmer, T., & Eronen, M. I. (2022) · Current Directions in Psychological Science · doi:10.1177/09637214221096485 · Cited by 221
“Although the lack of conceptual clarity has been observed to be a widespread and fundamental problem in psychology (Antonakis, 2017; Eronen & Bringmann, 2021; Flake, 2021; MacKenzie, 2003; Podsakoff et al., 2016; Scheel et al., 2021), conceptual clarification plays a mostly marginal role in psychological research (see…”
BackgroundRohrer, J. M., & Murayama, K. (2023) · Advances in Methods and Practices in Psychological Science · doi:10.1177/25152459221140842 · Cited by 240
Sweller, J. (2023) · Educational Psychology Review · doi:10.1007/s10648-023-09817-2 · Cited by 220
Metadata
- Authors
- Scheel, A. M., Tiokhin, L., Isager, P. M., & Lakens, D.
- Affiliations
- Eindhoven University of Technology (NL)
- Corresponding author
- Scheel, Anne M.
- Venue
- Perspectives on Psychological Science
- Publisher
- SAGE Publications
- Published
- 2020-12-16
- Volume
- 16
- Issue
- 4
- Pages
- 744-755
- ISSN
- 1745-6916 (ISSN-L)
- 1745-6924
- PMC
- PMC8273364
- OpenAlex
- W3113132966
- Cited by
- 382
- References
- 95
- Citations by year
- 2026: 32 · 2025: 48 · 2024: 82 · 2023: 69 · 2022: 80 · 2021: 59 · 2020: 12
- FWCI
- 143.27
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- Perspectives on Psychological Science (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- TU/e Research Portal (publishedVersion, cc-by)
- PsyArXiv (OSF Preprints) (submittedVersion, cc-by)
- PsyArXiv (OSF Preprints) (submittedVersion)
- TU/e Research Portal (submittedVersion, cc-by)
- Funders
- Nederlandse Organisatie voor Wetenschappelijk Onderzoek (VIDI grant 452-17-013)
- Topics
- Social Sciences › Psychology › General Psychology › Academic and Historical Perspectives in Psychology
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Keywords
- null hypothesis significance testing, hypothesis testing, replication crisis, Paul Meehl, theory testing, measurement validity, auxiliary hypotheses, boundary conditions
- MeSH terms
- Humans, Psychological Theory, Psychology, Causality
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
