The DOI matches its Crossref record; 99 of the 99 references it cites were found in a registry or an index.
Why Hypothesis Testers Should Spend Less Time Testing Hypotheses
Open accessPublished version of a preprint
Abstract
For almost half a century, Paul Meehl educated psychologists about how the mindless use of null-hypothesis significance tests made research on theories in the social sciences basically uninterpretable. In response to the replication crisis, reforms in psychology have focused on formalizing procedures for testing hypotheses. These reforms were necessary and influential. However, as an unexpected consequence, psychological scientists have begun to realize that they may not be ready to test hypotheses. Forcing researchers to prematurely test hypotheses before they have established a sound "derivation chain" between test and theory is counterproductive. Instead, various nonconfirmatory research activities should be used to obtain the inputs necessary to make hypothesis tests informative. Before testing hypotheses, researchers should spend more time forming concepts, developing valid measures, establishing the causal relationships between concepts and the functional form of those relationships, and identifying boundary conditions and auxiliary assumptions. Providing these inputs should be recognized and incentivized as a crucial goal in itself. In this article, we discuss how shifting the focus to nonconfirmatory research can tie together many loose ends of psychology's reform movement and help us to develop strong, testable theories, as Paul Meehl urged.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Due to this lack of theory and robust data, we as AI researchers might simply not be ready to test specific hypotheses for certain contexts-a conclusion that is not unheard of in other research fields, such as psychology (Scheel et al., 2021) and human-robot interaction (Leichtmann et al., 2022).”
From Effects of Explainable Artificial Intelligence on trust and human behavior in a high-risk decision task (2023) · cites this paper as Scheel et al., 2021
The cited source says what the claim attributes to it.
In the citing paper
“Scheel et al. (2021) suggested that, before testing hypotheses confirmatorily, prior steps such as exploratory experimentation can help to discover relationships between variables and to explore boundaries on the basis of which more concrete theoretical assumptions can be made.”
From Effects of Explainable Artificial Intelligence on trust and human behavior in a high-risk decision task (2023) · cites this paper as Scheel et al., 2021
The cited source says what the claim attributes to it.
In the citing paper
“Thus, it becomes evident that more specific theorizing enables more precise operationalizations and more adequate empirical testing (Scheel et al., 2021).”
From Do Salient Social Norms Moderate Mortality Salience Effects? A (Challenging) Meta-Analysis of Terror Management Studies (2023) · cites this paper as Scheel et al., 2021
The cited source says what the claim attributes to it.
In the citing paper
“Even though the studies have confirmed the significance and influence of BPN Satisfaction for HCI research, the inconsistent use of measures is problematic in terms of comparability between studies and their replicability.”
From Development and Validation of a Basic Psychological Needs Scale for Technology Use (2024) · cites this paper as Scheel et al. (2021)
The cited source says what the claim attributes to it.
In the citing paper
“If a study fails to apply a valid measure, theoretical inferences cannot be made as the findings could equally measure random noise or a related but nevertheless distinct construct (Scheel et al., 2021).”
From Development and Validation of a Basic Psychological Needs Scale for Technology Use (2024) · cites this paper as Scheel et al., 2021
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The genetical evolution of social behaviour. I
Hamilton, W. D. (1964) · Journal of Theoretical Biology · doi:10.1016/0022-5193(64)90038-4 · Cited by 15,725 · cited 2×
“For example, natural selection favors strategies that maximize inclusive fitness (Hamilton, 1964), which is often not accomplished by self-preservation (e.”
Background(2002) · Cited by 13,211
Causality: Models, Reasoning, and Inference
Pearl, J. (2009) · doi:10.1017/cbo9780511803161 · Cited by 11,653
Estimating the reproducibility of psychological science
Open Science Collaboration (2015) · Science · doi:10.1126/science.aac4716 · Cited by 8,515
Mate selection—A selection for a handicap
Zahavi, A. (1975) · Journal of Theoretical Biology · doi:10.1016/0022-5193(75)90111-3 · Cited by 5,251 · cited 1×
“For example, when Zahavi (1975) proposed the idea that the costliness of signals ensures their reliability (i.e., the handicap principle), many biologists found the idea implausible.”
BackgroundThe genetical evolution of social behaviour. II
Hamilton, W. D. O. (1964) · Journal of Theoretical Biology · doi:10.1016/0022-5193(64)90039-6 · Cited by 4,158 · cited 1×
“For example, natural selection favors strategies that maximize inclusive fitness (Hamilton, 1964), which is often not accomplished by self-preservation (e.g., people investing less in their future health when extrinsic mortality risks are high; Nettle, 2010).”
BackgroundA Next Generation Connectivity Map: L1000 Platform and the First 1,000,000 Profiles
Subramanian, A., Narayan, R., Corsello, S. M., Peck, D. D., Natoli, T. E., Lu, X., Gould, J., Davis, J. F., Tubelli, A. A., Asiedu, J. K., et al. (2017) · Cell · doi:10.1016/j.cell.2017.10.049 · Cited by 3,515
The Methodology of Scientific Research Programmes: Philosophical Papers
Lakatos, I. (1978) · doi:10.1017/cbo9780511621123 · Cited by 3,045
Hempel, C. G. (1966) · Medical Entomology and Zoology · Cited by 2,518
The Causes and Consequences of a Need for Self-Esteem: A Terror Management Theory
Greenberg, J., Pyszczynski, T., & Solomon, S. (1986) · Public Self and Private Self · doi:10.1007/978-1-4613-9564-5_10 · Cited by 2,326 · cited 1×
“For example, terror-management theory (TMT) assumes that humans have an instinct for self-preservation that led to the evolution of an incapacitating fear of death with which humans cope via an anxiety-reducing “terrormanagement” system (Greenberg et al., 1986).”
Background
Flagged first, then by citations
Research works printed but not linked: 7
- Dubin R., 1969, Theory building
- Hernán M., 2020, Causal inference: What if
- Nature Human Behaviour . (2020). Registered reports: Author and reviewer guidelines. https://web.archive.org/web/20200229230434/https://media.nature.com/original/nature-cms/uploads/ckeditor/attachments/4825/RegisteredReportsGuidelines_NatureHumanBehaviour.pdf
- Riesch H. (2008). Scientists’ views of the philosophy of science [Doctoral dissertation, University College London]. https://discovery.ucl.ac.uk/id/eprint/1446063
- Steinnes K. K. (2017). Too cute for words: Cuteness evokes the kama muta emotion and motivates communal sharing [Master’s thesis, University of Oslo]. http://urn.nb.no/URN:NBN:no-60030
- Tinbergen N., 1951, The study of instinct
- Zittermann A., 2014, Anticancer Research, 34, 4641
Cited by
Sorted by citations
Items 101–200 of 381
Personal Space in Human-Robot Interaction at Work: Effect of Room Size and Working Memory Load
Leichtmann, B., Lottermoser, A., Berger, J. L., & Nitsch, V. (2022) · ACM Transactions on Human-Robot Interaction · doi:10.1145/3536167 · Cited by 18
“[42] argued that a major cause of unreliable results is the flexibility in data analysis because a priori hypotheses still had too much room for interpretation, not allowing for precise predictions.”
BackgroundDefining aphasia: Content analysis of six aphasia diagnostic batteries
Castro, N. M., Hula, W. D., & Ashaie, S. A. (2023) · Cortex · doi:10.1016/j.cortex.2023.05.005 · Cited by 19
Subjective Experiences of Committed Meditators Across Practices Aiming for Contentless States
Woods, T. J., Windt, J., Brown, L., Carter, O. L., & Dam, N. T. V. (2023) · Mindfulness · doi:10.1007/s12671-023-02145-0 · Cited by 17
Idiographic Psychology: From Forms of Inference to Levels of Analysis, and Back AgainPreprint
Syed, M. (2022) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/hsp5v · Cited by 17
The replication crisis has led to positive structural, procedural, and community changesPreprint
Korbmacher, M., Azevedo, F., Pennington, C. R., Hartmann, H., Pownall, M., Schmidt, K., Elsherif, M. M., Breznau, N., Robertson, O. M., Kalandadze, T., et al. (2023) · MetArXiv (OSF Preprints) · doi:10.31222/osf.io/r6cvx · Cited by 18
The challenges of the experimental paradigm in narrative identity research
Mansfield, C. D., Pasupathi, M., & McLean, K. C. (2022) · Journal of Research in Personality · doi:10.1016/j.jrp.2022.104318 · Cited by 17
Perspectives on scientific error
van Ravenzwaaij, D., Bakker, M., Heesen, R., Romero, F., van Dongen, N. N. N., Crüwell, S., Field, S. M., Held, L., Munafo, M. R., Pittelkow, M.-M., et al. (2023) · Royal Society Open Science · doi:10.1098/rsos.230448 · Cited by 31
“Thus, the development of theories [31] and a strengthening of the derivation chain from theory to hypothesis test [32] could help to evaluate the relevance of known effects and guide us towards effects we should be looking for.”
BackgroundSelf-citationvan der Burght, C. L., Friederici, A. D., Maran, M., Papitto, G., Pyatigorskaya, E., Schroën, J. A. M., Trettenbrein, P. C., & Zaccarella, E. (2023) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_02058 · Cited by 16
A computational method to reveal psychological constructs from text data.
Herderich, A., Freudenthaler, H. H., & García, D. (2024) · Psychological Methods · doi:10.1037/met0000700 · Cited by 20
Opening up autism research: Bringing open research methods to our field
Hobson, H., Poole, D., Pearson, A., & Fletcher‐Watson, S. (2022) · Autism · doi:10.1177/13623613221105385 · Cited by 16
On the scope of scientific hypotheses
Thompson, W. H., & Skau, S. (2023) · Royal Society Open Science · doi:10.1098/rsos.230607 · Cited by 23
“One suggestion has been to do less confirmatory research [7,20].”
BackgroundRohrer, J. M., Seifert, I. S., Arslan, R. C., Sun, J., & Schmukle, S. C. (2024) · Collabra Psychology · doi:10.1525/collabra.121238 · Cited by 18
Harris, S., Sandal, G. M., Bye, H. H., Palinkas, L. A., & Binder, P. (2021) · Frontiers in Public Health · doi:10.3389/fpubh.2021.777582 · Cited by 22
“In line with calls for more non-confirmatory research, which may facilitate making hypothesis tests more informative (78), we encourage the use of our findings as inspiration and basis for future hypotheses.”
MethodLeichtmann, B., Meneweger, T., Busch, C., Reiterer, B., Meyer, K., Rammer, D., Haring, R., & Mara, M. (2024) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2024.2331878 · Cited by 15
“…approach thus places itself at the point in the research process that contributes to a refinement of theories and hypotheses even before concrete confirmatory tests, for which very precise predictions usually have to exist but are often missing in the literature as Scheel et al. (2021) points out.”
The fundamental importance of method to theory
Dale, R. A. C., Warlaumont, A. S., & Johnson, K. L. (2022) · Nature Reviews Psychology · doi:10.1038/s44159-022-00120-5 · Cited by 14
“Instead, researchers could engage in more nonconfirmatory investigation of the complex relationships among measures and paradigms to facilitate clearer conditions for rigorous, reliable and replicable links between effects and theories (see ref.(151) in particular for discussion).”
BackgroundCompensatory beliefs in the internet gratification behavior: A study of game-based assessment
Yin, B., & Shen, Y. (2023) · Frontiers in Public Health · doi:10.3389/fpubh.2023.997108 · Cited by 14
“In social and behavioral sciences, a theory is constructed to explain certain phenomenon in human behavior, which is bounded by the type of population it concerns, the context where the theory is at play and the methods used to test the hypothesis predicted by the theory (23, 24).”
MethodCausal complexity and psychological measurement
Eronen, M. I. (2024) · Philosophical Psychology · doi:10.1080/09515089.2023.2300693 · Cited by 15
“First, it is widely accepted that most psychological concepts are not well-defined (Flake, 2021; Franz, 2022; Podsakoff et al., 2016; Scheel et al., 2021).”
The Status and Value of Replications in Animal Behavior Science
Brecht, K. F., Legg, E. W., Nawroth, C., Fraser, H. S., & Ostojić, L. (2021) · Animal Behavior and Cognition · doi:10.26451/abc.08.02.01.2021 · Cited by 14
The Function of Literature in Psychological Science
Flis, I. (2022) · Review of General Psychology · doi:10.1177/10892680211066466 · Cited by 15
Reproducibility, Diversity, and the Crisis of Inference in PsychologyPreprint
Syed, M. (2021) · doi:10.31234/osf.io/89buj · Cited by 13
Improving statistical reporting in psychology
Schubert, A., Snijder-Steinhilber, M., Kang, H., & Quintana, D. S. (2025) · Communications Psychology · doi:10.1038/s44271-025-00356-w · Cited by 20
Intentions to purchase a sustainable mobile phone by network analysis in four European countries
Zwicker, M. V., van Harreveld, F., Zickfeld, J., & Brick, C. (2023) · Sustainable Production and Consumption · doi:10.1016/j.spc.2023.07.008 · Cited by 11
Behavioral Labeling: Prompting Consumer Behavior Through Activity Tags
Fritze, M. P., Völckner, F., & Melnyk, V. (2023) · Journal of Marketing · doi:10.1177/00222429231213011 · Cited by 13
“…are novel fi ndings and were tested in various contexts and with different labels, we did not have clear expectations about effect sizes to conduct a priori power analyses across the studies (Scheel et al. 2021), which would have been the preferred option for determining sample sizes (Lakens 2022).”
BackgroundMcLean, K. C. (2024) · doi:10.1093/oso/9780197764640.001.0001 · Cited by 16
Ampartzaki, M., Tassis, K., Kalogiannakis, M., Pavlidou, V., Christidis, K., Chatzoglidou, S., & Eleftherakis, G. (2024) · Education Sciences · doi:10.3390/educsci14060606 · Cited by 13
Fiske, A. P., Schubert, T. W., & Seibt, B. (2024) · Annual Review of Psychology · doi:10.1146/annurev-psych-020124-023338 · Cited by 13
Vivanco-Carlevari, A., Oosterwijk, S., & Kleef, G. A. V. (2024) · Cognition & Emotion · doi:10.1080/02699931.2024.2385691 · Cited by 13
“The initial qualitative phase allowed us to gain insight into the breadth of the phenomenon of interest, develop a preliminary understanding of relevant motives, and develop measures to tap into those motives (Scheel et al., 2021).”
Integrating Computer Prediction Methods in Social Science: A Comment on Hofman et al. (2021)
Breznau, N. (2022) · Social Science Computer Review · doi:10.1177/08944393211049776 · Cited by 12
“Social science gains are already emerging in “many analysts” studies which mimic crowdsourcing competitions of computer scientists but achieve goals of explanation not just developing a better (meta-)algorithm (Botvinik-Nezer et al., 2020; Breznau et al., 2021; Silberzahn et al., 2018).”
BackgroundKrypotos, A., Mertens, G., Klugkist, I. G., & Engelhard, I. M. (2022) · Springer eBooks · doi:10.1007/978-3-031-04968-2_15 · Cited by 13
Strategic ambiguity in the social sciencesPreprint
Frankenhuis, W. E., Panchanathan, K., & Smaldino, P. E. (2022) · doi:10.31222/osf.io/kep5b · Cited by 12
Exploring the meaning of unresolved loss and trauma in more than 1,000 Adult Attachment Interviews
Bakkum, L., Verhage, M. L., Schuengel, C., Duschinsky, R., Cornelisz, I., van Klaveren, C., van IJzendoorn, M. H., Raby, K. L., Roisman, G. I., Bakermans‐Kranenburg, M. J., et al. (2022) · Development and Psychopathology · doi:10.1017/s0954579421001735 · Cited by 14
“Meta-analytic findings have supported these predictions, with effect sizes ranging from .21 to .34 for the associations between parents ’ unresolved states of mind, parents ’ frightening behavior, and children ’ s disorganized attachment (Madigan et al., 2006; Schuengel et al., 1999).”
Büchner, A., Ewert, C., Hoffmann, C. F. A., Schröder–Abé, M., & Horstmann, K. T. (2024) · Journal of Personality · doi:10.1111/jopy.12924 · Cited by 11
“Importantly, the development of accurate measures not only represents the foundation for the empirical study of WP variation but can also enhance the accuracy of the underlying theory (Scheel et al., 2021).”
Jong, B. A. D., Lee, A., Gill, H., & Zheng, X. (2024) · Journal of Organizational Behavior · doi:10.1002/job.2838 · Cited by 15
Spruill, M., & Lewis, N. A. (2022) · Perspectives on Psychological Science · doi:10.1177/17456916221096110 · Cited by 10
Vornhagen, J. B., Bennett, D., Murphy, D., & Mekler, E. D. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3611071 · Cited by 12
“Since both systematic investigation and conceptualization of dis-/empowerment experiences is lacking at present, hypothesis testing would not have been appropriate, and descriptive ground work is called for [70].”
BackgroundDunn, R. C., & Tamminen, K. A. (2023) · International Review of Sport and Exercise Psychology · doi:10.1080/1750984x.2023.2188120 · Cited by 10
“As Scheel et al. (2021, p. 747) suggested, ‘ without knowing conditions in which a phenomenon should occur, it is not possible to evaluate the extent to which observing the phenomenon provides evidence for/against theory ’ .”
Coretta, S., Casillas, J. V., & Roettger, T. B. (2022) · doi:10.31234/osf.io/q8t2k · Cited by 10
Palitsky, R., Chen, Z. J., Rentscher, K. E., Friedman, S. E., Wilson, D. D. M., Ruiz, J. M., Sullivan, D., Grant, G. H., & O'Connor, M. (2023) · Aging Cell · doi:10.1111/acel.14014 · Cited by 10
“Correspondingly, this research was better suited for hypothesis formation rather than testing, as has been recommended elsewhere (Scheel et al., 2021 ).”
MethodWhat is the Replication Crisis a Crisis Of?
Feest, U. (2024) · Philosophy of Science · doi:10.1017/psa.2024.2 · Cited by 9
“Relatedly, Scheel et al. (2021) have pointed out that when psychologists test hypotheses by means of experiments, it is often hard for different researchers to agree on their correct interpretation, because the “ derivation chain ” (Meehl 1990) between theory, hypotheses, and data is underspecified.”
de Rivas, S., Lecuona, O. Z., Castro, Á., & Barrada, J. R. (2022) · Archives of Sexual Behavior · doi:10.1007/s10508-022-02483-5 · Cited by 9
A Metascientific Empirical Review of Cognitive Load Lie Detection
Neequaye, D. A. (2022) · Collabra Psychology · doi:10.1525/collabra.57508 · Cited by 11
Zhao, F., Lim, H., Morrow, E. L., Turkstra, L. S., Duff, M. C., & Mutlu, B. (2022) · Frontiers in Digital Health · doi:10.3389/fdgth.2022.991814 · Cited by 9
“We expect that findings would serve as the foundation for future hypothesis-driven research on technology-based social media interventions for individuals with TBI (59).”
BackgroundCorrespondence: Reward, but do not yet require, interval hypothesis tests
Lakens, D. (2022) · Journal of physiotherapy · doi:10.1016/j.jphys.2022.06.004 · Cited by 9
Self-citationTheorizing as Problem Solving: A Pragmatist Perspective on the Logic of Pursuit
Bhardwaj, A., Sergeeva, A., Mahoney, J. T., & Nickerson, J. A. (2025) · Strategy Science · doi:10.1287/stsc.2023.0075 · Cited by 9
Reflections on Conducting a Large Replication Project in Sports and Exercise Science
Murphy, J., Caldwell, A. R., & Warne, J. P. (2025) · Sports Medicine · doi:10.1007/s40279-025-02200-x · Cited by 9
“…should specify how their independent and dependent variables will be operationalised, how many participants they will collect data on, which exclusion criteria they will apply, which statistical method they will use, and how to decide whether the hypothesis was corroborated or falsified [40].”
InfluentialPriestley, D. R., Staph, J., Koneru, S., Rajtmajer, S. M., Cwiek, A. P., Vervoordt, S. M., & Hillary, F. G. (2022) · Brain Communications · doi:10.1093/braincomms/fcac322 · Cited by 7
“49 Instead, there should be a greater focus on establishing sound relationships between method and theory, as opposed to the universal emphasis on statistical significance testing seen to-day.”
Ten simple rules for socially responsible science
Zivony, A., Kardosh, R., Timmins, L., & Reggev, N. (2023) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1010954 · Cited by 9
The mismatch negativity to abstract relationship of tone pairs is independent of attention
Hsu, Y., Tu, C.-A., Chen, Y., & Liu, H. (2023) · Scientific Reports · doi:10.1038/s41598-023-37131-y · Cited by 8
The Bayesian One-Sample t-Test Supersedes Correlation Analysis as a Test of Validity
Phylactou, P., Παπαδάτου-Παστού, Μ., & Konstantinou, N. (2024) · European Journal of Psychology Open · doi:10.1024/2673-8627/a000069 · Cited by 8
Poor hypotheses and research waste in biology: learning from a theory crisis in psychology
Nakagawa, S., Armitage, D. W., Froese, T., Yang, Y., & Lagisz, M. (2025) · BMC Biology · doi:10.1186/s12915-025-02134-w · Cited by 13
A new beginning of intelligence research. Designing the playground
Protzko, J., & Colom, R. (2021) · Intelligence · doi:10.1016/j.intell.2021.101559 · Cited by 8
Theories and models: What they are, what they are for, and what they are aboutPreprint
Fried, E. I. (2020) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/dt6ev · Cited by 8
“The call by van Rooij & Baggio (this issue) aligns with recent work arguing that hypothesis testers should spend less time testing hypotheses—because coming up with informative hypotheses worth testing requires theoretical work first (Scheel et al., 2020).”
BackgroundGreene, J. A., & Robinson, K. A. (2024) · Educational Psychology Review · doi:10.1007/s10648-024-09971-1 · Cited by 9
“…as the theory is further defined and developed, it can be compared to other theories encompassing similar phenomena, such as by examining the epistemic virtues of each (e.g., the scope of the theory, its fruitfulness and accuracy, how practically useful it is to society; Scheel et al., 2021).”
Won’t Stop ’Til You Get Enough? Determinants of Disengaging From Mobile Media Apps in Daily Life
Ernst, A., & Schnauber-Stockmann, A. (2025) · Communication Research · doi:10.1177/00936502251378582 · Cited by 8
“Such a descriptive and naturalistic approach not only establishes a fundamental basis for theory development and research on disengagement (Scheel et al., 2021) but also offers vital insights into how media use is stopped in situ.”
Reframing oxytocin as a behavioral flexibility hormone
Walle, K. M., Sartorius, A. M., Deilhaug, E., Kang, H., Nærland, T., Andreassen, O. A., Westlye, L. T., & Quintana, D. S. (2026) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2026.106628 · Cited by 10
Technology mediation in child sexual exploitation and abuse in Africa and Asia
Ghai, S., Vuorre, M., Kardefelt‐Winther, D., Ferguson, A. M., Kurten, S., Livingstone, S., Przybylski, A. K., & Orben, A. C. (2026) · Nature · doi:10.1038/s41586-026-10525-4 · Cited by 8
“As such, all findings should be interpreted as identifying plausible associations that warrant future testing 77 through ethically appropriate and feasible methodological approaches.”
Lohse, K. R. (2022) · Physical Therapy · doi:10.1093/ptj/pzac118 · Cited by 7
“If we want to set an even higher standard, we could set H 0 ≤ δ , where δ > 0, but then we need to define what δ should be in a given population 5,10 and determine an adequate study design.”
Shorland, J., Douglas, J. M., & O’Halloran, R. (2023) · International Journal of Speech-Language Pathology · doi:10.1080/17549507.2023.2169352 · Cited by 8
“Given the exploratory nature of this study, descriptive statistics were used (Scheel et al., 2021).”
BackgroundMethodWhen the Going Gets Tough, How Do We Perceive the Future?
Tepper, S. J., & Lewis, N. A. (2021) · Social Cognition · doi:10.1521/soco.2021.39.4.526 · Cited by 7
Formalizing theories of child development: Introduction to the special section
Frankenhuis, W. E., Borsboom, D., Nettle, D., & Roisman, G. I. (2023) · Child Development · doi:10.1111/cdev.14020 · Cited by 7
“However, valuing formal theory does not imply devaluing non-formal theory (Nettle, 2021; Scheel et al., 2021).”
BackgroundRebalancing social & personality psychology methods: The case for naturalistic observation
Robbins, M. L., Jonnalagadda, P., & Spahr, C. M. (2023) · Social and Personality Psychology Compass · doi:10.1111/spc3.12917 · Cited by 6
Rodger, A., Barsalou, L. W., & Papies, E. K. (2024) · Applied Psychology Health and Well-Being · doi:10.1111/aphw.12598 · Cited by 6
“Water drinking is an emerging research domain still in the earlier stages of theory development (e.g. establishing the relationship between concepts; Scheel et al., 2020).”
The effects of contextual diversity on lexical processing: A scoping review
Norman, R. E., Rodd, J. M., & Taylor, J. S. H. (2025) · Psychonomic Bulletin & Review · doi:10.3758/s13423-025-02761-y · Cited by 6
“Conducting this review has enabled us to determine how each subfield of research on contextual diversity contributes to the stages of the ‘research cycle’ (Oberauer & Lewan-dowsky, 2019; Scheel et al., 2021) and to suggest at which stages future research should focus its efforts (Fig.”
McMurray, B., Oleson, J., & Kutlu, E. (2024) · Brain Research · doi:10.1016/j.brainres.2025.149786 · Cited by 6
Decoding the content of working memory in school-aged children
Turoman, N., Fiave, P. A., Zahnd, C., deBettencourt, M. T., & Vergauwe, E. (2023) · Cortex · doi:10.1016/j.cortex.2023.10.019 · Cited by 6
“Such methodological issues can weaken the derivation chain from hypothesis building based on extant theory, through their testing with a given set of methods, to yielding results that are used to build new theory (Meehl, 1990; Scheel et al., 2021).”
A decade of theory as reflected in Psychological Science (2009-2019)Preprint
McPhetres, J., Albayrak‐Aydemir, N., Mendes, A. B., Chow, E. C., Gonzalez-Marquez, P., Loukras, E., Maus, A., O’Mahony, A., Pomareda, C., Primbs, M. A., et al. (2020) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/hs5nx · Cited by 10
Better Inference in Neuroscience: Test Less, Estimate More
Calin-Jageman, R. J. (2022) · Journal of Neuroscience · doi:10.1523/jneurosci.1133-22.2022 · Cited by 6
The Benefits of Preregistration and Registered ReportsPreprint
Lakens, D., Mesquida, C., Rasti, S., & Ditroilo, M. (2024) · doi:10.31234/osf.io/dqap7 · Cited by 6
Self-citationPrivacy Calculus: Theory, studies, and new perspectivesPreprint
Dienlin, T. (2024) · doi:10.31235/osf.io/z3gw8 · Cited by 7
Vitality structures in ‘addictive’ game design
Karhulahti, V. (2024) · Open Research Europe · doi:10.12688/openreseurope.17177.1 · Cited by 5
Survey instructions bias perceptions of environmental health risks
Timmons, S., Papadopoulos, A., & Lunn, P. (2024) · Journal of Risk Research · doi:10.1080/13669877.2024.2421006 · Cited by 5
“The results thus provide inputs necessary to make future confirmatory testing more informative (Scheel et al., 2021).”
Using theoretical constraints and the TASI taxonomy to delineate predictably replicable findings
Fiedler, K., & Trafimow, D. (2024) · Psychonomic Bulletin & Review · doi:10.3758/s13423-024-02521-4 · Cited by 6
“Similar points have been made by Scheel et al. (2021) or by Muthukrishna and Henrich (2019), but leading to clearly different recommendation.”
Robinson, M. M., Williams, J. R., Wixted, J. T., & Brady, T. F. (2024) · Psychonomic Bulletin & Review · doi:10.3758/s13423-024-02562-9 · Cited by 5
“As part of this review, we also integrate and build on recent conceptual and technical discussions of best practices in theory assessment (e.g., Kellen et al., 2021a; Scheel et al., 2021; Zilker, 2022).”
Ronconi, A., Altoè, G., & Masón, L. (2024) · Learning and Individual Differences · doi:10.1016/j.lindif.2024.102604 · Cited by 17
A systematic review of pre-registration in autism research journals
Poole, D., Linden, A. H., Sedgewick, F., Allchin, O., & Hobson, H. (2024) · Autism · doi:10.1177/13623613241308312 · Cited by 7
“Positive results were reported in 44% of psychology/psychiatry studies published between 2013 and 2018, compared with 96% of articles taken from a comparable sample using the standard manuscript format (Scheel et al., 2021).”
The Theory Crisis in Criminology: Causes, Consequences, and Solutions
Ducate, C. S., Bostrom, S. R., Proctor, K. R., & Niemeyer, R. E. (2024) · Theoretical Criminology · doi:10.1177/13624806251372169 · Cited by 9
“Moreover, the reason other disciplines have been successful at developing formal models is because they have a stock of stable, reproducible phenomena; most of psychology does not (Scheel, 2022; Scheel et al., 2021).”
Strengthening derivation chains in cognitive neuroscience: A special issue of Cortex
Mirman, D., Scheel, A. M., Schubert, A., & McIntosh, R. D. (2022) · Cortex · doi:10.1016/j.cortex.2021.12.002 · Cited by 7
Self-citationA Metatheoretical Review of Cognitive Load Lie Detection
Neequaye, D. A. (2023) · Collabra Psychology · doi:10.1525/collabra.87497 · Cited by 6
Riesthuis, P., & Otgaar, H. (2024) · Journal of Criminal Psychology · doi:10.1108/jcp-04-2024-0031 · Cited by 5
Parent-Child Conversations About Inequality: Perpetuating or Disrupting an Inequitable Status Quo
McLean, K. C., Raihl, L. F., LaFever, B., Park, A., Williams, Q. M., & Gonzalez, A. M. (2025) · Journal of Adolescent Research · doi:10.1177/07435584251322693 · Cited by 5
“Finally, our study was qualitative and descriptive, meeting calls for understanding what the phenomena is before moving to the scientific stage of explanation or prediction (Scheel et al., 2021; see also Perry et al., 2021).”
Reconsidering Paul Meehl's disciplinary legacy
Davidson, I. J. (2021) · Social and Personality Psychology Compass · doi:10.1111/spc3.12623 · Cited by 5
Stower, R. E., Tatarian, K., Rudaz, D., Chamoux, M., Chétouani, M., & Kappas, A. (2022) · IEEE RO-MAN · doi:10.1109/ro-man53752.2022.9900782 · Cited by 5
“As these analyses were exploratory, we did not formulate any a-priori hypotheses regarding the association between participant’s attitudes and behaviour [36].”
Keller, E., Marsh, J. E., Richardson, B. H., & Ball, L. J. (2021) · doi:10.31234/osf.io/ntmuy · Cited by 5
“This can be used as a foundation on which to choose or develop suitable theory, aiming to explain the descriptive evidence that has been collected (Eronen & Bringmann, 2021; Scheel et al., 2020).”
MethodSchuengel, C. (2022) · Journal of Child Psychology and Psychiatry · doi:10.1111/jcpp.13577 · Cited by 5
Validating Measures of Mental AbstractionPreprint
Giolla, E. M., Luke, T. J., Calderon, S., & Ask, K. (2022) · doi:10.31234/osf.io/v6xt4 · Cited by 5
Schindler, S., Hilgard, J., Fritsche, I., Burke, B., & Pfattheicher, S. (2022) · doi:10.31234/osf.io/8y6me · Cited by 5
Beyond the bipolar: allowing satisfaction and dissatisfaction to coexist
Souders, D. J., & Yu, Y. (2025) · Frontiers in Psychology · doi:10.3389/fpsyg.2025.1572220 · Cited by 4
Teymoori, A., & Trappes, R. (2026) · Review of General Psychology · doi:10.1177/10892680261421875 · Cited by 7
“Registered reports seem effective in addressing publication bias, whereas 96% of standard studies report signi fi cant results, only about half of registered reports do so (Scheel et al., 2021).”
How does adversity relate to performance across different abilities within individuals?
Young, E. S., Vermeent, S., Frankenhuis, W. E., Nivison, M. D., Simpson, J. A., & Roisman, G. I. (2024) · Development and Psychopathology · doi:10.1017/s0954579424001433 · Cited by 4
Elegant theories and the problems of social psychology
Syed, M. (2025) · European Review of Social Psychology · doi:10.1080/10463283.2025.2494434 · Cited by 4
“It also means recognising when we do not have sufficient rationale to generate strong and reasonable hypotheses (Scheel et al., 2021), and conducting experiments within a systematic process of exploration rather than strict and premature confirmation (Devezer & Buzbas, 2021).”
Brauer, J. R., & Day, J. C. (2025) · Journal of Quantitative Criminology · doi:10.1007/s10940-025-09627-5 · Cited by 5
Strong-Form Frequentist Testing In Communication Science: Principles, Opportunities, And Challenges
Coenen, L., & Smits, T. (2022) · Communication Methods and Measures · doi:10.1080/19312458.2022.2086690 · Cited by 4
A Differentiated Field of Affordances: Redefining Perceptual Skill in Physical Education Teachers
McDonic, L., van der Kamp, J., & van Maarseveen, M. J. J. (2024) · Ecological Psychology · doi:10.1080/10407413.2024.2432645 · Cited by 4
“Ultimately, high-powered randomised controlled trials are essential for confirming which interventions are effective, which produce the largest effects, and which are most cost-effective (Lakens et al., 2018; Scheel et al., 2021).”
deJonge, M. L., Sabiston, C. M., Hamza, C. A., & Darnell, S. C. (2025) · Psychology of sport and exercise · doi:10.1016/j.psychsport.2025.102846 · Cited by 6
Riesthuis, P., Rassin, E., Bücken, C. A., Booker, A., Chin, J. M., Goldfarb, D., Deferme, D., & Otgaar, H. (2025) · The International Journal of Evidence & Proof · doi:10.1177/13657127251357630 · Cited by 4
“We recruited as many legal professionals as possible and because the research was exploratory, we did not conduct an a priori power analysis to estimate a sample size for inferential testing (Scheel et al., 2021).”
How open science can benefit bilingualism research: A lesson in six tales
Ben, R. D., Brouillard, M., Gonzalez‐Barrero, A. M., Killam, H., Kremin, L. V., Quirk, E., Sander‐Montant, A., Schott, E., Tsui, R. K. Y., & Byers‐Heinlein, K. (2022) · Bilingualism Language and Cognition · doi:10.1017/s1366728922000256 · Cited by 4
“Cutting-edge research pushes the frontiers of the unknown and provides more complete scientific explanations of our world (Scheel, Tiokhin, Isager & Lakens, 2020).”
BackgroundCombining statistics: the role of phonotactics on cross-situational word learning
Ben, R. D., Souza, D. D. H., & Hay, J. F. (2022) · Psicologia Reflexão e Crítica · doi:10.1186/s41155-022-00234-y · Cited by 4
“Finally, given the exploratory nature of our investigation, we do not report p-values for our Frequentist analyses (Scheel et al., 2020).”
MethodAI-Enhanced Hypothesis Development and Research Questions
Srivastava, A. P., & Shetye, S. K. (2024) · Advances in educational technologies and instructional design book series · doi:10.4018/979-8-3693-1798-3.ch007 · Cited by 4
Registered Reports: a process to safeguard high-quality evidence
Scheel, A. M. (2020) · Quality of Life Research · doi:10.1007/s11136-020-02698-w · Cited by 3
Self-citationNeuroscience Needs to Test Both Statistical and Scientific Hypotheses
Alger, B. E. (2022) · Journal of Neuroscience · doi:10.1523/jneurosci.1134-22.2022 · Cited by 5
Metadata
- Authors
- Scheel, A. M., Tiokhin, L., Isager, P. M., & Lakens, D.
- Affiliations
- Eindhoven University of Technology (NL)
- Corresponding author
- Scheel, Anne M.
- Venue
- Perspectives on Psychological Science
- Publisher
- SAGE Publications
- Published
- 2020-12-16
- Volume
- 16
- Issue
- 4
- Pages
- 744-755
- ISSN
- 1745-6916 (ISSN-L)
- 1745-6924
- PMC
- PMC8273364
- OpenAlex
- W3113132966
- Cited by
- 382
- References
- 95
- Citations by year
- 2026: 32 · 2025: 48 · 2024: 82 · 2023: 69 · 2022: 80 · 2021: 59 · 2020: 12
- FWCI
- 143.27
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- Perspectives on Psychological Science (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- TU/e Research Portal (publishedVersion, cc-by)
- PsyArXiv (OSF Preprints) (submittedVersion, cc-by)
- PsyArXiv (OSF Preprints) (submittedVersion)
- TU/e Research Portal (submittedVersion, cc-by)
- Funders
- Nederlandse Organisatie voor Wetenschappelijk Onderzoek (VIDI grant 452-17-013)
- Topics
- Social Sciences › Psychology › General Psychology › Academic and Historical Perspectives in Psychology
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Keywords
- null hypothesis significance testing, hypothesis testing, replication crisis, Paul Meehl, theory testing, measurement validity, auxiliary hypotheses, boundary conditions
- MeSH terms
- Humans, Psychological Theory, Psychology, Causality
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
