Small Telescopes: Detectability and the Evaluation of Replication Results
The DOI matches its Crossref record; 52 of the 52 references it cites were found in a registry or an index.
Abstract
This article introduces a new approach for evaluating replication results. It combines effect-size estimation with hypothesis testing, assessing the extent to which the replication results are consistent with an effect size big enough to have been detectable in the original study. The approach is demonstrated by examining replications of three well-known findings. Its benefits include the following: (a) differentiating “unsuccessful” replication attempts (i.e., studies yielding p > .05) that are too noisy from those that actively indicate the effect is undetectably different from zero, (b) “protecting” true findings from underpowered replications, and (c) arriving at intuitively compelling inferences in general and for the revisited replications in particular.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“our replications employed 2.5 times the sample size of the original paper (8).”
From Empirical audit and review and an assessment of evidentiary value in research on the psychological consequences of scarcity (2021) · cites this paper as [8]
In the citing paper
“Selective reporting of measures, treatments, and analyses, and the p-hacking they enable (7) has been identified as an existential threat to behavioral science (8, 9)”
From Empirical audit and review and an assessment of evidentiary value in research on the psychological consequences of scarcity (2021) · cites this paper as [8]
In the citing paper
“This approach limits the sample size required to test for equivalence to 2.5 times the sample size of the original study.”
From Equivalence Tests: A Practical Primer for t Tests, Correlations, and Meta-Analyses (2017) · cites this paper as Simonsohn (2015)
Citations
Cites
Statistical Power Analysis for the Behavioral Sciences
Muller, K. (1989) · Technometrics · doi:10.1080/00401706.1989.10488618 · Cited by 99,167
Statistical power analysis for the behavioral sciences
(1990) · Computers, Environment and Urban Systems · doi:10.1016/0198-9715(90)90050-4 · Cited by 65,269
Statistical Power Analysis for the Behavioral Sciences (2nd ed.).
Lachenbruch, P. A., & Cohen, J. (1989) · Journal of the American Statistical Association · doi:10.2307/2290095 · Cited by 37,395
Statistical Methods for Meta-Analysis
Wasserman, S., Hedges, L. V., & Olkin, I. (1988) · Journal of Educational Statistics · doi:10.2307/1164953 · Cited by 10,326
Power failure: why small sample size undermines the reliability of neuroscience
Button, K. S., Ioannidis, J. P. A., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S. J., & Munafò, M. R. (2013) · Nature Reviews Neuroscience · doi:10.1038/nrn3475 · Cited by 7,641 · cited 1×
“If the original study is powered to 50%, an optimistic scenario (Button et al., 2013; Cohen, 1962; Rossi, 1990; Sedlmeier & Gigerenzer, 1989), the resulting upward bias in effect size would be severe enough that replications with samples sizes set to obtain 80% power, given the reported effects, would on average have…”
Schwarz, N., & Clore, G. L. (1983) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.45.3.513 · Cited by 4,952
Cohen, J. (1994) · American Psychologist · doi:10.1037/0003-066x.49.12.997 · Cited by 4,003
Bayesian t tests for accepting and rejecting the null hypothesis
Rouder, J. N., Speckman, P. L., Sun, D., Morey, R. D., & Iverson, G. (2009) · Psychonomic Bulletin & Review · doi:10.3758/pbr.16.2.225 · Cited by 3,688
The New Statistics: Why and How
Cumming, G. (2014) · Psychological Science · doi:10.1177/0956797613504966 · Cited by 3,200
Experimental Tests of the Endowment Effect and the Coase Theorem
Kahneman, D., Knetsch, J. L., & Thaler, R. H. (1990) · Journal of Political Economy · doi:10.1086/261737 · Cited by 3,194
Research works printed but not linked: 4
- Fisher R. A., 1926, Journal of the Ministry of Agriculture, 33, 503
- Jeffreys H., 1961, Theory of probability
- 10.3102/10769986008002093 · doi:10.3102/10769986008002093 · Paper page
- 10.1037/0003-066X.40.1.73 · doi:10.1037/0003-066x.40.1.73 · Paper page
Cited by
Items 201–300 of 743
Geller, J., Thye, M. D., & Mirman, D. (2019) · NeuroImage · doi:10.1016/j.neuroimage.2019.01.020 · Cited by 29
“Recent recommendations indicate that a strongly powered replication should have a sample twice the size of the original study (Brandt et al., 2015; Simonsohn, 2015).”
Kelter, R. (2021) · BMC Medical Research Methodology · doi:10.1186/s12874-021-01341-7 · Cited by 33
On the selection of endogenous and exogenous signals
McCormick, C. R., Redden, R. S., Hurst, A., & Klein, R. M. (2019) · Royal Society Open Science · doi:10.1098/rsos.190134 · Cited by 26
“Each block consisted of four sets of 16 trials, one for each of the four possible signal delay conditions2 (no signal, 100, 250 and 850 ms), with the order of the sets being 1This satisfies Simonsohn’s [14] criterion of 2.5 times the original sample size (Posner et al. n = 9) to detect an effect the same size or large…”
InfluentialHow many decimals? Rounding descriptive and inferential statistics based on measurement precision
Cousineau, D. (2020) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2020.102362 · Cited by 32
Calderon, S., Giolla, E. M., Ask, K., & Granhag, P. A. (2020) · Journal of Personality and Social Psychology · doi:10.1037/pspa0000214 · Cited by 26
Kuwabara, K., Cao, J., Cho, S., & Ingram, P. L. (2022) · Academy of Management Journal · doi:10.5465/amj.2020.0770 · Cited by 27
Wang, X., & Kobayashi, T. (2020) · Chinese Journal of Communication · doi:10.1080/17544750.2020.1807372 · Cited by 26
“If a single source had been used, there inevitably would have been sampling variations, and the reliability of the findings could not be tested (Simonsohn, 2015).”
Reliable measurement in sport psychology: The case of performance outcome measures
Schweizer, G., Furley, P., Rost, N., & Barth, K. (2020) · Psychology of sport and exercise · doi:10.1016/j.psychsport.2020.101663 · Cited by 29
“What exactly constitutes a successful or unsuccessful replication remains a matter of 152 debate (for an enlightening discussion and one possible olution see Simonsohn, 2015).”
Changing the logic of replication: A case from infant studies
Margoni, F., & Shepperd, M. (2020) · Infant Behavior and Development · doi:10.1016/j.infbeh.2020.101483 · Cited by 22
Moeller, J., Dietrich, J., Neubauer, A. B., Brose, A., Kühnel, J., Dehne, M., Jaehne, M. F., Schmiedek, F., Bellhäuser, H., Malmberg, L., et al. (2022) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/5wsna · Cited by 30
A Framework for Evaluating Reproducibility and Replicability in EconomicsPreprint
Dreber, A., & Johannesson, M. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4458153 · Cited by 34
“The “small telescopes” indicator involves testing if the replication effect size is significantly smaller (at the 5% level in a one-sided test) than a “small effect size” defined as the effect size the original study had 33% power to detect (Simonsohn 2015).”
Psychological reactance as a function of thought versus behavioral control
Ma, A., Tang, S., & Kay, A. C. (2019) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2019.103825 · Cited by 28
“original sample size (Simonsohn, 2015), which meant 200 participants per”
Burns, D., Fox, E. L., Greenstein, M., Olbright, G., & Montgomery, D. A. (2019) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2018.10.001 · Cited by 28
The Replication Crisis and Open Science in Psychology: Methodological Challenges and Developments
Renkewitz, F., & Heene, M. (2019) · Zeitschrift für Psychologie · doi:10.1027/2151-2604/a000389 · Cited by 30
Dekkers, T. J., Huizenga, H. M., Bult, J., Popma, A., & Boyer, B. E. (2020) · European Child & Adolescent Psychiatry · doi:10.1007/s00787-020-01579-4 · Cited by 29
Kim, J. Y., Brockner, J., & Block, C. J. (2022) · Organizational Behavior and Human Decision Processes · doi:10.1016/j.obhdp.2022.104118 · Cited by 25
Lee, E., Kim, H. S., & Joo, M. H. (2022) · Health Communication · doi:10.1080/10410236.2022.2074781 · Cited by 31
“Based on (a) the recommendation that a replication study’s sample size should be at least 2.5 times the original study’s (Simonsohn, 2015) and (b) an a priori statistical power analysis using G*Power 3.1 (Faul et al., 2007) which suggested the sample size of 351 assuming a smaller effect size in…”
InfluentialVećkalov, B., Gligorić, V., & Petrović, M. (2023) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2023.104549 · Cited by 26
Cooperative preferences fluctuate across the menstrual cycle
Anderl, C., Hahn, T., Notebaert, K., Klotz, C., Rutter, B., & Windmann, S. (2015) · Judgment and Decision Making · doi:10.1017/s1930297500005544 · Cited by 30
“For Study 2, we envisaged about 2.5 times as many participants compared to Study 1 to provide acceptable power for replication (Simonsohn, 2015).”
InfluentialNuthmann, A., Vitu, F., Engbert, R., & Kliegl, R. M. (2016) · PLoS ONE · doi:10.1371/journal.pone.0162449 · Cited by 24
“5 times the original sample size (N = 4), which is in agreement with a proposal by Simonsohn [36].”
InfluentialTybur, J. M., Jones, B. C., DeBruine, L. M., Ackerman, J. M., & Fasolt, V. (2020) · Psychological Science · doi:10.1177/0956797620955209 · Cited by 22
“For replication studies, Simonsohn (2015) recommends testing an effect size that an original study had 33% power to detect.”
Is Red Really Romantic? Two Pre-Registered Replications of the Red-Romance Hypothesis
Lehmann, G. K., & Calin-Jageman, R. J. (2017) · Social Psychology · doi:10.1027/1864-9335/a000296 · Cited by 22
Behavioral Priming 2.0: Enter a Dynamical Systems Perspective
Krpan, D. (2017) · Frontiers in Psychology · doi:10.3389/fpsyg.2017.01204 · Cited by 25
Unlearning implicit social biases during sleep: A failure to replicate
Humiston, G. B., & Wamsley, E. J. (2019) · PLoS ONE · doi:10.1371/journal.pone.0211416 · Cited by 23
Gervais, W. M., McKee, S. E., & Malik, S. E. (2020) · Psychological Science · doi:10.1177/0956797620922477 · Cited by 21
“…results in the same direction as the original article, (b) the use of a “smalltelescopes” approach for assessing the detectability of effects (Simonsohn, 2015), and (c) a Bayes factor (BF) analysis comparing a point null hypothesis with an alternative hypothesis based on the original…”
InfluentialKramer, H. J., Goldfarb, D., Tashjian, S. M., & Lagattuta, K. H. (2021) · Cognitive Psychology · doi:10.1016/j.cogpsych.2021.101408 · Cited by 31
Equal Quality for Online and Lab Data: A Direct Comparison from Two Dual-Task Paradigms
Sauter, M., Stefani, M., & Mack, W. (2022) · Open Psychology · doi:10.1515/psych-2022-0003 · Cited by 32
Reviews, expectations, and the experience of stories
Tiede, K. E., & Appel, M. (2019) · Media Psychology · doi:10.1080/15213269.2019.1602055 · Cited by 27
“We determined the sample size a priori, following the guidelines for replication studies by Simonsohn (2015) who recommends a sample size 2.5 times as large as the original study’s sample size.”
InfluentialThe need for open science practices and well-conducted replications in the field of gambling studies
Wohl, M. J. A., Tabri, N., & Zelenski, J. M. (2019) · International Gambling Studies · doi:10.1080/14459795.2019.1672769 · Cited by 23
“Simonsohn (2015) suggested that replication studies should have 2.5 times the original number of participants.”
InfluentialIs there evidence for cross-domain congruency sequence effect? A replication of Kan et al . (2013)
Aczél, B., Kovács, M., Bognar, M., Pálfi, B., Hartanto, A., Onie, S., Tiong, L. E., & Evans, T. R. (2021) · Royal Society Open Science · doi:10.1098/rsos.191353 · Cited by 22
Crawford, J. T., & Ruscio, J. (2021) · Psychological Science · doi:10.1177/0956797620972367 · Cited by 20
Schwarz, N., & Clore, G. L. (2016) · Psychological Science · doi:10.1177/0956797616653102 · Cited by 24
The three R's of scientific integrity: Replicability, reproducibility, and robustness
McIntosh, R. D., & Chambers, C. (2020) · Cortex · doi:10.1016/j.cortex.2020.04.019 · Cited by 24
“…3 See Figure S1 of Chalkia et al (2020) for a graphical timeline of these events. because they must be sufficiently well-powered that null results can be meaningfully interpreted, which often requires a sample two-to-three times larger than that of the original study (Simonsohn, 2015).”
Improving entrepreneurial competencies in the classroom: an extension and in-study replication
Glackin, C., & Phelan, S. E. (2020) · New England journal of entrepreneurship · doi:10.1108/neje-04-2020-0005 · Cited by 36
Effects of Motor Tempo on Frontal Brain Activity: An fNIRS Study
Guérin, S. M. R., Vincent, M., Karageorghis, C. I., & Delevoye, Y. (2021) · NeuroImage · doi:10.1016/j.neuroimage.2020.117597 · Cited by 26
“The small telescopes approach was used to determine the smallest ffect size of interest ( Simonsohn, 2015 , SESOI; i.e., the difference that s considered too small to be meaningful”
Attention Guides the Motor-Timing Strategies in Finger-Tapping Tasks When Moving Fast and Slow
Guérin, S. M. R., Boitout, J., & Delevoye, Y. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2020.574396 · Cited by 22
“The small telescopes approach was used to determine the smallest effect size of interest (SESOI; i.e., the difference that is considered too small to be meaningful; Simonsohn, 2015).”
Wulff, J. N., & Taylor, L. (2024) · Strategic Organization · doi:10.1177/14761270231214429 · Cited by 24
Prototypes and same-gender bias in perceptions of hiring discrimination
Carlsson, R., & Sinclair, S. (2017) · The Journal of Social Psychology · doi:10.1080/00224545.2017.1341374 · Cited by 30
Białek, M., & Fugelsang, J. A. (2019) · Journal of Multilingual and Multicultural Development · doi:10.1080/01434632.2018.1547072 · Cited by 28
“Čavar and Tytus simply used too small a telescope to observe an effect of this size (Simonsohn 2015).”
Resolving the Tension Between Exploration and Confirmation in Preclinical Biomedical Research
Dirnagl, U. (2019) · Handbook of experimental pharmacology · doi:10.1007/164_2019_278 · Cited by 26
“For further details on sample size calculation of replication studies, see Simonsohn (2015).”
Positive Affect and Cognitive Restoration: Investigating the Role of Valence and Arousal
Nealis, L. J., van Allen, Z., & Zelenski, J. M. (2016) · PLoS ONE · doi:10.1371/journal.pone.0147275 · Cited by 24
Todd, A. R., Simpson, A. J., & Cameron, C. D. (2019) · Cognition · doi:10.1016/j.cognition.2019.03.002 · Cited by 24
“…the operating conditions of altercentric interference in L1-VPT versus L2-VPT (Surtees, Samson, et al., 2016, Experiment 1), settling on a target sample of at least 160 participants (i.e., 2.5 times the size of Surtees et al.’s sample of 64; for elaboration on this rationale, see Simonsohn, 2015).”
InfluentialWhat anthropologists can learn from psychologists, and the other way around
Weisman, K., & Luhrmann, T. M. (2020) · Journal of the Royal Anthropological Institute · doi:10.1111/1467-9655.13245 · Cited by 24
Ehrke, F., Bruckmüller, S., & Steffens, M. C. (2020) · European Journal of Social Psychology · doi:10.1002/ejsp.2709 · Cited by 25
Schofield, T. P., Haslam, N., & Butterworth, P. (2019) · Journal of Applied Social Psychology · doi:10.1111/jasp.12616 · Cited by 27
The organizational principles of de-differentiated topographic maps in somatosensory cortex
Liu, P., Chrysidou, A., Doehler, J., Hebart, M. N., Wolbers, T., & Kuehn, E. (2021) · eLife · doi:10.7554/elife.60090 · Cited by 23
Do more attractive women show stronger preferences for male facial masculinity?
Docherty, C., Lee, A. J., Hahn, A. C., DeBruine, L. M., & Jones, B. C. (2020) · Evolution and Human Behavior · doi:10.1016/j.evolhumbehav.2020.05.005 · Cited by 22
“Thus, our sample exceeds the size that Simonsohn (2015) recommends for replications.”
Can concurrent memory load reduce distraction? A replication study and beyond.
de Liaño, B. G.-G., Stablum, F., & Umiltà, C. A. (2015) · Journal of Experimental Psychology General · doi:10.1037/xge0000131 · Cited by 18
Power Poseur: Bodily Expansiveness Does Not Matter in Dyadic Interactions
Cesario, J., & Johnson, D. J. (2017) · Social Psychological and Personality Science · doi:10.1177/1948550617725153 · Cited by 23
Reassessing the impact of descriptive norms on charitable giving
Lindersson, L., Guntell, L., Carlsson, R., & Agerström, J. (2018) · International Journal of Nonprofit and Voluntary Sector Marketing · doi:10.1002/nvsm.1617 · Cited by 21
Against dullness: on what it means to be interesting in communication research
Goyanes, M. (2018) · Information Communication & Society · doi:10.1080/1369118x.2018.1495248 · Cited by 21
“In addition, all legitimate science (under this philosophy of science) must be reliable, valid and replicable to be considered as generalizable (Simonsohn, 2015) and therefore, should be synchronized with a certain level of orthodoxy (Bunge, 2015), showing respect for the norms and values that…”
McKelvie, S. J., & Standing, L. (2018) · Frontiers in Psychology · doi:10.3389/fpsyg.2018.02295 · Cited by 18
“Preferably, the teams can each try to replicate the same paper, which has the advantage of maximizing N, and we note that replications to be adequately powered should use more than the number of participants listed in the target article (Simonsohn, 2015).”
Pollet, T. V., & Saxton, T. K. (2020) · Personality and Social Psychology Bulletin · doi:10.1177/0146167220904512 · Cited by 18
Macpherson, M. C., Fay, N., & Miles, L. K. (2020) · Acta Psychologica · doi:10.1016/j.actpsy.2020.103140 · Cited by 22
Kelter, R. (2022) · British Journal of Mathematical and Statistical Psychology · doi:10.1111/bmsp.12267 · Cited by 22
The 27 Club: Music Lyrics Reflect Psychological Distress
Markowitz, D. M., & Hancock, J. T. (2016) · Communication Reports · doi:10.1080/08934215.2016.1210663 · Cited by 21
Grand Challenges for Personality and Social Psychology: Moving beyond the Replication Crisis
Bardi, A., & Zentner, M. R. (2017) · Frontiers in Psychology · doi:10.3389/fpsyg.2017.02068 · Cited by 20
The Collaborative Roots of Corruption? A Replication of Weisel & Shalvi (2015)
Wouda, J., Bijlstra, G., Frankenhuis, W. E., & Wigboldus, D. H. J. (2017) · Collabra Psychology · doi:10.1525/collabra.97 · Cited by 24
Identification of and Correction for Publication Bias
Andrews, I., & Kasy, M. (2017) · National Bureau of Economic Research · doi:10.3386/w23298 · Cited by 33
Chan, M., & Feldman, G. (2025) · Royal Society Open Science · doi:10.1098/rsos.250290 · Cited by 17
Benefits of power posing: effects on dominance and social sensitivity
Klaschinski, L., Schnabel, K., & Schröder–Abé, M. (2017) · Comprehensive Results in Social Psychology · doi:10.1080/23743603.2016.1248080 · Cited by 23
Like student like manager? Using student subjects in managerial debiasing research
Graf‐Vlachy, L. (2017) · Review of Managerial Science · doi:10.1007/s11846-017-0250-3 · Cited by 18
Rudloff, J. P., Hutmacher, F., & Appel, M. (2022) · Journal of Personality · doi:10.1111/jopy.12706 · Cited by 24
Association between Social Class, Greed, and Unethical Behaviour: A Replication Study
Clerke, A. S., Brown, M. F. D., Forchuk, C. A., & Campbell, L. (2018) · Collabra Psychology · doi:10.1525/collabra.166 · Cited by 16
A Bayes Factor for Replications of ANOVA Results
Harms, C. (2019) · The American Statistician · doi:10.1080/00031305.2018.1518787 · Cited by 24
“Simonsohn (2015) therefore proposed a method to take the power of the original study into account when evaluating a replication study.”
InfluentialExperiment aversion does not appear to generalize
Mažar, N., Elbaek, C. T., & Mitkidis, P. (2023) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2217551120 · Cited by 37
Kedron, P., Bardin, S., Holler, J., Gilman, J., Grady, B. W., Seeley, M. M., Wang, X., & Yang, W. (2023) · Geographical Analysis · doi:10.1111/gean.12370 · Cited by 20
“There is no universally agreed upon criteria to assess whether the results of an original study have been reproduced, and most literature focuses on replication (Verhagen and Wagenmakers 2014; Open Science Collaboration 2015; Simonsohn 2015; Lakens 2017).”
Mixed News about the Bad News Game
Graham, M. E., Skov, B., Gilson, Z. M., Heise, C., Fallow, K. M., Mah, E. Y., & Lindsay, D. S. (2023) · Journal of Cognition · doi:10.5334/joc.324 · Cited by 25
Examining the replicability of online experiments selected by a decision market
Holzmeister, F., Johannesson, M., Camerer, C. F., Chen, Y., Ho, T., Hoogeveen, S., Huber, J., Imai, N., Imai, T., Jin, L. J., et al. (2024) · Nature Human Behaviour · doi:10.1038/s41562-024-02062-9 · Cited by 26
Moery, E., & Calin-Jageman, R. J. (2016) · Social Psychological and Personality Science · doi:10.1177/1948550616639649 · Cited by 16
Does personality change following spousal bereavement?
Chopik, W. J. (2016) · Journal of Research in Personality · doi:10.1016/j.jrp.2016.08.010 · Cited by 17
Meta-ethics and the mortality: Mortality salience leads people to adopt a less subjectivist morality
Yilmaz, O., & Bahçekapılı, H. G. (2018) · Cognition · doi:10.1016/j.cognition.2018.06.014 · Cited by 18
“Thus, in accordance with Simonsohn’s (2015) suggestion that sample sizes of replication studies should be at least two and a half times more than the original sample sizes, we aimed to collect data from at least 315 participants to detect an effect.”
How (not) to measure replication
Fletcher, S. C. (2021) · European Journal for Philosophy of Science · doi:10.1007/s13194-021-00377-2 · Cited by 19
“This can be especially likely when the original’s estimated effect size was small (Simonsohn 2015, p. 561).”
Shielding working memory from distraction is more effortful than flexible updating
Papadopetraki, D., Froböse, M. I., Westbrook, A., Zandbelt, B. B., & Cools, R. (2025) · Scientific Reports · doi:10.1038/s41598-025-96980-x · Cited by 19
Thinking orientation and preference for research methodology
Murshed, F., & Zhang, Y. (2016) · Journal of Consumer Marketing · doi:10.1108/jcm-01-2016-1694 · Cited by 16
Olsen, J., Kogler, C., Brandt, M. J., Dezső, L., & Kirchler, E. M. (2019) · Journal of Economic Psychology · doi:10.1016/j.joep.2019.02.001 · Cited by 16
“…different from zero (p = .012) and not different from an effect that would have been detectable with the original sample size (N = 351); p = .426; Simonsohn (2015), “Small Telescopes” UK: inconclusive; the effect is not different from zero (p = .197) and also not different from an effect that…”
InfluentialHow to Choose between Different Bayesian Posterior Indices for Hypothesis Testing in Practice
Kelter, R. (2021) · Multivariate Behavioral Research · doi:10.1080/00273171.2021.1967716 · Cited by 20
Białek, M., Bergelt, M., Majima, Y., & Koehler, D. J. (2019) · Journal of Neuroscience Psychology and Economics · doi:10.1037/npe0000111 · Cited by 14
A toolbox to evaluate the trustworthiness of published findings
Adler, S. J., Röseler, L., & Schöniger, M. K. (2023) · Journal of Business Research · doi:10.1016/j.jbusres.2023.114189 · Cited by 21
Mehr, S. A., Scannell, D. A., & Winner, E. (2018) · PLoS ONE · doi:10.1371/journal.pone.0202075 · Cited by 14
Confounds in “Failed” Replications
Bressan, P. (2019) · Frontiers in Psychology · doi:10.3389/fpsyg.2019.01884 · Cited by 18
“…(based on the effect size obtained in the original study, a method that leads to inadequate statistical power per se: Maxwell et al., 2015; Simonsohn, 2015) were made pointless by the introduction of random and systematic noise that was absent in the original study—not just regular…”
Ignored visual context does not induce latent learning
Vadillo, M. A., Giménez‐Fernández, T., Aivar, M. P., & Cubillas, C. P. (2020) · Psychonomic Bulletin & Review · doi:10.3758/s13423-020-01722-x · Cited by 24
Günther, V., Hußlack, A., Weil, A.-S., Bujanow, A., Henkelmann, J., Kersting, A., Quirin, M., Hoffmann, K., Egloff, B., Lobsien, D., et al. (2020) · NeuroImage Clinical · doi:10.1016/j.nicl.2020.102441 · Cited by 20
“According to Simonsohn (2015), a replication sample should be 2.5-fold the size of the original sample in order to achieve a reasonable statistical power.”
InfluentialRaymondie, R. A., & Steiner, D. D. (2021) · Journal of Applied Social Psychology · doi:10.1111/jasp.12778 · Cited by 20
Stefan, A. M., Schönbrodt, F. D., Evans, N. J., & Wagenmakers, E. (2022) · Behavior Research Methods · doi:10.3758/s13428-021-01754-8 · Cited by 24
“Note that by definition a SESOI specification of the effect size parameter does not represent a researcher’s best guess, but a lower bound on what the researcher deems realistic or interesting (e.g., Button et al. 2015; Simonsohn 2015; Perugini et al. 2014; Lakens et al. 2018).”
Countering Youth's Negative Stereotypes of Teens Fosters Constructive Behavior
Qu, Y., Pomerantz, E. M., & Wu, G. (2020) · Child Development · doi:10.1111/cdev.13156 · Cited by 20
Want to make me happy? Tell me about your experiences but not your objects
Bastos, W. (2020) · Journal of Consumer Affairs · doi:10.1111/joca.12321 · Cited by 17
Guérin, S. M. R., Vincent, M., & Delevoye, Y. (2022) · Psychophysiology · doi:10.1111/psyp.14226 · Cited by 16
The Consequences of Revealing First-Generational Status
Belmi, P. R. V., Raz, K., Neale, M. A., & Thomas-Hunt, M. C. (2023) · Organization Science · doi:10.1287/orsc.2023.1682 · Cited by 21
Andres, E., Chuan-Peng, H., Gerlicher, A. M. V., Meyer, B., Tüscher, O., & Kalisch, R. (2024) · Nature Communications · doi:10.1038/s41467-024-46936-y · Cited by 15
Biderman, N., Luria, R., Teodorescu, A. R., Hajaj, R., & Goshen‐Gottstein, Y. (2018) · Psychological Science · doi:10.1177/0956797618813538 · Cited by 22
“Following Simonsohn (2015), we predetermined a sample size in all of our experiments (Ns = 24) that was substantially larger than Brady et al.’s (2013).”
InfluentialAKT1 and genetic vulnerability to bipolar disorder
Millischer, V., Matheson, G. J., Martinsson, L., Ek, I. R., Schalling, M., Lavebratt, C., & Backlund, L. (2019) · Psychiatry Research · doi:10.1016/j.psychres.2019.112677 · Cited by 18
Making Research Data Accessible
Kapiszewski, D., & Karcher, S. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108762519.008 · Cited by 16
Straying Off Course: The Negative Impact of Mind Wandering on Fine Motor Movements
da Silva, M. R. D., & Postma, M. (2021) · Journal of Motor Behavior · doi:10.1080/00222895.2021.1937032 · Cited by 17
“However, a variety of factors, including task difficulty (Seli et al., 2018), previous knowledge, level of expertise (Simonsohn, 2015; Smallwood & Andrews-Hanna, 2013), time on task (Thomson et al., 2014), working memory capacity (Randall et al., 2019; Smallwood, 2013), drowsiness (Stawarczyk et…”
Does testing increase spontaneous mediation in learning semantically related paired associates?
Cho, K. W., Neely, J. H., Brennan, M. K., Vitrano, D., & Crocco, S. (2017) · Journal of Experimental Psychology Learning Memory and Cognition · doi:10.1037/xlm0000414 · Cited by 18
Henderson, A., Thoelen, G., Nadler, A., Barraza, J. A., & Nave, G. (2018) · Scientific Reports · doi:10.1038/s41598-018-29928-z · Cited by 13
Revisiting the rigor–relevance relationship: An institutional logics perspective
Paterson, T. A., Harms, P. D., & Tuggle, C. S. (2018) · Human Resource Management · doi:10.1002/hrm.21911 · Cited by 18
Journal-based replication of experiments: An application to “Being Chosen to Lead”
Drazen, A., Dreber, A., Ozbay, E. Y., & Snowberg, E. (2021) · Journal of Public Economics · doi:10.1016/j.jpubeco.2021.104482 · Cited by 20
Does Power Reduce Temporal Discounting? Commentary on Joshi and Fast (2013)
Zhang, M., & Smith, P. K. (2018) · Psychological Science · doi:10.1177/0956797617754219 · Cited by 13
Heyard, R., Pawel, S., Frese, J., Voelkl, B., Würbel, H., McCann, S., Held, L., Wever, K. E., Hartmann, H., Townsin, L., et al. (2025) · Royal Society Open Science · doi:10.1098/rsos.242076 · Cited by 20
Metadata
- Authors
- Simonsohn, U.
- Affiliations
- University of Pennsylvania (US)
- Corresponding author
- Simonsohn, Uri
- ORCID
- Simonsohn, Uri
- Venue
- Psychological Science
- Publisher
- SAGE Publications
- Published
- 2015-03-23
- Volume
- 26
- Issue
- 5
- Pages
- 559-569
- ISSN
- 0956-7976 (ISSN-L)
- 1467-9280
- OpenAlex
- W2151509224
- Cited by
- 742
- References
- 52
- Citations by year
- 2026: 30 · 2025: 58 · 2024: 59 · 2023: 67 · 2022: 64 · 2021: 66 · 2020: 111 · 2019: 79 · 2018: 76 · 2017: 63 · 2016: 48 · 2015: 16
- FWCI
- 61.64
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
- Locations
- Psychological Science (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Keywords
- effect size estimation, hypothesis testing, replication study, statistical power, underpowered studies, small effect sizes
- MeSH terms
- Astronomy, Data Interpretation, Statistical, Research Design, Reproducibility of Results, Models, Statistical, Sample Size, Telescopes
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.