The DOI matches its Crossref record; 52 of the 52 references it cites were found in a registry or an index.
Small Telescopes: Detectability and the Evaluation of Replication Results
Abstract
This article introduces a new approach for evaluating replication results. It combines effect-size estimation with hypothesis testing, assessing the extent to which the replication results are consistent with an effect size big enough to have been detectable in the original study. The approach is demonstrated by examining replications of three well-known findings. Its benefits include the following: (a) differentiating “unsuccessful” replication attempts (i.e., studies yielding p > .05) that are too noisy from those that actively indicate the effect is undetectably different from zero, (b) “protecting” true findings from underpowered replications, and (c) arriving at intuitively compelling inferences in general and for the revisited replications in particular.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“our replications employed 2.5 times the sample size of the original paper (8).”
From Empirical audit and review and an assessment of evidentiary value in research on the psychological consequences of scarcity (2021) · cites this paper as [8]
In the citing paper
“Selective reporting of measures, treatments, and analyses, and the p-hacking they enable (7) has been identified as an existential threat to behavioral science (8, 9)”
From Empirical audit and review and an assessment of evidentiary value in research on the psychological consequences of scarcity (2021) · cites this paper as [8]
In the citing paper
“This approach limits the sample size required to test for equivalence to 2.5 times the sample size of the original study.”
From Equivalence Tests: A Practical Primer for t Tests, Correlations, and Meta-Analyses (2017) · cites this paper as Simonsohn (2015)
Citations
Cites
Statistical Power Analysis for the Behavioral Sciences
Muller, K. (1989) · Technometrics · doi:10.1080/00401706.1989.10488618 · Cited by 99,167
Statistical power analysis for the behavioral sciences
(1990) · Computers, Environment and Urban Systems · doi:10.1016/0198-9715(90)90050-4 · Cited by 65,269
Statistical Power Analysis for the Behavioral Sciences (2nd ed.).
Lachenbruch, P. A., & Cohen, J. (1989) · Journal of the American Statistical Association · doi:10.2307/2290095 · Cited by 37,395
Statistical Methods for Meta-Analysis
Wasserman, S., Hedges, L. V., & Olkin, I. (1988) · Journal of Educational Statistics · doi:10.2307/1164953 · Cited by 10,326
Power failure: why small sample size undermines the reliability of neuroscience
Button, K. S., Ioannidis, J. P. A., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S. J., & Munafò, M. R. (2013) · Nature Reviews Neuroscience · doi:10.1038/nrn3475 · Cited by 7,641 · cited 1×
“If the original study is powered to 50%, an optimistic scenario (Button et al., 2013; Cohen, 1962; Rossi, 1990; Sedlmeier & Gigerenzer, 1989), the resulting upward bias in effect size would be severe enough that replications with samples sizes set to obtain 80% power, given the reported effects, would on average have…”
Schwarz, N., & Clore, G. L. (1983) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.45.3.513 · Cited by 4,952
Cohen, J. (1994) · American Psychologist · doi:10.1037/0003-066x.49.12.997 · Cited by 4,003
Bayesian t tests for accepting and rejecting the null hypothesis
Rouder, J. N., Speckman, P. L., Sun, D., Morey, R. D., & Iverson, G. (2009) · Psychonomic Bulletin & Review · doi:10.3758/pbr.16.2.225 · Cited by 3,688
The New Statistics: Why and How
Cumming, G. (2014) · Psychological Science · doi:10.1177/0956797613504966 · Cited by 3,200
Experimental Tests of the Endowment Effect and the Coase Theorem
Kahneman, D., Knetsch, J. L., & Thaler, R. H. (1990) · Journal of Political Economy · doi:10.1086/261737 · Cited by 3,194
Research works printed but not linked: 3
- Fisher R. A., 1926, Journal of the Ministry of Agriculture, 33, 503
- Jeffreys H., 1961, Theory of probability
- 10.1037/0003-066X.40.1.73 · doi:10.1037/0003-066x.40.1.73 · Paper page
Cited by
Estimating the reproducibility of psychological science
Open Science Collaboration (2015) · Science · doi:10.1126/science.aac4716 · Cited by 8,515
Equivalence Tests: A Practical Primer for t Tests, Correlations, and Meta-Analyses
Lakens, D. (2017) · Social Psychological and Personality Science · doi:10.1177/1948550617697177 · Cited by 3,001
Lakens, D. (2022) · Collabra Psychology · doi:10.1525/collabra.33267 · Cited by 1,962
Bayesian inference for psychology. Part I: Theoretical advantages and practical ramifications
Wagenmakers, E.-J., Marsman, M., Jamil, T., Ly, A., Verhagen, J., Love, J., Selker, R., Gronau, Q. F., Šmíra, M., Epskamp, S., et al. (2018) · Psychonomic Bulletin & Review · doi:10.3758/s13423-017-1343-3 · Cited by 1,627
Equivalence Testing for Psychological Research: A Tutorial
Lakens, D., Scheel, A. M., & Isager, P. M. (2018) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245918770963 · Cited by 1,560
“Simonsohn (2015) recently proposed setting the SESOI as the effect size that an earlier study would have had 33% power to detect.”
Metadata
- Authors
- Simonsohn, U.
- Affiliations
- University of Pennsylvania (US)
- Corresponding author
- Simonsohn, Uri
- ORCID
- Simonsohn, Uri
- Venue
- Psychological Science
- Publisher
- SAGE Publications
- Published
- 2015-03-23
- Volume
- 26
- Issue
- 5
- Pages
- 559-569
- ISSN
- 0956-7976 (ISSN-L)
- 1467-9280
- OpenAlex
- W2151509224
- Cited by
- 742
- References
- 52
- Citations by year
- 2026: 30 · 2025: 58 · 2024: 59 · 2023: 67 · 2022: 64 · 2021: 66 · 2020: 111 · 2019: 79 · 2018: 76 · 2017: 63 · 2016: 48 · 2015: 16
- FWCI
- 61.64
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
- Locations
- Psychological Science (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Keywords
- effect size estimation, hypothesis testing, replication study, statistical power, underpowered studies, small effect sizes
- MeSH terms
- Astronomy, Data Interpretation, Statistical, Research Design, Reproducibility of Results, Models, Statistical, Sample Size, Telescopes
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Small Telescopes: Detectability and the Evaluation of Replication Results. https://app.cochat.ai/papers/10.1177/0956797614567341
- The paper
- Simonsohn, U. (2015). Small Telescopes: Detectability and the Evaluation of Replication Results. Psychological Science. https://doi.org/10.1177/0956797614567341
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.