The DOI matches its Crossref record; 25 of the 25 references it cites were found in a registry or an index.
Estimating the Reproducibility of Experimental Philosophy
Open accessOverview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505
Lakens, D. (2013) · Frontiers in Psychology · doi:10.3389/fpsyg.2013.00863 · Cited by 10,303
Estimating the reproducibility of psychological science
Open Science Collaboration (2015) · Science · doi:10.1126/science.aac4716 · Cited by 8,515 · cited 2×
“In 32 studies out of 39 that used participants (excluding Reuter, 2011, that used internet hits as data points), the replicationN was greater than the originalN.”
BackgroundResultPower failure: why small sample size undermines the reliability of neuroscience
Button, K. S., Ioannidis, J. P. A., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S. J., & Munafò, M. R. (2013) · Nature Reviews Neuroscience · doi:10.1038/nrn3475 · Cited by 7,641 · cited 3×
“Because initial effect size estimates in the literature tend to be inflated due to publication bias (Anderson et al. 2017; Button et al. 2013), we elected to use a higher than usual power assumption 9 In this respect, our methodology differed from the OSC’s methodology, which instructed replication…”
BackgroundMethodSimmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · CHANCE · doi:10.1080/09332480.2005.10722754 · Cited by 4,702
1,500 scientists lift the lid on reproducibility
Baker, M. (2016) · Nature · doi:10.1038/533452a · Cited by 4,209
A manifesto for reproducible science
Munafò, M. R., Nosek, B. A., Bishop, D. V. M., Button, K. S., Chambers, C. D., Percie du Sert, N., Simonsohn, U., Wagenmakers, E.-J., Ware, J. J., & Ioannidis, J. P. A. (2017) · Nature Human Behaviour · doi:10.1038/s41562-016-0021 · Cited by 3,397
Redefine statistical significance
Benjamin, D. J., Berger, J. O., Johannesson, M., Nosek, B. A., Wagenmakers, E.-J., Berk, R., Bollen, K. A., Brembs, B., Brown, L., Camerer, C., et al. (2017) · Nature Human Behaviour · doi:10.1038/s41562-017-0189-z · Cited by 2,772
The preregistration revolution
Nosek, B. A., Ebersole, C. R., DeHaven, A. C., & Mellor, D. T. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1708274114 · Cited by 1,998
Casler, K., Bickel, L., & Hackett, E. (2013) · Computers in Human Behavior · doi:10.1016/j.chb.2013.05.009 · Cited by 1,662 · cited 1×
“Thus, it seems that most original findings based on convenience samples such as pedestrians or university students could be generalized to online samples (Casler et al. 2013).”
BackgroundPublication bias in the social sciences: Unlocking the file drawer
Franco, A., Malhotra, N., & Simonovits, G. (2014) · Science · doi:10.1126/science.1255484 · Cited by 1,542 · cited 1×
“As many have argued, the systematic non-publication of null results – which contributes to the so-called filedrawer effect – is a leading factor in increasing the proportion of false positives in the literature and thus of non-replicable effects (Earp 2017; Franco et al. 2014; Rosenthal 1979).”
BackgroundP-curve: A key to the file-drawer.
Simonsohn, U., Nelson, L. D., & Simmons, J. P. (2014) · Journal of Experimental Psychology: General · doi:10.1037/a0033242 · Cited by 1,500 · cited 2×
“…Average replication effect size Content-based 90.3% 0.41 (0.17) 0.39 (0.21) Context-based 25.0% 0.22 (0.08) 0.01 (0.06) Demographic effect 25.0% 0.29 (0.10) 0.12 (0.10) (Simonsohn et al. 2014), whereas the p-curve for our target set of x-phi findings revealed a substantial rightward skew (see Fig.”
BackgroundInfluentialCognitive load selectively interferes with utilitarian moral judgment
Greene, J. D., Morelli, S. A., Lowenberg, K., Nystrom, L. E., & Cohen, J. D. (2008) · Cognition · doi:10.1016/j.cognition.2007.11.004 · Cited by 1,231 · cited 2×
“One ‘difficulty’ point was awarded to studies that were not simple surveys that could have potentially been run online (for example, studies that involved a cognitive load task and as such required an in-lab setting, such as Greene et al., 2008).”
BackgroundShall we Really do it Again? The Powerful Concept of Replication is Neglected in the Social Sciences
Schmidt, S. (2009) · Review of General Psychology · doi:10.1037/a0015108 · Cited by 1,125
Replications in Psychology Research: How Often Do They Really Occur?
Makel, M. C., Plucker, J. A., & Hegarty, B. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691612460688 · Cited by 750
Facts Are More Important Than Novelty: Replication in the Education Sciences
Makel, M. C., & Plucker, J. A. (2014) · Educational Researcher · doi:10.3102/0013189x14545513 · Cited by 609 · cited 1×
“…rate1 should be for a given field in light of trade-offs between, e.g., innovation and confirmation (Gilbert, King, Pettigrew, & Wilson, 2016; Makel & Plucker, 2014), many scientists regard the estimates that have been generated—less than 50% in each of the above cases—as worryingly low.”
BackgroundComment on “Estimating the reproducibility of psychological science.”
Gilbert, D. T., King, G., Pettigrew, S., & Wilson, T. D. (2016) · Science · doi:10.1126/science.aad7243 · Cited by 474 · cited 2×
“, innovation and confirmation (Gilbert et al. 2016; Makel & Plucker, 2014), many scientists regard the estimates that have been generated—less than 50% in each of the above cases—as worryingly low.”
BackgroundFraley, R. C., & Vazire, S. (2014) · PLoS ONE · doi:10.1371/journal.pone.0109019 · Cited by 409
Anderson, S. F., Kelley, K., & Maxwell, S. E. (2017) · Psychological Science · doi:10.1177/0956797617723724 · Cited by 406
Priming, Replication, and the Hardest Science
Cesario, J. (2014) · Perspectives on Psychological Science · doi:10.1177/1745691613513470 · Cited by 320
Questionable Research Practices Revisited
Fiedler, K., & Schwarz, N. (2016) · Social Psychological and Personality Science · doi:10.1177/1948550615612150 · Cited by 279 · cited 1×
“This in turn could promote so-called Questionable Research Practices (Fiedler and Schwarz 2016; John et al. 2012) which may increase the likelihood of committing a Type 1 error (Simmons et al. 2011).”
BackgroundNosek, B. A., & Errington, T. M. (2017) · eLife · doi:10.7554/elife.23383 · Cited by 242 · cited 1×
“…reproducibility of psychological science (Open Science Collaboration, 2015), the Reproducibility Project’s analogous initiative for cancer biology (Nosek & Errington, 2017), meta-scientist John Ioannidis’s modeling efforts in biomedicine and beyond (e.g., Ioannidis, 2005) and a 2015 estimate…”
BackgroundCullen, S. (2010) · Review of Philosophy and Psychology · doi:10.1007/s13164-009-0016-1 · Cited by 225 · cited 2×
“…field, it has been subjected to criticism from skeptical quarters, including the claim that it is little more than bad psychology—an upstart enterprise run by philosophers who mimic the methods of behavioral science without fully mastering or even understanding them (Cullen 2010; Woolfolk 2013).”
BackgroundIs Economics Research Replicable? Sixty Published Papers from Thirteen Journals Say 'Usually Not'
Chang, A. C., & Li, P. (2015) · doi:10.2139/ssrn.2669564 · Cited by 217 · cited 1×
“…initiative for cancer biology (Nosek and Errington 2017), meta-scientist John Ioannidis’s modeling efforts in biomedicine and beyond (e.g., Ioannidis 2005) and a 2015 estimate produced by the Board of Governors of the Federal Reserve System for research in economics (Chang and Li 2015).”
Background
Cited by
Replicability, Robustness, and Reproducibility in Psychological Science
Nosek, B. A., Hardwicke, T. E., Moshontz, H., Allard, A., Corker, K. S., Dreber, A., Fidler, F., Hilgard, J., Kline Struhl, M., Nuijten, M. B., et al. (2022) · Annual Review of Psychology · doi:10.1146/annurev-psych-020821-114157 · Cited by 770
Self-citationSoto, C. J. (2019) · Psychological Science · doi:10.1177/0956797619831612 · Cited by 470
Challenges for assessing replicability in preclinical cancer biology
Errington, T. M., Denis, A., Perfito, N., Iorns, E., & Nosek, B. A. (2021) · eLife · doi:10.7554/elife.67995 · Cited by 267
Many Labs 5: Testing Pre-Data-Collection Peer Review as an Intervention to Increase Replicability
Ebersole, C. R., Mathur, M. B., Baranski, E., Bart-Plange, D.-J., Buttrick, N. R., Chartier, C. R., Corker, K. S., Corley, M., Hartshorne, J. K., IJzerman, H., et al. (2020) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245920958687 · Cited by 105
Byrd, N., & Conway, P. (2019) · Cognition · doi:10.1016/j.cognition.2019.06.007 · Cited by 77
Metadata
- Authors
- Cova, F., Strickland, B., Abatista, A., Allard, A., Andow, J., Attie, M., Beebe, J., Berniūnas, R., Boudesseul, J., Colombo, M., Cushman, F., Diaz, R., N’Djaye Nikolai van Dongen, N., Dranseika, V., Earp, B. D., Torres, A. G., Hannikainen, I., Hernández-Conde, J. V., Hu, W., Jaquet, F., Khalifa, K., Kim, H., Kneer, M., Knobe, J., Kurthy, M., Lantian, A., Liao, S.-Y., Machery, E., Moerenhout, T., Mott, C., Phelan, M., Phillips, J., Rambharose, N., Reuter, K., Romero, F., Sousa, P., Sprenger, J., Thalabard, E., Tobia, K., Viciana, H., Wilkenfeld, D., & Zhou, X.
- Venue
- Review of Philosophy and Psychology
- Published
- 2018-06-14
- Volume
- 12
- Issue
- 1
- Pages
- 9-44
- ISSN
- 1878-5158 (ISSN-L)
- 1878-5166
- OpenAlex
- W2797413227
- Cited by
- 179
- References
- 25
- FWCI
- 23.14
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.