Outlier Removal and the Relation with Reporting Errors and Quality of Psychological Research
Open accessAbstract
BACKGROUND: The removal of outliers to acquire a significant result is a questionable research practice that appears to be commonly used in psychology. In this study, we investigated whether the removal of outliers in psychology papers is related to weaker evidence (against the null hypothesis of no effect), a higher prevalence of reporting errors, and smaller sample sizes in these papers compared to papers in the same journals that did not report the exclusion of outliers from the analyses. METHODS AND FINDINGS: We retrieved a total of 2667 statistical results of null hypothesis significance tests from 153 articles in main psychology journals, and compared results from articles in which outliers were removed (N = 92) with results from articles that reported no exclusion of outliers (N = 61). We preregistered our hypotheses and methods and analyzed the data at the level of articles. Results show no significant difference between the two types of articles in median p value, sample sizes, or prevalence of all reporting errors, large reporting errors, and reporting errors that concerned the statistical significance. However, we did find a discrepancy between the reported degrees of freedom of t tests and the reported sample size in 41% of articles that did not report removal of any data values. This suggests common failure to report data exclusions (or missingness) in psychological articles. CONCLUSIONS: We failed to find that the removal of outliers from the analysis in psychological articles was related to weaker evidence (against the null hypothesis of no effect), sample size, or the prevalence of errors. However, our control sample might be contaminated due to nondisclosure of excluded values in articles that did not report exclusion of outliers. Results therefore highlight the importance of more transparent reporting of statistical analyses.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Many reported p-values do not match their accompanying test statistic and degrees of freedom.
“However, there is evidence that many reported p-values do not match their accompanying test statistic and degrees of freedom (Bakker & Wicherts, 2011; Bakker & Wicherts, 2014; Berle & Starcevic, 2007; Caperos & Pardo, 2013; Garcia-Berthou & Alcaraz, 2004; Veldkamp, Nuijten, Dominguez-Alvarez, Van Assen, & Wicherts, 2014; Wicherts, Bakker, & Molenaar, 2011).”
From The prevalence of statistical reporting errors in psychology (1985–2013) (2016) · cites this paper as Bakker & Wicherts, 2014
Bakker and Wicherts (2014) only included completely reported t and F values.
“Bakker and Wicherts (2014) only included completely reported t and F values.”
From The prevalence of statistical reporting errors in psychology (1985–2013) (2016) · cites this paper as Bakker and Wicherts (2014)
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Statistical Methods for Research Workers
Fisher, R. A. (1945) · American Journal of Clinical Pathology · doi:10.1093/ajcp/15.11.552b · Cited by 10,883 · see citing works
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567 · see citing works
Publication Manual of the American Psychological Association
DeCleene, K. E., & Fogo, J. (2011) · Occupational Therapy In Health Care · doi:10.3109/07380577.2011.629024 · Cited by 3,177 · see citing works
Measuring the Prevalence of Questionable Research Practices With Incentives for Truth Telling
John, L. K., Loewenstein, G., & Prelec, D. (2012) · Psychological Science · doi:10.1177/0956797611430953 · Cited by 2,239 · see citing works
P-curve: A key to the file-drawer.
Simonsohn, U., Nelson, L. D., & Simmons, J. P. (2014) · Journal of Experimental Psychology: General · doi:10.1037/a0033242 · Cited by 1,500 · see citing works
Pashler, H., & Wagenmakers, E. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691612465253 · Cited by 1,321 · see citing works
An Agenda for Purely Confirmatory Research
Wagenmakers, E.-J., Wetzels, R., Borsboom, D., van der Maas, H. L. J., & Kievit, R. A. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691612463078 · Cited by 1,042 · see citing works
The power of outliers (and why researchers should ALWAYS check for them)
Osborne, J. W., & Overbay, A. (2020) · ScholarWorks@UMassAmherst (University of Massachusetts Amherst) · doi:10.7275/qf69-7k43 · Cited by 898 · see citing works
Sharing Detailed Research Data Is Associated with Increased Citation Rate
Piwowar, H. A., Day, R. S., & Fridsma, D. B. (2007) · PLoS ONE · doi:10.1371/journal.pone.0000308 · Cited by 881 · see citing works
Recommendations for Increasing Replicability in Psychology
Asendorpf, J. B., Conner, M. T., Fruyt, F. D., Houwer, J. D., Denissen, J. J. A., Fiedler, K., Fiedler, S., Funder, D. C., Kliegl, R. M., Nosek, B. A., et al. (2013) · European Journal of Personality · doi:10.1002/per.1919 · Cited by 818 · see citing works
Self-citationThe rules of the game called psychological science
Bakker, M., van Dijk, A., & Wicherts, J. M. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691612459060 · Cited by 800 · see citing works
Self-citationSimmons, J. P., Nelson, L. D., & Simonsohn, U. (2012) · doi:10.2139/ssrn.2160588 · Cited by 680 · see citing works
The poor availability of psychological research data for reanalysis.
Wicherts, J. M., Borsboom, D., Kats, J., & Molenaar, D. (2006) · American Psychologist · doi:10.1037/0003-066x.61.7.726 · Cited by 543 · see citing works
Self-citationStatistical Methods for Research Workers.
Jowett, G. H., & Fisher, R. A. (1956) · Journal of the Royal Statistical Society Series C (Applied Statistics) · doi:10.2307/2985747 · Cited by 517 · see citing works
Data Withholding in Academic Genetics
Campbell, E. G., Clarridge, B. R., Gokhale, M., Birenbaum, L., Hilgartner, S., Holtzman, N. A., & Blumenthal, D. (2002) · JAMA · doi:10.1001/jama.287.4.473 · Cited by 490 · see citing works
The Availability of Research Data Declines Rapidly with Article Age
Vines, T. H., Albert, A. Y. K., Andrew, R. L., Débarre, F., Bock, D. G., Franklin, M. T., Gilbert, K. J., Moore, J.-S., Renaut, S., & Rennison, D. J. (2014) · Current Biology · doi:10.1016/j.cub.2013.11.014 · Cited by 482 · see citing works
Wicherts, J. M., Bakker, M., & Molenaar, D. (2011) · PLoS ONE · doi:10.1371/journal.pone.0026828 · Cited by 424 · see citing works
Self-citationEmpirical Study of Data Sharing by Authors Publishing in PLoS Journals
Savage, C. J., & Vickers, A. J. (2009) · PLoS ONE · doi:10.1371/journal.pone.0007078 · Cited by 411 · see citing works
The (mis)reporting of statistical results in psychology journals
Bakker, M., & Wicherts, J. M. (2011) · Behavior Research Methods · doi:10.3758/s13428-011-0089-5 · Cited by 356 · see citing works
Self-citationEich, E. (2013) · Psychological Science · doi:10.1177/0956797613512465 · Cited by 258 · see citing works
Meta‐research: The art of getting it wrong
Ioannidis, J. P. A. (2010) · Research Synthesis Methods · doi:10.1002/jrsm.19 · Cited by 231 · see citing works
Selective Reporting Biases in Cancer Prognostic Factor Studies
Kyzas, P., Loizou, K. T., & Ioannidis, J. P. A. (2005) · JNCI Journal of the National Cancer Institute · doi:10.1093/jnci/dji184 · Cited by 231 · see citing works
Wolins, L. (1962) · American Psychologist · doi:10.1037/h0038819 · Cited by 228 · see citing works
Modern Statistics for the Social and Behavioral Sciences: A Practical Introduction
Wilcox, R. R. (2011) · Cited by 220 · see citing works
Verifying the Solution from a Nonlinear Solver: A Case Study
McCullough, B. D., & Vinod, H. D. (2003) · American Economic Review · doi:10.1257/000282803322157133 · Cited by 220 · see citing works
Bakker, M., & Wicherts, J. M. (2014) · Psychological Methods · doi:10.1037/met0000014 · Cited by 208 · see citing works
Self-citationReplication Standards for Quantitative Social Science
Jeremy Freese, A. (2007) · Sociological Methods & Research · doi:10.1177/0049124107306659 · Cited by 192 · see citing works
LeBel, E. P., Borsboom, D., Giner‐Sorolla, R., Hasselman, F., Peters, K. R., Ratliff, K. A., & Smith, C. T. (2013) · Perspectives on Psychological Science · doi:10.1177/1745691613491437 · Cited by 103 · see citing works
The Life of <i>p</i> : “Just Significant” Results are on the Rise
Leggett, N. C., Thomas, N. A., Loetscher, T., & Nicholls, M. E. R. (2013) · Quarterly Journal of Experimental Psychology · doi:10.1080/17470218.2013.863371 · Cited by 64 · see citing works
Data Sharing in Medical Research: An Empirical Investigation
Reidpath, D. D., & Allotey, P. A. (2001) · Bioethics · doi:10.1111/1467-8519.00220 · Cited by 63 · see citing works
Modern Statistics for the Social and Behavioral Sciences
Wilcox, R. (2011) · doi:10.1201/9781466503236 · Cited by 62 · see citing works
(2014) · Data Archiving and Networked Services (DANS) · Cited by 46 · see citing works
Consistency errors in p-values reported in Spanish psychology journals
Caperos, J., & Pardo, A. (2013) · Psicothema · doi:10.7334/psicothema2012.207 · Cited by 23 · see citing works
Replication Data Sets and Favored-Hypothesis Bias
Firebaugh, G. (2007) · Sociological Methods & Research · doi:10.1177/0049124107306663 · Cited by 19 · see citing works
How Often are Our Statistics Wrong? a Statistics Class Exercise
Rossi, J. S. (1987) · Teaching of Psychology · doi:10.1207/s15328023top1402_8 · Cited by 15 · see citing works
When ab ≠ c – c′: Published errors in the reports of single-mediator models
Petrocelli, J. V., Clarkson, J. J., Whitmire, M. B., & Moon, P. E. (2012) · Behavior Research Methods · doi:10.3758/s13428-012-0262-5 · Cited by 12 · see citing works
Flagged first, then by citations
1 of the 37 references couldn’t be matched to a known paper.
Research works printed but not linked: 2
- [28]Molenaar D, Wicherts JM, Borsboom D (2005) Participant loss in psychological research and the problem of exclusion: A validity issue. [Bachelor's thesis]. Amsterdam: University of Amsterdam.
- [30]Wicherts JM (2008) Expectancy bias, statistical blunders, and the importance of reanalysis. Invited talk at the 23 IOPS summer conference, June 19, Enschede, The Netherlands.
Cited by
Sorted by citations
The prevalence of statistical reporting errors in psychology (1985–2013)
Nuijten, M. B., Hartgerink, C. H. J., van Assen, M. A. L. M., Epskamp, S., & Wicherts, J. M. (2016) · Behavior Research Methods · doi:10.3758/s13428-015-0664-2 · Cited by 510 · see citing works
“…there is evidence that many reported p -values do not match their accompanying test statistic and degrees of freedom (Bakker & Wicherts, 2011; Bakker & Wicherts, 2014; Berle & Starcevic, 2007; Caperos & Pardo, 2013; Garcia-Berthou & Alcaraz, 2004; Veldkamp, Nuijten, Dominguez-Alvarez, Van…”
Self-citationSample Size, Statistical Power, and False Conclusions in Infant Looking‐Time Research
Oakes, L. M. (2017) · Infancy · doi:10.1111/infa.12186 · Cited by 248 · see citing works
van Aert, R. C. M., Wicherts, J. M., & van Assen, M. A. L. M. (2016) · Perspectives on Psychological Science · doi:10.1177/1745691616650874 · Cited by 219 · see citing works
Self-citationEditorial: Evidence on Questionable Research Practices: The Good, the Bad, and the Ugly
Banks, G. C., Rogelberg, S. G., Woznyj, H. M., Landis, R. S., & Rupp, D. E. (2016) · Journal of Business and Psychology · doi:10.1007/s10869-016-9456-7 · Cited by 176 · see citing works
Questions About Questionable Research Practices in the Field of Management: A Guest Commentary
Banks, G. C., O’Boyle, E. H., Pollack, J. M., White, C. D., Batchelor, J. H., Whelpley, C. E., Abston, K. A., Bennett, A. A., & Adkins, C. L. (2016) · Journal of Management · doi:10.1177/0149206315619011 · Cited by 163 · see citing works
Answers to 18 Questions About Open Science Practices
Banks, G. C., Field, J. G., Oswald, F. L., O’Boyle, E. H., Landis, R. S., Rupp, D. E., & Rogelberg, S. G. (2018) · Journal of Business and Psychology · doi:10.1007/s10869-018-9547-8 · Cited by 151 · see citing works
Closing the divide: Exploring higher education teachers’ perspectives on educational technology
Al‐Adwan, A. S., Meet, R. K., Kala, D., Smedley, J., Urbaníková, M., & Al-Rahmi, W. M. (2024) · Information Development · doi:10.1177/02666669241279181 · Cited by 145 · see citing works
Big little lies: a compendium and simulation of p -hacking strategies
Stefan, A. M., & Schönbrodt, F. D. (2023) · Royal Society Open Science · doi:10.1098/rsos.220346 · Cited by 149 · see citing works
Giofrè, D., Cumming, G., Fresc, L., Boedker, I., & Tressoldi, P. (2017) · PLOS ONE · doi:10.1371/journal.pone.0175583 · Cited by 98 · see citing works
Journal Data Sharing Policies and Statistical Reporting Inconsistencies in Psychology
Nuijten, M. B., Borghuis, J., Veldkamp, C. L. S., Dominguez-Alvarez, L., van Assen, M. A. L. M., & Wicherts, J. M. (2017) · Collabra: Psychology · doi:10.1525/collabra.102 · Cited by 79 · see citing works
Self-citationStatistical Reporting Errors and Collaboration on Statistical Analyses in Psychological Science
Veldkamp, C. L. S., Nuijten, M. B., Dominguez-Alvarez, L., van Assen, M. A. L. M., & Wicherts, J. M. (2014) · PLoS ONE · doi:10.1371/journal.pone.0114876 · Cited by 69 · see citing works
Self-citationCheatham, C. L., & Sheppard, K. W. (2015) · Nutrients · doi:10.3390/nu7115452 · Cited by 72 · see citing works
Pickett, J. T., & Roche, S. P. (2016) · Science and Engineering Ethics · doi:10.1007/s11948-017-9886-2 · Cited by 72 · see citing works
Matthes, J., Marquart, F., Naderer, B., Arendt, F., Schmuck, D., & Adam, K. (2015) · Communication Methods and Measures · doi:10.1080/19312458.2015.1096334 · Cited by 66 · see citing works
Artner, R., Verliefde, T., Steegen, S., Gomes, S. L. B. T., Traets, F., Tuerlinckx, F., & Vanpaemel, W. (2020) · Psychological Methods · doi:10.1037/met0000365 · Cited by 63 · see citing works
Douglas, I. P., Murnane, E. L., Bencharit, L. Z., Altaf, B., Costa, J. M. D. R., Yang, J., Ackerson, M., Srivastava, C., Cooper, M., Douglas, K., et al. (2022) · Building and Environment · doi:10.1016/j.buildenv.2022.109516 · Cited by 53 · see citing works
Hertenstein, E., Nissen, C., Riemann, D. W., Feige, B., Baglioni, C., & Spiegelhalder, K. (2015) · Journal of Sleep Research · doi:10.1111/jsr.12293 · Cited by 49 · see citing works
Predictors of post-traumatic growth in stroke survivors
Kelly, G., Morris, R., & Shetty, H. (2018) · Disability and Rehabilitation · doi:10.1080/09638288.2017.1363300 · Cited by 43 · see citing works
Vermeulen, I., Beukeboom, C. J., Batenburg, A., Avrămiea, A.-E., Stoyanov, D. S., van de Velde, B., & Oegema, D. (2015) · Communication Methods and Measures · doi:10.1080/19312458.2015.1096333 · Cited by 41 · see citing works
Scientific Misconduct in Psychology
Stricker, J., & Günther, A. (2019) · Zeitschrift für Psychologie · doi:10.1027/2151-2604/a000356 · Cited by 36 · see citing works
Bilewicz, M., Skrodzka, M., Olko, J., & Lewińska, T. (2021) · International Journal of Intercultural Relations · doi:10.1016/j.ijintrel.2021.06.009 · Cited by 34 · see citing works
The Validity of the Tool “statcheck” in Discovering Statistical Reporting InconsistenciesPreprint
Nuijten, M. B., van Assen, M. A. L. M., Hartgerink, C. H. J., Epskamp, S., & Wicherts, J. M. (2017) · doi:10.31234/osf.io/tcxaj · Cited by 34 · see citing works
Self-citationBig Little Lies: A Compendium and Simulation of p-Hacking StrategiesPreprint
Stefan, A. M., & Schönbrodt, F. D. (2022) · doi:10.31234/osf.io/xy2dk · Cited by 35 · see citing works
Verify original results through reanalysis before replicating
Nuijten, M. B., Bakker, M., Maassen, E., & Wicherts, J. M. (2018) · Behavioral and Brain Sciences · doi:10.1017/s0140525x18000791 · Cited by 30 · see citing works
Self-citationGreen‐Fulgham, S. M., Ball, J. B., Kwilasz, A. J., Harland, M. E., Frank, M. G., Dragavon, J. M., Grace, P. M., & Watkins, L. R. (2023) · Brain Behavior and Immunity · doi:10.1016/j.bbi.2023.10.004 · Cited by 32 · see citing works
A multi-method psychometric assessment of the affinity for technology interaction (ATI) scale
Lezhnina, O., & Kismihók, G. (2020) · Computers in Human Behavior Reports · doi:10.1016/j.chbr.2020.100004 · Cited by 28 · see citing works
Brake, H. J. V. D., van der Vegt, G. S., & Essens, P. J. M. D. (2023) · Journal of Applied Psychology · doi:10.1037/apl0001168 · Cited by 27 · see citing works
Agenis-Nevers, M., Wang, Y., Dugachard, M., Salvazet, R., Becker, G., & Chenu, D. (2021) · Energy and Buildings · doi:10.1016/j.enbuild.2021.111183 · Cited by 27 · see citing works
Statistical reporting inconsistencies in experimental philosophy
Colombo, M., Duev, G., Nuijten, M. B., & Sprenger, J. (2018) · PLoS ONE · doi:10.1371/journal.pone.0194360 · Cited by 25 · see citing works
Linder, C., & Farahbakhsh, S. (2020) · Business Ethics Quarterly · doi:10.1017/beq.2019.52 · Cited by 23 · see citing works
Aizenstros, A., Bakker, D., Hofmann, S. G., Curtiss, J. E., & Kazantzis, N. (2020) · Behavioural and Cognitive Psychotherapy · doi:10.1017/s1352465820000922 · Cited by 21 · see citing works
Re: The Charité cesarean birth: a family-orientated approach of cesarean section
Armbrust, R., & Henrich, W. (2016) · The Journal of Maternal-Fetal & Neonatal Medicine · doi:10.3109/14767058.2016.1153028 · Cited by 21 · see citing works
Have psychologists increased reporting of outliers in response to the reproducibility crisis?
Valentine, K. D., Buchanan, E. M., Cunningham, A., Hopke, T. G., Wikowsky, A., & Wilson, H. (2021) · Social and Personality Psychology Compass · doi:10.1111/spc3.12591 · Cited by 20 · see citing works
Comprehensive neural networks for guilty feelings in young adults
Nakagawa, S., Takeuchi, H., Taki, Y., Nouchi, R., Sekiguchi, A., Kotozaki, Y., Miyauchi, C. M., Iizuka, K., Yokoyama, R., Shinada, T., et al. (2014) · NeuroImage · doi:10.1016/j.neuroimage.2014.11.004 · Cited by 20 · see citing works
Bestiary of Questionable Research Practices in Psychology
Nagy, T., Hergert, J., Elsherif, M. M., Wallrich, L., Schmidt, K., Waltzer, T., Payne, J. W., Gjoneska, B., Seetahul, Y., Wang, Y. A., et al. (2025) · Advances in Methods and Practices in Psychological Science · doi:10.1177/25152459251348431 · Cited by 24 · see citing works
Siritzky, E. M., Cox, P., Nadler, S. M., Grady, J. N., Kravitz, D. J., & Mitroff, S. R. (2023) · Cognitive Research Principles and Implications · doi:10.1186/s41235-023-00520-y · Cited by 20 · see citing works
Jones, L. V., Barnett, A. G., & Vagenas, D. (2025) · PLoS ONE · doi:10.1371/journal.pone.0299617 · Cited by 16 · see citing works
A Footprint Extraction and Recognition Algorithm Based on Plantar Pressure
Zhang, C., Pan, S., Qi, Y., & Yang, Y. (2019) · Traitement du signal · doi:10.18280/ts.360506 · Cited by 14 · see citing works
Barakat, S., & Maguire, S. (2022) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph20010119 · Cited by 13 · see citing works
VanPatten, B., & Smith, M. (2018) · Second language Research · doi:10.1177/0267658318785652 · Cited by 13 · see citing works
Welhaf, M. S., Smeekens, B. A., Meier, M. E., Silvia, P. J., Kwapil, T. R., & Kane, M. J. (2020) · Journal of Intelligence · doi:10.3390/jintelligence8020025 · Cited by 12 · see citing works
Nuijten, M. B., Bakker, M., Maassen, E., & Wicherts, J. M. (2018) · doi:10.31234/osf.io/fuzkh · Cited by 12 · see citing works
Self-citationLearning from Outliers and Anomalies
Ruef, M., & Birkhead, C. (2024) · Academy of Management Perspectives · doi:10.5465/amp.2023.0481 · Cited by 11 · see citing works
Are Elementary Preservice Teachers Floating or Sinking in Their Understanding of Buoyancy?
Harrell, P. E., Kirby, B., Subramaniam, K., & Long, C. (2021) · International Journal of Science and Mathematics Education · doi:10.1007/s10763-021-10160-7 · Cited by 10 · see citing works
Neoh, M. J. Y., Carollo, A., Lee, A., & Esposito, G. (2023) · Royal Society Open Science · doi:10.1098/rsos.230677 · Cited by 8 · see citing works
Guan, Y., Qiang, Y., Li, B., Zhang, N., Yang, X. B., & Lu, W. (2025) · Journal of Hydrology · doi:10.1016/j.jhydrol.2025.133662 · Cited by 8 · see citing works
Sasson, D. C., Duan, K., Patel, S. M., Junn, A., & Hsia, H. C. (2023) · Advances in Skin & Wound Care · doi:10.1097/01.asw.0000904464.87749.c5 · Cited by 8 · see citing works
Green‐Fulgham, S. M., Lacagnina, M. J., Willcox, K. F., Li, J., Harland, M. E., Ciena, A. P., Rocha, I. R. C., Ball, J. B., Dreher, R. A., Zuberi, Y. A., et al. (2023) · Brain Behavior and Immunity · doi:10.1016/j.bbi.2023.10.027 · Cited by 7 · see citing works
Respiratory and cardiac interoceptive sensitivity in the first two years of lifePreprint
Tünte, M. R., Hoehl, S., Wunderwald, M., Bullinger, J., Boyadziheva, A., Maister, L., Elsner, B., Manos Tsakiris, A., & Kayhan, E. (2023) · eLife · doi:10.7554/elife.91579 · Cited by 7 · see citing works
When bigger looks better: CLASS results in public Montessori preschool classrooms
Lillard, A. S., LeBoeuf, L., Borgman, C., Мартынова, Е. А., Faria, A.-M., & Manship, K. (2024) · Early Childhood Research Quarterly · doi:10.1016/j.ecresq.2024.10.003 · Cited by 6 · see citing works
THE (NON‐)SIGNIFICANCE OF REPORTING ERRORS IN ECONOMICS: EVIDENCE FROM THREE TOP JOURNALS
Pütz, P., & Bruns, S. B. (2020) · Journal of Economic Surveys · doi:10.1111/joes.12397 · Cited by 6 · see citing works
Development of an Artificial White Spot Lesion Creation Protocol: A Preliminary Study
Seevalingam, R., Yahaya, N., Mohamed, A. M. F. S., & Kumar, H. A. (2024) · Cureus · doi:10.7759/cureus.60226 · Cited by 5 · see citing works
Rossi, M. A., Vermeir, E., Brooks, M., Pierce, M., Pukall, C. F., & Rosen, N. O. (2021) · The Journal of Sexual Medicine · doi:10.1016/j.jsxm.2021.11.004 · Cited by 5 · see citing works
Psychometric Properties of the Turkish Young-Rygh Avoidance Inventory
Soygüt, G., Gülüm, İ. V., & Karaosmanoğlu, H. A. (2021) · Journal of Cognitive Psychotherapy · doi:10.1891/jcpsy-d-19-00024 · Cited by 5 · see citing works
Veldkamp, C. L. S. (2017) · doi:10.31234/osf.io/g8cjq · Cited by 6 · see citing works
Risk as an Explanatory Factor for Researchers’ Inferential Interpretations
Hoekstra, R. (2015) · The Mathematics Enthusiast · doi:10.54870/1551-3440.1338 · Cited by 3 · see citing works
The pursuit of individual ambidexterity by middle managers and their psychological well‐being
Moccia, A., Dawson, D., & Darwish, T. K. (2025) · European Management Review · doi:10.1111/emre.70010 · Cited by 3 · see citing works
Donkey hair cortisol concentrations are associated with carrying heavy load and being beaten at work
Bukhari, S. S. U. H., Li, C. M., Kenéz, Á., Steagall, P. V. M., McElligott, A. G., & Parkes, R. S. V. (2025) · BMC Veterinary Research · doi:10.1186/s12917-025-05021-2 · Cited by 3 · see citing works
Sheanshang, D., White, P. A., & Keeler, D. G. (2021) · Statistical Modelling · doi:10.1177/1471082x211043946 · Cited by 3 · see citing works
Exploring personality traits and mental toughness in early career teachers in England
Taberner, J. E., & MacQuarrie, S. (2025) · British Educational Research Journal · doi:10.1002/berj.70041 · Cited by 3 · see citing works
Inconsistencies in Reportedp-Values in Spanish Journals of Psychology
Montalbán, J. M. C., Olmos, R., & Pardo, A. (2016) · Methodology · doi:10.1027/1614-2241/a000107 · Cited by 3 · see citing works
Relaksasi dan Katarsis Singkat Daring bagi Peningkatan Kebersyukuran dan Penurunan Stres Pengasuhan
Rahmandani, A., & Kahija, Y. F. L. (2022) · Jurnal Psikologi Integratif · doi:10.14421/jpsi.v10i1.2258 · Cited by 3 · see citing works
Pitfalls of Statistical Methods in Traffic Psychology
de Winter, J., & Dodou, D. (2021) · Elsevier eBooks · doi:10.1016/b978-0-08-102671-7.10665-7 · Cited by 3 · see citing works
Siddique, A. B., Shaw, B. P., Dwyer, J. T., Fields, D. A., Fontaine, K. R., Hand, D. J., Schekman, R. W., Alberts, J. R., Locher, J. L., & Allison, D. B. (2024) · Science and Engineering Ethics · doi:10.1007/s11948-024-00517-w · Cited by 2 · see citing works
Can, R. G., Schack, T., & Koester, D. (2022) · Psychological Research · doi:10.1007/s00426-022-01741-4 · Cited by 2 · see citing works
Respiratory and cardiac interoceptive sensitivity in the first two years of life
Tünte, M. R., Hoehl, S., Wunderwald, M., Bullinger, J., Boyadziheva, A., Maister, L., Elsner, B., Manos Tsakiris, A., & Kayhan, E. (2025) · eLife · doi:10.31234/osf.io/vbtzq · Cited by 2 · see citing works
Topalović, D., Tasić, V., Stanković-Petrović, J. S., Vlahović, J., Radenković, M., & Smičiklas, I. D. (2024) · Environmental Monitoring and Assessment · doi:10.1007/s10661-024-13069-0 · Cited by 2 · see citing works
StatBreak : Identifying “Lucky” Data Points Through Genetic Algorithms
Rosenbusch, H., Hilbert, L. P., Evans, A. M., & Zeelenberg, M. (2020) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245920917950 · Cited by 2 · see citing works
Dissertation R.C.M. van AertPreprint
van Aert, R. C. M. (2018) · doi:10.31222/osf.io/eqhjd · Cited by 3 · see citing works
Comparison between Statistical Approaches and Data Mining Algorithms for Outlier Detection
Utami, A. P., Fitrianto, A., & Notodiputro, K. A. (2024) · CAUCHY Jurnal Matematika Murni dan Aplikasi · doi:10.18860/ca.v9i1.25450 · Cited by 1 · see citing works
Finch, W. H. (2024) · Stats · doi:10.3390/stats7030051 · Cited by 1 · see citing works
Diplock, B. D., Ellis, J., Perry, J., Sahgal, A., Soliman, H., Moroney, C., Bilodeau, D., & Selchen, S. (2024) · Cancer Survivorship Research & Care · doi:10.1080/28352610.2024.2407120 · Cited by 1 · see citing works
Outliers (typically) cannot cause type I errors in one-sample/paired t-tests
Wisler, A. A. (2026) · PLoS ONE · doi:10.1371/journal.pone.0341720 · Cited by 1 · see citing works
Sullivan, E. C., Dallison, R., Montgomery, P., & Henderson, L. (2026) · Child and Adolescent Mental Health · doi:10.1111/camh.70079 · Cited by 2 · see citing works
The functional foetal brain: A systematic review and meta-analysis
Loehn, L., Dunn, K., To, M. P. S., & Reid, V. M. (2026) · Developmental Cognitive Neuroscience · doi:10.1016/j.dcn.2026.101708 · Cited by 1 · see citing works
Usuki, K., Ueda, H., Yamaguchí, T., Suzuki, T., & Hamaguchi, T. (2024) · PLoS ONE · doi:10.1371/journal.pone.0294301 · Cited by 2 · see citing works
Francis, C. (2025) · Children and youth services review · doi:10.1016/j.childyouth.2025.108128 · Cited by 1 · see citing works
Research on Research: A Meta-Scientific Study of Problems and Solutions in Psychological SciencePreprint
Nuijten, M. B. (2018) · doi:10.31234/osf.io/qtk7e · Cited by 1 · see citing works
Deception, trust, and credibility:a Gricean exploration
Howard, H. A. (2018) · Cited by 1 · see citing works
Testing a new non-arbitrary system for risk communication
Garvey, M. (2020) · doi:10.22215/etd/2020-14270 · Cited by 1 · see citing works
Respiratory and Cardiac Interoceptive Sensitivity in the First Two Years of LifePreprint
Tünte, M. R., Höhl, S., Wunderwald, M., Bullinger, J., Boyadziheva, A., Maister, L., Elsner, B., Manos Tsakiris, A., & Kayhan, E. (2023) · eLife · doi:10.7554/elife.91579.1 · Cited by 1 · see citing works
Tressoldi, P. E., Cumming, G., Fresc, L., & Giofrr, D. (2016) · SSRN Electronic Journal · doi:10.2139/ssrn.2712603
The underlying psychological processes of peer support in stroke.
Marshall, C. S. (2017) · ORCA Online Research @Cardiff (Cardiff University)
Xu, Y. (2019) · UWSpace (University of Waterloo)
Questionable Research Practices: p ‐Hacking, Replication, and Fraud
Arendt, F. (2020) · The International Encyclopedia of Media Psychology · doi:10.1002/9781119011071.iemp0008
CSR Perception Influence on Philadelphia Young Professionals: Career Attraction and ExpectationsPreprint
Croke, C. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.3974090
de Winter, J., & Dodou, D. (2017) · SpringerBriefs in applied sciences and technology · doi:10.1007/978-3-319-56964-2_3
Statistical reporting errors in economicsPreprint
Bruns, S. B., Herwartz, H., Ioannidis, J. P. A., Islam, C.-G., & Raters, F. H. C. (2023) · doi:10.31222/osf.io/mbx62
Tünte, M. R., Höhl, S., Wunderwald, M., et al. (2023) · eLife · doi:10.7554/elife.91579.1.sa0
Tünte, M. R., Höhl, S., Wunderwald, M., et al. (2023) · eLife · doi:10.7554/elife.91579.1.sa1
Francis, C. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4823268
Statistical Reporting Inconsistencies in Experimental LinguisticsPreprint
Etemady, D., & Roettger, T. B. (2024) · doi:10.31234/osf.io/pwjky
Who is Under the Spotlight in Peer Review Networks
Sun, X., Zhang, J., & Di, Z. (2024) · International Conference on Cloud and Network Computing (ICCNC) · doi:10.1109/iccnc63989.2024.00030
Respiratory and Cardiac Interoceptive Sensitivity in the First Two Years of LifePreprint
Tünte, M. R., Höhl, S., Wunderwald, M., Bullinger, J., Boyadziheva, A., Maister, L., Elsner, B., Manos Tsakiris, A., & Kayhan, E. (2024) · eLife · doi:10.7554/elife.91579.2
Rousseau, S., Avital, N., & Tolpyhina, Y. (2024) · Frontiers in Psychology · doi:10.3389/fpsyg.2024.1419159 · Cited by 1 · see citing works
Chenhao, 夏. X., Xinwei, 王. W., Liang, 孙. S., Bo, 宋. S., Songtao, 范. F., & Yan, 周. Z. (2024) · Infrared and Laser Engineering · doi:10.3788/irla20240204
Respiratory and Cardiac Interoceptive Sensitivity in the First Two Years of LifePreprint
Tünte, M. R., Hoehl, S., Wunderwald, M., Bullinger, J., Boyadziheva, A., Maister, L., Elsner, B., Manos Tsakiris, A., & Kayhan, E. (2025) · eLife · doi:10.7554/elife.91579.3
Episcopio‐Sturgeon, D. J., Steele, Z. T., & Pienaar, E. F. (2026) · Anthrozoös · doi:10.1080/08927936.2026.2614161
- Authors
- Bakker, M., & Wicherts, J. M.
- Affiliations
- University of Amsterdam (NL)
- Tilburg University (NL)
- Corresponding author
- Bakker, Marjan
- Venue
- PLoS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2014-07-29
- Volume
- 9
- Issue
- 7
- Pages
- e103360
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC4114807
- OpenAlex
- W2012571365
- Cited by
- 101
- References
- 37
- Citations by year
- 2026: 5 · 2025: 9 · 2024: 16 · 2023: 13 · 2022: 6 · 2021: 8 · 2020: 9 · 2019: 3 · 2018: 9 · 2017: 8 · 2016: 6 · 2015: 7 · 2014: 2
- FWCI
- 7.51
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- PLoS ONE (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- Research portal (Tilburg University) (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Data Archiving and Networked Services (DANS) (submittedVersion, other-oa)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Nederlandse Organisatie voor Wetenschappelijk Onderzoek (016-125-385)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Social Sciences › Safety Research › Academic integrity and plagiarism
- Keywords
- outlier removal, questionable research practices, error reporting, null hypothesis significance testing, sample size, research quality, psychology journals, data exclusion
- MeSH terms
- Humans, Psychology, Research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
