Bias in the evaluation of psychology studies: A comparison of parapsychology versus neuroscience

Abstract

Research suggests that scientists display confirmation biases with regard to the evaluation of research studies, in that they evaluate results as being stronger when a study confirms their prior expectations. These biases may influence the peer review process, particularly for studies that present controversial findings. The purpose of the current study was to compare the evaluation of a parapsychology study versus a neuroscience study. One hundred participants with a background in psychology were randomly assigned to read and evaluate one of two virtually identical study abstracts (50 participants per group). One of the abstracts described the findings as if they were from a parapsychology study, whereas the other abstract described the findings as if they were from a neuroscience study. The results revealed that participants rated the neuroscience abstract as having stronger findings and as being more valid and reliable than the parapsychology abstract, despite the fact that the two abstracts were identical. Participants also displayed confirmation bias in their ratings of the parapsychology abstract, in that their ratings were correlated with their scores on transcendentalism (a measure of beliefs and experiences related to parapsychology, consciousness and reality). Specifically, higher transcendentalism was associated with more favorable ratings of the parapsychology abstract, whereas lower transcendentalism was associated with less favorable ratings. The findings suggest that individuals with a background in psychology need to be vigilant about potential biases that could impact their evaluations of parapsychology research during the peer review process.