Fighting “bad science” in the information age: The effects of an intervention to stimulate evaluation and critique of false scientific claims

Abstract

With developments in technology (e.g., “Web 2.0” sites that allow users to author and create media content) and the removal of publication barriers, the quality of science information online now varies vastly. These changes in the review of published science information, along with increased facility of information distribution, have resulted in the spread of misinformation about science. As such, the role of evaluation when reading scientific claims has become a pressing issue when educating students. While recent studies have examined educational strategies for supporting evaluation of sources and plausibility of claims, there is little extant work on supporting students in critiquing the claims for flawed scientific reasoning. This study tested the efficacy of a structured reading support intervention for evaluation and critique on cultivating a critical awareness of flawed scientific claims in an online setting. We developed and validated a questionnaire to measure epistemic vigilance, implemented a large‐scale (N = 1081) Randomized Controlled Trial (RCT) of an original reading activity that elicits evaluation and critique of scientific claims, and measured whether the intervention increased epistemic vigilance of misinformation. Our RCT results suggested a moderate effect in students who complied with the treatment intervention. Furthermore, analyses of heterogeneous effects suggested that the intervention effects were driven by 11th‐grade students and students who self‐reported a moderate trust in science and medicine. Our findings point to the need for additional opportunities and instruction for students on critiquing scientific claims and the nature of specific errors in scientific reasoning.