Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat
PreprintCited by 14

The DOI matches its Crossref record; 1 of the 1 references it cites were found in a registry or an index.

Gamified Inoculation Interventions Do Not Improve Discrimination Between True and Fake News: Reanalyzing Existing Research With Receiver Operating Characteristic Analysis

Modirrousta-Galian, Ariana, Higham, Philip Anthony · 2022

Preprint — published as 10.1037/xge0001395

Abstract

Gamified inoculation interventions designed to improve detection of online misinformation are becoming increasingly prevalent. Two of the most notable interventions of this kind are Bad News and Go Viral!. To assess their efficacy, prior research has typically used pre-post designs in which participants rated the reliability or manipulativeness of true and fake news items before and after playing these games, while most of the time also including a control group who played an irrelevant game (Tetris) or did nothing at all. Mean ratings were then compared between pre-tests and post-tests and/or between the control and experimental conditions. Critically, these prior studies have not separated response bias effects (overall tendency to respond “true” or “fake”) from discrimination (ability to distinguish between true and fake news, commonly dubbed discernment). We reanalyzed the results from five prior studies using receiver operating characteristic (ROC) curves, a method common to signal detection theory (SDT) that allows for discrimination to be measured free from response bias. Across the studies, when comparable true and fake news items were used, Bad News and Go Viral! did not improve discrimination, but rather elicited more “false” responses to all news items (more conservative responding). These novel findings suggest that the current gamified inoculation interventions designed to improve fake news detection are not as effective as previously thought and may even be counterproductive. They also demonstrate the usefulness of ROC analysis, a largely unexploited method in this setting, for assessing the effectiveness of any intervention designed to improve fake news detection.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.31234/osf.io/4bgkd

Cites

1 of 1 references verified0 citation errors · 0 not checkable

Statistics

Figures

Figure 1 Equal-Variance Gaussian Distributions of True and Fake News Items Distributed Over a Dimension of Subjective Truth as Conceptualized by Signal Detection Theory
Figure 1 Equal-Variance Gaussian Distributions of True and Fake News Items Distributed Over a Dimension of Subjective Truth as Conceptualized by Signal Detection TheoryCC-BY · From the PDF
Note . ROC = Receiver operating characteristic. These ROC curves are obtained from equal-variance Gaussian distributions.
Note . ROC = Receiver operating characteristic. These ROC curves are obtained from equal-variance Gaussian distributions.CC-BY · From the PDF
Note . ROC = Receiver operating characteristic. These ROC curves are obtained from equal-variance Gaussian distributions, and discrimination was kept constant across all three scenarios. Although it appears as though the liberal figure has five points and the unbiased figure has six points, in fact all figures have seven points; three points in the upper-right portion of the liberal figure are overlapping, and two points in the upper-right portion of the unbiased figure are overlapping.
Note . ROC = Receiver operating characteristic. These ROC curves are obtained from equal-variance Gaussian distributions, and discrimination was kept constant across all three scenarios. Although it appears as though the liberal figure has five points and the unbiased figure has six points, in fact all figures have seven points; three points in the upper-right portion of the liberal figure are overlapping, and two points in the upper-right portion of the unbiased figure are overlapping.CC-BY · From the PDF
Figure 4 ROC Curves for the Pre-Test and Post-Test in Roozenbeek and van der Linden (2019)
Figure 4 ROC Curves for the Pre-Test and Post-Test in Roozenbeek and van der Linden (2019)CC-BY · From the PDF
Figure 5 ROC Curves for the Pre-Test and Post-Test in Basol et al.'s (2020) Treatment Condition and Control Condition
Figure 5 ROC Curves for the Pre-Test and Post-Test in Basol et al.'s (2020) Treatment Condition and Control ConditionCC-BY · From the PDF
Image 6
CC-BY · From the PDF
Figure 6 Design of Roozenbeek et al.'s (2020) Experiment 1 and the Pre-Post Differences of Interest
Figure 6 Design of Roozenbeek et al.'s (2020) Experiment 1 and the Pre-Post Differences of InterestCC-BY · From the PDF
Figure 7 Design of Roozenbeek et al.'s (2020) Experiment 2 and the Differences of Interest
Figure 7 Design of Roozenbeek et al.'s (2020) Experiment 2 and the Differences of InterestCC-BY · From the PDF
Figure 8 ROC Curves for the Pre-Test and Post-Test in Roozenbeek et al.'s (2020) Set A-A and Set B-B From Experiment 1 and Control Condition From Experiment 2
Figure 8 ROC Curves for the Pre-Test and Post-Test in Roozenbeek et al.'s (2020) Set A-A and Set B-B From Experiment 1 and Control Condition From Experiment 2CC-BY · From the PDF
Note . ROC = Receiver operating characteristic. T = time. T1 = just before playing Bad News or Tetris in Experiments 1 and 2. T2 = just after playing Bad News or Tetris in Experiments 1 and 2. T3 = 1 week after playing Bad News or Tetris in Experiment 1 and 9 weeks after playing Bad News or Tetris in Experiment 2. T4 = 5 weeks after playing Bad News or Tetris in Experiment 1. T5 = 13 weeks after playing Bad News or Tetris in Experiment 1.
Note . ROC = Receiver operating characteristic. T = time. T1 = just before playing Bad News or Tetris in Experiments 1 and 2. T2 = just after playing Bad News or Tetris in Experiments 1 and 2. T3 = 1 week after playing Bad News or Tetris in Experiment 1 and 9 weeks after playing Bad News or Tetris in Experiment 2. T4 = 5 weeks after playing Bad News or Tetris in Experiment 1. T5 = 13 weeks after playing Bad News or Tetris in Experiment 1.CC-BY · From the PDF
Note . ROC = Receiver operating characteristic. The follow-up test took place 1 week after playing Go Viral! or Tetris in Study 2.
Note . ROC = Receiver operating characteristic. The follow-up test took place 1 week after playing Go Viral! or Tetris in Study 2.CC-BY · From the PDF
Image 12
CC-BY · From the PDF
Image 13
CC-BY · From the PDF
Image 14
CC-BY · From the PDF
Image 15
CC-BY · From the PDF
Image 16
CC-BY · From the PDF
Image 17
CC-BY · From the PDF
Image 18
CC-BY · From the PDF

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 1 references

Research works printed but not linked: 75

  1. Adjin-Tettey, T. D. (2022). Combating fake news, disinformation, and misinformation: Experimental evidence for media literacy education. Cogent Arts & Humanities, 9(1), Article 2037229. https://doi.org/10.1080/23311983.2022.2037229 · DOI · Paper page
  2. Afolabi, A. A., & Ilesanmi, O. S. (2021). Dealing with vaccine hesitancy in Africa: The prospective COVID-19 vaccine context. The Pan African Medical Journal, 38, Article 3. https://doi.org/10.11604%2Fpamj.2021.38.3.27401 · DOI
  3. Aleci, C. (2021). Measuring the soul. EDP Sciences. https://doi.org/10.1051/978-2-7598- 2518-9 · doi:10.1051/978-2-7598-2518-9
  4. Allen, J., Howland, B., Mobius, M., Rothschild, D., & Watts, D. J. (2020). Evaluating the fake news problem at the scale of the information ecosystem. Science Advances, 6(14), Article eaay3539. https://doi.org/10.1126/sciadv.aay3539 · DOI · Paper page
  5. Banas, J. A., & Rains, S. A. (2010). A meta-analysis of research on inoculation theory. Communication Monographs, 77(3), 281-311. http://dx.doi.org/10.1080/03637751003758193 · DOI · Paper page
  6. Basol, M., Roozenbeek, J., Berriche, M., Uenal, F., McClanahan, W. P., & van der Linden, S. (2021). Towards psychological herd immunity: Cross-cultural evidence for two prebunking interventions against COVID-19 misinformation. Big Data & Society, 8(1), 1-18. https://doi.org/10.1177%2F20539517211013868 · DOI · Paper page
  7. Basol, M., Roozenbeek, J., & van der Linden, S. (2020). Good news about bad news: Gamified inoculation boosts confidence and cognitive immunity against fake news. Journal of Cognition, 3(1), Article 2. https://doi.org/10.5334/joc.91 · DOI · Paper page
  8. Batailler, C., Brannon, S. M., Teas, P. E., & Gawronski, B. (2022). A signal detection approach to understanding the identification of fake news. Perspectives on Psychological Science, 17(1), 79-98. https://doi.org/10.1177/1745691620986135 · DOI · Paper page
  9. Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2010). A basic introduction to fixed-effect and random-effects models for meta-analysis. Research Synthesis Methods, 1(2), 97-111. https://doi.org/10.1002/jrsm. · DOI · Paper page
  10. Brady, T. F., Robinson, M. M., Williams, J. R., & Wixted, J. T. (2022). Measuring memory is harder than you think: How to avoid problematic measurement practices in memory research. Psychonomic Bulletin & Review. https://doi.org/10.3758/s13423-022- 02179-w · DOI
  11. Brashier, N. M., Pennycook, G., Berinsky, A. J., & Rand, D. G. (2021). Timing matters when correcting fake news. Proceedings of the National Academy of Sciences, 118(5), Article e2020043118. https://doi.org/10.1073/pnas.2020043118 · DOI · Paper page
  12. Calvillo, D. P., Rutchick, A. M., & Garcia, R. J. B. (2021). Individual differences in belief in fake news about election fraud after the 2020 U.S. election. Behavioral Sciences, 11(12), 175. https://doi.org/10.3390/bs11120175 · DOI
  13. Clayton, K., Blair, S., Busam, J. A., Forstner, S., Glance, J., Green, G., Kawata, A., Kovvuri, A., Martin, J., Morgan, E., Sandhu, M., Sang, R., Scholz-Bright, R., Welch, A. T., Wolff, A. G., Zhou, A., & Nyhan, B. (2020). Real solutions for fake news? Measuring the effectiveness of general warnings and fact-check tags in reducing belief in false stories on social media. Political Behavior, 42(4), 1073-1095. https://doi.org/10.1007/s11109-019-09533-0 · DOI · Paper page
  14. Correll, J., Park, B., Judd, C. M., & Wittenbrink, B. (2002). The police officer's dilemma: Using ethnicity to disambiguate potentially threatening individuals. Journal of Personality and Social Psychology, 83(6), 1314-1329. https://doi.org/10.1037/0022- 3514.83.6.1314 · DOI · Paper page
  15. Corrigan, P. W. (2016). Lessons learned from unintended consequences about erasing the stigma of mental illness. World Psychiatry, 15(1), 67-73. https://doi.org/10.1002%2Fwps.20295 · DOI · Paper page
  16. de Gardelle, V., & Kouider, S. (2009). Cognitive theories of consciousness. In W. P. Banks (Ed.), Encyclopedia of consciousness (pp. 135-146). Elsevier. https://doi.org/10.1016/B978-012373873-8.00077-3 two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics, 44(3), 837-845. https://doi.org/10.2307/2531595 · DOI · Paper page
  17. Donaldson, W. (1992). Measuring recognition memory. Journal of Experimental Psychology: General, 121(3), 275-277. https://doi.org/10.1037/0096-3445.121.3.275 · DOI
  18. Donaldson, W., & Good, C. (1996). A′r: An estimate of area under isosensitivity curves. Behavior Research Methods, Instruments & Computers, 28(4), 590-597. https://doi.org/10.3758/BF03200547 · DOI
  19. Eriksen, C. W. (1954). The case for perceptual defense. Psychological Review, 61(3), 175- 182. https://doi.org/10.1037/h0058094 · DOI
  20. Glassner, B. (2004). Narrative techniques of fear mongering. Social Research: An International Quarterly, 71(4), 819-826. https://doi.org/10.1353/sor.2004.0001 · DOI
  21. Grace, L., & Hone, B. (2019). Factitious: large scale computer game to fight fake news and improve news literacy. Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems, 1-8. https://doi.org/10.1145/3290607.3299046 · doi:10.1145/3290607.3299046
  22. Grady, R. H., Ditto, P. H., & Loftus, E. F. (2021). Nevertheless, partisanship persisted: Fake news warnings help briefly, but bias returns with time. Cognitive Research: Principles and Implications, 6(1), Article 52. https://doi.org/10.1186/s41235-021-00315-z · DOI
  23. Guay, B., Berinsky, A. J., Pennycook, G., & Rand, D. (2022). How to think about whether misinformation interventions work. PsyArxiv. https://doi.org/10.31234/osf.io/gv8qx · doi:10.31234/osf.io/gv8qx
  24. Hart, P. S., Chinn, S., & Soroka, S. (2020). Politicization and polarization in COVID-19 news coverage. Science Communication, 42(5), 679-697. https://doi.org/10.1177%2F1075547020950735 · DOI · Paper page
  25. Heit, E., & Rotello, C. M. (2014). Traditional difference-score analyses of reasoning are flawed. Cognition, 131(1), 75-91. https://doi.org/10.1016/j.cognition.2013.12.003 · DOI
  26. Higham, P. A., & Higham, D. P. (2018). New improved gamma: Enhancing the accuracy of Goodman-Kruskal's gamma using ROC curves. Behavior Research Methods, 51(1), 108-125. https://doi.org/10.3758/s13428-018-1125-5 · DOI
  27. Higham, P. A., Zawadzka, K., & Hanczakowski, M. (2016). Internal mapping and its impact on measures of absolute and relative metacognitive accuracy. In J. Dunlosky & S. K. Tauber (Eds.), The Oxford handbook of metamemory (pp. 39-61). Oxford University Press. http://dx.doi.org/10.1093/oxfordhb/9780199336746.013. · DOI
  28. Howes, D. H., & Solomon, R. L. (1950). A note on McGinnies' 'Emotionality and perceptual defense.' Psychological Review, 57(4), 229-234. https://doi.org/10.1037/h0060881 · DOI
  29. Huotari, K., & Hamari, J. (2016). A definition for gamification: Anchoring gamification in the service marketing literature. Electronic Markets, 27(1), 21-31. https://doi.org/10.1007/s12525-015-0212-z · DOI · Paper page
  30. Iyengar, A., Gupta, P., & Priya, N. (2022). Inoculation against conspiracy theories: A consumer side approach to India's fake news problem. Applied Cognitive Psychology. Advance online publication. https://doi.org/10.1002/acp.3995 · DOI
  31. Jeffreys, H. (1961). Theory of probability (3 rd ed.). Oxford University Press.
  32. Jhangiani, R. S., Chiang, I.-C. A., Cuttler, C., & Leighton, D. C. (2019). Research methods in psychology (4 th ed.). Kwantlen Polytechnic University. https://doi.org/10.17605/OSF.IO/HF7DQ · doi:10.17605/osf.io/hf7dq
  33. Johnson, H. M., & Seifert, C. M. (1994). Sources of the continued influence effect: When misinformation in memory affects later inferences. Journal of Experimental Psychology: Learning, Memory, and Cognition, 20(6), 1420-1436. https://doi.org/10.1037/0278-7393.20.6.1420 · DOI · Paper page
  34. Kanozia, R., Kaur, S., & Arya, R. (2021). Infodemic during the COVID-19 lockdown in India. Media Asia, 48(1), 58-66. https://doi.org/10.1080/01296612.2021.1881286 · DOI
  35. Klatzky, R. L., & Erdelyi, M. H. (1985). The response criterion problem in tests of hypnosis and memory. International Journal of Clinical and Experimental Hypnosis, 33(3), 246-257. https://doi.org/10.1080/00207148508406653 · DOI
  36. Lazer, D. M. J., Baum, M. A., Benkler, Y., Berinsky, A. J., Greenhill, K. M., Menczer, F., Metzger, M. J., Nyhan, B., Pennycook, G., Rothschild, D., Schudson, M., Sloman, S. A., Sunstein, C. R., Thorson, E. A., Watts, D. J., & Zittrain, J. L. (2018). The science of fake news. Science, 359(6380), 1094-1096. https://doi.org/10.1126/science.aao2998 · DOI · Paper page
  37. Lee, M. D., & Wagenmakers, E.-J. (2013). Bayesian cognitive modelling: A practical course. Cambridge University Press. https://psycnet.apa.org/doi/10.1017/CBO9781139087759 · DOI · Paper page
  38. Lewandowsky, S., Ecker, U. K. H., Seifert, C. M., Schwarz, N., & Cook, J. (2012). Misinformation and its correction: continued influence and successful debiasing. Psychological Science in the Public Interest, 13(3), 106-131. https://doi.org/10.1177%2F1529100612451018 · DOI · Paper page
  39. Maertens, R., Roozenbeek, J., Basol, M., & van der Linden, S. (2021). Long-term effectiveness of inoculation against misinformation: three longitudinal experiments. Journal of Experimental Psychology: Applied, 27(1), 1-16. https://psycnet.apa.org/doi/10.1037/xap0000315 · DOI · Paper page
  40. Mandrekar, J. N. (2010). Receiver operating characteristic curve in diagnostic test assessment. Journal of Thoracic Oncology, 5(9), 1315-1316. https://doi.org/10.1097/jto.0b013e3181ec173d · DOI · Paper page
  41. McGinnies, E. (1949). Emotionality and perceptual defense. Psychological Review, 56(5), 244-251. https://doi.org/10.1037/h0056508 · DOI · Paper page
  42. McGuire, W. J. (1964). Some contemporary approaches. In L. Berkowitz (Ed.), Advances in Experimental Social Psychology (Vol. 1, pp. 191-229). Academic Press. https://doi.org/10.1016/S0065-2601(08)60052-0 · DOI · Paper page
  43. Memon, A., & Higham, P. A. (1999). A review of the cognitive interview. Psychology, Crime and Law, 5(1-2), 177-196. http://dx.doi.org/10.1080/10683169908415000 · DOI
  44. Messori, A., Damuzzo, V., Agnoletto, L., Leonardi, L., Chiumente, M., & Mengato, D. (2019). A model-independent method to determine restricted mean survival time in the analysis of survival curves. SN Comprehensive Clinical Medicine, 2(1), 66-68. https://doi.org/10.1007/s42399-019-00199-7 · doi:10.1007/s42399-019-00199-7
  45. Micallef, N., Avram, M., Menczer, F., & Patil, S. (2021). Fakey: a game intervention to improve news literacy on social media. Proceedings of the ACM on Human- Computer Interaction, 5(CSCW1), Article 6. https://doi.org/10.1145/3449080 · DOI
  46. Mickes, L., Flowe, H. D., & Wixted, J. T. (2012). Receiver operating characteristic analysis of eyewitness memory: Comparing the diagnostic accuracy of simultaneous versus sequential lineups. Journal of Experimental Psychology: Applied, 18(4), 361-376. https://doi.org/10.1037/a0030609 · DOI · Paper page
  47. Modirrousta-Galian, A., & Higham, P. A. (2022, December 21). Gamified Inoculation Interventions Do Not Improve Discrimination Between True and Fake News: Reanalyzing Existing Research With Receiver Operating Characteristic Analysis. https://doi.org/10.17605/OSF.IO/85BE7 · doi:10.17605/osf.io/85be7
  48. Modirrousta-Galian, A., Higham, P. A., & Seabrooke, T. (in press). Effects of inductive learning and gamification on news veracity discernment. Journal of Experimental Psychology: Applied.
  49. Morschheuser, B., Hamari, J., & Koivisto, J. (2016). Gamification in crowdsourcing: a review. Proceedings of the 49 th Annual Hawaii International Conference on System Sciences, 4375-4384. https://doi.org/10.1109/HICSS.2016.543 · DOI · Paper page
  50. Mukhtar, S. (2021). Psychology and politics of COVID-19 misinfodemics: Why and how do people believe in misinfodemics? International Sociology, 36(1), 111-123. https://doi.org/10.1177/0268580920948807 · DOI
  51. Mumpower, J. L., & McClelland, G. H. (2014). A signal detection theory analysis of racial and ethnic disproportionality in the referral and substantiation processes of the U.S. child welfare services system. Judgment and Decision Making, 9(2), 114-128. https://journal.sjdm.org/13/13422/jdm13422.html
  52. Pennycook, G., Cannon, T. D., & Rand, D. G. (2018). Prior exposure increases perceived accuracy of fake news. Journal of Experimental Psychology: General, 147(12), 1865-1880. https://doi.org/10.1037/xge0000465 · DOI · Paper page
  53. Pennycook, G., & Rand, D. G. (2019). Cognitive reflection and the 2016 U.S. presidential election. Personality and Social Psychology Bulletin, 45(2), 224-239. https://doi.org/10.1177/0146167218783192 · DOI
  54. Pertwee, E., Simas, C., & Larson, H. J. (2022). An epidemic of uncertainty: rumors, conspiracy theories and vaccine hesitancy. Nature Medicine, 28(3), 456-459. https://doi.org/10.1038/s41591-022-01728-z · DOI · Paper page
  55. Phelan, J. C., Cruz-Rojas, R., & Reiff, M. (2011). Genes and stigma: The connection between perceived genetic etiology and attitudes and beliefs about mental illness. Psychiatric Rehabilitation Skills, 6(2), 159-185. https://doi.org/10.1080/10973430208408431 · DOI
  56. Pollack, I., & Hsieh, R. (1969). Sampling variability of the area under the ROC-curve of d′e. Psychological Bulletin, 71(3), 161-173. https://psycnet.apa.org/doi/10.1037/h0026862 · DOI
  57. Ratcliff, R., Sheu, C.-F., & Gronlund, S. D. (1992). Testing global memory models using ROC curves. Psychological Review, 99(3), 518-535. https://doi.org/10.1037/0033- 295X.99.3.518 · doi:10.1037/0033-295x.99.3.518 · Paper page
  58. Read, J., & Harré, N. (2009). The role of biological and genetic causal beliefs in the stigmatisation of 'mental patients'. Journal of Mental Health, 10(2), 223-235. https://doi.org/10.1080/09638230123129 · DOI · Paper page
  59. Read, J. (2011). Why promoting biological ideology increases prejudice against people labelled "schizophrenic". Australian Psychologist, 42(2), 118-128. https://doi.org/10.1080/00050060701280607 · DOI
  60. Rhodes, M. G., & Jacoby, L. L. (2007). On the dynamic nature of response criterion in recognition memory: Effects of base rate, awareness, and feedback. Journal of Experimental Psychology: Learning, Memory, and Cognition, 33(2), 305-320. https://doi.org/10.1037/0278-7393.33.2.305 · DOI
  61. Roozenbeek, J., Maertens, R., McClanahan, W., & van der Linden, S. (2020). Disentangling item and testing effects in inoculation research on online misinformation: Solomon revisited. Educational and Psychological Measurement, 81(2), 340-362. https://doi.org/10.1177/0013164420940378 · DOI · Paper page
  62. Roozenbeek, J., Traberg, C. S., & van der Linden, S. (2022) Technique-based inoculation against real-world misinformation. Royal Society Open Science, 9(5), Article 211719. https://doi.org/10.1098/rsos.211719 · DOI
  63. Roozenbeek, J., & van der Linden, S. (2019). Fake news game confers psychological resistance against online misinformation. Palgrave Communications, 5, Article 65. https://doi.org/10.1057/s41599-019-0279-9 · DOI · Paper page
  64. Rotello, C. M., Heit, E., & Dubé, C. (2015). When more data steer us wrong: Replications with the wrong dependent measure perpetuate erroneous conclusions. Psychonomic Bulletin & Review, 22(4), 944-954. https://doi.org/10.3758/s13423-014-0759-2 · DOI
  65. Saleh, N. F., Roozenbeek, J., Makki, F. A., McClanahan, W. P., & van der Linden, S. (2021). Active inoculation boosts attitudinal resistance against extremist persuasion techniques: A novel approach towards the prevention of violent extremism. Behavioural Public Policy, 1-24. https://doi.org/10.1017/bpp.2020. · DOI
  66. See, J. E., Warm, J. S., Dember, W. N., & Howe, S. R. (1997). Vigilance and signal detection theory: An empirical evaluation of five measures of response bias. Human Factors: The Journal of the Human Factors and Ergonomics Society, 39(1), 14-19. https://doi.org/10.1518/001872097778940704 · DOI
  67. Snodgrass, J. G., & Corwin, J. (1988). Pragmatics of measuring recognition memory: applications to dementia and amnesia. Journal of Experimental Psychology: General, 117(1), 34-50. https://doi.org/10.1037/0096-3445.117.1.34 · DOI · Paper page
  68. Stanislaw, H., & Todorov, N. (1999). Calculation of signal detection theory measures. Behavior Research Methods, Instruments, & Computers, 31(1), 137-149. https://doi.org/10.3758/bf03207704 · DOI · Paper page
  69. Steblay, N. K., Dysart, J. E., & Wells, G. L. (2011). Seventy-two tests of the sequential lineup superiority effect: A meta-analysis and policy discussion. Psychology, Public Policy, and Law, 17(1), 99-139. https://doi.org/10.1037/a0021650 · DOI · Paper page
  70. Tasche, D. (2008). Validation of internal rating systems and PD estimates. In G. Christodoulakis & S. Satchell (Eds.), The analytics of risk model validation (pp. 169- 196). Elsevier. https://doi.org/10.1016/B978-075068158-2.50014-7 · doi:10.1016/b978-075068158-2.50014-7
  71. Tay, L. Q., Hurlstone, M. J., Kurz, T., & Ecker, U. K. H. (2021). A comparison of prebunking and debunking interventions for implied versus explicit misinformation. British Journal of Psychology, 113(3), 591-607. https://doi.org/10.1111/bjop.12551 · DOI
  72. Urban, A., Moore, J., & Hewitt, C. (2019). Fake it to make it, media literacy, and persuasive design: Using the functional triad as a tool for investigating persuasive elements in a fake news simulator. Proceedings of the Association for Information Science and Technology, 55(1), 915-916. https://doi.org/10.1002/pra2.2018.14505501174 · doi:10.1002/pra2.2018.14505501174
  73. Viechtbauer, W. (2010). Conducting meta-analyses in R with the metafor package. Journal of Statistical Software, 36(3), 1-48. https://doi.org/10.18637/jss.v036.i03 · DOI · Paper page
  74. Van Bavel, J. J., Harris, E. A., Pärnamets, P., Rathje, S., Doell, K. C., & Tucker, J. A. (2021). Political psychology in the digital (mis)information age: A model of news belief and sharing. Social Issues and Policy Review, 15(1), 84-113. https://doi.org/10.1111/sipr.12077 · DOI · Paper page
  75. Witt, J. K., Taylor, J. E. T., Sugovic, M., & Wixted, J. T. (2015). Signal Detection Measures Cannot Distinguish Perceptual Biases from Response Biases. Perception, 44(3), 289-300. https://doi.org/10.1068/ · DOI · Paper page

Cited by

Metadata

Authors
Modirrousta-Galian, A., & Higham, P. A.
Published
2022-08-18
Cited by
14
References
1
Type
Preprint

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Gamified Inoculation Interventions Do Not Improve Discrimination Between True and Fake News: Reanalyzing Existing Research With Receiver Operating Characteristic Analysis. https://app.cochat.ai/papers/10.31234/osf.io/4bgkd
The paper
Modirrousta-Galian, A., & Higham, P. A. (2022). Gamified Inoculation Interventions Do Not Improve Discrimination Between True and Fake News: Reanalyzing Existing Research With Receiver Operating Characteristic Analysis. https://doi.org/10.31234/osf.io/4bgkd

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.