CoChat

The Misinformation Susceptibility Test (MIST): A psychometrically validated measure of news veracity discernment

Open access
Maertens, Rakoen, Götz, Friedrich M., Golino, Hudson F., Roozenbeek, Jon, Schneider, Claudia R., Kyrychenko, Yara, Kerr, John R., Stieger, Stefan, McClanahan, William P., Drabot, Karly, He, James, van der Linden, Sander · 2023 · Behavior Research Methods

Published version of a preprint

Cited by 104

The DOI matches its Crossref record; 60 of the 60 references it cites were found in a registry or an index.

Abstract

Interest in the psychology of misinformation has exploded in recent years. Despite ample research, to date there is no validated framework to measure misinformation susceptibility. Therefore, we introduce V e r i f ication d o n e , a nuanced interpretation schema and assessment tool that simultaneously considers V eracity discernment , and its distinct, measurable abilities ( r eal/ f ake news detection ), and biases ( d istrust / n aïvité —negative/positive judgment bias). We then conduct three studies with seven independent samples ( N total = 8504) to show how to develop, validate, and apply the Misinformation Susceptibility Test (MIST). In Study 1 ( N = 409) we use a neural network language model to generate items, and use three psychometric methods—factor analysis, item response theory, and exploratory graph analysis—to create the MIST-20 (20 items; completion time < 2 minutes), the MIST-16 (16 items; < 2 minutes), and the MIST-8 (8 items; < 1 minute). In Study 2 ( N = 7674) we confirm the internal and predictive validity of the MIST in five national quota samples (US, UK), across 2 years, from three different sampling platforms—Respondi, CloudResearch, and Prolific. We also explore the MIST’s nomological net and generate age-, region-, and country-specific norm tables. In Study 3 ( N = 421) we demonstrate how the MIST—in conjunction with V e r i f ication d o n e —can provide novel insights on existing psychological interventions, thereby advancing theory development. Finally, we outline the versatile implementations of the MIST as a screening tool, covariate, and intervention evaluation framework. As all methods are transparently reported and detailed, this work will allow other researchers to create similar scales or adapt them for any population of interest.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.3758/s13428-023-02124-2

Cites

60 of 60 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 60 references

Research works printed but not linked: 105

  1. Aichholzer, J., & Kritzinger, S. (2016). Kurzskala politischer Zynismus (KPZ). [Short scale of political cynicism]. Zusammenstellung Sozialwissenschaftlicher Items und Skalen. https://doi.org/10.6102/zis245 · doi:10.6102/zis245
  2. Aird, M. J., Ecker, U. K. H., Swire, B., Berinsky, A. J., & Lewandowsky, S. (2018). Does truth matter to voters? The effects of correcting political misinformation in an Australian sample. Royal Society Open Science, 5(12), Article 180593. https://doi.org/10.1098/rsos.180593 · DOI · Paper page
  3. Anvari, F., & Lakens, D. (2021). Using anchor-based methods to determine the smallest effect size of interest. Journal of Experimental Social Psychology. Advance online publication. https://doi.org/10.1016/j.jesp.2021.104159 · DOI · Paper page
  4. Baron, J. (2019). Actively open-minded thinking in politics. Cognition, 188, 8–18. https://doi.org/10.1016/j.cognition.2018.10.004 · DOI · Paper page
  5. Basol, M., Roozenbeek, J., McClanahan, P., Berriche, M., Uenal, F., & van der Linden, S. (2021). Towards psychological herd immunity: Cross-cultural evidence for two prebunking interventions against COVID-19 misinformation. Big Data & Society, 8(1), 1–18. https://doi.org/10.1177/20539517211013868 · DOI · Paper page
  6. Batailler, C., Brannon, S. M., Teas, P. E., & Gawronski, B. (2022). A signal detection approach to understanding the identification of fake news. Perspectives on Psychological Science, 17(1), 78–98. https://doi.org/10.1177/1745691620986135 · DOI · Paper page
  7. Bentler, P. M., & Bonett, D. G. (1980). Significance tests and goodness of fit in the analysis of covariance structures. Psychological Bulletin, 88(3), 588–606. https://doi.org/10.1037/0033-2909.88.3.588 · DOI · Paper page
  8. Blondel, V. D., Guillaume, J.-L., Lambiotte, R., & Lefebvre, E. (2008). Fast unfolding of communities in large networks. Journal of Statistical Mechanics: Theory and Experiment, 2008(10), P10008. https://doi.org/10.1088/1742-5468/2008/10/P10008 · DOI · Paper page
  9. Boker, S. M. (2018). Longitudinal multivariate psychology (E. Ferrer, S. M. Boker, & K. J. Grimm, Eds.). Routledge. https://doi.org/10.4324/9781315160542 · doi:10.4324/9781315160542
  10. Borsboom, D., Cramer, A. O., Schmittmann, V. D., Epskamp, S., & Waldorp, L. J. (2011). The small world of psychopathology. PloS One, 6(11), e27407. https://doi.org/10.1371/journal.pone.0027407 · DOI · Paper page
  11. Bovet, A., & Makse, H. A. (2019). Influence of fake news in Twitter during the 2016 US presidential election. Nature Communications, 10(1), 7. https://doi.org/10.1038/s41467-018-07761-2 · DOI · Paper page
  12. Brick, C., Hood, B., Ekroll, V., & de-Wit, L. (2022). Illusory essences: A bias holding back theorizing in psychological science. Perspectives on Psychological Science, 17(2), 491–506. https://doi.org/10.1177/1745691621991838 · DOI
  13. Brotherton, R., French, C. C., & Pickering, A. D. (2013). Measuring belief in conspiracy theories: The generic conspiracist beliefs scale. Frontiers in Psychology, 4, 1–15. https://doi.org/10.3389/fpsyg.2013.00279 · DOI · Paper page
  14. Buhrmester, M. D., Talaifar, S., & Gosling, S. D. (2018). An evaluation of Amazon’s Mechanical Turk, its rapid rise, and its effective use. Perspectives on Psychological Science, 13(2), 149–154. https://doi.org/10.1177/1745691617706516 · DOI · Paper page
  15. Campbell, D. T., & Fiske, D. W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56(2), 81–105. https://www.ncbi.nlm.nih.gov/pubmed/13634291. · DOI · Paper page
  16. Chalmers, R. P. (2012). mirt: A multidimensional item response theory package for the Renvironment. Journal of Statistical Software, 48(6), 1–29. https://doi.org/10.18637/jss.v048.i06 · DOI · Paper page
  17. Chen, J., & Chen, Z. (2008). Extended Bayesian information criteria for model selection with large model spaces. Biometrika, 95(3), 759–771. https://doi.org/10.1093/biomet/asn034 · DOI · Paper page
  18. Chołoniewski, J., Sienkiewicz, J., Dretnik, N., Leban, G., Thelwall, M., & Hołyst, J. A. (2020). A calibrated measure to compare fluctuations of different entities across timescales. Scientific Reports, 10(1), Article 20673. https://doi.org/10.1038/s41598-020-77660-4 · DOI
  19. Christensen, A. P., Cotter, K. N., & Silvia, P. J. (2019). Reopening openness to experience: A network analysis of four openness to experience inventories. Journal of Personality Assessment, 101(6), 574–588. https://doi.org/10.1080/00223891.2018.1467428 · DOI · Paper page
  20. Christensen, A. P., Garrido, L. E., & Golino, H. (2020a). Unique variable analysis: A novel approach for detecting redundant variables in multivariate data. PsyArXiv. https://doi.org/10.31234/osf.io/4kra2 · DOI
  21. Christensen, A. P., Golino, H., & Silvia, P. J. (2020b). A psychometric network perspective on the validity and validation of personality trait questionnaires. European Journal of Personality, 34(6), 1095–1108. https://doi.org/10.1002/per.2265 · DOI · Paper page
  22. Christensen, A. P., & Golino, H. (2021a). Estimating the stability of psychological dimensions via bootstrap exploratory graph analysis: A Monte Carlo simulation and tutorial. Psych, 3(3), 479–500. https://doi.org/10.3390/psych3030032 · DOI · Paper page
  23. Christensen, A. P., & Golino, H. (2021b). Factor or network model? Predictions from neural networks. Journal of Behavioral Data Science, 1(1), 85–126. https://doi.org/10.35566/jbds/v1n1/p5 · DOI
  24. Christensen, A. P., & Golino, H. (2021c). On the equivalency of factor and network loadings. Behavior Research Methods, 53, 1563–1580. https://doi.org/10.3758/s13428-020-01500-6 · DOI · Paper page
  25. Cichocka, A., Marchlewska, M., & de Zavala, A. G. (2016). Does self-love or self-hate predict conspiracy beliefs? Narcissism, self-esteem, and the endorsement of conspiracy theories. Social Psychological and Personality Science, 7(2), 157–166. https://doi.org/10.1177/1948550615616170 · DOI · Paper page
  26. Clark, L. A., & Watson, D. (1995). Constructing validity: Basic issues in objective scale development. Psychological Assessment, 7(3), 309–319. https://doi.org/10.1037//1040-3590.7.3.309 · DOI · Paper page
  27. Cokely, E. T., Galesic, M., Schulz, E., Ghazal, S., & Garcia-Retamero, R. (2012). Measuring risk literacy: The Berlin numeracy test. Judgment and Decision Making, 7(1), 25–47. http://journal.sjdm.org/11/11808/jdm11808.pdf · DOI · Paper page
  28. Comrey, A. L., & Lee, H. B. (1992). A first course in factor analysis (2nd ed.). Erlbaum Associates
  29. Condon, D. M., Wood, D., Mõttus, R., Booth, T., Costantini, G., Greiff, S., ..., Zimmermann, J. (2020). Bottom up construction of a personality taxonomy. European Journal of Psychological Assessment, 36(6), 923–934. https://doi.org/10.1027/1015-5759/a000626 · DOI
  30. Cramer, A. O. (2012). Why the item “23+ 1” is not in a depression questionnaire: Validity from a network perspective. Measurement: Interdisciplinary Research & Perspective, 10(1-2), 50–54. https://doi.org/10.1080/15366367.2012.681973 · DOI
  31. Cronbach, L. J., & Meehl, P. E. (1955). Construct validity in psychological tests. Psychological Bulletin, 52(4), 281–302. https://doi.org/10.1037/h0040957 · DOI · Paper page
  32. Curran, P. J., Bollen, K. A., Chen, F., Paxton, P., & Kirby, J. B. (2003). Finite sampling properties of the point estimates and confidence intervals of the RMSEA. Sociological Methods & Research, 32(2), 208–252. https://doi.org/10.1177/0049124103256130 · DOI · Paper page
  33. de Vries, W., & Nissim, M. (2020). As good as new: How to successfully recycle English GPT-2 to make models for other languages. ArXiv. https://arxiv.org/abs/2012.05628. Accessed 10 Dec 2020. · DOI
  34. Deffner, D., Rohrer, J. M., & McElreath, R. (2022). A causal framework for cross-cultural generalizability. Advances in Methods and Practices in Psychological Science, 5(3), 1–18. https://doi.org/10.1177/25152459221106366 · DOI
  35. Dhami, M. K., Hertwig, R., & Hoffrage, U. (2004). The role of representative design in an ecological approach to cognition. Psychological Bulletin, 130(6), 959–988. https://doi.org/10.1037/0033-2909.130.6.959 · DOI · Paper page
  36. Dür, A., & Schlipphak, B. (2021). Elite cueing and attitudes towards trade agreements: The case of TTIP. European Political Science Review, 13(1), 41–57. https://doi.org/10.1017/S175577392000034X · DOI
  37. Ebert, T., Götz, F. M., Gladstone, J. J., Müller, S. R., & Matz, S. C. (2021). Spending reflects not only who we are but also who we are around: The joint effects of individual and geographic personality on consumption. Journal of Personality and Social Psychology, 121(2), 378–393. https://doi.org/10.1037/pspp0000344 · DOI
  38. Epskamp, S., Maris, G., Waldorp, L. J., & Borsboom, D. (2018). Network psychometrics. In B. Irwing Paul (Ed.), The Wiley handbook of psychometric testing: A multidisciplinary reference on survey, scale and test development (pp. 953–986). John Wiley & Sons Ltd.. https://doi.org/10.1002/9781118489772.ch30 · DOI · Paper page
  39. Epskamp, S., Rhemtulla, M., & Borsboom, D. (2017). Generalized network psychometrics: Combining network and latent variable models. Psychometrika, 82(4), 904–927. https://doi.org/10.1007/s11336-017-9557-x · DOI · Paper page
  40. Finch, J. F., & West, S. G. (1997). The investigation of personality structure: Statistical models. Journal of Research in Personality, 31(4), 439–485. https://doi.org/10.1006/jrpe.1997.2194 · DOI · Paper page
  41. Flake, J. K., Pek, J., & Hehman, E. (2017). Construct validation in social and personality research: Current practice and recommendations. Social Psychological and Personality Science, 8(4), 370–378. https://doi.org/10.1177/1948550617693063 · DOI · Paper page
  42. Ford, J. K., MacCallum, R. C., & Tait, M. (1986). The application of exploratory factor analysis in applied psychology: A critical review and analysis. Personnel Psychology, 39(2), 291–314. https://doi.org/10.1111/j.1744-6570.1986.tb00583.x · DOI · Paper page
  43. Golino, H. F., & Demetriou, A. (2017). Estimating the dimensionality of intelligence like data using exploratory graph analysis. Intelligence, 62, 54–70. https://doi.org/10.1016/j.intell.2017.02.007 · DOI
  44. Golino, H. F., & Epskamp, S. (2017). Exploratory graph analysis: A new approach for estimating the number of dimensions in psychological research. PloS One, 12(6), e0174035. https://doi.org/10.1371/journal.pone.0174035 · DOI · Paper page
  45. Golino, H. F., Christensen, A. P., & Garrido, L. E. (2022). Exploratory graph analysis in context. Revista Psicologia: Teoria e Prática, 24(3), ePTPPA14197. https://doi.org/10.5935/1980-6906/ePTPIC15531.en · DOI
  46. Golino, H. F., Lillard, A. S., Becker, I., & Christensen, A. P. (2021). Investigating the structure of the children’s concentration and empathy scale using exploratory graph analysis. Psychological Test Adaptation and Development, 2(1), 35–49. https://doi.org/10.1027/2698-1866/a000008 · DOI
  47. Golino, H. F., Moulder, R., Shi, D., Christensen, A., Garrido, L., Neto, M., et al. (2020a). Entropy fit indices: New fit measures for assessing the structure and dimensionality of multiple latent variables. Multivariate Behavioral Research, 56(6), 874–902. https://doi.org/10.1080/00273171.2020.1779642 · DOI
  48. Golino, H. F., Shi, D., Garrido, L. E., Christensen, A. P., Nieto, M. D., Sadana, R., et al. (2020b). Investigating the performance of exploratory graph analysis and traditional techniques to identify the number of latent factors: A simulation and tutorial. Psychological Methods, 25(3), 292–230. https://doi.org/10.1037/met0000255 · DOI · Paper page
  49. Goretzko, D., Pham, T. T. H., & Bühner, M. (2021). Exploratory factor analysis: Current use, methodological developments and recommendations for good practice. Current Psychology, 40(7), 3510–3521. https://doi.org/10.1007/s12144-019-00300-2 · DOI · Paper page
  50. Götz, F. M., Maertens, R., Loomba, S., & van der Linden, S. (2023). Let the algorithm speak: How to use neural networks for automatic item generation in psychological scale development. Psychological Methods. Advance online publication. https://doi.org/10.1037/met0000540 · DOI · Paper page
  51. Götz, F. M., Gosling, S. D., & Rentfrow, P. J. (2022). Small effects: The indispensable foundation for a cumulative psychological science. Perspectives on Psychological Science, 17(1), 205–215. https://doi.org/10.1177/17456916209844 · doi:10.1177/17456916209844
  52. Hair, J. F., Anderson, R. E., Babin, B. J., & Black, W. C. (2010). Multivariate data analysis: A global perspective (7th ed.). Pearson
  53. Heinsohn, T., Fatke, M., Israel, J., Marschall, S., & Schultze, M. (2019). Effects of voting advice applications during election campaigns: Evidence from a panel study at the 2014 European elections. Journal of Information Technology & Politics, 16(3), 250–264. https://doi.org/10.1080/19331681.2019.1644265 · DOI
  54. Ho, A. K., Sidanius, J., Kteily, N., Sheehy-Skeffington, J., Pratto, F., Henkel, K. E., Foels, R., & Stewart, A. L. (2015). The nature of social dominance orientation: Theorizing and measuring preferences for intergroup inequality using the new SDO7 scale. Journal of Personality and Social Psychology, 109(6), 1003–1028. https://doi.org/10.1037/pspi0000033 · DOI · Paper page
  55. Hommel, B. E., Wollang, F. J. M., Kotova, V., Zacher, H., & Schmukle, S. C. (2022). Transformer-based deep neural language modeling for construct-specific automatic item generation. Psychometrika, 87(2), 749–772. https://doi.org/10.1007/s11336-021-09823-9 · DOI · Paper page
  56. Horn, J. L. (1965). A rationale and test for the number of factors in factor analysis. Psychometrika, 30(2), 179–185. https://doi.org/10.1007/BF02289447 · DOI · Paper page
  57. Hotez, P., Batista, C., Ergonul, O., Figueroa, J. P., Gilbert, S., Gursel, M., Hassanain, M., Kang, G., Kim, J. H., Lall, B., Larson, H., Naniche, D., Sheahan, T., Shoham, S., Wilder-Smith, A., Strub-Wourgaft, N., Yadav, P., & Bottazzi, M. E. (2021). Correcting COVID-19 vaccine misinformation. EClinicalMedicine, 33, Article 100780. https://doi.org/10.1016/j.eclinm.2021.100780 · DOI
  58. Humphreys, L. G., & Ilgen, D. R. (1969). Note on a criterion for the number of common factors. Educational and Psychological Measurement, 29(3), 571–578. https://doi.org/10.1177/001316446902900303 · DOI · Paper page
  59. Jamison, L., Golino, H., & Christensen, A. P. (2022). Metric invariance in exploratory graph analysis via permutation testing. PsycArxiv. https://doi.org/10.31234/osf.io/j4rx9 · DOI
  60. Jimenez, M., Abad, F. J., Garcia-Garzon, E., Golino, H., Christensen, A. P., & Garrido, L. E. (2022). Dimensionality assessment in generalized bi-factor structures: A network psychometrics approach. PsyArXiv. https://doi.org/10.31234/osf.io/2ujdk · DOI
  61. Jolley, D., & Paterson, J. L. (2020). Pylons ablaze: Examining the role of 5G COVID-19 conspiracy beliefs and support for violence. British Journal of Social Psychology, 59(3), 628–640. https://doi.org/10.1111/bjso.12394 · DOI · Paper page
  62. Konrath, S., Meier, B. P., & Bushman, B. J. (2014). Development and validation of the Single Item Narcissism Scale (SINS). PloS One, 9(8), Article e103469. https://doi.org/10.1371/journal.pone.0103469 · DOI · Paper page
  63. Kumareswaran, D. J. (2014). The psychopathological foundations of conspiracy theorists. Victoria University of Wellington. http://hdl.handle.net/10063/3603
  64. Lawson, A., & Kakkar, H. (2021). Of pandemics, politics, and personality: The role of conscientiousness and political ideology in sharing of fake news. PsyArXiv. https://doi.org/10.31234/osf.io/ves5m · DOI
  65. Lewandowsky, S., Smillie, L., Garcia, D., Hertwig, R., Weatherall, J., Egidy, S., Robertson, R. E., O’Connor, C., Kozyreva, A., Lorenz-Spreen, P., Blaschke, Y., & Leiser, M. R. (2020). Technology and democracy: Understanding the influence of online technologies on political behaviour and decision-making. Publications Office of the European Union. https://doi.org/10.2760/709177 · doi:10.2760/709177 · Paper page
  66. Litman, L., Robinson, J., & Abberbock, T. (2017). TurkPrime.com: A versatile crowdsourcing data acquisition platform for the behavioral sciences. Behavior Research Methods, 49(2), 433–442. https://doi.org/10.3758/s13428-016-0727-z · DOI · Paper page
  67. Loevinger, J. (1957). Objective tests as instruments of psychological theory. Psychological Reports, 3(3), 635–694. https://doi.org/10.2466/pr0.1957.3.3.635 · DOI · Paper page
  68. Marsman, M., Borsboom, D., Kruis, J., Epskamp, S., van Bork, R., Waldorp, L., et al. (2018). An introduction to network psychometrics: Relating Ising network models to item response theory models. Multivariate Behavioral Research, 53(1), 15–35. · DOI · Paper page
  69. Maertens, R., Anseel, F., & van der Linden, S. (2020). Combatting climate change misinformation: Evidence for longevity of inoculation and consensus messaging effects. Journal of Environmental Psychology, 70, 101455. https://doi.org/10.1016/j.jenvp.2020.101455 · DOI · Paper page
  70. Maertens, R., Roozenbeek, J., Simons, J., Lewandowsky, S., Maturo, V., Goldberg, B., ...,, van der Linden, S. (2022). Psychological booster shots targeting memory increase long-term resistance against misinformation. [Manuscript in preparation] · DOI
  71. Markon, K. E. (2019). Bifactor and hierarchical models: Specification, inference, and interpretation. Annual Review of Clinical Psychology, 15, 51–69. https://doi.org/10.1146/annurev-clinpsy-050718-095522 · DOI · Paper page
  72. McDonald, R. P. (1999). Test theory: A unified treatment. Psychology Press. https://doi.org/10.4324/9781410601087 · DOI · Paper page
  73. Meehl, P. E. (1978). Theoretical risks and tabular asterisks: Sir Karl, Sir Ronald, and the slow progress of soft psychology. Journal of Consulting and Clinical Psychology, 46(4), 806–834. https://doi.org/10.1037/0022-006X.46.4.806 · DOI · Paper page
  74. Nguyen, T. H., Han, H.-R., Kim, M. T., & Chan, K. S. (2014). An introduction to item response theory for patient-reported outcome measurement. The Patient, 7(1), 23–35. https://doi.org/10.1007/s40271-013-0041-0 · DOI · Paper page
  75. Paulhus, D. L., Buckels, E. E., Trapnell, P. D., & Jones, D. N. (2020). Screening for dark personalities. European Journal of Psychological Assessment, 37(3), 208–222. https://doi.org/10.1027/1015-5759/a000602 · DOI · Paper page
  76. Pennycook, G., Binnendyk, J., Newton, C., & Rand, D. G. (2021a). A practical guide to doing behavioral research on fake news and misinformation. Collabra: Psychology, 7(1), 25293. https://doi.org/10.1525/collabra.25293 · DOI · Paper page
  77. Pituch, K. A., & Stevens, J. P. (2015). Applied multivariate statistics for the social sciences: Analyses with SAS and IBM’s SPSS. Routledge. https://doi.org/10.4324/9781315814919 · DOI · Paper page
  78. Pons, P., & Latapy, M. (2005). Computing communities in large networks using random walks. In Pi. Yolum, T. Güngör, F. Gürgen, & C. Özturan (Eds.), Computer and information sciences - ISCIS 2005 (pp. 284–293). Berlin, Heidelberg: Springer. https://doi.org/10.1007/11569596_31 · DOI · Paper page
  79. Preskill, J. (2018). Quantum Shannon entropy. In J. Preskill (Ed.), Quantum information (p. 94). Cambridge University Press. https://arxiv.org/pdf/1604.07450.pdf · arXiv
  80. Rammstedt, B., Lechner, C. M., & Danner, D. (2021). Short forms do not fall short: A comparison of three (extra-)short forms of the Big Five. European Journal of Psychological Assessment, 37(1), 23–32. https://doi.org/10.1027/1015-5759/a000574 · DOI
  81. Reise, S. P., Widaman, K. F., & Pugh, R. H. (1993). Confirmatory factor analysis and item response theory: Two approaches for exploring measurement invariance. Psychological Bulletin, 114(3), 552–566. https://doi.org/10.1037/0033-2909.114.3.552 · doi:10.1037/0033-2909.114.3.552 · Paper page
  82. Rentfrow, P. J., Gosling, S. D., Jokela, M., Stillwell, D. J., Kosinski, M., & Potter, J. (2013). Divided we stand: Three psychological regions of the United States and their political, economic, social, and health correlates. Journal of Personality and Social Psychology, 105(6), 996–1012. https://doi.org/10.1037/a0034434 · DOI · Paper page
  83. Rentfrow, P. J., Jokela, M., & Lamb, M. E. (2015). Regional personality differences in Great Britain. PloS One, 10(3), e0122245. https://doi.org/10.1371/journal.pone.0122245 · DOI · Paper page
  84. Revelle, W., & Condon, D. M. (2019). Reliability from α to ω: A tutorial. Psychological Assessment, 31(12), 1395–1411. https://doi.org/10.1037/pas0000754 · DOI · Paper page
  85. Roozenbeek, J., Culloty, E., & Suiter, J. (2023). Countering misinformation: Evidence, knowledge gaps, and implications of current interventions. European Psychologist. In press. https://doi.org/10.31234/osf.io/b52um · DOI
  86. Roozenbeek, J., Maertens, R., Herzog, S., Geers, M., Kurvers, R., Sultan, M., & van der Linden, S. (2022). Susceptibility to misinformation is consistent across question framings and response modes and better explained by myside bias and partisanship than analytical thinking. Judgment and Decision Making, 17(3), 547–573. http://journal.sjdm.org/22/220228/jdm220228.pdf · DOI
  87. Roozenbeek, J., & van der Linden, S. (2020). Breaking Harmony Square: A game that “inoculates” against political misinformation. Harvard Kennedy School Misinformation Review, 1(8), 1–26. https://doi.org/10.37016/mr-2020-47 · DOI · Paper page
  88. Rosellini, A. J., & Brown, T. A. (2021). Developing and validating clinical questionnaires. Annual Review of Clinical Psychology, 17, 55–81. https://doi.org/10.1146/annurev-clinpsy-081219-115343 · DOI
  89. Sass, D. A., & Schmitt, T. A. (2010). A comparative investigation of rotation criteria within exploratory factor analysis. Multivariate Behavioral Research, 45(1), 73–103. https://doi.org/10.1080/00273170903504810 · DOI · Paper page
  90. Satorra, A. (2000). Scaled and adjusted restricted tests in multi-sample analysis of moment structures. In Innovations in multivariate statistical analysis (pp. 233–247). Springer. https://doi.org/10.1007/978-1-4615-4603-0_17 · DOI · Paper page
  91. Schwartz, L. M., Woloshin, S., Black, W. C., & Welch, H. G. (1997). The role of numeracy in understanding the benefit of screening mammography. Annals of Internal Medicine, 127(11), 966–972. https://doi.org/10.7326/0003-4819-127-11-199712010-00003 · DOI · Paper page
  92. Shi, D., DiStefano, C., McDaniel, H. L., & Jiang, Z. (2018). Examining chi-square test statistics under conditions of large model size and ordinal data. Structural Equation Modeling: A Multidisciplinary Journal, 25(6), 924–945. https://doi.org/10.1080/10705511.2018.1449653 · DOI
  93. Simms, L. J. (2008). Classical and modern methods of psychological scale construction. Social and Personality Psychology Compass, 2(1), 414–433. https://doi.org/10.1111/j.1751-9004.2007.00044.x · DOI · Paper page
  94. Steiner, M., & Grieder, S. (2020). EFAtools: An R package with fast and flexible implementations of exploratory factor analysis tools. Journal of Open Source Software, 5(53), 2521. https://doi.org/10.21105/joss.02521 · DOI · Paper page
  95. Strauss, M. E., & Smith, G. T. (2009). Construct validity: Advances in theory and methodology. Annual Review of Clinical Psychology, 5, 1–25. https://doi.org/10.1146/annurev.clinpsy.032408.153639 · DOI · Paper page
  96. Swami, V., Furnham, A., Smyth, N., Weis, L., Lay, A., & Clow, A. (2016). Putting the stress on conspiracy theories: Examining associations between psychological stress, anxiety, and belief in conspiracy theories. Personality and Individual Differences, 99, 72–76. https://doi.org/10.1016/j.paid.2016.04.084 · DOI · Paper page
  97. Swire, B., Berinsky, A. J., Lewandowsky, S., & Ecker, U. K. H. (2017). Processing political misinformation: Comprehending the Trump phenomenon. Royal Society Open Science, 4(3), 160802. https://doi.org/10.1098/rsos.160802 · DOI · Paper page
  98. Tabachnick, B. G., & Fidell, L. S. (2007). Using multivariate statistics (5th ed.). Pearson. https://psycnet.apa.org/record/2006-03883-000
  99. Thalmayer, A. G., Saucier, G., & Eigenhuis, A. (2011). Comparative validity of brief to medium-length Big Five and Big Six Personality Questionnaires. Psychological Assessment, 23(4), 995–1009. https://doi.org/10.1037/a0024165 · DOI · Paper page
  100. Thurstone, L. L. (1944). Second-order factors. Psychometrika, 9(2), 71–100. https://doi.org/10.1007/BF02288715 · doi:10.1007/bf02288715
  101. Uenal, F., Sidanius, J., Maertens, R., Hudson, S. K. T., Davis, G., & Ghani, A. (2022). The roots of ecological dominance orientation: Assessing individual preferences for an anthropocentric and hierarchically organized world. Journal of Environmental Psychology, 81, 101783. https://doi.org/10.1016/j.jenvp.2022.101783 · DOI
  102. Van Bavel, J. J., Harris, E. A., Pärnamets, P., Rathje, S., Doell, K., & Tucker, J. A. (2020). Political psychology in the digital (mis)information age: A model of news belief and sharing. PsyArXiv. https://doi.org/10.31234/osf.io/u5yts · DOI
  103. van der Linden, S., & Roozenbeek, J. (2020). Psychological inoculation against fake news. In R. Greifeneder, M. Jaffé, E. J. Newman, & N. Schwarz (Eds.), The psychology of fake news: Accepting, sharing, and correcting misinformation. Routledge https://www.routledge.com/p/book/9780367271831
  104. Weiner, I. B., Schinka, J. A., & Velicer, W. F. (2012). Handbook of psychology: Research methods in psychology (2nd ed., Vol. 2). John Wiley & Sons
  105. Zickar, M. J. (2020). Measurement development and evaluation. Annual Review of Organizational Psychology and Organizational Behavior, 7, 213–232. https://doi.org/10.1146/annurev-orgpsych-012119-044957 · DOI

Cited by

See all 104

Metadata

Authors
Maertens, R., Götz, F. M., Golino, H. F., Roozenbeek, J., Schneider, C. R., Kyrychenko, Y., Kerr, J. R., Stieger, S., McClanahan, W. P., Drabot, K., He, J., & van der Linden, S.
Venue
Behavior Research Methods
Published
2023-06-29
Volume
56
Issue
3
Pages
1863-1899
ISSN
  • 1554-351X (ISSN-L)
  • 1554-3528
OpenAlex
W4382502700
Cited by
104
References
60
FWCI
43.66
Type
journal article
Language
en
Open access
hybrid
Licence

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: The Misinformation Susceptibility Test (MIST): A psychometrically validated measure of news veracity discernment. https://app.cochat.ai/papers/10.3758/s13428-023-02124-2
The paper
Maertens, R., Götz, F. M., Golino, H. F., Roozenbeek, J., Schneider, C. R., Kyrychenko, Y., Kerr, J. R., Stieger, S., McClanahan, W. P., Drabot, K., He, J., & van der Linden, S. (2023). The Misinformation Susceptibility Test (MIST): A psychometrically validated measure of news veracity discernment. Behavior Research Methods. https://doi.org/10.3758/s13428-023-02124-2

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.