CoChat

Debate Chatbots to Facilitate Critical Thinking on YouTube: Social Identity and Conversational Style Make A Difference

Tanprasert, Thitaree, Fels, Sidney S, Sinnamon, Luanne, Yoon, Dongwook · 2024 · Proceedings of the CHI Conference on Human Factors in Computing Systems
Cited by 51

The DOI matches its Crossref record; 19 of the 19 references it cites were found in a registry or an index.

Abstract

Exposure to diverse perspectives is helpful for bursting the filter bubble in online public video platforms. The recent advancement of Large Language Models (LLMs) illuminates the potential of creating a debate chatbot that prompts users to critically examine their stances on a topic formed by watching videos. However, whether the viewer is influenced by the chatbot may depend on its persona. In this paper, we investigated the effect of two relevant persona attributes - social identity and rhetorical styles - on critical thinking. In a mixed-methods study (n=36), we found that chatbots with outgroup (vs. ingroup) identity (t(33)=-2.33, p=0.03) and persuasive (vs. eristic) rhetoric (t(44)=1.98, p=0.05) induced critical thinking most effectively, making participants re-examine their arguments. However, participants’ stances remain largely unaffected, likely due to the chatbot’s lack of contextual knowledge and human touch. Our paper provides empirical groundwork for designing chatbot persona for remedying filter bubbles in online communities.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3613904.3642513

Cites

19 of 19 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 19 references

Research works printed but not linked: 88

  1. 10.18653/v1/2020.acl-main.632 · DOI
  2. Hussam Alkaissi and Samy I McFarlane. 2023. Artificial hallucinations in ChatGPT: implications in scientific writing. Cureus 15, 2 (2023).
  3. Christa SC Asterhan and Rakheli Hever. 2015. Learning from reading argumentive group discussions in Facebook: Rhetoric style matters (again). Computers in human behavior 53 (2015), 570–576.
  4. 10.1145/3449230 · DOI
  5. David Bawden 2008. Origins and concepts of digital literacy. Digital literacies: Concepts, policies and practices 30, 2008 (2008), 17–32.
  6. 10.1177/0146167210376059 · DOI · Paper page
  7. Kevin M Blasiak, Marten Risius, and Sabine Matook. 2021. “Social Bots for Peace”: A Dual-Process Perspective to Counter Online Extremist Messaging. Association for Information Systems.
  8. Kristen Bloom and Kelly Marie Johnston. 2010. Digging into YouTube videos: Using media literacy and participatory culture to promote cross-cultural understanding. Journal of Media Literacy Education 2, 2 (2010), 3.
  9. Virginia Braun and Victoria Clarke. 2006. Using thematic analysis in psychology. Qualitative research in psychology 3, 2 (2006), 77–101.
  10. Pablo Brinol and Richard E Petty. 2009. Source factors in persuasion: A self-validation approach. European review of social psychology 20, 1 (2009), 49–96.
  11. Megan A Brown, James Bisbee, Angela Lai, Richard Bonneau, Jonathan Nagler, and Joshua A Tucker. 2022. Echo Chambers, Rabbit Holes, and Algorithmic Bias: How YouTube Recommends Content to Real Users. Available at SSRN 4114905 (2022).
  12. 10.1515/opis-2020-0007 · DOI
  13. Katarzyna Budzynska. 2010. Argument analysis: Components of interpersonal argumentation. In Computational Models of Argument. IOS Press, 135–146.
  14. 10.1108/09600030710825694 · DOI · Paper page
  15. Richard Catrambone, John Stasko, and Jun Xiao. 2004. ECA as User Interface Paradigm: Experimental findings within a framework for research. From brows to trust: Evaluating embodied conversational agents (2004), 239–267.
  16. 10.18653/v1/2023.eacl-main.53 · DOI
  17. Guo Chen, Yin-Dong Zheng, Jiahao Wang, Jilan Xu, Yifei Huang, Junting Pan, Yi Wang, Yali Wang, Yu Qiao, Tong Lu, 2023. VideoLLM: Modeling Video Sequence with Large Language Models. arXiv preprint arXiv:2305.13292 (2023). · arXiv
  18. Juliet M Corbin and Anselm Strauss. 1990. Grounded theory research: Procedures, canons, and evaluative criteria. Qualitative sociology 13, 1 (1990), 3–21.
  19. Abidin Crystal. 2018. Gay, famous and working hard on YouTube: Influencers, queer microcelebrity publics and discursive activism. In Youth, sexuality and sexual citizenship. Routledge, 217–231.
  20. Lincoln Dahlberg. 2001. The Internet and democratic discourse: Exploring the prospects of online deliberative forums extending the public sphere. Information, communication & society 4, 4 (2001), 615–633.
  21. Daniela Dahle 2022. Adolescents YouTube and Media Literacy A mixed-method time series study of Norwegian youth’s social media usage and its implications for media literacy. Master’s thesis. OsloMet-Storbyuniversitetet.
  22. Jakob Sebastian Davis. 2022. Video Essays Academia and Remediation: How YouTube Video Essayists Refashion Media and Scholarship. Ph. D. Dissertation. Texas A&M University-Central Texas.
  23. 10.1145/3267305.3274189 · DOI
  24. D Christopher Dryer. 1999. Getting personal with computers: how to design personalities for agents. Applied artificial intelligence 13, 3 (1999), 273–295.
  25. 10.1386/eta.4.3.247_1 · doi:10.1386/eta.4.3.247_1
  26. 10.1145/3308558.3313676 · DOI
  27. 10.1145/344949.345004 · DOI · Paper page
  28. 10.1145/1753326.1753502 · DOI · Paper page
  29. Kathleen C Fraser, Svetlana Kiritchenko, and Isar Nejadgholi. 2023. Diversity is not a one-way street: Pilot study on ethical interventions for racial bias in text-to-image systems. ICCV, accepted (2023).
  30. Sebastian Fritz-Morgenthal, Bernhard Hein, and Jochen Papenbrock. 2022. Financial risk management and explainable, trustworthy, responsible AI. Frontiers in artificial intelligence 5 (2022), 779799.
  31. 10.1145/3172944.3172970 · DOI
  32. Frédéric Guay, Robert J Vallerand, and Céline Blanchard. 2000. On the assessment of situational intrinsic and extrinsic motivation: The Situational Motivation Scale (SIMS). Motivation and emotion 24 (2000), 175–213.
  33. 10.1007/s11518-021-5497-1 · DOI
  34. Dale Hample. 2003. Arguing skill. Handbook of communication and social interaction skills (2003), 439–478.
  35. John Hartley. 2009. Uses of YouTube: Digital literacy and the growth of knowledge. YouTube: Online video and participatory culture (2009), 126–143.
  36. 10.1145/3544548.3581328 · DOI
  37. Susan Herring, Kirk Job-Sluder, Rebecca Scheckler, and Sasha Barab. 2002. Searching for safety online: Managing" trolling" in a feminist forum. The information society 18, 5 (2002), 371–384.
  38. Michael A Hogg. 2016. Social identity theory. Springer.
  39. 10.1145/3392854 · DOI · Paper page
  40. 10.1007/s10726-021-09765-8 · DOI
  41. 10.1163/25900110-00101002 · DOI
  42. 10.1111/bjop.12563 · DOI
  43. 10.1080/10714410903482658 · DOI
  44. 10.1145/3449140 · DOI · Paper page
  45. 10.1016/j.chb.2020.106419 · DOI · Paper page
  46. Tibor Koltay. 2011. The media and the literacies: Media literacy, information literacy, digital literacy. Media, culture & society 33, 2 (2011), 211–221.
  47. Aaron Kuecker. 2014. Ethnicity and social identity. T&T Clark handbook to social idenfity in the New Testament (2014), 59–78.
  48. Brenda Laurel and S Joy Mountford. 1990. The art of human-computer interface design. Addison-Wesley Longman Publishing Co., Inc.
  49. Mark Ledwich, Anna Zaitsev, and Anton Laukemper. 2022. Radical bubbles on YouTube? Revisiting algorithmic extremism with personalised recommendations. First Monday (2022).
  50. Daniel Z Levin, Ellen M Whitener, and Rob Cross. 2006. Perceived trustworthiness of knowledge sources: The moderating impact of relationship length.Journal of applied psychology 91, 5 (2006), 1163.
  51. 10.1177/1527476419879919 · DOI · Paper page
  52. 10.1145/3544548.3580953 · DOI · Paper page
  53. Patricia W Linville, Gregory W Fischer, and Carolyn Yoon. 1996. Perceived covariation among the features of ingroup and outgroup members: The outgroup covariation effect.Journal of Personality and Social Psychology 70, 3 (1996), 421.
  54. 10.1145/3411764.3445467 · DOI · Paper page
  55. Carol B MacKnight. 2000. Teaching critical thinking through online discussions. Educause Quarterly 23, 4 (2000), 38–41.
  56. Nila Masrourisaadat. 2023. Quantitative and Qualitative Analysis of Text-to-Image models. Ph. D. Dissertation. Virginia Tech.
  57. Phaedra S Mohammed and Eleanor ‘Nell’Watson. 2019. Towards inclusive education in the age of artificial intelligence: Perspectives challenges and opportunities. Artificial Intelligence and Inclusive Education: Speculative futures and emerging practices (2019) 17–37. · DOI
  58. Fabio Motoki, Valdemar Pinho Neto, and Victor Rodrigues. 2023. More human than human: Measuring ChatGPT political bias. Public Choice (2023), 1–21.
  59. 10.1145/3544549.3573844 · DOI
  60. Sean Munson, Stephanie Lee, and Paul Resnick. 2013. Encouraging reading of diverse political viewpoints with a browser widget. In Proceedings of the international AAAI conference on web and social media, Vol. 7. 419–428.
  61. 10.1891/1061-3749.25.1.156 · DOI
  62. Vaibhavi Nandagiri and Leena Philip. 2018. Impact of influencers from Instagram and YouTube on their followers. International Journal of Multidisciplinary Research and Modern Education 4, 1 (2018), 61–65.
  63. 10.1145/3359243 · DOI · Paper page
  64. 10.1145/2566486.2568012 · DOI · Paper page
  65. 10.1371/journal.pone.0177865 · DOI
  66. 10.3390/educsci12120938 · DOI
  67. 10.2501/JAR-2023-008 · DOI
  68. 10.1177/0146167213499937 · DOI · Paper page
  69. 10.1177/1354856517736983 · DOI · Paper page
  70. Eun Rhee, James S Uleman, and Hoon Koo Lee. 1996. Variations in collectivism and individualism by ingroup and culture: Confirmatory factor analysis.Journal of Personality and Social Psychology 71, 5 (1996), 1037.
  71. Jürgen Rudolph, Samson Tan, and Shannon Tan. 2023. ChatGPT: Bullshit spewer or the end of traditional assessments in higher education?Journal of Applied Learning and Teaching 6, 1 (2023).
  72. Mustafa Safdari, Greg Serapio-García, Clément Crepy, Stephen Fitz, Peter Romero, Luning Sun, Marwa Abdulhai, Aleksandra Faust, and Maja Matarić. 2023. Personality traits in large language models. arXiv preprint arXiv:2307.00184 (2023). · arXiv
  73. 10.1109/TTS.2023.3257627 · DOI · Paper page
  74. Henri Tajfel and John C Turner. 2004. The social identity theory of intergroup behavior. In Political psychology. Psychology Press, 276–293.
  75. 10.34190/ecsm.9.1.106 · DOI
  76. Martin Tanis and Tom Postmes. 2005. A social identity approach to trust: Interpersonal perception, group membership and trusting behaviour. European journal of social psychology 35, 3 (2005), 413–424.
  77. 10.1145/3544548.3581153 · DOI
  78. 10.1016/j.lcsi.2019.03.009 · DOI · Paper page
  79. Achmad Tohe. 2021. YouTube, Learning, and Transformative Critical Pedagogy. KnE Social Sciences (2021), 15–29.
  80. Douglas Walton. 2009. Burden of proof in deliberation dialogs. In International workshop on argumentation in multi-agent systems. Springer, 1–22.
  81. Douglas Walton and Erik CW Krabbe. 1995. Commitment in dialogue: Basic concepts of interpersonal reasoning. SUNY press.
  82. 10.1007/BF00128147 · DOI · Paper page
  83. WM Westenberg. 2016. The influence of YouTubers on teenagers: a descriptive research about the role YouTubers play in the life of their teenage viewers. Master’s thesis. University of Twente.
  84. Matti Wiegmann, Khalid Al Khatib, Vishal Khanna, and Benno Stein. 2022. Analyzing Persuasion Strategies of Debaters on Social Media. In 29th International Conference on Computational Linguistics. International Committee on Computational Linguistics, 6897–6905.
  85. Wendy Wood. 2000. Attitude change: Persuasion and social influence. Annual review of psychology 51, 1 (2000), 539–570.
  86. Brahim Zarouali, Mykola Makhortykh, Mariella Bastian, and Theo Araujo. 2021. Overcoming polarization with chatbot news? Investigating the impact of news content containing opposing views on agreement and credibility. European journal of communication 36, 1 (2021), 53–68.
  87. 10.1016/j.paid.2020.110303 · DOI
  88. Daniel Zimmermann, Christian Noll, Lars Gräßer, Kai-Uwe Hugger, Lea Marie Braun, Tine Nowak, and Kai Kaspar. 2020. Influencers on YouTube: a quantitative study on young people’s use and perception of videos about political and societal topics. Current Psychology (2020), 1–17.

Cited by

See all 51

Metadata

Authors
Tanprasert, T., Fels, S. S., Sinnamon, L., & Yoon, D.
Venue
Proceedings of the CHI Conference on Human Factors in Computing Systems
Publisher
ACM
Published
2024-05-11
Pages
1-24
OpenAlex
W4396832241
Cited by
51
References
19
FWCI
35.28
Type
conference paper
Language
en
Open access
closed

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Debate Chatbots to Facilitate Critical Thinking on YouTube: Social Identity and Conversational Style Make A Difference. https://app.cochat.ai/papers/10.1145/3613904.3642513
The paper
Tanprasert, T., Fels, S. S., Sinnamon, L., & Yoon, D. (2024). Debate Chatbots to Facilitate Critical Thinking on YouTube: Social Identity and Conversational Style Make A Difference. Proceedings of the CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3613904.3642513

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.