Debate Chatbots to Facilitate Critical Thinking on YouTube: Social Identity and Conversational Style Make A Difference
The DOI matches its Crossref record; 19 of the 19 references it cites were found in a registry or an index.
Abstract
Exposure to diverse perspectives is helpful for bursting the filter bubble in online public video platforms. The recent advancement of Large Language Models (LLMs) illuminates the potential of creating a debate chatbot that prompts users to critically examine their stances on a topic formed by watching videos. However, whether the viewer is influenced by the chatbot may depend on its persona. In this paper, we investigated the effect of two relevant persona attributes - social identity and rhetorical styles - on critical thinking. In a mixed-methods study (n=36), we found that chatbots with outgroup (vs. ingroup) identity (t(33)=-2.33, p=0.03) and persuasive (vs. eristic) rhetoric (t(44)=1.98, p=0.05) induced critical thinking most effectively, making participants re-examine their arguments. However, participants’ stances remain largely unaffected, likely due to the chatbot’s lack of contextual knowledge and human touch. Our paper provides empirical groundwork for designing chatbot persona for remedying filter bubbles in online communities.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ChatGPT for Good? On Opportunities and Challenges of Large Language Models for Education
Kasneci, E., Seßler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., et al. (2023) · doi:10.35542/osf.io/5er8f · Cited by 7,552
Bartneck, C., Kulić, D., Croft, E., & Zoghbi, S. (2009) · International Journal of Social Robotics · doi:10.1007/s12369-008-0001-3 · Cited by 3,193 · cited 1×
“Finally, the perception of chatbot questionnaire is adapted from the embodied conversation agents evaluation framework [20] and the measurement instruments of the five key concepts of human-robot interactions [6].”
Sallam, M. (2023) · Healthcare · doi:10.3390/healthcare11060887 · Cited by 3,126 · cited 1×
“Such phenomena have been observed in many ChatGPT applications in academia [3, 90, 93].”
Analyzing and Interpreting Data From Likert-Type Scales
Sullivan, G. M., & Artino, A. R. (2013) · Journal of Graduate Medical Education · doi:10.4300/jgme-5-4-18 · Cited by 3,044
The filter bubble: how the new personalized web is changing what we read and how we think
(2012) · Choice Reviews Online · doi:10.5860/choice.50-0926 · Cited by 1,319
Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate
Liang, T., He, Z., Jiao, W., Wang, X., Wang, Y., Wang, R., Yang, Y., Shi, S., & Tu, Z. (2024) · Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2024.emnlp-main.992 · Cited by 1,062
Factors affecting YouTube influencer marketing credibility: a heuristic-systematic model
Xiao, M., Wang, R., & Chan-Olmsted, S. (2018) · Journal of Media Business Studies · doi:10.1080/16522354.2018.1501146 · Cited by 497
Dissociating language and thought in large language models
Mahowald, K., Ivanova, A. A., Blank, I. A., Kanwisher, N., Tenenbaum, J. B., & Fedorenko, E. (2024) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2024.01.011 · Cited by 345
Convergent and divergent responses to emotional displays of ingroup and outgroup.
van der Schalk, J., Fischer, A., Doosje, B., Wigboldus, D., Hawk, S., Rotteveel, M., & Hess, U. (2011) · Emotion · doi:10.1037/a0022582 · Cited by 275
Safe spaces in online places: social media and LGBTQ youth
Lucero, L. (2017) · Multicultural Education Review · doi:10.1080/2005615x.2017.1313482 · Cited by 195 · cited 1×
“Viewers may struggle to find a trustworthy and respectful discussion partner with opposing views and may feel apprehensive of offending the partners when discussing sensitive or controversial topics, such as political movements and trendy social issues [7, 45, 67].”
Background
Research works printed but not linked: 88
- 10.18653/v1/2020.acl-main.632 · DOI
- Hussam Alkaissi and Samy I McFarlane. 2023. Artificial hallucinations in ChatGPT: implications in scientific writing. Cureus 15, 2 (2023).
- Christa SC Asterhan and Rakheli Hever. 2015. Learning from reading argumentive group discussions in Facebook: Rhetoric style matters (again). Computers in human behavior 53 (2015), 570–576.
- 10.1145/3449230 · DOI
- David Bawden 2008. Origins and concepts of digital literacy. Digital literacies: Concepts, policies and practices 30, 2008 (2008), 17–32.
- 10.1177/0146167210376059 · DOI · Paper page
- Kevin M Blasiak, Marten Risius, and Sabine Matook. 2021. “Social Bots for Peace”: A Dual-Process Perspective to Counter Online Extremist Messaging. Association for Information Systems.
- Kristen Bloom and Kelly Marie Johnston. 2010. Digging into YouTube videos: Using media literacy and participatory culture to promote cross-cultural understanding. Journal of Media Literacy Education 2, 2 (2010), 3.
- Virginia Braun and Victoria Clarke. 2006. Using thematic analysis in psychology. Qualitative research in psychology 3, 2 (2006), 77–101.
- Pablo Brinol and Richard E Petty. 2009. Source factors in persuasion: A self-validation approach. European review of social psychology 20, 1 (2009), 49–96.
- Megan A Brown, James Bisbee, Angela Lai, Richard Bonneau, Jonathan Nagler, and Joshua A Tucker. 2022. Echo Chambers, Rabbit Holes, and Algorithmic Bias: How YouTube Recommends Content to Real Users. Available at SSRN 4114905 (2022).
- 10.1515/opis-2020-0007 · DOI
- Katarzyna Budzynska. 2010. Argument analysis: Components of interpersonal argumentation. In Computational Models of Argument. IOS Press, 135–146.
- 10.1108/09600030710825694 · DOI · Paper page
- Richard Catrambone, John Stasko, and Jun Xiao. 2004. ECA as User Interface Paradigm: Experimental findings within a framework for research. From brows to trust: Evaluating embodied conversational agents (2004), 239–267.
- 10.18653/v1/2023.eacl-main.53 · DOI
- Guo Chen, Yin-Dong Zheng, Jiahao Wang, Jilan Xu, Yifei Huang, Junting Pan, Yi Wang, Yali Wang, Yu Qiao, Tong Lu, 2023. VideoLLM: Modeling Video Sequence with Large Language Models. arXiv preprint arXiv:2305.13292 (2023). · arXiv
- Juliet M Corbin and Anselm Strauss. 1990. Grounded theory research: Procedures, canons, and evaluative criteria. Qualitative sociology 13, 1 (1990), 3–21.
- Abidin Crystal. 2018. Gay, famous and working hard on YouTube: Influencers, queer microcelebrity publics and discursive activism. In Youth, sexuality and sexual citizenship. Routledge, 217–231.
- Lincoln Dahlberg. 2001. The Internet and democratic discourse: Exploring the prospects of online deliberative forums extending the public sphere. Information, communication & society 4, 4 (2001), 615–633.
- Daniela Dahle 2022. Adolescents YouTube and Media Literacy A mixed-method time series study of Norwegian youth’s social media usage and its implications for media literacy. Master’s thesis. OsloMet-Storbyuniversitetet.
- Jakob Sebastian Davis. 2022. Video Essays Academia and Remediation: How YouTube Video Essayists Refashion Media and Scholarship. Ph. D. Dissertation. Texas A&M University-Central Texas.
- 10.1145/3267305.3274189 · DOI
- D Christopher Dryer. 1999. Getting personal with computers: how to design personalities for agents. Applied artificial intelligence 13, 3 (1999), 273–295.
- 10.1386/eta.4.3.247_1 · doi:10.1386/eta.4.3.247_1
- 10.1145/3308558.3313676 · DOI
- 10.1145/344949.345004 · DOI · Paper page
- 10.1145/1753326.1753502 · DOI · Paper page
- Kathleen C Fraser, Svetlana Kiritchenko, and Isar Nejadgholi. 2023. Diversity is not a one-way street: Pilot study on ethical interventions for racial bias in text-to-image systems. ICCV, accepted (2023).
- Sebastian Fritz-Morgenthal, Bernhard Hein, and Jochen Papenbrock. 2022. Financial risk management and explainable, trustworthy, responsible AI. Frontiers in artificial intelligence 5 (2022), 779799.
- 10.1145/3172944.3172970 · DOI
- Frédéric Guay, Robert J Vallerand, and Céline Blanchard. 2000. On the assessment of situational intrinsic and extrinsic motivation: The Situational Motivation Scale (SIMS). Motivation and emotion 24 (2000), 175–213.
- 10.1007/s11518-021-5497-1 · DOI
- Dale Hample. 2003. Arguing skill. Handbook of communication and social interaction skills (2003), 439–478.
- John Hartley. 2009. Uses of YouTube: Digital literacy and the growth of knowledge. YouTube: Online video and participatory culture (2009), 126–143.
- 10.1145/3544548.3581328 · DOI
- Susan Herring, Kirk Job-Sluder, Rebecca Scheckler, and Sasha Barab. 2002. Searching for safety online: Managing" trolling" in a feminist forum. The information society 18, 5 (2002), 371–384.
- Michael A Hogg. 2016. Social identity theory. Springer.
- 10.1145/3392854 · DOI · Paper page
- 10.1007/s10726-021-09765-8 · DOI
- 10.1163/25900110-00101002 · DOI
- 10.1111/bjop.12563 · DOI
- 10.1080/10714410903482658 · DOI
- 10.1145/3449140 · DOI · Paper page
- 10.1016/j.chb.2020.106419 · DOI · Paper page
- Tibor Koltay. 2011. The media and the literacies: Media literacy, information literacy, digital literacy. Media, culture & society 33, 2 (2011), 211–221.
- Aaron Kuecker. 2014. Ethnicity and social identity. T&T Clark handbook to social idenfity in the New Testament (2014), 59–78.
- Brenda Laurel and S Joy Mountford. 1990. The art of human-computer interface design. Addison-Wesley Longman Publishing Co., Inc.
- Mark Ledwich, Anna Zaitsev, and Anton Laukemper. 2022. Radical bubbles on YouTube? Revisiting algorithmic extremism with personalised recommendations. First Monday (2022).
- Daniel Z Levin, Ellen M Whitener, and Rob Cross. 2006. Perceived trustworthiness of knowledge sources: The moderating impact of relationship length.Journal of applied psychology 91, 5 (2006), 1163.
- 10.1177/1527476419879919 · DOI · Paper page
- 10.1145/3544548.3580953 · DOI · Paper page
- Patricia W Linville, Gregory W Fischer, and Carolyn Yoon. 1996. Perceived covariation among the features of ingroup and outgroup members: The outgroup covariation effect.Journal of Personality and Social Psychology 70, 3 (1996), 421.
- 10.1145/3411764.3445467 · DOI · Paper page
- Carol B MacKnight. 2000. Teaching critical thinking through online discussions. Educause Quarterly 23, 4 (2000), 38–41.
- Nila Masrourisaadat. 2023. Quantitative and Qualitative Analysis of Text-to-Image models. Ph. D. Dissertation. Virginia Tech.
- Phaedra S Mohammed and Eleanor ‘Nell’Watson. 2019. Towards inclusive education in the age of artificial intelligence: Perspectives challenges and opportunities. Artificial Intelligence and Inclusive Education: Speculative futures and emerging practices (2019) 17–37. · DOI
- Fabio Motoki, Valdemar Pinho Neto, and Victor Rodrigues. 2023. More human than human: Measuring ChatGPT political bias. Public Choice (2023), 1–21.
- 10.1145/3544549.3573844 · DOI
- Sean Munson, Stephanie Lee, and Paul Resnick. 2013. Encouraging reading of diverse political viewpoints with a browser widget. In Proceedings of the international AAAI conference on web and social media, Vol. 7. 419–428.
- 10.1891/1061-3749.25.1.156 · DOI
- Vaibhavi Nandagiri and Leena Philip. 2018. Impact of influencers from Instagram and YouTube on their followers. International Journal of Multidisciplinary Research and Modern Education 4, 1 (2018), 61–65.
- 10.1145/3359243 · DOI · Paper page
- 10.1145/2566486.2568012 · DOI · Paper page
- 10.1371/journal.pone.0177865 · DOI
- 10.3390/educsci12120938 · DOI
- 10.2501/JAR-2023-008 · DOI
- 10.1177/0146167213499937 · DOI · Paper page
- 10.1177/1354856517736983 · DOI · Paper page
- Eun Rhee, James S Uleman, and Hoon Koo Lee. 1996. Variations in collectivism and individualism by ingroup and culture: Confirmatory factor analysis.Journal of Personality and Social Psychology 71, 5 (1996), 1037.
- Jürgen Rudolph, Samson Tan, and Shannon Tan. 2023. ChatGPT: Bullshit spewer or the end of traditional assessments in higher education?Journal of Applied Learning and Teaching 6, 1 (2023).
- Mustafa Safdari, Greg Serapio-García, Clément Crepy, Stephen Fitz, Peter Romero, Luning Sun, Marwa Abdulhai, Aleksandra Faust, and Maja Matarić. 2023. Personality traits in large language models. arXiv preprint arXiv:2307.00184 (2023). · arXiv
- 10.1109/TTS.2023.3257627 · DOI · Paper page
- Henri Tajfel and John C Turner. 2004. The social identity theory of intergroup behavior. In Political psychology. Psychology Press, 276–293.
- 10.34190/ecsm.9.1.106 · DOI
- Martin Tanis and Tom Postmes. 2005. A social identity approach to trust: Interpersonal perception, group membership and trusting behaviour. European journal of social psychology 35, 3 (2005), 413–424.
- 10.1145/3544548.3581153 · DOI
- 10.1016/j.lcsi.2019.03.009 · DOI · Paper page
- Achmad Tohe. 2021. YouTube, Learning, and Transformative Critical Pedagogy. KnE Social Sciences (2021), 15–29.
- Douglas Walton. 2009. Burden of proof in deliberation dialogs. In International workshop on argumentation in multi-agent systems. Springer, 1–22.
- Douglas Walton and Erik CW Krabbe. 1995. Commitment in dialogue: Basic concepts of interpersonal reasoning. SUNY press.
- 10.1007/BF00128147 · DOI · Paper page
- WM Westenberg. 2016. The influence of YouTubers on teenagers: a descriptive research about the role YouTubers play in the life of their teenage viewers. Master’s thesis. University of Twente.
- Matti Wiegmann, Khalid Al Khatib, Vishal Khanna, and Benno Stein. 2022. Analyzing Persuasion Strategies of Debaters on Social Media. In 29th International Conference on Computational Linguistics. International Committee on Computational Linguistics, 6897–6905.
- Wendy Wood. 2000. Attitude change: Persuasion and social influence. Annual review of psychology 51, 1 (2000), 539–570.
- Brahim Zarouali, Mykola Makhortykh, Mariella Bastian, and Theo Araujo. 2021. Overcoming polarization with chatbot news? Investigating the impact of news content containing opposing views on agreement and credibility. European journal of communication 36, 1 (2021), 53–68.
- 10.1016/j.paid.2020.110303 · DOI
- Daniel Zimmermann, Christian Noll, Lars Gräßer, Kai-Uwe Hugger, Lea Marie Braun, Tine Nowak, and Kai Kaspar. 2020. Influencers on YouTube: a quantitative study on young people’s use and perception of videos about political and societal topics. Current Psychology (2020), 1–17.
Cited by
Lee, H.-P. (., Sarkar, A., Tankelevitch, L., Drosos, I., Rintel, S., Banks, R., & Wilson, N. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713778 · Cited by 490
Mei, P., Brewis, D. N., Nwaiwu, F., Sumanathilaka, D., Alva-Manchego, F., & Demaree-Cotton, J. (2025) · Computers in Human Behavior: Artificial Humans · doi:10.1016/j.chbah.2025.100140 · Cited by 50
“…(Danry et al., 2023) or forcing a ‘pause ’ (Buçinca et al., 2021) can improve elements of critical thinking, whether LLMs can be used to promote reflection (Du et al., 2024), or whether LLM ’ s can be used as part of pedagogical tools (Lee et al., 2023; Tanprasert et al., 2024; Yuan et al., 2023).”
Shen, J. J., King Chen, J., Findlater, L., & Dietz Smith, G. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713405 · Cited by 24
Metadata
- Authors
- Tanprasert, T., Fels, S. S., Sinnamon, L., & Yoon, D.
- Venue
- Proceedings of the CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2024-05-11
- Pages
- 1-24
- OpenAlex
- W4396832241
- Cited by
- 51
- References
- 19
- FWCI
- 35.28
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Debate Chatbots to Facilitate Critical Thinking on YouTube: Social Identity and Conversational Style Make A Difference. https://app.cochat.ai/papers/10.1145/3613904.3642513
- The paper
- Tanprasert, T., Fels, S. S., Sinnamon, L., & Yoon, D. (2024). Debate Chatbots to Facilitate Critical Thinking on YouTube: Social Identity and Conversational Style Make A Difference. Proceedings of the CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3613904.3642513
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.