Towards Mutual Theory of Mind in Human-AI Interaction: How Language Reflects What Students Perceive About a Virtual Teaching Assistant
The DOI matches its Crossref record; 98 of the 98 references it cites were found in a registry or an index.
Abstract
Building conversational agents that can conduct natural and prolonged conversations has been a major technical and design challenge, especially for community-facing conversational agents. We posit Mutual Theory of Mind as a theoretical framework to design for natural long-term human-AI interactions. From this perspective, we explore a community’s perception of a question-answering conversational agent through self-reported surveys and computational linguistic approach in the context of online education. We first examine long-term temporal changes in students’ perception of Jill Watson (JW), a virtual teaching assistant deployed in an online class discussion forum. We then explore the feasibility of inferring students’ perceptions of JW through linguistic features extracted from student-JW dialogues. We find that students’ perception of JW’s anthropomorphism and intelligence changed significantly over time. Regression analyses reveal that linguistic verbosity, readability, sentiment, diversity, and adaptability reflect student perception of JW. We discuss implications for building adaptive community-facing conversational agents as long-term companions and designing towards Mutual Theory of Mind in human-AI interaction.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Educating end users to spot what could potentially be AI-generated will empower them to establish an understanding of AI tools [67,109] and develop individual-specifc strategies to evolve to be more informed and mindful information consumers.”
From Synthetic Lies: Understanding AI-Generated Misinformation and Evaluating Algorithmic and Human Solutions (2023) · cites this paper as [109]
In the citing paper
“prior CHI studies [43, 50, 77, 79, 93] have investigated potential scenarios of interacting with multi-party based chatbots [77] as well as diverse roles of chatbots and their positive impact on familiarization among teammates [79], generating more ideas and diverse views [43, 50, 79], and nudging silent people to speak up [43].”
From The Promise and Peril of ChatGPT in Higher Education: Opportunities, Challenges, and Design Implications (2024) · cites this paper as [93]
In the citing paper
“Design guidelines can be difficult to put into practice [59, 113, 160, 164, 176, 192], often because they describe goals rather than actions [73].”
From Design Principles for Generative AI Applications (2024) · cites this paper as [176]
Citations
Cites
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071 · cited 1×
“Word embeddings represent words as vectors in a higher dimensional latent space, where lexico-semantically similar words tend to have vectors that are closer [21, 65, 74].”
BackgroundThe Presentation of Self in Everyday Life.
Naegele, K. D., & Goffman, E. (1956) · American Sociological Review · doi:10.2307/2089106 · Cited by 25,991
Does the autistic child have a “theory of mind” ?
Baron-Cohen, S., Leslie, A. M., & Frith, U. (1985) · Cognition · doi:10.1016/0010-0277(85)90022-8 · Cited by 8,599 · cited 8×
“Scholars posit that the Theory of Mind (ToM) is a basic cognitive and social characteristic that enables us to make conjectures about each others’ minds through observable or latent behavioral and verbal cues [6, 12, 37, 38, 94].”
BackgroundInfluentialVADER: a parsimonious rule-based model for sentiment analysis of social media text
Hutto, C., & Gilbert, E. (2014) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v8i1.14550 · Cited by 6,087 · cited 1×
“To measure the sentiment of each post to JW, we used the VADER sentiment analysis model [48], which is a rule-based sentiment analysis model that provides numerical scores ranging from -1”
MethodPresentation of Self in Everyday Life, the
(2008) · Encyclopedia of Political Communication · doi:10.4135/9781412953993.n530 · Cited by 3,771
Mindblindness: An Essay on Autism and Theory of Mind
(1995) · Choice Reviews Online · doi:10.5860/choice.33-2412 · Cited by 3,670 · cited 2×
“For example, building shared plans and goals is fundamental to collaborative task-completion— ToM enables us to recognize and mitigate each other’s plans and goals to work together [5, 6]; intentional communication is the basis for smooth communication— ToM enables us to understand the interlocutor has a belief or kno…”
BackgroundThe Uncanny Valley [From the Field]
Mori, M., MacDorman, K., & Kageki, N. (2012) · IEEE Robotics & Automation Magazine · doi:10.1109/mra.2012.2192811 · Cited by 3,542 · cited 1×
“Anthropomorphism is a widely studied yet highly debatable design characteristic of CA—on one hand, intentionally building CAs with more humanlike attributes can improve user trust[13, 32], make the CAmore approachable and ease user interactions [54, 97]; on the other hand, the famous “Uncanny Valley” effect [66] indi-”
BackgroundMindblindness: An Essay on Autism and Theory of Mind
Baron-Cohen, S. (1995) · doi:10.7551/mitpress/4635.001.0001 · Cited by 3,428
Bartneck, C., Kulić, D., Croft, E., & Zoghbi, S. (2009) · International Journal of Social Robotics · doi:10.1007/s12369-008-0001-3 · Cited by 3,193 · cited 6×
“Even though building artifcially “intelligent” machines has been a unfulflled promise due to various technical and feasibility challenges [8, 87], users tend to expect their CAs to be “smart” [98], thus creating a gap between user expectation and CA’s true intelligence.”
BackgroundMethodInfluentialGuidelines for Human-AI Interaction
Amershi, S., Weld, D., Vorvoreanu, M., Fourney, A., Nushi, B., Collisson, P., Suh, J., Iqbal, S., Bennett, P. N., Inkpen, K., et al. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300233 · Cited by 2,439 · cited 2×
“While CAs are relatively successful in task-oriented interactions [82, 96], the initial promise of building CAs that can carry out natural and coherent conversations with users has largely remained unfulflled due to both design and technical challenges [3, 18, 87].”
BackgroundNass, C., Steuer, J., & Tauber, E. R. (1994) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/191666.191703 · Cited by 2,148 · cited 1×
“Since people often treat computers as social actors [8, 70], perceived likeability is a potential factor that could infuence long-term relationship-building.”
Linear models in decision making.
Dawes, R. M., & Corrigan, B. (1974) · Psychological Bulletin · doi:10.1037/h0037613 · Cited by 1,875 · cited 1×
“Linear regression is known to help interpret conditionally monotone relationships with the dependent variable [22].”
BackgroundLuger, E., & Sellen, A. (2016) · Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858288 · Cited by 1,176 · cited 7×
“It is also well-established in the literature that people often have unrealistically high expectations for CAs [61, 98].”
BackgroundInfluentialWhy the child’s theory of mind really is a theory
GOPNIK, A., & WELLMAN, H. M. (1992) · Mind & Language · doi:10.1111/j.1468-0017.1992.tb00202.x · Cited by 1,158 · cited 1×
“Scholars posit that the Theory of Mind (ToM) is a basic cognitive and social characteristic that enables us to make conjectures about each others’ minds through observable or latent behavioral and verbal cues [6, 12, 37, 38, 94].”
BackgroundGo, E., & Sundar, S. S. (2019) · Computers in Human Behavior · doi:10.1016/j.chb.2019.01.020 · Cited by 1,089 · cited 1×
“Anthropomorphism is a widely studied yet highly debatable design characteristic of CA—on one hand, intentionally building CAs with more humanlike attributes can improve user trust[13, 32], make the CAmore approachable and ease user interactions [54, 97]; on the other hand, the famous “Uncanny Valley” effect [66] indi-”
BackgroundGrimm, P. (2010) · Wiley International Encyclopedia of Marketing · doi:10.1002/9781444316568.wiem02057 · Cited by 1,054
Using linguistic cues for the automatic recognition of personality in conversation and text
Mairesse, F., Walker, M. A., Mehl, M. R., & Moore, R. K. (2007) · Journal of Artificial Intelligence Research · doi:10.1613/jair.2349 · Cited by 1,046 · cited 1×
“Researchers have inferred users’ emotions towards the agent [90], personality traits [62], signs of conversation breakdowns [58, 99], politeness [20] from conversational cues.”
BackgroundDesigning Theory-Driven User-Centric Explainable AI
Wang, D., Yang, Q., Abdul, A., & Lim, B. Y. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300831 · Cited by 906 · cited 1×
“understanding of how their textual responses would be parsed by the CA to extract perceptions about their goals and intentions [91].”
BackgroundAleven, V. A. W. M. M., & Koedinger, K. R. (2002) · Cognitive Science · doi:10.1207/s15516709cog2602_1 · Cited by 895 · cited 2×
“learning, where community-facing CAs are commonly seen to provide informational and social support to student communities [1, 92, 95].”
BackgroundHill, J., Randolph Ford, W., & Farreras, I. G. (2015) · Computers in Human Behavior · doi:10.1016/j.chb.2015.02.026 · Cited by 856 · cited 7×
“In fact, prior work suggests that we are able to adapt our speech pattern accordingly based on whether the interlocutor is a human or a CA [45], suggesting that adaptability of our speech pattern could be an indicator of our perception of”
BackgroundMethodResultInfluentialMcKight, P. E., & Najab, J. (2010) · The Corsini Encyclopedia of Psychology · doi:10.1002/9780470479216.corpsy0491 · Cited by 802
Mindblindness—an essay on autism and theory of mind
Wilson, G. (1996) · Personality and Individual Differences · doi:10.1016/s0191-8869(96)80006-7 · Cited by 802
Carruthers, P., & Smith, P. K. (1996) · Cited by 718
In the shades of the uncanny valley: An experimental study of human–chatbot interaction
Ciechanowski, L., Przegalinska, A., Magnuski, M., & Gloor, P. (2019) · Future Generation Computer Systems · doi:10.1016/j.future.2018.01.055 · Cited by 651 · cited 2×
“negative feelings towards the CA [17] as well as setting unrealistic user expectations on CA’s capabilities [61].”
BackgroundAlmost human: Anthropomorphism increases trust resilience in cognitive agents.
de Visser, E. J., Monfort, S. S., McKendrick, R., Smith, M. A. B., McKnight, P. E., Krueger, F., & Parasuraman, R. (2016) · Journal of Experimental Psychology: Applied · doi:10.1037/xap0000092 · Cited by 614
A New Chatbot for Customer Service on Social Media
Xu, A., Liu, Z., Guo, Y., Sinha, V., & Akkiraju, R. (2017) · Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3025496 · Cited by 550 · cited 1×
“While CAs are relatively successful in task-oriented interactions [82, 96], the initial promise of building CAs that can carry out natural and coherent conversations with users has largely remained unfulflled due to both design and technical challenges [3, 18, 87].”
BackgroundHughes, C., & Leekam, S. (2004) · Social Development · doi:10.1111/j.1467-9507.2004.00285.x · Cited by 505 · cited 1×
“ToM has been extensively studied and continues to be a leading influence inmany fields such as cognitive science, developmental psychology, and autism research [5, 7, 47, 78].”
BackgroundWhat Makes a Good Conversation?: Challenges in Designing Truly Conversational Agents
Clark, L., Pantidi, N., Cooney, O., Doyle, P., Garaialde, D., Edwards, J., Spillane, B., Gilmartin, E., Murad, C., Munteanu, C., et al. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300705 · Cited by 461 · cited 1×
“While CAs are relatively successful in task-oriented interactions [82, 96], the initial promise of building CAs that can carry out natural and coherent conversations with users has largely remained unfulflled due to both design and technical challenges [3, 18, 87].”
BackgroundMori, M. (2020) · The Monster Theory Reader · doi:10.5749/j.ctvtv937f.7 · Cited by 433
Reflexively mindblind: Using theory of mind to interpret behavior requires effortful attention
Lin, S., Keysar, B., & Epley, N. (2010) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2009.12.019 · Cited by 412 · cited 2×
“Robots built with ToM have demonstrated positive outcomes in team operations [25, 44], collaborative decision-making [39], and are perceived to be more natural and intelligent [59].”
BackgroundDesigning robots for long-term social interaction
Gockley, R., Bruce, A., Forlizzi, J., Michalowski, M., Mundell, A., Rosenthal, S., Sellner, B., Simmons, R., Snipes, K., Schultz, A. C., et al. (2005) · 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems · doi:10.1109/iros.2005.1545303 · Cited by 407 · cited 2×
“CA responses have been shown to infuence user’s repair strategies [9] and foster engaging conversations [14, 33].”
BackgroundHAPPÉ, F. (1996) · Mind & Language · doi:10.1111/j.1468-0017.1996.tb00057.x · Cited by 392
Revisiting readability: a unified framework for predicting text quality
Pitler, E., & Nenkova, A. (2008) · Proceedings of the Conference on Empirical Methods in Natural Language Processing - EMNLP '08 · doi:10.3115/1613715.1613742 · Cited by 375 · cited 1×
“8, in which L is the average number of letters per 100 words and S is the average number of sentences per 100 words [77].”
BackgroundAlthoff, T., Clark, K., & Leskovec, J. (2016) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00111 · Cited by 322 · cited 1×
“We draw on prior work [2, 84] to obtain linguistic diversity, and use word embeddings for this purpose.”
MethodIJCAI-83: International joint conference on artificial intelligence
(1982) · Artificial Intelligence · doi:10.1016/0004-3702(82)90046-7 · Cited by 316
Communication Breakdowns Between Families and Alexa
Beneteau, E., Richards, O. K., Zhang, M., Kientz, J. A., Yip, J., & Hiniker, A. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300473 · Cited by 313 · cited 4×
“CA responses have been shown to infuence user’s repair strategies [9] and foster engaging conversations [14, 33].”
BackgroundInfluentialExternal manifestations of trustworthiness in the interface
Cassell, J., & Bickmore, T. (2000) · Communications of the ACM · doi:10.1145/355112.355123 · Cited by 255 · cited 1×
“Anthropomorphism is a widely studied yet highly debatable design characteristic of CA—on one hand, intentionally building CAs with more humanlike attributes can improve user trust[13, 32], make the CA more approachable and ease user interactions [54, 97];”
BackgroundNegotiated Collusion: Modeling Social Language and its Relationship Effects in Intelligent Agents
Cassell, J., & Bickmore, T. (2003) · User Modeling and User-Adapted Interaction · doi:10.1023/a:1024026532471 · Cited by 253 · cited 1×
“CA responses have been shown to influence user’s repair strategies [9] and foster engaging conversations [14, 33].”
BackgroundCONSTRUCTION OF A NEW SCALE: THE REYSEN LIKABILITY SCALE
Reysen, S. (2005) · Social Behavior and Personality: an international journal · doi:10.2224/sbp.2005.33.2.201 · Cited by 251 · cited 1×
“In human interactions, likeability has been suggested to induce positive afect, increase persuasiveness, and foster favorable perceptions [76, 80].”
BackgroundResilient Chatbots: Repair Strategy Preferences for Conversational Breakdowns
Ashktorab, Z., Jain, M., Liao, Q. V., & Weisz, J. D. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300484 · Cited by 224 · cited 1×
“This would also help alleviate the current one-sided communication burden on users, who had to constantly adjust their mental model of the CA through an arbitrary trial-and-error process to elicit desired CA responses [4, 9].”
BackgroundLee, S., Lee, N., & Sah, Y. J. (2020) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2019.1699748 · Cited by 204 · cited 1×
“Consequently, researchers and designers turn to traditional HCI design guidelines intended for Graphical User Interfaces, which is not always the optimal perspective to look at designing the interactions between humans and often anthropomorphized CAs [89]— researchers and designers face major obstacles in balancing un…”
BackgroundI'm sad you're sad: emotional contagion in CMC
Hancock, J. T., Gee, K., Ciaccio, K., & Lin, J. M.-H. (2008) · Proceedings of the 2008 ACM conference on Computer supported cooperative work · doi:10.1145/1460563.1460611 · Cited by 189 · cited 1×
“, unsatisfactory interaction with one individual might also frustrate others [42].”
BackgroundHolstein, K., McLaren, B. M., & Aleven, V. (2019) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-23204-7_14 · Cited by 189 · cited 2×
“However, while these CAs have been found to be efcient in facilitating students’ learning outcome, how students perceive these CAs in the online classroom is barely explored—students’ perceptions of the CA could infuence students’ interaction experience with the CA and thus potentially impacting their online learning…”
BackgroundMark My Words! Linguistic Style Accommodation in Social Media
Danescu-Niculescu-Mizil, C., Gamon, M., & Dumais, S. (2011) · arXiv (Cornell University) · doi:10.48550/arxiv.1105.0673 · Cited by 185 · cited 1×
“guistic cues on public online platforms, depending on the perceived audience [19, 49, 83].”
BackgroundAt Your Service: Designing Voice Assistant Personalities to Improve Automotive User Interfaces
Braun, M., Mainz, A., Chadowitz, R., Pfleging, B., & Alt, F. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300270 · Cited by 182 · cited 1×
“Whereas humans tend to leverage various social techniques such as altering appearances and manners to make certain impressions [36], embodied virtual agents and robots can change facial expressions or physical behaviors [73], voice-based conversation assistants can change the tone of their voice [11], text-based langu…”
BackgroundWinkler, R., Hobert, S., Salovaara, A., Söllner, M., & Leimeister, J. M. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376781 · Cited by 177 · cited 3×
“learning, where community-facing CAs are commonly seen to provide informational and social support to student communities [1, 92, 95].”
BackgroundQuizBot: A Dialogue-based Adaptive Learning System for Factual Knowledge
Ruan, S., Jiang, L., Xu, J., Tham, B. J.-K., Qiu, Z., Zhu, Y., Murnane, E. L., Brunskill, E., & Landay, J. A. (2019) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300587 · Cited by 176 · cited 1×
“While CAs are relatively successful in task-oriented interactions [82, 96], the initial promise of building CAs that can carry out natural and coherent conversations with users has largely remained unfulflled due to both design and technical challenges [3, 18, 87].”
BackgroundConversations Gone Awry: Detecting Early Signs of Conversational Failure
Zhang, J., Chang, J., Danescu-Niculescu-Mizil, C., Dixon, L., Hua, Y., Taraborelli, D., & Thain, N. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1125 · Cited by 175 · cited 1×
“Researchers have inferred users’ emotions towards the agent [90], personality traits [62], signs of conversation breakdowns [58, 99], politeness [20] from conversational cues.”
BackgroundAn implemented theory of mind to improve human-robot shared plans execution
Devin, S., & Alami, R. (2016) · ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1109/hri.2016.7451768 · Cited by 167 · cited 2×
“Specifically, ToM has been intentionally built in as an individual module of the system architecture to help robots monitor world state as well as the human state [25], construct simulation of hypothetical cognitive models of the human partner to account for human behaviors that deviate from original plans [44, 79], a…”
BackgroundI'm Sorry, Dave, I'm Afraid I Can't Do That: Chatbot Perception and Expectations
Zamora, J. (2017) · Proceedings of the 5th International Conference on Human Agent Interaction · doi:10.1145/3125739.3125766 · Cited by 164
The Bag of Communities: Identifying Abusive Behavior Online with Preexisting Internet Data
Chandrasekharan, E., Samory, M., Srinivasan, A., & Gilbert, E. (2017) · Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3026018 · Cited by 153 · cited 1×
“moderated, can not only have diverse and deviant discussions but can also include informal languages [15].”
BackgroundGoel, A. K., & Polepeddi, L. (2018) · Learning Engineering for Online Education · doi:10.4324/9781351186193-7 · Cited by 146
Self-citationKim, S., Eun, J., Oh, C., Suh, B., & Lee, J. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376785 · Cited by 133 · cited 2×
“these community-facing CAs face two unique challenges: the need to carry out smooth dyadic interactions with individual community members, and the need to respond accordingly based on the community’s shifting perceptions [53, 86].”
BackgroundWhat Makes a Good Conversation? Challenges in Designing Truly Conversational Agents
Clark, L., Pantidi, N., Cooney, O., Doyle, P. R., Garaialde, D., Edwards, J., Spillane, B., Murad, C., Munteanu, C., Wade, V., et al. (2019) · Cork Open Research Archive (University College Cork, Ireland) · doi:10.48550/arxiv.1901.06525 · Cited by 128 · cited 1×
“While CAs are relatively successful in task-oriented interactions [82, 96], the initial promise of building CAs that can carry out natural and coherent conversations with users has largely remained unfulflled due to both design and technical challenges [3, 18, 87].”
BackgroundMental Models of AI Agents in a Cooperative Game Setting
Gero, K. I., Ashktorab, Z., Dugan, C., Pan, Q., Johnson, J., Geyer, W., Ruiz, M., Miller, S., Millen, D. R., Campbell, M., et al. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376316 · Cited by 121 · cited 7×
“Research has explored ways along the realm of identifying user perceptions of CAs to facilitate dyadic human-AI interactions, including examining an individual’s mental model of CAs in a variety of contexts [31, 54, 61].”
BackgroundResultInfluentialLiao, Q. V., Mas-ud Hussain, M., Chandar, P., Davis, M., Khazaeni, Y., Crasso, M. P., Wang, D., Muller, M., Shami, N. S., & Geyer, W. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173577 · Cited by 116
Accommodating human variability in human-robot teams through theory of mind
Hiatt, L. M., Harrison, A. M., & Trafton, J. G. (2011) · International Joint Conference on Artificial Intelligence · doi:10.5591/978-1-57735-516-8/ijcai11-345 · Cited by 115
Mark my words!: linguistic style accommodation in social media
Danescu-Niculescu-Mizil, C., Gamon, M., & Dumais, S. (2011) · Proceedings of the 20th international conference on World wide web · doi:10.1145/1963405.1963509 · Cited by 113 · cited 1×
“guistic cues on public online platforms, depending on the perceived audience [19, 49, 83].”
Background‘Realness’ in Chatbots: Establishing Quantifiable Criteria
Morrissey, K., & Kirakowski, J. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-39330-3_10 · Cited by 110
Subtleties of facial expressions in embodied agents
Pélachaud, C., & Poggi, I. (2002) · The Journal of Visualization and Computer Animation · doi:10.1002/vis.299 · Cited by 105 · cited 1×
“Whereas humans tend to leverage various social techniques such as altering appearances and manners to make certain impressions [36], embodied virtual agents and robots can change facial expressions or physical behaviors [73], voice-based conversation assistants can change the tone of their voice [11], text-based langu…”
BackgroundMurphy, N. A. (2007) · Personality and Social Psychology Bulletin · doi:10.1177/0146167206294871 · Cited by 102 · cited 3×
“Contradictory to observations related to human-human interactions [68], our observations suggest that people’s linguistic diversity does not necessarily indicate how intelligent one perceives an agent to be.”
BackgroundInfluentialLinguistic Markers Indicating Therapeutic Outcomes of Social Media Disclosures of Schizophrenia
Ernala, S. K., Rizvi, A. F., Birnbaum, M. L., Kane, J. M., & De Choudhury, M. (2017) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3134678 · Cited by 99 · cited 4×
“research to derive a variety of linguistic attributes (features) from the language interactions which include verbosity, readability, sentiment, diversity, and adaptability [27, 84].”
BackgroundInfluentialBeyond Dyadic Interactions: Considering Chatbots as Community Members
Seering, J., Luria, M., Kaufman, G., & Hammer, J. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300680 · Cited by 97 · cited 5×
“Even though building artifcially “intelligent” machines has been a unfulflled promise due to various technical and feasibility challenges [8, 87], users tend to expect their CAs to be “smart” [98], thus creating a gap between user expectation and CA’s true intelligence.”
BackgroundInfluentialUnderstanding Affective Experiences with Conversational Agents
Yang, X., Aurisicchio, M., & Baxter, W. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300772 · Cited by 95
Causal Factors of Effective Psychosocial Outcomes in Online Mental Health CommunitiesPreprint
Saha, K., & Sharma, A. (2020) · medRxiv · doi:10.1101/2020.08.15.20175836 · Cited by 92 · cited 6×
“Psycholinguistic literature values readability to be a key indicator of people’s cognitive behavior, and prior work has adapted this measure to understand conversational patterns in online communities [27, 84, 85].”
BackgroundMethodInfluentialSelf-citationMeasuring Service Encounter Satisfaction with Customer Service Chatbots using Sentiment Analysis
Feine, J., Morana, S., & Gnewuch, U. (2019) · WIRTSCHAFTSINFORMATIK · Cited by 87
An Implemented Theory of Mind to Improve Human-Robot Shared Plans Execution
Devin, S., & Alami, R. (2016) · doi:10.5555/2906831.2906886 · Cited by 77
The Social Roles of Bots: Evaluating Impact of Bots on Discussions in Online Communities
Seering, J., Flores, J. P., Savage, S., & Hammer, J. (2018) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3274426 · Cited by 77
Causal Factors of Effective Psychosocial Outcomes in Online Mental Health Communities
Saha, K., & Sharma, A. (2020) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v14i1.7326 · Cited by 72
Self-citationGarg, R., & Sengupta, S. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376631 · Cited by 68 · cited 1×
“However, the majority of the research on human perception of CAs uses qualitative methods to either identify the various dimensions of user perception of the agent [31, 67] or provides one-time assessments of user perception after interacting with the agent [30, 41, 51, 55, 88, 98].”
BackgroundUser attitude towards an embodied conversational agent: Effects of the interaction mode
Novielli, N., de Rosis, F., & Mazzotta, I. (2010) · Journal of Pragmatics · doi:10.1016/j.pragma.2009.12.016 · Cited by 58
Computer-based readability indexes
McCallum, D. R., & Peterson, J. L. (1982) · ACM Annual Conference · doi:10.1145/800174.809754 · Cited by 55 · cited 1×
“Readability refers to the level of ease readers can comprehend a given text [63].”
BackgroundSaha, K., Weber, I., & De Choudhury, M. (2018) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v12i1.15016 · Cited by 55 · cited 1×
“Psycholinguistic literature values readability to be a key indicator of people’s cognitive behavior, and prior work has adapted this measure to understand conversational patterns in online communities [27, 84, 85].”
BackgroundSelf-citationLearning Engineering for Online Education: Theoretical Contexts and Design-Based Examples
(2018) · Routledge eBooks · doi:10.4324/9781351186193 · Cited by 55
Choi, D., Kwak, D., Cho, M., & Lee, S. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376569 · Cited by 54 · cited 1×
“This fnding also provides more evidence to the long-standing debate of whether CAs should be designed as humanlike as possible [16, 17], suggesting that user’s preference of whether CAs should be humanlike is highly dependent on CA’s role and use contexts.”
BackgroundModeling Organizational Culture with Workplace Experiences Shared on Glassdoor
Das Swain, V., Saha, K., Reddy, M. D., Rajvanshy, H., Abowd, G. D., & De Choudhury, M. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376793 · Cited by 54 · cited 1×
“Word embeddings represent words as vectors in a higher dimensional latent space, where lexico-semantically similar words tend to have vectors that are closer [21, 65, 74].”
BackgroundSelf-citationSounding Board: A User-Centric and Content-Driven Social Chatbot
Fang, H., Cheng, H., Sap, M., Clark, E., Holtzman, A., Choi, Y., Smith, N. A., & Ostendorf, M. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Demonstrations · doi:10.18653/v1/n18-5020 · Cited by 51 · cited 1×
“Contrary to prior research and popular belief [28, 45, 60], our fndings suggest that students who used more number of unique words per post or more complex language tended to perceive JW as less human-like, less intelligent, and less likeable.”
ResultIt Takes a Village: Integrating an Adaptive Chatbot into an Online Gaming Community
Seering, J., Luria, M., Ye, C., Kaufman, G., & Hammer, J. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376708 · Cited by 49 · cited 3×
“However, the majority of the research on human perception of CAs uses qualitative methods to either identify the various dimensions of user perception of the agent [31, 67] or provides one-time assessments of user perception after interacting with the agent [30, 41, 51, 55, 88, 98].”
BackgroundJudgment of the Humanness of an Interlocutor Is in the Eye of the Beholder
Lortie, C. L., & Guitton, M. J. (2011) · PLoS ONE · doi:10.1371/journal.pone.0025085 · Cited by 47 · cited 2×
“Contrary to prior research and popular belief [28, 45, 60], our fndings suggest that students who used more number of unique words per post or more complex language tended to perceive JW as less human-like, less intelligent, and less likeable.”
BackgroundResultLee, S., Kim, S., & Lee, S. (2019) · Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290607.3312796 · Cited by 45 · cited 1×
“However, the majority of the research on human perception of CAs uses qualitative methods to either identify the various dimensions of user perception of the agent [31, 67] or provides one-time assessments of user perception after interacting with the agent [30, 41, 51, 55, 88, 98].”
BackgroundLibRA: On LinkedIn based Role Ambiguity and Its Relationship with Wellbeing and Job Performance
Saha, K., Reddy, M. D., Mattingly, S. M., Moskal, E., Sirigiri, A., & Choudhury, M. D. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359239 · Cited by 43 · cited 1×
“guistic cues on public online platforms, depending on the perceived audience [19, 49, 83].”
Self-citationCo-Performing Agent: Design for Building User-Agent Partnership in Learning and Adaptive Services
Kim, D.-J., & Lim, Y.-K. (2019) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300714 · Cited by 42 · cited 2×
“However, the majority of the research on human perception of CAs uses qualitative methods to either identify the various dimensions of user perception of the agent [31, 67] or provides one-time assessments of user perception after interacting with the agent [30, 41, 51, 55, 88, 98].”
BackgroundGenie in the Bottle: Anthropomorphized Perceptions of Conversational Agents
Kuzminykh, A., Sun, J., Govindaraju, N., Avery, J., & Lank, E. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376665 · Cited by 41 · cited 3×
“While there is a lack of evidence on how sentiment in wording can be associated with perceived intelligence, intelligence is one of the key desired characteristics people expect from a CA [54].”
BackgroundGörür, O. C., Rosman, B., Hoffman, G., & Albayrak, Ş. (2017) · Cited by 39
Jill Watson SA: Design and Evaluation of a Virtual Agent to Build Communities Among Online Learners
Wang, Q., Jing, S., Camacho, I., Joyner, D. A., & Goel, A. K. (2020) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3334480.3382878 · Cited by 39 · cited 2×
“seen to provide informational and social support to student communities [1, 92, 95].”
BackgroundSelf-citationModeling Agents with a Theory of Mind
Harbers, M., den Bosch, K. V., & Meyer, J. C. (2009) · IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology (WI-IAT) · doi:10.1109/wi-iat.2009.153 · Cited by 35 · cited 1×
“Specifcally, ToM has been intentionally built in as an individual module of the system architecture to help robots monitor world state as well as the human state [25], construct simulation of hypothetical cognitive models of the human partner to account for human behaviors that deviate from original plans [44, 79], an…”
BackgroundExploring Effects of Conversational Fillers on User Perception of Conversational Agents
Jeong, Y., Lee, J., & Kang, Y. (2019) · Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290607.3312913 · Cited by 31 · cited 1×
“In particular, we adapted a validated survey instrumentmeasuring user perception of robots in human-robot interactions [8], also previously applied in human-CA interaction settings [50].”
MethodThe Good, the Bad and the Neutral: Affective Profile in Dialog System-User Communication
Skowron, M., Rank, S., Theunis, M., & Sienkiewicz, J. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-24600-5_37 · Cited by 28 · cited 1×
“Researchers have inferred users’ emotions towards the agent [90], personality traits [62], signs of conversation breakdowns [58, 99],”
BackgroundWang, Q., Jing, S., Joyner, D. A., Wilcox, L. G., Li, H., Plötz, T., & DiSalvo, B. J. (2020) · ACM Conference on Learning @ Scale (L@S) · doi:10.1145/3386527.3405917 · Cited by 23 · cited 1×
“example, identifying conversation breakdowns through conversational cues [58] or monitoring user emotions and satisfactions during interactions [29, 93] can be combined with the identified user perception of the CA.”
BackgroundSelf-citationPickard, M. D., Burgoon, J. K., & Derrick, D. C. (2014) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2014.888504 · Cited by 21 · cited 1×
“In human interactions, likeability has been suggested to induce positive affect, increase persuasiveness, and foster favorable perceptions [76, 80].”
BackgroundMutual Modelling in Robotics: Inspirations for the Next Steps
Lemaignan, S., & Dillenbourg, P. (2015) · ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1145/2696454.2696493 · Cited by 21 · cited 1×
“MToM is increasingly used as a theoretical framework for the design of human-centered AI, such as robots, that can be perceived as more “natural” and intelligent during collaborations with human partners [26, 57, 59, 75].”
Designing Chatbots for Guiding Online Peer Support Conversations for Adults with ADHD
Nordberg, O. E., Wake, J. D., Nordby, E. S., Flobak, E., Nordgreen, T., Mukhiya, S. K., & Guribye, F. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-39540-7_8 · Cited by 20
Goal Recognition over POMDPs: Inferring the Intention of a POMDP Agent
Ramirez, M., & Geffner, H. (2011) · RMIT Research Repository (RMIT University Library) · doi:10.5591/978-1-57735-516-8 · Cited by 18
Towards a simple robotic theory of mind
Kim, K.-J., & Lipson, H. (2009) · Workshop on Performance Metrics for Intelligent Systems (PerMIS) · doi:10.1145/1865909.1865937 · Cited by 17 · cited 1×
“Specifcally, ToM has been intentionally built in as an individual module of the system architecture to help robots monitor world state as well as the human state [25], construct simulation of hypothetical cognitive models of the human partner to account for human behaviors that deviate from original plans [44, 79], an…”
BackgroundAI-Powered Learning: Making Education Accessible, Affordable, and Achievable
Goel, A. K. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.01908 · Cited by 13 · cited 3×
“We deployed a community-facing question-answering (QA)CA named Jill Watson [24, 34, 35] ( JW for short) in an online class discussion forum to answer students’ questions for 10 weeks over the course of a semester.”
MethodSelf-citationAre CUIs Just GUIs with Speech Bubbles?
Simpson, J. (2020) · ACM Conference on Conversational User Interfaces (CUI) · doi:10.1145/3405755.3406143 · Cited by 9 · cited 1×
“Consequently, researchers and designers turn to traditional HCI design guidelines intended for Graphical User Interfaces, which is not always the optimal perspective to look at designing the interactions between humans and often anthropomorphized CAs [89]— researchers and designers face major obstacles in balancing un…”
BackgroundConstructionism of virtual humans to improve perceptions of conversational partners
Halan, S., Rossen, B., Crary, M. A., & Lok, B. C. L. (2012) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/2212776.2223807 · Cited by 7 · cited 2×
“However, the majority of the research on human perception of CAs uses qualitative methods to either identify the various dimensions of user perception of the agent [31, 67] or provides one-time assessments of user perception after interacting with the agent [30, 41, 51, 55, 88, 98].”
BackgroundWorking with a Recommendation Agent
Bigras, É., Jutras, M.-A., Sénécal, S., Léger, P., Frédette, M., Black, C., Robitaille, N., Grande, K., & Hudon, C. (2018) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3170427.3188639 · Cited by 4 · cited 2×
“of the CA could include global behavior, knowledge distribution, and local behavior [31]; people’s perception of a recommendation agent consists of trust, credibility, and satisfaction [10].”
BackgroundResult
Research works printed but not linked: 14
- Simon Baron-cohen. 1999. Evolution of a Theory of Mind?In The Descent of Mind: Psychological Perspectives on Hominid Evolution. Oxford University Press, 1–31.
- Cristian Danescu-Niculescu-Mizil Moritz Sudhof Dan Jurafsky Jure Leskovec and Christopher Potts. 2013. A computational approach to politeness with application to social factors. arXiv preprint arXiv:1306.6078(2013). · arXiv
- Bobbie Eicher, Kathryn Cunningham, Sydni Peterson Marissa Gonzales, and Ashok Goel. 2017. Toward mutual theory of mind as a foundation for co-creation. In International Conference on Computational Creativity, Co-Creation Workshop.
- Ashok K Goel and Lalith Polepeddi. 2016. Jill Watson: A virtual teaching assistant for online education. Technical Report. Georgia Institute of Technology.
- 10.4324/9781003320609-59 · DOI · Paper page
- Alvin I Goldman 2012. Theory of mind. The Oxford handbook of philosophy of cognitive science 1 (2012).
- Kokil Jaidka, Sharath Chandra Guntuku, Anneke Buffone, H Andrew Schwartz, and Lyle H Ungar. 2018. Facebook vs. Twitter: Cross-platform differences in self-disclosure and trait prediction. In Proceedings of the Twelfth International AAAI Conference on Web and Social Media. 141–150.
- 10.5555/1622637.1622649 · doi:10.5555/1622637.1622649
- Tomas Mikolov Ilya Sustskever Kai Chen Greg Corrado and Jeffrey Dean. 2013. Distributed Representations ofWords and Phrases and their Compositionality. In Neural Information Processing Systems (NIPS). 3111–3119.
- Clifford Nass and Youngme Moon. 2000. Machines and mindlessness: Social responses to computers.Journal of Social Issues 1, 56 (2000), 81–103.
- Christopher Peters. 2005. Foundations of an agent theory of mind model for conversation initiation in virtual environments. Virtual Social Agents(2005) 163. · DOI
- David Premack and Guy Woodruff. 1978. Does the chimpanzee have a theory of mind?Behavioral and brain sciences 1, 4 (1978), 515–526.
- Tina L Robbins and Angelo S DeNisi. 1994. A closer look at interpersonal affect as a distinct influence on cognitive processing in performance evaluations.Journal of Applied Psychology 79, 3 (1994), 341.
- Henry M Wellman. 1992. The child’s theory of mind.The MIT Press.
Cited by
Zhou, J., Zhang, Y., Luo, Q., Parker, A. G., & De Choudhury, M. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581318 · Cited by 288
“Educating end users to spot what could potentially be AI-generated will empower them to establish an understanding of AI tools [67, 109] and develop individual-specifc strategies to evolve to be more informed and mindful information consumers.”
BackgroundRoss, S. I., Martinez, F., Houde, S., Muller, M., & Weisz, J. D. (2023) · Proceedings of the 28th International Conference on Intelligent User Interfaces · doi:10.1145/3581641.3584037 · Cited by 258
“Such a pair of models is often described asMutual Theory ofMind [29, 100].”
BackgroundMethodKazemitabaar, M., Ye, R., Wang, X., Henley, A. Z., Denny, P., Craig, M., & Grossman, T. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642773 · Cited by 290
Design Principles for Generative AI Applications
Weisz, J. D., He, J., Muller, M., Hoefer, G., Miles, R., & Geyer, W. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642466 · Cited by 232
“After refning the principles and strategies for relevance and clarity, we evaluated their utility within the design process by conducting structured, exploratory workshops with design practitioners within our organization who work on generative AI applications.”
MethodWhat is Human-Centered about Human-Centered AI? A Map of the Research Landscape
Capel, T., & Brereton, M. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580959 · Cited by 228
Metadata
- Authors
- Wang, Q., Saha, K., Gregori, E., Joyner, D., & Goel, A.
- Affiliations
- Georgia Institute of Technology (US)
- Venue
- Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2021-05-06
- Pages
- 1-14
- OpenAlex
- W3160674997
- Cited by
- 120
- References
- 96
- Citations by year
- 2026: 20 · 2025: 26 · 2024: 33 · 2023: 26 · 2022: 12 · 2021: 3
- FWCI
- 31.56
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion)
- Funders
- Georgia Institute of Technology
- Topics
- Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
- Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
- Social Sciences › Business, Management and Accounting › Marketing › AI in Service Interactions
- Keywords
- human-AI interaction, virtual teaching assistants, chatbots, student perceptions, anthropomorphism, perceived intelligence, online learning, forum, linguistic features, sentiment analysis, readability, long-term interaction, adaptive agents
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.