A Second Chance to Make a First Impression? How Appearance and Nonverbal Behavior Affect Perceived Warmth and Competence of Virtual Agents over Time

“Moreover, a study by Bergmann et al. (2012) showed that appearance characteristics, such as human-like appearance (vs cartoonish appearance), have different effects than nonverbal behaviors, such as gestures.”

“Bergmann et al. (2012) found that impressions of AI warmth tended to decrease over time, whereas Pan et al. (2018) found that warmth ratings increased over time.”

“Most recently, Bergmann et al. [2012] investigated how appearance and non-verbal immediacy affect the perceived warmth and competence of virtual agents over time.”

“● The agent design features – age, gender and role – affect the perception of the characteristics of a static agent image at first glance; existing research confirms this finding (Bergmann et al., 2012).”

“peated interactions with multiple days of zero exposure in between, instead of multiple measurements within one interaction session [10][23][44][60].”

“Similarly, the strategic use of NVC also increases perceptions of warmth and likability in an agent (Bergmann et al., 2012; Cafaro et al., 2016).”

“Some reviews specifcally address empathy [270] or the nonverbal behavior [30] of agents.”

“But research conducted in other contexts has revealed that a bot’s ability to speak [31] or exhibit human-like non-verbal cues enhanced users’ perceived warmth and competence of the robots [4].”

“The same evaluations have been used occasionally in the past research to comment on particular attributes of robots (Bergmann et al., 2012; Eyssel & Hegel, 2012), but no research has looked at their applicability across a large sample of robots.”

“Several studies explored the warmth perceptions of artifcial entities, showing that people are open to the idea that such entities can possess human-like traits, such as warmth (e.g., [4, 6, 22]).”

“Nevertheless, people’s impression on chatbots can evolve in a short time [37], and future research is encouraged to explore this effect with more sessions spread in a longer period of time.”

“Researchers have proposed models of friendliness based on movement cues such as gaits, gestures, and gazing [34], [35], [36], [37] for robotics and VR applications.”

“For instance, anthropomorphic signals offer possibilities to shape time-dependent warmth and competence attributions of virtual agents [4].”

“While some verbal aspects, such as pitch or prosody, and the impact of empathic speech content have been developed and discussed as an aspect of IVA systems or a component of user studies, they were generally not the main contribution of the paper [7, 9, 13, 19, 21, 37, 47, 48, 66, 69, 70, 76, 77, 81, 85].”

“It has been shown that while the visual presence and appearance of IVAs can have a major impact on humans, for example, in motivation and affect, it is independent of the underlying technical sophistication [2] [4].”

“Previous work has already provided evidence for the relevance of warmth and competence attributions as underlying dimensions of social perception inHCI (Niewiadomski et al., 2010; Bergmann et al., 2012).”

“, [1, 4, 20, 27]).”

“Therefore, individuals also generate warmth or competence perceptions when evaluating robots (Bergmann et al., 2012; Mieczkowski et al., 2019).”

“To measure competence perception of the finance coach (e.g., Bergmann et al., 2012), we asked participants to rate the finance coach on a five-point Likert scale—from 1 (not at all) to 5 (very)—using five traits (competent, experienced, intelligent, efficient, and capable).”

“[4] found that the use of gestures in a virtual agent significantly increased a person’s perception of the agent’s competence.”

“We believe that this finding can add to a new line of research at the intersection of technology and (social) psychology that investigates how technological developments affect psychological processes (e.g., Bergmann et al., 2012).”

“Bergmann et al. (2012) examined the difference between first and second impressions between robot and human avatars.”

“Existing research provides evidence for implementation of animations of the agent’s embodiment, showing that animations positively affect users’ impressions of the agent [38-40] and interaction time [13,39].”

“First encounters (Bergmann et al. 2012) are particularly revealing because, as participants have no reliable knowledge of the robot’s interactional competencies, they rely on their own assumptions as well as what they discover about those competencies in the course of their interaction with the…”

“It has, for instance, been demonstrated that a virtual agent’s use of gestures might enhance the perceived verbal eloquence [7] as well as users’ ratings of the agent, mostly in terms of competence [3].”

“For example, interactions with embodied agents that exhibited nonverbal feedback such as gestures were shown to engender more favorable evaluations (Bergmann et al., 2012; Krämer et al., 2013) and increase a sense of realism (e.g., Garau et al., 2005).”

“People tend to interact with others who provide favorable impressions [1].”

“…et al., 2010), a lot of studies in the broader field of human-AI interaction focused on the human-like perception of AI systems (assessing constructs such as perceived warmth, competence, or anthropomorphism, e.g., Bergmann et al., 2012; Gilad et al., 2021; Gong, 2008; Purington et al., 2017).”

“With respect to [10], we consider more behaviours than only co-speech gestures, such as facial expressions, trunk leaning and head poses.”

“Third, while prior research on HRI suggests that various factors such as appearance (Bergmann et al., 2012; Kim et al., 2019), expression (i.e. smile) (Yu & Boyol Ngan, 2019), and gesture (Biancardi et al., 2017; Salem et al., 2013) a ff ect judgment of warmth for non-human entities, the e ff ect of…”

“Bergmann et al. [11] used gestures to communicate warmth in virtual agents.”

“…out that affective evaluation plays a crucial role in a user ’ s experience (Dirin et al., 2018; Knutson & Greer, 2008), and VR technologies are an effective medium for evoking affective experiences, such as impressions (Bergmann et al., 2012; Cafaro et al., 2016 ; Ghanbarzadeh & Ghapanchi, 2020).”

“Following [7], we also assessed participants’ perceptions of the virtual agents on the scales warmth and competence [17].”

“Prior work has shown negative impacts of multiple exposures on subjective perceptions of robots over time (Bergmann et al., 2012), and recent approaches in social neuroscience (Schilbach et al., 2013) highlight the need for more ecologically valid paradigms for studying mechanisms of social…”

“This finding contradicts previous research which demonstrated robot body language can enhance perception of robot warmth [45],[63].”

“When we interact with another human, or ECA, for the first time, we immediately form initial ideas about the other [16,17].”

“The perceived warmth and competence of the agent was measured using the Warmth and Competence Scales [7].”

“[17] found that human-like vs.”

“technologies may indeed appear more or less competent [49], persuasive [50], cooperative [51], authoritative [52], or dominant [53] to their human users.”

“It is also an important matter in studies on the agent to formulate positive long-term relationship with people through social interactions [63].”

“Literature showed that first impressions influenced whether humans build relations with others and find their interactions believable [Bergmann et al. 2012].”

“As users are more sensitive to human-like agents [8], the human agent’s negative feedback should be perceived more negatively than corresponding feedback from the robot and the mouse.”

“Bergmann et al. found that human-like vs. robot-like appearance positively affects impressions of warmth, while the presence of co-speech gestures increases competence judgements (Bergmann et al., 2012).”

“First impressions are, therefore, an important basis for whether humans will build relations with others and find their interactions believable [2].”

“The authors of [47] stated that a limitation of their study is the fact that it did not consider the influence of other elements, such as the agent’s voice or other nonverbal cues.”

“Warmth and competence were measured by warmth-related and competence-related trait adjectives in previous studies [21], [24].”

“W&C perception in virtual agents has been measured in several studies (e.g., [14], [3]).”

“[31] investigated agents with nonnaturalistic appearance with and without gesture behavior and found that the non-naturalistic agent was perceived having more warmth (e.”

“Such appearance change also impacts how users perceive the agent (Bergmann et al., 2012; Kwak, 2014; Li et al., 2010).”

“Differences in all these factors also affect the perception of, and ultimately the relationship with, robots over time (Bergmann et al., 2012; Paetzel et al., 2020).”

“Another study showed a short introductory video of an agent for 15 s to investigate participants’ first impression of it [8].”

“participants completed the Rapport scale [14], the scales for perceived usefulness for tasks (modified from [15]), a perceived competence scale [16], and the Godspeed anthropomorphism subscale [17].”

“Speaking about competence in the agents, it is often operationalised through making the agent to appear ‘intelligent’ and ‘expert’ [13].”

“This change in helpfulness could also be attributed to the phenomenon of impressions of an agent evolving over time [5].”

“For instance, in a study exploring two aspects of impressions: “warmth” and “competence”, Bergmann and colleagues [9] reported that ratings of the perceived warmth of a virtual interaction agent shaped in the form of a robot was influenced by the agents appearance and decreased over time (i.”

“Human-like agents are preferred by users since it is more likely that they give a steadier impression of warmth [4], which generates positive symbolic meaning [1].”

“warmth, and competence of a robot are made [1, 2].”

“Perceived warmth and competence of robots might be influenced by their appearance and non-verbal behavior [27].”

“It is arguable whether perceptions reported after having these interactions should be termed “initial perceptions”, because according to Bergmann’s report [10], first impressions are modifiable through direct interactions.”

“[62] has shown how over the course of an interaction, perceptions of warmth and competence of a virtual agent can change, whereas [63] has shown how during an interaction with a robot, positive interactions tend to result in more favourable perceptions of the robot.”

“Warmth and competence were measured by warmth-related and competence-related trait adjectives, as well as in previous studies [11], [12].”

“previous works, researchers have examined the utility of anthropomorphizing virtual agents by facial representation [12], gesture use [1], linguistic styles [26], and support-providing abilities [16], etc.”

“As in the case of people, in the first step the observer formulates the first impression in contact with the robot [55].”

“Form factor design has been shown to generally influence human expectations of the robot [12], perceived trust [13]– [15], intelligence [16], [17], and warmth and competence [18], [19].”

“For example, Bergmann et al. [11] found that human-like vs. robot-like appearance positively affected the perception of warmth, while the presence of co-speech gestures increased competence judgements.”

“With respect to Bergmann and colleagues [3], we choose to consider more behaviors than only co-speech gestures, such as facial expressions, trunk leaning and head poses.”

“[12,13,14,15]) we hypothesize that the relationship between freedom of choice in the selection procedure and satisfaction with the virtual agent as well as the rating of agent’s suitability for its task would be mediated by the evaluation of the resulting agent in terms of likeability, attractiveness, warmth, competen…”

“, continuing watching the virtual human) differed significantly from the first impression ratings, indicating for example an instability in the dimension “Warmth” for robotic appearance [5].”

“…to adapt their gaze behaviour and facial expressions to the virtual agent’s gaze and body direction (Marschner et al. 2015)⁠, and to produce verbal utterances and discussion topics resembling the ones produced in the presence of a real human interlocutor (Bergmann et al. 2012 ; Lugrin et al. 2018).”

“…ját - szanak az attitűdök kialakulásában, így a robot emberhez való hasonlósága (Katz & Halpern, 2014), a kognitív ké - pességeinek a szintje (Bergmann, Eyssel, & Kopp, 2012; Demeure, 2011; Demeure, Niewiadomski, & Pelachaud, 2010; Fraune et al., 2017), valamint a robottal kapcsolatos…”