The State of Speech in HCI: Trends, Themes and Challenges

“Inductive Thematic Analysis (Braun & Clarke, 2006) was conducted on the final search results to categorize the research themes discussed in each publication.”

“A key debate around the role of partner models seems to be emerging in HCI based publications, which reflects debates on language production more generally (Brennan & Clark, 1996; Horton & Keysar, 1996; Keysar, Barr, & Horton, 1998).”

“Approaches commonly used in interaction design research such as cultural probes (DeHaemer & Wallace, 1992), contextual inquiry (Holtzblatt & Jones, 1993), and participatory design more generally (Schuler & Namioka, 1993) tended not to be used in the research we reviewed.”

“This was to avoid studies where speech is confounded by issues of embodiment, such as gesture and emotive facial expressions (Breazeal, 2003), that can affect users’ interactions and experiences (Bruce, Nourbakhsh, & Simmons, 2002; Kuno et al., 2007).”

“Our review indicates that there has been some recent progress towards this (e.g. Luger & Sellen, 2016; Porcheron et al., 2017; Cowan et al., 2017).”

“In WoZ studies participants are led to believe they are interacting with a system, however, a (usually unseen) confederate controls the system’s output (Dahlbäck, Jönsson, & Ahrenberg, 1993).”

“In their overview, (Clark et al., 2019) point out interesting qualities to keep in consideration, such as user attitudes (towards the interface), task performance (total of dialogue turns, task completion, etc.), lexis and syntax choice, perceived usability, user recall (of specific aspects and…”

“Speech has even become the primary form of interaction with home devices like Amazon Echo, Google Home, and Apple HomePod (Clark et al., 2019).”

“Many articles use post-interaction forms as a research method in which users report their usability/performance experiences [32], [33].”

“They are used in a variety of academic disciplines, including speech recognition, human computer interaction (Clark et al., 2019), neuroscience (Anumanchipalli et al., 2019; Ak-bari et al., 2019), psychology (Pouw et al., 2020), and medicine (Anfinrud et al., 2020; Karan et al., 2020; Alhanai et…”

“Smart Speakers, Displays, and Mirrors are frequently introduced as smart home control interfaces (Clark et al., 2019; Athira et al., 2016; Ardito et al., 2015).”

“…uploaded to the cloud for spoken language understanding (SLU) is steadily increasing [1, 2, 12], particularly in ubiquitous wimpy devices where textual input is inconvenient [3, 17, 41], e.g., home automation devices [32], smartwatches [37], telehealth sensors [22] and smart factory sensors [29] .”

“For instance, voice input may be socially inappropriate in public settings [13] and degrades significantly in noisy environments [10]; gaze is effective for pointing but suffers from the Midas touch problem [30] and induces fatigue during continuous control [28, 39]; EMG signals are sensitive to…”

“We selected these databases by adopting the approach used by Clark et al. [20].”

“…a set of usability heuristics for CUIs developed by 2 Literature review Past decades have seen the development of tools to evaluate and assess graphical user interfaces (GUI), ranging from design guidelines and heuristics [24] [25] to task performance and self-report questionnaires [5] [8-9].”

“Notably, voice-specific UX factors such as appeal, anthropomor-phism, aversion, comfort, and trustworthiness remain largely un-explored [4, 29, 30, 32], especially in persuasive contexts.”

“Intelligent Personal Assistants (IPAs) have become ubiquitous over the past decade and a half, with prominent examples such as Siri, Google Assistant, and Alexa integrated in smartphones, home controllers and desktop computers [14, 27, 46].”

“[1, p. 36] Voice interfaces are convenient, but often lack reliability in their operation [28].”

“Despite of these advancements in HCIs, there are various challenges [4].”

“In user-centric applications, the choice of speaker voice can often impact user experience [8].”

“The presence of computers and the Internet in academic, work and personal environments has modified the forms of communication between humans [6].”

“For instance, Clark et al. (2019) highlighted a pressing need for theoretical advancements in voice-based HCI to support diverse interaction contexts and enhance user experience.”

“We are also seeing a large commercial push and demand for usable CUIs - many of which we’ve already seen the likes of, such as Google Assistant, Amazon Alexa, and Siri.”

“A recent review on the state of the art in research on spoken dialog systems showed that most of the reviewed studies explored “usability” or “concepts and theories from research in human communication” [4].”

“Rather than being truly conversational, many of these applications are tightly constrained and highly task-oriented in nature (Clark et al., 2019b), with users requesting the proxies to execute tasks on their behalf.”