The DOI matches its Crossref record; 69 of the 69 references it cites were found in a registry or an index.
Understanding User Perceptions of Proactive Smart Speakers
Abstract
Voice assistants, such as Amazon's Alexa and Google Home, increasingly find their way into consumer homes. Their functionality, however, is currently limited to being passive answer machines rather than proactively engaging users in conversations. Speakers' proactivity would open up a range of important application scenarios, including health services, such as checking in on patient states and triggering medication reminders. It remains unclear how passive speakers should implement proactivity. To better understand user perceptions, we ran a 3-week field study with 13 participants where we modified the off-the-shelf Google Home to become proactive. During the study, our speaker proactively triggered conversations that were essentially Experience Sampling probes allowing us to identify when to engage users. Applying machine-learning, we are able to predict user responsiveness with a 71.6% accuracy and find predictive features. We also identify self-reported factors, such as boredom and mood, that are significantly correlated with users' perceived availability. Our prototype and findings inform the design of proactive speakers that verbally engage users at opportune moments and contribute to the design of proactive application scenarios and voice-based experience sampling studies.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“However, existing commercial chatbot frameworks, such as Dialogflow[32] and Amazon Alexa [6], predominantly only support building rule-based and scripted chatbots [69, 106].”
From Leveraging Large Language Models to Power Chatbots for Collecting User Self-Reported Data (2024) · cites this paper as [106]
Citations
Cites
XGBoost: A Scalable Tree Boosting System
Chen, T., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939785 · Cited by 54,569 · cited 1×
“XGBoost is a gradient boosting regression tree algorithm, which is efficient and has a low computation cost [10].”
MethodMultiple Resources and Mental Workload
Wickens, C. D. (2008) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/001872008x288394 · Cited by 2,164
The Aware Home: A Living Laboratory for Ubiquitous Computing Research
Kidd, C. D., Orr, R., Abowd, G. D., Atkeson, C. G., Essa, I. A., MacIntyre, B., Mynatt, E., Starner, T. E., & Newstetter, W. (1999) · Lecture Notes in Computer Science · doi:10.1007/10705432_17 · Cited by 1,035
Dillman, D. A., Phelps, G., Tortora, R., Swift, K., Kohrell, J., Berck, J., & Messer, B. L. (2009) · Social Science Research · doi:10.1016/j.ssresearch.2008.03.007 · Cited by 850 · cited 1×
“In order to shorten the question-asking time, we did not include the label of each Likert scale point [16].”
BackgroundVoice Interfaces in Everyday Life
Porcheron, M., Fischer, J. E., Reeves, S., & Sharples, S. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174214 · Cited by 717 · cited 2×
“Multi-turn conversations with smart speakers can be more error-prone and can cause user confusion when interaction errors occur [57].”
BackgroundLau, J., Zimmerman, B., & Schaub, F. (2018) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3274371 · Cited by 568
Designing robots for long-term social interaction
Gockley, R., Bruce, A., Forlizzi, J., Michalowski, M., Mundell, A., Rosenthal, S., Sellner, B., Simmons, R., Snipes, K., Schultz, A. C., et al. (2005) · 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems · doi:10.1109/iros.2005.1545303 · Cited by 407 · cited 1×
“A greeting or utterance, which is learned from human-to-human conversations, has been used to initiate human-robot conversations [21].”
MethodUsing context-aware computing to reduce the perceived burden of interruptions from mobile devices
Ho, J., & Intille, S. S. (2005) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1054972.1055100 · Cited by 367 · cited 1×
“[23] reported that the mobile messages delivered at physical activity transitions were better received, such as sitting to walking and standing to sitting.”
BackgroundAn in-situ study of mobile phone notifications
Pielot, M., Church, K., & de Oliveira, R. (2014) · Proceedings of the 16th international conference on Human-computer interaction with mobile devices & services · doi:10.1145/2628363.2628364 · Cited by 345 · cited 2×
“It is reported that a person may receive on average more than 60 notifications from smartphones per day [53].”
BackgroundThe Personalization of Conversational Agents in Health Care: Systematic Review
Kocaballi, A. B., Berkovsky, S., Quiroz, J. C., Laranjo, L., Tong, H. L., Rezazadegan, D., Briatore, A., & Coiera, E. (2019) · Journal of Medical Internet Research · doi:10.2196/15360 · Cited by 332 · cited 1×
“…factor that determines user experience with proactive speakers, further studies can explore two directions for conversation starters: (1) context-aware utterances [45] or personalized utterances [31], and (2) compare the Earcon starter with the utterance starter that both require user consent.”
Background
Research works printed but not linked: 2
- Alexander Meschtscherjakov , Astrid Weiss , and Thomas Scherndl . 2009 . Utilizing emoticons on mobile devices within ESM studies to measure emotions in the field . Proc. MME in conjunction with MobileHCI 9 (2009), 3361--3366. Alexander Meschtscherjakov, Astrid Weiss, and Thomas Scherndl. 2009. Utilizing emoticons on mobile devices within ESM studies to measure emotions in the field. Proc. MME in conjunction with MobileHCI 9 (2009), 3361--3366.
- Chenyang Zhang and Yingli Tian . 2012 . RGB-D camera-based daily living activity recognition . Journal of computer vision and image processing 2 , 4 (2012), 12 . Chenyang Zhang and Yingli Tian. 2012. RGB-D camera-based daily living activity recognition. Journal of computer vision and image processing 2, 4 (2012), 12.
Cited by
Leveraging Large Language Models to Power Chatbots for Collecting User Self-Reported Data
Wei, J., Kim, S., Jung, H., & Kim, Y.-H. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3637364 · Cited by 84
“Lastly, we hope future research can investigate the user perceptions of LLM-driven chatbots, or even voice-based ones like smart speakers [96].”
BackgroundSelf-citationZargham, N., Reicherts, L., Bonfert, M., Voelkel, S. T., Schoening, J., Malaka, R., & Rogers, Y. (2022) · Proceedings of the 4th Conference on Conversational User Interfaces · doi:10.1145/3543829.3543834 · Cited by 84
“Previous studies have explored opportune moments for VA interventions in homes [7, 16, 19, 43], cars [17, 18, 33–36], and other settings [20].”
BackgroundPolite or Direct? Conversation Design of a Smart Display for Older Adults Based on Politeness Theory
Hu, Y., Qu, Y., Maus, A., & Mutlu, B. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517525 · Cited by 60
“On the other hand, recent work has proposed the need for proactive behavior in voice interaction design and for conversational systems to play a more active role in engaging users [4, 6, 38].”
BackgroundProactive behavior in voice assistants: A systematic review and conceptual model
Bérubé, C., Nißen, M., Vinay, R., Geiger, A., Budig, T., Bhandari, A., Benito, C. R. P., Ibarcena, N., Pistolese, O., Li, P., et al. (2024) · Computers in Human Behavior Reports · doi:10.1016/j.chbr.2024.100411 · Cited by 58
Oh, J., Kim, W., Kim, S., Im, H., & Lee, S. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642193 · Cited by 38
Lim, J., Koh, Y., Kim, A., & Lee, U. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642846 · Cited by 38
“IoT sensors effectively detect user presence in homes, bypassing the need for Bluetooth connectivity between smart speakers and phones [57], which may not always be carried by users.”
InfluentialDixon, E., Michaels, R. B., Xiao, X. H., Zhong, Y., Clary, P., Narayanan, A., Brewer, R. N., & Lazar, A. (2022) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/3517428.3544809 · Cited by 37
“, context-aware proactive smart-speakers [83, 102, 103]) by highlighting potentially useful features and providing regular reminders of these features if they begin to go unused.”
BackgroundPark, G. W., Panda, P., Tankelevitch, L., & Rintel, S. (2024) · Designing Interactive Systems Conference · doi:10.1145/3643834.3661507 · Cited by 39
“Proactive VUIs [90, 109, 115] can support time-critical events, which is essential for recognizing how well meeting talk matches meeting needs.”
Towards Detecting and Mitigating Cognitive Bias in Spoken Conversational Search
Ji, K., Cherumanal, S. P., Trippas, J. R., Hettiachchi, D., Salim, F. D., Scholer, F., & Spina, D. (2024) · International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI) · doi:10.1145/3640471.3680245 · Cited by 22
WorkFit: Designing Proactive Voice Assistance for the Health and Well-Being of Knowledge Workers
Ahire, S., Simon, B., & Rohs, M. (2024) · ACM Conference on Conversational User Interfaces (CUI) · doi:10.1145/3640794.3665561 · Cited by 17
Foody Talk: Exploring Opportunities for Conversational Food Journaling
Silva, L. M., Lu, X., Liang, E. X., & Epstein, D. A. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713875 · Cited by 18
Chang, F., Sheng, L., & Gu, Z. (2024) · JMIR mHealth and uHealth · doi:10.2196/47472 · Cited by 14
“The potential for emergency calls, while not frequently used, aligns with the findings of studies emphasizing the importance of safety features for older adults [39].”
BackgroundTickling Proactivity: Exploring the Use of Humor in Proactive Voice Assistants
Zargham, N., Reicherts, L., Avanesi, V., Rogers, Y., & Malaka, R. (2023) · International Conference on Mobile and Ubiquitous Multimedia (MUM) · doi:10.1145/3626705.3627777 · Cited by 12
“Several researchers have looked into opportune moments to proactively interact with people [7, 38, 40, 63, 71–73 , 94].”
BackgroundCho, S., Kim, Y., Jang, J., & Hwang, I. (2023) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3580812 · Cited by 14
“Proactive interactions have been studied extensively in push notifications [69, 95, 96, 108, 113, 116], proactive speakers [13, 17, 25, 72, 82, 83, 117, 144, 145], and interactive displays [5, 137, 142, 157, 159].”
BackgroundUnderstanding How to Administer Voice Surveys through Smart Speakers
Wei, J., Jiang, W., Wang, C., Yu, D., Goncalves, J., Dingler, T., & Kostakos, V. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3555606 · Cited by 9
“In the end, we provide participants with twelve in-home scenarios proposed in previous studies [9, 74, 84] (see Table 2), and ask them to choose whether they prefer the voice or the web-browser in each scenario.”
BackgroundMethodInfluentialSelf-citationKim, M., Lee, J., Koh, Y., Lee, C., Lee, U., & Kim, A. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642778 · Cited by 11
“Although studies have focused on visual–manual interactions with smartphones [38, 46, 47, 64], an increasing number of recent studies have investigated opportune moments for auditory–verbal (or speech) interactions in home environments [10, 51, 60, 61, 65].”
BackgroundResultInfluentialPark, G. W., Panda, P., Tankelevitch, L., & Rintel, S. (2024) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3613905.3650797 · Cited by 12
“LLMs combined with proactive Voice User In-terfaces (VUIs) can provide real-time support, aligning meeting dialogue with predefined objectives [21, 75, 87, 89].”
Koh, Y., Lee, C., Joung, E., Lee, H., & Lee, U. (2025) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3749485 · Cited by 9
“Experience Sampling Method (ESM) Data Collection: To collect self-reported emotional wellbeing data, we implemented a context-aware ESM system, following approaches from prior studies [56, 109].”
Exploring Cultural and Intergenerational Dynamics in Voice Assistant Design for Chinese Older Adults
Qian, Z., Fu, J., & Zhou, Y. (2025) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3712275 · Cited by 8
Beyond Subservience: Using Joint Commitment to Enable Proactive CUIs for Mood Logging
Bowman, R., Cowan, B. R., Thieme, A., & Doherty, G. J. (2022) · ACM Conference on Conversational User Interfaces (CUI) · doi:10.1145/3543829.3544512 · Cited by 5
“As the CUI’s beliefs, constructed using context-aware techniques [7, 27], are intrinsically uncertain, this uncertainty should be represented by the CUIs designed.”
BackgroundDual-Mode Interventions: Giving Agency to Knowledge Workers in Proactive Health Interventions
Ahire, S., Othman, S., & Rohs, M. (2024) · ACM Conference on Conversational User Interfaces (CUI) · doi:10.1145/3640794.3665578 · Cited by 4
Xuan, Z., Wu, Y., Xu, X., Namboodiri, V., Chuah, M. C., & Yang, Y. (2025) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3770661 · Cited by 5
“Following these criteria, we identified ten case studies [11, 12, 14, 18, 39, 44, 50, 64, 70, 71] that reflect four key themes in assistant research, including Personalization and Social Framing , Proactivity and Context-Awareness , Managing Attention and Interruptibility , and User Control and…”
InfluentialBystander Privacy Implications of Robots in Everyday Spaces: A Scoping Review
Dietrich, M., Sarkisian, A., & Weisswange, T. H. (2026) · ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1145/3757279.3785586 · Cited by 3
Nasrabadi, M. A., Beauregard, Y., & Ekhlassi, A. (2025) · IEEE Transactions on Engineering Management · doi:10.1109/tem.2025.3640012 · Cited by 2
“In smart speaker and display, inaccuracy has been a critical factor of obsolescence, primarily due to speech recognition errors [117], resulting in consistently high OI (jct) levels, a trend that has been increasing since 2022.”
Xuan, Z., Wu, Y., Yan, Z., Namboodiri, V., & Yang, Y. (2026) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3831999 · Cited by 2
“Studies show that user acceptance of actions from assistants depends not only on what an assistant does, but also on when it intervenes, how much initiative it takes, how it communicates, and whether it adapts to situational context [9, 11, 13, 32, 46, 55].”
Yan, X., Ding, J., Li, Y., & Yu, Z. (2026) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3831658 · Cited by 1
“Similarly, [4] and [52] explored interruptibility in smart speaker interactions, identifying key contextual factors such as user mood, activity, and social presence that influence the appropriateness of proactive engage-ments.”
Design and Prototype Conversational Agents for Research Data Collection
Wei, J., Kim, Y., Chan, S., & Dingler, T. (2022) · ACM International Conference on Interactive Surfaces and Spaces (ISS) · doi:10.1145/3532104.3571467
“…both text-based chatbots or voice assistants to administer standard survey administration [5, 7], collect personal information through Experience Sampling Method (ESM) surveys [2, 10], stage interventions [8], and explore use cases in customer service [4], education [6], or digital health [3, 9].”
BackgroundSelf-citationCho, S., Jang, J., Kim, Y., & Hwang, I. (2023) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/3594739.3610718
“Proactive interactions have been extensively studied in the HCI community, including push notifications [12, 14, 16, 24, 25, 27, 28], proactive speakers [3, 17, 31], home robotics & automation [13, 15, 18], gamifications [6, 9, 21], and interactive displays [1, 30, 33, 34].”
BackgroundElmasry, A. (2024) · Al-Mağallah Al-ʿilmiyyaẗ Lil-Maktabāt wa Al-Waṯā'iq wa Al-Ma'lūmāt · doi:10.21608/jslmf.2022.160027.1132
Deka, A., & Kalita, N. (2024) · Multimedia Tools and Applications · doi:10.1007/s11042-024-19373-8
“To accurately understand the perception of listeners in the caller state, knowing the information regarding the speaker’s age, gender, and state of emotion is important [13].”
Designing Cognitive ‘Copilots’: Augmenting Our Minds Through Prompting Conversational Interactions
Reicherts, L. (2024) · UCL Discovery (University College London)
Exploring the Design of Conversational Agents for Mental Health Self-Monitoring
Bowman, R. J. (2025) · Trinity's Access to Research Output (TARA) (Trinity College Dublin) · Cited by 1
Sensor-Driven Dialogue System for Collecting Community-Specific Knowledge Data
Tobase, N., & Sato-Shimokawara, E. (2026) · International Conference on Knowledge and Smart Technology · doi:10.1109/kst67832.2026.11432125
Proactive, But Not Creepy: Legitimacy and Disclosure Boundaries in Generative IR Assistants
Song, B., Kim, S., & Lee, S. (2026) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3772363.3798894
Exploring the impact of frequency of robot’s proactive utterances in product introduction dialogues
Nakanishi, J., Baba, J., Okafuji, Y., Yoshikawa, Y., & Ishiguro, H. (2026) · Scientific Reports · doi:10.1038/s41598-026-49412-3
Zhou, X., Liu, X., & Wang, Y. (2026) · Scientific Reports · doi:10.1038/s41598-026-59281-5
“This construct underscores the perception of the AI system as an active, self-driven agent capable of initiating interaction rather than simply responding to commands 59,60 .”
MiPP: Minimal Proactivity Policies for Timely, Low-Intrusion Proactivity
Sánchez-Adame, L. M., & Mendoza, S. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-30860-3_23
Multimodal-context Interruptibility Dataset for Proactive Services on Smart Speakers at Home
SUNATULLAEV, G., Hwang, J., Lee, J., Son, D., Son, K., & Kim, A. (2026) · Scientific Data · doi:10.1038/s41597-026-07618-0
Metadata
- Authors
- Wei, J., Dingler, T., & Kostakos, V.
- Affiliations
- The University of Melbourne (AU)
- Venue
- Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
- Published
- 2021-12-27
- Volume
- 5
- Issue
- 4
- Pages
- 1-28
- ISSN
- 2474-9567 (ISSN-L)
- OpenAlex
- W4200018492
- Cited by
- 33
- References
- 69
- Citations by year
- 2026: 6 · 2025: 5 · 2024: 13 · 2023: 3 · 2022: 6
- FWCI
- 3.37
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies (publishedVersion)
- Funders
- Australian Research Council (DP190102627)
- National Health and Medical Research Council (1170937, 2004316)
- AUSMURI (13203896)
- Topics
- Physical Sciences › Computer Science › Human-Computer Interaction › Innovative Human-Technology Interaction
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Social Sciences › Business, Management and Accounting › Marketing › AI in Service Interactions
- Keywords
- smart speakers, ecological momentary assessment, user mood, perceived availability, medication reminders, health monitoring, chatbots, voice user interface design
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.