“A Painless Way to Learn:” Designing an Interactive Storytelling Voice User Interface to Engage Older Adults in Informal Health Information Learning

May combine two claims.

Nearly all participants successfully used the various functions of Mystery Agent and found them useful.

“Nearly all the participants successfully used the various functions of Mystery Agent and found them useful.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Some participants oscillated between pronouns.

“Some participants oscillated between pronouns.”

May combine two claims.

Some participants found the speed of information and instructions too fast, while a few found them too slow.

“For example, some participants found the speed of information and instructions too fast, while a few participants found the information and instructions too slow.”

May combine two claims.

Some participants mentioned the VUI's accessibility when they cannot read for long periods without irritating their eyes, or as a listening option when they'd like to take their eyes away from a computer screen.

“Some participants mentioned the VUI's accessibility when they cannot read for long periods of time without irritating their eyes [P9], or accessible as a listening option when they'd like to take their eyes away from an often-used computer screen [P1].”

Probably not a substantial claim: may not pass S3 (load-bearing).

A few participants expressed that prioritizing empathy and focusing on user input would make an agent feel more trustworthy and compassionate.

“A few participants expressed that prioritizing empathy and focusing on user input would make an agent feel more trustworthy and compassionate [P1, P2].”

May combine two claims.

Some participants considered their interactions with the VUI as a strong starting point or gateway to further health information-seeking techniques like searching online, and some imagined the agent sharing further resources to encourage further health information seeking beyond the VUI interaction.

“Some participants also considered their interactions with the VUI as a strong starting point or gateway to further health informationseeking techniques like searching online, and some participants imagined the agent sharing further resources to encourage further health information seeking beyond the VUI interaction.”

Probably not a substantial claim: may not pass S3 (load-bearing).

A few participants imagined a social approach, where they would work together with their relatives and romantic partners.

“A few participants also imagined a social approach, where they would work together with their relatives [P8] and romantic partners [P10].”

Probably not a substantial claim: may not pass S3 (load-bearing).

To establish trust, some participants discussed a potential strategy in co-design to share a disclaimer at the beginning of the agent interaction that shares when, where, by whom, and with what expertise the health information is being drawn.

“To establish trust, some participants discussed a potential strategy in co-design to share a disclaimer at the beginning of the agent interaction that shares when, where, by whom, and with what expertise the health information is being drawn.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

In imagined at-home interactions, interaction time ranged from an hour and a half to 15-minute sessions, unless the story is too compelling and they want to spend longer.

“In their imagined at-home interactions with Mystery Agent, interaction time ranged from hour and a half [P1] to 15-minute sessions, unless the story is too compelling, and they want to spend longer [P7]”

May combine two claims.

Most participants were neutral towards the amount of feedback and encouragement from Mystery Agent, while a few found it very positive, and a few would have preferred not to have it at all.

“Most participants were neutral towards the amount of feedback and encouragement from Mystery Agent, while a few found it very positive, and a few would have preferred not to have it at all.”

Probably not a substantial claim: may not pass S3 (load-bearing).

An important personalization feature mentioned by a few participants was to have options based on the user's familiarity with medical terminology.

“An important personalization feature mentioned by a few participants was to have options based on the user's familiarity with medical terminology [P8, P9].”

Probably not a substantial claim: may not pass S3 (load-bearing).

For one participant, establishing credibility was also part of creating a trustworthy persona for the agent.

“For one participant, establishing credibility was also part of creating a trustworthy persona for the agent.”

May combine two claims.

Many participant comments were intimately related to anxieties about loss of memory with aging and feeling judged.

“Many of these participant comments were intimately related to anxieties about loss of memory with aging and feeling judged.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Participants hypothesized diverse approaches to interacting with Mystery Agent at home, from a casual approach while multi-tasking to a focused approach while sitting down and concentrating.

“Participants hypothesized diverse approaches to interacting with Mystery Agent at home, from a casual approach to interacting while multi-tasking around the house [P7] to a focused approach while sitting down and concentrating on the session.”

May combine two claims.

Participants mentioned the auditory medium's accessibility when they are tired or fatigued or when their dexterity becomes limited, making using GUIs and keyboards more difficult.

“Participants mentioned the auditory medium's accessibility when they are tired or fatigued [P6] or when their dexterity becomes limited, making using GUIs and keyboards more difcult [P2].”

Probably not a substantial claim: may not pass S3 (load-bearing).

Other strategies for developing compassion in an imagined VUI design included having a caring agent persona with extended back-and-forth interactions and personable statements beyond disseminating only health facts.

“Other strategies for developing compassion in an imagined VUI design included having a caring agent persona with extended backand-forth interactions and personable statements beyond disseminating only health facts.”

Probably not a substantial claim: may not pass S3 (load-bearing).

One participant perceived the VUI to be like a tour guide, emphasizing the story's movement through settings and even pace.

“Another participant perceived the VUI to be like a tour guide [P5], emphasizing the story's movement through settings and story progression at an even pace.”

May combine two claims.

All participants emphasized the importance of health information and found health information to be an appropriate topic for learning via interaction with an interactive storytelling VUI, but some shared caveats such as requiring trustworthiness, timeliness, and accuracy.

“All participants emphasized the importance of health information and found health information to be an appropriate topic for learning via interaction with an interactive storytelling VUI-but some participants shared caveats such as requiring trustworthiness, timeliness [P5], and accuracy.”

Probably not a substantial claim: may not pass S1 (asserted), S3 (load-bearing). May combine two claims.

Priorities for this disclaimer included using a well-known and respected source like the Mayo Clinic, mentioning the specific studies information is drawn from, including a date to establish how current the information is, and offering references for further information gathering.

“Priorities for this disclaimer included using a well-known and respected source like the Mayo Clinic [P2, P3], mentioning the specifc studies information is drawn from [P3], including a date to establish how current the information is [P5], and ofering references for further information gathering [P5].”

May combine two claims.

While one participant would have preferred not to have the story interrupted, most found the back-and-forth interactions more fun and engaging, and a few considered the interaction a game or puzzle.

“While one participant would have preferred not to have the story interrupted [P4], most found the back-and-forth interactions more fun and engaging [P8], and a few considered the interaction a game [P3] or puzzle [P7, P10].”

May combine two claims.

Participants had diverse ideas of what voice they would prefer for Mystery Agent (gender, nationality, friendliness), what the agent's speech cadence and speed should be, and even desired options for different story genres.

“Participants also had diverse ideas of what voice they would prefer for Mystery Agent (gender, nationality, friendliness), what the agent's speech cadence and speed should be, and even desiring options for diferent story genres.”

Probably not a substantial claim: may not pass S3 (load-bearing).

One participant described how an interactive storytelling VUI could make learning more accessible to users who are home-bound, have diverse levels of formal education, or are currently under or unemployed in their home environments.

“Last, one participant described how an interactive storytelling VUI could make learning more accessible to users who are home-bound, have diverse levels of formal education, or are currently under or unemployed in their home environments.”

May combine two claims.

While all participants thought the questions kept them more attentive, a few wanted more questions, and a few would have preferred fewer questions or moving the questions to the end.

“While all participants thought the questions kept them more attentive, a few wanted more questions, and a few would have preferred fewer questions or moving the questions to the end.”

May combine two claims.

Anticipation of the questions caused all participants to pay more attention to the storytelling, with one participant becoming curious and looking forward to the questions.

“Anticipation of the questions caused all of the participants to pay more attention to the storytelling, with one participant becoming curious and looking forward to the questions [P10].”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Participants that imagined a focused approach included actions like taking notes during interactions and treating Mystery Agent like a puzzle, sitting at their kitchen table with pen and paper.

“Participants that imagined a focused approach included actions like taking notes during interactions [P2, P3, P6, P10] and treating Mystery Agent like a puzzle [P10], sitting at their kitchen table with pen and paper [P10].”

May combine two claims.

Most participants considered learning health information to be anxiety-inducing and therefore prioritized a calming, patient voice for the VUI.

“Most participants considered learning health information to be anxiety-inducing and therefore prioritized a calming, patient voice for the VUI.”

May combine two claims.

Most participants specifically mentioned that the story session was fun and that the story itself helped hold their attention.

“Most of the participants specifcally mentioned that the story session was fun and that the story itself helped hold their attention, with a few participants enjoying the puzzle-like [P3] or game-like [P7] aspect of listening for clues.”

May combine two claims.

The interactive design was perceived as accessible for neurodiverse users unrelated to aging, with one participant sharing it was accessible and helpful for users with ADHD.

“The interactive design was also perceived as accessible for neurodiverse users unrelated to aging.One participant shared how the experience with checkpoints and section-related questions was accessible and helpful for users with attention defcit/hyperactivity disorder (ADHD).”

May combine two claims.

Nearly all participants were able to easily imagine using Mystery Agent in their own homes, though one participant had privacy reservations.

“Nearly all participants were able to easily imagine using Mystery Agent in their own homes, though one participant had privacy reservations.”

Probably not a substantial claim: may not pass S1 (asserted), S3 (load-bearing).

Another participant described how having increased control options could also broaden accessibility for hearing loss.

“Another participant described how having these increased control options could also broaden accessibility for hearing loss [P4].”

May combine two claims.

A few participants desired a better balance between story and health information or more integration of health details throughout the story.

“In contrast, a few others wanted to see a greater balance of story and health information [P10] or greater integrations of the health details throughout the story session [P2].”

Classified as probably not a substantial claim. May combine two claims.

Among these older adults, credibility was defined as recent information, information drawn from peer-reviewed research studies, and well-known and established sources, like the Mayo Clinic or NIH.

“Among these older adults, credibility was defned as recent information, information drawn from peer-reviewed research studies, and well-known and established sources, like the Mayo Clinic or NIH.”

Probably not a substantial claim: may not pass S3 (load-bearing).

A few participants mentioned they would enjoy different interactive storytelling genres such as historical fiction and travelogues.

“A few participants also discussed how they would enjoy diferent interactive storytelling genres like historical fction [P4] and travelogues [P5]”

Classified as probably not a substantial claim.

The sense of compassion, some participants shared, is especially important when asking users to discuss sometimes anxiety-inducing health topics.

“The sense of compassion, some participants shared, is especially important when asking users to discuss sometimes anxiety-inducing health topics.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

A few participants indicated that they would like to be introduced to health information with Mystery Agent but would also like references on how and where they could learn about the topic more.

“Further, a few participants indicated that they would like to be introduced to health information with Mystery Agent but would also like references on how and where they could learn about the topic more.”

May combine two claims.

Participants characterized Mystery Agent as soothing, professional, non-judgmental, and affirming.

“Participants also described Mystery Agent as soothing [P1], professional and non-judgmental [P10], and afrming [P7].”

May combine two claims.

The repetitive nature of the instructions was helpful and necessary to feel confident using the VUI for some, and unnecessary and even condescending for a few.

“The repetitive nature of the instructions was helpful and necessary to feel confdent using the VUI for some, and unnecessary and even condescending for a few.”

May combine two claims.

Other participants described the storytelling learning experience as likeable, refreshing, engaging, enjoyable, novel, neat, cool, or unexpected.

“Similarly, other participants described their experience learning via storytelling as likeable and refreshing [P10], the listening experience engaging and enjoyable [P1], and the approach novel [P10], neat [P8], cool [P2], or unexpected [P7].”

May combine two claims.

Prior VUI learning studies are limited to second-language learning or STEM learning, and the resulting voice-based learning agents are built for younger adults or children.

“However, these studies are limited to second-language learning [16,23] or STEM learning [22,36], and the resulting voice-based learning agents are built for younger adults or children.”

• cites [16] sourceNot checked yet

  Not checked in the free run.

• cites [23] sourceNot checked yet

  Not checked in the free run.

• cites [22] sourceNot checked yet

  Not checked in the free run.

• cites [36] sourceNot checked yet

  Not checked in the free run.

May combine two claims.

The System Usability Scale has face validity for VUIs and face and construct validity for a specific use scenario.

“For VUIs, SUS has been successfully tested for face validity [28], and face and construct validity for a specifc use scenario [10].”

• cites [28] sourceNot checked yet

  Not checked in the free run.

• cites [10] sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Interactive fiction is a highly engaging informal learning medium that incorporates story and user interaction.

“These choices are drawn from interactive fction [45], another highly engaging informal learning medium that incorporates story and user interaction [30].”

• cites [45]Not checked yet

  Not checked in the free run.

• cites [30] sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Older adults tend to be more verbose than younger adults in speaking style and often take longer to respond.

“Older adults tend to be more verbose than younger adults in their speaking style and often take longer to respond [18], leading to interruptions [9] and high error rates [38].”

• cites [18] sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Older adults are at risk for low health literacy and have more difficulties finding, understanding, and using health information.

“Despite the high interest and need for health information seeking, older adults are at risk for low health literacy [27,39] and have more difculties fnding, understanding, and using health information [46].”

• cites [27] sourceNot checked yet

  Not checked in the free run.

• cites [39]Not checked yet

  Not checked in the free run.

• cites [46] sourceNot checked yet

  Not checked in the free run.

May combine two claims.

In scenario-based interviews with 35 older adults, participants had difficulties formulating complex queries and often did not find the information they sought.

“Brewer et al. [9]-in scenario-based interviews with 35 older adults-found that participants had difculties formulating complex queries and often did not fnd the information they sought.”

• cites [9] sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Older adults are often unfamiliar with how to initiate a conversation with voice user interfaces and what to do during a conversational breakdown.

“Additionally, older adults are often unfamiliar with how to initiate a conversation with VUIs [38,51] and what to do during a conversational breakdown.”

• cites [38] sourceNot checked yet

  Not checked in the free run.

• cites [51] sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Older adults often do not know how to start a conversation with a VUI and what to do during conversational failure.

“The rationale for developing the demo function was based on previous studies [9,20,52] investigating the use of VUIs by older adults, which found that older adults often do not know how to start a conversation with a VUI and what to do during conversational failure.”

• cites [9] sourceNot checked yet

  Not checked in the free run.

• cites [20] sourceNot checked yet

  Not checked in the free run.

• cites [52] sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

There are no item dependencies between likeability, intelligence, and trust in Godspeed measures.

“Here, it is important to note that there are no item dependencies between likeability, intelligence, and trust in Godspeed measures.”

• cites [4] sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

The Godspeed questionnaire has been successfully modified to accommodate VUIs in previous studies.

“This questionnaire from robotics research has been successfully modifed to accommodate VUIs in previous studies (e.g., [24,57]).”

• cites [24] sourceNot checked yet

  Not checked in the free run.

• cites [57] sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In previous studies with text-based pedagogical agents, self-regulated learning strategies such as scaffolding building, self-explanation, self-assessment, and motivation have been used.

“In previous studies [2,29] with text-based pedagogical agents (e.g., MetaTutor), SRL strategies like scafolding building, self-explanation, self-assessment, and motivations have been used.”

• cites [2] sourceNot checked yet

  Not checked in the free run.

• cites [29] sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S4 (based). May combine two claims.

Repeat is a common feature of VUI interactions but is often not used because users are not informed of its affordance.

“Although repeat is a common feature of interactions with VUIs, often it is not used because users are not signifed of its afordance.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

Older adults' health information-seeking behavior has transferred to information-seeking activities with voice user interfaces.

“Current literature in HCI suggests that older adults' health information-seeking behavior has also transferred to information-seeking activities with VUIs.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Auditory storytelling is a foundational medium of human culture.

“Auditory storytelling is a foundational medium of human culture [43] and has been found to be more cognitively and emotionally engaging than text and even visual stories [54].”

• cites [43] sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

The quantitative result of positive affect is corroborated by the qualitative analysis of participants' impressions.

“This result is corroborated by our qualitative analysis of participants' impressions.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The variability in intentions to continue using Mystery Agent could be because a few participants considered themselves nonauditory learners, and some indicated they would use it on their own terms, not frequently or daily.

“This could be because a few participants considered themselves nonauditory learners, and some indicated that they would use Mystery Agent on their own terms, not frequently or on a day-to-day basis.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The qualitative analysis used Nowell et al.'s recommendations to meet trustworthiness criteria.

“We used Nowell et al. 's [48] recommendations to meet the trustworthiness criteria.”

• cites [48] sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

A third researcher reviewed the codes to develop the final codes.

“A third researcher reviewed the codes to develop the fnal codes.”

May combine two claims.

Mystery Agent offered a low entry point and a comparatively less steep learning curve, since nearly all participants were not frequent VUI users and a few were first-time users.

“These ratings also refect that Mystery Agent ofered a low entry point and presented a comparatively less steep learning curve since nearly all the participants were not frequent users of VUIs, with a few participants indicating that this was their frst time using any VUI.”

Probably not a substantial claim: may not pass S3 (load-bearing).

This is consistent with the broader field of 'sociophonetics', which treats voice as an important design element.

“This is consistent with the broader feld of 'sociophonetics' [61], which treats voice as an important design element.”

• cites [61] sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Two researchers individually familiarized themselves with the primary data using reflexive journaling over two weeks.

“Two researchers individually familiarized themselves with the primary data using refexive journaling over two weeks.”

May combine two claims. The quote only partly states this claim.

Giving older adults more control makes the system more usable, evidenced by participants rating Mystery Agent's usability as good.

“Moreover, it makes the system more usable-evidenced by the participants rating Mystery Agent's usability as good.”

May combine two claims.

Internal reliability was excellent for likeability and intelligence and acceptable for trust.

“The internal reliability for likeability and intelligence was at an excellent level, and for trust it was at an acceptable level (summarized in Table 3).”

Classified as probably not a substantial claim.

Giving older adults more control over their interaction prepares them for conversational errors.

“Thus, giving them more control over their interaction and preparing them for conversational errors.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Internal consistency was good for positive affect and acceptable for negative affect.

“The internal consistency for positive afect was at a good level, and for negative afect, it was at an acceptable level.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

All three researchers have experience in qualitative analysis and a background in HCI.

“All three researchers have experience in qualitative analysis and a background in HCI.”

May combine two claims.

Older adults suffer a comparatively high rate of conversational errors, resulting in more confusion on how to progress or recover the conversation in case of a breakdown.

“Further, they sufer a comparatively high rate of conversational errors [38]-resulting in more confusion on how to progress or recover the conversation in the case of a breakdown.”

• cites [38] sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

The qualitative analysis used Thematic Analysis.

“The audio recordings were transcribed and analyzed by two researchers using Thematic Analysis [8].”

• cites [8] sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Codes were discussed over three data sessions until conflicts about interpretation, importance, and relevance were resolved.

“Over three data sessions, the codes were discussed until all conficts of opinion regarding the codes' interpretation, importance, and relevance were resolved.”

May combine two claims.

Although increased control increases cognitive load, it allows users to explore all interaction elements.

“Although this increases the cognitive load on the users, it allows them to explore all interaction elements.”

May combine two claims.

Participants were recruited via university email, so all participants were familiar with email and considerably tech-savvy.

“The participants were recruited via university email, so all participants were familiar with email and considerably tech-savvy.”

May combine two claims.

Unlike searching online health information using the internet, in which users would be able to track where they found the information, the quick and short answer provided by VUIs may be easy for users to grab, but lack opportunities to track the source of information.

“Unlike searching online health information using the internet, in which users would be able to track where they found the information, the quick and short answer provided by VUIs may be easy for users to grab, but lack of opportunities to track the source of information [52].”

• cites [52] sourceNot checked yet

  Not checked in the free run.

May combine two claims.

In a study with 54 participants, Bickmore et al. found that the health information participants' voice assistants shared, if applied in practice, could have resulted in some degree of harm in 29% and potential death in 16% of the participants' health situation instances.

“In a study with 54 participants, Bickmore et al. [7] found that the health information the participants' voice assistants shared, if it had been applied in practice, could have resulted in some degree of harm in 29% and potential death in 16% of the participants' health situation instances.”

• cites [7] sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Although SUS, Godspeed, and PANAS are popularly used in VUI literature, none of these scales were developed for VUIs, and their validity for VUIs remains limited.

“Although popularly used in VUI literature [57], none of these scales were developed for VUIs, and their validity for VUIs remains limited.”

• cites [57] sourceNot checked yet

  Not checked in the free run.

The quote only partly states this claim.

The Godspeed measures and PANAS used in the study demonstrated acceptable internal consistency, as indicated by Cronbach's alpha values.

“Table 3 : Reliability of the measures used in Godspeed measures and PANAS”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

This presents a limitation in our study and highlights a major gap in VUI literature.

“This presents a limitation in our study and highlights a major gap in VUI literature.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The lab environment was a limitation which could have increased participant anxiety, and contexts of home use could only be speculated.

“The lab environment was also a limitation which could have increased participant anxiety, and contexts of home use could only be speculated.”

May combine two claims.

All participants had a college degree, and nearly all participants were white, which is a major limitation.

“All participants had a college degree, and nearly all participants were white, which is a major limitation.”

Probably not a substantial claim: may not pass S2 (checkable).

The study's findings have implications for design that prioritizes older adults' specific needs and design priorities.

“More broadly, the fndings in this study have design implications for prioritizing older adults' specifc needs and design priorities.”