kBot: Knowledge-Enabled Personalized Chatbot for Asthma Self-Management

Half of the included studies (n=6) did not report any system measures.

“Half of the studies (n=6) did not report on any system measures [59,60,62,65-67]”

Three studies reported technology acceptance through a System Usability Survey (SUS).

“A total of 3 studies [60,66,67] reported technology acceptance through a System Usability Survey (SUS).”

Two included studies used validated questionnaires only, 2 used adapted questionnaires only, and 1 used both validated and adapted questionnaires.

“Although 2 studies [60,69] used validated questionnaires only and 2 [62,67] used adapted questionnaires only, 1 study used both validated and adapted questionnaires [66].”

The majority of included studies (n=7) did not describe the user interface of their VCAs, whereas the remaining 5 papers did.

“The majority of the included studies (n=7) did not describe the user interface of their VCAs [60-62,64,68,70], whereas the remaining 5 papers did [59,65-67,69].”

Smartphones were the most used device to test the VCA (n=5), followed by smart speakers (n=3) and tablets (n=2).

“When considering the devices used to test the VCA, we found that smartphones were the most used (n=5) [61,65-67,69], followed by smart speakers (n=3) [62,64,68] and tablets (n=2) [59,60].”

The vast majority of VCAs (n=10) were not commercially available at the time of the systematic literature review.

“The vast majority of the VCAs (n=10) were not commercially available [59,60,62-69] at the time of this systematic literature review.”

Only 3 of the 7 studies that included a measure of technology acceptance through a questionnaire used a validated questionnaire, whereas the others adapted them.

“In particular, only 3 [60,66,69] of the 7 studies that included a measure of technology acceptance through a questionnaire [59,60,62,65-67,69] used a validated questionnaire, whereas the others adapted them.”

Ten of the selected papers developed and evaluated a VCA, whereas 2 aimed to report a criterion-based performance evaluation of existing commercial conversational agents.

“The majority (n=10) of the selected papers developed and evaluated VCA [59,60,62-69], whereas 2 [61,70] aimed to report a criterion-based performance evaluation of existing commercial conversational agents (eg, Google Assistant and Apple Siri).”

Among papers developing and evaluating a VCA, 6 assessed technology acceptance, 3 assessed system accuracy, and 1 assessed both.

“Among the papers developing and evaluating a VCA, 6 [59,60,62,65-67] assessed the technology acceptance of the VCA, whereas 3 [63,64,68] assessed the system accuracy. Only one [69] assessed both performance and acceptance.”

Of the 12 studies, 7 reported technology acceptance findings, whereas 5 did not.

“Of the 12 studies, 7 [59,60,62,65-67,69] reported technology acceptance findings, whereas the others (n=5) did not [61,63,64,68,70].”

Four papers did not explicitly specify the study design they used, whereas the other papers provided labels.

“A total of 4 papers [61,66,68,70] did not explicitly specify the study design they used, whereas the other papers provided labels.”

The majority of the studies (n=10) did not report any behavioral measures, whereas 2 papers did.

“The majority of the studies (n=10) did not report any behavioral measures [59-63,65-67,69,70], whereas 2 papers [64,68] did.”

The majority of included studies (n=10) did not provide measures of attitude toward the target health behavior.

“The majority of the included studies (n=10) did not provide measures of attitude toward the target health behavior [61-70].”

The majority of included studies (n=8) did not report controlling for participants' previous experience with technology.

“The majority of the included studies (n=8) also did not report controlling for participants’ previous experience with technology [59-63,66,69,70].”

Of the 12 included studies, 7 reported technology acceptance measures, whereas the remaining 5 did not.

“Of the 12 studies, 7 [59,60,62,65-67,69] reported technology acceptance measures, whereas the remaining studies [61,63,64,68,70] did not.”

Other discussed conditions were hearing impairment [68], asthma [66], and Parkinson disease [65].

“Other discussed conditions were hearing impairment [68], asthma [66], and Parkinson disease [65].”

Eight included studies named their VCA, whereas 2 did not specify any name.

“A total of 8 studies [60,62,63,65-69] named their VCA, whereas 2 studies [59,64] did not specify any name (Multimedia Appendix 5).”

The underlying architecture of the investigated VCAs was described in 7 included studies, whereas 3 papers did not provide this information.

“The underlying architecture of the investigated VCAs was described in 7 of the included studies [62,63,66-70], whereas 3 papers did not provide this information [61,64,65].”

The other included studies targeted hearing-impaired individuals [68], smokers [61], patients with asthma [66], patients with glaucoma and diabetes [69], people with intellectual disability [64], and patients with chronic heart failure [63].

“The others targeted hearing-impaired individuals [68], smokers [61], patients with asthma [66], patients with glaucoma and diabetes [69], people with intellectual disability [64], and patients with chronic heart failure [63].”

Three included studies provided a lookup of health information only, whereas 2 also provided targeted alerts and reminders.

“A total of 3 studies provided such a lookup of health information only [61,62,66], whereas 2 [62,66] also provided targeted alerts and reminders.”

Monitoring activities were mainly implemented as active data capture and documentation (n=5), whereas 1 study also focused on self-monitoring of health or diagnostic data.

“Monitoring activities were mainly implemented as active data capture and documentation (n=5) [62,63,66-69], whereas 1 study [66] also focused on self-monitoring of health or diagnostic data.”

Five VCAs exclusively focused on support, 3 exclusively focused on monitoring, and 4 investigated a VCA providing both monitoring and support.

“A total of 5 VCAs [59-61,64,70] exclusively focused on support, and 3 studies [63,65,68] exclusively focused on monitoring. In total, 4 studies investigated a VCA providing both monitoring and support [62,66,67,69].”

The usability level (SUS score of 61.6) is comparable with the 2 psychological therapy chatbots with SUS scores of 63.6 and 57.0 but lower than that of asthma management (SUS score of about 83) and depression prevention chatbots (SUS score of approximately 75.7).

“This level of usability (SUS scores of 61.6) is comparable with the 2 psychological therapy chatbots with SUS scores of 63.6 and 57.0 [34] but lower than that of asthma management (SUS score of about 83) [35] and depression prevention chatbots (SUS score of approximately 75.7) [36].”