CoChat
Sign Up
PreprintCited by 14Cervical Cancer and HPV ResearchSocial Robot Interaction and HRIMobile Health and mHealth Applications

Examining Potential Usability and Health Beliefs Among Young Adults Using a Conversational Agent for HPV Vaccine Counseling

Open access
Amith, Muhammad, Lin, Rebecca, Cunningham, Rachel, Wu, Qiwei Luna, Savas, Lara S., Gong, Yang, Boom, Julie A., Tang, Lu +1 more·2020·arXiv
DOI 10.48550/arxiv.2001.02306arXiv 2001.02306PMC PMC7233050
Open this paper in CoChat — free account

Abstract

The human papillomavirus (HPV) vaccine is the most effective way to prevent HPV-related cancers. Integrating provider vaccine counseling is crucial to improving HPV vaccine completion rates. Automating the counseling experience through a conversational agent could help improve HPV vaccine coverage and reduce the burden of vaccine counseling for providers. In a previous study, we tested a simulated conversational agent that provided HPV vaccine counseling for parents using the Wizard of OZ protocol. In the current study, we assessed the conversational agent among young college adults (n=24), a population that may have missed the HPV vaccine during their adolescence when vaccination is recommended. We also administered surveys for system and voice usability, and for health beliefs concerning the HPV vaccine. Participants perceived the agent to have high usability that is slightly better or equivalent to other voice interactive interfaces, and there is some evidence that the agent impacted their beliefs concerning the harms, uncertainty, and risk denials for the HPV vaccine. Overall, this study demonstrates the potential for conversational agents to be an impactful tool for health promotion endeavors.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2001.02306

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Half of the included studies (n=6) did not report any system measures.

    “Half of the studies (n=6) did not report on any system measures [59,60,62,65-67]”

    From Voice-Based Conversational Agents for the Prevention and Management of Chronic and Mental Health Conditions: Systematic Literature Review (2021) · cites this paper as [60]

    The cited paper reports measures of its conversational agent rather than reporting none. Its Methods (p. 4) identify the SUS and SUISQ instruments, and its Results (pp. 4–5) report system-usability and speech-interface scores. The paper cannot establish how many of the other included studies reported measures. (LLM, conf=0.80)

    Checked by a reasoning model — may be wrong.

    Experimental — accuracy not yet measured

  • Three included studies targeted older people [62,67,70], whereas 2 targeted either parents of adolescents [59] or pediatric patients [60].

    “When inspecting the target population, we observed that 3 of the included studies [62,67,70] targeted older people, whereas 2 targeted either parents of adolescents [59] or pediatric patients [60].”

    From Voice-Based Conversational Agents for the Prevention and Management of Chronic and Mental Health Conditions: Systematic Literature Review (2021) · cites this paper as [60]

    The Methods (p. 3) say participants were young adults aged 18–26, with an average age of 20, not pediatric patients. The Discussion (p. 7) distinguishes this recruited group from parents of pediatric patients, whom it describes as another intended demographic. (LLM, conf=0.90)

    Checked by a reasoning model — may be wrong.

    Experimental — accuracy not yet measured

  • Three included studies targeted older people [62,67,70], whereas 2 targeted either parents of adolescents [59] or pediatric patients [60].

    “When inspecting the target population, we observed that 3 of the included studies [62,67,70] targeted older people, whereas 2 targeted either parents of adolescents [59] or pediatric patients [60].”

    From Voice-Based Conversational Agents for the Prevention and Management of Chronic and Mental Health Conditions: Systematic Literature Review (2021) · cites this paper as [59]

    The Methods (p. 3) say this study recruited young adults aged 18–26, not pediatric patients. The Discussion (p. 7) distinguishes those participants from parents of pediatric patients, whom the authors describe as another intended demographic for the conversational agent. (LLM, conf=0.90)

    Checked by a reasoning model — may be wrong.

    Experimental — accuracy not yet measured

  • Two included studies could not provide any architectural information because of their WOz study design.

    “A total of 2 studies [59,60] could not provide any architectural information, given the nature of their study design (ie, WOz).”

    From Voice-Based Conversational Agents for the Prevention and Management of Chronic and Mental Health Conditions: Systematic Literature Review (2021) · cites this paper as [60]

    The Methods (p. 3) confirm that this was a Wizard of Oz study, but the Methods (p. 4) provide architectural information: the speech-interaction system used iOS SDK for a tablet and Java for a desktop application that interacted with it remotely. The Conclusion (p. 7) says a fully automated agent had not yet been tested; that does not support the claim that the study could provide no architectural information at all. (LLM, conf=0.86)

    Checked by a reasoning model — may be wrong.

    Experimental — accuracy not yet measured

  • Of the 12 studies, 7 reported technology acceptance findings, whereas 5 did not.

    “Of the 12 studies, 7 [59,60,62,65-67,69] reported technology acceptance findings, whereas the others (n=5) did not [61,63,64,68,70].”

    From Voice-Based Conversational Agents for the Prevention and Management of Chronic and Mental Health Conditions: Systematic Literature Review (2021) · cites this paper as [60]

    The Results (p. 4) report usability findings for this conversational agent, so this paper can serve as an example of a study reporting user-facing technology findings. It does not review the 12 studies or establish that seven reported technology acceptance findings and five did not; that tally cannot be verified from this source. (LLM, conf=0.82)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

No references could be resolved yet.

Research works printed but not linked: 59

  1. [3]3. Gilkey MB, Calo WA, Moss JL, et al. Provider communication and HPV vaccination: The impact of recommendation quality. Vaccine 2016; 34: 1187-1192.
  2. [4]4. Lau M, Lin H and Flores G. Factors associated with human papillomavirus vaccine-series initiation and healthcare provider recommendation in US adolescent females: 2007 National Survey of Children's Health. Vaccine 2012; 30: 3112-3118.
  3. [5]5. Dempsey AF, Pyrzanowski J, Lockhart S, et al. Parents' perceptions of provider communication regarding adolescent vaccines. Human vaccines & immunotherapeutics 2016; 12: 1469-1475.
  4. [6]6. Bratic JS, Seyferth ER and Bocchini JA. Update on barriers to human papillomavirus vaccination and effective strategies to promote vaccine acceptance. Current opinion in pediatrics 2016; 28: 407-412.
  5. [7]7. Gerend MA, Shepherd MA, Lustria MLA, et al. Predictors of provider recommendation for HPV vaccine among young adult men and women: findings from a cross-sectional survey. Sex Transm Infect 2016; 92: 104-107.
  6. [8]8. Moss JL, Reiter PL, Rimer BK, et al. Collaborative patient-provider communication and uptake of adolescent vaccines. Social Science & Medicine 2016; 159: 100-107.
  7. [9]9. Smith PJ, Stokley S, Bednarczyk RA, et al. HPV vaccination coverage of teen girls: the influence of health care providers. Vaccine 2016; 34: 1604-1610.
  8. [10]10. Valentino K and Poronsky CB. Human papillomavirus infection and vaccination. Journal of pediatric nursing 2016; 31: e155-e166.
  9. [12]12. Ong LM, De Haes JC, Hoos AM, et al. Doctor-patient communication: a review of the literature. Social science & medicine 1995; 40: 903-918.
  10. [13]13. Strecher VJ. Improving physician-patient interactions: a review. Patient counselling and health education 1983; 4: 129-136.
  11. [14]14. Larsen KM and Smith CK. Assessment of nonverbal communication in the patient-physician interview. J Fam Pract 1981; 12: 481-488.
  12. [15]15. Buller MK and Buller DB. Physicians' communication style and patient satisfaction. Journal of health and social behavior 1987: 375-388.
  13. [16]16. Arora NK. Interacting with cancer patients: the significance of physicians' communication behavior. Social science & medicine 2003; 57: 791-806.
  14. [17]17. Street Jr RL, Makoul G, Arora NK, et al. How does communication heal? Pathways linking clinician-patient communication to health outcomes. Patient education and counseling 2009; 74: 295-301.
  15. [18]18. Stewart MA. Effective physician-patient communication and health outcomes: a review. CMAJ: Canadian Medical Association Journal 1995; 152: 1423.
  16. [19]19. Stewart M, Brown JB, Donner A, et al. The impact of patient-centered care on outcomes. The Journal of Family Practice 2000; 49: 796-804.
  17. [20]20. Miller DP, Brownlee CD, McCoy TP, et al. The effect of health literacy on knowledge and receipt of colorectal cancer screening: a survey study. BMC family practice 2007; 8: 16.
  18. [21]21. Scott TL, Gazmararian JA, Williams MV, et al. Health literacy and preventive health care use among Medicare enrollees in a managed care organization. Medical care 2002; 40: 395-404.
  19. [22]22. Miller TA. Health literacy and adherence to medical treatment in chronic and acute illness: A meta-analysis. Patient education and counseling 2016; 99: 1079-1086.
  20. [23]23. Epstein RM, Fiscella K, Lesser CS, et al. Why the nation needs a policy push on patient-centered health care. Health affairs 2010; 29: 1489-1495.
  21. [24]24. Teutsch C. Patient-doctor communication. Medical Clinics of North America 2003; 87: 1115-1146.
  22. [25]25. Finney Rutten LJ, Augustson E and Wanke K. Factors associated with patients' perceptions of health care providers' communication behavior. Journal of Health Communication 2006; 11: 135-146.
  23. [26]26. Ha JF and Longnecker N. Doctor-patient communication: a review. Ochsner Journal 2010; 10: 38-43.
  24. [27]27. Levinson W, Lesser CS and Epstein RM. Developing physician communication skills for patient-centered care. Health affairs 2010; 29: 1310-1318.
  25. [28]28. Stewart M, Brown JB, Boon H, et al. Evidence on patient-doctor communication. Cancer 1999; 25: 30.
  26. [29]29. Tongue JR, Epps HR and Forese LL. Communication skills for patient-centered care: research-based, easily learned techniques for medical interviews that benefit orthopaedic surgeons and their patients. JBJS 2005; 87: 652-658.
  27. [30]30. Epstein RM, Franks P, Shields CG, et al. Patient-centered communication and diagnostic testing. The Annals of Family Medicine 2005; 3: 415-421.
  28. [31]31. Rosenthal S, Weiss TW, Zimet GD, et al. Predictors of HPV vaccine uptake among women aged 19-26: importance of a physician's recommendation. Vaccine 2011; 29: 890-895.
  29. [32]32. Kester LM, Zimet GD, Fortenberry JD, et al. A national study of HPV vaccination of adolescent girls: rates, predictors, and reasons for non-vaccination. Maternal and child health journal 2013; 17: 879-885.
  30. [33]33. Gilkey MB, Malo TL, Shah PD, et al. Quality of physician communication about human papillomavirus vaccine: findings from a national survey. Cancer Epidemiology and Prevention Biomarkers 2015; 24: 1673-1679.
  31. [34]34. Vadaparampil ST, Kahn JA, Salmon D, et al. Missed clinical opportunities: provider recommendations for HPV vaccination for 11-12 year old girls are limited. Vaccine 2011; 29: 8634-8641. 2011/09/20. DOI: 10.1016/j.vaccine.2011.09.006. · DOI
  32. [35]35. Daley MF, Crane LA, Markowitz LE, et al. Human papillomavirus vaccination practices: a survey of US physicians 18 months after licensure. Pediatrics 2010; 126: 425-433.
  33. [36]36. Katz ML, Krieger JL and Roberto AJ. Human papillomavirus (HPV): college male's knowledge, perceived risk, sources of information, vaccine barriers and communication. Journal of men's health 2011; 8: 175-184.
  34. [37]37. Ahken S, Fleming N, Dumont T, et al. HPV Awareness in Higher-Risk Young Women: The Need for a Targeted HPV Catch-Up Vaccination Program. Journal of Obstetrics and Gynaecology Canada 2015; 37: 122-128. DOI: 10.1016/S1701-2163(15)30333-9. · DOI
  35. [38]38. Albright AE and Allen RS. HPV misconceptions among college students: The role of health literacy. Journal of community health 2018; 43: 1192-1200.
  36. [39]39. Brown DR, Shew ML, Qadadri B, et al. A longitudinal study of genital human papillomavirus infection in a cohort of closely followed adolescent women. Journal of Infectious Diseases 2005; 191: 182-192.
  37. [40]40. Bosch FX and de Sanjosé S. The epidemiology of human papillomavirus infection and cervical cancer. Disease markers 2007; 23: 213-227.
  38. [41]41. Amith M, Anna Z, Cunningham R, et al. Early usability assessment of a conversational agent for HPV vaccination. Studies in health technology and informatics 2019; 257: 17.
  39. [42]42. Fraser NM and Gilbert GN. Simulating speech systems. Computer Speech & Language 1991; 5: 81-99.
  40. [44]44. Janz NK and Becker MH. The health belief model: A decade later. Health education quarterly 1984; 11: 1-47.
  41. [45]45. Brooke J. SUS: a retrospective. Journal of usability studies 2013; 8: 29-40.
  42. [46]46. Brooke J. SUS-A quick and dirty usability scale. Usability evaluation in industry 1996; 189: 4-7.
  43. [48]48. Polkosky MD. Machines as mediators: The challenge of technology for interpersonal communication theory and research. Mediated interpersonal communication . Routledge, 2008, pp.48-71.
  44. [49]49. McRee A-L, Brewer NT, Reiter PL, et al. The Carolina HPV immunization attitudes and beliefs scale (CHIAS): scale development and associations with intentions to vaccinate. Sexually transmitted diseases 2010; 37: 234239.
  45. [50]50. Sauro J and Lewis JR. Quantifying the user experience: Practical statistics for user research . Morgan Kaufmann, 2016.
  46. [51]51. Bangor A, Kortum P and Miller J. Determining what individual SUS scores mean: Adding an adjective rating scale. Journal of usability studies 2009; 4: 114-123.
  47. [52]52. Lewis JR and Hardzinski ML. Investigating the psychometric properties of the Speech User Interface Service Quality questionnaire. International Journal of Speech Technology 2015; 18: 479-487.
  48. [53]53. Brewer NT, Gottlieb SL, Reiter PL, et al. Longitudinal predictors of HPV vaccine initiation among adolescent girls in a high-risk geographic area. Sexually transmitted diseases 2011; 38: 197.
  49. [54]54. Nicol C, Duska L, Hornsby P, et al. HPV vaccination rates and attitudes: A cross-sectional survey of college men and women. Gynecologic Oncology 2016; 141: 112.
  50. [55]55. Dempsey AF, Fuhrel-Forbis A and Konrath S. Use of the Carolina HPV Immunization Attitudes and Beliefs Scale (CHIAS) in young adult women. PloS one 2014; 9: e100193.
  51. [56]56. Kamimura A, Trinh HN, Weaver S, et al. Knowledge and beliefs about HPV among college students in Vietnam and the United States. Journal of infection and public health 2018; 11: 120-125.
  52. [57]57. Hanson KE, McLean HQ, Belongia EA, et al. Sociodemographic and clinical correlates of human papillomavirus vaccine attitudes and receipt among Wisconsin adolescents. Papillomavirus Research 2019; 8: 100168.
  53. [58]58. Lewis JR and Sauro J. Can I leave this one out?: the effect of dropping an item from the SUS. Journal of Usability Studies 2017; 13: 38-46.
  54. [61]61. Amith M, Gong Y, Cunningham R, et al. Developing VISO: vaccine information statement ontology for patient education. Journal of biomedical semantics 2015; 6: 23.
  55. [62]62. Wang D, Cunningham R, Boom J, et al. Towards a HPV Vaccine Knowledgebase for Patient Education Content. Studies in health technology and informatics 2016; 225: 432.
  56. [63]63. Lin R, Liang C, Duan R, et al. Visualized Emotion Ontology: a model for representing visual cues of emotions. BMC medical informatics and decision making 2018; 18: 64.
  57. [64]64. Amith M, Roberts K and Tao C. Conceiving an application ontology to model patient human papillomavirus vaccine counseling for dialogue management. BMC Bioinformatics In Press.
  58. [65]65. Amith M, Lin R, Cui L, et al. An Ontology-Powered Dialogue Engine For Patient Communication of Vaccines. 4th International Workshop on Semantics- Powered Data Mining and Analytics (SEPDA 2019) in Conjunction with the 19th International Semantic Web Conference (ISWC 2019) . Aukland, New Zealand 2019.
  59. [66]66. Amith M, Lin R, Liang C, et al. VEO-Engine: Interfacing and Reasoning with an Emotion Ontology for Device Visual Expression. In: International Conference on Human-Computer Interaction 2018, pp.349-355. Springer.

Cited by

Sorted by citations

Authors
Amith, M., Lin, R., Cunningham, R., Wu, Q. L., Savas, L. S., Gong, Y., Boom, J. A., Tang, L., & Tao, C.
Affiliations
  • The University of Texas Health Science Center at Houston (US)
  • Johns Hopkins University (US)
  • Texas Children's Hospital (US)
  • Texas A&M University (US)
  • Texas Medical Center (US)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2020
Volume
2020
Pages
43-52
DOI
10.48550/arxiv.2001.02306
arXiv
2001.02306
PMC
PMC7233050
OpenAlex
W2999713800
Cited by
14
References
0
Citations by year
2025: 1 · 2024: 4 · 2023: 4 · 2022: 2 · 2021: 2 · 2020: 1
FWCI
1.61
Type
journal article
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • PubMed Central (publishedVersion)
  • PubMed (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
  • PubMed
Funders
  • University of Texas Health Science Center at Houston (RP160015)
  • National Institutes of Health (RP160015)
  • National Institute of Allergy and Infectious Diseases (R01AI130460)
  • U.S. National Library of Medicine (R01LM012607, R01LM011829)
Topics
  • Health Sciences › Medicine › Epidemiology › Cervical Cancer and HPV Research
  • Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
  • Health Sciences › Health Professions › General Health Professions › Mobile Health and mHealth Applications
Keywords
HPV vaccine, vaccine counseling, chatbots, voice assistants, young adults, usability, health beliefs, vaccine hesitancy, missed vaccination, health promotion

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review