CoChat
Sign Up
Cited by 91Artificial Intelligence in Healthcare and EducationBRCA gene mutations in cancerMobile Health and mHealth Applications

Ask Rosa—the making of a digital genetic conversation tool, a chatbot, about hereditary breast and ovarian cancer

Open access
Siglen, Elen, Vetti, Hildegunn Høberg, Lunde, Aslaug Beathe Forberg, Hatlebrekke, Thomas Akselberg, Strømsvik, Nina, Hamang, Anniken, Hovland, Sigrid Tronsli, Rettberg, Jill Walker +2 more·2022·Patient Education and Counseling
DOI 10.1016/j.pec.2021.09.027
Open this paper in CoChat — free accountOpen PDF

Abstract

OBJECTIVE: We aimed at developing a pilot version of an app (Rosa) that can perform digital conversations with breast or ovarian cancer patients about genetic BRCA testing, using chatbot technology, to identify best practices for future patient-focused chatbots. METHODS: We chose a commercial chatbot platform and participatory methodology with a team of patient representatives, IT engineers, genetic counselors and clinical geneticists, within a nationwide collaboration. An iterative approach ensured extensive user and formal usability testing during the development process. RESULTS: The development phase lasted for two years until the pilot version was completed in December 2019. The iteration steps disclosed major challenges in the artificial intelligence (AI)-based matching of user provided questions with predefined information in the database, leading initially to high level of fallback answers. We therefore developed strategies to reduce potential language ambiguities (e.g. BRCA1 vs BRCA2) and overcome dialogue confusion. The first prototype contained a database with 500 predefined questions and 67 corresponding predefined answers, while the final version included 2257 predefined questions and 144 predefined answers. Despite the limited AI functionality of the chatbot, the testing revealed that the users liked the layout and found the chatbot trustworthy and reader friendly. CONCLUSIONS: Building a health chatbot is challenging, expensive and time consuming with today's technology. The users had a positive attitude to the chatbot, and would use it in a real life setting, if given to them by health care personnel. PRACTICE IMPLICATIONS: We here present a framework for future health chatbot initiatives. The participatory methodology in combination with an iterative approach ensured that the patient perspective was incorporated at every level of the development process. We strongly recommend this approach in patient-centered health innovations.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1016/j.pec.2021.09.027

Cites

2 of 2 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Patients are receptive to receiving education, treatment planning, and emotional support from AI systems.

    “Moreover, the high levels of user satisfaction reported in interventions such as Vivibot [14], Vik [17], Rosa [18], and social media–based studies [193257] underscore that patients are receptive to receiving education, treatment planning, and emotional support from AI systems.”

    From Empathic AI for Patient-Centered Cancer Care: A Scoping Review of Patient Navigation, Support, and Clinical Practice (2026) · cites this paper as [18]

    The Abstract (page unavailable) reports that Rosa’s testers liked the chatbot and would use it if recommended by health care personnel; the Conclusions describe its potential to help patients educate themselves before genetic counseling. This supports receptiveness to AI-delivered education, but the paper does not establish receptiveness to treatment planning. The Introduction (page unavailable) expressly says that chatbots cannot offer psychosocial support with today’s technology, contrary to the emotional-support part of the claim. (LLM, conf=0.88)

    Experimental — accuracy not yet measured

  • Chaix et al and Siglen et al reported that chatbot-delivered education improved adherence to treatment regimens and facilitated greater engagement in shared decision-making.

    “Similarly, Chaix et al [17] and Siglen et al [18] reported that chatbot-delivered education improved adherence to treatment regimens and facilitated greater engagement in shared decision-making.”

    From Empathic AI for Patient-Centered Cancer Care: A Scoping Review of Patient Navigation, Support, and Clinical Practice (2026) · cites this paper as [18]

    The Introduction and Conclusions describe Ask Rosa as a source of genetic information and suggest it may help patients prepare for counseling. Our copy does not report improved treatment adherence or greater engagement in shared decision-making; searches for both outcomes found no supporting passage. (LLM, conf=0.76)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 24

  1. Sun J, Meng H, Yao L, Lv M, Bai J, Zhang J, et al. Germline mutations in cancer susceptibility genes in a large series of unselected breast cancer patients. Pers Med Imaging 2017;23:6113-9. https://doi.org/10.1158/1078-0432.CCR-16-3227%. (J Clinical Cancer Researc). · DOI
  2. Høberg-Vetti H, Bjorvatn C, Fiane BE, Aas T, Woie K, Espelid H, et al. BRCA1/2 testing in newly diagnosed breast and ovarian cancer patients without prior genetic counselling: the DNA-BONus study. Eur J Hum Genet 2016;24:881-8. https://doi.org/10.1038/ejhg.2015.196 · DOI
  3. Pal T, Permuth-Wey J, Betts JA, Krischer JP, Fiorica J, Arango H, et al. BRCA1 and BRCA2 mutations account for a large proportion of ovarian carcinoma cases. Cancer 2005;104:2807-16. https://doi.org/10.1002/cncr.21536 · DOI
  4. Tung NM, Boughey JC, Pierce LJ, Robson ME, Bedrosian I, Dietz JR, et al. Management of hereditary breast cancer: American society of clinical oncology, American society for radiation oncology, and society of surgical oncology guideline. J Clin Oncol 2020;38:2080-106. https://doi.org/10.1200/jco.20.00299 · DOI
  5. Wright S, Porteous M, Stirling D, Lawton J, Young O, Gourley C, et al. Patients' views of Treatment-Focused Genetic Testing (TFGT): some lessons for the mainstreaming of BRCA1 and BRCA2 testing. J Genet Couns 2018;27:1459-72. https://doi.org/10.1007/s10897-018-0261-5 · DOI
  6. Høberg-Vetti H, Eide GE, Siglen E, Listøl W, Haavind MT, Hoogerbrugge N, et al. Cancer-related distress in unselected women with newly diagnosed breast or ovarian cancer undergoing BRCA1/2 testing without pretest genetic counseling. Acta Oncol 2019;58:175-81. https://doi.org/10.1080/0284186X. 2018.1502466 · DOI
  7. National Society of Genetic Counselors' Definition Task F, Resta R, Biesecker BB, Bennett RL, Blum S, Hahn SE, et al. A new definition of genetic counseling: na- tional society of genetic counselors' task force report. J Genet Couns 2006;15:77-83. https://doi.org/10.1007/s10897-005-9014-3 · DOI
  8. Forde C, Brunstrom K, Woodward E, Bowers N, Pereira M, Wallace AJ, et al. Uptake of pre-symptomatic testing for BRCA1 and BRCA2 is age, gender, off- spring and time-dependent. J Med Genet 2020;0:1-5. https://doi.org/10.1136/ jmedgenet-2019-106544 · DOI
  9. Wevers MR, Ausems MG, Verhoef S, Bleiker EM, Hahn DE, Brouwer T, et al. Does rapid genetic counseling and testing in newly diagnosed breast cancer patients cause additional psychosocial distress? Results from a randomized clinical trial. Genet Med 2016;18:117-26. https://doi.org/10.1038/gim.2015.50 · DOI
  10. Nilsson MP, Törngren T, Henriksson K, Kristoffersson U, Kvist A, Silfverberg B, et al. BRCAsearch: written pre-test information and BRCA1/2 germline mutation testing in unselected patients with newly diagnosed breast cancer. Breast Cancer Res Treat 2018;168:117-26. https://doi.org/10.1007/s10549-017-4584-y · DOI
  11. Kearney E, Wojcik A, Babu D. Artificial intelligence in genetic services delivery: utopia or apocalypse? J Genet Couns 2020;29:8-17. https://doi.org/10.1002/jgc4. 1192 · DOI
  12. Greene A, Greene CC, Greene C. Artificial intelligence, chatbots, and the future of medicine. Lancet Oncol 2019;20:481-2. https://doi.org/10.1016/S1470-2045(19) 30142-1 · DOI · Paper page
  13. Joerin A, Rauws M, Fulmer R, Black V. Ethical artificial intelligence for digital health organizations. Cureus 2020;12:e7202https://doi.org/10.7759/cureus.7202 · DOI
  14. Nazareth S, Nussbaum RL, Siglen E, Wicklund CA. Chatbots & artificial in- telligence to scale genetic information delivery. Chatbots Artif Intell Scale Genet Inf Deliv 2021;30:7-10. https://doi.org/10.1002/jgc4.1359 · DOI
  15. L. Reis, C. Maier, J. Mattke, T. Weitzel Chatbots in Healhcare: Status quo, appli- cation scenarios for physicians and patients and future directions Proceedings of the 28th European Conference on Information Systems (ECIS) 2020 An Online AIS Conference,.June 15-17, 2020〈https://aisel.aisnet.org/ecis2020_rp/163〉.
  16. Schmidlen T, Schwartz M, DiLoreto K, Kirchner HL, Sturm AC. Patient assessment of chatbots for the scalable delivery of genetic counseling. J Genet Couns 2019;28:1166-77. https://doi.org/10.1002/jgc4.1169 · DOI
  17. Simonsen JE, Robertson T. Routledge International Handbook of Participatory Design. London: Routledge,; 2012.
  18. Merle C. User and usability testing -how it should be undertaken? Australas J Educ Technol 1995;11. https://doi.org/10.14742/ajet.2075 · DOI
  19. Cameron G, Cameron DW, Megaw G, Bond D, Mulvenna M, O'Neill S, et al. Best practices for designing chatbots in mental healthcare -a case study on iHelpr. Proceedings of the 32nd International BCS Human Computer Interaction Conference 2018;129:1-5. https://doi.org/10.14236/ewic/HCI2018.129 · DOI
  20. Baharuddin R, Singh D, Razali R. Usability dimensions for mobile applications -a review. Res J Appl Sci Eng Technol 2013;5:2225-31. https://doi.org/10.19026/ rjaset.5.4776 · DOI
  21. Zapata BC, Fernández-Alemán JL, Idri A, Toval AIA. Empirical studies on usability of mHealth apps: a systematic literature review. J Med Syst 2015;39:1. https:// doi.org/10.1007/s10916-014-0182-2 · DOI · Paper page
  22. Hays RT, Singer MJ. Simulation fidelity in training system design. Bridging the Gap Between Reality and Training. Springer-Verlag,; 1989.
  23. Reshmi S, Balakrishnan K. Implementation of an inquisitive chatbot for database supported knowledge bases. Sādhanā 2016;41:1173-8. https://doi.org/10.1007/ s12046-016-0544-1 · DOI
  24. Powell J, Me Trust. I'm a chatbot: how artificial intelligence in health care fails the turing test. J Med Internet Res 2019;21:e16222https://doi.org/10.2196/16222 · DOI · Paper page

Cited by

Sorted by citations

Authors
Siglen, E., Vetti, H. H., Lunde, A. B. F., Hatlebrekke, T. A., Strømsvik, N., Hamang, A., Hovland, S. T., Rettberg, J. W., Steen, V. M., & Bjorvatn, C.
Affiliations
  • Haukeland University Hospital (NO)
  • University Hospital of North Norway (NO)
  • St Olav's University Hospital (NO)
  • Oslo University Hospital (NO)
  • University of Bergen (NO)
Corresponding author
Siglen, Elen
ORCID
Venue
Patient Education and Counseling
Publisher
Elsevier BV
Published
2022-06
Volume
105
Issue
6
Pages
1488-1494
ISSN
0738-3991 (ISSN-L)
DOI
10.1016/j.pec.2021.09.027
OpenAlex
W3203698165
Cited by
91
References
2
Citations by year
2026: 19 · 2025: 26 · 2024: 13 · 2023: 25 · 2022: 7 · 2021: 1
FWCI
3.35
Type
journal article
Language
en
Open access
Licence
Locations
  • Patient Education and Counseling (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • NORA - Norwegian Open Research Archives (submittedVersion, cc-by)
Topics
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Genetics › BRCA gene mutations in cancer
  • Health Sciences › Health Professions › General Health Professions › Mobile Health and mHealth Applications
Keywords
chatbots, hereditary breast and ovarian cancer, BRCA testing, genetic counseling, participatory design, usability testing, digital health tools
MeSH terms
Artificial Intelligence, Communication, Female, Humans, Ovarian Neoplasms, Software, Rosa

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review