CoChat
Sign Up
Cited by 59Expert finding and Q&A systemsMisinformation and Its ImpactsCommunication and COVID-19 Impact

Powering an AI Chatbot with Expert Sourcing to Support Credible Health Information Access

Open access
Xiao, Ziang, Liao, Q. Vera, Zhou, Michelle, Grandison, Tyrone, Li, Yunyao·2023·Proceedings of the 28th International Conference on Intelligent User Interfaces
DOI 10.1145/3581641.3584031
Open this paper in CoChat — free accountOpen PDF

Abstract

During a public health crisis like the COVID-19 pandemic, a credible and easy-to-access information portal is highly desirable. It helps with disease prevention, public health planning, and misinformation mitigation. However, creating such an information portal is challenging because 1) domain expertise is required to identify and curate credible and intelligible content, 2) the information needs to be updated promptly in response to the fast-changing environment, and 3) the information should be easily accessible by the general public; which is particularly difficult when most people do not have the domain expertise about the crisis. In this paper, we presented an expert-sourcing framework and created Jennifer, an AI chatbot, which serves as a credible and easy-to-access information portal for individuals during the COVID-19 pandemic. Jennifer was created by a team of over 150 scientists and health professionals around the world, deployed in the real world and answered thousands of user questions about COVID-19. We evaluated Jennifer from two key stakeholders’ perspectives, expert volunteers and information seekers. We first interviewed experts who contributed to the collaborative creation of Jennifer to learn about the challenges in the process and opportunities for future improvement. We then conducted an online experiment that examined Jennifer’s effectiveness in supporting information seekers in locating COVID-19 information and gaining their trust. We share the key lessons learned and discuss design implications for building expert-sourced and AI-powered information portals, along with the risks and opportunities of misinformation mitigation and beyond.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3581641.3584031

Cites

7 of 7 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Integration with EHR may come across several obstacles, including demand for budget and resources, handling sensitive information, and interoperability issues.

    “Admittedly, integration with EHR may come across several obstacles, including demand for budget and resources, handling sensitive information, and interoperability issues [33,52,118].”

    From Talk2Care: An LLM-based Voice Assistant for Communication between Healthcare Providers and Older Adults (2024) · cites this paper as [118]

    The Introduction and §6.3 discuss the effort and resources needed to maintain a public-health information chatbot, not to integrate it with EHRs. Searches for EHR, electronic health record, interoperability, and sensitive information found no passage addressing the claimed integration obstacles. (LLM, conf=0.76)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 79

  1. Bill Adair, Chengkai Li, Jun Yang, and Cong Yu. 2017. Progress Toward "the Holy Grail": The Continued Quest to Automate Fact-Checking. In Proceedings of the 2017 Computation+Journalism Symposium.
  2. Emily M Agree, Abby C King, Cynthia M Castro, Adrienne Wiley, and Dina LG Borzekowski. 2015. "It's got to be on this page": Age and cognitive style in a study of online health information seeking. Journal of medical Internet research 17, 3 (2015), e79.
  3. Mabrook S Al-Rakhami and Atif M Al-Amri. 2020. Lies Kill, Facts Save: Detecting COVID-19 Misinformation in Twitter. IEEE Access 8 (2020), 155961-155970.
  4. Firoj Alam, Ferda Ofli, and Muhammad Imran. 2018. Processing social media images by combining human and machine computing during crises. International Journal of Human-Computer Interaction 34, 4 (2018), 311-327.
  5. Sacha Altay, Anne-Sophie Hacquin, Coralie Chevallier, and Hugo Mercier. 2021. Information delivered by a chatbot has a positive impact on COVID-19 vaccines attitudes and intentions. Journal of Experimental Psychology: Applied (2021).
  6. Cynthia Andrews, Elodie Fichet, Yuwei Ding, Emma S Spiro, and Kate Starbird. 2016. Keeping up with the tweet-dashians: The impact of'official'accounts on on- line rumoring. In Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing. 452-465.
  7. Ahmer Arif, Kelley Shanahan, Fang-Ju Chou, Yoanna Dosouto, Kate Starbird, and Emma S Spiro. 2016. How information snowballs: Exploring the role of exposure in online rumor propagation. In Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing. 466-477.
  8. Ana I Bento, Thuy Nguyen, Coady Wing, Felipe Lozano-Rojas, Yong-Yeol Ahn, and Kosali Simon. 2020. Evidence from internet search data shows information- seeking responses to news of local COVID-19 cases. Proceedings of the National Academy of Sciences 117, 21 (2020), 11220-11222.
  9. Gretchen K Berland, Marc N Elliott, Leo S Morales, Jeffrey I Algazy, Richard L Kravitz, Michael S Broder, David E Kanouse, Jorge A Muñoz, Juan-Antonio Puyol, Marielena Lara, et al. 2001. Health information on the Internet: accessibility, quality, and readability in English and Spanish. jama 285, 20 (2001), 2612-2621.
  10. Michael S Bernstein, Greg Little, Robert C Miller, Björn Hartmann, Mark S Ackerman, David R Karger, David Crowell, and Katrina Panovich. 2010. Soylent: a word processor with a crowd inside. In Proceedings of the 23nd annual ACM symposium on User interface software and technology. 313-322.
  11. Timothy W Bickmore, Dina Utami, Robin Matsuyama, and Michael K Paasche- Orlow. 2016. Improving access to online health information with conversational agents: a randomized controlled experiment. Journal of medical Internet research 18, 1 (2016), e1.
  12. Leticia Bode and Emily K Vraga. 2018. See something, say something: Correction of global health misinformation on social media. Health communication 33, 9 (2018), 1131-1140.
  13. Maged N Kamel Boulos, Bernd Resch, David N Crowley, John G Breslin, Gunho Sohn, Russ Burtner, William A Pike, Eduardo Jezierski, and Kuo-Yu Slayer Chuang. 2011. Crowdsourcing, citizen sensing and sensor web technologies for public and environmental health surveillance and crisis management: trends, OGC standards and application examples. International journal of health geographics 10, 1 (2011), 1-29.
  14. Josip Bozic, Oliver A Tazl, and Franz Wotawa. 2019. Chatbot testing using AI planning. In 2019 IEEE International Conference On Artificial Intelligence Testing (AITest). IEEE, 37-44.
  15. J Scott Brennen, Felix Simon, Philip N Howard, and Rasmus Kleis Nielsen. 2020. Types, sources, and claims of COVID-19 misinformation. Reuters Institute 7 (2020), 3-1.
  16. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Pra- fulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language Models are Few-Shot Learners. In Advances in Neural Information Processing Systems, H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (Eds.), Vol. 33. Curran Associates, Inc., 1877-1901. https://proceedings. neurips.cc/paper/2020/file/1457c0d6bfcb4967418bfb8ac142f64a-Paper.pdf
  17. Jiyoung Chae. 2016. Who avoids cancer information? Examining a psychological process leading to cancer information avoidance. Journal of health communication 21, 7 (2016), 837-844.
  18. Yan Chen, Steve Oney, and Walter S Lasecki. 2016. Towards providing on-demand expert support for software developers. In Proceedings of the 2016 CHI conference on human factors in computing systems. 3192-3203.
  19. Seonhwa Choi and Byunggul Bae. 2015. The real-time monitoring system of social big data for disaster management. In Computer science and its applications. Springer, 809-815.
  20. Mark É Czeisler, Michael A Tynan, Mark E Howard, Sally Honeycutt, Erika B Fulmer, Daniel P Kidder, Rebecca Robbins, Laura K Barger, Elise R Facer-Childs, Grant Baldwin, et al. 2020. Public attitudes, behaviors, and beliefs related to COVID-19, stay-at-home orders, nonessential business closures, and public health guidance-United States, New York City, and Los Angeles, May 5-12, 2020. Mor- bidity and Mortality Weekly Report 69, 24 (2020), 751.
  21. Eamonn Fahy, Rohan Hardikar, Adrian Fox, and Sean Mackay. 2014. Quality of patient health information on the Internet: reviewing a complex and evolving landscape. The Australasian medical journal 7, 1 (2014), 24.
  22. Julian J Faraway. 2016. Extending the linear model with R: generalized linear, mixed effects and nonparametric regression models. CRC press.
  23. Asbjørn Følstad and Petter Bae Brandtzæg. 2017. Chatbots and the new world of HCI. interactions 24, 4 (2017), 38-42.
  24. Allen Foster. 2004. A nonlinear model of information-seeking behavior. Journal of the American society for information science and technology 55, 3 (2004), 228-237.
  25. Mingkun Gao, Ziang Xiao, Karrie Karahalios, and Wai-Tat Fu. 2018. To label or not to label: The effect of stance and credibility labels on readers' selection and perception of news articles. Proceedings of the ACM on Human-Computer Interaction 2, CSCW (2018), 1-16.
  26. Miaomiao Gong, Yuling Sun, and Liang He. 2019. A Social Network Engaged Crowdsourcing Framework for Expert Tasks. In 2019 IEEE 23rd International Conference on Computer Supported Cooperative Work in Design (CSCWD). IEEE, 249-254.
  27. Yuanyuan Guo, Zhenzhen Lu, Haibo Kuang, and Chaoyou Wang. 2020. Infor- mation avoidance behavior on social network sites: Information irrelevance, overload, and the moderating role of time pressure. International Journal of Information Management 52 (2020), 102067.
  28. Christine Hagar. 2015. Crisis informatics. In Encyclopedia of Information Science and Technology, Third Edition. IGI Global, 1350-1358.
  29. Ting-Hao Huang, Joseph Chee Chang, and Jeffrey P Bigham. 2018. Evorus: A crowd-powered conversational assistant built to automate itself over time. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. 1-13.
  30. Amanda Lee Hughes and Andrea H. Tapia. 2015. Social Media in Crisis: When Professional Responders Meet Digital Volunteers. Journal of Homeland Security and Emergency Management 12 (2015). Issue 3.
  31. Salil S Kanhere. 2013. Participatory sensing: Crowdsourcing data from mobile smartphones in urban spaces. In International Conference on Distributed Comput- ing and Internet Technology. Springer, 19-26.
  32. Marc-André Kaufhold, Nicola Rupp, Christian Reuter, and Matthias Habdank. 2020. Mitigating information overload in social media during conflicts and crises: design and evaluation of a cross-platform alerting system. Behaviour & Information Technology 39, 3 (2020), 319-342.
  33. Melanie Kellar, Carolyn Watters, and Michael Shepherd. 2007. A field study characterizing Web-based information-seeking tasks. Journal of the American Society for information science and technology 58, 7 (2007), 999-1018.
  34. Marina Kogan and Leysia Palen. 2018. Conversations in the eye of the storm: At-scale features of conversational structure in a high-tempo, high-stakes mi- croblogging environment. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. 1-13.
  35. Walter S Lasecki, Rachel Wesley, Jeffrey Nichols, Anand Kulkarni, James F Allen, and Jeffrey P Bigham. 2013. Chorus: a crowd-powered conversational assistant. In Proceedings of the 26th annual ACM symposium on User interface software and technology. 151-162.
  36. Yi-Chieh Lee, Naomi Yamashita, and Yun Huang. 2020. Designing a chatbot as a mediator for promoting deep self-disclosure to a real mental health professional. Proceedings of the ACM on Human-Computer Interaction 4, CSCW1 (2020), 1-27.
  37. James R Lewis. 1991. Psychometric evaluation of an after-scenario questionnaire for computer usability studies: the ASQ. ACM Sigchi Bulletin 23, 1 (1991), 78-81.
  38. Xukun Li, Doina Caragea, Cornelia Caragea, Muhammad Imran, and Ferda Ofli. 2019. Identifying disaster damage images using a domain adaptation approach. In Proceedings of the 16th International Conference on Information Systems for Crisis Response And Management.
  39. Yunyao Li, Tyrone Grandison, Patricia Silveyra, Ali Douraghy, Xinyu Guan, Thomas Kieselbach, Chengkai Li, and Haiqi Zhang. 2020. Jennifer for COVID-19: An NLP-Powered Chatbot Built for the People and by the People to Combat Misinformation. In Proceedings of the 1st Workshop on NLP for COVID-19 at ACL 2020.
  40. Q Vera Liao, Matthew Davis, Werner Geyer, Michael Muller, and N Sadat Shami. 2016. What can you do? Studying social-agent orientation and agent proactive interactions with an agent for employees. In Proceedings of the 2016 acm conference on designing interactive systems. 264-275.
  41. Zihan Liu, Genta Indra Winata, Zhaojiang Lin, Peng Xu, and Pascale Fung. 2020. Attention-Informed Mixed-Language Training for Zero-Shot Cross-Lingual Task- Oriented Dialogue Systems. In AAAI. AAAI Press, 8433-8440.
  42. Thomas Ludwig, Christoph Kotthaus, Christian Reuter, Sören Van Dongen, and Volkmar Pipek. 2017. Situated crowdsourcing during disasters: Managing the tasks of spontaneous volunteers through public displays. International Journal of Human-Computer Studies 102 (2017), 103-121.
  43. Wenhong Luo and Mohammad Najdawi. 2004. Trust-building measures: a review of consumer health portals. Commun. ACM 47, 1 (2004), 108-113.
  44. Aman Madaan, Amrith Setlur, Tanmay Parekh, Barnabas Poczos, Graham Neubig, Yiming Yang, Ruslan Salakhutdinov, Alan W Black, and Shrimai Prabhumoye. 2020. Politeness Transfer: A Tag and Generate Approach. In ACL.
  45. Alistair Martin, Jama Nateqi, Stefanie Gruarin, Nicolas Munsch, Isselmou Ab- darahmane, Marc Zobel, and Bernhard Knapp. 2020. An artificial intelligence- based first-line defence against COVID-19: digitally screening citizens for risks via a chatbot. Scientific reports 10, 1 (2020), 1-7.
  46. D Harrison McKnight, Vivek Choudhury, and Charles Kacmar. 2002. The impact of initial consumer trust on intentions to transact with a web site: a trust building model. The journal of strategic information systems 11, 3-4 (2002), 297-323.
  47. Lisa M Soederberg Miller and Robert A Bell. 2012. Online health information seeking: the influence of age, information trustworthiness, and search challenges. Journal of aging and health 24, 3 (2012), 525-541.
  48. Adam S Miner, Liliana Laranjo, and A Baki Kocaballi. 2020. Chatbots in the fight against the COVID-19 pandemic. npj Digital Medicine 3, 1 (2020), 1-4.
  49. Clifford Nass, Jonathan Steuer, and Ellen R Tauber. 1994. Computers are social actors. In Proceedings of the SIGCHI conference on Human factors in computing systems. ACM, 72-78.
  50. Ida Norheim-Hagtun and Patrick Meier. 2010. Crowdsourcing for crisis mapping in Haiti. Innovations: Technology, Governance, Globalization 5, 4 (2010), 81-89.
  51. Kyo-Joong Oh, Dongkun Lee, Byungsoo Ko, and Ho-Jin Choi. 2017. A chatbot for psychiatric counseling in mental healthcare service based on emotional dialogue analysis and sentence generation. In 2017 18th IEEE International Conference on Mobile Data Management (MDM). IEEE, 371-375.
  52. Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instructions with human feedback. arXiv preprint arXiv:2203.02155 (2022). · arXiv
  53. Leysia Palen, Kenneth M Anderson, Gloria Mark, James Martin, Douglas Sicker, Martha Palmer, and Dirk Grunwald. 2010. A vision for technology-mediated support for public participation & assistance in mass emergencies & disasters. ACM-BCS Visions of Computer Science 2010 (2010), 1-12.
  54. Leysia Palen, Sarah Vieweg, Jeannette Sutton, Sophia B Liu, and Amanda Hughes. 2007. Crisis informatics: Studying crisis in a networked world. In Proceedings of the Third International Conference on E-Social Science. 7-9.
  55. Gordon Pennycook, Jonathon McPhetres, Yunhao Zhang, Jackson G Lu, and David G Rand. 2020. Fighting COVID-19 misinformation on social media: Ex- perimental evidence for a scalable accuracy-nudge intervention. Psychological science 31, 7 (2020), 770-780.
  56. Marta Poblet, Esteban García-Cuesta, and Pompeu Casanovas. 2018. Crowdsourc- ing roles, methods and tools for data-intensive disaster management. Information Systems Frontiers 20, 6 (2018), 1363-1379.
  57. Filip Radlinski and Nick Craswell. 2017. A theoretical framework for conversa- tional search. In Proceedings of the 2017 conference on conference human informa- tion interaction and retrieval. 117-126.
  58. Daniela Retelny, Michael S Bernstein, and Melissa A Valentine. 2017. No workflow can ever be enough: How crowdsourcing workflows constrain complex work. Proceedings of the ACM on Human-Computer Interaction 1, CSCW (2017), 1-23.
  59. Daniela Retelny, Sébastien Robaszkiewicz, Alexandra To, Walter S Lasecki, Jay Patel, Negar Rahmati, Tulsee Doshi, Melissa Valentine, and Michael S Bernstein. 2014. Expert crowdsourcing with flash teams. In Proceedings of the 27th annual ACM symposium on User interface software and technology. 75-85.
  60. Bram Rochwerg, Rachael Parke, Srinivas Murthy, Shannon M Fernando, Jeanna Parsons Leigh, John Marshall, Neill KJ Adhikari, Kirsten Fiest, Rob Fowler, François Lamontagne, et al. 2020. Misinformation during the coronavirus disease 2019 outbreak: How knowledge emerges from noise. Critical Care Explorations 2, 4 (2020).
  61. Corbin Rosset, Chenyan Xiong, Xia Song, Daniel Campos, Nick Craswell, Saurabh Tiwary, and Paul Bennett. 2020. Leading Conversational Search by Suggesting Useful Questions. In Proceedings of The Web Conference 2020. 1160-1170.
  62. Alessandro Rovetta and Akshaya Srikanth Bhagavathula. 2020. COVID-19-related web search behaviors and infodemic attitudes in Italy: Infodemiological study. JMIR public health and surveillance 6, 2 (2020), e19374.
  63. Katherine E Rowan. 1991. When simple language fails: Presenting difficult science to the public. Journal of technical writing and communication 21, 4 (1991), 369-382.
  64. Fadi Safieddine, Wassim Masri, and Pardis Pourghomi. 2016. Corporate respon- sibility in combating online misinformation. International Journal of Advanced Computer Science and Applications (IJACSA) 7, 2 (2016), 126-132.
  65. Irina Shklovski, Moira Burke, Sara Kiesler, and Robert Kraut. 2010. Technology adoption and use in the aftermath of Hurricane Katrina in New Orleans. American Behavioral Scientist 53, 8 (2010), 1228-1246.
  66. Madhumita Shrotri, Tui Swinnen, Beate Kampmann, and Edward PK Parker. 2021. An interactive website tracking COVID-19 vaccine development. The Lancet Global Health 9, 5 (2021), e590-e592.
  67. Lee Sproull, Mani Subramani, Sara Kiesler, Janet H Walker, and Keith Waters. 1996. When the interface is a face. Human-computer interaction 11, 2 (1996), 97-124.
  68. Bobby Swar, Tahir Hameed, and Iris Reychav. 2017. Information overload, psy- chological ill-being, and behavioral intention to continue online healthcare in- formation search. Computers in Human Behavior 70 (2017), 416-425.
  69. Briony Swire-Thompson and David Lazer. 2020. Public health and online misin- formation: challenges and recommendations. Annual Review of Public Health 41 (2020), 433-451.
  70. Leila Tavakoli. 2020. Generating Clarifying Questions in Conversational Search Systems. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management. 3253-3256.
  71. Johanne R Trippas, Damiano Spina, Lawrence Cavedon, Hideo Joho, and Mark Sanderson. 2018. Informing the design of spoken conversational search: Perspec- tive paper. In Proceedings of the 2018 Conference on Human Information Interaction & Retrieval. 32-41.
  72. Jinke D Van Der Laan, Adriaan Heino, and Dick De Waard. 1997. A simple procedure for the assessment of acceptance of advanced transport telematics. Transportation Research Part C: Emerging Technologies 5, 1 (1997), 1-10.
  73. Alexandra Vtyurina, Denis Savenkov, Eugene Agichtein, and Charles LA Clarke. 2017. Exploring conversational search with humans, assistants, and wizards. In Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems. 2187-2193.
  74. Joanne I White and Leysia Palen. 2015. Expertise in the wired wild west. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing. 662-675.
  75. Ziang Xiao, Sarah Mennicken, Bernd Huber, Adam Shonkoff, and Jennifer Thom. 2021. Let Me Ask You This: How Can a Voice Assistant Elicit Explicit User Feedback? Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1-24.
  76. Ziang Xiao, Michelle X Zhou, Wenxi Chen, Huahai Yang, and Changyan Chi. 2020. If I Hear You Correctly: Building and Evaluating Interview Chatbots with Active Listening Skills. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1-14.
  77. Ziang Xiao, Michelle X Zhou, Q Vera Liao, Gloria Mark, Changyan Chi, Wenxi Chen, and Huahai Yang. 2020. Tell Me About Yourself: Using an AI-Powered Chatbot to Conduct Conversational Surveys with Open-ended Questions. ACM Transactions on Computer-Human Interaction (TOCHI) 27, 3 (2020), 1-37.
  78. Man-Ching Yuen, Irwin King, and Kwong-Sak Leung. 2011. A survey of crowd- sourcing systems. In 2011 IEEE third international conference on privacy, security, risk and trust and 2011 IEEE third international conference on social computing. IEEE, 766-773.
  79. Fengbin Zhu, Wenqiang Lei, Chao Wang, Jianming Zheng, Soujanya Poria, and Tat-Seng Chua. 2021. Retrieving and reading: A comprehensive survey on open- domain question answering. arXiv preprint arXiv:2101.00774 (2021). · arXiv

Cited by

Sorted by citations

Authors
Xiao, Z., Liao, Q. V., Zhou, M., Grandison, T., & Li, Y.
Affiliations
  • Johns Hopkins University (US)
  • Microsoft (Canada) (CA)
  • Juji, Inc., United States
  • The Data-Driven Institute, United States
  • Apple (United States) (US)
ORCID
Venue
Proceedings of the 28th International Conference on Intelligent User Interfaces
Publisher
ACM
Published
2023-03-27
Pages
2-18
DOI
10.1145/3581641.3584031
OpenAlex
W4360991106
Cited by
59
References
7
Citations by year
2026: 10 · 2025: 23 · 2024: 19 · 2023: 7
FWCI
15.17
Type
conference paper
Language
en
Open access
Licence
Locations
  • ACM International Conference on Intelligent User Interfaces (IUI) (publishedVersion, cc-by-nc)
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • arXiv (Cornell University) (cc-by)
Topics
  • Physical Sciences › Computer Science › Information Systems › Expert finding and Q&A systems
  • Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
  • Social Sciences › Communication › Communication and COVID-19 Impact
Keywords
AI chatbots, health information credibility, misinformation mitigation, information seeking, trust in AI, public health crisis communication

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review