User reactions to COVID-19 screening chatbots from reputable providers
Open accessAbstract
Objectives: The objective was to understand how people respond to coronavirus disease 2019 (COVID-19) screening chatbots. Materials and Methods: We conducted an online experiment with 371 participants who viewed a COVID-19 screening session between a hotline agent (chatbot or human) and a user with mild or severe symptoms. Results: The primary factor driving user response to screening hotlines (human or chatbot) is perceptions of the agent’s ability. When ability is the same, users view chatbots no differently or more positively than human agents. The primary factor driving perceptions of ability is the user’s trust in the hotline provider, with a slight negative bias against chatbots’ ability. Asian individuals perceived higher ability and benevolence than did White individuals. Conclusions: Ensuring that COVID-19 screening chatbots provide high-quality service is critical but not sufficient for widespread adoption. The key is to emphasize the chatbot’s ability and assure users that it delivers the same quality as human agents.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
COVID-19 screening is an ideal application for chatbots because it is a well-structured process involving clearly defined questions and a risk score.
“COVID-19 screening is considered an ideal application for chatbots because it is a well-structured process that involves asking patients a series of clearly defined questions and determining a risk score (Dennis et al. 2020).”
From Chatbot breakthrough in the 2020s? An ethical reflection on the trend of automated consultations in health care (2022) · cites this paper as Dennis et al. 2020
Relieving pressure on contact centres by chatbots is especially important in the present COVID-19 situation, thus making chatbots cost-effective.
“This relieving of pressure on contact centres is especially important in the present COVID-19 situation (Dennis et al. 2020, p. 1727), thus making chatbots cost-effective.”
From Chatbot breakthrough in the 2020s? An ethical reflection on the trend of automated consultations in health care (2022) · cites this paper as Dennis et al. 2020
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Statistical Power Analysis for the Behavioral Sciences
Muller, K. E., & Cohen, J. (1989) · Technometrics · doi:10.2307/1270020 · Cited by 99,588 · see citing works
Statistical power analysis for the behavioral sciences
Cohen, J. (1990) · Computers Environment and Urban Systems · doi:10.1016/0198-9715(90)90050-4 · Cited by 65,406 · see citing works
An integrative model of organizational trust
Mayer, R. C., Davis, J. H., & Schoorman, F. D. (1995) · The Academy of Management Review · doi:10.2307/258792 · Cited by 10,736 · see citing works
Trust and TAM in Online Shopping: An Integrated Model1
Gefen, D., Karahanna, E., & Straub, D. W. (2003) · MIS Quarterly · doi:10.2307/30036519 · Cited by 8,635 · see citing works
Aguinis, H., & Bradley, K. J. (2014) · Organizational Research Methods · doi:10.1177/1094428114547952 · Cited by 2,047 · see citing works
Luo, X., Tong, S., Fang, Z., & Qu, Z. (2019) · Marketing Science · doi:10.1287/mksc.2019.1192 · Cited by 1,027 · cited 1× · see citing works
“Therefore, to offset users’ biases [33], a necessary component in deploying chatbots for COVID-19 screening is a strong messaging campaign that emphasizes the chatbot’s ability.”
BackgroundTrust In and Adoption of Online Recommendation Agents
Benbasat, I., & Wang, W. (2005) · Journal of the Association for Information Systems · doi:10.17705/1jais.00065 · Cited by 847 · see citing works
Nadarzynski, T., Miles, O., Cowie, A., & Ridge, D. (2019) · DIGITAL HEALTH · doi:10.1177/2055207619871808 · Cited by 810 · see citing works
Towards a Human-like Open-Domain Chatbot
Adiwardana, D., Luong, M.-T., So, D. R., Hall, J., Fiedel, N., Thoppilan, R., Yang, Z., Kulshreshtha, A., Nemade, G., Lu, Y., et al. (2020) · arXiv · doi:10.48550/arxiv.2001.09977 · Cited by 614 · see citing works
Technology, Humanness, and Trust: Rethinking Trust in Technology
Marshall University, Lankton, N., McKnight, D. H., Michigan State University, Tripp, J., & Baylor University (2015) · Journal of the Association for Information Systems · doi:10.17705/1jais.00411 · Cited by 553 · cited 1× · see citing works
“Trust in humans is influenced by three primary factors [19] that also have parallels for trust in technology [20].”
BackgroundTo justify or excuse?: A meta-analytic review of the effects of explanations.
Shaw, J. C., Wild, E., & Colquitt, J. A. (2003) · Journal of Applied Psychology · doi:10.1037/0021-9010.88.3.444 · Cited by 446 · see citing works
Data Collection in the Digital Age: Innovative Alternatives to Student Samples1
Steelman, Z. R., Hammer, B. I., & Limayem, M. (2014) · MIS Quarterly · doi:10.25300/misq/2014/38.2.02 · Cited by 406 · see citing works
Physician-Patient Relationship and Medication Compliance: A Primary Care Investigation
Kerse, N. M. (2004) · The Annals of Family Medicine · doi:10.1370/afm.139 · Cited by 391 · see citing works
Cram, W. A., D’Arcy, J., & Proudfoot, J. G. (2019) · MIS Quarterly · doi:10.25300/misq/2019/15117 · Cited by 337 · see citing works
Chatbots in the fight against the COVID-19 pandemic
Miner, A. S., Laranjo, L., & Kocaballi, A. B. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-0280-0 · Cited by 322 · see citing works
Wang, W., Qiu, L., Kim, D., & Benbasat, I. (2016) · Decision Support Systems · doi:10.1016/j.dss.2016.03.007 · Cited by 208 · see citing works
HOW THE PACKAGING OF DECISION EXPLANATIONS AFFECTS PERCEPTIONS OF TRUSTWORTHINESS.
Elsbach, K. D., & Elofson, G. (2000) · Academy of Management Journal · doi:10.2307/1556387 · Cited by 145 · see citing works
Data Breaches of Protected Health Information in the United States
Liu, V., Musen, M. A., & Chou, T. (2015) · JAMA · doi:10.1001/jama.2015.2252 · Cited by 144 · see citing works
Lu, X., Zhang, R., Wu, W., Shang, X., & Liu, M. (2018) · Journal of Medical Internet Research · doi:10.2196/jmir.9364 · Cited by 138 · see citing works
Xu, D. J., Cenfetelli, R. T., & Aquino, K. F. (2015) · The Journal of Strategic Information Systems · doi:10.1016/j.jsis.2015.10.004 · Cited by 118 · see citing works
Implementation of a digital chatbot to screen health system employees during the COVID-19 pandemic
Judson, T. J., Odisho, A. Y., Young, J. J., Bigazzi, O., Steuer, D., Gonzales, R., & Neinstein, A. B. (2020) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaa130 · Cited by 106 · see citing works
Quro: Facilitating User Symptom Check Using a Personalised Chatbot-Oriented Dialogue System
Ghosh, S., Bhatia, S., & Bhatia, A. (2018) · Studies in health technology and informatics · doi:10.3233/978-1-61499-890-7-51 · Cited by 60 · see citing works
Shawar, B. A., & Atwell, E. (2016) · International Journal of Speech Technology · doi:10.1007/s10772-015-9330-4 · Cited by 31 · see citing works
Flagged first, then by citations
Research works printed but not linked: 5
- Mauldin M. ChatterBots, tinymuds, and the turing test: entering the Loeb- ner prize competition. In: proceedings of the Eleventh National Confer- ence on Artificial Intelligence; 1994: 16-21.
- Mittal A, Agrawal A, Chouksey A, Shriwas R, Agrawal S. A comparative study of chatbots and humans. Int J Adv Res Comput Commun Eng 2016; 5 (3): 1055-7.
- Følstad A, Skjuve M. Chatbots for customer service: user experience and motivation. In: proceedings of the 1st International Conference on Con- versational User Interfaces New York; 2019: 1-9.
- Xu A, Liu Z, Guo Y, Sinha V, Akkiraju R. A new chatbot for customer ser- vice on social media. In: proceedings of the 2017 CHI Conference on Hu- man Factors in Computing Systems (CHI'17); 2017: 3506-10.
- Reis L, Maier C, Mattke J, Weitzel T. Chatbots in healthcare: status quo, application scenarios for physicians and patients, and future directions. In: proceedings in of the 28th European Conference on Information Sys- tems; 2020.
Cited by
Sorted by citations
Parviainen, J., & Rantala, J. (2022) · Medicine, Health Care and Philosophy · doi:10.1007/s11019-021-10049-w · Cited by 146 · see citing works
- Authors
- Dennis, A. R., Kim, A., Raimi, R., & Ayabakan, S.
- Affiliations
- Indiana University Bloomington (US)
- Temple University (US)
- Corresponding author
- Dennis, Alan R
- Venue
- Journal of the American Medical Informatics Association
- Publisher
- Oxford University Press (OUP)
- Published
- 2020-07-06
- Volume
- 27
- Issue
- 11
- Pages
- 1727-1731
- ISSN
- 1067-5027 (ISSN-L)
- 1527-974X
- PMC
- PMC7454579
- OpenAlex
- W3040564084
- Cited by
- 59
- References
- 23
- Citations by year
- 2026: 1 · 2025: 6 · 2024: 6 · 2023: 14 · 2022: 18 · 2021: 13 · 2020: 1
- FWCI
- 4.52
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- academic.oup.com/journals/pages/open_access/funder_policies/chorus/standard_publication_model (vor)
- Locations
- Journal of the American Medical Informatics Association (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- TUScholarShare (Temple University) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › AI in Service Interactions
- Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Keywords
- COVID-19 screening, chatbots, user perception, perceived competence, trust in providers, racial and ethnic differences, benevolence, chatbot adoption
- MeSH terms
- COVID-19, COVID-19 Testing, Communication, Female, Humans, Male, Medical Informatics Applications, Pneumonia, Viral, Telemedicine, Coronavirus Infections, Clinical Laboratory Techniques, Trust, Pandemics, Text Messaging
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
