Exploring healthcare professionals’ perceptions of artificial intelligence: Validating a questionnaire using the e-Delphi method
Open accessThe DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index.
Abstract
Objective: The aim of this study was to draw upon the collective knowledge of experts in the fields of health and technology to develop a questionnaire that measured healthcare professionals’ perceptions of Artificial Intelligence (AI). Methods: The panel for this study were carefully selected participants who demonstrated an interest and/or involvement in AI from the fields of health or information technology. Recruitment was accomplished via email which invited the panel member to participate and included study and consent information. Data were collected from three rounds in the form of an online survey, an online group meeting and email communication. A 75% median threshold was used to define consensus. Results: Between January and March 2019, five healthcare professionals and three IT experts participated in three rounds of study to reach consensus on the structure and content of the questionnaire. In Round 1 panel members identified issues about general understanding of AI and achieved consensus on nine draft questionnaire items. In Round 2 the panel achieved consensus on demographic questions and comprehensive group discussion resulted in the development of two further questionnaire items for inclusion. In a final e-Delphi round, a draft of the final questionnaire was distributed via email to the panel members for comment. No further amendments were put forward and 100% consensus was achieved. Conclusion: A modified e-Delphi method was used to validate and develop a questionnaire to explore healthcare professionals’ perceptions of AI. The e-Delphi method was successful in achieving consensus from an interdisciplinary panel of experts from health and IT. Further research is recommended to test the reliability of this questionnaire.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Perceived usefulness, perceived ease of use, and user acceptance of information technology
Davis, F. D. (1989) · MIS Quarterly · doi:10.2307/249008 · Cited by 68,483
The potential for artificial intelligence in healthcare
Davenport, T., & Kalakota, R. (2019) · Future Healthcare Journal · doi:10.7861/futurehosp.6-2-94 · Cited by 3,800
Diamond, I. R., Grant, R. C., Feldman, B. M., Pencharz, P. B., Ling, S. C., Moore, A. M., & Wales, P. W. (2014) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2013.12.002 · Cited by 3,740
Artificial intelligence in medicine
Hamet, P., & Tremblay, J. (2017) · Metabolism · doi:10.1016/j.metabol.2017.01.011 · Cited by 2,334
Boulkedid, R., Abdoul, H., Loustau, M., Sibony, O., & Alberti, C. (2011) · PLoS ONE · doi:10.1371/journal.pone.0020476 · Cited by 2,045 · cited 1×
“Traditionally, the Delphi method uses a paper-based questionnaire to collect data from participants, however digital methods, called e-Delphi methods, are now being used.(50,51) The use of electronic and internet-based questionnaires allows for a faster response time, points of anonymity and a reduction of resource co…”
MethodConsensus measurement in Delphi studies
von der Gracht, H. A. (2012) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2012.04.013 · Cited by 1,831
Chan, K. S., & Zary, N. (2019) · JMIR Medical Education · doi:10.2196/13930 · Cited by 877 · cited 1×
“This could be really interesting” 65–67 These studies acknowledge that understanding of AI is limited amongst medical students and that deciding on the content of education is challenging, but training will be necessary to realise the full capacity of this technology.”
BackgroundNadarzynski, T., Miles, O., Cowie, A., & Ridge, D. (2019) · DIGITAL HEALTH · doi:10.1177/2055207619871808 · Cited by 810
Patient care information systems and health care work: a sociotechnical approach
Berg, M. (1999) · International Journal of Medical Informatics · doi:10.1016/s1386-5056(99)00011-8 · Cited by 786
Medical students' attitude towards artificial intelligence: a multicentre survey
Pinto dos Santos, D., Giese, D., Brodehl, S., Chon, S. H., Staab, W., Kleinert, R., Maintz, D., & Baeßler, B. (2019) · European Radiology · doi:10.1007/s00330-018-5601-1 · Cited by 754 · cited 1×
“This could be really interesting” 65–67 These studies acknowledge that understanding of AI is limited amongst medical students and that deciding on the content of education is challenging, but training will be necessary to realise the full capacity of this technology.”
Background
Cited by
Ahmed, Z., Bhinder, K. K., Tariq, A., Tahir, M. J., Mehmood, Q., Tabassum, M. S., Malik, M., Aslam, S., Asghar, M. S., & Yousaf, Z. (2022) · Annals of Medicine & Surgery · doi:10.1016/j.amsu.2022.103493 · Cited by 208
“Machine learning and deep learning are two subsets of AI that are being explored globally in the health sector [6].”
BackgroundPotential of large language models in health care: Delphi study
Denecke, K., May, R., LLMHealthGroup, & Rivera Romero, O. (2024) · Journal of Medical Internet Research · doi:10.2196/52399 · Cited by 111
Are physicians and medical students ready for artificial intelligence applications in healthcare?
AlZaabi, A., AlMaskari, S., & AalAbdulsalam, A. (2023) · DIGITAL HEALTH · doi:10.1177/20552076231152167 · Cited by 93
Catalina, Q. M., Fuster-Casanovas, A., Vidal-Alaball, J., Escalé-Besa, A., Marin-Gomez, F. X., Femenia, J., & Solé-Casals, J. (2023) · DIGITAL HEALTH · doi:10.1177/20552076231180511 · Cited by 71
Application of artificial intelligence in medical technologies: a systematic review of main trends.
Bitkina, O. V., Park, J., & Kim, H. K. (2023) · Digital Health · doi:10.1177/20552076231189331 · Cited by 66
Metadata
- Authors
- Shinners, L., Aggar, C., Grace, S., & Smith, S.
- Venue
- DIGITAL HEALTH
- Publisher
- SAGE Publications
- Published
- 2021-01
- Volume
- 7
- Pages
- 20552076211003433-20552076211003433
- ISSN
- 2055-2076 (ISSN-L)
- OpenAlex
- W3136425357
- Cited by
- 62
- References
- 21
- FWCI
- 2.15
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Exploring healthcare professionals’ perceptions of artificial intelligence: Validating a questionnaire using the e-Delphi method. https://app.cochat.ai/papers/10.1177/20552076211003433
- The paper
- Shinners, L., Aggar, C., Grace, S., & Smith, S. (2021). Exploring healthcare professionals’ perceptions of artificial intelligence: Validating a questionnaire using the e-Delphi method. DIGITAL HEALTH. https://doi.org/10.1177/20552076211003433
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.