Responsible AI for Digital Health: a Synthesis and a Research Agenda
Open accessThe DOI matches its Crossref record; 86 of the 86 references it cites were found in a registry or an index; 6 of the 71 sources its claims cite were checked for support.
Abstract
Responsible AI is concerned with the design, implementation and use of ethical, transparent, and accountable AI technology in order to reduce biases, promote fairness, equality, and to help facilitate interpretability and explainability of outcomes, which are particularly pertinent in a healthcare context. However, the extant literature on health AI reveals significant issues regarding each of the areas of responsible AI, posing moral and ethical consequences. This is particularly concerning in a health context where lives are at stake and where there are significant sensitivities that are not as pertinent in other domains outside of health. This calls for a comprehensive analysis of health AI using responsible AI concepts as a structural lens. A systematic literature review supported our data collection and sampling procedure, the corresponding analysis, and extraction of research themes helped us provide an evidence-based foundation. We contribute with a systematic description and explanation of the intellectual structure of Responsible AI in digital health and develop an agenda for future research.
Overview
CoChat Summary
TL;DR
This systematic review synthesizes 34 papers (2009–2020) on responsible AI in digital health using Mittelstadt et al.’s six ethical concerns, mapping research themes via co-word and network analysis. It finds big data/predictive modelling as the mainstream theme, ethics/healthcare as a basic transversal theme, and proposes a research agenda around inconclusive, inscrutable, misguided evidence, unfair outcomes, transformative effects, and traceability.
Key findings
- From 83 initially retrieved papers, 34 were selected; 30 papers had author-assigned keywords (133 unique keywords, M = 4.43 per paper), and machine extraction produced 158 unique key-phrases (M = 4.65 per paper), with 47 keywords and 57 key-phrases retained for analysis.
- Co-word analysis identified seven clusters in both author-assigned and machine-extracted datasets; the mainstream Q1 theme covered big data, big data analytics, and predictive modelling, while ethics/healthcare appeared as a basic and transversal Q4 theme.
- Peripheral Q2 themes included ethical AI, primary ethical risks, ethical design, and IoT/medical devices; author-keyword Q3 emerging/declining themes included information ethics, trust, security, privacy, and health information exchange.
- Core-periphery analysis found ethics, artificial intelligence, privacy, big data, and healthcare among the most popular/core/backbone topics; big data analytics was frequent but not core or backbone.
Methods
- Qualitative systematic literature review with a protocol covering research questions, sources, search terms, search strategy, and inclusion/exclusion criteria; searches covered AIS basket-of-eight journals, leading management journals, EBSCOhost, Web of Science, Scopus, ACM Digital Library, and INFORMS, for articles published until October 2020.
- Meta-data analysis used correspondence analysis, co-word analysis, network analysis, and core-periphery analysis on author-assigned keywords and machine-extracted key-phrases; TextRank was used for abstract key-phrase extraction with a sliding window of 3, nouns/adjectives/proper nouns, and top-10 phrases.
- Synthesis was structured using Mittelstadt et al.’s (2016) six types of ethical concerns: inconclusive evidence, inscrutable evidence, misguided evidence, unfair outcomes, transformative effects, and traceability.
Limitations
- Stated by the paper: the focus is limited to healthcare, the analysis is limited by using Mittelstadt et al.’s framework, and other technologies such as Electronic Healthcare Records were not included.
- Evident from the text: the search ended in October 2020, included only English-language peer-reviewed journals and complete conference papers, and excluded research in progress, abstracts, workshop proposals, book chapters, demos, and blogs.
- Evident from the text: the final corpus is small (34 papers), and the co-word analysis retained only terms co-appearing more than 2 times, which may narrow the themes captured.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
5 Valid · 4 Not valid · 58 Not checked · 67 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“The application of AI in healthcare therefore raises significant concerns of fairness, responsibility, human rights (Floridi & Taddeo, 2016)”
§ Responsible AI for Digital Health: a Synthesis and a Research Agenda > 1 Introduction
What the sources say
- Partly supportedcites Floridi & Taddeo, 2016 sourceExperimental — accuracy not yet measured
The Introduction (page not specified) says that growing reliance on algorithms, including AI, raises issues of fairness, responsibility and respect for human rights. It discusses biomedical data and an NHS programme separately, but does not specifically make this three-part claim about AI applications in healthcare. (LLM, conf=0.84)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Recent empirical articles highlight how deploying AI is coupled with significant ethical challenges (Floridi & Taddeo, 2016)”
§ Responsible AI for Digital Health: a Synthesis and a Research Agenda > 1 Introduction
What the sources say
- Supportedcites Floridi & Taddeo, 2016 source
The Introduction (p. 1) states verbatim that the opportunities of data science 'are also coupled to significant ethical challenges', explicitly listing 'the growing reliance on algorithms ... (including machine learning, artificial intelligence and robotics)' and reduced human oversight as posing 'pressing issues of fairness, responsibility and respect of human rights'. The same section's passage on the ethics of algorithms ties AI and artificial agents to 'crucial challenges' of moral responsibility, accountability and unforeseen/undesired consequences, and the Abstract reinforces 'the complexity of the ethical challenges posed by data science'. Only caveat: the source is a conceptual framing paper rather than an empirical study, and it treats AI within the wider scope of data science, but the attributed content itself is stated. (LLM, conf=0.85)
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
None of the sources cited for this claim have been checked yet.
In the paper
“On other occasions a false negative occurred when AI incorrectly fails to indicate the presence of a disease when in reality it is present.”
§ Responsible AI for Digital Health: a Synthesis and a Research Agenda > 4.2 Inconclusive Evidence
What the sources say
- Could not verifycites Burr et al., 2020 sourceExperimental — accuracy not yet measured
The Health and Healthcare section discusses possible false positives in ML detection of at-risk patients, but does not discuss false negatives or AI failing to identify a disease that is present. Searches for false negatives and missed diagnoses found no passage addressing that claim. (LLM, conf=0.72)
- Not checked yetcites Maher et al., 2019 source
plainstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Could not verify
Every source the paper cites for this claim was checked and supports it.
In the paper
“In these cases, AI is subject to significant uncertainty, which called for precautionary and safety principles in the data collection and analysis for decision making (Floridi et al., 2020).”
§ Responsible AI for Digital Health: a Synthesis and a Research Agenda > 4.2 Inconclusive Evidence
What the sources say
- Supportedcites Floridi et al., 2020 source
In “Falsifiability and Incremental Deployment” (page unavailable), the paper says complete certainty about an AI-for-social-good application’s safety is out of reach in an uncertain world. This supports the attributed statement that AI in such cases is subject to significant uncertainty. (LLM, conf=0.88)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Although AI can augment or surpass human abilities by identifying, interpreting, making inferences, and learning from data to achieve predetermined organizational and societal goals (Mikalef & Gupta, 2021), it provides suggestions that are by nature uncertain as it identifies correlation relationships (Floridi et al., 2020).”
§ Responsible AI for Digital Health: a Synthesis and a Research Agenda > 5.1 Inconclusive Evidence
What the sources say
- Partly supportedcites Floridi et al., 2020 sourceExperimental — accuracy not yet measured
In “Safeguards Against the Manipulation of Predictors” (page unavailable), the paper says AI prediction can rely on indicators correlated with, but not causal of, an outcome. It describes this as a risk for AI4SG interventions; it does not say AI suggestions are inherently uncertain because they identify correlations. (LLM, conf=0.85)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
46 more claims
Create a free account to see all 134 claims and to verify the 65 references still open.
- Probably not a substantial claim. 52
- It may state two separate claims as one. 46
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“For example, irresponsible AI may lead to a loss of medical staff and patients [18].”
From Survey of Explainable AI Techniques in Healthcare (2023) · cites this paper as [18]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Responsible AI for Digital Health: a Synthesis and a Research Agenda > 4.1.3 Thematic Analysis along the Four Quadrants of Strategic Diagram
“According to author assigned keywords, big data, responsible development, ethical impact resulted to be motor theme about ethical issues stemming from AI (custer C4).”
§ Responsible AI for Digital Health: a Synthesis and a Research Agenda > Network and Core-Periphery Analysis
“C4 Q1 2 8 55 0,622 2000 bigdata, biomedicine”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Responsible AI for Digital Health: a Synthesis and a Research Agenda > 4.1.1 Insights from the Correspondence Analysis Factor Map
“Density: how cohesive is the cluster of terms, i.e., the number of direct ties observed for the cluster divided by the maximum number of possible ones (M.Callon et al., 1991).”
§ Responsible AI for Digital Health: a Synthesis and a Research Agenda > 4.1.1 Insights from the Correspondence Analysis Factor Map
“The value range can be any positive number, and can be greater than 1, as density is not "interpreted" as a proportion, but rather as the average number of observed lines (Knoke & Yang, 2019, p. 107).”
Citations
Cites
Arrieta, A. B., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., García, S., Gil-López, S., Molina, D., Benjamins, R., et al. (2019) · arXiv · doi:10.48550/arxiv.1910.10045 · Cited by 11,321
Barredo Arrieta, A., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., Garcia, S., Gil-Lopez, S., Molina, D., Benjamins, R., et al. (2020) · Information Fusion · doi:10.1016/j.inffus.2019.12.012 · Cited by 10,423
Structural Holes and Good Ideas
Burt, R. S. (2004) · American Journal of Sociology · doi:10.1086/421787 · Cited by 5,642
The ethics of algorithms: Mapping the debate
Mittelstadt, B. D., Allo, P., Taddeo, M., Wachter, S., & Floridi, L. (2016) · Big Data & Society · doi:10.1177/2053951716679679 · Cited by 4,567
TextRank: Bringing Order into Text
Mihalcea, R. F., & Tarau, P. (2004) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · Cited by 3,631
Science mapping software tools: Review, analysis, and cooperative study among tools
Cobo, M. J., López-Herrera, A. G., Herrera-Viedma, E., & Herrera, F. (2011) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.21525 · Cited by 3,242
Callon, M., Courtial, J. P., & Laville, F. (1991) · Scientometrics · doi:10.1007/bf02019280 · Cited by 2,090
Synthesizing information systems knowledge: A typology of literature reviews
Paré, G., Trudel, M.-C., Jaana, M., & Kitsiou, S. (2015) · Information & Management · doi:10.1016/j.im.2014.08.008 · Cited by 1,987 · cited 1×
“In the first step, we specified the objective of our review and the road map towards achieving this objective (Templier & Paré, 2015).”
MethodFrom translations to problematic networks: An introduction to co-word analysis
Callon, M., Courtial, J.-P., Turner, W. A., & Bauin, S. (1983) · Social Science Information · doi:10.1177/053901883022002003 · Cited by 1,951
Principles alone cannot guarantee ethical AI
Mittelstadt, B. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0114-4 · Cited by 1,886 · cited 2×
“Most of the highlevel interventions into the ethics of AI discussion are principle-based (Floridi & Cowls, 2019) but principles alone cannot guarantee ethical AI (Mittelstadt, 2019).”
BackgroundMikalef, P., & Gupta, M. (2021) · Information & Management · doi:10.1016/j.im.2021.103434 · Cited by 1,700
Self-citationA Unified Framework of Five Principles for AI in Society
Floridi, L., & Cowls, J. (2019) · Harvard Data Science Review · doi:10.1162/99608f92.8cd550d1 · Cited by 1,631 · cited 1×
“Most of the high-level interventions into the ethics of AI discussion are principle-based (Floridi & Cowls, 2019) but principles alone cannot guarantee ethical AI (Mittelstadt, 2019).”
BackgroundCommunications of the Association for Information Systems
(2017) · Communications of the Association for Information Systems · doi:10.17705/1cais · Cited by 1,603
Explainable AI: From black box to glass box
Rai, A. (2020) · Journal of the Academy of Marketing Science · doi:10.1007/s11747-019-00710-5 · Cited by 1,317 · cited 1×
“Inscrutable evidence raise concerns about explainability, and transparency of the procedures AI followed to generate results (Barredo Arrieta et al., 2020; Rai, 2020).”
BackgroundResponsible Artificial Intelligence: How to Develop and Use AI in a Responsible Way
Dignum, V. (2019) · Artificial Intelligence: Foundations, Theory, and Algorithms · doi:10.1007/978-3-030-30371-6 · Cited by 1,197
Correspondence Analysis in Practice
Greenacre, M. (2017) · Chapman & Hall/CRC eBooks · doi:10.1201/9781315369983 · Cited by 1,108
The ethics of AI in health care: A mapping review
Morley, J., Machado, C. C. V., Burr, C., Cowls, J., Joshi, I., Taddeo, M., & Floridi, L. (2020) · Social Science & Medicine · doi:10.1016/j.socscimed.2020.113172 · Cited by 1,105
It's About Time: Temporal Structuring in Organizations
Orlikowski, W. J., & Yates, J. (2002) · Organization Science · doi:10.1287/orsc.13.6.684.501 · Cited by 1,038 · cited 1×
“Others have a temporal personality that exhibits this feeling of slow passage of time even when there is no logical reason (Ancona et al., 2001; Mosakowski & Earley, 2000; Orlikowski & Yates, 2002).”
BackgroundThe ethics of artificial intelligence
Bostrom, N., & Yudkowsky, E. (2014) · The Cambridge Handbook of Artificial Intelligence · doi:10.1017/cbo9781139046855.020 · Cited by 918
Ancona, D. G., Goodman, P. S., Lawrence, B. S., & Tushman, M. L. (2001) · The Academy of Management Review · doi:10.2307/3560246 · Cited by 870
Ethical Implications and Accountability of Algorithms
Martin, K. (2019) · Journal of Business Ethics · doi:10.1007/s10551-018-3921-3 · Cited by 757 · cited 11×
“Martin (2019a) suggests to share the responsibility among all actors involved in data sharing and analysis including the engineers, who developed those AI tools because autonomous robots and other AI technologies behave according to ethical standards and principles inscribed in them by the…”
BackgroundInfluentialArtificial Intelligence and the ‘Good Society’: the US, EU, and UK approach
Cath, C., Wachter, S., Mittelstadt, B., Taddeo, M., & Floridi, L. (2017) · Science and Engineering Ethics · doi:10.1007/s11948-017-9901-7 · Cited by 733 · cited 2×
“In response to this, many studies suggested that transparent and controlled approaches for data collection, analysis, and interpretation would solve the black-boxed medicine (Astromsk ė et al., 2020; Cath et al., 2017; Floridi et al., 2020).”
BackgroundThe ethics of big data: current and foreseeable issues in biomedical contexts
Mittelstadt, B. D., & Floridi, L. (2016) · Science and Engineering Ethics · doi:10.1007/s11948-015-9652-2 · Cited by 714
Floridi, L., & Taddeo, M. (2016) · Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences · doi:10.1098/rsta.2016.0360 · Cited by 637
Knowledge Discovery through Co-Word Analysis.
Qin, H. (1999) · Illinois Digital Environment for Access to Learning and Scholarship (University of Illinois at Urbana-Champaign) · Cited by 636
Artificial Intelligence Transforms the Future of Health Care
Noorbakhsh-Sabet, N., Zand, R., Zhang, Y., & Abedi, V. (2019) · The American Journal of Medicine · doi:10.1016/j.amjmed.2019.01.017 · Cited by 626
A Framework for Guiding and Evaluating Literature Reviews
Templier, M., & Paré, G. (2015) · Communications of the Association for Information Systems · doi:10.17705/1cais.03706 · Cited by 553 · cited 1×
“In the first step, we specified the objective of our review and the road map towards achieving this objective (Templier & Paré, 2015).”
MethodHawaii international conference on system sciences
Schultz, E. E. (1992) · ACM SIGCHI Bulletin · doi:10.1145/134421.134436 · Cited by 517
Newell, S., & Marabelli, M. (2015) · The Journal of Strategic Information Systems · doi:10.1016/j.jsis.2015.02.001 · Cited by 494
Ethics in artificial intelligence: introduction to the special issue
Dignum, V. (2018) · Ethics and Information Technology · doi:10.1007/s10676-018-9450-z · Cited by 459 · cited 1×
“Accountability means understanding the rationale behind the processes followed during decision making (Dignum, 2018; Smith, 2020).”
BackgroundInformation ethics: On the philosophical foundation of computer ethics
Floridi, L. (1999) · Ethics and Information Technology · doi:10.1023/a:1010018611096 · Cited by 430 · cited 1×
“In addition to these well-established ethical theories, there are more recent approaches specific to technological applications such as information ethics (Floridi, 1999), data ethics (Floridi & Taddeo, 2016), big data ethics (Mittelstadt & Floridi, 2016).”
BackgroundHow to Design AI for Social Good: Seven Essential Factors
Floridi, L., Cowls, J., King, T. C., & Taddeo, M. (2020) · Science and Engineering Ethics · doi:10.1007/s11948-020-00213-5 · Cited by 430
The ethics of information transparency
Turilli, M., & Floridi, L. (2009) · Ethics and Information Technology · doi:10.1007/s10676-009-9187-9 · Cited by 419
The Ethics of Digital Well-Being: A Thematic Review
Burr, C., Taddeo, M., & Floridi, L. (2020) · Science and Engineering Ethics · doi:10.1007/s11948-020-00175-8 · Cited by 374
Cuccurullo, C., Aria, M., & Sarto, F. (2016) · Scientometrics · doi:10.1007/s11192-016-1948-8 · Cited by 365
Leydesdorff, L., & Vaughan, L. (2006) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.20335 · Cited by 361
Breaking the vicious cycle of algorithmic management: A virtue ethics approach to people analytics
Gal, U., Jensen, T. B., & Stein, M.-K. (2020) · Information and Organization · doi:10.1016/j.infoandorg.2020.100301 · Cited by 342
The Ethics of Artificial Intelligence
Bostrom, N., & Yudkowsky, E. (2018) · Artificial Intelligence Safety and Security · doi:10.1201/9781351251389-4 · Cited by 333
The legal and ethical concerns that arise from using complex predictive analytics in health care
Cohen, I. G., Amarasingham, R., Shah, A., Xie, B., & Lo, B. (2014) · Health Affairs · doi:10.1377/hlthaff.2014.0048 · Cited by 300
A Selective Review of Time Assumptions in Strategy Research
Mosakowski, E., & Earley, P. C. (2000) · The Academy of Management Review · doi:10.2307/259207 · Cited by 279
Artificial Intelligence and Patient-Centered Decision-Making
Bjerring, J. C., & Busch, J. (2021) · Philosophy & Technology · doi:10.1007/s13347-019-00391-6 · Cited by 274 · cited 4×
“…but this is not sufficient to posit the existence of a causal connection, which can lead to unjustified actions (Astromsk ė et al., 2020; Bjerring & Busch, 2020; Crnkovic Dodig & Çürüklü, 2012; Floridi et al., 2020; Garattini et al., 2019; Henriksen & Bechmann, 2020; Morley et al.,…”
BackgroundInfluentialClinical AI: opacity, accountability, responsibility and liability
Smith, H. (2021) · AI & SOCIETY · doi:10.1007/s00146-020-01019-6 · Cited by 245 · cited 5×
“The decision making process is delegated not only to healthcare professional but also to the AI technology used to support these professionals during the decision making process (Guan, 2019; Kaplan, 2016; Smith, 2020).”
BackgroundMethodInfluentialEthics of healthcare robotics: Towards responsible research and innovation
Stahl, B. C., & Coeckelbergh, M. (2016) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2016.08.018 · Cited by 235
Artificial intelligence in healthcare: a critical analysis of the legal and ethical implications
Schönberger, D. (2019) · International Journal of Law and Information Technology · doi:10.1093/ijlit/eaz004 · Cited by 230 · cited 3×
“…is associated with the design of AI, the companies that developed AI for specific tasks, the healthcare professions who used AI during decision making process and the type of liability when AI-driven outcomes cause harm to patients or to the society (Davison, 2000; Schoenberger, 2019).”
BackgroundMethodKnoke, D., & Yang, S. (2020) · Sage eBooks · doi:10.4135/9781506389332 · Cited by 211
Artificial intelligence for human flourishing – Beyond principles for machine learning
Stahl, B. C., Andreou, A., Brey, P., Hatzakis, T., Kirichenko, A., Macnish, K., Laulhé Shaelou, S., Patel, A., Ryan, M., & Wright, D. (2021) · Journal of Business Research · doi:10.1016/j.jbusres.2020.11.030 · Cited by 203
From individual to group privacy in big data analytics
Mittelstadt, B. (2017) · Philosophy & Technology · doi:10.1007/s13347-017-0253-7 · Cited by 184 · cited 4×
“Since AI is deeply influenced by the data it uses and analysis to provide suggestions, it faces a well-known limitationwhere the output of data processing can never exceed the input (quality of data used) (Mittelstadt, 2017a).”
BackgroundInfluentialCHI 1994-2013: mapping two decades of intellectual progress through co-word analysis
Liu, Y., Goncalves, J., Ferreira, D., Xiao, B., Hosio, S., & Kostakos, V. (2014) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/2556288.2556969 · Cited by 174 · cited 5×
“To find those hubs, a smaller number of highly frequent, i.e., popular terms can be used, as suggested in (Cobo et al., 2011; Liu et al., 2014).”
InfluentialArtificial intelligence in healthcare and medicine: promises, ethical challenges and governance
Guan, J. (2019) · Chinese Medical Sciences Journal · doi:10.24920/003611 · Cited by 170 · cited 6×
“The identification of these problems requires input from different disciplines such as computer science, social science, medical science, economics, and others (Guan, 2019).”
BackgroundMethodInfluentialOn being ‘systematic’ in literature reviews
Boell, S. K., & Cecez-Kecmanovic, D. (2015) · Formulating Research Methods for Information Systems · doi:10.1057/9781137509888_3 · Cited by 159 · cited 1×
“We followed the guidelines provided by Boell and Cecez-Kecmanovic (2015) and developed a protocol with five subsections, namely research questions , sources searched , search terms , search strategy , inclusion , and exclusion criteria .”
MethodEthics of the health-related internet of things: a narrative review
Mittelstadt, B. (2017) · Ethics and Information Technology · doi:10.1007/s10676-017-9426-4 · Cited by 154 · cited 4×
“Since AI is deeply influenced by the data it uses and analysis to provide suggestions, it faces a well-known limitationwhere the output of data processing can never exceed the input (quality of data used) (Mittelstadt, 2017a).”
BackgroundInfluentialToward an Understanding of Responsible Artificial Intelligence Practices
Wang, Y., Xiong, M., & Olya, H. (2020) · Proceedings of the Annual Hawaii International Conference on System Sciences · doi:10.24251/hicss.2020.610 · Cited by 143 · cited 2×
“The close collaboration between human and artificial intelligence during medical decision making calls for important considerations about shared responsibility (Dignum, 2019; Wang et al., 2020).”
BackgroundGeorge Washington University School of Business, & Martin, K. (2019) · MIS Quarterly Executive · doi:10.17705/2msqe.00012 · Cited by 138 · cited 11×
“…can lead to issues with informational privacy and moral responsibility (Cohen et al., 2014; Crnkovic Dodig & Çürüklü, 2012; Krutzinna et al., 2019; Martin,2019b ; Morley et al., 2019; Ocak et al.,2020 ) decision making in health are untranslatable between different care settings, which challenges…”
BackgroundInfluentialAstromskė, K., Peičius, E., & Astromskis, P. (2020) · AI & Society · doi:10.1007/s00146-020-01008-9 · Cited by 116 · cited 6×
“The lack of transparency of AI in terms of its content, calculations, and procedures is one of the most discussed concerns, also called “ black-boxed medicine ” (Astromsk ė et al., 2020; Crnkovic Dodig & Çürüklü, 2012; Gray & Thorpe, 2015).”
BackgroundMethodInfluentialTrust me, i'm a chatbot: how artificial intelligence in health care fails the Turing Test
Powell, J. (2019) · Journal of Medical Internet Research · doi:10.2196/16222 · Cited by 115 · cited 2×
“Due to the ‘black-boxed’ nature of many algorithms, it is challenging to understand the processes followed, the data used, the rules applied, and the people involved (Guan, 2019; Powell, 2019).”
BackgroundMethodResponsible AI for conservation
Wearn, O. R., Freeman, R., & Jacoby, D. M. P. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0022-7 · Cited by 112
Review and Theory Symbiosis: An Introspective Retrospective
Baylor University and Lund University, & Leidner, D. (2018) · Journal of the Association for Information Systems · doi:10.17705/1jais.00501 · Cited by 102 · cited 2×
“We conducted a systematic literature review (Leidner, 2018; Paré et al., 2015; Schryen et al., 2020) to identify relevant research papers.”
MethodHealthcare uses of artificial intelligence: Challenges and opportunities for growth
Racine, E., Boehlen, W., & Sample, M. (2019) · Healthcare Management Forum · doi:10.1177/0840470419843831 · Cited by 102
Passive data collection and use in healthcare: A systematic review of ethical issues
Maher, N. A., Senders, J. T., Hulsbergen, A. F. C., Lamba, N., Parker, M., Onnela, J.-P., Bredenoord, A. L., Smith, T. R., & Broekman, M. L. D. (2019) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2019.06.015 · Cited by 91
Morality, Ethics, and Reflection: A Categorization of Normative IS Research
De Montfort University, & Stahl, B. (2012) · Journal of the Association for Information Systems · doi:10.17705/1jais.00304 · Cited by 90 · cited 1×
“Ethics has been discussed by philosophers for millennia with the attempt of developing moral statements related to what is good, right, or acceptable (Stahl, 2012).”
BackgroundDodig Crnkovic, G., & Çürüklü, B. (2012) · Ethics and Information Technology · doi:10.1007/s10676-011-9278-2 · Cited by 90
Big Data Analytics, Infectious Diseases and Associated Ethical Impacts
Garattini, C., Raffle, J., Aisyah, D. N., Sartain, F., & Kozlakidis, Z. (2019) · Philosophy & Technology · doi:10.1007/s13347-017-0278-y · Cited by 84
How Should Health Data Be Used?: Privacy, Secondary Use, and Big Data Sales
KAPLAN, B. (2016) · Cambridge Quarterly of Healthcare Ethics · doi:10.1017/s0963180115000614 · Cited by 84 · cited 3×
“EU Data Protection Directive in engaged with protecting users with principles such as transparency, legitimacy, and proportionality (Floridi et al., 2019; Kaplan, 2016; Turilli & Floridi, 2009).”
BackgroundMethodPaderborn University, Schryen, G., Wagner, G., University of Regensburg, Benlian, A., University of Technology Darmstadt, Paré, G., & Montréal, H. (2021) · Communications of the Association for Information Systems · doi:10.17705/1cais.04607 · Cited by 72 · cited 1×
“We conducted a systematic literature review (Leidner, 2018; Paré et al., 2015; Schryen et al., 2020) to identify relevant research papers.”
MethodProfessional Ethics in Information Systems: A Personal Perspective
Davison, R. M. (2000) · Communications of the Association for Information Systems · doi:10.17705/1cais.00308 · Cited by 66 · cited 4×
“…is associated with the design of AI, the companies that developed AI for specific tasks, the healthcare professions who used AI during decision making process and the type of liability when AI-driven outcomes cause harm to patients or to the society (Davison, 2000; Schoenberger, 2019).”
BackgroundInfluentialThe Debate on the Ethics of AI in Health Care: a Reconstruction and Critical Review
Morley, J., Machado, C., Burr, C., Cowls, J., Taddeo, M., & Floridi, L. (2019) · SSRN Electronic Journal · doi:10.2139/ssrn.3486518 · Cited by 66
Designing the Health-related Internet of Things: Ethical Principles and Guidelines
Mittelstadt, B. (2017) · Information · doi:10.3390/info8030077 · Cited by 62 · cited 4×
“Since AI is deeply influenced by the data it uses and analysis to provide suggestions, it faces a well-known limitationwhere the output of data processing can never exceed the input (quality of data used) (Mittelstadt, 2017a).”
BackgroundInfluentialGovernance by Glass-Box: Implementing Transparent Moral Bounds for AI Behaviour
Tubella, A. A., Theodorou, A. A., Dignum, F. P. M., & Dignum, V. (2019) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/802 · Cited by 61
Building truths in AI: Making predictive algorithms doable in healthcare
Henriksen, A., & Bechmann, A. (2020) · Information, Communication & Society · doi:10.1080/1369118x.2020.1751866 · Cited by 58
Jiang, J. J., Klein, G. L., & Fernandez, W. D. (2018) · Journal of the Association for Information Systems · doi:10.17705/1jais.00480 · Cited by 56
Responsible AI and moral responsibility: a common appreciation
Tigard, D. W. (2021) · AI and Ethics · doi:10.1007/s43681-020-00009-0 · Cited by 56 · cited 3×
“This can lead to issues with moral responsibility (Tigard, 2020) and thus epistemic and normative ethical issues related to the use of algorithms.”
BackgroundA Deontological Approach to Designing Ethical Collaboration
Millikin University, Chatterjee, S., Sarker, S., Washington State University, U. S. A., Fuller, M., & Washington State University, U. S. A. (2009) · Journal of the Association for Information Systems · doi:10.17705/1jais.00190 · Cited by 54 · cited 2×
“Virtue ethics makes a theoretical distinction between good and bad based on individual ’ s virtues of mind, character, and sense of honesty and not on external aspects of an action (Chatterjee et al., 2009; Gal et al., 2020).”
BackgroundToward responsible AI for the next billion users
Sambasivan, N., & Holbrook, J. (2018) · Interactions · doi:10.1145/3298735 · Cited by 51
Hörnle, J. (2019) · International Journal of Law and Information Technology · doi:10.1093/ijlit/eaz002 · Cited by 43 · cited 3×
“…is associated with the design of AI, the companies that developed AI for specific tasks, the healthcare professions who used AI during decision making process and the type of liability when AI-driven outcomes cause harm to patients or to the society (Davison, 2000; Schoenberger, 2019).”
BackgroundMethodMittelstadt, B., Stahl, B. C., & Fairweather, N. B. (2015) · Ethical Theory and Moral Practice · doi:10.1007/s10677-015-9582-8 · Cited by 40
Krutzinna, J. I., Taddeo, M., & Floridi, L. (2018) · Science and Engineering Ethics · doi:10.1007/s11948-018-0067-8 · Cited by 36
Computing Education Research Landscape through an Analysis of Keywords
Papamitsiou, Z. K., Giannakos, M. N., Simon, & Luxton-Reilly, A. (2020) · ACM Conference on International Computing Education Research (ICER) · doi:10.1145/3372782.3406276 · Cited by 36 · cited 2×
“The strategic diagrams use the centrality and density of each cluster to help us understand the relative “ positions ” of these clusters within the overall landscape of ethics stemming from AI in healthcare (Liu et al., 2014; Papamitsiou et al., 2020).”
BackgroundMethodSelf-citationGray, E. A., & Thorpe, J. H. (2015) · Journal of Comparative Effectiveness Research · doi:10.2217/cer.14.51 · Cited by 34
Key ethical challenges in the European medical information framework
Floridi, L., Luetge, C., Pagallo, U., Schafer, B., Valcke, P., Vayena, E., Addison, J., Hughes, N., Lea, N., Sage, C., et al. (2019) · Minds and Machines · doi:10.1007/s11023-018-9467-4 · Cited by 33
Chen, L., Baird, A. M., & Straub, D. W. (2019) · Journal of the Association for Information Systems · doi:10.17705/1jais.00561 · Cited by 30
Ethical implications of social stratification in information systems research
Berente, N., Gal, U., & Hansen, S. (2010) · Information Systems Journal · doi:10.1111/j.1365-2575.2010.00353.x · Cited by 27
Let’s Claim the Authority to Speak Out on the Ethics of Smart Information Systems
Stahl, B. C., & Markus, M. L. (2021) · MIS Quarterly · doi:10.25300/misq/2021/15434.1.6 · Cited by 11 · cited 1×
“Future scholars might consider other frameworks or core values such as compliance, acceptability, proactivity, reflexivity as suggested by (Stahl & Markus, 2021).”
Business ethics research in healthcare management: A systematic review
Ocak, S., Köseoğlu, M. A., & Yildiz, M. (2017) · International Journal of Healthcare Management · doi:10.1080/20479700.2017.1336882 · Cited by 10
Woolley, J. P. (2019) · Philosophy & Technology · doi:10.1007/s13347-017-0288-9 · Cited by 10 · cited 3×
“Therefore, authors call for digital health fiduciaries to protect users during information sharing (Mittelstadt & Floridi, 2016; Woolley, 2019).”
BackgroundLee, M. S. A., Floridi, L., & Singh, J. (2020) · SSRN Electronic Journal · doi:10.2139/ssrn.3679975 · Cited by 10
(2019)
Cited by
Ooi, K.-B., Tan, G. W.-H., Al-Emran, M., Al-Sharafi, M. A., Capatina, A., Chakraborty, A., Dwivedi, Y. K., Huang, T.-L., Kar, A. K., Lee, V.-H., et al. (2025) · Journal of Computer Information Systems · doi:10.1080/08874417.2023.2261010 · Cited by 746
“130 This can be achieved by analyzing the patients’ medical results and identifying potential health issues that may be inadvertently missed by clinicians.”
Self-citationSurvey of Explainable AI Techniques in Healthcare
Chaddad, A., Peng, J., Xu, J., & Bouridane, A. (2023) · Sensors · doi:10.3390/s23020634 · Cited by 591
“For example, irresponsible AI may lead to a loss of medical staff and patients [18].”
BackgroundWhat influences algorithmic decision-making? A systematic literature review on algorithm aversion
Mahmud, H., Islam, A. K. M. N., Ahmed, S. I., & Smolander, K. (2022) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2021.121390 · Cited by 601
Responsible Artificial Intelligence Governance: A Review and Research Framework
Papagiannidis, E., Mikalef, P., & Conboy, K. (2025) · The Journal of Strategic Information Systems · doi:10.1016/j.jsis.2024.101885 · Cited by 565
Self-citationThinking responsibly about responsible AI and ‘the dark side’ of AI
Mikalef, P., Conboy, K., Lundström, J. E., & Popovič, A. (2022) · European Journal of Information Systems · doi:10.1080/0960085x.2022.2026621 · Cited by 435
“Time also plays an immediate key role at the very point that AI or analytics are used, and it is this timing that may determine whether said analytics or AI is, in fact, dark (Conboy et al., 2020; Trocin et al., 2021).”
BackgroundSelf-citation
Metadata
- Authors
- Trocin, C., Mikalef, P., Papamitsiou, Z., & Conboy, K.
- Affiliations
- Norwegian University of Science and Technology (NO)
- Ollscoil na Gaillimhe – University of Galway (IE)
- Corresponding author
- Trocin, Cristina
- Venue
- Information Systems Frontiers
- Published
- 2021-06-26
- Volume
- 25
- Issue
- 6
- Pages
- 2139-2157
- ISSN
- 1387-3326 (ISSN-L)
- 1572-9419
- OpenAlex
- W3173496715
- Cited by
- 205
- References
- 86
- Citations by year
- 2026: 36 · 2025: 56 · 2024: 48 · 2023: 30 · 2022: 24 · 2021: 11
- FWCI
- 35.36
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- Information Systems Frontiers (publishedVersion, cc-by)
- RePEc: Research Papers in Economics (submittedVersion)
- NORA - Norwegian Open Research Archives (submittedVersion, cc-by)
- Newcastle University ePrints (Newcastle Univesity) (cc-by)
- Funders
- European Research Consortium for Informatics and Mathematics
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Keywords
- responsible AI, digital health, health AI, algorithmic bias, fairness, transparency, accountability, explainability, interpretability, AI ethics, research agenda
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-05, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-05 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol and qwen3.8-omni-flash).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.