The DOI matches its Crossref record; 88 of the 89 references it cites were found in a registry or an index.
Bridging the Gap Between Ethics and Practice: Guidelines for Reliable, Safe, and Trustworthy Human-centered AI Systems
Abstract
This article attempts to bridge the gap between widely discussed ethical principles of Human-centered AI (HCAI) and practical steps for effective governance. Since HCAI systems are developed and implemented in multiple organizational structures, I propose 15 recommendations at three levels of governance: team, organization, and industry. The recommendations are intended to increase the reliability, safety, and trustworthiness of HCAI systems: (1) reliable systems based on sound software engineering practices, (2) safety culture through business management strategies, and (3) trustworthy certification by independent oversight. Software engineering practices within teams include audit trails to enable analysis of failures, software engineering workflows, verification and validation testing, bias testing to enhance fairness, and explainable user interfaces. The safety culture within organizations comes from management strategies that include leadership commitment to safety, hiring and training oriented to safety, extensive reporting of failures and near misses, internal review boards for problems and future plans, and alignment with industry standard practices. The trustworthiness certification comes from industry-wide efforts that include government interventions and regulation, accounting firms conducting external audits, insurance companies compensating for failures, non-governmental and civil society organizations advancing design principles, and professional organizations and research institutes developing standards, policies, and novel ideas. The larger goal of effective governance is to limit the dangers and increase the benefits of HCAI to individuals, organizations, and society.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“These challenges raise ethical questions around accountability-who is responsible when AI-generated code causes harm?-and the need for governance mechanisms that ensure human oversight and traceability [34].”
From Redefining the Programmer: Human-AI Collaboration, LLMs, and Security in Modern Software Engineering (2025) · cites this paper as [34]
In the citing paper
“In short, currently available knowledge related to responsible and ethical AI is often of a high level of abstraction.”
From Contestable AI by Design: Towards a Framework (2022) · cites this paper as Shneiderman, 2020
In the citing paper
“while AI tools are essential in detecting harmful content, their effectiveness remains highly variable, requiring structured evaluation frameworks to assess their reliability and ethical implications (Shneiderman, 2020).”
From Virtual Reality Risks for Children in the Metaverse: An Examination through Criminological Perspectives (2026) · cites this paper as Shneiderman, 2020
In the citing paper
“When these mechanisms are aligned with efforts needed to establish social, organizational, and regulatory ecosystems for the assurance of trustworthy AI [9, 30, 54], they are likely to satisfy the calibration condition.”
From Designing for Responsible Trust in AI Systems: A Communication Perspective (2022) · cites this paper as [54]
In the citing paper
“Finally, as monitoring and auditing of AIPMs in practice are widely recommended (covered in more detail in phase 6), the developed software should facilitate this10,22,26,32,58,62,104.”
From Guidelines and quality criteria for artificial intelligence-based prediction models in healthcare: a scoping review (2022) · cites this paper as [58]
5 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Research in Organizational Behavior.
Grimes, A. J., Cummings, L. L., & Staw, B. M. (1986) · Administrative Science Quarterly · doi:10.2307/2392975 · Cited by 10,899
Rudin, C. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0048-x · Cited by 10,561 · cited 1×
“Du et al. [28] make useful distinctions between intrinsically understandable machine learning models, such as decision trees or rule-based models [67, 26, 97], and the more common approach of post hoc explanations.”
BackgroundExplanation in artificial intelligence: Insights from the social sciences
Miller, T. (2019) · Artificial Intelligence · doi:10.1016/j.artint.2018.07.007 · Cited by 5,302
Towards A Rigorous Science of Interpretable Machine Learning
Doshi-Velez, F., & Kim, B. (2017) · arXiv · doi:10.48550/arxiv.1702.08608 · Cited by 3,121 · cited 1×
“Du et al. [28] make useful distinctions between intrinsically understandable machine learning models, such as decision trees or rule-based models [67, 26, 97], and the more common approach of post hoc explanations.”
BackgroundResilience Engineering: Concepts and Precepts
Wears, R. L. (2006) · Quality and Safety in Health Care · doi:10.1136/qshc.2006.018390 · Cited by 2,900
Raji, I. D., Smart, A., White, R. N., Mitchell, M., Gebru, T., Hutchinson, B., Smith-Loud, J., Theron, D., & Barnes, P. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372873 · Cited by 2,245 · cited 2×
“However, they can produce important internal efforts such as Google’s internal review and algorithmic auditing framework [92] (see Section 3.”
BackgroundEuropean Union Regulations on Algorithmic Decision Making and a “Right to Explanation”
Goodman, B., & Flaxman, S. (2017) · AI Magazine · doi:10.1609/aimag.v38i3.2741 · Cited by 2,131 · cited 1×
“Furthermore, explanations have become a legal requirement in many countries based on the European Union’s General Data Protection Regulation (GDPR) requirement of a “right to explanation” [41, 115].”
BackgroundThe nature of safety culture: a review of theory and research
Guldenmund, F. W. (2000) · Safety Science · doi:10.1016/s0925-7535(00)00014-x · Cited by 1,831 · cited 1×
“A fourth theme described how organizations cultivate safety cultures [43, 8], especially in healthcare [32] by open management strategies.”
BackgroundHuman-Centered Artificial Intelligence: Reliable, Safe & Trustworthy
Shneiderman, B. (2020) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2020.1741118 · Cited by 1,743 · cited 2×
“In contrast, HCAI focuses on amplifying, augmenting, and enhancing human performance [103] in ways that make systems reliable, safe, and trustworthy [102].”
BackgroundSelf-citationRule-Based Expert Systems: The MYCIN Experiments of the Stanford Heuristic Programming Project
(1986) · Telematics and Informatics · doi:10.1016/s0736-5853(86)80073-9 · Cited by 1,701
Bellamy, R. K. E., Dey, K., Hind, M., Hoffman, S. C., Houde, S., Kannan, K., Lohia, P., Martino, J., Mehta, S., Mojsilovic, A., et al. (2018) · arXiv · doi:10.48550/arxiv.1810.01943 · Cited by 1,581
Engineering a Safer World: Systems Thinking Applied to Safety
(2012) · Choice Reviews Online · doi:10.5860/choice.49-6305 · Cited by 1,566 · cited 2×
“Leveson [68] develops a systems engineering approach to safety engineering that includes design, hazard analysis, and failure investigations.”
BackgroundTechniques for interpretable machine learning
Du, M., Liu, N., & Hu, X. (2019) · Communications of the ACM · doi:10.1145/3359786 · Cited by 1,268 · cited 2×
“In addition to the distinctions between intrinsically understandable models, post hoc , and prospective explanations, Du et al. [28] follow other researchers in distinguishing between global explanations that give an overview of what the algorithm does and local explanations that deal with specific…”
BackgroundCapability maturity model, version 1.1
Paulk, M. C., Curtis, B., Chrissis, M. B., & Weber, C. V. (1993) · IEEE Software · doi:10.1109/52.219617 · Cited by 1,263 · cited 1×
“Another approach to improve software quality is the Capability Maturity Model, developed by the Software Engineering Institute in the late 1980s [55], which is regularly updated [85, 113].”
MethodFriedman, B., & Nissenbaum, H. (1996) · ACM Transactions on Information Systems · doi:10.1145/230538.230561 · Cited by 1,203 · cited 1×
“An early review by Friedman and Nissenbaum [39] described three kind of bias, which remain as a helpful guide: preexisting biases based on social practices and attitudes, technical bias based on design constraints in hardware and software, and emergent bias that arises from changing the use context.”
BackgroundCounterfactual explanations without opening the black box: automated decisions and the GDPR
Wachter, S., Mittelstadt, B., & Russell, C. (2017) · SSRN Electronic Journal · doi:10.2139/ssrn.3063289 · Cited by 1,061
Fjeld, J., Achten, N., Hilligoss, H., Nagy, A., & Srikumar, M. (2020) · SSRN Electronic Journal · doi:10.2139/ssrn.3518482 · Cited by 998 · cited 4×
“The HCAI movement has raised these issues again with frequent calls for Algo-rithmic Accountability and Transparency [40], Ethically Aligned Design [56], and Professional Responsibility [36].”
BackgroundInfluentialA governance model for the application of AI in health care
Reddy, S., Allan, S., Coghlan, S., & Cooper, P. (2020) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocz192 · Cited by 979 · cited 1×
“Implementations of safety cultures in HCAI-based organizations is emerging with initial efforts to support AI governance in medical care [19, 93].”
BackgroundArtificial intelligence, bias and clinical safety
Challen, R., Denny, J., Pitt, M., Gompels, L., Edwards, T., & Tsaneva-Atanasova, K. (2019) · BMJ Quality & Safety · doi:10.1136/bmjqs-2018-008370 · Cited by 956 · cited 1×
“Implementations of safety cultures in HCAI-based organizations is emerging with initial efforts to support AI governance in medical care [19, 93].”
BackgroundDesigning Theory-Driven User-Centric Explainable AI
Wang, D., Yang, Q., Abdul, A., & Lim, B. Y. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300831 · Cited by 906 · cited 1×
“This controversial GDPR requirement is vague and difficult to satisfy in general, but international research efforts at developing Explainable AI (XAI) have blossomed [9, 47, 48, 106, 116, 75, 71].”
BackgroundMachine Learning Testing: Survey, Landscapes and Horizons
Zhang, J. M., Harman, M., Ma, L., & Liu, Y. (2022) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2019.2962027 · Cited by 897
AI Fairness 360: An extensible toolkit for detecting and mitigating algorithmic bias
Bellamy, R. K. E., Dey, K., Hind, M., Hoffman, S. C., Houde, S., Kannan, K., Lohia, P., Martino, J., Mehta, S., Mojsilović, A., et al. (2019) · IBM Journal of Research and Development · doi:10.1147/jrd.2019.2942287 · Cited by 884 · cited 1×
“Beyond detecting biases, researchers offer fairness-enhancing interventions [38] and companies have developed commercial grade toolkits for detecting and mitigating algorithmic bias [7].”
BackgroundMittelstadt, B., Russell, C., & Wachter, S. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287574 · Cited by 794 · cited 2×
“This controversial GDPR requirement is vague and difficult to satisfy in general, but international research efforts at developing Explainable AI (XAI) have blossomed [9, 47, 48, 106, 116, 75, 71].”
BackgroundImproving Fairness in Machine Learning Systems: What Do Industry Practitioners Need?
Holstein, K., Wortman Vaughan, J., Daumé, H., Dudik, M., & Wallach, H. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300830 · Cited by 777 · cited 1×
“Commercial practices began to shift when serious problems emerged [51] from biased decisions that influenced parole granting, when hate-filled chatbots learned from malicious social media postings, and when job hiring biases were exposed [87].”
BackgroundLetham, B., Rudin, C., McCormick, T. H., & Madigan, D. (2015) · The Annals of Applied Statistics · doi:10.1214/15-aoas848 · Cited by 766 · cited 1×
“Du et al. [28] make useful distinctions between intrinsically understandable machine learning models, such as decision trees or rule-based models [67, 26, 97], and the more common approach of post hoc explanations.”
BackgroundCharacterizing the software process: a maturity framework
Humphrey, W. S. (1988) · IEEE Software · doi:10.1109/52.2014 · Cited by 659 · cited 1×
“Another approach to improve software quality is the Capability Maturity Model, developed by the Software Engineering Institute in the late 1980s [55], which is regularly updated [85, 113].”
MethodA comparative study of fairness-enhancing interventions in machine learning
Friedler, S. A., Scheidegger, C., Venkatasubramanian, S., Choudhary, S., Hamilton, E. P., & Roth, D. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287589 · Cited by 601 · cited 1×
“Beyond detecting biases, researchers offer fairness-enhancing interventions [38] and companies have developed commercial grade toolkits for detecting and mitigating algorithmic bias [7].”
BackgroundHigh Reliability Organizations: Unlikely, Demanding and At Risk
La Porte, T. R. (1996) · Journal of Contingencies and Crisis Management · doi:10.1111/j.1468-5973.1996.tb00078.x · Cited by 599
Invisible Women Exposing Data Bias in a World Designed for Men
Williams, N. (2020) · Occupational Medicine · doi:10.1093/occmed/kqaa038 · Cited by 573
Meaningful Human Control over Autonomous Systems: A Philosophical Account
Santoni de Sio, F., & van den Hoven, J. (2018) · Frontiers in Robotics and AI · doi:10.3389/frobt.2018.00015 · Cited by 566
The Black Box Society: The Secret Algorithms That Control Money and Information
(2015) · Choice Reviews Online · doi:10.5860/choice.190706 · Cited by 533 · cited 1×
“These constructive steps are a positive sign, but the persistence of bias remains a problem as applications such as facial recognition become more widely used for police work and commercial applications [82, 84].”
BackgroundEthical governance is essential to building trust in robotics and artificial intelligence systems
Winfield, A. F. T., & Jirotka, M. (2018) · Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences · doi:10.1098/rsta.2018.0085 · Cited by 493 · cited 2×
“These and other ethical principles are an important foundation for clear thinking, but as Winfield and Jirotka [122] note: “The gap between principles and practice is an important theme.””
BackgroundReliability Engineering and Risk Analysis: A Practical Guide
Modarres, M., Kaminskiy, M. P., & Krivtsov, V. (2016) · CRC Press eBooks · doi:10.1201/9781315382425 · Cited by 457
Normal Accidents: Living with High Risk Technologies - Updated Edition
Perrow, C. (2000) · Princeton University Press eBooks · doi:10.1515/9781400828494 · Cited by 456
Toward human-centered AI: a perspective from human-computer interaction
Xu, W. (2019) · Interactions · doi:10.1145/3328485 · Cited by 454
Baeza-Yates, R. (2018) · Communications of the ACM · doi:10.1145/3209581 · Cited by 432 · cited 1×
“Baeza-Yates [5] described additional forms of bias, such as geography, language, and culture, which were embedded in web-based algorithms, databases, and user interfaces.”
BackgroundA Review of User Interface Design for Interactive Machine Learning
Dudley, J. J., & Kristensson, P. O. (2018) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3185517 · Cited by 343 · cited 1×
“These guidelines build on a long history of user interface design [104, 100] and newer research on designing interfaces for HCAI systems [29, 4, 103].”
BackgroundA survey on provenance: What for? What form? What from?
Herschel, M., Diestelkämper, R., & Ben Lahmar, H. (2017) · The VLDB Journal · doi:10.1007/s00778-017-0486-1 · Cited by 300 · cited 1×
“Lessons from database systems [46] and information visualization [91] are useful.”
BackgroundAgency plus automation: Designing artificial intelligence into interactive systems
Heer, J. (2019) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1807184115 · Cited by 279
Sources of Power: How People Make Decisions
(2001) · International Journal of Manpower · doi:10.1108/ijm.2001.22.7.664.2 · Cited by 271
The challenge of crafting intelligible intelligence
Weld, D. S., & Bansal, G. (2019) · Communications of the ACM · doi:10.1145/3282486 · Cited by 253 · cited 3×
“Interactive HCAI approaches are endorsed by Weld and Bansal [118], who recommend that designers should “make the explanation system interactive so users can drill down until they are satisfied with their understanding.””
BackgroundInfluentialEssential Characteristics of Resilience
Woods, D. D. (2017) · Resilience Engineering · doi:10.1201/9781315605685-4 · Cited by 248 · cited 1×
“More positive approaches were offered in descriptions of High Reliability Organizations [61, 12, 25], which emerged from organizational design and business administration, and Resilience Engineering [123, 44], which grew out of cognitive science and human factors engineering.”
BackgroundRagan, E. D., Endert, A., Sanyal, J., & Chen, J. (2016) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2015.2467551 · Cited by 240 · cited 1×
“Lessons from database systems [46] and information visualization [91] are useful.”
BackgroundWeapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy
Verma, S. (2019) · Vikalpa: The Journal for Decision Makers · doi:10.1177/0256090919853933 · Cited by 219 · cited 1×
“…learning algorithms were applied to consequential applications such as parole granting, mortgage loan approval, and job interviewing, many critics [81] raised questions about their: —opacity (difficulty in challenging algorithmic decisions), —scale (use by large companies and governments for…”
MethodHuman factors in model interpretability: Industry practices, challenges, and needs
Hong, S. R., Hullman, J., & Bertini, E. (2020) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3392878 · Cited by 219 · cited 1×
“An interview study with 22 machine learning professionals documented the value of explainability for developers, testers, managers, and users [52].”
BackgroundHohman, F., Park, H., Robinson, C., & Polo Chau, D. H. (2020) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2019.2934659 · Cited by 213 · cited 1×
“For complex decisions, more elaborate user interfaces and data analytics should clarify which of the many features are the most relevant [49], even in deep learning for image understanding [50].”
BackgroundDon’t ask if artificial intelligence is good or fair, ask how it shifts power
Kalluri, P. (2020) · Nature · doi:10.1038/d41586-020-02003-2 · Cited by 213 · cited 1×
“While governments and corporations often make strong positive statements about their commitment to serving stakeholder needs, when leaders have to make difficult decisions about power and money, they may favor their personal needs, political pressures, and stockholder expectations [58].”
BackgroundExecution monitoring in robotics: A survey
Pettersson, O. (2005) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2005.09.004 · Cited by 194 · cited 1×
“FDRs provide important lessons for HCAI designers of audit trails (also called product logs) to record the actions of robots [89, 121, 106, 74].”
BackgroundRaji, I. D., Smart, A., White, R. N., Mitchell, M., Gebru, T., Hutchinson, B., Smith-Loud, J., Theron, D., & Barnes, P. (2020) · arXiv · doi:10.48550/arxiv.2001.00973 · Cited by 179 · cited 2×
“However, they can produce important internal efforts such as Google’s internal review and algorithmic auditing framework [92] (see Section 3.”
BackgroundTrusting Virtual Agents: The Effect of Personality
Zhou, M. X., Mark, G., Li, J., & Yang, H. (2019) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3232077 · Cited by 176 · cited 1×
“Further gains come from enabling adaptable user interfaces to fit different needs and personalities [127, 119].”
BackgroundBerry, J. C., Davis, J. T., Bartman, T., Hafer, C. C., Lieb, L. M., Khan, N., & Brilli, R. J. (2020) · Journal of Patient Safety · doi:10.1097/pts.0000000000000251 · Cited by 162 · cited 2×
“A fourth theme described how organizations cultivate safety cultures [43, 8], especially in healthcare [32] by open management strategies.”
BackgroundDesigning and implementing transparency for real time inspection of autonomous robots
Theodorou, A. A., Wortham, R. H., & Bryson, J. J. (2017) · Connection Science · doi:10.1080/09540091.2017.1310182 · Cited by 154 · cited 2×
“This controversial GDPR requirement is vague and difficult to satisfy in general, but international research efforts at developing Explainable AI (XAI) have blossomed [9, 47, 48, 106, 116, 75, 71].”
BackgroundA model for unpacking big data analytics in high-frequency trading
Seddon, J. J. M., & Currie, W. L. (2016) · Journal of Business Research · doi:10.1016/j.jbusres.2016.08.003 · Cited by 153 · cited 1×
“Audit trails for stock market trading algorithms are now widely used to log trades so that managers, customers, and the U.S. Securities and Exchange Commission can study errors, detect fraud, or recover from flash crash events [99, 112].”
BackgroundDesign Lessons From AI’s Two Grand Goals: Human Emulation and Useful Applications
Shneiderman, B. (2020) · IEEE Transactions on Technology and Society · doi:10.1109/tts.2020.2992669 · Cited by 130 · cited 3×
“These traditional humancomputer interaction methods and guidelines [104, 100] are being updated by leading corporations and researchers to meet the needs of HCAI [102, 103].”
BackgroundMethodSelf-citationSystematic literature review of usability capability/maturity models
Lacerda, T. C., & von Wangenheim, C. G. (2017) · Computer Standards & Interfaces · doi:10.1016/j.csi.2017.06.001 · Cited by 127 · cited 1×
“…Maturity Model Integration version comes with the claim that it helps “inte-grate traditionally separate organizational functions, set process improvement goals and priorities, provide guidance for quality processes, and provide a point of reference for appraising current pro-cesses” [37, 62].”
BackgroundA health data analytics maturity model for hospitals information systems
Carvalho, J. V., Rocha, Á., de Vasconcelos, J. B., & Abreu, A. J. (2018) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2018.07.001 · Cited by 108
Machine Learning Testing: Survey, Landscapes and HorizonsPreprint
Zhang, J. M., Harman, M., Ma, L., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.10742 · Cited by 108 · cited 4×
“[126] describe five typical tasks for machine learning, which may need distinctive workflows: “(1) Classification: to assign a category to each data instance, e.”
BackgroundInfluentialHRO and RE: A pragmatic perspective
Haavik, T. K., Antonsen, S., Rosness, R., & Hale, A. (2016) · Safety Science · doi:10.1016/j.ssci.2016.08.010 · Cited by 97 · cited 1×
“More positive approaches were offered in descriptions of High Reliability Organizations [61, 12, 25], which emerged from organizational design and business administration, and Resilience Engineering [123, 44], which grew out of cognitive science and human factors engineering.”
BackgroundToward algorithmic transparency and accountability
Garfinkel, S., Matthews, J. N., Shapiro, S. S., & Smith, J. M. (2017) · Communications of the ACM · doi:10.1145/3125780 · Cited by 92 · cited 2×
“The HCAI movement has raised these issues again with frequent calls for Algo-rithmic Accountability and Transparency [40], Ethically Aligned Design [56], and Professional Responsibility [36].”
BackgroundExplaining Explanation, Part 1: Theoretical Foundations
Hoffman, R. R., & Klein, G. A. (2017) · IEEE Intelligent Systems · doi:10.1109/mis.2017.54 · Cited by 86 · cited 1×
“This controversial GDPR requirement is vague and difficult to satisfy in general, but international research efforts at developing Explainable AI (XAI) have blossomed [9, 47, 48, 106, 116, 75, 71].”
BackgroundCreating software process capability / maturity models
von Wangenheim, C. G., Hauck, J. C. R., Zoucas, A., Salviano, C. F., McCaffery, F., & Shull, F. J. (2010) · STOR (Dundalk Institute of Technology) · doi:10.34961/689 · Cited by 85
Turning point: policymaking in the era of artificial intelligence
Chen, K. (2022) · International Affairs · doi:10.1093/ia/iiac074 · Cited by 81 · cited 1×
“” A broad review of policymaking pointed to AI’s distinctive features of intentionality, intelligence, and adaptability, which necessitated new initiatives from government and other bodies [120].”
Forensic Media : Reconstructing Accidents in Accelerated Modernity
Siegel, G. (2014) · Project Muse (Johns Hopkins University) · Cited by 75
The dangers of faulty, biased, or malicious algorithms requires independent oversight
Shneiderman, B. (2016) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1618211113 · Cited by 73 · cited 5×
“Three independent oversight methods are common (Figure 4) [101]:”
BackgroundMethodInfluentialSelf-citationBell, S. K., Smulowitz, P. B., Woodward, A. C., Mello, M. M., DUVA, A. M., Boothman, R. C., & Sands, K. (2012) · Milbank Quarterly · doi:10.1111/j.1468-0009.2012.00679.x · Cited by 66 · cited 2×
“Professional and organizational pride also increased [6].”
BackgroundCyber risk research impeded by disciplinary barriers
Falco, G. J., Eling, M., Jablanski, D., Weber, M., Miller, V., Gordon, L. A., Wang, S. S., Schmit, J. T., Thomas, R. C., Elvedi, M., et al. (2019) · Science · doi:10.1126/science.aaz4795 · Cited by 66
Survey of Research for Performance Measurement of Mobile Manipulators
Bostelman, R. V., Hong, T. H., & Marvel, J. A. (2016) · Journal of Research of the National Institute of Standards and Technology · doi:10.6028/jres.121.015 · Cited by 63 · cited 1×
“Metrics for “safe operation, task completion, time to complete the task, quality, and quantity of tasks completed” will guide development [10].”
BackgroundPerez, A., Garcia, M. I., Nieto, M., Pedraza, J. L., Rodriguez, S., & Zamorano, J. (2010) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2010.2046323 · Cited by 61 · cited 1×
“Similar proposals have been made for highly automated (also called self-driving or driverless) cars [86, 125], which extend current work on electronic logging devices, which are installed on many cars to support better maintenance.”
BackgroundExplaining Explanation, Part 2: Empirical Foundations
Hoffman, R. R., Mueller, S. T., & Klein, G. A. (2017) · IEEE Intelligent Systems · doi:10.1109/mis.2017.3121544 · Cited by 56 · cited 1×
“This controversial GDPR requirement is vague and difficult to satisfy in general, but international research efforts at developing Explainable AI (XAI) have blossomed [9, 47, 48, 106, 116, 75, 71].”
BackgroundProceedings of the 26th International Workshop on Principles of Diagnosis (DX-2015)
Pencolé, Y., Travé-Massuyès, L., & Dague, P. (2015) · HAL (Le Centre pour la Communication Scientifique Directe) · Cited by 54
Pasquale, F. (2017) · Digital Commons at University of Maryland Carey Law (University of Maryland Francis King Carey School of Law) · Cited by 49
User Evaluations on Sentiment-based Recommendation Explanations
Chen, L., Yan, D., & Wang, F. (2019) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3282878 · Cited by 41
Robust artificial intelligence and robust human organizations
Dietterich, T. G. (2018) · Frontiers of Computer Science · doi:10.1007/s11704-018-8900-4 · Cited by 36 · cited 1×
“More positive approaches were offered in descriptions of High Reliability Organizations [61, 12, 25], which emerged from organizational design and business administration, and Resilience Engineering [123, 44], which grew out of cognitive science and human factors engineering.”
BackgroundThe Smart Black Box: A Value-Driven High-Bandwidth Automotive Event Data Recorder
Yao, Y., & Atkins, E. (2021) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2020.2971385 · Cited by 33 · cited 1×
“Similar proposals have been made for highly automated (also called self-driving or driverless) cars [86, 125], which extend current work on electronic logging devices, which are installed on many cars to support better maintenance.”
BackgroundValidation of Autonomous Systems
Ebert, C., & Weyrich, M. (2019) · IEEE Software · doi:10.1109/ms.2019.2921037 · Cited by 32 · cited 1×
“As new requirements are added or the context of use changes, new test cases can be added [30].”
BackgroundKavi, K. M. (2010) · IEEE Spectrum · doi:10.1109/mspec.2010.5520630 · Cited by 22 · cited 1×
“This history of FDRs and the Cockpit Voice Recorders (CVR) demonstrates the value of using these tools to understand aviation crashes [42, 16, 59], which have contributed strongly to safe civil aviation.”
BackgroundProtecting Individual Privacy in the Struggle Against Terrorists: A Framework for Program Assessment
Telecommunications Board (2008) · Cited by 21
Explaining Decisions Made with AIPreprint
Leslie, D. (2020) · SSRN Electronic Journal · doi:10.2139/ssrn.4033308 · Cited by 20 · cited 1×
“A useful and practical resource are the three reports on “Explaining decisions made with AI” from the U.K. Information Commissioner’s Office and the Alan Turing Institute [57].”
BackgroundEventAction: A Visual Analytics Approach to Explainable Recommendation for Event Sequences
Du, F., Catherine Plaisant, B., Spring, N. T., Crowley, K., & Shneiderman, B. (2019) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3301402 · Cited by 19
Self-citationWe need a building code for building code
Landwehr, C. E. (2015) · Communications of the ACM · doi:10.1145/2700341 · Cited by 18 · cited 1×
“Building codes could be a model for software engineers, as described in Landwehr’s proposal for “a building code for building code” [64, 65].”
MethodWhen machine learning is facially invalid
Pasquale, F. (2018) · Communications of the ACM · doi:10.1145/3241367 · Cited by 18 · cited 1×
“These constructive steps are a positive sign, but the persistence of bias remains a problem as applications such as facial recognition become more widely used for police work and commercial applications [82, 84].”
BackgroundSoftware Black Box: an alternative mechanism for failure analysis
Elbaum, S., & Munson, J. C. (2002) · Proceedings 11th International Symposium on Software Reliability Engineering. ISSRE 2000 · doi:10.1109/issre.2000.885887 · Cited by 15 · cited 1×
“These robot versions of aviation flight data recorders have been called software, smart, ethical, or robot black boxes, but the consistent intention of designers is to collect relevant evidence for retrospective analyses of failures [105, 31].”
BackgroundA Visual Approach for Interactive Keyterm-Based Clustering
Nourashrafeddin, S., Sherkat, E., Minghim, R., & Milios, E. (2018) · ACM Trans. Interact. Intell. Syst. · doi:10.1145/3181669 · Cited by 15 · cited 1×
“In general, exploratory user interfaces are welcomed by users who spend more time developing an understanding of the sensitivity of variables and digging more deeply into aspects that interest them, leading to greater satisfaction and compliance with recommendations [80, 27, 21].”
BackgroundPutting the “I” in Interaction: Interactive Interfaces Personalized to Individuals
Wenskovitch, J. E., Zhou, M. X., Collins, C. M., Chang, R., Dowling, M., Endert, A., & Xu, K. (2020) · IEEE Computer Graphics and Applications · doi:10.1109/mcg.2020.2982465 · Cited by 15 · cited 2×
“Each idea requires research and testing to validate effectiveness, but they are oriented to designing HCAI systems that are reliable, safe, and trustworthy, which in turn bring benefits to individuals, organizations, and society [68, 119].”
BackgroundAuditChain: A Trading Audit Platform Over Blockchain
Vishnia, G. R., & Peters, G. W. (2020) · Frontiers in Blockchain · doi:10.3389/fbloc.2020.00009 · Cited by 10 · cited 1×
“Securities and Exchange Commission can study errors, detect fraud, or recover from flash crash events [99, 112].”
BackgroundA building code for building code: putting what we know works to work
Landwehr, C. E. (2013) · Annual Computer Security Applications Conference · doi:10.1145/2523649.2530278 · Cited by 8
Tracking Minor and Near-Miss Events and Sharing Lessons Learned as a Way to Prevent Accidents
Landon, P., Weaver, P., & Fitch, J. P. (2016) · Applied Biosafety · doi:10.1177/1535676016646642 · Cited by 8
Keeping the Lights On: A Comparison of Normal Accidents and High Reliability Organizations
Brown, H. (2018) · IEEE Technology and Society Magazine · doi:10.1109/mts.2018.2826059 · Cited by 5 · cited 1×
“More positive approaches were offered in descriptions of High Reliability Organizations [61, 12, 25], which emerged from organizational design and business administration, and Resilience Engineering [123, 44], which grew out of cognitive science and human factors engineering.”
Background
Research works printed but not linked: 32
- A. Abdul , J. Vermeulen , D. Wang , B. Y. Lim , and M. Kankanhalli . 2018. Trends and trajectories for explainable, accountable and intelligible systems: An HCI research agenda . Proceedings of the CHI Conference on Human Factors in Computing Systems. ACM, 1--18 . A. Abdul, J. Vermeulen, D. Wang, B. Y. Lim, and M. Kankanhalli. 2018. Trends and trajectories for explainable, accountable and intelligible systems: An HCI research agenda. Proceedings of the CHI Conference on Human Factors in Computing Systems. ACM, 1--18.
- S. Alsheibani , C. Messom , and Y. Cheung . 2019. Towards an artificial intelligence maturity model: From science fiction to business facts . Proceedings of the 23rd Pacific Asia Conference on Information Systems. Association for Information Systems. Retrieved from http://www.pacis2019.org/wd/Submissions/PACIS 2019 _paper_146.pdf. S. Alsheibani, C. Messom, and Y. Cheung. 2019. Towards an artificial intelligence maturity model: From science fiction to business facts. Proceedings of the 23rd Pacific Asia Conference on Information Systems. Association for Information Systems. Retrieved from http://www.pacis2019.org/wd/Submissions/PACIS2019_paper_146.pdf.
- S. Amershi , A. Begel , C. Bird , R. DeLine , H. Gall , E. Kamar , N. Nagappan , B. Nushi , and T. Zimmermann . 2019a. Software engineering for machine learning: A case study . In Proceedings of the IEEE/ACM 41st International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP’19) . IEEE, 291--300. S. Amershi, A. Begel, C. Bird, R. DeLine, H. Gall, E. Kamar, N. Nagappan, B. Nushi, and T. Zimmermann. 2019a. Software engineering for machine learning: A case study. In Proceedings of the IEEE/ACM 41st International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP’19). IEEE, 291--300.
- S. Amershi , D. Weld , M. Vorvoreanu , A. Fourney , B. Nushi , P. Collisson , and E. Horvitz . 2019b. Guidelines for human-AI interaction . In Proceedings of the CHI Conference on Human Factors in Computing Systems. ACM, 1--13 . S. Amershi, D. Weld, M. Vorvoreanu, A. Fourney, B. Nushi, P. Collisson, and E. Horvitz. 2019b. Guidelines for human-AI interaction. In Proceedings of the CHI Conference on Human Factors in Computing Systems. ACM, 1--13.
- O. Biran and C. Cotton . 2017. Explanation and justification in machine learning: A survey . In Proceedings of the International Joint Conference on Artificial Inteeligence Workshop on Explainable AI (XAI’17) . O. Biran and C. Cotton. 2017. Explanation and justification in machine learning: A survey. In Proceedings of the International Joint Conference on Artificial Inteeligence Workshop on Explainable AI (XAI’17).
- E. Breck , N. Polyzotis , S. Roy , S. E. Whang , and M. Zinkevich . 2019. Data validation for machine learning . In Proceedings of the Conference on Systems and Machine Learning (SysML’19) . Retrieved from https://www.sysml.cc/doc/ 2019 /167.pdf. E. Breck, N. Polyzotis, S. Roy, S. E. Whang, and M. Zinkevich. 2019. Data validation for machine learning. In Proceedings of the Conference on Systems and Machine Learning (SysML’19). Retrieved from https://www.sysml.cc/doc/2019/167.pdf.
- J. Buolamwini and T. Gebru . 2018. Gender shades: Intersectional accuracy disparities in commercial gender classification . Proc. Mach. Learn. Res. 81 , ( 2018 ), 77--91. J. Buolamwini and T. Gebru. 2018. Gender shades: Intersectional accuracy disparities in commercial gender classification. Proc. Mach. Learn. Res. 81, (2018), 77--91.
- Calo Ryan . 2016. Robots in American law . University of Washington School of Law Research Paper. Retrieved from https ://papers.ssrn.com/sol3/papers.cfm?abstract_id=2737598. Calo Ryan. 2016. Robots in American law. University of Washington School of Law Research Paper. Retrieved from https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2737598.
- N. Campbell . 2007 . The evolution of flight data analysis . Proceedings of Australian Society of Air Safety Investigators. Retrieved from https://asasi.org/papers/2007/The_Evolution_of_Flight_Data_Analysis_Neil_Campbell.pdf. N. Campbell. 2007. The evolution of flight data analysis. Proceedings of Australian Society of Air Safety Investigators. Retrieved from https://asasi.org/papers/2007/The_Evolution_of_Flight_Data_Analysis_Neil_Campbell.pdf.
- H. F. Cheng , R. Wang , Z. Zhang , F. O'Connell , T. Gray , F. M. Harper , and H. Zhu . 2019. Explaining decision-making algorithms through UI: Strategies to help non-expert stakeholders . In Proceedings of the CHI Conference on Human Factors in Computing Systems ACM, 1--12 . H. F. Cheng, R. Wang, Z. Zhang, F. O'Connell, T. Gray, F. M. Harper, and H. Zhu. 2019. Explaining decision-making algorithms through UI: Strategies to help non-expert stakeholders. In Proceedings of the CHI Conference on Human Factors in Computing Systems ACM, 1--12.
- 10.5555/16244.16245 · doi:10.5555/16244.16245
- J. Couzin-Frankel . 2019. Medicine contends with how to use artificial intelligence. Science 354, 6446 ( 2019 ), 1119--1120. J. Couzin-Frankel. 2019. Medicine contends with how to use artificial intelligence. Science 354, 6446 (2019), 1119--1120.
- P. R. Daugherty and H. J. Wilson . 2018 . Human + Machine: Reimagining Work in the Age of AI . Harvard Business Press . P. R. Daugherty and H. J. Wilson. 2018. Human+ Machine: Reimagining Work in the Age of AI. Harvard Business Press.
- S. M. Erickson , J. Wolcott , J. M. Corrigan , and P. Aspden (Eds.). 2004 . Patient Safety: Achieving a New Standard for Care . National Academies Press , Washington, DC . S. M. Erickson, J. Wolcott, J. M. Corrigan, and P. Aspden (Eds.). 2004. Patient Safety: Achieving a New Standard for Care. National Academies Press, Washington, DC.
- P. Fraser , J. Moultrie , and M. Gregory . 2002. The use of maturity models/grids as a tool in assessing product development capability . In Proceedings of the IEEE International Engineering Management Conference. IEEE, 244--249 . P. Fraser, J. Moultrie, and M. Gregory. 2002. The use of maturity models/grids as a tool in assessing product development capability. In Proceedings of the IEEE International Engineering Management Conference. IEEE, 244--249.
- D. R. Grossi . 1999 . Aviation Recorder Overview . In Proceedings of the International Symposium on Transportation Recorders. 153--164 . D. R. Grossi. 1999. Aviation Recorder Overview. In Proceedings of the International Symposium on Transportation Recorders. 153--164.
- F. Hohman , A. Head , R. Caruana , R. DeLine , and S. M. Drucker . 2019. Gamut: A design probe to understand how data scientists understand machine learning models . In Proceedings of the CHI Conference on Human Factors in Computing Systems. ACM, 1--13 . F. Hohman, A. Head, R. Caruana, R. DeLine, and S. M. Drucker. 2019. Gamut: A design probe to understand how data scientists understand machine learning models. In Proceedings of the CHI Conference on Human Factors in Computing Systems. ACM, 1--13.
- A. Hopkins . 1999 . The limits of normal accident theory . Safety Sci. 32 , 2 (1999), 93 -- 102 . A. Hopkins. 1999. The limits of normal accident theory. Safety Sci. 32, 2 (1999), 93--102.
- R. Hull , B. Kumar , D. Lieuwen , P. F. Patel-Schneider , A. Sahuguet , S. Varadarajan , and A. Vyas . 2003. Everything personal, not just business: Improving user experience through rule-based service customization . In Proceedings of the International Conference on Service-Oriented Computing. Springer , Berlin, 149--164. R. Hull, B. Kumar, D. Lieuwen, P. F. Patel-Schneider, A. Sahuguet, S. Varadarajan, and A. Vyas. 2003. Everything personal, not just business: Improving user experience through rule-based service customization. In Proceedings of the International Conference on Service-Oriented Computing. Springer, Berlin, 149--164.
- IEEE Global Initiative on Ethics of Autonomous and Intelligent Systems ( 2019 ). Ethically Aligned Design: A Vision for Prioritizing Human Well-being with Autonomous and Intelligent Systems , 1 st ed. IEEE. Retrieved from https://standards.ieee.org/content/ieee-standards/en/industry-connections/ec/autonomous-systems.html https://ethicsinaction.ieee.org/. IEEE Global Initiative on Ethics of Autonomous and Intelligent Systems (2019). Ethically Aligned Design: A Vision for Prioritizing Human Well-being with Autonomous and Intelligent Systems, 1st ed. IEEE. Retrieved from https://standards.ieee.org/content/ieee-standards/en/industry-connections/ec/autonomous-systems.html https://ethicsinaction.ieee.org/.
- X. Liang , S. Shetty , D. Tosh , C. Kamhoua , K. Kwiat , and L. Njilla . 2017. Provchain: A blockchain-based data provenance architecture in cloud environment with enhanced privacy and availability . In Proceedings of the 17th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGRID’17) , 468--477. X. Liang, S. Shetty, D. Tosh, C. Kamhoua, K. Kwiat, and L. Njilla. 2017. Provchain: A blockchain-based data provenance architecture in cloud environment with enhanced privacy and availability. In Proceedings of the 17th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGRID’17), 468--477.
- Q. V. Liao , D. Gruen , and S. Miller . 2020. Questioning the AI: Informing design practices for explainable ai user experiences . Proceedings of the ACM CHI Conference on Human Factors in Computing Systems. ACM , New York, 1--15. Q. V. Liao, D. Gruen, and S. Miller. 2020. Questioning the AI: Informing design practices for explainable ai user experiences. Proceedings of the ACM CHI Conference on Human Factors in Computing Systems. ACM, New York, 1--15.
- T. Mai , R. Khanna , J. Dodge , J. Irvine , K. H. Lam , Z. Lin , N. Kiddle , E. Newman , S. Raja , C. Matthews , C. Perdriau , M. Burnett , and A. Fern . 2020. Keeping it “organized and logical”: After-action review for AI (AAR/AI) . In Proceedings of the 25th International Conference on Intelligent User Interfaces. ACM, 465--476 . T. Mai, R. Khanna, J. Dodge, J. Irvine, K. H. Lam, Z. Lin, N. Kiddle, E. Newman, S. Raja, C. Matthews, C. Perdriau, M. Burnett, and A. Fern. 2020. Keeping it “organized and logical”: After-action review for AI (AAR/AI). In Proceedings of the 25th International Conference on Intelligent User Interfaces. ACM, 465--476.
- A. Mitrevski , S. Thoduka , A. O. Sáinz , M. Schöbel , P. Nagel , P. G. Plöger , and E. Prassler . 2018. Deploying robots in everyday environments: Toward dependable and practical robotic systems . In Proceedings of the 29th International Workshop Principles of Diagnosis (DX’18) . Retrieved from http://www.ropod.org/downloads/dx18.pdf. A. Mitrevski, S. Thoduka, A. O. Sáinz, M. Schöbel, P. Nagel, P. G. Plöger, and E. Prassler. 2018. Deploying robots in everyday environments: Toward dependable and practical robotic systems. In Proceedings of the 29th International Workshop Principles of Diagnosis (DX’18). Retrieved from http://www.ropod.org/downloads/dx18.pdf.
- M. R. Morris . 2020. AI and accessibility: A discussion of ethical considerations. Commun. ACM 63, 6 ( 2016 ), ACM 35--37. 10.1145/3356727 M. R. Morris. 2020. AI and accessibility: A discussion of ethical considerations. Commun. ACM 63, 6 (2016), ACM 35--37. 10.1145/3356727 · DOI · Paper page
- Nadella Satya ( 2016 ). The partnership of the future , Slate. Retrieved from https://slate.com/technology/2016/06/microsoft-ceo-satya-nadella-humans-and-a-i-can-work-together-to-solve-societys-challenges.html. Nadella Satya (2016). The partnership of the future, Slate. Retrieved from https://slate.com/technology/2016/06/microsoft-ceo-satya-nadella-humans-and-a-i-can-work-together-to-solve-societys-challenges.html.
- F. Rosenberg and S. Dustdar . 2005. Design and implementation of a service-oriented business rules broker . In Proceedings of the 7th IEEE International Conference on E-Commerce Technology Workshops. IEEE, 55--63 . F. Rosenberg and S. Dustdar. 2005. Design and implementation of a service-oriented business rules broker. In Proceedings of the 7th IEEE International Conference on E-Commerce Technology Workshops. IEEE, 55--63.
- M. Rotenberg . 2020. The AI Policy Sourcebook 2020 . Electronic Privacy Information Center, Washington , DC. M. Rotenberg. 2020. The AI Policy Sourcebook 2020. Electronic Privacy Information Center, Washington, DC.
- H. Sharp , J. Preece , and Y. Rogers . 2019 . Interaction Design: Beyond Human-Computer Interaction , 5 th ed. Wiley Publishers . H. Sharp, J. Preece, and Y. Rogers. 2019. Interaction Design: Beyond Human-Computer Interaction, 5th ed. Wiley Publishers.
- B. Shneiderman , C. Plaisant , M. Cohen , S. Jacobs , and N. Elmqvist . 2016 . Designing the User Interface: Strategies for Effective Human-Computer Interaction , 6 th ed. Pearson . B. Shneiderman, C. Plaisant, M. Cohen, S. Jacobs, and N. Elmqvist. 2016. Designing the User Interface: Strategies for Effective Human-Computer Interaction, 6th ed. Pearson.
- 10.1109/MS.2010.96 · doi:10.1109/ms.2010.96
- A. F. Winfield and M. Jirotka . 2017. The case for an ethical black box . In Proceedings of the Annual Conference Towards Autonomous Robotic Systems, Springer 262--273 . Retrieved from https://link.springer.com/chapter/10.1007/978-3-319-64107-2_21. A. F. Winfield and M. Jirotka. 2017. The case for an ethical black box. In Proceedings of the Annual Conference Towards Autonomous Robotic Systems, Springer 262--273. Retrieved from https://link.springer.com/chapter/10.1007/978-3-319-64107-2_21. · DOI · Paper page
Cited by
Aldoseri, A., Al-Khalifa, K. N., & Hamouda, A. M. (2023) · Applied Sciences · doi:10.3390/app13127082 · Cited by 716
“Data Poisoning: Data poisoning attacks involve tampering with training data to degrade the performance of an AI model [106]. replicate intent”
Trustworthy AI: From Principles to Practices
Li, B., Qi, P., Liu, B., Di, S., Liu, J., Pei, J., Yi, J., & Zhou, B. (2023) · ACM Computing Surveys · doi:10.1145/3555803 · Cited by 612
Responsible Artificial Intelligence Governance: A Review and Research Framework
Papagiannidis, E., Mikalef, P., & Conboy, K. (2025) · The Journal of Strategic Information Systems · doi:10.1016/j.jsis.2024.101885 · Cited by 565
Six Human-Centered Artificial Intelligence Grand Challenges
Ozmen Garibay, O., Winslow, B., Andolina, S., Antona, M., Bodenschatz, A., Coursaris, C., Falco, G., Fiore, S. M., Garibay, I., Grieman, K., et al. (2023) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2022.2153320 · Cited by 501
“…to date, human-centered AI (HCAI) seeks to re-position humans at the center of the AI lifecycle (Bond et al., 2019; Riedl, 2019; Xu, 2019) and improve human performance in ways that are reliable, safe, and trustworthy by augmenting rather than replacing human capabilities (Shneiderman, 2020a).”
Self-citationde Hond, A. A. H., Leeuwenberg, A. M., Hooft, L., Kant, I. M. J., Nijman, S. W. J., van Os, H. J. A., Aardoom, J. J., Debray, T. P. A., Schuit, E., van Smeden, M., et al. (2022) · npj Digital Medicine · doi:10.1038/s41746-021-00549-7 · Cited by 499
Metadata
- Authors
- Shneiderman, B.
- Affiliations
- University of Maryland, College Park (US)
- Corresponding author
- Shneiderman, Ben
- ORCID
- Shneiderman, Ben
- Venue
- ACM Transactions on Interactive Intelligent Systems
- Published
- 2020-10-16
- Volume
- 10
- Issue
- 4
- Pages
- 1-31
- ISSN
- 2160-6455 (ISSN-L)
- 2160-6463
- OpenAlex
- W3094233077
- Cited by
- 768
- References
- 89
- Citations by year
- 2026: 185 · 2025: 208 · 2024: 153 · 2023: 105 · 2022: 78 · 2021: 38 · 2020: 1
- FWCI
- 75.54
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- ACM Transactions on Interactive Intelligent Systems (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Engineering › Safety, Risk, Reliability and Quality › Safety Systems Engineering in Autonomy
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Keywords
- human-centered AI, AI governance, software engineering practices, safety culture, audit trail, verification and validation, bias testing, failure reporting, near misses, AI regulation, organizational leadership
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.