Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 88 of the 89 references it cites were found in a registry or an index.

Bridging the Gap Between Ethics and Practice: Guidelines for Reliable, Safe, and Trustworthy Human-centered AI Systems

Shneiderman, Ben · 2020 · ACM Transactions on Interactive Intelligent Systems

Abstract

This article attempts to bridge the gap between widely discussed ethical principles of Human-centered AI (HCAI) and practical steps for effective governance. Since HCAI systems are developed and implemented in multiple organizational structures, I propose 15 recommendations at three levels of governance: team, organization, and industry. The recommendations are intended to increase the reliability, safety, and trustworthiness of HCAI systems: (1) reliable systems based on sound software engineering practices, (2) safety culture through business management strategies, and (3) trustworthy certification by independent oversight. Software engineering practices within teams include audit trails to enable analysis of failures, software engineering workflows, verification and validation testing, bias testing to enhance fairness, and explainable user interfaces. The safety culture within organizations comes from management strategies that include leadership commitment to safety, hiring and training oriented to safety, extensive reporting of failures and near misses, internal review boards for problems and future plans, and alignment with industry standard practices. The trustworthiness certification comes from industry-wide efforts that include government interventions and regulation, accounting firms conducting external audits, insurance companies compensating for failures, non-governmental and civil society organizations advancing design principles, and professional organizations and research institutes developing standards, policies, and novel ideas. The larger goal of effective governance is to limit the dangers and increase the benefits of HCAI to individuals, organizations, and society.

Overview

Integrity signals

Registry

Crossref · 10.1145/3419764

Cites

88 of 88 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

5 more claims about this paper

Create a free account to see every claim other papers make about this one.

See all claims — free account

Citations

Cites

Show all 89 references

Research works printed but not linked: 32

  1. A. Abdul , J. Vermeulen , D. Wang , B. Y. Lim , and M. Kankanhalli . 2018. Trends and trajectories for explainable, accountable and intelligible systems: An HCI research agenda . Proceedings of the CHI Conference on Human Factors in Computing Systems. ACM, 1--18 . A. Abdul, J. Vermeulen, D. Wang, B. Y. Lim, and M. Kankanhalli. 2018. Trends and trajectories for explainable, accountable and intelligible systems: An HCI research agenda. Proceedings of the CHI Conference on Human Factors in Computing Systems. ACM, 1--18.
  2. S. Alsheibani , C. Messom , and Y. Cheung . 2019. Towards an artificial intelligence maturity model: From science fiction to business facts . Proceedings of the 23rd Pacific Asia Conference on Information Systems. Association for Information Systems. Retrieved from http://www.pacis2019.org/wd/Submissions/PACIS 2019 _paper_146.pdf. S. Alsheibani, C. Messom, and Y. Cheung. 2019. Towards an artificial intelligence maturity model: From science fiction to business facts. Proceedings of the 23rd Pacific Asia Conference on Information Systems. Association for Information Systems. Retrieved from http://www.pacis2019.org/wd/Submissions/PACIS2019_paper_146.pdf.
  3. S. Amershi , A. Begel , C. Bird , R. DeLine , H. Gall , E. Kamar , N. Nagappan , B. Nushi , and T. Zimmermann . 2019a. Software engineering for machine learning: A case study . In Proceedings of the IEEE/ACM 41st International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP’19) . IEEE, 291--300. S. Amershi, A. Begel, C. Bird, R. DeLine, H. Gall, E. Kamar, N. Nagappan, B. Nushi, and T. Zimmermann. 2019a. Software engineering for machine learning: A case study. In Proceedings of the IEEE/ACM 41st International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP’19). IEEE, 291--300.
  4. S. Amershi , D. Weld , M. Vorvoreanu , A. Fourney , B. Nushi , P. Collisson , and E. Horvitz . 2019b. Guidelines for human-AI interaction . In Proceedings of the CHI Conference on Human Factors in Computing Systems. ACM, 1--13 . S. Amershi, D. Weld, M. Vorvoreanu, A. Fourney, B. Nushi, P. Collisson, and E. Horvitz. 2019b. Guidelines for human-AI interaction. In Proceedings of the CHI Conference on Human Factors in Computing Systems. ACM, 1--13.
  5. O. Biran and C. Cotton . 2017. Explanation and justification in machine learning: A survey . In Proceedings of the International Joint Conference on Artificial Inteeligence Workshop on Explainable AI (XAI’17) . O. Biran and C. Cotton. 2017. Explanation and justification in machine learning: A survey. In Proceedings of the International Joint Conference on Artificial Inteeligence Workshop on Explainable AI (XAI’17).
  6. E. Breck , N. Polyzotis , S. Roy , S. E. Whang , and M. Zinkevich . 2019. Data validation for machine learning . In Proceedings of the Conference on Systems and Machine Learning (SysML’19) . Retrieved from https://www.sysml.cc/doc/ 2019 /167.pdf. E. Breck, N. Polyzotis, S. Roy, S. E. Whang, and M. Zinkevich. 2019. Data validation for machine learning. In Proceedings of the Conference on Systems and Machine Learning (SysML’19). Retrieved from https://www.sysml.cc/doc/2019/167.pdf.
  7. J. Buolamwini and T. Gebru . 2018. Gender shades: Intersectional accuracy disparities in commercial gender classification . Proc. Mach. Learn. Res. 81 , ( 2018 ), 77--91. J. Buolamwini and T. Gebru. 2018. Gender shades: Intersectional accuracy disparities in commercial gender classification. Proc. Mach. Learn. Res. 81, (2018), 77--91.
  8. Calo Ryan . 2016. Robots in American law . University of Washington School of Law Research Paper. Retrieved from https ://papers.ssrn.com/sol3/papers.cfm?abstract_id=2737598. Calo Ryan. 2016. Robots in American law. University of Washington School of Law Research Paper. Retrieved from https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2737598.
  9. N. Campbell . 2007 . The evolution of flight data analysis . Proceedings of Australian Society of Air Safety Investigators. Retrieved from https://asasi.org/papers/2007/The_Evolution_of_Flight_Data_Analysis_Neil_Campbell.pdf. N. Campbell. 2007. The evolution of flight data analysis. Proceedings of Australian Society of Air Safety Investigators. Retrieved from https://asasi.org/papers/2007/The_Evolution_of_Flight_Data_Analysis_Neil_Campbell.pdf.
  10. H. F. Cheng , R. Wang , Z. Zhang , F. O'Connell , T. Gray , F. M. Harper , and H. Zhu . 2019. Explaining decision-making algorithms through UI: Strategies to help non-expert stakeholders . In Proceedings of the CHI Conference on Human Factors in Computing Systems ACM, 1--12 . H. F. Cheng, R. Wang, Z. Zhang, F. O'Connell, T. Gray, F. M. Harper, and H. Zhu. 2019. Explaining decision-making algorithms through UI: Strategies to help non-expert stakeholders. In Proceedings of the CHI Conference on Human Factors in Computing Systems ACM, 1--12.
  11. 10.5555/16244.16245 · doi:10.5555/16244.16245
  12. J. Couzin-Frankel . 2019. Medicine contends with how to use artificial intelligence. Science 354, 6446 ( 2019 ), 1119--1120. J. Couzin-Frankel. 2019. Medicine contends with how to use artificial intelligence. Science 354, 6446 (2019), 1119--1120.
  13. P. R. Daugherty and H. J. Wilson . 2018 . Human + Machine: Reimagining Work in the Age of AI . Harvard Business Press . P. R. Daugherty and H. J. Wilson. 2018. Human+ Machine: Reimagining Work in the Age of AI. Harvard Business Press.
  14. S. M. Erickson , J. Wolcott , J. M. Corrigan , and P. Aspden (Eds.). 2004 . Patient Safety: Achieving a New Standard for Care . National Academies Press , Washington, DC . S. M. Erickson, J. Wolcott, J. M. Corrigan, and P. Aspden (Eds.). 2004. Patient Safety: Achieving a New Standard for Care. National Academies Press, Washington, DC.
  15. P. Fraser , J. Moultrie , and M. Gregory . 2002. The use of maturity models/grids as a tool in assessing product development capability . In Proceedings of the IEEE International Engineering Management Conference. IEEE, 244--249 . P. Fraser, J. Moultrie, and M. Gregory. 2002. The use of maturity models/grids as a tool in assessing product development capability. In Proceedings of the IEEE International Engineering Management Conference. IEEE, 244--249.
  16. D. R. Grossi . 1999 . Aviation Recorder Overview . In Proceedings of the International Symposium on Transportation Recorders. 153--164 . D. R. Grossi. 1999. Aviation Recorder Overview. In Proceedings of the International Symposium on Transportation Recorders. 153--164.
  17. F. Hohman , A. Head , R. Caruana , R. DeLine , and S. M. Drucker . 2019. Gamut: A design probe to understand how data scientists understand machine learning models . In Proceedings of the CHI Conference on Human Factors in Computing Systems. ACM, 1--13 . F. Hohman, A. Head, R. Caruana, R. DeLine, and S. M. Drucker. 2019. Gamut: A design probe to understand how data scientists understand machine learning models. In Proceedings of the CHI Conference on Human Factors in Computing Systems. ACM, 1--13.
  18. A. Hopkins . 1999 . The limits of normal accident theory . Safety Sci. 32 , 2 (1999), 93 -- 102 . A. Hopkins. 1999. The limits of normal accident theory. Safety Sci. 32, 2 (1999), 93--102.
  19. R. Hull , B. Kumar , D. Lieuwen , P. F. Patel-Schneider , A. Sahuguet , S. Varadarajan , and A. Vyas . 2003. Everything personal, not just business: Improving user experience through rule-based service customization . In Proceedings of the International Conference on Service-Oriented Computing. Springer , Berlin, 149--164. R. Hull, B. Kumar, D. Lieuwen, P. F. Patel-Schneider, A. Sahuguet, S. Varadarajan, and A. Vyas. 2003. Everything personal, not just business: Improving user experience through rule-based service customization. In Proceedings of the International Conference on Service-Oriented Computing. Springer, Berlin, 149--164.
  20. IEEE Global Initiative on Ethics of Autonomous and Intelligent Systems ( 2019 ). Ethically Aligned Design: A Vision for Prioritizing Human Well-being with Autonomous and Intelligent Systems , 1 st ed. IEEE. Retrieved from https://standards.ieee.org/content/ieee-standards/en/industry-connections/ec/autonomous-systems.html https://ethicsinaction.ieee.org/. IEEE Global Initiative on Ethics of Autonomous and Intelligent Systems (2019). Ethically Aligned Design: A Vision for Prioritizing Human Well-being with Autonomous and Intelligent Systems, 1st ed. IEEE. Retrieved from https://standards.ieee.org/content/ieee-standards/en/industry-connections/ec/autonomous-systems.html https://ethicsinaction.ieee.org/.
  21. X. Liang , S. Shetty , D. Tosh , C. Kamhoua , K. Kwiat , and L. Njilla . 2017. Provchain: A blockchain-based data provenance architecture in cloud environment with enhanced privacy and availability . In Proceedings of the 17th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGRID’17) , 468--477. X. Liang, S. Shetty, D. Tosh, C. Kamhoua, K. Kwiat, and L. Njilla. 2017. Provchain: A blockchain-based data provenance architecture in cloud environment with enhanced privacy and availability. In Proceedings of the 17th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGRID’17), 468--477.
  22. Q. V. Liao , D. Gruen , and S. Miller . 2020. Questioning the AI: Informing design practices for explainable ai user experiences . Proceedings of the ACM CHI Conference on Human Factors in Computing Systems. ACM , New York, 1--15. Q. V. Liao, D. Gruen, and S. Miller. 2020. Questioning the AI: Informing design practices for explainable ai user experiences. Proceedings of the ACM CHI Conference on Human Factors in Computing Systems. ACM, New York, 1--15.
  23. T. Mai , R. Khanna , J. Dodge , J. Irvine , K. H. Lam , Z. Lin , N. Kiddle , E. Newman , S. Raja , C. Matthews , C. Perdriau , M. Burnett , and A. Fern . 2020. Keeping it “organized and logical”: After-action review for AI (AAR/AI) . In Proceedings of the 25th International Conference on Intelligent User Interfaces. ACM, 465--476 . T. Mai, R. Khanna, J. Dodge, J. Irvine, K. H. Lam, Z. Lin, N. Kiddle, E. Newman, S. Raja, C. Matthews, C. Perdriau, M. Burnett, and A. Fern. 2020. Keeping it “organized and logical”: After-action review for AI (AAR/AI). In Proceedings of the 25th International Conference on Intelligent User Interfaces. ACM, 465--476.
  24. A. Mitrevski , S. Thoduka , A. O. Sáinz , M. Schöbel , P. Nagel , P. G. Plöger , and E. Prassler . 2018. Deploying robots in everyday environments: Toward dependable and practical robotic systems . In Proceedings of the 29th International Workshop Principles of Diagnosis (DX’18) . Retrieved from http://www.ropod.org/downloads/dx18.pdf. A. Mitrevski, S. Thoduka, A. O. Sáinz, M. Schöbel, P. Nagel, P. G. Plöger, and E. Prassler. 2018. Deploying robots in everyday environments: Toward dependable and practical robotic systems. In Proceedings of the 29th International Workshop Principles of Diagnosis (DX’18). Retrieved from http://www.ropod.org/downloads/dx18.pdf.
  25. M. R. Morris . 2020. AI and accessibility: A discussion of ethical considerations. Commun. ACM 63, 6 ( 2016 ), ACM 35--37. 10.1145/3356727 M. R. Morris. 2020. AI and accessibility: A discussion of ethical considerations. Commun. ACM 63, 6 (2016), ACM 35--37. 10.1145/3356727 · DOI · Paper page
  26. Nadella Satya ( 2016 ). The partnership of the future , Slate. Retrieved from https://slate.com/technology/2016/06/microsoft-ceo-satya-nadella-humans-and-a-i-can-work-together-to-solve-societys-challenges.html. Nadella Satya (2016). The partnership of the future, Slate. Retrieved from https://slate.com/technology/2016/06/microsoft-ceo-satya-nadella-humans-and-a-i-can-work-together-to-solve-societys-challenges.html.
  27. F. Rosenberg and S. Dustdar . 2005. Design and implementation of a service-oriented business rules broker . In Proceedings of the 7th IEEE International Conference on E-Commerce Technology Workshops. IEEE, 55--63 . F. Rosenberg and S. Dustdar. 2005. Design and implementation of a service-oriented business rules broker. In Proceedings of the 7th IEEE International Conference on E-Commerce Technology Workshops. IEEE, 55--63.
  28. M. Rotenberg . 2020. The AI Policy Sourcebook 2020 . Electronic Privacy Information Center, Washington , DC. M. Rotenberg. 2020. The AI Policy Sourcebook 2020. Electronic Privacy Information Center, Washington, DC.
  29. H. Sharp , J. Preece , and Y. Rogers . 2019 . Interaction Design: Beyond Human-Computer Interaction , 5 th ed. Wiley Publishers . H. Sharp, J. Preece, and Y. Rogers. 2019. Interaction Design: Beyond Human-Computer Interaction, 5th ed. Wiley Publishers.
  30. B. Shneiderman , C. Plaisant , M. Cohen , S. Jacobs , and N. Elmqvist . 2016 . Designing the User Interface: Strategies for Effective Human-Computer Interaction , 6 th ed. Pearson . B. Shneiderman, C. Plaisant, M. Cohen, S. Jacobs, and N. Elmqvist. 2016. Designing the User Interface: Strategies for Effective Human-Computer Interaction, 6th ed. Pearson.
  31. 10.1109/MS.2010.96 · doi:10.1109/ms.2010.96
  32. A. F. Winfield and M. Jirotka . 2017. The case for an ethical black box . In Proceedings of the Annual Conference Towards Autonomous Robotic Systems, Springer 262--273 . Retrieved from https://link.springer.com/chapter/10.1007/978-3-319-64107-2_21. A. F. Winfield and M. Jirotka. 2017. The case for an ethical black box. In Proceedings of the Annual Conference Towards Autonomous Robotic Systems, Springer 262--273. Retrieved from https://link.springer.com/chapter/10.1007/978-3-319-64107-2_21. · DOI · Paper page

Cited by

See all 771

Metadata

Authors
Shneiderman, B.
Affiliations
University of Maryland, College Park (US)
Corresponding author
Shneiderman, Ben
Venue
ACM Transactions on Interactive Intelligent Systems
Published
2020-10-16
Volume
10
Issue
4
Pages
1-31
ISSN
  • 2160-6455 (ISSN-L)
  • 2160-6463
OpenAlex
W3094233077
Cited by
768
References
89
Citations by year
2026: 185 · 2025: 208 · 2024: 153 · 2023: 105 · 2022: 78 · 2021: 38 · 2020: 1
FWCI
75.54
Type
journal article
Language
en
Open access
closed
Locations
ACM Transactions on Interactive Intelligent Systems (publishedVersion)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
  • Physical Sciences › Engineering › Safety, Risk, Reliability and Quality › Safety Systems Engineering in Autonomy
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
Keywords
human-centered AI, AI governance, software engineering practices, safety culture, audit trail, verification and validation, bias testing, failure reporting, near misses, AI regulation, organizational leadership

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Bridging the Gap Between Ethics and Practice: Guidelines for Reliable, Safe, and Trustworthy Human-centered AI Systems. https://app.cochat.ai/papers/10.1145/3419764
The paper
Shneiderman, B. (2020). Bridging the Gap Between Ethics and Practice: Guidelines for Reliable, Safe, and Trustworthy Human-centered AI Systems. ACM Transactions on Interactive Intelligent Systems. https://doi.org/10.1145/3419764

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.