Bridging the Gap Between Ethics and Practice: Guidelines for Reliable, Safe, and Trustworthy Human-centered AI Systems
The DOI matches its Crossref record; 91 of the 91 references it cites were found in a registry or an index.
Abstract
This article attempts to bridge the gap between widely discussed ethical principles of Human-centered AI (HCAI) and practical steps for effective governance. Since HCAI systems are developed and implemented in multiple organizational structures, I propose 15 recommendations at three levels of governance: team, organization, and industry. The recommendations are intended to increase the reliability, safety, and trustworthiness of HCAI systems: (1) reliable systems based on sound software engineering practices, (2) safety culture through business management strategies, and (3) trustworthy certification by independent oversight. Software engineering practices within teams include audit trails to enable analysis of failures, software engineering workflows, verification and validation testing, bias testing to enhance fairness, and explainable user interfaces. The safety culture within organizations comes from management strategies that include leadership commitment to safety, hiring and training oriented to safety, extensive reporting of failures and near misses, internal review boards for problems and future plans, and alignment with industry standard practices. The trustworthiness certification comes from industry-wide efforts that include government interventions and regulation, accounting firms conducting external audits, insurance companies compensating for failures, non-governmental and civil society organizations advancing design principles, and professional organizations and research institutes developing standards, policies, and novel ideas. The larger goal of effective governance is to limit the dangers and increase the benefits of HCAI to individuals, organizations, and society.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“These challenges raise ethical questions around accountability-who is responsible when AI-generated code causes harm?-and the need for governance mechanisms that ensure human oversight and traceability [34].”
From Redefining the Programmer: Human-AI Collaboration, LLMs, and Security in Modern Software Engineering (2025) · cites this paper as [34]
In the citing paper
“In short, currently available knowledge related to responsible and ethical AI is often of a high level of abstraction.”
From Contestable AI by Design: Towards a Framework (2022) · cites this paper as Shneiderman, 2020
In the citing paper
“while AI tools are essential in detecting harmful content, their effectiveness remains highly variable, requiring structured evaluation frameworks to assess their reliability and ethical implications (Shneiderman, 2020).”
From Virtual Reality Risks for Children in the Metaverse: An Examination through Criminological Perspectives (2026) · cites this paper as Shneiderman, 2020
In the citing paper
“When these mechanisms are aligned with efforts needed to establish social, organizational, and regulatory ecosystems for the assurance of trustworthy AI [9, 30, 54], they are likely to satisfy the calibration condition.”
From Designing for Responsible Trust in AI Systems: A Communication Perspective (2022) · cites this paper as [54]
In the citing paper
“Finally, as monitoring and auditing of AIPMs in practice are widely recommended (covered in more detail in phase 6), the developed software should facilitate this10,22,26,32,58,62,104.”
From Guidelines and quality criteria for artificial intelligence-based prediction models in healthcare: a scoping review (2022) · cites this paper as [58]
5 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Research in Organizational Behavior.
Grimes, A. J., Cummings, L. L., & Staw, B. M. (1986) · Administrative Science Quarterly · doi:10.2307/2392975 · Cited by 10,899
Rudin, C. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0048-x · Cited by 10,561 · cited 1×
“Du et al. [28] make useful distinctions between intrinsically understandable machine learning models, such as decision trees or rule-based models [67, 26, 97], and the more common approach of post hoc explanations.”
BackgroundExplanation in artificial intelligence: Insights from the social sciences
Miller, T. (2019) · Artificial Intelligence · doi:10.1016/j.artint.2018.07.007 · Cited by 5,302
Towards A Rigorous Science of Interpretable Machine Learning
Doshi-Velez, F., & Kim, B. (2017) · arXiv · doi:10.48550/arxiv.1702.08608 · Cited by 3,121 · cited 1×
“Du et al. [28] make useful distinctions between intrinsically understandable machine learning models, such as decision trees or rule-based models [67, 26, 97], and the more common approach of post hoc explanations.”
BackgroundResilience Engineering: Concepts and Precepts
Wears, R. L. (2006) · Quality and Safety in Health Care · doi:10.1136/qshc.2006.018390 · Cited by 2,900
Raji, I. D., Smart, A., White, R. N., Mitchell, M., Gebru, T., Hutchinson, B., Smith-Loud, J., Theron, D., & Barnes, P. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372873 · Cited by 2,245 · cited 2×
“However, they can produce important internal efforts such as Google’s internal review and algorithmic auditing framework [92] (see Section 3.”
BackgroundEuropean Union Regulations on Algorithmic Decision Making and a “Right to Explanation”
Goodman, B., & Flaxman, S. (2017) · AI Magazine · doi:10.1609/aimag.v38i3.2741 · Cited by 2,131 · cited 1×
“Furthermore, explanations have become a legal requirement in many countries based on the European Union’s General Data Protection Regulation (GDPR) requirement of a “right to explanation” [41, 115].”
BackgroundThe nature of safety culture: a review of theory and research
Guldenmund, F. W. (2000) · Safety Science · doi:10.1016/s0925-7535(00)00014-x · Cited by 1,831 · cited 1×
“A fourth theme described how organizations cultivate safety cultures [43, 8], especially in healthcare [32] by open management strategies.”
BackgroundHuman-Centered Artificial Intelligence: Reliable, Safe & Trustworthy
Shneiderman, B. (2020) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2020.1741118 · Cited by 1,743 · cited 2×
“In contrast, HCAI focuses on amplifying, augmenting, and enhancing human performance [103] in ways that make systems reliable, safe, and trustworthy [102].”
BackgroundSelf-citationRule-Based Expert Systems: The MYCIN Experiments of the Stanford Heuristic Programming Project
(1986) · Telematics and Informatics · doi:10.1016/s0736-5853(86)80073-9 · Cited by 1,701
Research works printed but not linked: 30
- S. Alsheibani , C. Messom , and Y. Cheung . 2019. Towards an artificial intelligence maturity model: From science fiction to business facts . Proceedings of the 23rd Pacific Asia Conference on Information Systems. Association for Information Systems. Retrieved from http://www.pacis2019.org/wd/Submissions/PACIS 2019 _paper_146.pdf. S. Alsheibani, C. Messom, and Y. Cheung. 2019. Towards an artificial intelligence maturity model: From science fiction to business facts. Proceedings of the 23rd Pacific Asia Conference on Information Systems. Association for Information Systems. Retrieved from http://www.pacis2019.org/wd/Submissions/PACIS2019_paper_146.pdf.
- S. Amershi , A. Begel , C. Bird , R. DeLine , H. Gall , E. Kamar , N. Nagappan , B. Nushi , and T. Zimmermann . 2019a. Software engineering for machine learning: A case study . In Proceedings of the IEEE/ACM 41st International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP’19) . IEEE, 291--300. S. Amershi, A. Begel, C. Bird, R. DeLine, H. Gall, E. Kamar, N. Nagappan, B. Nushi, and T. Zimmermann. 2019a. Software engineering for machine learning: A case study. In Proceedings of the IEEE/ACM 41st International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP’19). IEEE, 291--300.
- S. Amershi , D. Weld , M. Vorvoreanu , A. Fourney , B. Nushi , P. Collisson , and E. Horvitz . 2019b. Guidelines for human-AI interaction . In Proceedings of the CHI Conference on Human Factors in Computing Systems. ACM, 1--13 . S. Amershi, D. Weld, M. Vorvoreanu, A. Fourney, B. Nushi, P. Collisson, and E. Horvitz. 2019b. Guidelines for human-AI interaction. In Proceedings of the CHI Conference on Human Factors in Computing Systems. ACM, 1--13.
- O. Biran and C. Cotton . 2017. Explanation and justification in machine learning: A survey . In Proceedings of the International Joint Conference on Artificial Inteeligence Workshop on Explainable AI (XAI’17) . O. Biran and C. Cotton. 2017. Explanation and justification in machine learning: A survey. In Proceedings of the International Joint Conference on Artificial Inteeligence Workshop on Explainable AI (XAI’17).
- E. Breck , N. Polyzotis , S. Roy , S. E. Whang , and M. Zinkevich . 2019. Data validation for machine learning . In Proceedings of the Conference on Systems and Machine Learning (SysML’19) . Retrieved from https://www.sysml.cc/doc/ 2019 /167.pdf. E. Breck, N. Polyzotis, S. Roy, S. E. Whang, and M. Zinkevich. 2019. Data validation for machine learning. In Proceedings of the Conference on Systems and Machine Learning (SysML’19). Retrieved from https://www.sysml.cc/doc/2019/167.pdf.
- J. Buolamwini and T. Gebru . 2018. Gender shades: Intersectional accuracy disparities in commercial gender classification . Proc. Mach. Learn. Res. 81 , ( 2018 ), 77--91. J. Buolamwini and T. Gebru. 2018. Gender shades: Intersectional accuracy disparities in commercial gender classification. Proc. Mach. Learn. Res. 81, (2018), 77--91.
- Calo Ryan . 2016. Robots in American law . University of Washington School of Law Research Paper. Retrieved from https ://papers.ssrn.com/sol3/papers.cfm?abstract_id=2737598. Calo Ryan. 2016. Robots in American law. University of Washington School of Law Research Paper. Retrieved from https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2737598.
- N. Campbell . 2007 . The evolution of flight data analysis . Proceedings of Australian Society of Air Safety Investigators. Retrieved from https://asasi.org/papers/2007/The_Evolution_of_Flight_Data_Analysis_Neil_Campbell.pdf. N. Campbell. 2007. The evolution of flight data analysis. Proceedings of Australian Society of Air Safety Investigators. Retrieved from https://asasi.org/papers/2007/The_Evolution_of_Flight_Data_Analysis_Neil_Campbell.pdf.
- H. F. Cheng , R. Wang , Z. Zhang , F. O'Connell , T. Gray , F. M. Harper , and H. Zhu . 2019. Explaining decision-making algorithms through UI: Strategies to help non-expert stakeholders . In Proceedings of the CHI Conference on Human Factors in Computing Systems ACM, 1--12 . H. F. Cheng, R. Wang, Z. Zhang, F. O'Connell, T. Gray, F. M. Harper, and H. Zhu. 2019. Explaining decision-making algorithms through UI: Strategies to help non-expert stakeholders. In Proceedings of the CHI Conference on Human Factors in Computing Systems ACM, 1--12.
- 10.5555/16244.16245 · doi:10.5555/16244.16245
- J. Couzin-Frankel . 2019. Medicine contends with how to use artificial intelligence. Science 354, 6446 ( 2019 ), 1119--1120. J. Couzin-Frankel. 2019. Medicine contends with how to use artificial intelligence. Science 354, 6446 (2019), 1119--1120.
- P. R. Daugherty and H. J. Wilson . 2018 . Human + Machine: Reimagining Work in the Age of AI . Harvard Business Press . P. R. Daugherty and H. J. Wilson. 2018. Human+ Machine: Reimagining Work in the Age of AI. Harvard Business Press.
- S. M. Erickson , J. Wolcott , J. M. Corrigan , and P. Aspden (Eds.). 2004 . Patient Safety: Achieving a New Standard for Care . National Academies Press , Washington, DC . S. M. Erickson, J. Wolcott, J. M. Corrigan, and P. Aspden (Eds.). 2004. Patient Safety: Achieving a New Standard for Care. National Academies Press, Washington, DC.
- P. Fraser , J. Moultrie , and M. Gregory . 2002. The use of maturity models/grids as a tool in assessing product development capability . In Proceedings of the IEEE International Engineering Management Conference. IEEE, 244--249 . P. Fraser, J. Moultrie, and M. Gregory. 2002. The use of maturity models/grids as a tool in assessing product development capability. In Proceedings of the IEEE International Engineering Management Conference. IEEE, 244--249.
- D. R. Grossi . 1999 . Aviation Recorder Overview . In Proceedings of the International Symposium on Transportation Recorders. 153--164 . D. R. Grossi. 1999. Aviation Recorder Overview. In Proceedings of the International Symposium on Transportation Recorders. 153--164.
- F. Hohman , A. Head , R. Caruana , R. DeLine , and S. M. Drucker . 2019. Gamut: A design probe to understand how data scientists understand machine learning models . In Proceedings of the CHI Conference on Human Factors in Computing Systems. ACM, 1--13 . F. Hohman, A. Head, R. Caruana, R. DeLine, and S. M. Drucker. 2019. Gamut: A design probe to understand how data scientists understand machine learning models. In Proceedings of the CHI Conference on Human Factors in Computing Systems. ACM, 1--13.
- A. Hopkins . 1999 . The limits of normal accident theory . Safety Sci. 32 , 2 (1999), 93 -- 102 . A. Hopkins. 1999. The limits of normal accident theory. Safety Sci. 32, 2 (1999), 93--102.
- R. Hull , B. Kumar , D. Lieuwen , P. F. Patel-Schneider , A. Sahuguet , S. Varadarajan , and A. Vyas . 2003. Everything personal, not just business: Improving user experience through rule-based service customization . In Proceedings of the International Conference on Service-Oriented Computing. Springer , Berlin, 149--164. R. Hull, B. Kumar, D. Lieuwen, P. F. Patel-Schneider, A. Sahuguet, S. Varadarajan, and A. Vyas. 2003. Everything personal, not just business: Improving user experience through rule-based service customization. In Proceedings of the International Conference on Service-Oriented Computing. Springer, Berlin, 149--164.
- IEEE Global Initiative on Ethics of Autonomous and Intelligent Systems ( 2019 ). Ethically Aligned Design: A Vision for Prioritizing Human Well-being with Autonomous and Intelligent Systems , 1 st ed. IEEE. Retrieved from https://standards.ieee.org/content/ieee-standards/en/industry-connections/ec/autonomous-systems.html https://ethicsinaction.ieee.org/. IEEE Global Initiative on Ethics of Autonomous and Intelligent Systems (2019). Ethically Aligned Design: A Vision for Prioritizing Human Well-being with Autonomous and Intelligent Systems, 1st ed. IEEE. Retrieved from https://standards.ieee.org/content/ieee-standards/en/industry-connections/ec/autonomous-systems.html https://ethicsinaction.ieee.org/.
- X. Liang , S. Shetty , D. Tosh , C. Kamhoua , K. Kwiat , and L. Njilla . 2017. Provchain: A blockchain-based data provenance architecture in cloud environment with enhanced privacy and availability . In Proceedings of the 17th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGRID’17) , 468--477. X. Liang, S. Shetty, D. Tosh, C. Kamhoua, K. Kwiat, and L. Njilla. 2017. Provchain: A blockchain-based data provenance architecture in cloud environment with enhanced privacy and availability. In Proceedings of the 17th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGRID’17), 468--477.
- T. Mai , R. Khanna , J. Dodge , J. Irvine , K. H. Lam , Z. Lin , N. Kiddle , E. Newman , S. Raja , C. Matthews , C. Perdriau , M. Burnett , and A. Fern . 2020. Keeping it “organized and logical”: After-action review for AI (AAR/AI) . In Proceedings of the 25th International Conference on Intelligent User Interfaces. ACM, 465--476 . T. Mai, R. Khanna, J. Dodge, J. Irvine, K. H. Lam, Z. Lin, N. Kiddle, E. Newman, S. Raja, C. Matthews, C. Perdriau, M. Burnett, and A. Fern. 2020. Keeping it “organized and logical”: After-action review for AI (AAR/AI). In Proceedings of the 25th International Conference on Intelligent User Interfaces. ACM, 465--476.
- A. Mitrevski , S. Thoduka , A. O. Sáinz , M. Schöbel , P. Nagel , P. G. Plöger , and E. Prassler . 2018. Deploying robots in everyday environments: Toward dependable and practical robotic systems . In Proceedings of the 29th International Workshop Principles of Diagnosis (DX’18) . Retrieved from http://www.ropod.org/downloads/dx18.pdf. A. Mitrevski, S. Thoduka, A. O. Sáinz, M. Schöbel, P. Nagel, P. G. Plöger, and E. Prassler. 2018. Deploying robots in everyday environments: Toward dependable and practical robotic systems. In Proceedings of the 29th International Workshop Principles of Diagnosis (DX’18). Retrieved from http://www.ropod.org/downloads/dx18.pdf.
- M. R. Morris . 2020. AI and accessibility: A discussion of ethical considerations. Commun. ACM 63, 6 ( 2016 ), ACM 35--37. 10.1145/3356727 M. R. Morris. 2020. AI and accessibility: A discussion of ethical considerations. Commun. ACM 63, 6 (2016), ACM 35--37. 10.1145/3356727 · DOI · Paper page
- Nadella Satya ( 2016 ). The partnership of the future , Slate. Retrieved from https://slate.com/technology/2016/06/microsoft-ceo-satya-nadella-humans-and-a-i-can-work-together-to-solve-societys-challenges.html. Nadella Satya (2016). The partnership of the future, Slate. Retrieved from https://slate.com/technology/2016/06/microsoft-ceo-satya-nadella-humans-and-a-i-can-work-together-to-solve-societys-challenges.html.
- F. Rosenberg and S. Dustdar . 2005. Design and implementation of a service-oriented business rules broker . In Proceedings of the 7th IEEE International Conference on E-Commerce Technology Workshops. IEEE, 55--63 . F. Rosenberg and S. Dustdar. 2005. Design and implementation of a service-oriented business rules broker. In Proceedings of the 7th IEEE International Conference on E-Commerce Technology Workshops. IEEE, 55--63.
- M. Rotenberg . 2020. The AI Policy Sourcebook 2020 . Electronic Privacy Information Center, Washington , DC. M. Rotenberg. 2020. The AI Policy Sourcebook 2020. Electronic Privacy Information Center, Washington, DC.
- H. Sharp , J. Preece , and Y. Rogers . 2019 . Interaction Design: Beyond Human-Computer Interaction , 5 th ed. Wiley Publishers . H. Sharp, J. Preece, and Y. Rogers. 2019. Interaction Design: Beyond Human-Computer Interaction, 5th ed. Wiley Publishers.
- B. Shneiderman , C. Plaisant , M. Cohen , S. Jacobs , and N. Elmqvist . 2016 . Designing the User Interface: Strategies for Effective Human-Computer Interaction , 6 th ed. Pearson . B. Shneiderman, C. Plaisant, M. Cohen, S. Jacobs, and N. Elmqvist. 2016. Designing the User Interface: Strategies for Effective Human-Computer Interaction, 6th ed. Pearson.
- 10.1109/MS.2010.96 · doi:10.1109/ms.2010.96
- A. F. Winfield and M. Jirotka . 2017. The case for an ethical black box . In Proceedings of the Annual Conference Towards Autonomous Robotic Systems, Springer 262--273 . Retrieved from https://link.springer.com/chapter/10.1007/978-3-319-64107-2_21. A. F. Winfield and M. Jirotka. 2017. The case for an ethical black box. In Proceedings of the Annual Conference Towards Autonomous Robotic Systems, Springer 262--273. Retrieved from https://link.springer.com/chapter/10.1007/978-3-319-64107-2_21. · DOI · Paper page
Cited by
Aldoseri, A., Al-Khalifa, K. N., & Hamouda, A. M. (2023) · Applied Sciences · doi:10.3390/app13127082 · Cited by 716
“Data Poisoning: Data poisoning attacks involve tampering with training data to degrade the performance of an AI model [106]. replicate intent”
Trustworthy AI: From Principles to Practices
Li, B., Qi, P., Liu, B., Di, S., Liu, J., Pei, J., Yi, J., & Zhou, B. (2023) · ACM Computing Surveys · doi:10.1145/3555803 · Cited by 612
Responsible Artificial Intelligence Governance: A Review and Research Framework
Papagiannidis, E., Mikalef, P., & Conboy, K. (2025) · The Journal of Strategic Information Systems · doi:10.1016/j.jsis.2024.101885 · Cited by 565
Six Human-Centered Artificial Intelligence Grand Challenges
Ozmen Garibay, O., Winslow, B., Andolina, S., Antona, M., Bodenschatz, A., Coursaris, C., Falco, G., Fiore, S. M., Garibay, I., Grieman, K., et al. (2023) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2022.2153320 · Cited by 501
“…to date, human-centered AI (HCAI) seeks to re-position humans at the center of the AI lifecycle (Bond et al., 2019; Riedl, 2019; Xu, 2019) and improve human performance in ways that are reliable, safe, and trustworthy by augmenting rather than replacing human capabilities (Shneiderman, 2020a).”
Self-citationde Hond, A. A. H., Leeuwenberg, A. M., Hooft, L., Kant, I. M. J., Nijman, S. W. J., van Os, H. J. A., Aardoom, J. J., Debray, T. P. A., Schuit, E., van Smeden, M., et al. (2022) · npj Digital Medicine · doi:10.1038/s41746-021-00549-7 · Cited by 499
Thinking responsibly about responsible AI and ‘the dark side’ of AI
Mikalef, P., Conboy, K., Lundström, J. E., & Popovič, A. (2022) · European Journal of Information Systems · doi:10.1080/0960085x.2022.2026621 · Cited by 435
“However, such occurrences can allow us to deploy novel technological innovations such as AI in a much more safe, effective, and responsible way (Shneiderman, 2020).”
BackgroundA Systematic Literature Review of User Trust in AI-Enabled Systems: An HCI Perspective
Bach, T. A., Khan, A., Hallock, H., Beltrão, G., & Sousa, S. (2024) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2022.2138826 · Cited by 377
“In this case, a collective, multidisciplinary effort to close this gap is urgently needed (Hagendorff, 2020; Shneiderman, 2020c).”
BackgroundMethodInfluentialMunn, L. (2023) · AI and Ethics · doi:10.1007/s43681-022-00209-w · Cited by 355
Xu, W., Dainoff, M. J., Ge, L., & Gao, Z. (2023) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2022.2041900 · Cited by 363
A Review of Trustworthy and Explainable Artificial Intelligence (XAI)
Chamola, V., Hassija, V., Sulthana, A. R., Ghosh, D., Dhingra, D., & Sikdar, B. (2023) · IEEE Access · doi:10.1109/access.2023.3294569 · Cited by 352
Shaping the future of healthcare: ethical clinical challenges and pathways to trustworthy AI
Goktas, P., & Grzybowski, A. (2025) · Journal of Clinical Medicine · doi:10.3390/jcm14051605 · Cited by 336
Human-centric artificial intelligence architecture for industry 5.0 applications
Rožanec, J. M., Novalija, I., Zajec, P., Kenda, K., Tavakoli Ghinani, H., Suh, S., Bian, S., Veliou, E., Papamartzivanos, D., Giannetsos, T., et al. (2023) · International Journal of Production Research · doi:10.1080/00207543.2022.2138611 · Cited by 314
“To ensure artificial intelligence systems are human-centered, much research is being invested in ensuring such systems remain secure and comply with ethical principles (Shneiderman (2020)).”
AI governance: themes, knowledge gaps and future agendas
Birkstedt, T., Minkkinen, M., Tandon, A., & Mäntymäki, M. (2023) · Internet Research · doi:10.1108/intr-01-2022-0042 · Cited by 309
Nazar, M., Alam, M. M., Yafi, E., & Su'ud, M. M. (2021) · IEEE Access · doi:10.1109/access.2021.3127881 · Cited by 312
Defining organizational AI governance
Mäntymäki, M., Minkkinen, M., Birkstedt, T., & Viljanen, M. (2022) · AI and Ethics · doi:10.1007/s43681-022-00143-x · Cited by 285
Schiavo, G., Businaro, S., & Zancanaro, M. (2024) · Technology in Society · doi:10.1016/j.techsoc.2024.102537 · Cited by 284
Draxler, F., Werner, A., Lehmann, F., Hoppe, M., Schmidt, A., Buschek, D., & Welsch, R. (2023) · arXiv · doi:10.48550/arxiv.2303.03283 · Cited by 284
Human-centred learning analytics and AI in education: A systematic literature review
Alfredo, R., Echeverria, V., Jin, Y., Yan, L., Swiecki, Z., Gašević, D., & Martinez-Maldonado, R. (2024) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2024.100215 · Cited by 281
“This approach stands in line with the objectives of HCAI (Shneiderman, 2020b; Usmani et al., 2023; Renz and Vladova, 2021).”
BackgroundMethodResultInfluentialComputer Science Curricula 2023
Kumar, A. N., Raj, R. K., Aly, S. G., Anderson, M. D., Becker, B. A., Blumenthal, R. L., Eaton, E., Epstein, S. L., Goldweber, M., Jalote, P., et al. (2024) · ACM eBooks · doi:10.1145/3664191 · Cited by 223
Design Principles for Generative AI Applications
Weisz, J. D., He, J., Muller, M., Hoefer, G., Miles, R., & Geyer, W. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642466 · Cited by 232
What is Human-Centered about Human-Centered AI? A Map of the Research Landscape
Capel, T., & Brereton, M. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580959 · Cited by 228
Investigating Explainability of Generative AI for Code through Scenario-based Design
Sun, J., Liao, Q. V., Muller, M., Agarwal, M., Houde, S., Talamadupula, K., & Weisz, J. D. (2022) · 27th International Conference on Intelligent User Interfaces · doi:10.1145/3490099.3511119 · Cited by 219
Wanner, J., Herm, L.-V., Heinrich, K., & Janiesch, C. (2022) · Electronic Markets · doi:10.1007/s12525-022-00593-5 · Cited by 208
“Building trust in new technologies is initially hindered by unknown risk factors and thus uncertainty, as well as a lack of total user control (McKnight et al., 2011; Shneiderman, 2020).”
BackgroundAssessing behavioral data science privacy issues in government artificial intelligence deployment
Saura, J. R., Ribeiro-Soriano, D., & Palacios-Marqués, D. (2022) · Government Information Quarterly · doi:10.1016/j.giq.2022.101679 · Cited by 209
“Therefore, if governments have access to third party data, and these are linked to their national intelligence which already has access to massive data, this could led to possible manipulation and decision making focused on surveillance capitalism (see also Shneiderman, 2020).”
BackgroundHuman-Centered Design to Address Biases in Artificial Intelligence
Chen, Y., Clayton, E. W., Novak, L. L., Anders, S., & Malin, B. (2023) · Journal of Medical Internet Research · doi:10.2196/43251 · Cited by 199
Friha, O., Amine Ferrag, M., Kantarci, B., Cakmak, B., Ozgun, A., & Ghoualmi-Zine, N. (2024) · IEEE Open Journal of the Communications Society · doi:10.1109/ojcoms.2024.3456549 · Cited by 189
Mohsin Khan, M., Shah, N., Shaikh, N., Thabet, A., alrabayah, T., & Belkhair, S. (2025) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2024.105780 · Cited by 177
Draxler, F., Werner, A., Lehmann, F., Hoppe, M., Schmidt, A., Buschek, D., & Welsch, R. (2024) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3637875 · Cited by 186
Singhal, A., Neveditsin, N., Tanveer, H., & Mago, V. (2024) · JMIR Medical Informatics · doi:10.2196/50048 · Cited by 172
Designing for Responsible Trust in AI Systems: A Communication Perspective
Liao, Q. V., & Sundar, S. S. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533182 · Cited by 165
How to explain AI systems to end users: a systematic literature review and research agenda
Laato, S., Tiainen, M., Najmul Islam, A. K. M., & Mäntymäki, M. (2022) · Internet Research · doi:10.1108/intr-08-2021-0600 · Cited by 155
Understanding Human-Centred AI: a review of its defining elements and a research agenda
Schmager, S., Pappas, I. O., & Vassilakopoulou, P. (2025) · Behaviour and Information Technology · doi:10.1080/0144929x.2024.2448719 · Cited by 255
“It involves capturing and documenting information about the inputs, and algorithms via audit trails to examine system failures and unexpected behaviour (Ozmen Garibay et al. 2023; Shneiderman 2020a, 2020c).”
InfluentialWeber-Lewerenz, B. C. (2021) · SN Applied Sciences · doi:10.1007/s42452-021-04776-1 · Cited by 219
A Human-AI Collaborative Approach for Clinical Decision Making on Rehabilitation Assessment
Lee, M. H., Siewiorek, D. P., Smailagic, A., Bernardino, A., & Bermúdez i Badia, S. B. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445472 · Cited by 137
Human-machine Collaborative Decision-making: An Evolutionary Roadmap Based on Cognitive Intelligence
Ren, M., Chen, N., & Qiu, H. (2023) · International Journal of Social Robotics · doi:10.1007/s12369-023-01020-1 · Cited by 177
Nahar, N., Zhou, S., Lewis, G., & Kästner, C. (2022) · Proceedings of the 44th International Conference on Software Engineering · doi:10.1145/3510003.3510209 · Cited by 131
“of providing clear requirements for an ML model has also been raised in the literature [49, 55, 80, 92, 104, 111], partially arguing that uncertainty makes it difficult to specify model requirements”
BackgroundMhlanga, D. (2022) · Sustainability · doi:10.3390/su14137804 · Cited by 130
Responsible AI Pattern Catalogue: A Collection of Best Practices for AI Governance and Engineering
Lu, Q., Zhu, L., Xu, X., Whittle, J., Zowghi, D., & Jacquet, A. (2024) · ACM Computing Surveys · doi:10.1145/3626234 · Cited by 121
What Determines AI Adoption in Companies? Mixed-Method Evidence
Polisetty, A., Chakraborty, D., G, S., Kar, A. K., & Pahari, S. (2024) · Journal of Computer Information Systems · doi:10.1080/08874417.2023.2219668 · Cited by 128
Laine, J., Minkkinen, M., & Mäntymäki, M. (2024) · Information & Management · doi:10.1016/j.im.2024.103969 · Cited by 222
“Shneider-man [133] proposed recommendations on three governance levels — team, organization, and industry — to increase reliability, safety, and trustworthiness.”
InfluentialDesigning AI Using a Human-Centered Approach: Explainability and Accuracy Toward Trustworthiness
Schoenherr, J. R., Abbas, R., Michael, K., Rivas, P., & Anderson, T. D. (2023) · IEEE Transactions on Technology and Society · doi:10.1109/tts.2023.3257627 · Cited by 105
Human-XAI Interaction: A Review and Design Principles for Explanation User Interfaces
Chromik, M., & Butz, A. M. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-85616-8_36 · Cited by 207
“However, most XAI research focuses on computational aspects of generating explanations while limited research is reported concerning the human-centered design of the XUI [89, 85, 102].”
BackgroundInfluentialHariri‐Ardebili, M. A., Mahdavi, G., Nuss, L. K., & Lall, U. (2023) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2023.106813 · Cited by 142
What about investors? ESG analyses as tools for ethics-based AI auditing
Minkkinen, M., Niukkanen, A., & Mäntymäki, M. (2024) · AI & SOCIETY · doi:10.1007/s00146-022-01415-0 · Cited by 109
“AI governance requires not only ethical principles and guidelines but also and especially actionable tools and mechanisms (Brundage et al. 2020; Shneiderman 2020; Mäntymäki et al. 2022).”
Contestable AI by Design: Towards a Framework
Alfrink, K., Keller, I., Kortuem, G., & Doorn, N. (2022) · Minds and Machines · doi:10.1007/s11023-022-09611-z · Cited by 169
“To illustrate this point we briefly summarize a number of prominent systematic reviews and meta-analyses drawn from across disciplines (Jobin et al., 2019; Morley et al., 2019; Shneiderman, 2020).”
Project management: openings for disruption from AI and advanced analytics
Niederman, F. (2021) · Information Technology & People · doi:10.1108/itp-09-2020-0639 · Cited by 99
Interactive Model Cards: A Human-Centered Approach to Model Documentation
Crisan, A., Drouhard, M., Vig, J., & Rajani, N. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533108 · Cited by 101
A Review on Human–Machine Trust Evaluation: Human-Centric and Machine-Centric Perspectives
Gebru, B., Zeleke, L., Blankson, D., Nabil, M., Nateghi, S., Homaifar, A., & Tunstel, E. (2022) · IEEE Transactions on Human-Machine Systems · doi:10.1109/thms.2022.3144956 · Cited by 101
AI Ethics Principles in Practice: Perspectives of Designers and Developers
Sanderson, C., Douglas, D., Lu, Q., Schleiger, E., Whittle, J., Lacey, J., Newnham, G., Hajkowicz, S., Robinson, C., & Hansen, D. (2023) · IEEE Transactions on Technology and Society · doi:10.1109/tts.2023.3257303 · Cited by 99
Empowering local communities using artificial intelligence
Hsu, Y.-C., Huang, T.-H., Verma, H., Mauri, A., Nourbakhsh, I., & Bozzon, A. (2022) · Patterns · doi:10.1016/j.patter.2022.100449 · Cited by 133
“tunity to study co-design methods and software engineering workflows of translating the predictions of AI models and their internal decision-making process into human-intelligible insights in the hyperlocal context.(1,48,49) We believe the pipeline of such translation into explainable evidence can be a methodological…”
BackgroundMinkkinen, M., Laine, J., & Mäntymäki, M. (2022) · Digital Society · doi:10.1007/s44206-022-00022-2 · Cited by 88
“The rationale for CA is linked to the aspired human oversight of AI systems (Floridi et al., 2018; Shneiderman, 2020).”
BackgroundStakeholder roles in artificial intelligence projects
Miller, G. J. (2022) · Project Leadership and Society · doi:10.1016/j.plas.2022.100068 · Cited by 142
Georgieva, I., Lazo, C., Timan, T., & van Veenstra, A. F. (2022) · AI and Ethics · doi:10.1007/s43681-021-00127-3 · Cited by 89
Nadarzynski, T., Knights, N., Husbands, D., Graham, C. A., Llewellyn, C. D., Buchanan, T., Montgomery, I., & Ridge, D. (2024) · PLOS Digital Health · doi:10.1371/journal.pdig.0000492 · Cited by 117
Artificial Intelligence and Public Values: Value Impacts and Governance in the Public Sector
Chen, Y., Ahn, M. J., & Wang, Y. (2023) · Sustainability · doi:10.3390/su15064796 · Cited by 161
“Service Orientation Transparency [1,17,35,50,60,63,65–74] Effectiveness [15,23,39,75] Efficiency [15,39]”
BackgroundInfluentialWho needs explanation and when? Juggling explainable AI and user epistemic uncertainty
Jiang, J., Kahai, S., & Yang, M. (2022) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2022.102839 · Cited by 84
Human-AI Collaboration in Thematic Analysis using ChatGPT: A User Study and Design Recommendations
Yan, L., Echeverría, V., Fernandez-Nieto, G. M., Jin, Y., Swiecki, Z., Zhao, L., Gašević, D., & Martínez‐Maldonado, R. (2024) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3613905.3650732 · Cited by 148
Evaluating Trustworthiness in AI: Risks, Metrics, and Applications Across Industries
Nastoska, A., Jancheska, B., Rizinski, M., & Trajanov, D. (2025) · Electronics · doi:10.3390/electronics14132717 · Cited by 167
Creating Design Resources to Scaffold the Ideation of AI Concepts
Yildirim, N., Oh, C., Sayar, D., Brand, K., Challa, S., Turri, V., Crosby Walton, N., Wong, A. E., Forlizzi, J., McCann, J., et al. (2023) · Proceedings of the 2023 ACM Designing Interactive Systems Conference · doi:10.1145/3563657.3596058 · Cited by 76
Chow, J. C. L., & Li, K. (2024) · JMIR Bioinformatics and Biotechnology · doi:10.2196/64406 · Cited by 75
Towards a roadmap on software engineering for responsible AI
Lu, Q., Zhu, L., Xu, X., Whittle, J., & Xing, Z. (2022) · IEEE/ACM International Conference on AI Engineering – Software Engineering for AI (CAIN) · doi:10.1145/3522664.3528607 · Cited by 103
Akbarighatar, P., Pappas, I., & Vassilakopoulou, P. (2023) · International Journal of Information Management Data Insights · doi:10.1016/j.jjimei.2023.100193 · Cited by 72
Shneiderman, B. (2021) · Communications of the ACM · doi:10.1145/3445973 · Cited by 84
Self-citationDixon, R. B. L. (2022) · AI and Ethics · doi:10.1007/s43681-022-00205-0 · Cited by 106
AI Adoption and Educational Sustainability in Higher Education in the UAE
Shwedeh, F., Salloum, S. A., Aburayya, A., Fatin, B., Elbadawi, M. A., Ghurabli, Z. A., & Aldabbagh, T. (2024) · Studies in big data · doi:10.1007/978-3-031-52280-2_14 · Cited by 120
Better Together? An Evaluation of AI-Supported Code Translation
Weisz, J. D., Muller, M., Ross, S. I., Martinez, F., Houde, S., Agarwal, M., Talamadupula, K., & Richards, J. T. (2022) · 27th International Conference on Intelligent User Interfaces · doi:10.1145/3490099.3511157 · Cited by 69
Schultz, M. D., & Seele, P. (2022) · AI and Ethics · doi:10.1007/s43681-022-00150-y · Cited by 96
Call for the responsible artificial intelligence in the healthcare
Upadhyay, U., Gradišek, A., Iqbal, U., Dhar, E., Li, Y., & Syed-Abdul, S. (2023) · BMJ Health & Care Informatics · doi:10.1136/bmjhci-2023-100920 · Cited by 92
“21 This can be harmful to under-represented and vulnerable groups.”
BackgroundNordic Human-Computer Interaction Conference
(2022) · doi:10.1145/3546155 · Cited by 67
Holding AI to Account: Challenges for the Delivery of Trustworthy AI in Healthcare
Procter, R., Tolmie, P., & Rouncefield, M. (2022) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3577009 · Cited by 92
Management perspective of ethics in artificial intelligence
Baker-Brunnbauer, J. (2020) · AI and Ethics · doi:10.1007/s43681-020-00022-3 · Cited by 82
Minkkinen, M., Zimmer, M. P., & Mäntymäki, M. (2022) · Information Systems Frontiers · doi:10.1007/s10796-022-10269-2 · Cited by 92
“The conceptual components for understanding RAI ecosystems are summarized in Table 1. industry-level requirements exert pressure on organizations and development teams (Gasser & Almeida, 2017; Kamin-ski, 2019; Rakova et al., 2021; Shneiderman, 2020).”
BackgroundRequirements engineering framework for human-centered artificial intelligence software systems
Ahmad, K., Abdelrazek, M., Arora, C., Baniya, A. A., Bano, M., & Grundy, J. (2023) · Applied Soft Computing · doi:10.1016/j.asoc.2023.110455 · Cited by 84
“Shneiderman [87] presented a framework that proposes 15 recommendations for engineering trustworthy, reliable, and safe human-centered AI-based systems.”
BackgroundVerma, H., Mlynář, J., Schaer, R., Reichenbach, J., Jreige, M., Prior, J. O., Évéquoz, F., & Depeursinge, A. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581506 · Cited by 78
Zhao, J., Han, X., Wu, Y., Wang, Z., & Burke, A. F. (2024) · Journal of Energy Chemistry · doi:10.1016/j.jechem.2024.11.011 · Cited by 73
Young, J., Jawara, L. M., Nguyen, D. N., Daly, B., Huh-Yoo, J., & Razi, A. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642574 · Cited by 59
Chin, T., Cheng, T. C. E., Wang, C., & Huang, L. (2024) · Journal of Knowledge Management · doi:10.1108/jkm-06-2023-0458 · Cited by 70
Innovative applications of artificial intelligence during the COVID-19 pandemic
Lv, C., Guo, W., Yin, X., Liu, L., Huang, X., Li, S., & Zhang, L. (2024) · Infectious Medicine · doi:10.1016/j.imj.2024.100095 · Cited by 76
“Rigorous testing for biases and continuous monitoring of outcomes are imperative to avoid disproportionate impacts on marginalized communities [ 134 , 135 ].”
BackgroundYang, Q. (2021) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3485875 · Cited by 81
Textor, C., Zhang, R., Lopez, J., Schelble, B. G., McNeese, N. J., Freeman, G., Pak, R., Tossell, C., & de Visser, E. J. (2022) · Journal of Cognitive Engineering and Decision Making · doi:10.1177/15553434221113964 · Cited by 57
“Others have emphasized the importance of reliability, safety, and trustworthiness (Shneiderman, 2020).”
BackgroundSmart Interactive Technologies in the Human-Centric Factory 5.0: A Survey
Brunetti, D., Cristina Gena, A., & Vernero, F. (2022) · Applied Sciences · doi:10.3390/app12167965 · Cited by 64
Human-Centered augmented translation: against antagonistic dualisms
O’Brien, S. (2023) · Perspectives · doi:10.1080/0907676x.2023.2247423 · Cited by 106
“…augmentation, however, the paper will fi rst consider the idea of ‘ antagonistic dualisms ’ (Haraway, 1985) and Human-Centered Arti fi cal Intelligence (HCAI – Shneiderman, 2020a, 2020b) and how they might apply to the study and practice of translation in particular in discussions centering on…”
BackgroundInfluentialWhat's next for responsible artificial intelligence: a way forward through responsible innovation
Herrmann, H. (2023) · Heliyon · doi:10.1016/j.heliyon.2023.e14379 · Cited by 75
“Today ’ s reality is that an organization ’ s governance drives toward profitability objectives (i.e., economic sustainability) often overriding social responsibilities [66,223].”
BackgroundMaking It Possible for the Auditing of AI: A Systematic Review of AI Audits and AI Auditability
Li, Y., & Goel, S. (2024) · Information Systems Frontiers · doi:10.1007/s10796-024-10508-8 · Cited by 103
“Researchers consider the independent oversight by external review organizations as valuable in achieving trustworthy systems that receive wide public acceptance (Li et al., 2023; Shneiderman, 2020) and wider transparency (Ayling & Chapman, 2022).”
Karpov, O. E., Николаевна, П. Е., Kurkin, S., Александрович, М. В., Gusev, A. V., Shusharina, N. N., & Hramov, A. E. (2023) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph20075335 · Cited by 72
Li, Y., & Goel, S. (2025) · International Journal of Accounting Information Systems · doi:10.1016/j.accinf.2025.100739 · Cited by 121
Maiti, M., Kayal, P., & Vujko, A. (2025) · Future Business Journal · doi:10.1186/s43093-025-00462-5 · Cited by 126
Koshiyama, A., Kazim, E., & Treleaven, P. (2022) · Computer · doi:10.1109/mc.2021.3067225 · Cited by 55
Taxonomy of Machine Learning Safety: A Survey and Primer
Mohseni, S., Wang, H., Xiao, C., Yu, Z., Wang, Z., & Yadawa, J. (2023) · ACM Computing Surveys · doi:10.1145/3551385 · Cited by 53
El–Sappagh, S., Alonso, J. M., Abuhmed, T., Ali, F., & Bugarín, A. (2023) · Artificial Intelligence Review · doi:10.1007/s10462-023-10415-5 · Cited by 69
Sigfrids, A., Nieminen, M., Leikas, J., & Pikkuaho, P. (2022) · Frontiers in Human Dynamics · doi:10.3389/fhumd.2022.858108 · Cited by 50
“…addressing ethical issues of AI governance can be perceived at three levels (Stahl, 2021): policy and legislation (e.g., Jobin et al., 2019; European Commission, 2022), organizations (Shneiderman, 2020; Tsamados et al., 2021), and guidance mechanisms for individuals (e.g., Morley et al., 2020).”
BackgroundRhim, J., Lee, J., Chen, M., & Lim, A. (2021) · Frontiers in Robotics and AI · doi:10.3389/frobt.2021.632394 · Cited by 65
“Third, human-centered AI (HCAI) provides clear goals to achieve reliable, safe, and trustworthy AI-embedded systems Shneiderman (2020), yet how to achieve these goals is unclear.”
BackgroundWaqar, A., Andri, A., Qureshi, A. H., Almujibah, H. R., Tanjung, L. E., & Utami, C. (2023) · Ain Shams Engineering Journal · doi:10.1016/j.asej.2023.102551 · Cited by 72
Butt, M. A., Qayyum, A., Ali, H. M., Al‐Fuqaha, A., & Qadir, J. (2022) · Computers & Security · doi:10.1016/j.cose.2022.103058 · Cited by 58
Visualization for Recommendation Explainability: A Survey and New Perspectives
Chatti, M. A., Guesmi, M., & Muslim, A. (2024) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3672276 · Cited by 70
Seven HCI Grand Challenges Revisited: Five-Year Progress
Stephanidis, C., Salvendy, G., Antona, M., Duffy, V. G., Gao, Q., Karwowski, W., Konomi, S., Nah, F. F., Ntoa, S., Rau, P. P., et al. (2025) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2025.2450411 · Cited by 69
“In parallel, new approaches and guidelines have been proposed to improve human safety (Shneiderman, 2020c).”
InfluentialQuantitative Evaluation of Machine Learning Explanations: A Human-Grounded Benchmark
Mohseni, S., Block, J. E., & Ragan, E. D. (2021) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3397481.3450689 · Cited by 74
Symphony: Composing Interactive Interfaces for Machine Learning
Bäuerle, A., Cabrera, Á. A., Hohman, F., Maher, M., Koski, D., Suau, X., Barik, T., & Moritz, D. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3502102 · Cited by 69
Human-Centered AI Design in Reality: A Study of Developer Companies’ Practices
Hartikainen, M., Väänänen, K., Lehtiö, A., Ala-Luopa, S., & Olsson, T. (2022) · Nordic Human-Computer Interaction Conference · doi:10.1145/3546155.3546677 · Cited by 48
Abulibdeh, A. O., Chatti, C. B., Alkhereibi, A. H., & El-Menshawy, S. (2025) · European Journal of Education · doi:10.1111/ejed.12908 · Cited by 93
Metadata
- Authors
- Shneiderman, B.
- Affiliations
- University of Maryland, College Park (US)
- Corresponding author
- Shneiderman, Ben
- ORCID
- Shneiderman, Ben
- Venue
- ACM Transactions on Interactive Intelligent Systems
- Published
- 2020-10-16
- Volume
- 10
- Issue
- 4
- Pages
- 1-31
- ISSN
- 2160-6455 (ISSN-L)
- 2160-6463
- OpenAlex
- W3094233077
- Cited by
- 768
- References
- 89
- Citations by year
- 2026: 185 · 2025: 208 · 2024: 153 · 2023: 105 · 2022: 78 · 2021: 38 · 2020: 1
- FWCI
- 75.54
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- ACM Transactions on Interactive Intelligent Systems (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Engineering › Safety, Risk, Reliability and Quality › Safety Systems Engineering in Autonomy
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Keywords
- human-centered AI, AI governance, software engineering practices, safety culture, audit trail, verification and validation, bias testing, failure reporting, near misses, AI regulation, organizational leadership
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.