Responsible AI Pattern Catalogue: A Collection of Best Practices for AI Governance and Engineering
Open accessThe DOI matches its Crossref record; 27 of the 27 references it cites were found in a registry or an index.
Abstract
Responsible Artificial Intelligence (RAI) is widely considered as one of the greatest scientific challenges of our time and is key to increase the adoption of Artificial Intelligence (AI). Recently, a number of AI ethics principles frameworks have been published. However, without further guidance on best practices, practitioners are left with nothing much beyond truisms. In addition, significant efforts have been placed at algorithm level rather than system level, mainly focusing on a subset of mathematics-amenable ethical principles, such as fairness. Nevertheless, ethical issues can arise at any step of the development lifecycle, cutting across many AI and non-AI components of systems beyond AI algorithms and models. To operationalize RAI from a system perspective, in this article, we present an RAI Pattern Catalogue based on the results of a multivocal literature review. Rather than staying at the principle or algorithm level, we focus on patterns that AI system stakeholders can undertake in practice to ensure that the developed AI systems are responsible throughout the entire governance and engineering lifecycle. The RAI Pattern Catalogue classifies the patterns into three groups: multi-level governance patterns, trustworthy process patterns, and RAI-by-design product patterns. These patterns provide systematic and actionable guidance for stakeholders to implement RAI.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The global landscape of AI ethics guidelines
Jobin, A., Ienca, M., & Vayena, E. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0088-2 · Cited by 6,220
A Survey on Bias and Fairness in Machine Learning
Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., & Galstyan, A. (2019) · arXiv · doi:10.48550/arxiv.1908.09635 · Cited by 5,812
A Survey on Bias and Fairness in Machine Learning
Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., & Galstyan, A. (2022) · ACM Computing Surveys · doi:10.1145/3457607 · Cited by 5,812
Practical Secure Aggregation for Privacy-Preserving Machine Learning
Bonawitz, K., Ivanov, V., Kreuter, B., Marcedone, A., McMahan, H. B., Patel, S., Ramage, D., Segal, A., & Seth, K. (2017) · Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3133956.3133982 · Cited by 3,956 · cited 1×
“The federated learner [15, 18, 68–70, 112, 113, 117] preserves the data privacy by training models locally on the client devices and formulating a global model on a central server based on the local model updates (e.g., train the visual perception model locally in each vehicle).”
MethodModel Cards for Model Reporting
Mitchell, M., Wu, S., Zaldivar, A., Barnes, P., Vasserman, L., Hutchinson, B., Spitzer, E., Raji, I. D., & Gebru, T. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287596 · Cited by 2,681 · cited 2×
“In addition to supply chain details of the components, context documents (like model cards [81] for reporting AI models, and datasheets for the datasets [41] used to train AI models) can be integrated to the bill of materials registry.”
MethodGebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., Daumé, H., & Crawford, K. (2018) · arXiv · doi:10.48550/arxiv.1803.09010 · Cited by 2,474
Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., III, H. D., & Crawford, K. (2021) · Communications of the ACM · doi:10.1145/3458723 · Cited by 2,469 · cited 2×
“In addition to supply chain details of the components, context documents (like model cards [81] for reporting AI models, and datasheets for the datasets [41] used to train AI models) can be integrated to the bill of materials registry.”
MethodRaji, I. D., Smart, A., White, R. N., Mitchell, M., Gebru, T., Hutchinson, B., Smith-Loud, J., Theron, D., & Barnes, P. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372873 · Cited by 2,245 · cited 3×
“Continuous documentation using templates helps track the evolution of artifacts and clarify the context in which AI systems are trustworthy [7, 8, 9, 10, 11, 12].”
BackgroundCapability maturity model, version 1.1
Paulk, M. C., Curtis, B., Chrissis, M. B., & Weber, C. V. (1993) · IEEE Software · doi:10.1109/52.219617 · Cited by 1,263 · cited 1×
“Verifiable ethical credentials can be used as evidence of ethical compliance for AI systems, components, models, developers, operators, 96 users, organizations, and development processes [21, 72, 92].”
BackgroundFjeld, J., Achten, N., Hilligoss, H., Nagy, A., & Srikumar, M. (2020) · SSRN Electronic Journal · doi:10.2139/ssrn.3518482 · Cited by 998
Questioning the AI: Informing Design Practices for Explainable AI User Experiences
Liao, Q. V., Gruen, D., & Miller, S. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376590 · Cited by 925
Swarm Learning for decentralized and confidential clinical machine learning
Warnat-Herresthal, S., Schultze, H., Shastry, K. L., Manamohan, S., Mukherjee, S., Garg, V., Sarveswara, R., Händler, K., Pickkers, P., Aziz, N. A., et al. (2021) · Nature · doi:10.1038/s41586-021-03583-3 · Cited by 884 · cited 1×
“Decentralized learning is a variant of federated learning, which could use blockchain to remove the single point of failure and coordinate the learning process in a fully decentralized way [114].”
BackgroundGarousi, V., Felderer, M., & Mäntylä, M. V. (2019) · Information and Software Technology · doi:10.1016/j.infsof.2018.09.006 · Cited by 831
Shneiderman, B. (2020) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3419764 · Cited by 768
DeepChain: Auditable and Privacy-Preserving Deep Learning with Blockchain-based Incentive
Weng, J., Weng, J., Zhang, J., Li, M., Zhang, Y., & Luo, W. (2019) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2019.2952332 · Cited by 610 · cited 1×
“An incentive registry records the rewards that correspond to the AI system’s ethical behavior and outcome of decisions [123, 124], e.”
BackgroundFormalizing Trust in Artificial Intelligence: Prerequisites, Causes and Goals of Human Trust in AI
Jacovi, A., Marasović, A., Miller, T., & Goldberg, Y. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445923 · Cited by 545 · cited 2×
“To try to fill the principle-algorithmic gap, further guidance such as guidebooks, 2 , 3 questions to generate discussions [66, 67], checklists [44, 62], and documentation templates [1, 4, 52, 55, 94, 127] have started to appear.”
BackgroundFactSheets: Increasing trust in AI services through supplier's declarations of conformity
Arnold, M., Bellamy, R. K. E., Hind, M., Houde, S., Mehta, S., Mojsilović, A., Nair, R., Ramamurthy, K. N., Olteanu, A., Piorkowski, D., et al. (2019) · IBM Journal of Research and Development · doi:10.1147/jrd.2019.2942288 · Cited by 434 · cited 2×
“Role-level accountability contracts make the developers keep ethics in mind at every step, but may create stress for employees at all levels within an organization.”
BackgroundHutchinson, B., Smart, A., Hanna, A., Denton, R., Greer, C., Kjartansson, O., Barnes, P., & Mitchell, M. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445918 · Cited by 279
Governing AI safety through independent audits
Falco, G., Shneiderman, B., Badger, J., Carrier, R., Dahbura, A., Danks, D., Eling, M., Goodloe, A., Gupta, J., Hart, C., et al. (2021) · Nature Machine Intelligence · doi:10.1038/s42256-021-00370-7 · Cited by 250
Beyond the promise: implementing ethical AI
Eitel-Porter, R. (2021) · AI and Ethics · doi:10.1007/s43681-020-00011-6 · Cited by 245
Recommendations for the ethical use and design of artificial intelligent care providers
Luxton, D. D. (2014) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2014.06.004 · Cited by 243
Raji, I. D., Smart, A., White, R. N., Mitchell, M., Gebru, T., Hutchinson, B., Smith-Loud, J., Theron, D., & Barnes, P. (2020) · arXiv · doi:10.48550/arxiv.2001.00973 · Cited by 179
Legal requirements on explainability in machine learning
Bibal, A., Lognoul, M., de Streel, A., & Frénay, B. (2021) · Artificial Intelligence and Law · doi:10.1007/s10506-020-09270-4 · Cited by 165 · cited 1×
“Ethical requirements need to be derived from the AI ethics principles to fit into a specific domain and system context [16, 49, 93, 118, 128].”
BackgroundZ-Inspection ® : A Process to Assess Trustworthy AI
Zicari, R. V., Brodersen, J., Brusseau, J., Dudder, B., Eichhorn, T., Ivanov, T., Kararigas, G., Kringen, P., McCullough, M., Moslein, F., et al. (2021) · IEEE Transactions on Technology and Society · doi:10.1109/tts.2021.3066209 · Cited by 134
McCradden, M. D., Joshi, S., Anderson, J. A., Mazwi, M., Goldenberg, A., & Zlotnik Shaul, R. (2020) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaa085 · Cited by 111
Designing Contestability: Interaction Design, Machine Learning, and Mental Health
Hirsch, T., Merced, K., Narayanan, S., Imel, Z. E., & Atkins, D. C. (2017) · Proceedings of the 2017 Conference on Designing Interactive Systems · doi:10.1145/3064663.3064703 · Cited by 110 · cited 1×
“Phased deployment means deploying AI systems for a subset group of users initially to reduce ethical risk [96].”
MethodQuestion-Driven Design Process for Explainable AI User Experiences
Liao, Q. V., Pribić, M., Han, J., Miller, S., & Sow, D. (2021) · arXiv · doi:10.48550/arxiv.2104.03483 · Cited by 18 · cited 2×
“Checklists (e.g., a question bank) are often used to help design the explainable user interfaces [62, 66, 67] and understand the user needs, choices of XAI techniques, and XAI design factors [67].”
Background
Cited by
Alshehhi, K., Cheaitou, A., & Rashid, H. (2024) · Sustainability · doi:10.3390/su16177724 · Cited by 19
Kallina, E., & Singh, J. (2025) · Proceedings of the Extended Abstracts of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706599.3719726 · Cited by 6
“It’s Just a Wild, Wild West”: Harnessing Public Procurement as an AI Governance Mechanism
Hudig, A. I., Kallina, E., & Singh, J. (2026) · Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791968 · Cited by 3
Emerging models of national competent authorities under the EU AI Act
Parisini, E., & Dervishaj, E. (2025) · Conference on Digital Government Research · doi:10.59490/dgo.2025.1007 · Cited by 6
“The AI Act, which came into force in August 2024, sets the standard framework that European Member States must comply with to ensure that society understands AI’s capabilities and limitations, while promoting development and innovation (Lu, 2024).”
Metadata
- Authors
- Lu, Q., Zhu, L., Xu, X., Whittle, J., Zowghi, D., & Jacquet, A.
- Venue
- ACM Computing Surveys
- Published
- 2024-04-09
- Volume
- 56
- Issue
- 7
- Pages
- 1-35
- ISSN
- 0360-0300 (ISSN-L)
- 1557-7341
- OpenAlex
- W4387377782
- Cited by
- 121
- References
- 27
- FWCI
- 29.54
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.