Machine Learning Techniques for Accountability
Open accessAbstract
Artificial intelligence systems have provided us with many everyday conveniences. We can easily search for information across millions of web‐pages via text and voice. Paperwork processing is increasingly automated. Artificial intelligence systems flag potentially fraudulent credit‐card transactions and filter our e‐mail. Yet these artificial intelligence systems have also experienced significant failings. Across a range of applications, including loan approvals, disease severity scores, hiring algorithms, and face recognition, artificial‐intelligence‐based scoring systems have exhibited gender and racial bias. Self‐driving cars have had serious accidents. As these systems become more prevalent, it is increasingly important that we identify the best ways to keep them accountable.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Tools to attain accountability involve algorithmic accountability policy toolkits, the post-hoc analysis of the output of the model, or algorithms for causal inference and reasoning.
“Tools to attain this requirement involve algorithmic accountability policy toolkits (e.g., [118]), the post-hoc analysis of the output of the model (e.g. via local relevance attribution methods) or algorithms for causal inference and reasoning [119].”
From Connecting the dots in trustworthy Artificial Intelligence: From AI principles, ethics, and key requirements to responsible AI systems and regulation (2023) · cites this paper as [119]
The “Post Hoc Inspection of Model Outputs” section describes inspecting model outputs as a mechanism for accountability, and “Approaches” includes saliency maps as a feature-attribution method. Our copy does not identify causal-inference or reasoning algorithms as accountability tools (searched: “causal inference and reasoning” and “causal”). The policy-toolkit example is cited separately to [118]. (LLM, conf=0.85)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Model Cards for Model Reporting
Mitchell, M., Wu, S., Zaldivar, A., Barnes, P., Vasserman, L., Hutchinson, B., Spitzer, E., Raji, I. D., & Gebru, T. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287596 · Cited by 2,681 · cited 1× · see citing works
“Mitchell et al. (2019) describes an analogous approach for describing the models and algorithms that a system uses to make predictions, including relevant parameter and training choices.”
MethodIntelligible models for classification and regression
Lou, Y., Caruana, R., & Gehrke, J. (2012) · Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/2339530.2339556 · Cited by 547 · see citing works
Making AI Forget You: Data Deletion in Machine Learning
Ginart, A., Guan, M. Y., Valiant, G., & Zou, J. (2019) · arXiv · doi:10.48550/arxiv.1907.05012 · Cited by 175 · see citing works
Flagged first, then by citations
Research works printed but not linked: 25
- Adebayo, J.; Gilmer, J.; Muelly, M.; Goodfellow, I.; Hardt, M.; and Kim, B. 2018. Sanity Checks for Saliency Maps. arXiv preprint. arXiv:1810.03292[cs.CV]. Ithaca, NY: Cornell Uni- versity Library. · arXiv
- Anil, C.; Lucas, J.; and Grosse, R. 2018. Sorting Out Lipschitz Function Approximation. arXiv preprint. arXiv:1811.05381 [cs.LG]. Ithaca, NY: Cornell University Library. · arXiv
- Chaudhuri, K.; Monteleoni, C.; and Sarwate, A. D. 2011. Differentially Private Empirical Risk Minimization. Journal of Machine Learning Research 12:1069-109.
- Doshi-Velez, F.; Wallace, B.; and Adams, R. 2015. Graph- Sparse LDA: A Topic Model with Structured Sparsity. In Pro- ceedings of the Twenty-Ninth Association for the Advancement of Artificial Intelligence (AAAI) Conference on Artificial Intelligence, 2575-81. Palo Alto, CA: AAAI Press.
- Forde, J.; Head, T.; Holdgraf, H.; Panda, Y.; Nalvarete, N.; Ragan-Kelley, B.; and Sundell, E. (2018) Reproducible Research Environments with Repo2Docker. Paper presented at the Reproducibility in Machine Learning Workshop 2018. OpenReview.net.
- Gebru, T.; Morgenstern, J.; Vecchione, B.; Vaughan, J.; Wallach, W.; Daumeé, H.; and Crawford, K. 2018. Data- sheets for Datasets. arXiv preprint. arXiv:1803.09010[cs. ]. Ithaca, NY: Cornell University Library. · arXiv
- Wallach, H.; Larochelle, H.; Beygelzimer, A.; d'Alché-Buc, F.; Fox, E.; and Garnett, R., in Advances in Neural Informa- tion Processing Systems 32 (NeurIPS 2019). papers.nips.cc/ paper/2019/hash/cb79f8fa58b91d3af6c9c991f63962d3- Abstract.html.
- Gupta, M.; Cotter, A.; Pfeifer, J.; Voevodski, K.; Canini, K.; Mangylov, A.; Moczydlowski, W.; and van Esbroeck, A. 2016. Monotonic Calibrated Interpolated Look-Up Tables. Journal of Machine Learning Research 17:109:1-109:47.
- Hase, P.; Chen, C.; Li, O.; and Rudin, C. 2019. Interpretable Image Recognition with Hierarchical Prototypes. In Proceedings of the 7th Association for the Advancement of Artificial Intelligence (AAAI) Conference on Human Computation and Crowdsourcing, 32-40. Palo Alto, CA: AAAI Press.
- Kim, B.; Rudin, C.; and Shah, J. 2014. The Bayesian Case Model: A Generative Approach for Case-Based Reasoning and Prototype Classification. In Neural Information Processing Systems 2014. arXiv preprint. ArXiv:1503.01161[stat.ML]. Ithaca, NY: Cornell University Library. · arXiv
- Kim, B.; Shah, J.; and Doshi-Velez, F. 2015. Mind the Gap: A Generative Approach to Interpretable Feature Selection and Extraction. Paper presented at the Advances in Neural Infor- mation Processing Systems 28 (NIPS 2015). papers.neurips. cc/paper/5957-mind-the-gap-a-generative-approach-to- interpretable-feature-selection-and-extraction 23719621, 2021, 1, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/j.2371-9621.2021.tb00010.x, Wiley Online Library on [18/12/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
- Kim, B.; Wattenberg, M.; Gilmer, J.; Cai, C.; Wexler, J.; Viegas, F.; and Sayres, R. 2018. Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV). Proceedings of Machine Learning Research 80:2668-77.
- Koh, P. W., and Liang, P. 2017. Understanding Black-Box Predictions via Influence Functions. arXiv preprint. arXiv: 1703.04730[stat.ML]. Ithaca, NY: Cornell University Library. · arXiv
- McInnes, L.; Healy, J.; and Melville, J. 2018. Umap: Uni- form Manifold Approximation and Projection for Dimen- sion Reduction. arXiv preprint. arXiv:1802.03426[stat.ML]. Ithaca, NY: Cornell University Library. · arXiv
- McMahan, H. B.; Moore, E.; Ramage, D.; Hampson, S.; and Arcas, B. 2016. Communication-Efficient Learning of Deep Networks from Decentralized Data. arXiv preprint. arXiv:1602.05629[cs.LG]. Ithaca, NY: Cornell University Library. · arXiv
- Nie, W.; Zhang, Y.; and Patel, A. 2018. A Theoretical Expla- nation for Perplexing Behaviors of Backpropagation-Based Visualizations. Proceedings of Machine Learning Research 80:3806-15.
- Rudin, C. 2019. Stop Explaining Black Box Machine Learn- ing Models for High Stakes Decisions and Use Interpretable Models Instead. Nature Machine Intelligence 1(5): 206-15. doi.org/10.1038/s42256-019-0048-x · DOI · Paper page
- Selvaraju, R.; Das, A.; Vedantam, R.; Cogswell, M.; Parikh, D.; and Batra, D. 2016. Grad-Cam: Why Did You Say That? Visual Explanations from Deep Networks via Gradient-Based Localization. arXiv preprint. arXiv:1610.02391[cs.CV]. Ithaca, NY: Cornell University Library. · arXiv
- Smilkov, D.; Thorat, N.; Kim, B.; Viégas, F.; and Wattenberg, M. 2017. Smoothgrad: Removing Noise by Adding Noise. arXiv preprint. arXiv:1706.03825[cs.LG]. Ithaca, NY: Cornell University Library. · arXiv
- Steinhardt, J.; Wei, P.; Koh, W.; and Liang, P. S. 2017. Cer- tified Defenses for Data Poisoning Attacks. In Advances in Neural Information Processing Systems 30, Annual Conference on Neural Information Processing Systems 2017 (NIPS 2017). papers.nips.cc/paper/2017/hash/ 9d7311ba459f9e45ed746755a32dcd11-Abstract.html
- Sundararajan, M.; Taly, A.; and Yan, Q. 2017. Axiomatic Attribution for Deep Networks. Proceedings of Machine Learning Research 70:3319-28.
- Ulyanov, D.; Vedaldi, A.; and Lempitsky, V. 2017. Deep Image Prior. arXiv preprint. arXiv:1711.10925[cs.CV]. Ithaca, NY: Cornell University Library. · arXiv
- Ustun, B., and Rudin, C. 2014. Methods and Models for Interpretable Linear Classification. ArXiv preprint. arXiv:1405.4047[stat.ML]. Ithaca, NY: Cornell University Library. · arXiv
- van der Maaten, L., and Hinton, G. 2008. Visualizing Data Using T-SNE. Machine Learning 87(1): 33-55.
- You, S.; Ding, D.; Canini, K.; Pfeifer, J.; and Gupta, M. 2017. Deep Lattice Networks and Partial Monotonic Func- tions. arXiv preprint. arXiv:1709.06680[stat.ML]. Ithaca, NY: Cornell University Library. · arXiv
Cited by
Sorted by citations
Díaz-Rodríguez, N., Del Ser, J., Coeckelbergh, M., López de Prado, M., Herrera-Viedma, E., & Herrera, F. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.101896 · Cited by 800 · see citing works
Toward Accountable and Explainable Artificial Intelligence Part Two: The Framework Implementation
Vice, J., & Khan, M. M. (2022) · IEEE Access · doi:10.1109/access.2022.3163523 · Cited by 11 · see citing works
- Authors
- Kim, B., & Doshi‐Velez, F.
- Affiliations
- Google (United States) (US)
- Harvard University (US)
- Venue
- AI Magazine
- Publisher
- Wiley
- Published
- 2021-03
- Volume
- 42
- Issue
- 1
- Pages
- 47-52
- ISSN
- 0738-4602 (ISSN-L)
- 2371-9621
- OpenAlex
- W3165101951
- Cited by
- 18
- References
- 3
- Citations by year
- 2025: 6 · 2024: 3 · 2023: 3 · 2022: 6
- FWCI
- 3.76
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- AI Magazine (publishedVersion)
- AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) (submittedVersion, cc-by)
- Topics
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Social Sciences › Law › Digitalization, Law, and Regulation
- Keywords
- accountability, artificial intelligence, algorithmic bias, algorithmic fairness, explainability, machine learning, algorithmic decision-making, algorithmic auditing
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
