Why We Need to Know More: Exploring the State of AI Incident Documentation Practices
Open accessAbstract
To enable the development and use of safe and equitable artificial intelligence (AI) systems, AI engineers must monitor deployed AI systems and learn from past AI incidents where failures have occurred. Around the world, public databases for cataloging AI systems and resulting harms are instrumental in promoting awareness of potential AI harms among policymakers, researchers, and the public. However, despite growing recognition of the potential of AI systems to produce harms, causes of AI systems failure remain elusive and AI incidents continue to occur. For example, incidents of AI bias are frequently reported and discussed, yet biased systems continue to be developed and deployed.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Taxonomies based on incident databases and media reports capture risks that have already occurred and were visible enough to be recorded, and are therefore less suited to identifying risks that develop gradually or have not yet produced visible harms.
“Third, taxonomies based on incident databases and media reports (Abercrombie et al. 2024;Turri and Dzombak 2023;Paeth et al. 2025) capture risks that have already occurred and were visible enough to be recorded.They are therefore less suited to identifying risks that develop gradually or have not yet produced visible harms, such as the erosion of professional skills caused by prolonged reliance on”
From Unaccountable Delegation, Fading Skills: Mapping the Risks of Workplace AI Agents (2026) · cites this paper as Turri and Dzombak 2023
While we did find multiple popular databases for recording and collating incidents of harm, these databases record harms after they have already occurred, often rely on second-hand reports, and may not delve deeply into causes of harms or impacts.
“While we did find multiple popular databases for recording and collating incidents of harm (such as the AI Incident Database), these databases record harms after they have already occurred, often rely on second-hand reports, and may not delve deeply into causes of harms or impacts [139].”
From Towards AI Accountability Infrastructure: Gaps and Opportunities in AI Audit Tooling (2025) · cites this paper as [139]
Cites
A Survey on Bias and Fairness in Machine Learning
Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., & Galstyan, A. (2022) · ACM Computing Surveys · doi:10.1145/3457607 · Cited by 5,812 · see citing works
On the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676 · see citing works
Preventable Anesthesia Mishaps
Cooper, J. B., Newbower, R. S., Long, C. D., & McPeek, B. (1978) · Anesthesiology · doi:10.1097/00000542-197812000-00004 · Cited by 929 · see citing works
Improving Fairness in Machine Learning Systems: What Do Industry Practitioners Need?
Holstein, K., Wortman Vaughan, J., Daumé, H., Dudik, M., & Wallach, H. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300830 · Cited by 777 · see citing works
Roselli, D., Matthews, J., & Talagala, N. (2019) · Companion Proceedings of The 2019 World Wide Web Conference · doi:10.1145/3308560.3317590 · Cited by 237 · see citing works
Analyzing aviation safety: Problems, challenges, opportunities
Oster, C. V., Strong, J. S., & Zorn, C. K. (2013) · Research in Transportation Economics · doi:10.1016/j.retrec.2012.12.001 · Cited by 201 · see citing works
Incident Reporting and Patient Safety
Vincent, C. (2007) · BMJ · doi:10.1136/bmj.39071.441609.80 · Cited by 161 · see citing works
Preventing repeated real world AI failures by cataloging incidents: the AI incident database
McGregor, S. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i17.17817 · Cited by 156 · see citing works
REVISE: A Tool for Measuring and Mitigating Bias in Visual Datasets
Wang, A., Liu, A., Zhang, R., Kleiman, A., Kim, L., Zhao, D., Shirai, I., Narayanan, A., & Russakovsky, O. (2022) · International Journal of Computer Vision · doi:10.1007/s11263-022-01625-5 · Cited by 85 · see citing works
Srinivasan, R., & Chander, A. (2021) · Queue · doi:10.1145/3466132.3466134 · Cited by 38 · see citing works
On managing vulnerabilities in AI/ML systems
Spring, J., Galyardt, A., Householder, A. D., & VanHoudnos, N. (2020) · New Security Paradigms Workshop · doi:10.1145/3442167.3442177 · Cited by 21 · see citing works
Flagged first, then by citations
1 of the 12 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Artificial intelligence risk management framework :
National Institute of Standards and Technology (US) (2024) · NIST Trustworthy and Responsible AI · doi:10.6028/nist.ai.600-1 · Cited by 274 · see citing works
Policy advice and best practices on bias and fairness in AI
Álvarez, J. M., Colmenarejo, A. B., Elobaid, A., Fabbrizzi, S., Fahimi, M., Ferrara, A., Ghodsi, S., Mougan, C., Papageorgiou, I. T., Lobo, P. R., et al. (2024) · Ethics and Information Technology · doi:10.1007/s10676-024-09746-w · Cited by 128 · see citing works
Towards AI Accountability Infrastructure: Gaps and Opportunities in AI Audit Tooling
Ojewale, V., Steed, R., Vecchione, B., Birhane, A., & Raji, I. D. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713301 · Cited by 81 · see citing works
A five-layer framework for AI governance: integrating regulation, standards, and certification
Agarwal, A., & Nene, M. J. (2025) · Transforming Government People Process and Policy · doi:10.1108/tg-03-2025-0065 · Cited by 56 · see citing works
Ananny, M. (2024) · Javnost - The Public · doi:10.1080/13183222.2024.2319000 · Cited by 20 · see citing works
Accountability and AI: Redundancy, Overlaps and Blind-Spots
Elliott, M. T. J., & MacCárthaigh, M. (2025) · Public Performance & Management Review · doi:10.1080/15309576.2025.2493889 · Cited by 17 · see citing works
Addressing AI Risks in Critical Infrastructure: Formalising the AI Incident Reporting Process
Agarwal, A., & Nene, M. J. (2024) · IEEE International Conference on Electronics, Computing and Communication Technologies (CONECCT) · doi:10.1109/conecct62155.2024.10677312 · Cited by 14 · see citing works
An HCAI Methodological Framework: Putting it Into Action to Enable Human-Centered AI
Xu, W., Gao, Z., & Dainoff, M. J. (2026) · IEEE Transactions on Human-Machine Systems · doi:10.1109/thms.2025.3631590 · Cited by 75 · see citing works
Leveraging Knowledge Graphs for AI System Auditing and Transparency
Waltersdorfer, L., & Sabou, M. (2024) · Journal of Web Semantics · doi:10.1016/j.websem.2024.100849 · Cited by 20 · see citing works
Exploring Research and Tools in AI Security: A Systematic Mapping Study
Narula, S., GhasemiGol, M., Carnerero-Cano, J., Minnich, A., Lupu, E., & Takabi, D. (2025) · IEEE Access · doi:10.1109/access.2025.3567195 · Cited by 15 · see citing works
Misinformation, Fraud, and Stereotyping: Towards a Typology of Harm Caused by Deepfakes
Trifonova, P., & Venkatagiri, S. (2024) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/3678884.3685938 · Cited by 11 · see citing works
Avinash, A., & Nene, M. J. (2024) · ITU Kaleidoscope Academic Conference (ITU K) · doi:10.23919/ituk62727.2024.10772925 · Cited by 8 · see citing works
Hadan, H., Mogavi, R. H., Zhang-Kennedy, L., & Nacke, L. E. (2025) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2025.2549073 · Cited by 10 · see citing works
Decoding Real-World Artificial Intelligence Incidents
Velázquez, J. D. M., Šćepanović, S., Gvirtz, A., & Quercia, D. (2024) · Computer · doi:10.1109/mc.2024.3432492 · Cited by 7 · see citing works
Standardised Schema and Taxonomy for AI Incident Databases in Critical Digital Infrastructure
Agarwal, A., & Nene, M. J. (2024) · IEEE Pune Section International Conference (PuneCon) · doi:10.1109/punecon63413.2024.10895867 · Cited by 6 · see citing works
Incorporating AI incident reporting into telecommunications law and policy: Insights from India
Agarwal, A., & Nene, M. J. (2026) · Computer law & security review · doi:10.1016/j.clsr.2026.106263 · Cited by 4 · see citing works
From Incidents to Insights: Patterns of Responsibility following AI Harms
Richards, I., Benn, C., & Zilka, M. (2025) · ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization (EAAMO) · doi:10.1145/3757887.3763018 · Cited by 6 · see citing works
Raghupathi, W., Ren, J., & Kulkarni, T. (2026) · AppliedMath · doi:10.3390/appliedmath6010011 · Cited by 2 · see citing works
Advancing AI Incidents Classification: Leveraging LLMs with Strategic Prompting
Chen, Y., Do, L., Yi, L., Baeza-Yates, R. A., & Guerra‐Gomez, J. A. (2025) · Communications in computer and information science · doi:10.1007/978-3-031-91328-0_4 · Cited by 2 · see citing works
AI Risk, Safety, and Incident Reporting
Paeth, K., & McGregor, S. (2025) · Springer eBooks · doi:10.1007/978-981-95-3658-0_89 · Cited by 1 · see citing works
Safeguarding AI-driven digital health – An adaptation of the Swiss cheese model for safety
Denecke, K. (2026) · Digital Health · doi:10.1177/20552076261459551 · Cited by 1 · see citing works
Measuring the Effectiveness of Ethics-Based Auditing – A Semi-Systematic Literature ReviewPreprint
Shafik, E., Puncheva, P., Bharathy, G., & Pannai, E. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5498160 · Cited by 1 · see citing works
Unaccountable Delegation, Fading Skills: Mapping the Risks of Workplace AI Agents
La Malfa, G., Meegahapola, L., Bogucka, E., Zhang, J. M., Luck, M., Black, E., & Quercia, D. (2026) · arXiv · doi:10.48550/arxiv.2608.08601
“Third, taxonomies based on incident databases and media reports (Abercrom-bie et al. 2024; Turri and Dzombak 2023; Paeth et al. 2025) capture risks that have already occurred and were visible enough to be recorded.”
Chessa, G. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7001915
From Concepts to Practice: Challenges of Risk-Benefit Assessment in AI Research & Applications
Eslami, M., Xie, S., Nikookar, N., Chang, M. L., Peng, C., Kim, J., Zimmerman, J., & Heidari, H. (2026) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3805689.3812216
AI Risk Mitigation in Finance: Resource Allocation Under Uncertainty and Information QualityPreprint
Lai, C. J. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7637378
A federated architecture for sector-led AI governance: lessons from India
Agarwal, A., & Nene, M. J. (2026) · Transforming Government: People, Process and Policy · doi:10.1108/tg-09-2025-0310
- Authors
- Turri, V., & Dzombak, R.
- Affiliations
- Software Engineering Institute (US)
- Venue
- Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society
- Publisher
- ACM
- Published
- 2023-08-08
- Pages
- 576-583
- ISSN
- 3065-8365 (ISSN-L)
- OpenAlex
- W4386242300
- Cited by
- 24
- References
- 12
- Citations by year
- 2026: 7 · 2025: 9 · 2024: 8
- FWCI
- 6.01
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Proceedings of the AAAI/ACM Conference on AI Ethics and Society (publishedVersion, cc-by)
- Topics
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Physical Sciences › Engineering › Safety, Risk, Reliability and Quality › Safety Systems Engineering in Autonomy
- Social Sciences › Law › Digitalization, Law, and Regulation
- Keywords
- AI harms, algorithmic bias, AI monitoring, AI safety, AI equity, AI Incident Database
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
