AI Documentation: A path to accountability
Open accessAbstract
Artificial Intelligence (AI) promises huge potential for businesses but due to its black-box character has also substantial drawbacks. This is a particular challenge in regulated use cases, where software needs to be certified or validated before deployment. Traditional software documentation is not sufficient to provide the required evidence to auditors and AI-specific guidelines are not available yet. Thus, AI faces significant adoption barriers in regulated use cases, since accountability of AI cannot be ensured to a sufficient extent. This interview study aims to determine the current state of documenting AI in regulated use cases. We found that the risk level of AI use cases has an impact on the AI adoption and the scope of AI documentation. Further, we discuss how AI is currently documented and which challenges practitioners face when documenting AI.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The composite and gradient nature of openness can be grounded directly in prior work on openness and accountability in AI systems.
“The composite and gradient nature of openness can be grounded directly in prior work on openness and accountability in AI systems [29, 37, 42, 58].”
From Rethinking open source generative AI: open washing and the EU AI Act (2024) · cites this paper as [29]
Work on AI safety and reproducibility has long pointed out the crucial importance of training data for understanding model performance, ensuring reproducibility, and assessing legal exposure.
“Work on AI safety and reproducibility has long pointed out the crucial importance of training data for understanding model performance, ensuring reproducibility, and assessing legal exposure [2, 25, 29, 46, 58].”
From Rethinking open source generative AI: open washing and the EU AI Act (2024) · cites this paper as [29]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
(2014) · Medical Entomology and Zoology · Cited by 6,828 · see citing works
The global landscape of AI ethics guidelines
Jobin, A., Ienca, M., & Vayena, E. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0088-2 · Cited by 6,220 · see citing works
Barocas, S., & Selbst, A. D. (2014) · doi:10.2139/ssrn.2477899 · Cited by 3,048 · see citing works
Model Cards for Model Reporting
Mitchell, M., Wu, S., Zaldivar, A., Barnes, P., Vasserman, L., Hutchinson, B., Spitzer, E., Raji, I. D., & Gebru, T. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287596 · Cited by 2,681 · see citing works
Revisiting Unreasonable Effectiveness of Data in Deep Learning Era
Sun, C., Shrivastava, A., Singh, S., & Gupta, A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.97 · Cited by 2,484 · see citing works
Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., Daumé, H., & Crawford, K. (2018) · arXiv · doi:10.48550/arxiv.1803.09010 · Cited by 2,474
Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., III, H. D., & Crawford, K. (2021) · Communications of the ACM · doi:10.1145/3458723 · Cited by 2,469 · see citing works
The Unreasonable Effectiveness of Data
Halevy, A. Y., Norvig, P., & Pereira, F. M. B. (2009) · IEEE Intelligent Systems · doi:10.1109/mis.2009.36 · Cited by 1,776 · see citing works
Documenting software architectures: views and beyond
Clements, P., Garlan, D., Little, R., Nord, R. L., & Stafford, J. A. (2003) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse.2003.1201264 · Cited by 1,434 · see citing works
Liu, X., Cruz Rivera, S., Moher, D., Calvert, M. J., Denniston, A. K., The SPIRIT-AI and CONSORT-AI Working Group, SPIRIT-AI and CONSORT-AI Steering Group, Chan, A.-W., Darzi, A., Holmes, C., et al. (2020) · Nature Medicine · doi:10.1038/s41591-020-1034-x · Cited by 1,421 · see citing works
Deep learning-enabled medical computer vision
Esteva, A., Chou, K., Yeung, S., Naik, N., Madani, A., Mottaghi, A., Liu, Y., Topol, E., Dean, J., & Socher, R. (2021) · npj Digital Medicine · doi:10.1038/s41746-020-00376-2 · Cited by 1,406 · see citing works
Raji, I. D., Smart, A., White, R. N., Mitchell, M., Gebru, T., Hutchinson, B., Smith-Loud, J., Theron, D., & Barnes, P. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372873 · Cited by 1,372 · see citing works
Credit rating analysis with support vector machines and neural networks: a market comparative study
Huang, Z., Chen, H., Hsu, C., Chen, W.-H., & Wu, S. (2003) · Decision Support Systems · doi:10.1016/s0167-9236(03)00086-1 · Cited by 1,015 · see citing works
Consumer credit-risk models via machine-learning algorithms
Khandani, A. E., Kim, A. J., & Lo, A. W. (2010) · Journal of Banking & Finance · doi:10.1016/j.jbankfin.2010.06.001 · Cited by 864 · see citing works
Improving Fairness in Machine Learning Systems: What Do Industry Practitioners Need?
Holstein, K., Wortman Vaughan, J., Daumé, H., Dudik, M., & Wallach, H. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300830 · Cited by 777 · see citing works
Accountability in algorithmic decision making
Diakopoulos, N. (2016) · Communications of the ACM · doi:10.1145/2844110 · Cited by 763 · see citing works
Cruz Rivera, S., Liu, X., Chan, A.-W., Denniston, A. K., Calvert, M. J., The SPIRIT-AI and CONSORT-AI Working Group, SPIRIT-AI and CONSORT-AI Steering Group, Darzi, A., Holmes, C., Yau, C., et al. (2020) · Nature Medicine · doi:10.1038/s41591-020-1037-7 · Cited by 748 · see citing works
Big data and analytics in the modern audit engagement: Research needs
Appelbaum, D., Kogan, A., & Vasarhelyi, M. A. (2017) · Auditing: A Journal of Practice & Theory · doi:10.2308/ajpt-51684 · Cited by 515 · see citing works
FactSheets: Increasing trust in AI services through supplier's declarations of conformity
Arnold, M., Bellamy, R. K. E., Hind, M., Houde, S., Mehta, S., Mojsilović, A., Nair, R., Ramamurthy, K. N., Olteanu, A., Piorkowski, D., et al. (2019) · IBM Journal of Research and Development · doi:10.1147/jrd.2019.2942288 · Cited by 434 · see citing works
Benbya, H., Pachidi, S., & Järvenpää, S. L. (2021) · Journal of the Association for Information Systems · doi:10.17705/1jais.00662 · Cited by 340 · see citing works
Königstorfer, F., & Thalmann, S. (2020) · Journal of Behavioral and Experimental Finance · doi:10.1016/j.jbef.2020.100352 · Cited by 299 · see citing works
Self-citationRisk and Risk Management in the Credit Card Industry
Butaru, F., Chen, Q., Clark, B., et al. (2015) · Journal of Banking & Finance · doi:10.1016/j.jbankfin.2016.07.015 · Cited by 259 · see citing works
Predicting mortgage default using convolutional neural networks
Kvamme, H., Sellereite, N., Aas, K., & Sjursen, S. (2018) · Expert Systems with Applications · doi:10.1016/j.eswa.2018.02.029 · Cited by 184 · see citing works
Raji, I. D., Smart, A., White, R. N., Mitchell, M., Gebru, T., Hutchinson, B., Smith-Loud, J., Theron, D., & Barnes, P. (2020) · arXiv · doi:10.48550/arxiv.2001.00973 · Cited by 179 · see citing works
Between Subjectivity and Imposition: Power Dynamics in Data Annotation for Computer Vision
Miceli, M., Schuessler, M., & Yang, T. (2020) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3415186 · Cited by 149 · see citing works
Mining Massive Fine-Grained Behavior Data to Improve Predictive Analytics1
Martens, D., Provost, F., Clark, J., & de Fortuny, E. J. (2016) · MIS Quarterly · doi:10.25300/misq/2016/40.4.04 · Cited by 140 · see citing works
Cost, benefits and quality of software development documentation: A systematic mapping
Zhi, J., Garousi-Yusifoğlu, V., Sun, B., Garousi, G., Shahnewaz, S., & Ruhe, G. (2015) · Journal of Systems and Software · doi:10.1016/j.jss.2014.09.042 · Cited by 112 · see citing works
Documenting Computer Vision Datasets
Miceli, M., Yang, T., Naudts, L., Schuessler, M., Serbanescu, D. A., & Hanna, A. (2021) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3442188.3445880 · Cited by 94 · see citing works
Causal Discovery in Manufacturing: A Structured Literature Review
Vuković, M., & Thalmann, S. (2022) · Journal of Manufacturing and Materials Processing · doi:10.3390/jmmp6010010 · Cited by 86 · see citing works
Self-citationUsage and usefulness of technical software documentation: An industrial case study
Garousi, G., Garousi, V., Ruhe, G., Zhi, J., Moussavi, M., & Smith, B. (2014) · Information and Software Technology · doi:10.1016/j.infsof.2014.08.003 · Cited by 58 · see citing works
Artificial Intelligence in Pathology: From Prototype to Product
Homeyer, A., Lotz, J., Schwen, L. O., Weiss, N., Romberg, D., Höfener, H., Zerbe, N., & Hufnagl, P. (2021) · Journal of Pathology Informatics · doi:10.4103/jpi.jpi_84_20 · Cited by 45 · see citing works
BBVA’s Data Monetization Journey
BBVA Group, Alfaro, E., Bressan, M., et al. (2019) · MIS Quarterly Executive · doi:10.17705/2msqe.00011 · Cited by 42 · see citing works
Improving cash logistics in bank branches by coupling machine learning and robust optimization
Lázaro, J. L., Jiménez, Á. B., & Takeda, A. (2017) · Expert Systems with Applications · doi:10.1016/j.eswa.2017.09.043 · Cited by 41 · see citing works
The future of software engineering
Burnett, M. (2010) · FSE/SDP Workshop on Future of Software Engineering Research (FoSER) · doi:10.1145/1882362.1882378 · Cited by 40 · see citing works
Unexpected challenges in large scale machine learning
Parker, C. (2012) · doi:10.1145/2351316.2351317 · Cited by 36 · see citing works
A software development process model for artificial neural networks in critical applications
Rodvold, D. M. (2003) · IJCNN'99. International Joint Conference on Neural Networks. Proceedings (Cat. No.99CH36339) · doi:10.1109/ijcnn.1999.836192 · Cited by 34 · see citing works
Regulating artificial intelligence
Ellul, J., Pace, G. J., McCarthy, S., Sammut, T., Brockdorff, J., & Scerri, M. (2021) · International Conference on Artificial Intelligence and Law · doi:10.1145/3462757.3466093 · Cited by 31 · see citing works
Data Analytics for Industrial Process Improvement A Vision Paper
Thalmann, S., Mangler, J., Schreck, T., et al. (2018) · IEEE Conference on Business Informatics · doi:10.1109/cbi.2018.10051 · Cited by 25 · see citing works
Self-citationSoftware documentation is not enough! Requirements for the documentation of AI
Königstorfer, F., & Thalmann, S. (2021) · Digital Policy Regulation and Governance · doi:10.1108/dprg-03-2021-0047 · Cited by 25 · see citing works
Self-citationGashi, M., Vuković, M., Jekic, N., Thalmann, S., Holzinger, A., Jean-Quartier, C., & Jeanquartier, F. (2022) · BioMedInformatics · doi:10.3390/biomedinformatics2010009 · Cited by 25 · see citing works
Self-citationImproving wind power prediction with retraining machine learning algorithms
Barque, M., Martin, S., Vianin, J., et al. (2018) · International Workshop on Big Data and Information Security (IWBIS) · doi:10.1109/iwbis.2018.8471713 · Cited by 21 · see citing works
Auditing service providers: supporting auditors in cross-organizational settings
Bachlechner, D., Thalmann, S., & Manhart, M. (2014) · Managerial Auditing Journal · doi:10.1108/maj-05-2013-0861 · Cited by 18 · see citing works
Self-citationSoftware Documentation Management Issues and Practices: A Survey
Satish, C. J., & Mahendran, A. (2016) · Indian Journal of Science and Technology · doi:10.17485/ijst/2016/v9i20/86869 · Cited by 15 · see citing works
Dealing with missing usage data in defect prediction: A case study of a welding supplier
Gashi, M., Ofner, P., Ennsbrunner, H., & Thalmann, S. (2021) · Computers in Industry · doi:10.1016/j.compind.2021.103505 · Cited by 15 · see citing works
Self-citationPolzer, A. K., Fleiß, J., Ebner, T., Kainz, P., Koeth, C., & Thalmann, S. (2022) · Hawaii International Conference on System Sciences (HICSS) · doi:10.24251/hicss.2022.186 · Cited by 9 · see citing works
Self-citationA Network-based Tool for Identifying Knowledge Risks in Data-Driven Business Models
Fruhwirth, M., Pammer‐Schindler, V., & Thalmann, S. (2021) · Proceedings of the ... Annual Hawaii International Conference on System Sciences/Proceedings of the Annual Hawaii International Conference on System Sciences · doi:10.24251/hicss.2021.636 · Cited by 8 · see citing works
Self-citationMEDEP: Maintenance Event Detection for Multivariate Time Series Based on the PELT Approach
Gashi, M., Gursch, H., Hinterbichler, H., Pichler, S., Lindstaedt, S., & Thalmann, S. (2022) · Sensors · doi:10.3390/s22082837 · Cited by 8 · see citing works
Self-citationISO standards for software user documentation
Hayhoe, G. F. (2012) · IEEE International Professional Communication Conference (ProComm) · doi:10.1109/ipcc.2012.6408631 · Cited by 4 · see citing works
Flagged first, then by citations
Research works printed but not linked: 1
- Smit, K., Zoet, M. and van Meerten, J. (eds) (2020) A review of AI principles in practice.
Cited by
Sorted by citations
Lekadir, K., Frangi, A. F., Porras, A. R., Glocker, B., Cintas, C., Langlotz, C. P., Weicken, E., Asselbergs, F. W., Prior, F., Collins, G. S., et al. (2025) · BMJ · doi:10.1136/bmj-2024-081554 · Cited by 957 · see citing works
“The traceability principle states that medical AI tools should be developed together with mechanisms for documenting and monitoring the complete trajectory of the AI tool, from development and validation to deployment and usage.”
Wach, K., Duong, C. D., Ejdys, J., Kazlauskaitė, R., Korzynski, P., Mazurek, G., Paliszkiewicz, J., & Ziemba, E. (2023) · Entrepreneurial Business and Economics Review · doi:10.15678/eber.2023.110201 · Cited by 584 · see citing works
Rethinking open source generative AI: open washing and the EU AI Act
Liesenfeld, A., & Dingemanse, M. (2024) · The 2024 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3630106.3659005 · Cited by 67 · see citing works
Tan, M., & Subramonyam, H. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642592 · Cited by 40 · see citing works
- Authors
- Königstorfer, F., & Thalmann, S.
- Affiliations
- University of Graz (AT)
- Corresponding author
- Königstorfer, Florian
- Venue
- Journal of Responsible Technology
- Publisher
- Elsevier BV
- Published
- 2022-10
- Volume
- 11
- Pages
- 100043
- ISSN
- 2666-6596 (ISSN-L)
- OpenAlex
- W4289832037
- Cited by
- 55
- References
- 48
- Citations by year
- 2026: 12 · 2025: 18 · 2024: 19 · 2023: 6
- FWCI
- 3.54
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Journal of Responsible Technology (publishedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Karl-Franzens-Universität Graz
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Physical Sciences › Engineering › Safety, Risk, Reliability and Quality › Safety Systems Engineering in Autonomy
- Keywords
- AI documentation, accountability, AI adoption barriers, black-box AI, software certification, AI validation, audit evidence, responsible AI
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
