The DOI matches its Crossref record; 42 of the 44 references it cites were found in a registry or an index.
FactSheets: Increasing trust in AI services through supplier's declarations of conformity
Open accessAbstract
Accuracy is an important concern for suppliers of artificial intelligence (AI) services, but considerations beyond accuracy, such as safety (which includes fairness and explainability), security, and provenance, are also critical elements to engender consumers’ trust in a service. Many industries use transparent, standardized, but often not legally required documents called supplier's declarations of conformity (SDoCs) to describe the lineage of a product along with the safety and performance testing it has undergone. SDoCs may be considered multidimensional fact sheets that capture and quantify various aspects of the product and its development to make it worthy of consumers’ trust. In this article, inspired by this practice, we propose FactSheets to help increase trust in AI services. We envision such documents to contain purpose, performance, safety, security, and provenance information to be completed by AI service providers for examination by consumers. We suggest a comprehensive set of declaration items tailored to AI in the Appendix of this article.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“AI documentation is advocated to increase the transparency and facilitate understanding of AI [35,36,40,75,76,82], and is embodied in recently proposed features of AI Factsheets [8,81] and Model Cards [64].”
From Investigating Explainability of Generative AI for Code through Scenario-based Design (2022) · cites this paper as [8]
In the citing paper
“There is some work on fairness relating to AI services, though this tends to focus on either exposing or 'auditing' certain AI services [22,38,126,133], or devising tools, methods and interventions relevant for those building (rather than using) such services [12,96,106,128].”
From Out of Context: Investigating the Bias and Fairness Concerns of “Artificial Intelligence as a Service” (2023) · cites this paper as [12]
In the citing paper
“There appears a clear role for adopting and extending various documentation approaches, be they relating to data and models (such as datasheets [53], model cards [106] and factsheets [12]), their interactions and interconnections (e.g.decision provenance [139]), and broader, holistic socio-technical system reviews (e.g.reviewability [27] and traceability [86]).”
From Out of Context: Investigating the Bias and Fairness Concerns of “Artificial Intelligence as a Service” (2023) · cites this paper as [12]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The Market for "Lemons": Quality Uncertainty and the Market Mechanism
Akerlof, G. A. (1970) · The Quarterly Journal of Economics · doi:10.2307/1879431 · Cited by 23,016
Axford, N. (2017) · Routledge eBooks · doi:10.4324/9781315258324-16 · Cited by 11,329
Statsmodels: Econometric and Statistical Modeling with Python
Seabold, S., & Perktold, J. (2010) · Proceedings of the Python in Science Conferences · doi:10.25080/majora-92bf1922-011 · Cited by 6,968
Another look at measures of forecast accuracy
Hyndman, R. J., & Koehler, A. B. (2006) · International Journal of Forecasting · doi:10.1016/j.ijforecast.2006.03.001 · Cited by 5,757
The Limitations of Deep Learning in Adversarial Settings
Papernot, N., McDaniel, P., Jha, S., Fredrikson, M., Celik, Z. B., & Swami, A. (2016) · 2016 IEEE European Symposium on Security and Privacy (EuroS&P) · doi:10.1109/eurosp.2016.36 · Cited by 3,993
A survey on concept drift adaptation
Gama, J., Žliobaitė, I., Bifet, A., Pechenizkiy, M., & Bouchachia, A. (2014) · ACM Computing Surveys · doi:10.1145/2523813 · Cited by 3,735
Towards A Rigorous Science of Interpretable Machine Learning
Doshi-Velez, F., & Kim, B. (2017) · arXiv · doi:10.48550/arxiv.1702.08608 · Cited by 3,121
Barocas, S., & Selbst, A. D. (2014) · doi:10.2139/ssrn.2477899 · Cited by 3,048
Model Cards for Model Reporting
Mitchell, M., Wu, S., Zaldivar, A., Barnes, P., Vasserman, L., Hutchinson, B., Spitzer, E., Raji, I. D., & Gebru, T. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287596 · Cited by 2,681 · cited 2×
“…et al. proposed model cards to convey information that characterizes the evaluation of a machine learning model in a variety of conditions and disclose the context in which models are intended to be used, details of the performance evaluation procedures, and other relevant information [11].”
BackgroundMethodInfluentialGebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., Daumé, H., & Crawford, K. (2018) · arXiv · doi:10.48550/arxiv.1803.09010 · Cited by 2,474
Flagged first, then by citations
2 of the 44 references couldn’t be matched to a known paper.
Research works printed but not linked: 1
- varshney, 0, Data science of the people, for the people, by the people: A viewpoint on an emerging dichotomy, Proc Data for Good Exchange Conf
Cited by
Sorted by citations
Items 201–203 of 203
Human-Centered Explainable AI (XAI): From Algorithms to User Experiences
Liao, Q. V., & Varshney, K. R. (2021) · arXiv · doi:10.48550/arxiv.2110.10790 · Cited by 120
Self-citationResponsible AI by Design in Practice
Benjamins, R., Barbado, A., & Sierra, D. (2019) · arXiv · doi:10.48550/arxiv.1909.12838 · Cited by 38
A Methodology for Creating AI FactSheets
Richards, J., Piorkowski, D., Hind, M., Houde, S., & Mojsilović, A. (2020) · arXiv · doi:10.48550/arxiv.2006.13796 · Cited by 28
Self-citation
Metadata
- Authors
- Arnold, M., Bellamy, R. K. E., Hind, M., Houde, S., Mehta, S., Mojsilović, A., Nair, R., Ramamurthy, K. N., Olteanu, A., Piorkowski, D., Reimer, D., Richards, J., Tsay, J., & Varshney, K. R.
- Venue
- IBM Journal of Research and Development
- Publisher
- IBM
- Published
- 2019-07-01
- Volume
- 63
- Issue
- 4/5
- Pages
- 6:1-6:13
- ISSN
- 0018-8646 (ISSN-L)
- OpenAlex
- W2911599914
- Cited by
- 434
- References
- 44
- Citations by year
- 2026: 47 · 2025: 44 · 2024: 62 · 2023: 93 · 2022: 58 · 2021: 78 · 2020: 32 · 2019: 18
- FWCI
- 81.25
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IBM Journal of Research and Development (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Engineering › Safety, Risk, Reliability and Quality › Safety Systems Engineering in Autonomy
- Keywords
- trust in AI, AI safety, AI security, AI provenance, algorithmic fairness, explainable AI, consumer trust in AI
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
