The DOI matches its Crossref record.
Detecting Systematic Deviations in Data and Models
Open accessAbstract
Trustworthy artificial intelligence researchers should seek to better detect and characterize systematic deviations in data and models (that is, bias). This article provides data scientists with motivation, theory, code, and examples on how to perform disciplined discovery of systematic deviations in data and models at the subset level.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In both cases, AI maintenance frameworks can detect model degradation through time by detecting systematic deviations of the aforementioned metrics in data and models [34].”
From Connecting the dots in trustworthy Artificial Intelligence: From AI principles, ethics, and key requirements to responsible AI systems and regulation (2023) · cites this paper as [34]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
No references could be resolved yet.
Research works printed but not linked: 19
- 10.1145/170036.170072 · DOI · Paper page
- Agrawal, 1994, Fast algorithms for mining association rules in large databases, 487
- Bellamy, 2018, AI fairness 360: An extensible toolkit for detecting, understanding, and mitigating unwanted algorithmic bias
- 10.1145/253262.253327 · DOI
- 10.1016/j.patrec.2021.12.007 · DOI
- 10.1016/s2214-109x(19)30261-x · DOI
- 10.1007/springerreference_205144 · DOI
- 10.1093/jamia/ocaa196 · DOI · Paper page
- Idrees, 2021, Successes and misses of global health development: Detecting temporal concept drift of under-5 mortality prediction models with bias scan, 2021, 286
- Kohavi, 1996, Scaling up the accuracy of naive-bayes classifiers: A decision-tree hybrid, 96, 202
- 10.2139/ssrn.3155352 · DOI
- 10.1111/j.1467-9868.2011.01014.x · DOI
- 10.1002/sim.5675 · DOI
- Ogallo, 2021, Detection of anomalous patterns associated with the impact of medications on 30-day hospital readmission rates in diabetes care, 2021, 495
- Piatetsky-Shapiro, 1991, Discovery, analysis, and presentation of strong rules, 229
- Simon, 1979, Rational decision making in business organizations, Amer. Econ. Rev., 69, 493
- 10.1109/69.149926 · DOI
- 10.1080/10618600.2015.1029578 · DOI
- 10.2307/2682991 · DOI
Cited by
Sorted by citations
Díaz-Rodríguez, N., Del Ser, J., Coeckelbergh, M., López de Prado, M., Herrera-Viedma, E., & Herrera, F. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.101896 · Cited by 800
Metadata
- Authors
- Speakman, S., Tadesse, G. A., Cintas, C., Ogallo, W., Akumu, T., & Oshingbesan, A.
- Affiliations
- IBM Research - Africa (ZA)
- IBM (United States) (US)
- ORCID
- Venue
- Computer
- Published
- 2023-02
- Volume
- 56
- Issue
- 2
- Pages
- 82-92
- ISSN
- 0018-9162 (ISSN-L)
- 1558-0814
- OpenAlex
- W4319879392
- Cited by
- 9
- References
- 0
- Citations by year
- 2025: 3 · 2024: 4 · 2023: 2
- FWCI
- 2.26
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Computer (publishedVersion, cc-by-nc-nd)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
- Keywords
- systematic deviations, bias detection, data bias, model bias, trustworthy AI
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
