Auditing the AI auditors: A framework for evaluating fairness and bias in high stakes AI predictive models.
Open accessThe DOI matches its Crossref record.
Abstract
as a standardized approach for evaluating the fairness and bias of AI systems that make predictions about humans across disciplinary perspectives. We present 12 crucial components to audits across three categories: (a) components related to AI models in terms of their source data, design, development, features, processes, and outputs, (b) components related to how information about models and their applications are presented, discussed, and understood from the perspectives of those employing the algorithm, those affected by decisions made using its predictions, and third-party observers, and (c) meta-components that must be considered across all other auditing components, including cultural context, respect for persons, and the integrity of individual research designs used to support all model developer claims. (PsycInfo Database Record (c) 2023 APA, all rights reserved).
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Frameworks such as the one developed by Landers and Behrend [91] comprehensively outline questions that would be asked at each stage of AI development.”
From Barriers to and facilitators of artificial intelligence adoption in health care: scoping review (2024) · cites this paper as [91]
In the citing paper
“The impact of GenAI on digital equity depends on its implementation: if equitable access to these technologies is not guaranteed, the digital divide could widen (Landers and Behrend, 2023).”
From Ethical and regulatory challenges of Generative AI in education: a systematic review (2025) · cites this paper as Landers and Behrend, 2023
In the citing paper
“This has caused widespread concern about conceptual confusion in the field (Landers & Behrend, 2022).”
From Auditing of AI: Legal, Ethical and Technical Approaches (2023) · cites this paper as Landers & Behrend, 2022
In the citing paper
“Finally, a small but growing community of researchers are interested in how feasible and effective auditing is as an AI system governance mechanism (Costanza-Chock et al., 2022;Landers & Behrend, 2022).”
From Auditing of AI: Legal, Ethical and Technical Approaches (2023) · cites this paper as Landers & Behrend, 2022
Citations
Cites
Still being checked.
Cited by
Items 201–233 of 233
García-López, I. M., Espinosa, J. M. M., & Ramírez-Montoya, M. S. (2026) · Journal of Interactive Media in Education · doi:10.5334/jime.1051
Beyond the Black Box: Equipping Users to Recognize and Challenge AI Bias in Hiring
Volini, E., Lemmon, G., & Hwang, Y. (2026) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2026.2650563 · Cited by 1
Tugwell, O. O., & Uro, E. E. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/407441
Achieving fair AI in Africa? Ideas from the African charter on human and peoples’ rights
Jimoh, M. A. (2026) · Science and Public Policy · doi:10.1093/scipol/scag008
How to Get Started with Artificial Intelligence-Data Science in Industrial-Organizational Psychology
Doshi, D. V., Albritton, B. H., Bradley, J. A., & Tonidandel, S. (2026) · doi:10.1093/9780197807309.003.0020
Health and Stress and Artificial Intelligence
Shoss, M. K., Behrend, T. S., Crandell, H. A., & Tracy, M. M. (2026) · Oxford University Press eBooks · doi:10.1093/9780197807309.003.0015
Self-citationImplications of Artificial Intelligence for Workplace Monitoring and Privacy
Blachly, B. D., Kyeremateng, T. K., Shafiei, M., & Ravid, D. M. (2026) · Oxford University Press eBooks · doi:10.1093/9780197807309.003.0017
Navigating the Labyrinth: Essential Conditions for Generative Artificial Intelligence Governance
Wu, Y., & Lin, C. (2026) · IT Professional · doi:10.1109/mitp.2026.3663787 · Cited by 1
Fairness at Risk: Where Bias Emerges in Machine Learning
Albuquerque, O. D. P., Fantinato, M., & Peres, S. M. (2026) · Expert Systems · doi:10.1111/exsy.70265 · Cited by 1
Grenzebach, J., & Radüntz, T. (2026) · Communications in computer and information science · doi:10.1007/978-3-032-25308-8_7
Obschonka, M., Fisch, C. O., Beliaeva, T., Jabeur, S. B., Fernando, T., & Fookes, C. (2026) · Small Business Economics · doi:10.1007/s11187-026-01221-8
Personnel Psychology's 40 Questions Series: Artificial Intelligence
Campion, E. D., & Tonidandel, S. (2026) · Personnel Psychology · doi:10.1111/peps.70031
Sakowska, K. (2026) · Springer proceedings in business and economics · doi:10.1007/978-981-95-6415-6_2
Neutral or Positive? Understanding Finance Professionals' Attitudes Toward AI Adoption
Hu, X., Veenstra, K., & Yoon, C. E. (2026) · Accounting and Finance · doi:10.1111/acfi.70247
Billingsley, F. (2026) · IGI Global eBooks · doi:10.4018/979-8-3373-1509-6.ch008
Kantrowitz, T. M., & Segall, D. O. (2026) · Oxford University Press eBooks · doi:10.1093/9780197809013.003.0033
Algorithmic Accountability in Greenwashing Detection: Hybrid Governance in ESG Auditing
Harke, G. (2026) · IGI Global eBooks · doi:10.4018/979-8-3373-8392-7.ch008
An explainable fairness-aware AI framework for exam score verification
Olaniyan, J., Verkijika, S. F., & Obagbuwa, I. C. (2026) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2026.1824849
AI Tools for Energy Management: Revolutionising Smart Grids, Microgrids, Homes and Renewables
Ali, S. M., Khan, B., Ullah, K., Ullah, Z., Mehmood, C. A., Gulfam, S. M., & Gruosso, G. (2026) · International Journal of Energy Research · doi:10.1155/er/7310134 · Cited by 1
Sun, T., Oswald, F. L., & Hough, L. M. (2026) · Oxford University Press eBooks · doi:10.1093/9780197809013.003.0012
Person–Environment Fit and Organizational Selection
Kristof‐Brown, A. L., Yang, Y., & Sekiguchi, T. (2026) · Oxford University Press eBooks · doi:10.1093/9780197809013.003.0013
Borman, W., Pulakos, E. D., & Raghavan, M. (2026) · Oxford University Press eBooks · doi:10.1093/9780197809013.003.0025
Quantifying AI impact in energy transitions: The Energy Justice Impact Assessment (EJIA) frameworkPreprint
Bringmann, E., Kutzner, F., & Kacperski, C. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7352634 · Cited by 1
Viewing Neurotech Through the Lens of the Ethical Principles: Considerations and Calls to Action
Hilsabeck, R. C., & Benge, J. F. (2026) · Oxford University Press eBooks · doi:10.1093/9780197814116.003.0004
Amor, R. B., & Lajmi, A. (2026) · International Journal of Ethics and Systems · doi:10.1108/ijoes-04-2026-0299 · Cited by 1
Yuan, T., & Rau, P. P. (2026) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2026.103939
Kordoni, A., Sachoulidou, A., & de Oliveira, C. R. (2026) · Cambridge University Press eBooks · doi:10.1017/9781009623605.006
Halim, J. A., Radika, M. A. W., Diana, & Priccilia, S. (2026) · International Conference on ICT for Smart Society (ICISS) · doi:10.1109/iciss71466.2026.11715458
Farina, M., Lavazza, A., & Kladko, S. (2026) · AI and Ethics · doi:10.1007/s43681-026-01423-6
Wojda, T. R., & Stawicki, S. P. (2026) · Artificial intelligence · doi:10.5772/intechopen.1016859
Rebocho, A. T., Marques, S. S., Santos, J. P., & Pinto, C. S. (2026) · Frontiers in Digital Health · doi:10.3389/fdgth.2026.1888744
Metadata
- Authors
- Landers, R. N., & Behrend, T. S.
- Venue
- American Psychologist
- Published
- 2023-01
- Volume
- 78
- Issue
- 1
- Pages
- 36-49
- ISSN
- 0003-066X (ISSN-L)
- 1935-990X
- OpenAlex
- W4213278005
- Cited by
- 233
- References
- 0
- Citations by year
- 2026: 52 · 2025: 60 · 2024: 71 · 2023: 37
- FWCI
- 52.41
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- American Psychologist (publishedVersion, other-oa)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Social Sciences › Business, Management and Accounting › Organizational Behavior and Human Resource Management › AI and HR Technologies
- Keywords
- artificial intelligence in auditing, algorithmic fairness, algorithmic bias, prediction model, high-stakes decision making, evaluation framework, third-party observers, cultural context, respect for persons, research integrity, algorithmic accountability
- MeSH terms
- Artificial Intelligence, Humans
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.