Considerations for AI fairness for people with disabilities
Abstract
In society today, people experiencing disability can face discrimination. As artificial intelligence solutions take on increasingly important roles in decision-making and interaction, they have the potential to impact fair treatment of people with disabilities in society both positively and negatively. We describe some of the opportunities and risks across four emerging AI application areas: employment, education, public safety, and healthcare, identified in a workshop with participants experiencing a range of disabilities. In many existing situations, non-AI solutions are already discriminatory, and introducing AI runs the risk of simply perpetuating and replicating these flaws. We next discuss strategies for supporting fairness in the context of disability throughout the AI development lifecycle. AI systems should be reviewed for potential impact on the user in their broader context of use. They should offer opportunities to redress errors, and for users and those impacted to raise fairness concerns. People with disabilities should be included when sourcing data to build models, and in testing, to create a more inclusive and robust system. Finally, we offer pointers into an established body of literature on human-centered design processes and philosophies that may assist AI and ML engineers in innovating algorithms that reduce harm and ultimately enhance the lives of people with disabilities.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
These discriminatory issues are primarily a reflection of training set deficiencies, namely the underrepresentation of certain parts of the input space and the lack of awareness, consideration and attention of the people building these systems.
“Many of these works conclude that these issues are primarily a refection of training set defciencies, namely the underrepresentation of certain parts of the input space [15,22] and the lack of awareness, consideration and attention of the people building these systems [97,136,146].”
From Out of Context: Investigating the Bias and Fairness Concerns of “Artificial Intelligence as a Service” (2023) · cites this paper as [146]
Cites
Ries, E. (2011) · Cited by 3,379 · see citing works
Fairness and Abstraction in Sociotechnical Systems
Selbst, A. D., Boyd, D., Friedler, S. A., Venkatasubramanian, S., & Vertesi, J. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287598 · Cited by 1,631 · see citing works
Bellamy, R. K. E., Dey, K., Hind, M., Hoffman, S. C., Houde, S., Kannan, K., Lohia, P., Martino, J., Mehta, S., Mojsilovic, A., et al. (2018) · arXiv · doi:10.48550/arxiv.1810.01943 · Cited by 1,581 · see citing works
Bardzell, S. (2010) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/1753326.1753521 · Cited by 1,139 · see citing works
REGULATION (EU) 2019/518 OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL
de Farmacoterapia, R. P. (2015) · doi:10.5040/9781782258674.0010 · Cited by 1,104 · see citing works
Persons with disabilities as an unrecognized health disparity population
Krahn, G. L., Walker, D. K., & Correa-De-Araujo, R. (2015) · American Journal of Public Health · doi:10.2105/ajph.2014.302182 · Cited by 1,080 · see citing works
A cascade of disparities: Health and health care access for people with intellectual disabilities
Krahn, G. L., Hammond, L., & Turner, A. (2006) · Mental Retardation and Developmental Disabilities Research Reviews · doi:10.1002/mrdd.20098 · Cited by 775 · see citing works
Computers and Democracy: A Scandinavian Challenge
Bjerknes, G., Ehn, P., Kyng, M., & Nygaard, K. (1987) · Medical Entomology and Zoology · Cited by 732 · see citing works
Lepri, B., Oliver, N., Letouzé, E., Pentland, A., & Vinck, P. (2018) · Philosophy & Technology · doi:10.1007/s13347-017-0279-x · Cited by 709 · see citing works
Wobbrock, J. O., Kane, S. K., Gajos, K. Z., Harada, S., & Froehlich, J. E. (2011) · ACM Transactions on Accessible Computing · doi:10.1145/1952383.1952384 · Cited by 590 · see citing works
Data preparation for data mining
Zhang, S., Zhang, C., & Yang, Q. (2003) · Applied Artificial Intelligence · doi:10.1080/713827180 · Cited by 521 · see citing works
The Misgendering Machines: Trans/HCI Implications of Automatic Gender Recognition
Keyes, O. (2018) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3274357 · Cited by 505 · see citing works
The Human–Computer Interaction Handbook
(2012) · CRC Press eBooks · doi:10.1201/b11963 · Cited by 468 · see citing works
Pradhan, A., Mehta, K., & Findlater, L. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174033 · Cited by 448 · see citing works
A Survey of Value Sensitive Design Methods
Friedman, B., Hendry, D. G., & Borning, A. (2017) · Foundations and Trends® in Human–Computer Interaction · doi:10.1561/1100000015 · Cited by 389 · see citing works
Values as lived experience: evolving value sensitive design in support of value discovery
Dantec, C. A. L., Poole, E., & Wyche, S. (2009) · International Conference on Human Factors in Computing Systems · doi:10.1145/1518701.1518875 · Cited by 386 · see citing works
Next steps for value sensitive design
Borning, A., & Muller, M. (2012) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/2207676.2208560 · Cited by 365 · see citing works
Self-citationThe challenges and limits of big data algorithms in technocratic governance
Janssen, M., & Kuk, G. (2016) · Government Information Quarterly · doi:10.1016/j.giq.2016.08.011 · Cited by 363 · cited 1× · see citing works
“Owing to intended or unintended bias in the data, the model may inadvertently enforce or reinforce discriminatory patterns that work against people with disabilities (Janssen & Kuk, 2016).”
BackgroundHealth Disparities of Adults with Intellectual Disabilities: What Do We Know? What Do We Do?
Krahn, G. L., & Fox, M. H. (2013) · Journal of Applied Research in Intellectual Disabilities · doi:10.1111/jar.12067 · Cited by 313 · see citing works
Iezzoni, L. I. (2011) · Health Affairs · doi:10.1377/hlthaff.2011.0613 · Cited by 298 · see citing works
The Disability Employment Puzzle: A Field Experiment on Employer Hiring Behavior
Ameri, M., Schur, L. A., Adya, M., Bentley, F. S., McKay, P. F., & Kruse, D. L. (2017) · Industrial and Labor Relations Review · doi:10.1177/0019793917717474 · Cited by 298 · see citing works
Perspectives on Disability Disclosure: The Importance of Employer Practices and Workplace Climate
von Schrader, S., Malzer, V., & Bruyère, S. (2014) · Employee Responsibilities and Rights Journal · doi:10.1007/s10672-013-9227-9 · Cited by 263 · see citing works
Automatically generating user interfaces adapted to users' motor and vision capabilities
Gajos, K. Z., Wobbrock, J. O., & Weld, D. S. (2007) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/1294211.1294253 · Cited by 242 · see citing works
Gender Recognition or Gender Reductionism?
Hamidi, F., Scheuerman, M. K., & Branham, S. M. (2018) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173582 · Cited by 222 · see citing works
Self-citationHow Algorithms Discriminate Based on Data They Lack: Challenges, Solutions, and Policy Implications
Williams, B. A., Brooks, C., & Shmargad, Y. (2018) · Journal of Information Policy · doi:10.5325/jinfopoli.8.2018.0078 · Cited by 203 · see citing works
A journey of hope: lessons learned from studies on rare diseases and orphan drugs
Wästfelt, M., Fadeel, B., & Henter, J. (2006) · Journal of Internal Medicine · doi:10.1111/j.1365-2796.2006.01666.x · Cited by 184 · see citing works
Online e-learning and cognitive disabilities: A systematic review
Cinquin, P.-A., Guitton, P., & Sauzeon, H. (2018) · Computers & Education · doi:10.1016/j.compedu.2018.12.004 · Cited by 181 · see citing works
Deaf sign language users, health inequities, and public health: opportunity for social justice.
Barnett, S. L., McKee, M. M., Smith, S. R., et al. (2011) · PubMed · Cited by 173 · see citing works
Hankerson, D., Marshall, A., Booker, J., Elmimouni, H., Walker, I., & Rode, J. A. (2016) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/2851581.2892578 · Cited by 137 · see citing works
Bias Mitigation Post-processing for Individual and Group Fairness
Lohia, P., Ramamurthy, K. N., Bhide, M. A., Saha, D., Varshney, K. R., & Puri, R. (2019) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2019.8682620 · Cited by 135 · see citing works
Making Inclusion Work in General Education Classrooms
Obiakor, F. E., Harris, M. K., Mutua, K., et al. (2012) · Education and Treatment of Children · doi:10.1353/etc.2012.0020 · Cited by 133 · see citing works
Toward fairness in AI for people with disabilities SBG@a research roadmap
Guo, A., Kamar, E., Vaughan, J. W., Wallach, H., & Morris, M. R. (2020) · ACM SIGACCESS Accessibility and Computing · doi:10.1145/3386296.3386298 · Cited by 128 · see citing works
Two Perspectives On Inclusion in The United States
Dudley‐Marling, C., & Burns, M. B. (2013) · Global education review · Cited by 108 · see citing works
Reframing Assistive Robots to Promote Successful Aging
Lee, H. R., & Riek, L. D. (2018) · ACM Transactions on Human-Robot Interaction · doi:10.1145/3203303 · Cited by 106 · see citing works
Chander, A. (2017) · Michigan Law Review · doi:10.36644/mlr.115.6.racist · Cited by 105 · see citing works
Designing technology for and with developmentally diverse children
Börjesson, P., Barendregt, W., Eriksson, E. W., et al. (2015) · ACM International Conference on Interaction Design and Children (IDC) · doi:10.1145/2771839.2771848 · Cited by 104 · see citing works
(In)Visibility in Disability and Assistive Technology
Faucett, H. A., Ringland, K. E., Cullen, A. L. L., & Hayes, G. R. (2017) · ACM Transactions on Accessible Computing · doi:10.1145/3132040 · Cited by 101 · see citing works
Opening Doors or Slamming Them Shut? Online Learning Practices and Students with Disabilities
Burgstahler, S. E. (2015) · Social Inclusion · doi:10.17645/si.v3i6.420 · Cited by 91 · see citing works
The Assessment of Attitudes Toward Individuals With Disabilities in the Workplace
Popovich, P. M., Scherbaum, C. A., Scherbaum, K. L., & Polinko, N. K. (2003) · The Journal of Psychology · doi:10.1080/00223980309600606 · Cited by 84 · see citing works
Designing interaction, voice, and inclusion in AAC research
Pullin, G., Treviranus, J., Patel, R. M., & Higginbotham, D. J. (2017) · Augmentative and Alternative Communication · doi:10.1080/07434618.2017.1342690 · Cited by 74 · see citing works
Self-citationUnderstanding the Power of Control in Autonomous Vehicles for People with Vision Impairment
Brewer, R. N., & Kameswaran, V. (2018) · Proceedings of the 20th International ACM SIGACCESS Conference on Computers and Accessibility · doi:10.1145/3234695.3236347 · Cited by 72 · see citing works
Storni, C. (2010) · International Journal of Human-Computer Interaction · doi:10.1080/10447311003720001 · Cited by 66 · see citing works
Pollard, R. Q. (1994) · Rehabilitation Psychology · doi:10.1037/h0080318 · Cited by 64 · see citing works
Three tensions in participatory design for inclusion
Holone, H., & Herstad, J. (2013) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2470654.2481401 · Cited by 63 · see citing works
"Is Someone There? Do They Have a Gun"
Branham, S. M., Abdolrahmani, A., Easley, W., Scheuerman, M. K., Ronquillo, E., & Hurst, A. K. (2017) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/3132525.3132534 · Cited by 53 · see citing works
Self-citationKatan, S., Grierson, M., & Fiebrink, R. (2015) · Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702474 · Cited by 50 · see citing works
Incorporating Social Factors in Accessible Design
Shinohara, K., Wobbrock, J. O., & Pratt, W. (2018) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/3234695.3236346 · Cited by 50 · see citing works
Values, Identity, and Social Translucence
Zolyomi, A., Ross, A. S., Bhattacharya, A., Milne, L. R., & Munson, S. A. (2018) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174073 · Cited by 48 · see citing works
Downey, G. P. (2008) · Johns Hopkins University Press eBooks · doi:10.1353/book.3337 · Cited by 47 · see citing works
The Universal Design File: Designing for People of All Ages and Abilities. Revised Edition.
Story, M. F., Mueller, J. L., & Mace, R. L. (1998) · Cited by 45 · see citing works
Co-designing interactive applications with adults with intellectual disability
Sitbon, L., & Farhin, S. (2017) · Australian Conference on Computer-Human Interaction (OzCHI) · doi:10.1145/3152771.3156163 · Cited by 45 · see citing works
Oswal, S. K. (2014) · Communication Design Quarterly · doi:10.1145/2644448.2644452 · Cited by 44 · see citing works
Integrating roboticist values into a Value Sensitive Design framework for humanoid robots
Cheon, E., & Su, N. M. (2016) · ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1109/hri.2016.7451775 · Cited by 44 · see citing works
Safe walking technology for people with dementia
Holbø, K., Bøthun, S., & Dahl, Y. (2013) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/2513383.2513434 · Cited by 42 · see citing works
AI Fairness for People with Disabilities: Point of View
Trewin, S. (2018) · arXiv · doi:10.48550/arxiv.1811.10670 · Cited by 38 · see citing works
Self-citationInsights from the aphasia project
McGrenere, J., Davies, R., Findlater, L., et al. (2002) · doi:10.1145/957205.957225 · Cited by 36 · see citing works
Toward Fairness in AI for People with Disabilities: A Research RoadmapPreprint
Guo, A., Kamar, E. S., Vaughan, J. W., Wallach, H., & Morris, M. R. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.02227 · Cited by 35 · see citing works
IEEE P7003™ standard for algorithmic bias considerations
Koene, A. R., Dowthwaite, L., & Seth, S. (2018) · doi:10.1145/3194770.3194773 · Cited by 34 · see citing works
Exploring Aesthetic Enhancement of Wearable Technologies for Deaf Women
Wilde, D., & Marti, P. (2018) · Designing Interactive Systems Conference · doi:10.1145/3196709.3196777 · Cited by 33 · see citing works
Broadening accessibility through special interests
Morris, R. R., Kirschbaum, C. R., & Picard, R. W. (2010) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/1878803.1878834 · Cited by 31 · see citing works
Collaborative Appropriation: How Couples, Teams, Groups and Communities Adapt and Adopt Technologies
Muller, M. J., Neureiter, K., Verdezoto, N. X., et al. (2016) · doi:10.1145/2818052.2855508 · Cited by 28 · see citing works
Self-citationDraper, H., Sorell, T., Bedaf, S., Syrdal, D. S., Gutierrez-Ruiz, C., Duclos, A., & Amirabdollahian, F. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-11973-1_14 · Cited by 26 · see citing works
Fairness-Aware Machine Learning: Practical Challenges and Lessons Learned
Bird, S., Hutchinson, B., Kenthapadi, K., Kıcıman, E., & Mitchell, M. (2019) · International World Wide Web Conference (WWW) · doi:10.1145/3308560.3320086 · Cited by 26 · see citing works
Shaheen, N. L., & Lazar, J. K. (2017) · Journal of Special Education Technology · doi:10.1177/0162643417734557 · Cited by 22 · see citing works
Felzmann, H., Beyan, T., Ryan, M., et al. (2016) · ACM SIGCAS Computers and Society · doi:10.1145/2874239.2874279 · Cited by 21 · see citing works
Treviranus, J. (2019) · International Web for All Conference (W4A) · doi:10.1145/3315002.3332429 · Cited by 17 · see citing works
Self-citationPublic policy and the global public inclusive infrastructure project
Lewis, C., & Treviranus, J. (2013) · interactions · doi:10.1145/2510123 · Cited by 13 · see citing works
Self-citationLife-long learning on the inclusive web
Treviranus, J. (2016) · International Web for All Conference (W4A) · doi:10.1145/2899475.2899476 · Cited by 12 · see citing works
Self-citationTorres, I. G., Parmar, G., Aggarwal, S., et al. (2019) · Extended Abstracts of the CHI Conference on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3290607.3308463 · Cited by 11 · see citing works
Potentials of the "Unexpected"
Tscheligi, M., Krischkowsky, A., Neureiter, K., et al. (2014) · International Conference on Supporting Group Work · doi:10.1145/2660398.2660427 · Cited by 10 · see citing works
Shaffer, I. R. (2018) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/3234695.3240986 · Cited by 10 · see citing works
Implications of developments in machine learning for people with cognitive disabilities
Lewis, C. (2020) · ACM SIGACCESS Accessibility and Computing · doi:10.1145/3386308.3386309 · Cited by 10 · see citing works
Krishnaswamy, K. (2017) · ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1145/3029798.3034815 · Cited by 8 · see citing works
Appropriation of social networking by older people: two case studies
Harley, D., & Fitzpatrick, G. (2012) · University of Brighton Repository (University of Brighton) · Cited by 5 · see citing works
A Model for Evaluating Algorithmic Systems AccountabilityPreprint
Kanellopoulos, Y. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.06083 · Cited by 4 · see citing works
Engaging with Users and Stakeholders
Hamidi, F., Müller, C., Baljko, M., Schorch, M., Lewkowicz, M., & Stangl, A. J. (2016) · International ACM SIGGROUP Conference on Supporting Group Work (GROUP) · doi:10.1145/2957276.2996292 · Cited by 3 · see citing works
The Tactics of Everyday Practice: A Semiotic Approach to Appropriation
Derboven, J., Geerts, D., & Grooff, D. D. (2016) · Interaction design & architecture(s)/ID&A Interaction design & architecture(s) · doi:10.55612/s-5002-029-005 · Cited by 3 · see citing works
Many more stories: Co-design and creative communities
Clark, C. B. D. (2018) · OCAD University Open Research Repository (OCAD University) · Cited by 1 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Design Principles for Generative AI Applications
Weisz, J. D., He, J., Muller, M., Hoefer, G., Miles, R., & Geyer, W. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642466 · Cited by 232 · see citing works
Self-citationFairness and Bias in Algorithmic Hiring: A Multidisciplinary Survey
Fabris, A., Baranowska, N., Dennis, M. J., Graus, D., Hacker, P., Saldivar, J., Zuiderveen Borgesius, F., & Biega, A. J. (2025) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3696457 · Cited by 117 · see citing works
The FATE Landscape of Sign Language AI Datasets: An Interdisciplinary Perspective
Bragg, D., Caselli, N., Hochgesang, J. A., Huenerfauth, M., Katz-Hernandez, L., Koller, O., Kushalnagar, R., Vogler, C., & Ladner, R. E. (2021) · ACM Transactions on Accessible Computing · doi:10.1145/3436996 · Cited by 55 · see citing works
Theodorou, L., Massiceti, D., Zintgraf, L., Stumpf, S., Morrison, C., Cutrell, E., Harris, M. T., & Hofmann, K. (2021) · Proceedings of the 23rd International ACM SIGACCESS Conference on Computers and Accessibility · doi:10.1145/3441852.3471225 · Cited by 42 · see citing works
Lewicki, K., Lee, M. S. A., Cobbe, J., & Singh, J. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581463 · Cited by 30 · see citing works
“Many of these works conclude that these issues are primarily a reflection of training set deficiencies, namely the underrepresentation of certain parts of the input space [15, 22] and the lack of awareness, consideration and attention of the people building these systems [94, 133, 142].”
BackgroundAyobi, A., Hughes, J., Duckworth, C. J., Dylag, J. J., Gordon James, S., Marshall, P., Guy, M., Kumaran, A., Chapman, A., Boniface, M., et al. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581424 · Cited by 26 · see citing works
Cripping AI: Reimagining AI Through Lived Disability Experiences
Tang, X., Lin, T.-A., Li, J., & Wu, S. (2026) · Proceedings of the 2026 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3805689.3806744 · Cited by 2 · see citing works
“Growing evidence shows that AI models exhibit both performance degradation and bias in disability-related tasks [79, 256, 266].”
- Authors
- Trewin, S., Basson, S., Muller, M., Branham, S., Treviranus, J., Gruen, D., Hebert, D., Lyckowski, N., & Manser, E.
- Affiliations
- IBM
- Google (United States) (US)
- IBM (United States) (US)
- University of California, Irvine (US)
- Ontario College of Art and Design (CA)
- Venue
- AI Matters
- Published
- 2019-12-06
- Volume
- 5
- Issue
- 3
- Pages
- 40-63
- ISSN
- 2372-3483 (ISSN-L)
- OpenAlex
- W2994948545
- Cited by
- 143
- References
- 78
- Citations by year
- 2026: 21 · 2025: 29 · 2024: 31 · 2023: 20 · 2022: 27 · 2021: 11 · 2020: 4
- FWCI
- 18.35
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- AI Matters (publishedVersion)
- Topics
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Social Sciences › Human Factors and Ergonomics › Digital Accessibility for Disabilities
- Social Sciences › Safety Research › Disability Rights and Representation
- Keywords
- algorithmic fairness, disability, discrimination, people with disabilities, algorithmic bias, employment, public safety, health care, AI development lifecycle, human-centered design
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
