A Human-Centric Perspective on Fairness and Transparency in Algorithmic Decision-Making
Open accessAbstract
Automated decision systems (ADS) are increasingly used for consequential decision-making. These systems often rely on sophisticated yet opaque machine learning models, which do not allow for understanding how a given decision was arrived at. This is not only problematic from a legal perspective, but non-transparent systems are also prone to yield unfair outcomes because their sanity is challenging to assess and calibrate in the first place—which is particularly worrisome for human decision-subjects. Based on this observation and building upon existing work, I aim to make the following three main contributions through my doctoral thesis: (a) understand how (potential) decision-subjects perceive algorithmic decisions (with varying degrees of transparency of the underlying ADS), as compared to similar decisions made by humans; (b) evaluate different tools for transparent decision-making with respect to their effectiveness in enabling people to appropriately assess the quality and fairness of ADS; and (c) develop human-understandable technical artifacts for fair automated decision-making. Over the course of the first half of my PhD program, I have already addressed substantial pieces of (a) and (c), whereas (b) will be the major focus of the second half.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Non-transparent systems are more likely to produce unfair outcomes, as it is challenging to assess and adjust their fairness.
“Non-transparent systems are more likely to produce unfair outcomes, as it is challenging to assess and adjust their fairness, which is especially concerning for those subject to human decisions (Schoeffer, 2022).”
From Reconciling Administrative Efficiency with Procedural Safeguards in the ADM System (2025) · cites this paper as Schoeffer, 2022
Cites
A Unified Approach to Interpreting Model Predictions
Lundberg, S., & Lee, S.-I. (2017) · arXiv · doi:10.48550/arxiv.1705.07874 · Cited by 23,707 · see citing works
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555 · see citing works
“Why Should I Trust You?”: Explaining the Predictions of Any Classifier
Ribeiro, M., Singh, S., & Guestrin, C. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Demonstrations · doi:10.18653/v1/n16-3020 · Cited by 6,682 · see citing works
Justice at the millennium: A meta-analytic review of 25 years of organizational justice research.
Colquitt, J. A., Conlon, D. E., Wesson, M. J., Porter, C. O. L. H., & Ng, K. Y. (2001) · Journal of Applied Psychology · doi:10.1037/0021-9010.86.3.425 · Cited by 5,942 · see citing works
A Survey on Bias and Fairness in Machine Learning
Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., & Galstyan, A. (2022) · ACM Computing Surveys · doi:10.1145/3457607 · Cited by 5,812 · see citing works
Proceedings of the 21st International Conference on Neural Information Processing Systems
Koller, D., Schuurmans, D., Bengio, Y., & Bottou, L. (2008) · Cited by 4,461 · see citing works
Running experiments on Amazon Mechanical Turk
Paolacci, G., Chandler, J., & Ipeirotis, P. G. (2010) · Judgment and Decision Making · doi:10.1017/s1930297500002205 · Cited by 3,767 · see citing works
(2018) · Cited by 3,510 · see citing works
Dwork, C., Hardt, M., Pitassi, T., Reingold, O., & Zemel, R. (2012) · Proceedings of the 3rd Innovations in Theoretical Computer Science Conference · doi:10.1145/2090236.2090255 · Cited by 3,501 · see citing works
Prolific.ac—A subject pool for online experiments
Palan, S., & Schitter, C. (2018) · Journal of Behavioral and Experimental Finance · doi:10.1016/j.jbef.2017.12.004 · Cited by 3,450 · see citing works
Equality of Opportunity in Supervised Learning
Hardt, M., Price, E., & Srebro, N. (2016) · arXiv · doi:10.48550/arxiv.1610.02413 · Cited by 2,536 · see citing works
The utilization of e-government services: citizen trust, innovation and acceptance factors
Carter, L., & Bélanger, F. (2005) · Information Systems Journal · doi:10.1111/j.1365-2575.2005.00183.x · Cited by 2,207 · see citing works
Fair prediction with disparate impact: a study of bias in recidivism prediction instruments
Chouldechova, A. (2017) · Big Data · doi:10.1089/big.2016.0047 · Cited by 2,159 · see citing works
Lee, M. K. (2018) · Big Data & Society · doi:10.1177/2053951718756684 · Cited by 1,338 · see citing works
Dark Patterns at Scale: Findings from a Crawl of 11K Shopping Websites
Mathur, A., Acar, G., Friedman, M. J., Lucherini, E., Mayer, J., Chetty, M., & Narayanan, A. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359183 · Cited by 816 · see citing works
'It's Reducing a Human Being to a Percentage': Perceptions of Justice in Algorithmic Decisions
Binns, R., Van Kleek, M., Veale, M., Lyngs, U., Zhao, J., & Shadbolt, N. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173951 · Cited by 691 · cited 3× · see citing works
“2 Moreover, while the general need for transparency appears obvious, it is also understood that there exists no “one-size-fits-all” explanation technique that addresses all desiderata of different stakeholders of algorithmic decision-making [3, 14, 35].”
Langer, M., Oster, D., Speith, T., Hermanns, H., Kästner, L., Schmidt, E., Sesing, A., & Baum, K. (2021) · Artificial Intelligence · doi:10.1016/j.artint.2021.103473 · Cited by 652 · see citing works
How Much Information?: Effects of Transparency on Trust in an Algorithmic Interface
Kizilcec, R. F. (2016) · Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858402 · Cited by 615 · see citing works
On the ethics of algorithmic decision-making in healthcare
Grote, T., & Berens, P. (2020) · Journal of Medical Ethics · doi:10.1136/medethics-2019-105586 · Cited by 551 · see citing works
Newell, S., & Marabelli, M. (2015) · The Journal of Strategic Information Systems · doi:10.1016/j.jsis.2015.02.001 · Cited by 494 · see citing works
Building Classifiers with Independency Constraints
Calders, T., Kamiran, F., & Pechenizkiy, M. (2009) · 2009 IEEE International Conference on Data Mining Workshops · doi:10.1109/icdmw.2009.83 · Cited by 468 · see citing works
The Role of Explanations on Trust and Reliance in Clinical Decision Support Systems
Bussone, A., Stumpf, S., & O'Sullivan, D. (2015) · 2015 International Conference on Healthcare Informatics · doi:10.1109/ichi.2015.26 · Cited by 441 · see citing works
Too much, too little, or just right? Ways explanations impact end users' mental models
Kulesza, T., Stumpf, S., Burnett, M., Yang, S., Kwan, I., & Wong, W.-K. (2013) · 2013 IEEE Symposium on Visual Languages and Human Centric Computing · doi:10.1109/vlhcc.2013.6645235 · Cited by 388 · see citing works
Avoiding Discrimination through Causal Reasoning
Kilbertus, N., Rojas-Carulla, M., Parascandolo, G., Hardt, M., Janzing, D., & Schölkopf, B. (2017) · arXiv · doi:10.48550/arxiv.1706.02744 · Cited by 358 · see citing works
Inherent Tradeoffs in the Fair Determination of Risk Scores
Raghavan, M. (2023) · ACM eBooks · doi:10.1145/3603195.3603197 · Cited by 347 · see citing works
Explaining models: an empirical study of how explanations impact fairness judgment
Dodge, J., Liao, Q. V., Zhang, Y., Bellamy, R. K. E., & Dugan, C. (2019) · Proceedings of the 24th International Conference on Intelligent User Interfaces · doi:10.1145/3301275.3302310 · Cited by 286 · see citing works
Procedural Justice in Algorithmic Fairness
Lee, M. K., Jain, A., Jin, H., Ojha, S., & Kusbit, D. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359284 · Cited by 236 · see citing works
A Case for Humans-in-the-Loop: Decisions in the Presence of Erroneous Algorithmic Scores
De-Arteaga, M., Fogliato, R., & Chouldechova, A. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376638 · Cited by 221 · see citing works
Productivity and Selection of Human Capital with Machine Learning
Chalfin, A. J., Danieli, O., Hillis, A., Jelveh, Z., Luca, M., Ludwig, J., & Mullainathan, S. (2016) · American Economic Review · doi:10.1257/aer.p20161029 · Cited by 190 · see citing works
User attitudes towards algorithmic opacity and transparency in online reviewing platforms
Eslami, M., Vaccaro, K., Lee, M. K., Elazari Bar On, A., Gilbert, E., & Karahalios, K. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300724 · Cited by 184 · see citing works
Srivastava, M., Heidari, H., & Krause, A. (2019) · Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3292500.3330664 · Cited by 179 · see citing works
Automation bias: Empirical results assessing influencing factors
Goddard, K., Roudsari, A., & Wyatt, J. C. (2014) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2014.01.001 · Cited by 163 · see citing works
Feuerriegel, S., Dolata, M., & Schwabe, G. (2020) · Business & Information Systems Engineering · doi:10.1007/s12599-020-00650-3 · Cited by 160 · see citing works
Measuring Justice and Fairness
Cropanzano, R., Ambrose, M. L., Colquitt, J. A., & Rodell, J. B. (2015) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780199981410.013.8 · Cited by 149 · see citing works
Lee, M. K., & Rich, K. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445570 · Cited by 145 · see citing works
Automated Decision Making Comes of Age
Davenport, T. H., & Harris, J. G. (2005) · MIT Sloan Management Review · Cited by 121 · see citing works
Research in the crowdsourcing age: a case study
Hitlin, P. (2016) · Cited by 119 · see citing works
The Tyranny of Data? The Bright and Dark Sides of Data-Driven Decision-Making for Social Good
Lepri, B., Staiano, J., Sangokoya, D., Letouzé, E., & Oliver, N. (2017) · Studies in Big Data · doi:10.1007/978-3-319-54024-5_1 · Cited by 111 · see citing works
This Thing Called Fairness: Disciplinary Confusion Realizing a Value in Technology
Mulligan, D. K., Kroll, J. A., Kohli, N., & Wong, R. Y. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359221 · Cited by 111 · see citing works
Lakkaraju, H., Kleinberg, J., Leskovec, J., Ludwig, J., & Mullainathan, S. (2017) · Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/3097983.3098066 · Cited by 110 · see citing works
Auditing for Discrimination in Algorithms Delivering Job Ads
Imana, B., Korolova, A., & Heidemann, J. (2021) · International World Wide Web Conference (WWW) · doi:10.1145/3442381.3450077 · Cited by 109 · see citing works
Triberti, S., Durosini, I., & Pravettoni, G. (2020) · Frontiers in Public Health · doi:10.3389/fpubh.2020.00117 · Cited by 89 · see citing works
Explainability pitfalls: beyond dark patterns in explainable AI
Ehsan, U., & Riedl, M. O. (2024) · Patterns · doi:10.1016/j.patter.2024.100971 · Cited by 72 · see citing works
Transparent Data Mining for Big and Small Data
Cerquitelli, T., Cerquitelli, T., Quercia, D., & Pasquale, F. (2017) · Studies in big data · doi:10.1007/978-3-319-54024-5 · Cited by 61 · see citing works
Langer, M., Baum, K. L., König, C. J., Hähne, V., Oster, D., & Speith, T. (2021) · International Journal of Selection and Assessment · doi:10.1111/ijsa.12325 · Cited by 56 · see citing works
Dark Patterns of Explainability, Transparency, and User Control for Intelligent Systems.
Chromik, M., Eiband, M., Völkel, S. T., & Buschek, D. (2019) · IUI Workshops · Cited by 42 · see citing works
Srivastava, M., Heidari, H., & Krause, A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.04783 · Cited by 41 · see citing works
Understanding Decision-Making in Recruitment
Koivunen, S., Olsson, T., Olshannikova, E., & Lindberg, A. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3361123 · Cited by 33 · see citing works
A Study on Fairness and Trust Perceptions in Automated Decision MakingPreprint
Schoeffer, J., Machowski, Y., & Kuehl, N. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.04757 · Cited by 31 · see citing works
Self-citationFair Decisions Despite Imperfect Predictions
Kilbertus, N., Gomez-Rodriguez, M., Schölkopf, B., Muandet, K., & Valera, I. (2020) · Cited by 24 · see citing works
Schoeffer, J., & Kuehl, N. (2021) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/3462204.3481742 · Cited by 24 · see citing works
Self-citationSchoeffer, J., Machowski, Y., & Kühl, N. (2022) · Hawaii International Conference on System Sciences (HICSS) · doi:10.24251/hicss.2022.134 · Cited by 16 · see citing works
Self-citationThe Role of Abduction in Learning to Use a Computer System.
Lewis, C., & Mack, R. L. (1982) · Cited by 15 · see citing works
Deontological Ethics By Monotonicity Shape ConstraintsPreprint
Wang, S., & Gupta, M. R. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.11990 · Cited by 9 · see citing works
A Ranking Approach to Fair Classification
Schoeffer, J., Kuehl, N., & Valera, I. (2021) · ACM SIGCAS Conference on Computing and Sustainable Societies (COMPASS) · doi:10.1145/3460112.3471950 · Cited by 7 · see citing works
Self-citation
Flagged first, then by citations
3 of the 60 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
The Algorithmic Divide: A Systematic Review on AI-Driven Racial Disparities in Healthcare
Haider, S. A., Borna, S., Gomez-Cabello, C. A., Pressman, S. M., Haider, C. R., & Forte, A. J. (2024) · Journal of Racial and Ethnic Health Disparities · doi:10.1007/s40615-024-02237-0 · Cited by 66 · see citing works
Tal, A. S., Kuflik, T., Kliger, D., & Mancini, A. (2024) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2024.2348843 · Cited by 26 · see citing works
Cabitza, F., Campagner, A., Fregosi, C., Cameli, M., Gallazzi, E., Sconfienza, L. M., & Tontini, G. E. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757601 · Cited by 9 · see citing works
How Small Businesses Transform PDF Agreements into Action
So, J., Lipka, N., Siu, A. F., Rossi, R. A., & Dernoncourt, F. (2023) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/3584931.3606972 · Cited by 1 · see citing works
Reconciling Administrative Efficiency with Procedural Safeguards in the ADM System
Sharaf Addin, E. (2025) · Bratislava Law Review · doi:10.46282/blr.2025.9.2.1116
“Non-transparent systems are more likely to produce unfair outcomes, as it is challenging to assess and adjust their fairness, which is especially concerning for those subject to human decisions (Schoeffer, 2022).”
Designing Emotional Bonds: AI Devices as Solutions for Pet Separation Anxiety
Ji, L., & Yang, S. (2025) · Communications in computer and information science · doi:10.1007/978-3-031-94162-7_11
Koswara, A., Narimawati, U., & Kurniawan, B. (2026) · Journal of Islamic Civilization · doi:10.33086/jic.v7i2.8471
Carlsson, L., Olsson, A., & Eriksson, K. M. (2026) · IFIP advances in information and communication technology · doi:10.1007/978-3-032-16779-8_1
Whose Values? Demographic Influences on Perceptions of Conversational AI Value Alignment
Sadek, M., & Mougenot, C. (2026) · ACM Conference on Conversational User Interfaces (CUI) · doi:10.1145/3816046.3816265
- Authors
- Schoeffer, J.
- Affiliations
- Karlsruhe Institute of Technology (DE)
- Corresponding author
- Schoeffer, Jakob
- ORCID
- Schoeffer, Jakob
- Venue
- CHI Conference on Human Factors in Computing Systems Extended Abstracts
- Publisher
- ACM
- Published
- 2022-04-27
- Pages
- 1-6
- OpenAlex
- W4225113193
- Cited by
- 9
- References
- 60
- Citations by year
- 2026: 3 · 2025: 3 · 2024: 2 · 2023: 1
- FWCI
- 0.83
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Social Sciences › Psychology › Social Psychology › Human-Automation Interaction and Safety
- Keywords
- automated decision systems, algorithmic fairness, algorithmic transparency, explainable AI
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
