Accountability in algorithmic decision making

Diakopoulos, Nicholas

doi:10.1145/2844110

Cited by 763Ethics and Social Impacts of AIExplainable Artificial Intelligence (XAI)Digitalization, Law, and Regulation

Accountability in algorithmic decision making

Diakopoulos, Nicholas·2016·Communications of the ACM

DOI 10.1145/2844110

Open this paper in CoChat — free account

Abstract

A view from computational journalism.

Integrity signals

Registry

Crossref · 10.1145/2844110

Cites

18 of 18 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Such inferences can have important negative implications.
“While such inferences are increasingly used in decision- and policy-making, they can also have important negative implications (Diakopoulos, 2016; O'Neil, 2016).”
From Social Data: Biases, Methodological Pitfalls, and Ethical Boundaries (2019) · cites this paper as Diakopoulos, 2016
A driving concern throughout the implementation of algorithmic management is organizational accountability: holding algorithmically-driven decisions accountable to various stakeholders within and outside the boundaries of the organization.
“A driving concern throughout the implementation of algorithmic management is organizational account-ability: holding algorithmically-driven decisions accountable to various stakeholders within and outside the boundaries of the organization (Diakopoulos, 2016; Mateescu and Nguyen, 2019).”
From Algorithmic management in a work context (2021) · cites this paper as Diakopoulos, 2016
The increasingly interconnected nature of systems means that many systems do not operate discretely and in isolation but are employed as part of a system-of-systems.
“However, the increasingly interconnected nature of systems means that in many cases they do not operate discretely and in isolation but are employed as part of a system-of-systems [14], [15], with potentially many entities involved [16], [17].”
From Decision Provenance: Harnessing Data Flow for Accountable Systems (2019) · cites this paper as [14]

Certifying and Removing Disparate Impact
Feldman, M., Friedler, S. A., Moeller, J., Scheidegger, C., & Venkatasubramanian, S. (2015) · Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2783258.2783311 · Cited by 1,827 · see citing works
Explaining collaborative filtering recommendations
Herlocker, J. L., Konstan, J. A., & Riedl, J. (2000) · Proceedings of the 2000 ACM conference on Computer supported cooperative work · doi:10.1145/358916.358995 · Cited by 1,731 · see citing works
On "Software engineering"
Miya, E. N. (1985) · ACM SIGSOFT Software Engineering Notes · doi:10.1145/1012443.1012444 · Cited by 1,053 · see citing works
Interpretable classifiers using rules and Bayesian analysis: Building a better stroke prediction model
Letham, B., Rudin, C., McCormick, T. H., & Madigan, D. (2015) · The Annals of Applied Statistics · doi:10.1214/15-aoas848 · Cited by 766 · see citing works
Algorithmic Accountability: Journalistic investigation of computational power structures
Diakopoulos, N. (2015) · Digital Journalism · doi:10.1080/21670811.2014.976411 · Cited by 733 · see citing works
Self-citation
"I always assumed that I wasn't really that close to [her]"
Eslami, M., Rickman, A., Vaccaro, K., Aleyasen, A., Vuong, A., Karahalios, K. G., Hamilton, K., & Sandvig, C. (2015) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702556 · Cited by 665 · see citing works
A Survey of Explanations in Recommender Systems
Tintarev, N., & Masthoff, J. (2007) · IEEE International Conference on Data Engineering Workshops (ICDEW) · doi:10.1109/icdew.2007.4401070 · Cited by 584 · see citing works
The search engine manipulation effect (SEME) and its possible impact on the outcomes of elections
Epstein, R., & Robertson, R. E. (2015) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1419828112 · Cited by 578 · see citing works
Enter the Robot Journalist: Users' perceptions of automated content
Clerwall, C. (2014) · Journalism Practice · doi:10.1080/17512786.2014.883116 · Cited by 377 · see citing works
Accountability in a computerized society
Nissenbaum, H. (1996) · Science and Engineering Ethics · doi:10.1007/bf02639315 · Cited by 346 · see citing works
Is there an ethics of algorithms?
Kraemer, F., van Overveld, K., & Peterson, M. (2010) · Ethics and Information Technology · doi:10.1007/s10676-010-9233-7 · Cited by 170 · see citing works
Opening the Black Box: Strategies for Increased User Involvement in Existing Algorithm Implementations
Mühlbacher, T., Piringer, H., Gratzl, S., Sedlmair, M., & Streit, M. (2014) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2014.2346578 · Cited by 131 · see citing works
Software Engineering Code of Ethics and Professional Practice
Gotterbarn, D. (2014) · Chapman & Hall/CRC eBooks · doi:10.1201/9780429171529-74 · Cited by 69 · see citing works
Turkers, Scholars, "Arafat" and "Peace": Cultural Communities and Algorithmic Gold Standards
Sen, S., Giesel, M. E., Gold, R., Hillmann, B., Lesicko, M., Naden, S., Russell, J., Wang, Z. (., & Hecht, B. (2015) · Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing · doi:10.1145/2675133.2675285 · Cited by 47 · see citing works
Getting the Message?
Schaffer, J., Giridhar, P., Jones, D. G., Höllerer, T., Abdelzaher, T., & O’Donovan, J. (2015) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/2678025.2701406 · Cited by 39 · see citing works
Systematic Bias and Nontransparency in US Social Security Administration Forecasts
Kashin, K., King, G., & Soneji, S. S. (2015) · The Journal of Economic Perspectives · doi:10.1257/jep.29.2.239 · Cited by 22 · see citing works
Facebook Wants You to Vote on Tuesday. Here's How It Messed With Your Feed in 2012.
Fitzgerald, B. (2015) · Cited by 7 · see citing works
Ethics and Professional Conduct
Whitaker, S. (2016) · Apress eBooks · doi:10.1007/978-1-4842-2074-0_12 · Cited by 5 · see citing works

Flagged first, then by citations

Sorted by citations

Seeing without knowing: Limitations of the transparency ideal and its application to algorithmic accountability
Ananny, M., & Crawford, K. (2018) · New Media & Society · doi:10.1177/1461444816676645 · Cited by 1,836 · see citing works
Towards Intelligent-TPACK: An empirical study on teachers’ professional knowledge to ethically integrate artificial intelligence (AI)-based tools into education
Celik, I. (2023) · Computers in Human Behavior · doi:10.1016/j.chb.2022.107468 · Cited by 1,012 · see citing works
Data Statements for Natural Language Processing: Toward Mitigating System Bias and Enabling Better Science
Bender, E. M., & Friedman, B. (2018) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00041 · Cited by 854 · see citing works
Social Data: Biases, Methodological Pitfalls, and Ethical Boundaries
Olteanu, A., Castillo, C., Diaz, F., & Kıcıman, E. (2019) · Frontiers in Big Data · doi:10.3389/fdata.2019.00013 · Cited by 763 · see citing works
“In the case of inactive accounts, Gong et al. (2015, 2016) found that a signiﬁcant fraction of users, though interested in a given topic, choose to remain silent.”
Background
Trends and Trajectories for Explainable, Accountable and Intelligible Systems: An HCI Research Agenda
Abdul, A., Vermeulen, J., Wang, D., Lim, B. Y., & Kankanhalli, M. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174156 · Cited by 743 · see citing works
Actionable Auditing: Investigating the Impact of Publicly Naming Biased Performance Results of Commercial AI Products
Raji, I. D., & Buolamwini, J. (2019) · Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3306618.3314244 · Cited by 651 · see citing works
Role of fairness, accountability, and transparency in algorithmic affordance
Shin, D., & Park, Y. J. (2019) · Computers in Human Behavior · doi:10.1016/j.chb.2019.04.019 · Cited by 643 · see citing works
Towards transparency by design for artificial intelligence
Felzmann, H., Fosch-Villaronga, E., Lutz, C., & Tamò-Larrieux, A. (2020) · Science and Engineering Ethics · doi:10.1007/s11948-020-00276-4 · Cited by 588 · see citing works
Trustworthy AI: From Principles to Practices
Li, B., Qi, P., Liu, B., Di, S., Liu, J., Pei, J., Yi, J., & Zhou, B. (2023) · ACM Computing Surveys · doi:10.1145/3555803 · Cited by 612 · see citing works
“Note that in terms of information disclosure, transparency contributes the basic mechanism used to facilitate the accountability of an AI system [94, 100].”
Background
Algorithmic management in a work context
Jarrahi, M. H., Newlands, G., Lee, M. K., Wolf, C. T., Kinder, E., & Sutherland, W. (2021) · Big Data & Society · doi:10.1177/20539517211020332 · Cited by 532 · see citing works
“A driving concern throughout the implementation of algorithmic management is organizational accountability: holding algorithmically-driven decisions accountable to various stakeholders within and outside the boundaries of the organization (Diakopoulos, 2016; Mateescu and Nguyen, 2019).”
Background
Algorithmic Accountability and Public Reason
Binns, R. (2018) · Philosophy & Technology · doi:10.1007/s13347-017-0263-5 · Cited by 457 · see citing works
“This phenomenon has been termed algorithmic decision-making, and there are increasing calls to make algorithmic decision-makers accountable for their activities.1 1See e.g. (Diakopoulos, 2016), and section 2 for further examples.”
Background
Transparency and accountability in AI systems: Safeguarding wellbeing in the age of algorithmic decision-making
Cheong, B. C. (2024) · Frontiers in Human Dynamics · doi:10.3389/fhumd.2024.1421273 · Cited by 445 · see citing works
Artificial intelligence versus Maya Angelou: Experimental evidence that people cannot differentiate AI-generated from human-written poetry
Köbis, N., & Mossink, L. D. (2021) · Computers in Human Behavior · doi:10.1016/j.chb.2020.106553 · Cited by 406 · see citing works
Explanations as Mechanisms for Supporting Algorithmic Transparency
Rader, E., Cotter, K., & Cho, J. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173677 · Cited by 388 · see citing works
The Challenges of Algorithm-Based HR Decision-Making for Personal Integrity
Leicht-Deobald, U., Busch, T., Schank, C., Weibel, A., Schafheitle, S., Wildhaber, I., & Kasper, G. (2019) · Journal of Business Ethics · doi:10.1007/s10551-019-04204-w · Cited by 347 · see citing works
What to account for when accounting for algorithms: a systematic literature review on algorithmic accountability
Wieringa, M. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372833 · Cited by 321 · see citing works
Explaining Decision-Making Algorithms through UI: Strategies to Help Non-Expert Stakeholders
Cheng, H.-F., Wang, R., Zhang, Z., O'Connell, F., Gray, T., Harper, F. M., & Zhu, H. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300789 · Cited by 292 · see citing works
Putting AI ethics to work: are the tools fit for purpose?
Ayling, J., & Chapman, A. (2022) · AI and Ethics · doi:10.1007/s43681-021-00084-x · Cited by 253 · see citing works
Examining the Impact of Algorithmic Control on Uber Drivers’ Technostress
Cram, W. A., Wiener, M., Tarafdar, M., & Benlian, A. (2022) · Journal of Management Information Systems · doi:10.1080/07421222.2022.2063556 · Cited by 246 · see citing works
Algorithmic Impact Assessments and Accountability: The Co-construction of Impacts
Metcalf, J., Moss, E., Watkins, E. A., Singh, R., & Elish, M. C. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445935 · Cited by 195 · see citing works
Who Audits the Auditors? Recommendations from a field scan of the algorithmic auditing ecosystem
Costanza-Chock, S., Raji, I. D., & Buolamwini, J. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533213 · Cited by 191 · see citing works
“Auditing is imagined as a way to explicitly gather and expose evidence of how deployments fall short of performance claims [21].”
Background
A Scholarly Definition of Artificial Intelligence (AI): Advancing AI as a Conceptual Framework in Communication Research
Gil de Zúñiga, H., Goyanes, M., & Durotoye, T. (2024) · Political Communication · doi:10.1080/10584609.2023.2290497 · Cited by 198 · see citing works
Behavioural evidence for a transparency–efficiency tradeoff in human–machine cooperation
Ishowo-Oloko, F., Bonnefon, J.-F., Soroye, Z., Crandall, J., Rahwan, I., & Rahwan, T. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0113-5 · Cited by 178 · see citing works
Reviewable Automated Decision-Making: A Framework for Accountable Algorithmic Systems
Cobbe, J., Lee, M. S. A., & Singh, J. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445921 · Cited by 150 · see citing works
CheXplain: Enabling Physicians to Explore and Understand Data-Driven, AI-Enabled Medical Imaging Analysis
Xie, Y., Chen, M., Kao, D., Gao, G., & Chen, X. '. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376807 · Cited by 141 · see citing works
Decision Provenance: Harnessing Data Flow for Accountable Systems
Singh, J., Cobbe, J., & Norval, C. (2019) · IEEE Access · doi:10.1109/access.2018.2887201 · Cited by 110 · see citing works
Social Data: Biases, Methodological Pitfalls, and Ethical Boundaries
Olteanu, A., Castillo, C., Diaz, F., & Kiciman, E. (2016) · doi:10.2139/ssrn.2886526 · Cited by 763 · see citing works
Responsible artificial intelligence in healthcare: predicting and preventing insurance claim denials for economic and social wellbeing
Johnson, M., Albizri, A., & Harfouche, A. (2023) · Information Systems Frontiers · doi:10.1007/s10796-021-10137-5 · Cited by 88 · see citing works
Automated Decision-Making and Big Data: Concerns for People With Mental Illness
Monteith, S., & Glenn, T. (2016) · Current Psychiatry Reports · doi:10.1007/s11920-016-0746-6 · Cited by 62 · see citing works
When people and algorithms meet: user-reported problems in intelligent everyday applications
Eiband, M., Völkel, S. T., Buschek, D., Cook, S., & Hussmann, H. (2019) · Proceedings of the 24th International Conference on Intelligent User Interfaces · doi:10.1145/3301275.3302262 · Cited by 62 · see citing works
Death of a reviewer or death of peer review integrity? The challenges of using AI tools in peer reviewing and the need to go beyond publishing policies
Mollaki, V. (2024) · Research Ethics · doi:10.1177/17470161231224552 · Cited by 64 · see citing works
“Such biases include first impression bias (Checco et al., 2021), cultural and organizational biases (Diakopoulos, 2016) including language skills, and biases towards highly reputable research institutes or wealthy countries with high research and development expenditure.”
Background
AI Documentation: A path to accountability
Königstorfer, F., & Thalmann, S. (2022) · Journal of Responsible Technology · doi:10.1016/j.jrt.2022.100043 · Cited by 55 · see citing works
Toward Fair, Accountable, and Transparent Algorithms: Case Studies on Algorithm Initiatives in Korea and China
Shin, D. (. (2019) · Javnost - The Public · doi:10.1080/13183222.2019.1589249 · Cited by 53 · see citing works
Challenges and efforts in managing AI trustworthiness risks: a state of knowledge
Polemi, N., Praça, I., Kioskli, K., & Bécue, A. (2024) · Frontiers in Big Data · doi:10.3389/fdata.2024.1381163 · Cited by 46 · see citing works
Technology-mediated Control: Case Examples and Research Directions for the Future of Organizational Control
Cram, W. A., & Wiener, M. (2020) · Communications of the Association for Information Systems · doi:10.17705/1cais.04604 · Cited by 56 · see citing works
Taking Algorithms to Courts: A Relational Approach to Algorithmic Accountability
Metcalf, J., Singh, R., Moss, E., Tafesse, E., & Watkins, E. A. (2023) · 2023 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3593013.3594092 · Cited by 21 · see citing works
Accountability in Government Use of <scp>AI</scp> : Citizen Concerns and Preferences
Feng, N., & Chandra, Y. (2025) · Public Administration · doi:10.1111/padm.70030 · Cited by 18 · see citing works
A framework to contest and justify algorithmic decisions
Henin, C., & Le Métayer, D. (2021) · AI and Ethics · doi:10.1007/s43681-021-00054-3 · Cited by 14 · see citing works
A discussion frame for explaining records that are based on algorithmic output
Andresen, H. (2019) · Records Management Journal · doi:10.1108/rmj-04-2019-0019 · Cited by 10 · see citing works
L'acte administratif algorithmique
Stepanov, A. (2026) · doi:10.70675/adc05be3za62ez4b4fz81e7zabf744e60dbf

Authors: Diakopoulos, N.
Affiliations: University of Maryland, College Park (US)
Corresponding author: Diakopoulos, Nicholas
ORCID: Diakopoulos, Nicholas
Venue: Communications of the ACM
Publisher: Association for Computing Machinery (ACM)
Published: 2016-01-25
Volume: 59
Issue: 2
Pages: 56-62
ISSN: 0001-0782 (ISSN-L)
1557-7317
DOI: 10.1145/2844110
OpenAlex: W2263936035
Cited by: 763
References: 18
Citations by year: 2026: 190 · 2025: 104 · 2024: 98 · 2023: 51 · 2022: 68 · 2021: 76 · 2020: 45 · 2019: 62 · 2018: 42 · 2017: 18 · 2016: 9
FWCI: 106.70
Type: journal article
Language: en
Open access: closed
Licence: www.acm.org/publications/policies/copyright_policy#Background (vor)
Locations: Communications of the ACM (publishedVersion)
Topics: Social Sciences › Safety Research › Ethics and Social Impacts of AI
Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
Social Sciences › Law › Digitalization, Law, and Regulation
Keywords: algorithmic accountability, algorithmic decision-making, computational journalism, algorithmic transparency, algorithmic fairness

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Accountability in algorithmic decision making

Abstract

Integrity signals

Claims the paper makes

Claims other papers make about this one

Cites

Cited by

Related

Record

About these checks