Algorithmic accountability
Open accessAbstract
There is enormous opportunity for positive social impact from the rise of algorithms and machine learning. But this requires a licence to operate from the public, based on trustworthiness. There are a range of concerns relating to how algorithms might be held to account in areas affecting the public sphere. This paper outlines a number of approaches including greater transparency, monitoring of outcomes and improved governance. It makes a case that public sector bodies that hold datasets should be more confident in negotiating terms with the private sector. It also argues that all regulators (not just data regulators) need to wake up to the challenges posed by changing technology. Other improvements include diversity of the workforce, ethics training, codes of conduct for data scientists, and new deliberative bodies. Even if these narrower issues are solved, the paper poses some wider concerns including data monopolies, the challenge to democracy, public participation and maintaining the public interest. This article is part of a discussion meeting issue ‘The growing ubiquity of algorithms in society: implications, impacts and innovations’.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Internal audits complement external accountability, generating artifacts or transparent information that third parties can use for external auditing, or even end-user communication.
“Ultimately, internal audits complement external accountability, generating artifacts or transparent information [70] that third parties can use for external auditing, or even end-user communication.”
From Closing the AI accountability gap: defining an end-to-end framework for internal algorithmic auditing (2020) · cites this paper as [70]
Section 2 (‘Algorithmic accountability’; page not provided) recommends publishing the model, data and provenance metadata to allow external scrutiny. It does not say that internal audits generate those materials or that they are used for end-user communication. (LLM, conf=0.86)
Experimental — accuracy not yet measuredSimpler approaches to mitigating bias in data involve piloting algorithms in different contexts and with various datasets.
“Simpler approaches to mitigating bias in data involve piloting algorithms in different contexts and with various datasets (Shah 2018).”
From The ethics of algorithms: key problems and solutions (2022) · cites this paper as Shah 2018
Having a model, its datasets, and metadata on provenance published to enable external scrutiny can help correct unseen or unwanted bias.
“Having a model, its datasets, and metadata (on provenance) published to enable external scrutiny can also help correct unseen or unwanted bias (Shah 2018).”
From The ethics of algorithms: key problems and solutions (2022) · cites this paper as Shah 2018
Data used to train an algorithm is one of the main sources from which bias emerges, through preferentially sampled data or from data reflecting existing societal bias.
“The data used to train an algorithm is one of the main sources from which bias emerges (Shah 2018), through preferentially sampled data or from data reflecting existing societal bias (Diakopoulos and Koliska 2017;Danks and London 2017;Binns 2018;Malhotra et al. 2018).”
From The ethics of algorithms: key problems and solutions (2022) · cites this paper as Shah 2018
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Semantics derived automatically from language corpora contain human-like biases
Caliskan, A., Bryson, J. J., & Narayanan, A. (2017) · Science · doi:10.1126/science.aal4230 · Cited by 2,887 · see citing works
Counterfactual explanations without opening the black box: automated decisions and the GDPR
Wachter, S., Mittelstadt, B., & Russell, C. (2017) · SSRN Electronic Journal · doi:10.2139/ssrn.3063289 · Cited by 1,061 · see citing works
The DeepMind debacle demands dialogue on data
Shah, H. (2017) · Nature · doi:10.1038/547259a · Cited by 25 · see citing works
Self-citationFairness and Transparency in the Age of the Algorithm
Olhede, S. C., & Rodrigues, R. (2017) · Significance · doi:10.1111/j.1740-9713.2017.01012.x · Cited by 24 · see citing works
The opportunities and ethics of big data: practical priorities for a national Council of Data Ethics
Varley-Winter, O., & Shah, H. (2016) · Philosophical Transactions of the Royal Society A Mathematical Physical and Engineering Sciences · doi:10.1098/rsta.2016.0116 · Cited by 16 · see citing works
Self-citationUse our personal data for the common good
Shah, H. (2018) · Nature · doi:10.1038/d41586-018-03912-z · Cited by 11 · see citing works
Self-citation
Flagged first, then by citations
Research works printed but not linked: 1
- Deepmind Health Independent Review Panel. 2017. Annual Report. See https://deepmind. com/documents/85/DeepMind%20Health%20Independent%20Review%20Annual%20Rep ort%202017.pdf.
Cited by
Sorted by citations
Raji, I. D., Smart, A., White, R. N., Mitchell, M., Gebru, T., Hutchinson, B., Smith-Loud, J., Theron, D., & Barnes, P. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372873 · Cited by 1,372 · see citing works
“Ultimately, internal audits complement external accountability, generating artifacts or transparent information [70] that third parties can use for external auditing, or even end-user communication.”
BackgroundArtificial intelligence and algorithmic bias: implications for health systems
Panch, T., Mattie, H., & Atun, R. (2019) · Journal of Global Health · doi:10.7189/jogh.09.020318 · Cited by 533 · see citing works
The ethics of algorithms: key problems and solutions
Tsamados, A., Aggarwal, N., Cowls, J., Morley, J., Roberts, H., Taddeo, M., & Floridi, L. (2022) · AI & SOCIETY · doi:10.1007/s00146-021-01154-8 · Cited by 384 · see citing works
Brown, A., Chouldechova, A., Putnam-Hornstein, E., Tobin, A., & Vaithianathan, R. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300271 · Cited by 207 · see citing works
Giovanola, B., & Tiribelli, S. (2023) · AI & SOCIETY · doi:10.1007/s00146-022-01455-6 · Cited by 193 · see citing works
The growing ubiquity of algorithms in society: implications, impacts and innovations
Olhede, S. C., & Wolfe, P. J. (2018) · Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences · doi:10.1098/rsta.2017.0364 · Cited by 120 · see citing works
Should Users Trust Advanced AI Assistants? Justified Trust As a Function of Competence and Alignment
Manzini, A., Keeling, G., Marchal, N., McKee, K. R., Rieser, V., & Gabriel, I. (2024) · The 2024 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3630106.3658964 · Cited by 28 · see citing works
“With the increasing likelihood that AI assistants will become deeply integrated in our individual and collective lives and that they will be socially transformative, the need to protect users’ rights via effective governance has risen to the fore.”
BackgroundLammert, J. M., Roberts, A. C., McRae, K., Batterink, L. J., & Butler, B. E. (2025) · Journal of Speech, Language, and Hearing Research · doi:10.1044/2024_jslhr-24-00515 · Cited by 17 · see citing works
Reconciling Administrative Efficiency with Procedural Safeguards in the ADM System
Sharaf Addin, E. (2025) · Bratislava Law Review · doi:10.46282/blr.2025.9.2.1116
“Among the methods are better governance, greater transparency, and outcome monitoring (Shah, 2018).”
- Authors
- Shah, H.
- Affiliations
- Royal Statistical Society (GB)
- Corresponding author
- Shah, Hetan
- ORCID
- Shah, Hetan
- Venue
- Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences
- Publisher
- The Royal Society
- Published
- 2018-08-06
- Volume
- 376
- Issue
- 2128
- Pages
- 20170362
- ISSN
- 1364-503X (ISSN-L)
- 1471-2962
- OpenAlex
- W4254943578
- Cited by
- 101
- References
- 6
- Citations by year
- 2026: 8 · 2025: 16 · 2024: 11 · 2023: 33 · 2022: 17 · 2021: 8 · 2020: 3 · 2019: 4 · 2018: 1
- FWCI
- 7.95
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Philosophical Transactions of the Royal Society A Mathematical Physical and Engineering Sciences (publishedVersion, public-domain)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Social Sciences › Law › Digitalization, Law, and Regulation
- Social Sciences › Political Science and International Relations › E-Government and Public Services
- Keywords
- algorithmic accountability, algorithmic transparency, algorithmic governance, government data, data regulation, workplace diversity, ethics training, data monopolies, democracy, citizen participation, public interest, machine learning
- MeSH terms
- Machine Learning, Algorithms, Social Control, Formal, Social Responsibility, Policy
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
