Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 87 of the 88 references it cites were found in a registry or an index; 10 of the 43 sources its claims cite were checked for support.

Fairness and Accountability Design Needs for Algorithmic Support in High-Stakes Public Sector Decision-Making

Open access
Veale, Michael, Van Kleek, Max, Binns, Reuben · 2018 · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems

Published version of a preprint

Abstract

Calls for heightened consideration of fairness and accountability in algorithmically-informed public decisions-like taxation, justice, and child protection-are now commonplace. How might designers support such human values? We interviewed 27 public sector machine learning practitioners across 5 OECD countries regarding challenges understanding and imbuing public values into their work. The results suggest a disconnect between organisational and institutional realities, constraints and needs, and those addressed by current research into usable, transparent and 'discrimination-aware' machine learning-absences likely to undermine practical initiatives unless addressed. We see design opportunities in this disconnect, such as in supporting the tracking of concept drift in secondary data sources, and in building usable transparency tools to identify risks and incorporate domain knowledge, aimed both at managers and at the 'street-level bureaucrats' on the frontlines of public service. We conclude by outlining ethical challenges and future directions for collaboration in these high-stakes applications.

Overview

CoChat Summary

TL;DR

Interviews with 27 public-sector machine-learning practitioners across five OECD countries reveal a disconnect between organisational realities and current fairness, accountability, and transparency research. The authors identify design opportunities around data drift, discretion, performance communication, and social practices, while cautioning that FAT/ML tools cannot simply be airdropped into high-stakes public decision-making.

Key findings

  • Practitioners often chose simpler, more transparent models to secure internal buy-in: tax agencies used logistic regression or random forest, and one police contractor reduced 18,000 variables to 200, then 20, then 8, targeting around 75% accuracy.
  • Performance communication was difficult: one police modeller reported “100% accuracy” but only 20% precision because of 40 million records with few crashes, while a tax analytics lead said people “tend to lose faith” if their preferred risk indicators are absent.
  • Data changes and feedback loops were prominent: a human trafficking model became a car-wash detector after intelligence focused on car washes, and a predictive policing project saw the probability of being correct “tank” before re-evaluating learning every 2–3 weeks.
  • Discretion and reliance varied by context: helicopter pilots resisted directions while motorcycle police liked sequential high-risk lists; an ethics committee advised working down rank-ordered vulnerability lists, with professional judgement always overriding the system.

Methods

  • Semi-structured interviews with 27 individuals in late 2016, predominantly public servants and attached contractors in modelling or project management, across 5 OECD countries over 3 continents; 17 interviews were in person and 10 by telephone, each lasting 40–60 minutes with one interviewer; the US was not included.
  • Ad hoc sampling drew on grey literature, freedom of information requests, snowball sampling, direct inquiries, and news databases; application areas included taxation, child protection, policing, justice, emergency response, and interior security.
  • Conversations were not recorded; verbose notes were taken and typed the same day, then analysed with open coding in NVivo 11 and iteratively grouped using a public sector values framework.

Limitations

  • Stated in the paper: the sample was not intended to be representative, used ad hoc sampling, and the countries are not named to reduce informant identification; the US was excluded; audio recording was avoided due to sensitivity, so the study relied on notes; the authors did not seek to validate existing theoretical frameworks.
  • Evident from the text: findings rest on interviewer notes rather than transcripts, and the paper does not describe independent coding, member checking, or quantification of themes; the small, snowballed sample may overrepresent engaged practitioners.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1145/3173574.3174014

Cites

87 of 87 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

10 Valid · 2 Not valid · 26 Not checked · 60 Own results and interpretations

34 more claims · 2 with their cited sources checked

Create a free account to see all 98 claims and to verify the 1 reference still open.

See all claims — free account
  • Probably not a substantial claim. 48
  • It may state two separate claims as one. 22

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

15 more claims about this paper

Create a free account to see every claim other papers make about this one.

See all claims — free account

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

1 of 1

Citations

Cites

Show all 88 references

Research works printed but not linked: 13

  1. Julia Angwin, Jeff Larson, Surya Mattu, and Lauren Kirchner. 2016. Machine bias. ProPublica (2016). https://www.propublica.org/article/ machine-bias-risk-assessments-in-criminal-sentencing
  2. danah boyd. 2016. Undoing the neutrality of Big Data. Florida Law Review Forum 16 (2016), 226--232.
  3. Matthew Chalmers and Ian MacColl. 2003. Seamful and seamless design in ubiquitous computing. In Workshop at the crossroads: The interaction of HCI and systems issues in UbiComp, Vol. 8. https://perma.cc/2A3D-NMJP
  4. 10.5555/1213575 · doi:10.5555/1213575
  5. House of Common Science and Technology Committee. 2016. Robotics and artificial intelligence (HC 145). The House of Commons, London. http://www.publications.parliament.uk/pa/cm201617/ cmselect/cmsctech/145/145.pdf
  6. House of Commons Science and Technology Committee. 2016. The big data dilemma (HC 468). House of Commons, London. http://www.publications.parliament. uk/pa/cm201516/cmselect/cmsctech/468/468.pdf
  7. 10.5555/1964882 · doi:10.5555/1964882
  8. Robin Moore (Ed.). 2015. A compendium of research and analysis on the Offender Assessment System. Ministry of Justice Analytical Series London. DOI: http://dx.doi.org/https://perma.cc/W2FT-NFWZ
  9. BBC News. 2016. Kent slavery raids 'uncover 21 victims'. BBC News (7 Dec. 2016). https://perma.cc/AM4S-RMHR
  10. 10.5555/1462129 · doi:10.5555/1462129
  11. Nick Seaver. 2013. Knowing algorithms. Media in Transition 8 (2013). https://perma.cc/8USJ-VTWS
  12. The Royal Society. 2017. Machine learning: The power and promise of computers that learn by example. The Royal Society, London. https://royalsociety.org/~/ media/policy/projects/machine-learning/publications/ machine-learning-report.pdf
  13. The Royal Society and the British Academy. 2017. Data management and use: Governance in the 21st Century. The Royal Society and the British Academy, London. https://royalsociety.org/~/media/policy/projects/ data-governance/data-management-governance.pdf

Cited by

See all 490

Metadata

Authors
Veale, M., Van Kleek, M., & Binns, R.
Affiliations
  • University College London (GB)
  • University of Oxford, Oxford, Oxfordshire, United Kingdom
  • University of Oxford (GB)
Venue
Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems
Publisher
ACM
Published
2018-04-21
Pages
1-14
OpenAlex
W2786242872
Cited by
490
References
88
Citations by year
2026: 54 · 2025: 67 · 2024: 66 · 2023: 61 · 2022: 69 · 2021: 68 · 2020: 54 · 2019: 40 · 2018: 11
FWCI
78.16
Type
conference paper
Language
en
Open access
Locations
  • CHI Conference on Human Factors in Computing Systems (publishedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion)
  • RePEc: Research Papers in Economics (submittedVersion)
  • UCL Discovery (University College London) (submittedVersion)
  • SocArXiv (OSF Preprints) (submittedVersion)
  • arXiv (Cornell University) (cc-by)
Funders
EPSRC (EP/M507970/1)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
  • Social Sciences › Political Science and International Relations › E-Government and Public Services
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
Keywords
algorithmic fairness, accountability, public sector decision-making, algorithmic transparency, public value, concept drift, secondary data sources, street-level bureaucrats, public services

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Fairness and Accountability Design Needs for Algorithmic Support in High-Stakes Public Sector Decision-Making. https://app.cochat.ai/papers/10.1145/3173574.3174014
The paper
Veale, M., Van Kleek, M., & Binns, R. (2018). Fairness and Accountability Design Needs for Algorithmic Support in High-Stakes Public Sector Decision-Making. Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3173574.3174014

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.