Algorithmic pollution: Making the invisible visible
Open accessThe DOI matches its Crossref record; 31 of the 31 references it cites were found in a registry or an index.
Abstract
In this article, we focus on the growing evidence of unintended harmful societal effects of automated algorithmic decision-making in transformative services (e.g. social welfare, healthcare, education, policing and criminal justice), for individuals, communities and society at large. Drawing from the long-established research on social pollution, in particular its contemporary ‘pollution-as-harm’ notion, we put forward a claim – and provide evidence – that these harmful effects constitute a new type of digital social pollution, which we name ‘algorithmic pollution’. Words do matter, and by using the term ‘pollution’, not as a metaphor or an analogy, but as a transformative redefinition of the digital harm performed by automated algorithmic decision-making, we seek to make it visible and recognized. By adopting a critical performative perspective, we explain how the execution of automated algorithmic decision-making produces harm and thus performs algorithmic pollution. Recognition of the potential for unintended harmful effects of algorithmic pollution, and their examination as such, leads us to articulate the need for transformative actions to prevent, detect, redress, mitigate and educate about algorithmic harm. These actions, in turn, open up new research challenges for the information systems community.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Meeting the Universe Halfway: Quantum Physics and the Entanglement of Matter and Meaning
Barad, K. (2006) · doi:10.2307/j.ctv12101zq · Cited by 10,472
The Black Box Society: The Secret Algorithms That Control Money and Information
Pasquale, F. (2015) · doi:10.4159/harvard.9780674736061 · Cited by 3,796
Sociomaterial practices: exploring technology at work
Orlikowski, W. J. (2007) · Organization Studies · doi:10.1177/0170840607081138 · Cited by 3,248
Big other: Surveillance Capitalism and the Prospects of an Information Civilization
Zuboff, S. (2015) · Journal of Information Technology · doi:10.1057/jit.2015.5 · Cited by 3,205
Artificial intelligence and the future of work: Human-AI symbiosis in organizational decision making
Jarrahi, M. H. (2018) · Business Horizons · doi:10.1016/j.bushor.2018.03.007 · Cited by 2,772 · cited 3×
“These are the arguments for proposing human-machine collaboration in which humans and AI technologies would have complementary roles, drawing on their comparative strengths (Davenport, 2016; Jarrahi, 2018).”
BackgroundAlgorithms of Oppression: How Search Engines Reinforce Racism
Noble, S. U. (2020) · doi:10.2307/j.ctt1pwt9w5 · Cited by 2,709
10 Sociomateriality: Challenging the Separation of Technology, Work and Organization
Orlikowski, W. J., & Scott, S. V. (2008) · Academy of Management Annals · doi:10.5465/19416520802211644 · Cited by 2,571
Fair prediction with disparate impact: a study of bias in recidivism prediction instruments
Chouldechova, A. (2017) · Big Data · doi:10.1089/big.2016.0047 · Cited by 2,159 · cited 1×
“…now argue, the widely-used ethical frameworks of fairness, accountability, and transparency (FAT-ML) of algorithms do not go far enough, and need to be augmented with the concepts of justice and human rights when considering AADM in transformative services (Chouldechova, 2017; Gurses et al.,”
BackgroundLicklider, J. C. R. (1960) · IRE Transactions on Human Factors in Electronics · doi:10.1109/thfe2.1960.4503259 · Cited by 1,938
Bellamy, R. K. E., Dey, K., Hind, M., Hoffman, S. C., Houde, S., Kannan, K., Lohia, P., Martino, J., Mehta, S., Mojsilovic, A., et al. (2018) · arXiv · doi:10.48550/arxiv.1810.01943 · Cited by 1,581
Service research priorities in a rapidly changing context
Ostrom, A. L., Parasuraman, A., Bowen, D. E., Patrício, L., & Voss, C. A. (2015) · Journal of Service Research · doi:10.1177/1094670515576315 · Cited by 1,469 · cited 1×
“The emerging multidisciplinary field of Transformative Services Research (TSR) focuses on services that transform human lives by having a direct impact on the wellbeing of individuals, communities and the wider society (Anderson et al., 2013; Anderson and Ostrom, 2015; Ostrom et al., 2015).”
BackgroundThe Black Box Society: The Secret Algorithms That Control Money and Information
(2016) · Contemporary Sociology: A Journal of Reviews · doi:10.1177/0094306116641409c · Cited by 1,334
Loebbecke, C., & Picot, A. (2015) · The Journal of Strategic Information Systems · doi:10.1016/j.jsis.2015.08.002 · Cited by 1,317 · cited 1×
“…studies that focus on the harmful effects of AI and automated algorithmic decisionmaking on individuals, organizations and society – see for example Loebbecke and Picot (2015), Newell and Mirabelli (2015), Zuboff (2015, Galliers et al. (2017), Markus (2017), Schultze et al. (2018), Clarke…”
BackgroundThe social power of algorithms
Beer, D. (2017) · Information, Communication & Society · doi:10.1080/1369118x.2016.1216147 · Cited by 945 · cited 1×
“Through datafication practices, algorithms are starting to perform new worlds,2 by creating and enacting ‘new behaviours, new expressions, new actors and new realities’ (Muller, 2015: 29) and exerting power over us (Beer, 2017) through these effects (Diakopoulos, 2013).”
MethodAI Fairness 360: An extensible toolkit for detecting and mitigating algorithmic bias
Bellamy, R. K. E., Dey, K., Hind, M., Hoffman, S. C., Houde, S., Kannan, K., Lohia, P., Martino, J., Mehta, S., Mojsilović, A., et al. (2019) · IBM Journal of Research and Development · doi:10.1147/jrd.2019.2942287 · Cited by 884
Newell, S., & Marabelli, M. (2015) · The Journal of Strategic Information Systems · doi:10.1016/j.jsis.2015.02.001 · Cited by 494 · cited 3×
“…harmful effects of AI and automated algorithmic decisionmaking on individuals, organizations and society – see for example Loebbecke and Picot (2015), Newell and Mirabelli (2015), Zuboff (2015, Galliers et al. (2017), Markus (2017), Schultze et al. (2018), Clarke (2019a, 2019b), CecezKecmanovic…”
BackgroundWe Are Data: Algorithms and the Making of Our Digital Selves
Cheney-Lippold, J. (2020) · doi:10.2307/j.ctt1gk0941 · Cited by 436
Breaking the vicious cycle of algorithmic management: A virtue ethics approach to people analytics
Gal, U., Jensen, T. B., & Stein, M.-K. (2020) · Information and Organization · doi:10.1016/j.infoandorg.2020.100301 · Cited by 342 · cited 2×
“…in IS has only recently recognized the unintended harmful consequences of AI and AADM (Bohme, 2019; Cecez-Kecmanovic, 2019; Clarke, 2019a, 2019b; Gal et al., 2020; Galliers et al., 2017; Leidner, 2019; March, 2019; Markus, 2017; Schultze et al., 2018), but it is yet to consider the context of…”
BackgroundAlgorithms, Governance, and Governmentality: On Governing Academic Writing
Introna, L. D. (2016) · Science, Technology, & Human Values · doi:10.1177/0162243915587360 · Cited by 339 · cited 1×
“While the harmful effects of datafication practices and ‘algorithmic doing’ (Introna, 2016) continue to be reported in the literature by more and more studies, there is a paucity of research that engages with a theoretical explanation of how algorithms perform these effects.”
BackgroundExploring Data Justice: Conceptions, Applications and Directions
Dencik, L., Hintz, A., Redden, J., & Treré, E. (2019) · Information, Communication & Society · doi:10.1080/1369118x.2019.1606268 · Cited by 300 · cited 3×
“The credit agency, Experian, is making its demographic segmentation software called Mosaic (2019) and data available to local government agencies in the UK where it is used in a variety of ways, such as for risk assessment of defendants in court cases (Dencik et al., 2019a).”
BackgroundFraser, N. (2008) · Critical Inquiry · doi:10.1086/589478 · Cited by 236 · cited 2×
“Drawing from the work by social justice theorist Nancy Fraser (2008), we propose that mitigation of algorithmic pollution requires an ongoing dialogical process, between a civil society track with grass-root insights into harm experienced by individuals and an institutional track (e.g. government regulators), with leg…”
Classification with Fairness Constraints: A Meta-Algorithm with Provable Guarantees
Celis, L. E., Huang, L., Keswani, V., & Vishnoi, N. K. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287586 · Cited by 231 · cited 3×
“Consequently, the solutions suggested in the related literature (see, for example, Bellamy et al, 2018; Celis et al., 2018; Zemel et al., 2013) are also grounded in a technical rationality.”
BackgroundAlgorithmic Impact Assessments and Accountability: The Co-construction of Impacts
Metcalf, J., Moss, E., Watkins, E. A., Singh, R., & Elish, M. C. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445935 · Cited by 195
We Are Data: Algorithms and the Making of Our Digital Selves
(2017) · Project Muse (Johns Hopkins University) · doi:10.18574/9781479888702 · Cited by 166
The Rule of Law and Automation of Government Decision‐Making
Zalnieriute, M., Moses, L. B., & Williams, G. (2019) · The Modern Law Review · doi:10.1111/1468-2230.12412 · Cited by 140 · cited 3×
“…the intended beneficiaries of AADM often being the most vulnerable members of society, these algorithms, Eubanks (2018: 11) warns, are fast becoming ‘the new tools for digital poverty management’, sometimes with life-threatening consequences (Eubanks, 2018; Carney, 2018; Zalnieriute et al., 2019).”
BackgroundThe Rule of Law and Automation of Government Decision-Making
Zalnieriute, M., Bennett Moses, L., & Williams, G. (2019) · doi:10.2139/ssrn.3348831 · Cited by 140
Gebru, T. (2020) · The Oxford Handbook of Ethics of AI · doi:10.1093/oxfordhb/9780190067397.013.16 · Cited by 111
Keddell, E. (2019) · Social Sciences · doi:10.3390/socsci8100281 · Cited by 109
New Games, New Rules, New Scoreboards: The Potential Consequences of Big Data
Markus, M. L. (2015) · Journal of Information Technology · doi:10.1057/jit.2014.28 · Cited by 72
Galliers, R. D., Newell, S., Shanks, G., & Topi, H. (2017) · The Journal of Strategic Information Systems · doi:10.1016/j.jsis.2017.08.002 · Cited by 70
(2020) · doi:10.1145/3351095.3375784 · Cited by 29 · cited 2×
“For example, the 2020 ACM conference on Fairness, Accountability and Transparency (now known as FAccT), has broadened its scope to include ‘ethics and policy’, with papers investigating the social good (Washington and Kuo, 2020), collective freedom (Terzis, 2020), and algorithmic targeting of social policies (Noriega-…”
Background
Cited by
International Business, digital technologies and sustainable development: Connecting the dots
Ciulli, F., & Kolk, A. (2023) · Journal of World Business · doi:10.1016/j.jwb.2023.101445 · Cited by 86
Fan, Y., & Liu, X. (2022) · Frontiers in Psychology · doi:10.3389/fpsyg.2022.1009173 · Cited by 40
Kumar, A., Bhattacharyya, S. S., & Krishnamoorthy, B. (2023) · Journal of Enterprise Information Management · doi:10.1108/jeim-09-2022-0307 · Cited by 34
Metadata
- Authors
- Marjanovic, O., Cecez-Kecmanovic, D., & Vidgen, R.
- Venue
- Journal of Information Technology
- Publisher
- SAGE Publications
- Published
- 2021-05-28
- Volume
- 36
- Issue
- 4
- Pages
- 391-408
- ISSN
- 0268-3962 (ISSN-L)
- 1466-4437
- OpenAlex
- W3161540876
- Cited by
- 51
- References
- 31
- FWCI
- 9.89
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.