The DOI matches its Crossref record; 45 of the 53 references it cites were found in a registry or an index; none of the 36 sources its claims cite is checked for support yet.
The fallacy of inscrutability
Open accessAbstract
Contrary to the criticism that mysterious, unaccountable black-box software systems threaten to make the logic of critical decisions inscrutable, we argue that algorithms are fundamentally understandable pieces of technology. Software systems are designed to interact with the world in a controlled way and built or operated for a specific purpose, subject to choices and assumptions. Traditional power structures can and do turn systems into opaque black boxes, but technologies can always be understood at a higher level, intensionally in terms of their designs and operational goals and extensionally in terms of their inputs, outputs and outcomes. The mechanisms of a system's operation can always be examined and explained, but a focus on machinery obscures the key issue of power dynamics. While structural inscrutability frustrates users and oversight entities, system creators and operators always determine that the technologies they deploy are fit for certain uses, making no system wholly inscrutable. We investigate the contours of inscrutability and opacity, the way they arise from power dynamics surrounding software systems, and the value of proposed remedies from disparate disciplines, especially computer ethics and privacy by design. We conclude that policy should not accede to the idea that some systems are of necessity inscrutable. Effective governance of algorithms comes from demanding rigorous science and engineering in system design, operation and evaluation to make systems verifiably trustworthy. Rather than seeking explanations for each behaviour of a computer system, policies should formalize and make known the assumptions, choices, and adequacy determinations associated with a system. This article is part of the theme issue ‘Governing artificial intelligence: ethical, legal, and technical opportunities and challenges’.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
0 of 36 cited sources checked · 0 supported · 0 flagged
In the paper
“the audit nonetheless reveals a fairness issue that was not previously observed: black arrestees are rated incorrectly as having a high risk of being re-arrested upon release nearly twice as often as white arrestees.”
§ The fallacy of inscrutability > 3. Extensional understanding: inputs, outputs, outcomes, audit and review
What the sources say
- Not checked yetcites [39]
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIn the paper
“For example, a review by ProPublica famously discovered a disparity in false positive rates for black arrestees in the COMPAS evaluation, which is used to support bail decisions and other criminal justice professional judgement [39].”
§ The fallacy of inscrutability > 3. Extensional understanding: inputs, outputs, outcomes, audit and review
What the sources say
- Not checked yetcites [39]
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesA conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“That is, the claimed inscrutability of computer systems is a category error: by claiming that a system's actions cannot be understood, critics ascribe values to mechanical technologies and not to the humans who designed, built and fielded them.”
§ The fallacy of inscrutability > 1. Introduction
Tests✓ Asserted: passes? Checkable: unclear✓ Load-bearing: passes✓ Based: passesA conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Traditional power structures can and do turn systems into opaque black boxes, but technologies can always be understood at a higher level, intensionally in terms of their designs and operational goals and extensionally in terms of their inputs, outputs and outcomes.”
§ The fallacy of inscrutability > Abstract
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes? Based: unclearA conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Because such learning technologies are, as a general matter, difficult to understand at a mechanistic level, such an approach serves to create a barrier to understanding.”
§ The fallacy of inscrutability > (a) Responses to inscrutability: explanations and the abdication of fault
Tests✓ Asserted: passes? Checkable: unclear✓ Load-bearing: passes✓ Based: passesA conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Computer systems are not pure, neutral tools, but products of their sociotechnical context, and must be considered as such.”
§ The fallacy of inscrutability > 1. Introduction
Tests✓ Asserted: passes? Checkable: unclear✓ Load-bearing: passes✓ Based: passesA conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The mechanisms of a system's operation can always be examined and explained, but a focus on machinery obscures the key issue of power dynamics.”
§ The fallacy of inscrutability > Abstract
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes? Based: unclearIn the paper
“Explanations risk being 'just-so' stories, which lend false credence to an incorrect construct [14].”
§ The fallacy of inscrutability > (a) Responses to inscrutability: explanations and the abdication of fault
What the sources say
- Not checked yetcites [14]
Tests✓ Asserted: passes? Checkable: unclear✓ Load-bearing: passes✓ Based: passesA conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Contrary to the criticism that mysterious, unaccountable black-box software systems threaten to make the logic of critical decisions inscrutable, we argue that algorithms are fundamentally understandable pieces of technology.”
- Also stated in: § 1. Introduction · plain
“Instead, this paper argues that, while software systems may be opaque to certain observers in certain cases, at base they can and must be understood, both at a technical level and in a human context.”
§ The fallacy of inscrutability > Abstract
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes? Checkable: unclear✓ Load-bearing: passes✓ Based: passes- Also stated in: § 1. Introduction · plain
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“While demanding perfect foresight puts too much faith in engineering, it is certainly true that systems which are designed to achieve certain goals in a trustworthy and verifiable way can reasonably be judged on an absolute scale.”
§ The fallacy of inscrutability > (a) Responses to inscrutability: explanations and the abdication of fault
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes? Checkable: unclear✓ Load-bearing: passes✓ Based: passes
- Probably not a substantial claim. 60
- It may state two separate claims as one. 30
In the paper
“For this reason, tests are often considered to be some of the best and most up-to-date documentation of a software system, to the point that some methodologies advocate writing tests in advance of writing the actual software, as tests constitute a de facto specification for a system [27].”
§ The fallacy of inscrutability > 2. Intensional understanding: goals, requirements and mechanisms
What the sources say
- Not checked yetcites [27]
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Such a right sidesteps questions of aggregate effects like group fairness and nondiscrimination, while failing to provide individual understanding of particular decisions or to address power dynamics between individuals and computer systems or to facilitate governance of those”
§ The fallacy of inscrutability > 1. Introduction
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The mechanisms of a system's operation can always be examined and explained, but a focus on machinery obscures the key issue of power dynamics.”
§ The fallacy of inscrutability > Abstract
Tests✓ Asserted: passes✗ Checkable: fails✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim. It may be too vague to check against evidence.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Further, software products have far and away more defects and bugs than other kinds of engineered artefacts, to the point that most people inside and outside the industry consider bugs and defects in software inevitable.”
§ The fallacy of inscrutability > 2. Intensional understanding: goals, requirements and mechanisms
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. The paper may give no reason why it holds. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The role of detailed, mechanistic explanations in creating understandable software systems is much smaller than the role of careful validation, experimentation, and a focus on capturing robust mechanisms, especially causal ones.”
§ The fallacy of inscrutability > (a) Responses to inscrutability: explanations and the abdication of fault
Tests✓ Asserted: passes✗ Checkable: fails✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Explanation is not an unalloyed good, both because it is only useful when it properly engages the context of the tool being explained and because explanations, at a technical level, do not necessarily provide understanding or improve the interpretability of a particular technical tool.”
§ The fallacy of inscrutability > (a) Responses to inscrutability: explanations and the abdication of fault
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes? Checkable: unclear✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Instead, the priorities of the organization managing the software development are mediated into the product and tested for fit with the market rapidly, causing the benefits of sustained iteration to accrue not to overall social good, but to the software development organization alone.”
§ The fallacy of inscrutability > 2. Intensional understanding: goals, requirements and mechanisms
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. The paper may give no reason why it holds. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Systems can be understood in terms of their design goals and the mechanisms of their construction and operation.”
§ The fallacy of inscrutability > 1. Introduction
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✗ Checkable: fails✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“To improve these sequential engineering paradigms, the software industry invented iterative methodologies, where the phases form a true cycle and the things learned during development, implementation and maintenance can be fed back into design objectives to create requirements for future iterations of the software product.”
§ The fallacy of inscrutability > 2. Intensional understanding: goals, requirements and mechanisms
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Such a right sidesteps questions of aggregate effects like group fairness and nondiscrimination, while failing to provide individual understanding of particular decisions or to address power dynamics between individuals and computer systems or to facilitate governance of those”
§ The fallacy of inscrutability > 1. Introduction
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. The paper may give no reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“Transparency of these instruments alone does not provide understanding, however: there are fundamental technical limits on the capacity for software analysis [15,16].”
§ The fallacy of inscrutability > 2. Intensional understanding: goals, requirements and mechanisms
What the sources say
- Not checked yetcites [15]
- Not checked yetcites [16]
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Such a right sidesteps questions of aggregate effects like group fairness and nondiscrimination, while failing to provide individual understanding of particular decisions or to address power dynamics between individuals and computer systems or to facilitate governance of those”
§ The fallacy of inscrutability > 1. Introduction
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In these models, design goals are a living product of the process of software engineering rather than a static input to software development.”
§ The fallacy of inscrutability > 2. Intensional understanding: goals, requirements and mechanisms
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Understanding how computer systems are designed and built, including understanding when trade-offs between competing goals were made by the designers and why, dispels much of the inscrutability the systems may have when viewed as a whole or from the perspective of someone affected by such a system.”
§ The fallacy of inscrutability > 2. Intensional understanding: goals, requirements and mechanisms
Tests✓ Asserted: passes✗ Checkable: fails✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“It may in many cases be unnecessary to understand the precise mechanisms of an algorithmic system, just as we do not understand how humans make decisions, so long as we describe the outlines of the system's interaction with the world (e.g.doing good science, understanding causal mechanisms versus correlations).”
§ The fallacy of inscrutability > (a) Responses to inscrutability: explanations and the abdication of fault
Load-bearingStrength:hedged· not yet calibratedTests✓ Asserted: passes✗ Checkable: fails✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Further, because both the software product and its specification are living outputs of the engineering process, approval, review and control suffer.”
§ The fallacy of inscrutability > 2. Intensional understanding: goals, requirements and mechanisms
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Software systems are designed to interact with the world in a controlled way and built or operated for a specific purpose, subject to choices and assumptions.”
§ The fallacy of inscrutability > Abstract
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. The paper may give no reason why it holds. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“But this barrier is illusory-inscrutability in these contexts is a choice, perhaps one born of laziness, on the part of the system's controllers.”
§ The fallacy of inscrutability > (a) Responses to inscrutability: explanations and the abdication of fault
Tests✓ Asserted: passes✗ Checkable: fails✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim. It may be too vague to check against evidence.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“That is not to say that all systems are of necessity understandable-too often, systems are built (in whole or in part) irresponsibly by simply applying learning technologies to data without considering what phenomena the results should capture or to what ends the resulting system should function.”
§ The fallacy of inscrutability > (a) Responses to inscrutability: explanations and the abdication of fault
Tests✓ Asserted: passes✗ Checkable: fails✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Sometimes, it is necessary to consider a system's context dynamically and to account for how the system changes and is changed by its context.”
§ The fallacy of inscrutability > (a) Responses to inscrutability: explanations and the abdication of fault
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✗ Checkable: fails✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim. It may be too vague to check against evidence. It may state two separate claims as one.
In the paper
“Regardless of the development methodology being used, it is common and a recognized best practice for software to be tested for consistency with its specification [26].”
§ The fallacy of inscrutability > 2. Intensional understanding: goals, requirements and mechanisms
What the sources say
- Not checked yetcites [26]
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Traditional power structures can and do turn systems into opaque black boxes, but technologies can always be understood at a higher level, intensionally in terms of their designs and operational goals and extensionally in terms of their inputs, outputs and outcomes.”
§ The fallacy of inscrutability > Abstract
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes? Checkable: unclear✓ Load-bearing: passes? Based: unclearIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In turn, because they have determined that it is appropriate to use technology for particular ends, a system's controllers have also provided an understanding of the outcomes created by their technology: the reasons they have for believing that the system is adequate to that purpose.”
§ The fallacy of inscrutability > 1. Introduction
Tests✓ Asserted: passes✗ Checkable: fails✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim. It may be too vague to check against evidence.
In the paper
“Because software systems are developed without the process rigidities of other types of engineering work or the constraints of physical reality, large software development projects are more prone to failing both to produce outputs that meet the objectives of their customers and to produce outputs at all [22].”
§ The fallacy of inscrutability > 2. Intensional understanding: goals, requirements and mechanisms
What the sources say
- Not checked yetcites [22]
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“But the presence of such defects is a choice, not an inevitability.”
§ The fallacy of inscrutability > 2. Intensional understanding: goals, requirements and mechanisms
Tests✓ Asserted: passes✗ Checkable: fails✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In many domains where perfect foresight is impossible and where the outcomes from interventions are not foreseeable, such as medicine, we nonetheless hold practitioners to a standard of practice and are comfortable punishing practitioners when their actions do not rise to a sufficient standard of care.”
§ The fallacy of inscrutability > (a) Responses to inscrutability: explanations and the abdication of fault
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Testing measures the functionality of software both in the sense that it reveals defects and also in the sense that it verifies that the software acts according to its goals.”
§ The fallacy of inscrutability > 2. Intensional understanding: goals, requirements and mechanisms
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Any inability to describe how or why a particular advertisement was presented in a particular situation is merely a design choice, not an inevitability of the complexity of large systems.”
§ The fallacy of inscrutability > (a) Responses to inscrutability: explanations and the abdication of fault
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✗ Checkable: fails✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, this flexibility comes at a cost: because the process focuses on iterative improvement of an early prototype, outcomes can be path dependent and it is common for only the most obvious requirements to be addressed, leaving little time for features which are important, but only affect a small part of a system's user base.”
§ The fallacy of inscrutability > 2. Intensional understanding: goals, requirements and mechanisms
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“While agile software development can improve the speed at which functional software is delivered, it does so by creating incentives to avoid thinking deeply about the consequences of technical requirements or engaging stakeholders outside the day-today development process.”
§ The fallacy of inscrutability > 2. Intensional understanding: goals, requirements and mechanisms
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes? Based: unclearIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“But even here, systems can be designed to sidestep these limits and to support that interrogation-inscrutability remains a choice that can be accommodated for in system design.”
§ The fallacy of inscrutability > (a) Responses to inscrutability: explanations and the abdication of fault
Tests✓ Asserted: passes✗ Checkable: fails✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Additionally, systems can also be understood in terms of their inputs and outputs and the outcomes that result from their application in a particular context.”
§ The fallacy of inscrutability > 1. Introduction
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✗ Checkable: fails✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We offer that explanations must speak to the decisions made during the design of a computer system, as such information is always available and always fulfils the key requirements of a meaningful explanation.”
§ The fallacy of inscrutability > 2. Intensional understanding: goals, requirements and mechanisms
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✗ Checkable: fails✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Even carefully designed systems can miss important facts about the world, and it is important to verify that systems are operating as intended and satisfying their articulated requirements.”
§ The fallacy of inscrutability > 3. Extensional understanding: inputs, outputs, outcomes, audit and review
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“If a system's specification properly reflects the system's requirements and design goals-that is, if the system is the result of proper requirements engineering-then fidelity to that specification demonstrates fitness for purpose.”
§ The fallacy of inscrutability > 4. Explanation, understanding, validation and policy interventions
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
None of the sources cited for this claim have been checked yet.
In the paper
“While audits cannot, for technical reasons, ever guarantee that they will completely suss out the mechanisms of a computer system [15,16], they can, in many cases, provide useful information about when a system is causing harm or when it is behaving in a way different from what is expected.”
§ The fallacy of inscrutability > 3. Extensional understanding: inputs, outputs, outcomes, audit and review
What the sources say
- Not checked yetcites [15]
- Not checked yetcites [16]
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, transparency neither requires nor is provided by either open source or source availability alone.”
§ The fallacy of inscrutability > (a) Transparency as a solution to inscrutability
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Because the reproducibility of computer outputs is a design choice, we can again say that failing to make a system reproducible when it should be constitutes malpractice.”
§ The fallacy of inscrutability > (a) Transparency as a solution to inscrutability
Tests✓ Asserted: passes✗ Checkable: fails✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim. It may be too vague to check against evidence.
In the paper
“To that end, sufficient transparency may simply mean disclosing the fact and scope of data processing in a computer system, as is required by the GDPR in the EU.”
§ The fallacy of inscrutability > (a) Transparency as a solution to inscrutability
What the sources say
- Not checked yetcites GDPR
Strength:hedged· not yet calibratedCausal:conditional· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Ideally, bias is accounted for as assumptions are passed through a system; explicitly debiasing data risks inventing counterfactuals which are not supportable or testable during later validation phases.”
§ The fallacy of inscrutability > 5. Conclusion
Tests✓ Asserted: passes✗ Checkable: fails✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Beyond individual decisions, examination of group-level system outcomes is a cornerstone of quantitative evaluation for policy interventions of all kinds.”
§ The fallacy of inscrutability > 3. Extensional understanding: inputs, outputs, outcomes, audit and review
Strength:plain· not yet calibratedTests✓ Asserted: passes✗ Checkable: fails✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Understanding the design process that led to a software system helps elucidate the mechanisms by which the system operates, in turn providing insight into why particular outcomes or outputs were produced.”
§ The fallacy of inscrutability > 2. Intensional understanding: goals, requirements and mechanisms
Tests✓ Asserted: passes? Checkable: unclear✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“Similar audits have turned up issues of bias reinforcement in systems that guide police to portions of a city to patrol [36], and in e-commerce platforms [35].”
§ The fallacy of inscrutability > 3. Extensional understanding: inputs, outputs, outcomes, audit and review
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Viewing computer system behaviours as though they are uncontrollable ignores the fact that these systems are human artefacts, built to a purpose by some human agency that must be accountable for the behaviours of those artefacts.”
§ The fallacy of inscrutability > 2. Intensional understanding: goals, requirements and mechanisms
Tests✓ Asserted: passes✗ Checkable: fails✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim. It may be too vague to check against evidence.
- Rests on an assumption: Human agents who design, build, and field computer systems are accountable for the behaviours of those systems.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“All software is designed and built by people, and responsible organizations will create traces of their decision-making processes which provide insight into the meaning and function of the artefacts they produce.”
§ The fallacy of inscrutability > 2. Intensional understanding: goals, requirements and mechanisms
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. The paper may give no reason why it holds. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In turn, black arrestees find themselves systematically less able to mount a defence to the case against them, reinforcing historical and structural discrimination.”
§ The fallacy of inscrutability > 3. Extensional understanding: inputs, outputs, outcomes, audit and review
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The extent to which particular details are or are not foreseeable may differ depending on the discipline and background of the viewer.”
§ The fallacy of inscrutability > 2. Intensional understanding: goals, requirements and mechanisms
Strength:hedged· not yet calibratedCausal:conditional· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Yet, calls for requiring explanations of computer system behaviour as a path to governing those systems often do not address the question of who is meant to receive what knowledge through the explanation.”
§ The fallacy of inscrutability > 2. Intensional understanding: goals, requirements and mechanisms
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Importantly, the mere fact of a mistake is insufficient to define malpractice; rather, malpractice involves situations where bad outcomes could have been avoided by more responsible behaviour on the part of a system's controllers.”
§ The fallacy of inscrutability > 5. Conclusion
Tests✓ Asserted: passes✗ Checkable: fails✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In either case, explanations must create understanding by humans of computer systems in context, though they are not on their own necessary or sufficient to create that understanding.”
§ The fallacy of inscrutability > 4. Explanation, understanding, validation and policy interventions
Tests✓ Asserted: passes✗ Checkable: fails✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim. It may be too vague to check against evidence. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Any system is a model of the world, and too often the called-for explanation is only a description of that model.”
§ The fallacy of inscrutability > 4. Explanation, understanding, validation and policy interventions
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✗ Checkable: fails✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds.
In the paper
“Good explanations must not only explain why a behaviour occurred or why some other behaviour did not occur as a product of a system, but also capture and transmit the knowledge of the context underlying a system's mechanism of action as a process within a broad sociotechnical system [30].”
§ The fallacy of inscrutability > 2. Intensional understanding: goals, requirements and mechanisms
What the sources say
- Not checked yetcites [30]
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✗ Checkable: fails✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim. It may be too vague to check against evidence. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Of course, this does not mean that all computer systems are easily analysable.”
§ The fallacy of inscrutability > 5. Conclusion
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The concept of 'open-source' software, or software which has source code that is both freely available and permissively licensed, strengthens transparency by combining detailed source code disclosure with the ability to derive additional knowledge by recompiling or modifying the software.”
§ The fallacy of inscrutability > (a) Transparency as a solution to inscrutability
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
In the paper
“Because systems are designed to achieve particular ends, we cannot disentangle the environmental contributions to a particular behaviour from the designed mechanism [31]-the mechanism must, after all, compensate for its environment if it correctly embodies its goals.”
§ The fallacy of inscrutability > 2. Intensional understanding: goals, requirements and mechanisms
What the sources say
- Not checked yetcites [31] source
Tests✓ Asserted: passes? Checkable: unclear✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“No artefact is properly comprehended without reference to its human context, and software systems are no different.”
§ The fallacy of inscrutability > 5. Conclusion
Tests✓ Asserted: passes✗ Checkable: fails✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim. It may be too vague to check against evidence. It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“The theoretical potential for discrimination in automated systems is well established [15,41,42].”
§ The fallacy of inscrutability > 3. Extensional understanding: inputs, outputs, outcomes, audit and review
What the sources say
- Not checked yetcites [15]
- Not checked yetcites [41]
- Not checked yetcites [42]
Strength:boosted· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Additionally, it can be difficult to understand at the design stage how a system will interact with the world.”
§ The fallacy of inscrutability > 3. Extensional understanding: inputs, outputs, outcomes, audit and review
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✗ Checkable: fails✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds.
In the paper
“Fidelity to a system's specification is one aspect of determining whether that system is correct, although we might also demand that the specification itself satisfy design goals or relate properly to the real world [16].”
§ The fallacy of inscrutability > 4. Explanation, understanding, validation and policy interventions
What the sources say
- Not checked yetcites [16]
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The design efforts that chose that tool as the right one for that context-and such efforts exist at least at an informal level for all systems-explain who believes that tool is the proper one and why it is correct to make use of it.”
§ The fallacy of inscrutability > 5. Conclusion
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✗ Checkable: fails✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“While these technologies promise much in the way of making systems more trustworthy, good explanations require careful consideration of who receives the explanation and what facts those people must extract from the explanations.”
§ The fallacy of inscrutability > 4. Explanation, understanding, validation and policy interventions
Tests✓ Asserted: passes✗ Checkable: fails✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim. It may be too vague to check against evidence.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In other cases, as with human decision-makers, it is sufficient to produce an unrelated justification of the result or a general description of how it was reached.”
§ The fallacy of inscrutability > 4. Explanation, understanding, validation and policy interventions
Strength:plain· not yet calibratedTests✓ Asserted: passes✗ Checkable: fails✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.
In the paper
“While transparency is often taken to mean the disclosure of source code or data, possibly to a trusted entity such as a regulator, this is neither necessary nor sufficient for improving understanding of a system [15], and it does not capture the full meaning of transparency (though transparency is often equated with disclosure of system internals in technical communities and scholarship).”
§ The fallacy of inscrutability > (a) Transparency as a solution to inscrutability
What the sources say
- Not checked yetcites [15]
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The choice of this task can also dispel inscrutability, as it takes the focus off of possibly complex software systems and puts it on the sociotechnical system involving humans and their tools, concentrating analysis on where trustworthiness is necessary.”
§ The fallacy of inscrutability > 4. Explanation, understanding, validation and policy interventions
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Further, they help the controllers of a system understand how to set their goals, or how to achieve their goals in light of the world as it is.”
§ The fallacy of inscrutability > 2. Intensional understanding: goals, requirements and mechanisms
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For example, even well-designed systems following the pattern above may in some cases err or cause negative outcomes for their subjects.”
§ The fallacy of inscrutability > 5. Conclusion
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Non-causal reasoning is possible, but the way in which such systems can be validated is limited by their lack of undergirding mechanism and the extent to which their constructs are valid.”
§ The fallacy of inscrutability > 5. Conclusion
Tests✓ Asserted: passes✗ Checkable: fails✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“While information about the design process almost always exists and can improve understanding of a computer system, such information generally is not available to the people who need it for review.”
§ The fallacy of inscrutability > 3. Extensional understanding: inputs, outputs, outcomes, audit and review
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. The paper may give no reason why it holds. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In some cases, a mechanistic description of how a system computed a particular output will sufficiently explain that output.”
§ The fallacy of inscrutability > 4. Explanation, understanding, validation and policy interventions
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. The paper may give no reason why it holds.
In the paper
“Several scholars propose the use of an impact assessment process to aid in governing those systems [17][18][19]33], and assessments can even be legally required, as in Article 35 of the General Data Protection Regulation (GDPR) in Europe.”
§ The fallacy of inscrutability > 2. Intensional understanding: goals, requirements and mechanisms
What the sources say
- Not checked yetcites [17][18][19]33]
Tests✓ Asserted: passes✗ Checkable: fails✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim. It may be too vague to check against evidence.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We further reject that computer system behaviours should be held to a 'best-effort' standard because they are unforeseeable.”
§ The fallacy of inscrutability > 2. Intensional understanding: goals, requirements and mechanisms
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✗ Checkable: fails✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim. It may be too vague to check against evidence.
None of the sources cited for this claim have been checked yet.
In the paper
“In safety-critical applications, software systems are presented together with assurance cases, documents that are meant to convince a sceptical expert that the system operates as intended, and which detail the steps taken to assure fidelity of the system to its stated goals [28,29].”
§ The fallacy of inscrutability > 2. Intensional understanding: goals, requirements and mechanisms
What the sources say
- Not checked yetcites [28]
- Not checked yetcites [29]
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Failure to understand why relationships exist between data elements risks designing systems which codify existing unfairness and which reinforce problematic power dynamics.”
§ The fallacy of inscrutability > 5. Conclusion
Tests✓ Asserted: passes✗ Checkable: fails✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Inscrutability is a difficult problem in practice, in no small part because software systems have become incredibly powerful.”
§ The fallacy of inscrutability > 5. Conclusion
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✗ Checkable: fails✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
The paper relies on this without support.
Human agents who design, build, and field computer systems are accountable for the behaviours of those systems.
“built to a purpose by some human agency that must be accountable for the behaviours of those artefacts.”
“Responsibility and ethics attach not to the specifics of a technical tool, but rather to the ways that tool is used in a sociotechnical context, which are always considered when tools are created.”
The paper relies on this without support.
Interventions to protect users or subjects of computer systems should engage power relationships between controllers and affected parties.
“Interventions that aim to protect the subjects or users of a computer system must therefore engage these power relationships.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Yet, it is not impossible if the right questions are asked (Green 2018; Wachter et al. 2017) and closer attention is payed to how the design process can influence (Kroll 2018) whether an algorithm is more or less ‘ethically-aligned.’”
From From What to How: An Initial Review of Publicly Available AI Ethics Tools, Methods and Research to Translate Principles into Practices (2020) · cites this paper as Kroll 2018
In the citing paper
“As showed by Kroll (2018), the thesis that the understanding and transparency of algorithmic-assisted decision-making is limited by the inscrutability of the machine learning models and their algorithms (i.e. the fact that they are opaque or “black boxes”) is criticizable.”
From Transparency as design publicity: explaining and justifying inscrutable algorithms (2021) · cites this paper as Kroll (2018)
Citations
Cites
Causality: Models, Reasoning, and Inference
Pearl, J. (2009) · doi:10.1017/cbo9780511803161 · Cited by 11,653
Explanation in artificial intelligence: Insights from the social sciences
Miller, T. (2019) · Artificial Intelligence · doi:10.1016/j.artint.2018.07.007 · Cited by 5,302
Manifesto for agile software development
Beck, K., Beedle, M., van Bennekum, A., Cockburn, A., Cunningham, W., Fowler, M., & Thomas, D. A. (2001) · Cited by 4,284
The Black Box Society: The Secret Algorithms That Control Money and Information
Pasquale, F. (2015) · doi:10.4159/harvard.9780674736061 · Cited by 3,796
How the machine “thinks”: Understanding opacity in machine learning algorithms
Burrell, J. (2016) · Big Data & Society · doi:10.1177/2053951715622512 · Cited by 3,196 · cited 2×
“Such criticisms arise in the context of technical descriptions of complex computer systems or machine learning algorithms [4,5], as part of social critiques of computer systems [3], and in legal scholarship which aims to derive the appropriate regimes for computer system governance [1,6].”
BackgroundTowards A Rigorous Science of Interpretable Machine Learning
Doshi-Velez, F., & Kim, B. (2017) · arXiv · doi:10.48550/arxiv.1702.08608 · Cited by 3,121
Barocas, S., & Selbst, A. D. (2014) · doi:10.2139/ssrn.2477899 · Cited by 3,048
The Black Box Society: The Secret Algorithms That Control Money and Information
(2016) · Contemporary Sociology: A Journal of Reviews · doi:10.1177/0094306116641409c · Cited by 1,334
Wachter, S., Mittelstadt, B., & Floridi, L. (2017) · International Data Privacy Law · doi:10.1093/idpl/ipx005 · Cited by 1,308
Test Driven Development: By Example
Beck (2002) · Addison-Wesley Longman Publishing Co., Inc. eBooks · Cited by 1,278
Flagged first, then by citations
8 of the 53 references couldn’t be matched to a known paper.
Research works printed but not linked: 4
- Selbst AD Barocas S. The intuitive appeal of explainable machines. Preprint available at SSRN. (https://ssrn.com/abstract=3126971)
- Jones C, 2010, Software engineering best practices: lessons from successful projects in top companies
- Argyris C, 1977, Double loop learning in organizations, Harv. Bus. Rev., 55, 115
- Myers GJ, 1979, The art of software testing
Cited by
Sorted by citations
Items 101–131 of 131
Garaganov, A. V. (2024) · Sociopolitical sciences · doi:10.33693/2223-0092-2024-14-3-143-149 · Cited by 2
Accountable Algorithmic Decision-Making in Pharmaceutical Technology Development
Dupont, C., & Martin, J. (2025) · Journal of Applied Pharmaceutical Technologies and Systems · doi:10.68159/b722721918 · Cited by 2
Anić, N., & Anić, P. (2021) · National security and the future · doi:10.37458/nstf.21.3.4 · Cited by 3
Designing Legal Systems for an Algorithm Society
Barton, T. D. (2022) · Law for professionals · doi:10.1007/978-3-031-14240-6_5 · Cited by 2
Differential Replication in Machine LearningPreprint
Unceta, I., Nin, J., & Pujol, O. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.07981 · Cited by 2
AI and Corporate Responsibility
Martin, K. E., & Villegas-Galaviz, C. (2022) · Springer eBooks · doi:10.1007/978-3-319-23514-1_1297-1 · Cited by 2
Ethics and the explainable artificial intelligence(XAI) movementPreprint
Adamson, G. (2022) · doi:10.36227/techrxiv.20439192 · Cited by 2
Westover, J. (2026) · Future of Work The Journal of Labor Transformation Technology Integration and Human Adaptation · doi:10.70175/futureofworkjournal.2026.1.1.3
THE UNINTENDED CONSEQUENCES OF ALGORITHMIC TRANSPARENCY ON TRUST: A MULTI-DISCIPLINARY ANALYSIS
Čvokić, D. D. (2025) · Acta scientifica Balcanica. · doi:10.63356/asb.2025.002 · Cited by 1
A review of ethical AI frameworks in product development: taking stock and moving forward
Wecker, L., Nielsen, J. A., Nasrollahi, K., & Ploug, T. (2025) · AI and Ethics · doi:10.1007/s43681-025-00952-w · Cited by 2
Holst, J. (2020) · Advances in human and social aspects of technology book series · doi:10.4018/978-1-7998-4894-3.ch005 · Cited by 1
الذكاء الاصطناعي بين الاعتبار الأخلاقي والمقياس القانوني "أين نحن؟"
الخطيب, م. ع. (2023) · The Jordanian Journal of Law and Political Science · doi:10.35682/jjlps.v15i3.572 · Cited by 2
Ethical AI: Designing Responsible And Trustworthy Systems
Mana, S. C., Kalaiarasi, G., Selvi, D. M., Velvizhi, M. R., & Nair, M. M. C. (2023) · Jupiter Publications Consortium eBooks · doi:10.47715/jpc.978-93-91303-66-2 · Cited by 2
Normative Challenges of Risk Regulation of Artificial Intelligence and Automated Decision-MakingPreprint
Orwat, C., Bareis, J., Folberth, A., Jahnel, J., & Wadephul, C. (2022) · SocArXiv (OSF Preprints) · doi:10.31235/osf.io/qkx8b · Cited by 1
Smart cities: private means, public ends?
Voorwinden, A. (2023) · University of Groningen research database (University of Groningen / Centre for Information Technology) · doi:10.33612/diss.689858230 · Cited by 2
S. Scott Graham, The Doctor and the Algorithm. Promise, Peril, and the Future of Health AI
Lazcano, I. C. (2023) · Prometheus · doi:10.13169/prometheus.39.1.0051 · Cited by 1
Algorithmic Decoupling and the Adoption of `Privacy-Preserving' AnalyticsPreprint
Steed, R., & Acquisti, A. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4718865 · Cited by 3
Outlining Traceability: A Principle for Operationalizing Accountability in Computing Systems
Kroll, J. A. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445937 · Cited by 2
Self-citationOn Algorithmic Opacity and Moral CertaintyPreprint
Litvinski, O. (2020) · doi:10.31235/osf.io/rjsvk
AI and Corporate Responsibility
Martin, K. E., & Villegas-Galaviz, C. (2023) · Springer eBooks · doi:10.1007/978-3-030-22767-8_1297
A Transnational Perspective on Artificial Intelligence Cultural Challenges
Kromer, S. (2024) · IFIP advances in information and communication technology · doi:10.1007/978-3-031-67535-5_1
Plotting the Rule of Law Affordances
Bisoyi, A. (2025) · Law, governance and technology series · doi:10.1007/978-3-031-98712-0_10
Ethical AI Testing and Evaluation Against Governance and Frameworks
Ramachandran, M. (2025) · Springer eBooks · doi:10.1007/978-981-95-2909-4_11
Bridging the Gap: Translating AI Regulations into Socio-Technical Measures for Responsible AI
Ahmad, T. (2025) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/3715070.3747344
Gouvernance de l’IA en santé : médiations éthiques et modèles décentralisés
Nikabou, M. M. (2025) · Communication technologies et développement · doi:10.4000/15gpr
Rethinking technology adoption: responsible AI use as ethical practice in rural SMEs
Cenizo, C. (2026) · Small Enterprise Research · doi:10.1080/13215906.2026.2685206
TRACE: Runtime Governance, Containment, and Evidence for Autonomous AI Agent Operations
Calboreanu, E. (2026) · IEEE Access · doi:10.1109/access.2026.3733578
Artificial Empathy? AI’s Role in Amplifying (or Erasing) Queer Identities in Music Education
McBride, N. R. (2026) · Oxford University Press eBooks · doi:10.1093/9780197824818.003.0015
Ungureanu, V. (2026) · AI & Society · doi:10.1007/s00146-026-03352-8
A light framework for AI ethics
Kuo, H.-J. (2026) · AI and Ethics · doi:10.1007/s43681-026-01452-1
Versioned Governance for Frontier AI: A Software-Engineering-Inspired ApproachPreprint
Xue, Z. (2026) · Research Square · doi:10.21203/rs.3.rs-11082416/v1
Metadata
- Authors
- Kroll, J. A.
- Affiliations
- University of California, Berkeley (US)
- Corresponding author
- Kroll, Joshua A.
- ORCID
- Kroll, Joshua A.
- Venue
- Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences
- Publisher
- The Royal Society
- Published
- 2018-10-15
- Volume
- 376
- Issue
- 2133
- Pages
- 20180084
- ISSN
- 1364-503X (ISSN-L)
- 1471-2962
- PMC
- PMC6191668
- OpenAlex
- W2897439270
- Semantic Scholar
- 95362d732fa70608c2640336d17ac8ff0895e10a
- Cited by
- 132
- References
- 53
- Citations by year
- 2026: 9 · 2025: 13 · 2024: 15 · 2023: 22 · 2022: 27 · 2021: 19 · 2020: 17 · 2019: 8 · 2018: 2
- FWCI
- 22.85
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Philosophical Transactions of the Royal Society A Mathematical Physical and Engineering Sciences (publishedVersion)
- Europe PMC (PubMed Central) (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- eScholarship (California Digital Library) (submittedVersion)
- Funders
- Berkeley Center for Law and Technology
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Social Sciences › Law › Digitalization, Law, and Regulation
- Keywords
- algorithmic opacity, algorithmic governance, algorithmic accountability, computer ethics, privacy by design, algorithmic transparency, algorithmic decision-making
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-05, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-05 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
