The Fallacy of AI Functionality

In the paper

“Firstly, the sheer number of points of failure we were able to identify illustrates the scope of the problem.”

In the paper

“For example, audits of AI hiring tools focus primarily on ensuring an 80% selection rate for protected classes (the so-called 4/5ths rule) is satisfied, and rarely mention product validation processes, demonstrating an assumed validity of the prediction task [52,148,199].”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [52]
• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [148]
• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [199]

In the paper

“As an overlooked aspect of AI policy, functionality is often presented as a consideration secondary to other ethical challenges.In this paper, we argue that it is a primary concern that often precedes such problems.”

In the paper

“However, despite this reality, scholars, the press, and policymakers pay too little attention to functionality.”

In the paper

“We start by calling out what we perceive to be a functionality assumption, prevalent in much of the discourse on AI risks.”

In the paper

“It is unsurprising that promoters of AI do not tend to question its functionality.”

In the paper

“Similarly, to the extent regulators are looking into functionality, explanations aimed at regulators can assist oversight, but typically explanations are desired to check the basis for decisions, while assuming the systems work as intended.”

In the paper

“It is possible for designers to preempt misuse by implementing instructions, warnings, or error conditions, and failure to do so creates a system that does not function properly.”

In the paper

“The accuracy of ETS's method was disputed between experts sought by the National Union of Students and the Home Office; the resulting estimates of error rates ranged from 1% to 30%.”

In the paper

“Under the hood, however, Instead of software producing financial statements, dozens of accountants did most of it manually from ScaleFactor's Austin headquarters or from an outsourcing office in the Philippines, according to former employees.”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [100]

In the paper

“Due to upstream policy, data or societal choices, AI tasks can be practically impossible for one set of developers and not for another, or for different reasons in different contexts.”

In the paper

“We then discuss the existing accountability tools to address functionality issues, that are often overlooked in AI policy literature and in practice, due in large part to this assumption of functionality.”

In the paper

“While this in and of itself may not create unsafe products, expectations based on unreasonable claims can create unearned trust, and a potential overreliance that hurts parties who purchase the product.”

In the paper

“This research question presumes that an unconstrained solution without fairness restrictions is the optimal solution to the problem.”

In the paper

“Functionality can be difficult to define precisely.”

In the paper

“Similarly, Raji et al. [149] argue that AI makes claims to generality while modeling behaviour that is determined by highly constrained and context-specific data.”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [149]

In the paper

“Sometimes model failures are anticipated yet difficult to prevent; in this case, engineers can sometimes take steps to ensure these points of failure will not cause harm.”

In the paper

“Other papers discuss how the AI label lends itself to inflated claims of functionality that the systems cannot meet.”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [102]
• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [28]

In the paper

“Of the 81 non-randomized studies, they found the median number of clinical experts compared to the AI was 4, full access to datasets and code were unavailable in over 90% of studies, the overall risk of bias was high, and adherence to reporting standards were suboptimal, and therefore poorly substantiate their claims.”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [129]

In the paper

“Similarly, the Epic sepsis prediction model, a product actually implemented at hundreds of hospitals, was recently externally validated by Wong et al. [203], who found that the model had poor calibration to other hospital settings and discriminated against under-represented demographics.”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [203]

In the paper

“There is also some work demonstrating that AI products are challenging to engineer correctly in practice.”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [194]

In the paper

“Yet out of 12,500 people who appealed their immigration decisions, only 3,600 won their cases-and only a fraction of these were won through actually disproving the allegations of cheating.”

In the paper

“Communication failures often co-occur with other functional safety problems, and the lack of accountability for false claims -intentional or otherwise -makes these particularly pernicious and likely to occur as AI hype continues absent effective regulation.”

In the paper

“Similarly, as an early example of technical fairness solutions, Feldman et al. [60] created a method to remove disparate impact from a model while preserving rank, which only makes sense if the unconstrained system output is correct in the first place.”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [60]

In the paper

“Some customers say they received books filled with errors, and were forced to re-hire accountants, or clean up the mess themselves.[100]”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [100]

In the paper

“As an example, he points to disinformation researchers, who embrace corporate talking points of a recommendation model that can meaningfully influence consumer behavior to the point of controlling their purchases or voting activity-when in actuality, these algorithms have little ability to do either [22,75,88,95,162].”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [22]
• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [75]
• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [88]
• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [95]
• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [162]

In the paper

“These proposals thus often fall short of acknowledging the functionality issues in AI deployments and the role of the lack of functional safety in contributing to the harm perpetuated by these systems.”

In the paper

“From the engineering side, interpretability is usually desired for debugging purposes [23], so it is focused on functionality.”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [23]

In the paper

“Recent work details that the AI research field suffers from scientific validity and evaluation problems [48,79].”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [48]
• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [79]

In the paper

“As with other areas of software development, roles in AI development and deployment are becoming more specialized.”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [44]

In the paper

“These mentions suggest that functionality is not always ignored outright.Sometimes, it is considered in policy, but in many cases, that consideration lacks the emphasis of the other concerns presented.”

In the paper

“Many of these cases are direct refutations of the specific instances of the functionality assumptions in Section 3.”

In the paper

“Functionality could also be seen as a statement that a product lives up to the vendor's performance claims, but this, too, is incomplete; specifications chosen by the vendor could be insufficient to solve the problem at hand.”

In the paper

“For instance, in a classification model, choices such as which input and target variables to use, whether to prioritize accepting true positives or rejecting false negatives, and how to process the training data all factor into determining model outcomes.”

In the paper

“Communication failures often co-occur with other functional safety problems, and the lack of accountability for false claims -intentional or otherwise -makes these particularly pernicious and likely to occur as AI hype continues absent effective regulation.”

In the paper

“While these are credible threats, concerns around large language models tend to dismiss the practical limitations of what these models can achieve [18], neglecting to address more mundane hazards tied to the premature deployment of a system that does not work [55,189].”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [18]
• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [55]
• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [189]

In the paper

“Data validation is a critical step in the construction of a ML system, and the team that built the benefit system chose to use a highly problematic dataset to train their model.”

In the paper

“While it is not controversial to claim that such concerns are far from being realized [13,42,146], this fear of misspecified objectives, runaway feedback loops, and AI alignment presumes the existence of an industry that can get AI systems to execute on any clearly declared objectives, and that the main challenge is to choose and design an appropriate goal.”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [13]
• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [42]
• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [146]

In the paper

“However, the lack of a failsafe to make sure the carbon monoxide detector could not be turned off accidentally made the product dangerous.”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [200]

In the paper

“We argue that functionality is a meaningful AI policy challenge, operating as a necessary first step towards protecting affected communities from algorithmic harm.”

In the paper

“For example, when large language models are used to process the queries of AI-powered web search [131], the models' fragility to misspellings [125,147], or trivial changes to format [19] and context [18] can lead to unexpected results.”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [131]
• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [125]
• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [147]
• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [19]

In the paper

“the data they used to come up with their formula for setting people's assistance limits was corrupt.”

• Also stated in: § Model Implementation Failures. · plain

  “they had to throw out two-thirds of the records they had before they came up with the formula because of data entry errors and data that didn't make sense.”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [177]

In the paper

“Some roles focus on managing the data that feeds into models, others specialize in modeling, and others optimally engineer models for speed and scale [44].”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [44]

In the paper

“The draft AI Act in the EU includes requirements for pre-marketing controls to establish products' safety and performance, as well as quality management for high risk systems [190].”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [190]

In the paper

“Krafft et al. [110] found that policymakers often define AI with respect to how human-like a system is, and concluded that this could lead to deprioritizing issues more grounded in reality.”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [110]

In the paper

“US EHR data is often erroneous, miscoded, fragmented, and incomplete [91,92], creating a mismatch between available data and intended use [74].”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [91]
• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [92]
• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [74]

In the paper

“For example, expressed concerns around large language models centers on hyped narratives of the models' ability to generate hyper-realistic online content, which could theoretically be used by malicious actors to facilitate harmful misinformation campaigns [176,195].”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [176]
• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [195]

In the paper

“The hype surrounding IBM's Watson in healthcare represents another example where a product that may have been fully capable of performing specific helpful tasks was sold as a panacea to health care's ills.”

In the paper

“Sales of goods typically come with a set of implied warranties.”

In the paper

“Thankfully, as AI operates as a general purpose technology prevalent in many of these industries, there already exists a plethora of governance infrastructure to address this issue in related fields of application.”

In the paper

“In addition to legal levers, there are many organizational interventions that can be deployed to address the range of functionality issues discussed.”

In the paper

“In general, if a person is injured by a defective product, they can sue the producer or seller in products liability.”

In the paper

“There are several other organizational factors that can determine and assess the functional safety of a system.”

In the paper

“Existing functional safety standards from the automobile (ISO 26262), aerospace (US RTCA DO-178C), defense (MIL-STD-882E) and electronics (IEEE IEC 61508 / IEC 61511) industries, amongst others, can provide a template on how to approach this challenge within the AI industry.”

In the paper

“The AI field seems to now be facing this same challenge.”

In the paper

“Products liability could especially easily apply to engineering failures, where the error was foreseeable and an alternative, working version of the product should have been built.”

In the paper

“Fraud has many specific meanings in law, but invariably it involves a knowing or intentional misrepresentation that the victim relied on in good faith.”

In the paper

“Defunct AI will breach both these warranties.”

In the paper

“CEN (European Committee for Standardisation) and CENELEC (European Committee for Electrotechnical Standardisation), two of three European Standardisation Organisations (ESOs) were heavily involved in the creation of the EU's draft AI Act [190].”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [190]

In the paper

“We cannot take for granted that AI products work.”

In the paper

“The AI field could certainly adopt similar techniques for increasing the scrutiny of their systems, especially given the nascent state of reflection and standardization common in ML evaluation processes [118].”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [118]

In the paper

“Similarly, Eubanks [56] demonstrates how a $1.4 billion contract with a coalition of hightech companies led an Indiana governor to stubbornly continue a welfare automation algorithm that resulted in a 54% increase in the denials of welfare applications.”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [56]

In the paper

“Choi analogizes this issue to car crashes: car crashes are inevitable, but courts developed the idea of crashworthiness to ask about the car's contribution to the total harm, even if the initial injury was attributable to a product defect [36].”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [36]

In the paper

“It would stand to reason that a functionality failure in an AI system could be deemed a product defect.”

In the paper

“In extreme cases, the sale of defunct AI may constitute fraud.”

In the paper

“Thus, where dysfunctional AI is being sold and its failures causes substantial harm to consumers, the FTC could step in.”

In the paper

“The National Highway Safety Administration offers similar oversight for automobiles specifically.”

In the paper

“If a system is not functional, in the sense we describe, a court should have no problem finding that it is unreasonably defective.”

In the paper

“The fact that faulty AI products are on the market today makes this problem particularly urgent.”

In the paper

“In addition to various federal agencies, all states have consumer protection statutes that bar deceptive practices and many bar unfair practices as well, like the FTC Act [33].”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [33]

In the paper

“Another area of law implicated by product failure is warranty law, which protects the purchasers of defunct AI and certain third parties who stand to benefit from the sale.”

In the paper

“State attorneys general often take active roles as enforcers of those laws [38].”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [38]

In the paper

“This follows a pattern seen with machine learning products in general, where the advertised performance on a simple and static data benchmark, is much lower than the performance on the often more complex and diverse data encountered in practice.”

In the paper

“This would likely not be a problem for software that resides within a physical system, but for non-embodied AI, it might pose a hurdle.”

In the paper

“Similarly, cultural expectations for safety and engineering responsibility impact the quality of the output from the product development process -setting these expectations internally and fostering a healthy safety culture can increase cooperation on other industrywide and organizational measures [163].”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [163]

In the paper

“Injuries not related to the breach of contract are meant to be worked out in tort law, as described above.”

In the paper

“This is particularly a problem in the context of algorithms developed by public agencies -where the AI systems can be adopted as symbols for progress, or smokescreens for undesirable policy outcomes, and thus liable to inflated narratives of performance.”

In the paper

“The Commission does not have general rulemaking authority, so most of its actions to date have taken the form of public education and enforcement.”

In the paper

“Taking the form of anything from documentation exercises to challenge datasets as benchmarks, these processes raised the bar for deployment criteria and matured the product development pipeline in the process [152].”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [152]

In the paper

“Building on previous work [31], the ACLU ran Rekognition with a database of mugshots against members of U.S. Congress using the default setting and found 28 members falsely matched within the database, with people of color shown as a disproportionate share of these errors [175].”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [31]
• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [175]

In the paper

“A trickier intervention is the avenue of product certification and standards development for AI products.”

In the paper

“As discussed above, a product could be placed on the market claiming the ability to do something it cannot achieve in theory or in practice, or it can fail to be robust to unanticipated but foreseeable uses by consumers.”

In the paper

“The Commission does, however, have authority to make rules regarding unfair or deceptive practices under the Magnuson-Moss Warranty Act.”

In the paper

“In many cases, it required the active advocacy of concerned citizens to lead to the policy interventions that would effectively change the tide of these industries.”

In the paper

“False advertising laws are related and also common.”

In the paper

“Poorly vetted products permeate our lives, and while many readily accept the potential for harms as a tradeoff, the claims of the products' benefits go unchallenged.”

In the paper

“Although functionality issues are not completely ignored in AI policy, the lack of awareness of the range in which these issues arise leads to the problems being inadequately emphasized and poorly addressed by the full scope of accountability tools available.”

In the paper

“Another avenue for legal accountability may come from the tort of products liability, though there are some potential hurdles.”

In the paper

“Though it has created no new rules since 1980, in July 2021, the FTC voted to change internal agency policies to make it easier to do so [41].”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [41]

In the paper

“Fraud is difficult to prove, and our examples do not clearly indicate fraud, but it is theoretically possible if someone is selling snake oil.”

In the paper

“As a result, assessment outcomes are mainly for internal use only, often just to set internal quality assurance standards for deployment and prompt further engineering reflection during the evaluation process.”

In the paper

“Due process requires decisions to not be arbitrary, and AI that doesn't work loses its claim to making decisions on a sound basis [37].”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [37]

In the paper

“Fraud can lead to criminal liability as well.”

In the paper

“At Memorial-Sloan Kettering, after $62 million dollars spent and may years of effort, MD Anderson famously cancelled IBM Watson contracts with no results to show for it [89].”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [89]

In the paper

“However, the study used to make this claim (still prominently displayed in press release materials) was quickly debunked by a series of outside researchers who found that at the document-level, when provided with context from nearby sentences, and/or compared to human experts, the machine translation model did not indeed achieve equal accuracy to human translators [114,185].”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [186]
• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [114]
• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [185]

In the paper

“Also, as functionality is a safety risk aligned with profit-oriented goals, many model logging and evaluation operations tools are available for organizations to leverage in the internal inspection of their systems -including tools for more continuous monitoring of deployed systems [154,169].”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [154]
• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [169]

In the paper

“In tort law, a claim of fraud can lead to compensation necessary to rectify any harms that come from the fraud, as well as punitive damages in egregious cases.”

In the paper

“The implied warranty of fitness for particular purpose applies when a seller knows that the buyer has a specific purpose in mind and the buyer is relying on the seller's skill or judgment about the good's fitness, stating that the good is fit for that purpose [188].”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [188]

In the paper

“These efforts resemble the Partnership on AI's attempt to establish norms on model documentation processes [153].”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [153]

In the paper

“Collective industry-wide decision-making on critical issues can raise the bar for the entire industry and raise awareness within the industry of the importance of handling functionality challenges.”

In the paper

“Based on the FTC's approach to data security, in which the Commission has sued companies for failing to adequately secure consumer data in their possession against unknown thirdparty attackers [121], even post-deployment failures-if foreseeable and harmful-can be included among unfair practices, though they partially attributable to external actors.”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [121]

In the paper

“The plaintiff need not have purchased or used the product; it is enough that they were injured by it, and the product has a defect that rendered it unsafe.”

In the paper

“Due to clear conflicts of interest, the self-regulatory approaches described are far from adequate oversight for these challenges, and the presence of regulation does a lot to incentivise organizations to take these actions in the first place.”

In the paper

“Similarly, the transportation industry has a rich history of thorough accident reports and investigations, through organizations such as the National Transportation and Safety Board (NTSB), who have already been responsible for assessing the damage from the few known cases of self-driving car crashes from Uber and Tesla [81].”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [81]

In the paper

“Deceptive claims can be expressed explicitly-for example, representation in the sales materials that is inaccurate-or implied, such as an aspect of the design that suggests a functionality the product lacks [82,93].”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [82]
• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [93]

In the paper

“However, they do provide an immediate path forward in addressing these issues.”

In the paper

“In broader terms, the concept of functional safety in engineering design literature [163,174] well encapsulates the concerns articulated in this paper-namely that a system can be deployed without working very well, and that such performance issues can cause harm worth preventing.”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [163]
• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [174]

In the paper

“For software of nontrivial complexity, it is provably impossible to guarantee bug-free code.”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [36]

In the paper

“Writing about software and tort law, Choi [36] identifies the complexity of software as a more fundamental type of hurdle.”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [36]

In the paper

“One particular state law worth noting is California's Unfair Competition Law, which allows individuals to sue for injunctive relief to halt conduct that violates other laws, even if individuals could not otherwise sue under that law [2].”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [2]

In the paper

“The Consumer Finance Protection Bureau can regulate harms from products dealing with loans, banking, or other consumer finance issues [4].”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [4]

In the paper

“Formally, the FTC does not have the power to issue fines under its Section 5 authority, but the Commission frequently enters into long-term consent decrees with companies that it sues, permitting continuing jurisdiction, monitoring, and fines for future violations [3,40].”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [3]
• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [40]

In the paper

“On the U.S. front, industry groups IEEE and ISO regularly shape conversations, with IEEE going so far as to attempt the development of a certification program [72,84].”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [72]
• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [84]

In the paper

“The Federal Trade Commission is the federal consumer protection agency within the United States with the broadest subject matter jurisdiction.”

In the paper

“The challenge with such audits, however, is that the results are rarely communicated externally and disclosure is not mandatory, nor is it incentivized.”

In the paper

“A lack of construct validity would easily subject a model's user to liability.”

In the paper

“But as no amount of testing can guarantee bug-free software, it will difficult to determine how much testing is enough to be considered reasonable or non-negligent [36,94].”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [36]
• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [94]