Human intervention in automated decision-making: Toward the construction of contestable systems

Abstract

Concerns about "black box" machine learning algorithms have influenced why modern data protection laws and regulations on their establishment of a right to human intervention on decision-making supported by artificial intelligence. Such interventions provide data subjects with means to protect their rights, freedoms, and legitimate interests, either as a bare minimum requirement for data processing or as a central norm governing decision-aiding artificial intelligence. In this paper, I present contestability by design as an approach to two kinds of issues with current legal implementations of the right to human intervention. The first kind is the uncertainty about what kind of decision should be covered by this right: should intervention be restricted to those decisions with no human involvement, or should it be interpreted in a broader sense, encompassing all decisions that are effectively shaped by automated processing? The second class of issues ensues from practical limitations of this right to intervention: even within a clear conceptual framework, data subjects might still lack the information they need to the concrete exercise of their right, or the human intervention itself might introduce biases and limitations that result in undesirable outcomes. After discussing how those effects can be identified and measured, I then advance the thesis that proper protection of the rights of data subjects is feasible only if there are means for contesting decisions based solely on…