Shaping Our Tools: Contestability as a Means to Promote Responsible Algorithmic Decision Making in the Professions

Abstract

Algorithmic systems, particularly those based on machine learning, are increasingly being used to help us reason and make decisions. Effective systems that also align with societal values require not only designs that foster in-the-moment human engagement with such systems but also governance models that support ongoing critical engagement with system processes and outputs. Using the case of expert decision-support systems, we introduce the concept of contestability. We argue that contestability has distinct advantages over transparency and explainability, two policy objectives often offered as antidotes to the challenges posed by black-box algorithmic systems. We then discuss contestable design and governance principles as applied to clinical decision support (CDS) systems used by health-care professionals to aid medical decisions. We explain current governance frameworks around the use of these systems — particularly laws and professional standards — and point out their limitations. We argue that approaches focused on contestability better promote professionals’ continued, active engagement with algorithmic systems than current frameworks.