Improving Advisor Systems: Algorithms that Explain Themselves

Abstract

Human advisors typically explain their reasoning, which is absent when advice is given by an algorithm in the form of a mere number. We hypothesized that decision maker perceptions (e.g., trust), use of algorithmic advice, and hence judgment consistency and accuracy would improve if an algorithm ‘explains itself’. We recruited 1,202 English-speaking adults via Prolific who predicted the performance of a draw of 40 job candidates based on their assessment information and algorithmic advice. We used a 2 (narrative advice: yes/no) × 2 (narrative algorithm information: yes/no) × 2 (algorithmic advice as default: yes/no) between-subjects design. The first factor varied whether participants received mere numeric algorithmic advice or numeric advice plus a short case-by-case narrative explanation based on the specific candidate information. The second factor varied whether, before the task, the algorithm’s design and predictor weight choice were introduced in a narrative manner by a human character, using first-person language or in a descriptive manner. The third factor varied whether participants’ predictions defaulted to the algorithmic advice or an irrelevant value. Most effects were detectable but small in magnitude. The results showed that participants used narrative advice somewhat more than mere numeric advice, but only when their prediction did not default to the advice. Furthermore, participants had more trust, stronger feelings of human interaction, higher judgment consistency, and higher intentions to use the algorithm for future decisions when they received case-by-case narrative advice. People seem to feel more comfortable with algorithmic advice when receiving an explanation for each decision.