The weighing of evidence and the determinants of confidence

Abstract

The pattern of overconfidence and underconfidence observed in studies of intuitive judgment is explained by the hypothesis that people focus on the strength or extremeness of the available evidence (e.g., the warmth of a letter or the size of an effect) with insufficient regard for its weight or credence (e.g., the credibility of the writer or the size of the sample). This mode of judgment yields overconfidence when strength is high and weight is low, and underconfidence when strength is low and weight is high. We first demonstrate this phenomenon in a chance setup where strength is defined by sample proportion and weight is defined by sample size, and then extend the analysis to more complex evidential problems, including general knowledge questions and predicting the behavior of self and of others. We propose that people's confidence is determined by the balance of arguments for and against the competing hypotheses, with insufficient regard for the weight of the evidence. We show that this account can explain the effect of item difficulty on overconfidence, and we relate the observed discrepancy between confidence judgments and frequency estimates to the illusion of validity. Finally, we contrast the present account with a frequentistic model of confidence proposed by Gigerenzer and his colleagues, and present data that refute their model.