A tutorial on conformal prediction

“408–410 of his Statistical Learning Theory [27], support-vector machines grew out of the idea of separating two groups of examples with a hyperplane in a way that makes as few mistakes as possible—i.”

“1 Example: Classifying iris flowers In 1936 [11], R.”

“, zn−1) can be framed as an application of the widely accepted Neyman-Pearson theory for hypothesis testing and confidence intervals [17].”

“Readers familiar with Bayes nets [4] will recognize this diagram as an example; conditional on each variable, a joint probability distribution is given for its children (the variables to which arrows from it point), and given the variable, its descendants are independent of its ancestors.”

“This gives the following protocol: The Backward-Looking Betting Protocol Players: Joe, Bill KN := 1.”

“82; [21], pp.”

“Our concept of validity is consistent with a tradition that can be traced back to Jerzy Neyman’s introduction of confidence intervals for parameters in 1937 [19] and even to work by Laplace and others in the late 18th century.”

“On the other hand, as we have already said, it will make no difference if we replace the absolute difference d(z, z′) = |z − z′| with the squared difference d(z, z′) = (z − z′)2, thus squaring A.”

“In addition to calculating the average zn−1, calculate s2n−1 := 1 n − 2 n−1 ∑ i=1 (zi − zn−1) 2, which is sometimes called the sample variance.”

“But the efficiency of conformal prediction will depend on Q and the nonconformity measure.”

“This definition makes it easy to see that independent and identically distributed random variables are exchangeable.”

“The probability is valid if the offer does not put the person making it at a disadvantage, inasmuch as a long sequence of equally reasonable offers will not allow an opponent to multiply the capital he or she risks by a large factor (Shafer and Vovk, 2001).”

“But this turned out not to be the case (Robinson, 1975).”

“This allows us to express the empirical interpretation of given probabilities in terms of betting, using what we call Cournot’s principle: the odds determined by the probabilities will not allow a bettor to multiply the capital he or she risks by a large factor [23].”

“Exchangeability is closely related to the idea that examples are drawn independently from a probability distribution.”

“Conformal prediction (Vovk et al., 2003; Shafer and Vovk, 2008; Balasubramanian et al., 2014) is a framework for reliable prediction that is rooted in classical frequentist statistics, more specifically in hypothesis testing.”

“Training-based calibration methods comprise supervised techniques that modify the network’s weights and various types of self-verbalization, where the model qualifies and refines its outputs based on its own reasoning or feedback about uncertainty.”