Ordinal Regression by Extended Binary Classification

Abstract

We present a reduction framework from ordinal regression to binary classification based on extended examples. The framework consists of three steps: extracting \nextended examples from the original examples, learning a binary classifier on the extended examples with any binary classification algorithm, and constructing a \nranking rule from the binary classifier. A weighted 0/1 loss of the binary classifier would then bound the mislabeling cost of the ranking rule. Our framework \nallows not only to design good ordinal regression algorithms based on well-tuned binary classification approaches, but also to derive new generalization bounds for \nordinal regression from known bounds for binary classification. In addition, our framework unifies many existing ordinal regression algorithms, such as perceptron \nranking and support vector ordinal regression. When compared empirically on benchmark data sets, some of our newly designed algorithms enjoy advantages \nin terms of both training speed and generalization performance over existing algorithms, which demonstrates the usefulness of our framework.