Feature selection, perception learning, and a usability case study for text categorization

Abstract

In this paper, we describe an automated learning approach to text categorization based on perception learning and a new feature selection metric, called correlation coefficient.Our approach has been teated on the standard Reuters text categorization collection.Empirical results indicate that our approach outperforms the best published results on this % uters collection.In particular, our new feature selection method yields comiderable improvement.We also investigate the usability of our automated hxu-n-~approach by actually developing a system that categorizes texts into a tree of categories.We compare tbe accuracy of our learning approach to a rrddmsed, expert system ap preach that uses a text categorization shell built by Cams gie Group.Although our automated learning approach still gives a lower accuracy, by appropriately inmrporating a set of manually chosen worda to use as f~ures, the combined, semi-automated approach yields accuracy close to the * baaed approach.