Min-Uncertainty & Max-Certainty Criteria of Neighborhood Rough-Mutual Feature Selection

Abstract

Feature Selection (FS) is viewed as an important preprocessing step for pattern recognition, machine learning, and data mining. Most existing FS methods based on rough set theory use the dependency function for evaluating the goodness of a feature subset. However, these FS methods may unsuccessfully be applied on dataset with noise, which determine only information from a positive region but neglect a boundary region. This paper proposes a criterion of the maximal lower approximation information (Max-Certainty) and minimal boundary region information (Min-Uncertainty), based on neighborhood rough set and mutual information for evaluating the goodness of a feature subset. We combine this proposed criterion with neighborhood rough set, which is directly applicable to numerical and heterogeneous features, without involving a discretization of numerical features. Comparing it with the rough set based approaches, our proposed method improves accuracy over various experimental data sets. Experimental results illustrate that much valuable information can be extracted by using this idea. This proposed technique is demonstrated on discrete, continuous, and heterogeneous data, and is compared with other FS methods in terms of subset size and classification accuracy.