Min-Uncertainty & Max-Certainty Criteria of Neighborhood Rough-Mutual Feature Selection
Keywords:Feature selection, mutual information, neighborhood rough sets, classification, boundary region
Feature Selection (FS) is viewed as an important preprocessing step for pattern recognition, machine learning, and data mining. Most existing FS methods based on rough set theory use the dependency function for evaluating the goodness of a feature subset. However, these FS methods may unsuccessfully be applied on dataset with noise, which determine only information from a positive region but neglect a boundary region. This paper proposes a criterion of the maximal lower approximation information (Max-Certainty) and minimal boundary region information (Min-Uncertainty), based on neighborhood rough set and mutual information for evaluating the goodness of a feature subset. We combine this proposed criterion with neighborhood rough set, which is directly applicable to numerical and heterogeneous features, without involving a discretization of numerical features. Comparing it with the rough set based approaches, our proposed method improves accuracy over various experimental data sets. Experimental results illustrate that much valuable information can be extracted by using this idea. This proposed technique is demonstrated on discrete, continuous, and heterogeneous data, and is compared with other FS methods in terms of subset size and classification accuracy.
A Jain and D Zongker. Feature Selection: Evaluation, application, and small sample performance. IEEE Trans. Pattern Anal. Mach. Intell. 1997; 19, 153-8.
S Mitra. An evolutionary rough partitive clustering. Pattern Recogn. Lett. 2004; 25, 1439-49.
AL Blum and P Langley. Selection of relevant features and examples in machine learning. Artif. Intell. 1997; 97, 245-71.
R Kohavi and GH John. Wrappers for feature subset selection. Artif. Intell. 1997; 97, 273-324.
R Ruiz, JC Riquelme and JS Aguilar-Ruiz. Incremental wrapperbased gene selection from microarray data for cancer classification. Pattern Recogn. 2006; 39, 2383-92.
M Dash and H Liu. Feature selection for classification. Intell. Data Anal. 1997; 1, 131-56.
Y Kim, W Street and F Menczer. Feature selection for unsupervised learning via evolutionary search. In: Proceeding of the 6th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. Boston, Massachusetts, USA, 2000, p. 365-9.
MH Aghdam, N Ghasem-Aghaee and ME Basiri. Text feature selection using ant colony optimization. Expert Syst. Appl. 2009; 36, 6843-53.
W Shang, H Huang, H Zhu, Y Lin, Y Qu and Z Wang. A novel feature selection algorithm for text categorization. Expert Syst. Appl. 2007; 33, 1-5.
W Lee, SJ Stolfo and KW Mok. Adaptive intrusion detection: A data mining approach. Artif. Intell. Rev. 2000; 14, 533-67.
GJ Mun, BN Noh and YM Kim. Enhanced stochastic learning for feature selection in intrusion classification. Int. J. Innov. Comput. Inform. Contr. 2009; 5, 3625-35.
JC Patra, GP Lim, PK Meher and EL Ang. DNA microarray data analysis: Effective feature selection for accurate cancer classification. In: Proceedings of the International Joint Conference on Neural Networks, Orlando, Florida, USA, 2007, p. 260-5.
E Xing, M Jordan and R Karp. Feature selection for high-dimensional genomic microarray data. In: Proceedings of the 18th International Conference on Machine Learning. San Francisco, USA, 2001, p. 601-8.
Z Pawlak. Rough sets. Int. J. Inform. Comput. Sci. 1982; 11, 314-56.
Z Pawlak. Rough Sets: Theoretical Aspects of Reasoning about Data. Kluwer Academic Publishing, Dordrecht, 1991.
TM Cover and JA Thomas. Elements of Information Theory. Wiley, New York, 1991.
Y Chen, D Miao and R Wang. A rough set approach to feature selection based on ant colony optimization. Pattern Recogn. Lett. 2010; 31, 226-33.
JH Chiang and SH Ho. A combination of rough-based feature selection and RBF neural network for classification using gene expression data. IEEE Trans. Nanobiosci. 2008; 7, 91-9.
A Hassanien. Rough set approach for attribute reduction and rule generation: A case of patients with suspected breast cancer. J. Am. Soc. Inform. Sci. Tech. 2004; 55, 954-62.
AR Hedar, J Wang and M Fukushima. Tabu search for attribute reduction in rough set theory. Soft Comput. 2008; 12, 909-18.
YH Hung. A neural network classifier with rough set-based feature selection to classify multiclass IC package products. Adv. Eng. Inform. 2009; 23, 348-57.
R Jensen and Q Shen. Semantics-preserving dimensionality reduction: Rough and fuzzy-rough-based approaches. IEEE Trans. Knowl. Data Eng. 2004; 16, 1457-71.
Y Li, SCK Shiu, SK Pal and JNK Liu. A rough setbased case-based reasoner for text categorization. Int. J. Approx. Reason. 2006; 41, 229-55.
P Maji and S Paul. Rough sets for selection of molecular descriptors to predict biological activity of molecules. IEEE Trans. Syst. Man Cybern. Syst. C: Appl. Rev. 2010; 40, 639-48.
RW Swiniarski and A Skowron. Rough set methods in feature selection and recognition. Pattern Recogn. Lett. 2003; 24, 833-49.
R Battiti. Using mutual information for selecting features in supervised neural net learning. IEEE Trans. Neural Netw. 1994; 5, 537-50.
PA Estevez, M Tesmer, CA Perez and JM Zurada. Normalized mutual information feature selection. IEEE Trans. Neural Netw. 2009; 20, 189-201.
N Kwak and CH Choi. Input feature selection for classification problems. IEEE Trans. Neural Netw. 2002; 13, 143-59.
H Peng, F Long and C Ding. Feature selection based on mutual information: Criteria of max-dependency, max-relevance and min-redundancy. IEEE Trans. Pattern Anal. Mach. Intell. 2005; 27, 1226-38.
Q Hu, D Yu, J Liu and C Wu. Neighborhood rough set based heterogeneous feature subset selection. Inform. Sci. 2008; 178, 3577-94.
JS Deogun, VV Raghavan and H Sever. Exploiting upper approximation in the rough set methodology. In: Proceedings of the 1st International Conference Knowledge Discovery and Data Mining. Montréal, Québec, Canada, 1995, p. 69-74.
M Inuiguchi and M Tsurumi. Measures based on upper approximations of rough sets for analysis of attribute importance and interaction. Int. J. Innov. Comput. Inform. Contr 2006; 2, 1-12.
D Miao, Q Duan, H Zhang and N Jiao. Rough set based hybrid algorithm for text classification. Expert Syst. Appl. 2009; 36, 9168-74.
S Foitong, B Attachoo and O Pinngern. Feature subset selection wrapper based on mutual information and rough sets. Expert Syst. Appl. 2012; 39, 574-84.
H Liu and L Yu. Toward integrating feature selection algorithms for classification and clustering. IEEE Trans. Knowl. Data Eng. 2005; 17, 1-12.
MA Hall. Correlation-based feature selection for discrete and numeric class machine learning. In: Proceedings of the 17th International Conference on Machine Learning. San Francisco, CA, USA, 2000, p. 359-66.
H Liu and R Setiono. A probabilistic approach to feature selection: A filter solution. In: Proceedings of the 13th International Conference on Machine Learning. San Francisco, CA, USA, 1996, p. 319-27.
N Parthalain, Q Shen and R Jensen. A distance measure approach to exploring the rough set boundary region for attribute reduction. IEEE Trans. Knowl. Data Eng. 2010; 22, 305-17.
L Yu and H Liu. Efficient feature selection via analysis of relevance and redundancy. J. Mach. Learn. Res. 2004; 5, 1205-24.
MR Sikonja and I Kononenko. Theoretical and empirical analysis of ReliefF and RReliefF. Mach. Learn. 2003; 53, 23-69.
CE Shannon and W Weaver. The Mathematical Theory of Communication. University of Illinois Press, Urbana, Israel, 1994.
Q Hu, W Pedrycz, D Yu and J Lang. Selecting discrete and continuous features based on neighborhood decision error minimization. IEEE Trans. Syst. Man Cybern. Syst. B 2010; 40, 137-50.
CJ Merz and PM Murphy. UCI Repository of Machine Learning Databases, Irvine. Department of Information and Computer Science, University of California, 1996.
How to Cite
Copyright (c) 2016 Walailak Journal of Science and Technology (WJST)
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.