Hybrid wrapper-filter approaches for input feature selection using maximum relevance-minimum redundancy and artificial neural network input gain measurement approximation (ANNIGMA)

Feature selection processes improve the accuracy, computational efficiency and scalability of classification process in data mining applications. This paper proposes two filter and wrapper hybrid approaches for feature selection techniques by combining the filter's feature ranking score in the wrapper stage. The first approach hybridizes a Mutual Information (MI) based Maximum Relevance (MR) filter ranking heuristic with an Artificial Neural Network (ANN) based wrapper approach where Artificial Neural Network Input Gain Measurement Approximation (ANNIGMA) has been combined with MR (MR-ANNIGMA) to guide the search process in the wrapper. The second hybrid combines an improved version of MI based (Maximum Relevance and Minimum Redundancy; MaxRel-MinRed) filter ranking heuristic with the wrapper heuristic ANNIGMA (MaxRel-MinRed-ANNIGMA). The novelty of our approach is that we integrate the capability of wrapper approach to find better feature subset by combining filter's ranking score with the wrapper-heuristic's score that take advantages of both filter and wrapper heuristics. The performances of the hybrid approaches have been verified using synthetic, bench mark data sets and real life data set and compared to both independent filter and wrapper based approaches. Experimental results show that hybrid approaches (MRANNIGMA and MaxRel-MinRed-ANNIGMA) achieve more compact feature sets and higher accuracies than filter and wrapper approaches alone.