SVMs and data dependent distance metric

Support Vector Machine (SVM) is an efficient classification tool. Based on the principle of structured risk minimization, SVM is designed to generalize well. But it has been shown that SVM is not immune to the curse of dimensionality. Also SVM performance is not only critical to the choice of kernel but also to the kernel parameters which are generally tuned through computationally expensive cross-validation procedures. Typical kernels do not have any information about the subspace to ignore irrelevant features or making relevant features explicit. Recently, a lot of progress has been made for learning a data dependent distance metric for improving the efficiency of κ-Nearest Neighbor (KNN) classifier. Metric learning approaches have not been investigated in the context of SVM. In this paper, we study the impact of learning a data dependent distance metric on classification performance of an SVM classifier. Our novel approach in this paper is a formulation relying on a simple Mean Square Error (MSE) gradient based metric learning method to tune kernel's parameters. Experiments are conducted on major UCIML, faces and digit databases. We have found that tuning kernel parameters through a metric learning approach can improve the classification performance of an SVM classifier. © 2010 IEEE.