Clustering aided support vector machines

Support Vector Machines (SVMs) have proven to be an effective approach to learning a classifier from complex datasets. However, highly nonhomogeneous data distributions can pose a challenge for SVMs when the underlying dataset comprises clusters of instances with varying mixtures of class labels. To address this challenge we propose a novel approach, called a cluster-supported Support Vector Machine, in which information derived from clustering can be incorporated directly into the SVM learning process. We provide a theoretical derivation to show that when the total empirical loss is expressed in terms of the combined quadratic empirical loss from each cluster, we can still find a formulation of the optimisation problem that is a convex quadratic programming problem. We discuss the scenarios where this type of model would be beneficial, and present empirical evidence that demonstrates the improved accuracy of our combined model.