Openly accessible

Approximation vector machines for large-scale online learning

Le, Trung, Nguyen, Tu Dinh, Nguyen, Vu and Phung, Dinh 2017, Approximation vector machines for large-scale online learning, Journal of machine learning research, vol. 18, pp. 1-55.

Attached Files
Name Description MIMEType Size Downloads
le-approximationvector-2017.pdf Published version application/pdf 2.15MB 1

Title Approximation vector machines for large-scale online learning
Author(s) Le, TrungORCID iD for Le, Trung orcid.org/0000-0002-7070-8093
Nguyen, Tu DinhORCID iD for Nguyen, Tu Dinh orcid.org/0000-0002-9977-8247
Nguyen, Vu
Phung, DinhORCID iD for Phung, Dinh orcid.org/0000-0002-9977-8247
Journal name Journal of machine learning research
Volume number 18
Start page 1
End page 55
Total pages 55
Publisher MIT Press
Place of publication Cambridge, Mass.
Publication date 2017-11-01
ISSN 1532-4435
1533-7928
Keyword(s) Science & Technology
Technology
Automation & Control Systems
Computer Science, Artificial Intelligence
Computer Science
kernel
online learning
large-scale machine learning
sparsity
big data
core set
stochastic gradient descent
convergence analysis
PERCEPTRON
SVM
cs.LG
stat.ML
Summary One of the most challenging problems in kernel online learning is to bound the model size and to promote model sparsity. Sparse models not only improve computation and memory usage, but also enhance the generalization capacity -- a principle that concurs with the law of parsimony. However, inappropriate sparsity modeling may also significantly degrade the performance. In this paper, we propose Approximation Vector Machine (AVM), a model that can simultaneously encourage sparsity and safeguard its risk in compromising the performance. In an online setting context, when an incoming instance arrives, we approximate this instance by one of its neighbors whose distance to it is less than a predefined threshold. Our key intuition is that since the newly seen instance is expressed by its nearby neighbor the optimal performance can be analytically formulated and maintained. We develop theoretical foundations to support this intuition and further establish an analysis for the common loss functions including Hinge, smooth Hinge, and Logistic (i.e., for the classification task) and ℓ1, ℓ2, and ε-insensitive (i.e., for the regression task) to characterize the gap between the approximation and optimal solutions. This gap crucially depends on two key factors including the frequency of approximation (i.e., how frequent the approximation operation takes place) and the predefined threshold. We conducted extensive experiments for classification and regression tasks in batch and online modes using several benchmark datasets. The quantitative results show that our proposed AVM obtained comparable predictive performances with current state-of-the-art methods while simultaneously achieving significant computational speed-up due to the ability of the proposed AVM in maintaining the model size.
Language eng
HERDC Research category C1 Refereed article in a scholarly journal
ERA Research output type C Journal article
Copyright notice ©2017, Trung Le, Tu Dinh Nguyen, Vu Nguyen, and Dinh Phung.
Free to Read? Yes
Use Rights Creative Commons Attribution licence
Persistent URL http://hdl.handle.net/10536/DRO/DU:30105909

Connect to link resolver
 
Unless expressly stated otherwise, the copyright for items in DRO is owned by the author, with all rights reserved.

Every reasonable effort has been made to ensure that permission has been obtained for items included in DRO. If you believe that your rights have been infringed by this repository, please contact drosupport@deakin.edu.au.

Versions
Version Filter Type
Citation counts: TR Web of Science Citation Count  Cited 0 times in TR Web of Science
Scopus Citation Count Cited 1 times in Scopus
Google Scholar Search Google Scholar
Access Statistics: 9 Abstract Views, 3 File Downloads  -  Detailed Statistics
Created: Tue, 29 May 2018, 10:23:44 EST

Every reasonable effort has been made to ensure that permission has been obtained for items included in DRO. If you believe that your rights have been infringed by this repository, please contact drosupport@deakin.edu.au.