A new transfer learning framework with application to model-agnostic multi-task learning

Gupta, Sunil, Rana, Santu, Saha, Budhaditya, Phung, Dinh and Venkatesh, Svetha 2016, A new transfer learning framework with application to model-agnostic multi-task learning, Knowledge and information systems, vol. 49, no. 3, pp. 933-973, doi: 10.1007/s10115-016-0926-z.

Attached Files
Name Description MIMEType Size Downloads

Title A new transfer learning framework with application to model-agnostic multi-task learning
Author(s) Gupta, SunilORCID iD for Gupta, Sunil orcid.org/0000-0002-3308-1930
Rana, SantuORCID iD for Rana, Santu orcid.org/0000-0003-2247-850X
Saha, BudhadityaORCID iD for Saha, Budhaditya orcid.org/0000-0001-8011-6801
Phung, DinhORCID iD for Phung, Dinh orcid.org/0000-0002-9977-8247
Venkatesh, SvethaORCID iD for Venkatesh, Svetha orcid.org/0000-0001-8675-6631
Journal name Knowledge and information systems
Volume number 49
Issue number 3
Start page 933
End page 973
Total pages 41
Publisher Springer
Place of publication London, England
Publication date 2016-12
ISSN 0219-1377
Keyword(s) multi-task learning
model-agnostic framework
meta algorithm
Summary Learning from small number of examples is a challenging problem in machine learning. An effective way to improve the performance is through exploiting knowledge from other related tasks. Multi-task learning (MTL) is one such useful paradigm that aims to improve the performance through jointly modeling multiple related tasks. Although there exist numerous classification or regression models in machine learning literature, most of the MTL models are built around ridge or logistic regression. There exist some limited works, which propose multi-task extension of techniques such as support vector machine, Gaussian processes. However, all these MTL models are tied to specific classification or regression algorithms and there is no single MTL algorithm that can be used at a meta level for any given learning algorithm. Addressing this problem, we propose a generic, model-agnostic joint modeling framework that can take any classification or regression algorithm of a practitioner’s choice (standard or custom-built) and build its MTL variant. The key observation that drives our framework is that due to small number of examples, the estimates of task parameters are usually poor, and we show that this leads to an under-estimation of task relatedness between any two tasks with high probability. We derive an algorithm that brings the tasks closer to their true relatedness by improving the estimates of task parameters. This is achieved by appropriate sharing of data across tasks. We provide the detail theoretical underpinning of the algorithm. Through our experiments with both synthetic and real datasets, we demonstrate that the multi-task variants of several classifiers/regressors (logistic regression, support vector machine, K-nearest neighbor, Random Forest, ridge regression, support vector regression) convincingly outperform their single-task counterparts. We also show that the proposed model performs comparable or better than many state-of-the-art MTL and transfer learning baselines.
Language eng
DOI 10.1007/s10115-016-0926-z
Field of Research 080109 Pattern Recognition and Data Mining
Socio Economic Objective 970108 Expanding Knowledge in the Information and Computing Sciences
HERDC Research category C1 Refereed article in a scholarly journal
ERA Research output type C Journal article
Copyright notice ©2016, Springer-Verlag
Persistent URL http://hdl.handle.net/10536/DRO/DU:30082819

Connect to link resolver
Unless expressly stated otherwise, the copyright for items in DRO is owned by the author, with all rights reserved.

Version Filter Type
Citation counts: TR Web of Science Citation Count  Cited 3 times in TR Web of Science
Scopus Citation Count Cited 3 times in Scopus
Google Scholar Search Google Scholar
Access Statistics: 598 Abstract Views, 6 File Downloads  -  Detailed Statistics
Created: Wed, 13 Apr 2016, 16:07:04 EST

Every reasonable effort has been made to ensure that permission has been obtained for items included in DRO. If you believe that your rights have been infringed by this repository, please contact drosupport@deakin.edu.au.