File(s) under permanent embargo
Learning parts-based representations with nonnegative restricted boltzmann machine
conference contribution
posted on 2013-11-15, 00:00 authored by Tu Dinh Nguyen, Truyen TranTruyen Tran, Quoc-Dinh Phung, Svetha VenkateshSvetha VenkateshThe success of any machine learning system depends critically on effective representations of data. In many cases, especially those in vision, it is desirable that a representation scheme uncovers the parts-based, additive nature of the data. Of current representation learning schemes, restricted Boltzmann machines (RBMs) have proved to be highly effective in unsupervised settings. However, when it comes to parts-based discovery, RBMs do not usually produce satisfactory results. We enhance such capacity of RBMs by introducing nonnegativity into the model weights, resulting in a variant called nonnegative restricted Boltzmann machine (NRBM). The NRBM produces not only controllable decomposition of data into interpretable parts but also offers a way to estimate the intrinsic nonlinear
dimensionality of data. We demonstrate the capacity of our model on well-known datasets of handwritten digits, faces and documents. The decomposition quality on images is comparable with or better than what produced by the nonnegative matrix factorisation (NMF), and the thematic features uncovered from text are qualitatively interpretable in a similar manner to that of the latent Dirichlet allocation (LDA). However, the learnt features, when used for classification, are more discriminative than those discovered by both NMF and LDA and comparable with those by RBM.
dimensionality of data. We demonstrate the capacity of our model on well-known datasets of handwritten digits, faces and documents. The decomposition quality on images is comparable with or better than what produced by the nonnegative matrix factorisation (NMF), and the thematic features uncovered from text are qualitatively interpretable in a similar manner to that of the latent Dirichlet allocation (LDA). However, the learnt features, when used for classification, are more discriminative than those discovered by both NMF and LDA and comparable with those by RBM.
History
Event
Asian Conference on Machine Learning (5th : 2013 : Canberra, Australia)Volume
29Pagination
133 - 148Publisher
JMLR Workshop and Conference ProceedingsLocation
Canberra, AustraliaPlace of publication
[The Conference]Start date
2013-11-13End date
2013-11-15Language
engPublication classification
E Conference publication; E1.1 Full written paper - refereedCopyright notice
2013, The AuthorsTitle of proceedings
ACML 2013 : Proceedings of the 5th Asian Conference on Machine LearningUsage metrics
Categories
No categories selectedKeywords
Licence
Exports
RefWorks
BibTeX
Ref. manager
Endnote
DataCite
NLM
DC