Faster training of very deep networks via p-norm gates Pham, Trang, Tran, Truyen, Phung, Dinh and Venkatesh, Svetha 2016, Faster training of very deep networks via p-norm gates, in ICPR 2016: Proceedings of the 23rd International Conference on Pattern Recognition, IEEE, Piscataway, N.J., pp. 3542-3547, doi: 10.1109/ICPR.2016.7900183. Attached Files Name Description MIMEType Size Downloads pham-fastertraining-post-2016.pdf Accepted version application/pdf 802.94KB 17 Title Faster training of very deep networks via p-norm gates Author(s) Pham, Trang Tran, Truyen orcid.org/0000-0001-6531-8907 Phung, Dinh orcid.org/0000-0002-9977-8247 Venkatesh, Svetha orcid.org/0000-0001-8675-6631 Conference name Pattern Recognition. International Conference (23rd : 2016 : Cancun, Mexico) Conference location Cancun, Mexico Conference dates 4-8 Dec. 2016 Title of proceedings ICPR 2016: Proceedings of the 23rd International Conference on Pattern Recognition Publication date 2016 Conference series Pattern Recognition International Conference Start page 3542 End page 3547 Total pages 6 Publisher IEEE Place of publication Piscataway, N.J. Summary A major contributing factor to the recent advances in deep neural networks is structural units that let sensory information and gradients to propagate easily. Gating is one such structure that acts as a flow control. Gates are employed in many recent state-of-the-art recurrent models such as LSTM and GRU, and feedforward models such as Residual Nets and Highway Networks. This enables learning in very deep networks with hundred layers and helps achieve record-breaking results in vision (e.g., ImageNet with Residual Nets) and NLP (e.g., machine translation with GRU). However, there is limited work in analysing the role of gating in the learning process. In this paper, we propose a flexible p-norm gating scheme, which allows usercontrollable flow and as a consequence, improve the learning speed. This scheme subsumes other existing gating schemes, including those in GRU, Highway Networks and Residual Nets as special cases. Experiments on large sequence and vector datasets demonstrate that the proposed gating scheme helps improve the learning speed significantly without extra overhead. ISBN 9781509048472 Language eng DOI 10.1109/ICPR.2016.7900183 Field of Research 080109 Pattern Recognition and Data Mining Socio Economic Objective 0 Not Applicable HERDC Research category E1 Full written paper - refereed ERA Research output type E Conference publication Copyright notice ©2016, IEEE Free to Read? Yes Persistent URL http://hdl.handle.net/10536/DRO/DU:30096794 Document type: Conference Paper Collections: Faculty of Science, Engineering and Built Environment School of Information Technology Centre for Pattern Recognition and Data Analytics (PRADA) Open Access Collection 2018 ERA Submission Related Links Link Description Connect to published version Go to link with your DU access privileges     Unless expressly stated otherwise, the copyright for items in DRO is owned by the author, with all rights reserved. Every reasonable effort has been made to ensure that permission has been obtained for items included in DRO. If you believe that your rights have been infringed by this repository, please contact drosupport@deakin.edu.au. Versions Version Filter Type Thu, 18 May 2017, 17:05:50 EST Thu, 18 May 2017, 17:07:16 EST Fri, 19 May 2017, 05:00:30 EST Sat, 27 May 2017, 02:32:31 EST Tue, 30 May 2017, 17:05:35 EST Tue, 30 May 2017, 17:06:28 EST Wed, 31 May 2017, 06:05:23 EST Fri, 02 Jun 2017, 13:06:17 EST Fri, 02 Jun 2017, 13:12:09 EST Fri, 02 Jun 2017, 13:13:07 EST Tue, 01 Aug 2017, 11:49:15 EST Sat, 07 Oct 2017, 01:30:13 EST Fri, 07 Sep 2018, 12:51:16 EST Filtered Full Citation counts:   Cited 2 times in TR Web of Science Cited 4 times in Scopus Search Google Scholar Access Statistics: 489 Abstract Views, 23 File Downloads  -  Detailed Statistics Created: Tue, 30 May 2017, 17:06:28 EST

Every reasonable effort has been made to ensure that permission has been obtained for items included in DRO. If you believe that your rights have been infringed by this repository, please contact drosupport@deakin.edu.au.