File(s) under permanent embargo
Max-Variance Convolutional Neural Network Model Compression
conference contribution
posted on 2020-01-01, 00:00 authored by Tanya Boone SifuentesTanya Boone Sifuentes, Antonio Robles-KellyAntonio Robles-Kelly, Asef NazariAsef NazariIn this paper, we present a method for convolutional neural network model compression which is based on the removal of filter banks that correspond to unimportant weights. To do this, we depart from the relationship between consecutive layers so as to obtain a factor that can be used to assess the degree upon which each pair of filters are coupled to each other. This allows us to use the unit-response of the coupling between two layers so as to remove pathways int he network that are negligible. Moreover, since the back-propagation gradients tend to diminish as the chain rule is applied from the output to the input layer, here we maximise the variance on the coupling factors while enforcing a monotonicity constraint that assures the most relevant pathways are preserved. We show results on widely used networks employing classification and facial expression recognition datasets. In our experiments, our approach delivers a very competitive trade-off between compression rates and performance as compared to both, the uncompressed models and alternatives elsewhere in the literature.
History
Event
Digital Image Computing: Techniques and Applications. Conference (2020 : Melbourne, Victoria)Pagination
1 - 6Publisher
IEEELocation
Melbourne, VictoriaPlace of publication
Piscataway, N.J.Publisher DOI
Start date
2020-11-30End date
2020-12-02ISBN-13
9781728191089Language
engPublication classification
E1 Full written paper - refereedTitle of proceedings
DICTA 2020 : Proceedings of the Digital Image Computing: Techniques and Applications ConferenceUsage metrics
Categories
No categories selectedKeywords
Licence
Exports
RefWorks
BibTeX
Ref. manager
Endnote
DataCite
NLM
DC