Intelligent facial emotion recognition using a layered encoding cascade optimization model

Neoh, Siew Chin, Zhang, Li, Mistry, Kamlesh, Hossain, Mohammed Alamgir, Lim, Chee Peng, Aslam, Nauman and Kinghorn, Philip 2015, Intelligent facial emotion recognition using a layered encoding cascade optimization model, Applied soft computing, vol. 34, pp. 72-93, doi: 10.1016/j.asoc.2015.05.006.

Attached Files
Name Description MIMEType Size Downloads

Title Intelligent facial emotion recognition using a layered encoding cascade optimization model
Author(s) Neoh, Siew Chin
Zhang, Li
Mistry, Kamlesh
Hossain, Mohammed Alamgir
Lim, Chee PengORCID iD for Lim, Chee Peng
Aslam, Nauman
Kinghorn, Philip
Journal name Applied soft computing
Volume number 34
Start page 72
End page 93
Total pages 22
Publisher Elsevier
Place of publication Amsterdam, The Netherlands
Publication date 2015-09
ISSN 1568-4946
Keyword(s) Cascade optimization
Ensemble classifier
Evolutionary algorithm
Facial expression recognition
Feature selection
Layered representation structure
Summary In this research, we propose a facial expression recognition system with a layered encoding cascade optimization model. Since generating an effective facial representation is a vital step to the success of facial emotion recognition, a modified Local Gabor Binary Pattern operator is first employed to derive a refined initial face representation and we then propose two evolutionary algorithms for feature optimization including (i) direct similarity and (ii) Pareto-based feature selection, under the layered cascade model. The direct similarity feature selection considers characteristics within the same emotion category that give the minimum within-class variation while the Pareto-based feature optimization focuses on features that best represent each expression category and at the same time provide the most distinctions to other expressions. Both a neural network and an ensemble classifier with weighted majority vote are implemented for the recognition of seven expressions based on the selected optimized features. The ensemble model also automatically updates itself with the most recent concepts in the data. Evaluated with the Cohn-Kanade database, our system achieves the best accuracies when the ensemble classifier is applied, and outperforms other research reported in the literature with 96.8% for direct similarity based optimization and 97.4% for the Pareto-based feature selection. Cross-database evaluation with frontal images from the MMI database has also been conducted to further prove system efficiency where it achieves 97.5% for Pareto-based approach and 90.7% for direct similarity-based feature selection and outperforms related research for MMI. When evaluated with 90° side-view images extracted from the videos of the MMI database, the system achieves superior performances with >80% accuracies for both optimization algorithms. Experiments with other weighting and meta-learning combination methods for the construction of ensembles are also explored with our proposed ensemble showing great adpativity to new test data stream for cross-database evaluation. In future work, we aim to incorporate other filtering techniques and evolutionary algorithms into the optimization models to further enhance the recognition performance.
Language eng
DOI 10.1016/j.asoc.2015.05.006
Field of Research 080108 Neural, Evolutionary and Fuzzy Computation
Socio Economic Objective 970108 Expanding Knowledge in the Information and Computing Sciences
HERDC Research category C1 Refereed article in a scholarly journal
ERA Research output type C Journal article
Copyright notice ©2015, Elsevier
Persistent URL

Connect to link resolver
Unless expressly stated otherwise, the copyright for items in DRO is owned by the author, with all rights reserved.

Version Filter Type
Citation counts: TR Web of Science Citation Count  Cited 29 times in TR Web of Science
Scopus Citation Count Cited 33 times in Scopus
Google Scholar Search Google Scholar
Access Statistics: 448 Abstract Views, 2 File Downloads  -  Detailed Statistics
Created: Wed, 12 Aug 2015, 14:36:19 EST

Every reasonable effort has been made to ensure that permission has been obtained for items included in DRO. If you believe that your rights have been infringed by this repository, please contact