A Novel Video Emotion Recognition System in the Wild Using a Random Forest Classifier

© Springer Nature Singapore Pte Ltd 2020. Emotions are expressed by humans to demonstrate their feelings in daily life. Video emotion recognition can be employed to detect various human emotions captured in videos. Recently, many researchers have been attracted to this research area and attempted to improve video emotion detection in both lab controlled and unconstrained environments. While the recognition rate of existing methods is high on lab-controlled datasets, they achieve much lower accuracy rates in a real-world uncontrolled environment. This is because of a variety of challenges present in real-world environments such as variations in illumination, head pose, and individual appearance. To address these challenges, in this paper, we propose a framework to recognize seven human emotions by extracting robust visual features from the videos captured in the wild and handle the head pose variation using a new feature extraction technique. First, sixty-eight face landmarks are extracted from different video sequences. Then, the Generalized Procrustes analysis (GPA) method is employed to normalize the extracted features. Finally, a random forest classifier is applied to recognize emotions. We have evaluated the proposed method using Acted Facial Expressions in the Wild (AFEW) dataset and obtained better accuracy than three existing video emotion recognition methods. It is noticeable that the proposed system can be applied to various contextual applications such as smart homes, healthcare, game industry and marketing in a smart city.