Development of Facial Expression Recognition System Using Machine Learning Techniques

Abstract

Emotion can be defined as a vital part of the physiological process, which can be described as a stimulus to any changes based on the events occurring around a person. It can be revealed through various channels, such as the variation in the body gestures, postures, speed and pitch of the tone, and most significantly through facial expressions. The face is the most dynamic and visible part of the body, thereby allowing efficient and easy transmission of knowledge about the emotions. Automated Facial Expression Recognition (FER) is a challenging task and has been an active topic of research in the domain of pattern recognition and computer vision due to its wide applications in natural Human-computer Interaction (HCI), behavioral science, clinical practice, telecommunication, animations, video games, security, surveillance, marketing and advertisement. In these applications, robust emotional awareness is a significant point to accomplish the assistive task at best. However, to make our day-to-day life more comfortable, smooth, and exciting, there is a great necessity for the development of a real-time, improved and robust FER system. This research is focused towards the improvement of the FER system in all its vital stages including, facial image acquisition, facial feature extraction, feature reduction, and classification. The key stages of the FER framework are feature extraction and classification. The existing literature reveals that most of the feature extraction techniques suffer from various issues like occlusion, illumination, face shape variation, rotation, translation, etc. To address these challenges, the first research contribution is focused on improving the recognition performance with the use of a powerful feature extractor called Histogram of Oriented Gradients (HOG). The performance of the proposed scheme is also tested using an important variant of HOG, namely Pyramid HOG (PHOG). According to the earlier investigations, features based on frequency domain techniques have outperformed the spacial domain methods. Therefore, a frequency domain based image transform specifically Stationary Wavelet Transform (SWT) is utilized as a feature extractor in the second contribution. SWT addresses the translation issues of the standard wavelet transform. In the subsequent contribution, we have integrated SWT coefficients and texture features to obtain improved performance. To preserve the capabilities of both frequency and spatial domain features, PHOG has been applied over SWT sub-bands to retrieve the prominent facial features. A combined Principal Component Analysis and Linear Discriminant Analysis (PCA+LDA) method is applied to generate a set of reduced and discriminant features. Further, a multi-scale and multi-directional feature extraction technique called Ripplet Transform Type II (RT-II) is explored. This method not only represents the image along various scales and directions but also represent the image with better edge information. The classification is carried out using Least Squares SVM (LS-SVM) with Radial Basis Function (RBF) kernel which offers several advantages such as low computational cost and local minima avoidance, and provides a solution to a convex optimization problem. The existing FER systems employ the traditional grid-search method to fine-tune the parameters of the classifiers, which is a time consuming task. To mitigate this issue, recent meta-heuristic approaches like Jaya Optimization (JO), and Whale Optimization (WO) are explored to fine-tune classifier parameters which prompts to low computational cost and better generalization performance. In the last contribution, the Variational Mode Decomposition (VMD) is used for feature extraction. For classification, a hybrid classifier is proposed by combining WO with Kernel Extreme Learning Machine (KELM). The WO algorithm is employed to find the optimal parameter of KELM with RBF kernel. The efficacy of different proposed methods is evaluated using two benchmark facial expression datasets, namely Extended Cohn-Kanade (CK+) and Japanese Female Facial Expressions (JAFFE). The simulation results are compared with the current state-of-the-art methods and the results demonstrate the superiority of the proposed schemes. The VMD and KELM based approach has been discovered to be the most precise and effective method for recognizing facial expressions and hence can be utilized to detect emotions in real-time.