Hand gesture recognition based on fusion of moments

Abstract

This work is focussed on three main issues in developing a gesture recognition system. These are (i) Threshold independent skin colour segmentation using Modified K-means clustering and Mahalanobish distance (ii) illumination normalization (iii) user independent gesture recognition based on fusion of Moments. Since skin pixels can vary with different illumination condition, to find the range of skin pixels, becomes a hard task in case of colour space based skin colour segmentation. This work proposes a semi-supervised learning algorithm based on modified K-means clustering and Mahalanobis distance to extract human skin colour regions from the static hand gesture colour images. An efficient illumination invariant algorithm based on power law transform and averaging RGB colour space is proposed. Normalized binary silhouette is extracted from the hand gesture image and background and object noise is removed by Morphological filtering. Non-orthogonal moments like geometric moments and orthogonal moments like Tchebichef and Krawtchouk moments are used here as features. The Krawtchouk moment features are found to be very effective in hand gesture recognition compared to Tchebichef and Geometric moment features. To make the system real time efficient, different users are used for training and testing. In user-independent situation, neither of these moments has shown efficient classification accuracy. To improve the performance of classification, two feature fusion strategies have been proposed in this work; serial feature fusion and parallel feature fusion. A feed-forward multi-layer perceptron (MLP) based artificial neural network classifier is used in this work as a classifier. The proposed two fusion based moment features especially parallel fusion of Krawtchouk and Tchebichef moment has shown better performance as user-independent. The proposed hand gesture recognition system can be well realized for real time implementation of gesture based applications.