Static and Dynamic Hand Gesture Recognition Technique using RGB-D Information.

Abstract

The natural way of communication of the hearing impaired society is mainly through hand gestures. This technique has delineated as well as instinctive features which has a high ocular effect due to which it can be utilized in virtual reality or as a communication system whose foundation is laid by sign language and finger spelling. In the method of finger spelling, each letters/ words are represented by discrete and peculiar movement of the hand.

For this purpose, we have used depth cameras since the task of hand segmentation and hand tracking is easy with depth images. Depth images have high dimension, sothey cannot directly be fed to the machine learning models as their inputs. Therefore, Bag-of-Visual-Words (BOVW) approach plays a big role in dimensionality reduction and collect the information which is semantically useful. Thereafter, as an input to the Support Vector Machine (SVM) classifier, these extracted features are utilized. This approach his validated using a dataset which has static images of subjects showing American Sign Language (ASL) hand shapes. This dataset consists of 20 words with 5 subjects captured by Kinect sensor. Two different experiments are performed, one using only DATASET A which consists of ASL alphabet signs and the other DATASET B created during the course of the project. The experimental results show SURF performs better than SIFT when only depth information is used on the newly collected complex dataset of bare hand dynamic gestures shows higher level of recognition and outperforms other state-of-the-art method.