Assistive System for Visually Impaired using Object Recognition

Abstract

Object recognition based electronic aid is most promising for visually impaired people to get description of nearby objects. With the advancement in computer vision and computing technologies we can afford to develop a system for visually impaired people, which can give audio feedback of surrounding objects and context. This thesis explore object recognition method to assist visually impaired people. We proposed an object recognition algorithm, and an assistive system which is very useful for their safety, quality life and freedom from other person all the time. Consideration of Gabor-recursive neural network along with convolutional recursive neural network(CRNN) with less number of maps have been found to be very promising to achieve better accuracy with less time complexity as compare to CRNN, the extracted feature vector is used to train Softmax classifier which is then used to classify query image into one of the trained categories. Our color recognition algorithm is simple and fast, which is the desiderata of color recognition module, we are using HSI color space with observed threshold and random sampling. The second contribution of this thesis is to use above stated methods to assist visually impaired people, the proposed assistive system is ensemble of two modules (i) object and (ii) color recognition, it is implemented using multimedia processor equipped embedded board and OpenCV. Object recognition module recognize objects such as door, chair, stairs, mobile phone etc. and generate an audio feedback to the user. Color recognition module generates audio description about object color in front of the camera, it is useful in recognizing clothing colors, fruits color etc. The system modules and operation can be selected using an on demand push button panel which contains two push buttons. The object recognition algorithm is evaluated on on-line available dataset as well as on our dataset and compared with state of art methods.