Human Action Recognition using Extended Motion History Image and Fusion of Features

Abstract

The problem of human action recognition is solved as a machine learning problem. The research work solves the human action recognition in different ways. First, the Histogram of Oriented Gradient based human action recognition is studied. This research work introduces a concept of region of interest in terms of extending the Motion History Image by projecting the motion information in a video onto the 3D spatial temporal planes. There are 3 planes are projected which are plane-XY, plane-XT and plane-YT where X, Y and T represents the spatial (X, Y) and temporal (T) coordinates. The edges detected over the noise removed frames are projected onto the plane-XY. The actions over the interested regions are detected by higher order differences between the adjacent frames in a video. To eliminate smaller unwanted motion between the adjacent frames, the higher order differenced frames are further proceed with binarization using the predefined threshold. The binarized frame sequences are projected onto the plane-XT and plane-YT. Now the HOG features are extracted over all the projected planes separately and extracted features are concatenated together to represents a video. The extracted HOG features are high in dimensions. It may consist of redundant and irrelevant features to the learning model. It leads occurrence of over fitting problem and increases the computation time to train the classifier. In order to overcome these issues the Principle Component Analysis (PCA) based dimension reduction technique is employed to reduce high dimensional features into the lower dimensional space. Further to optimize the learning model, Pearson correlation and Spearman correlation filters based feature selection techniques are adopted. Second, the Histogram of Optical Flow (HOF) with Bag of Word (BoW) descriptor based human action recognition is studied. In the preprocessing stage the videos are converted into the gray scales and median filter is used for smoothening the frames which removes the noises and the same time edges are preserved. The optical flows between preprocessed frame sequences are calculated. The human object segmentation is achieved by combining the sobel edge detection in both horizontal and vertical directions. Further the segmented objects are enhanced by the morphological process which includes dilation, fill and erode operations. It will eliminate the sparse noise in the edge detected frame sequences. Next, the optical flow over the boundary of segmented human object is used for computing the HOF. The region wise HOF is determined by partitioning the segmented boundary into four sub-regions. Further the features are extracted by applying BoW descriptor over the computed HOF. Finally the extracted features are vi fused with the best HOG features obtained from the first method. The multiclass SVM based classifier with radial basis kernel is trained to recognize different human actions. The experiments are conducted on the benchmark KTH dataset. The performance of the proposed human action recognition methods HOG, HOF + Bow and HOG + HOF + BoW are evaluated separately and results are presented. The experimental results are compared with the state-of-the-art methods.