Semi-Supervised Learning in Random Forest Classifier for Human Action Recognition

Abstract

The objective of human action recognition is the interpretation of ongoing events and context from a video data for automated systems. Because of intra class variations and complex background, human action recognition is a very challenging task. In this work, a feature extraction technique viz histogram of 3D oriented gradients (HOG3D) and histogram of3D optical flow (HOF3D) are used to describe an action prominently. Video sequence is characterized by spatio-temporal interest points (STIPs) to represent the video sparsely and effectively for less complexity. STIPs describe the significant variations in both spatial and temporal directions of a video sequence. The extracted STIPs may contain noisy points due to the background clutter and illumination changes. To avoid such noisy points in proposed work, extracted STIPs are passed through motion history image (MHI). The classification accuracy is improved because of elimination of noisy STIPs. A sub volume patch is extracted around each noisy free STIPs. Initially the 2D gradients based histogram of oriented gradients (HOG) and 2D optical flow based histogram of optical flow (HOF) features are computed for action recognition. After that the improved 3D gradients and optical flow are computed along the spatial and temporal dimensions of the sub volume patches. The gradient and optical flow sub-volumes split into small cells. For each cell, an eight bin histogram estimated and all cells histograms are concatenated to form a histogram of 3Doriented gradient and optical flow features. Semi-supervised random forest classifier trained by HOG3D and HOF3D features for action classification. Both unsupervised and supervised learning methods used while building the trees. Unsupervised learning based splitting is done at the initial nodes of the tree and the predefined depth of the tree is reached learning is changed to supervised. The combination of both unsupervised and supervised learning at various nodes level of tree improves the classification accuracy. Mutual information estimated for STIP features towards all action classes by voting scores when passed through the random trees. Experiments performed on KTH, Weizmann and UCF sports datasets. The region of interest of MHI helps to reduce the noisy points effect. HOG3D, HOF3Dcombination features with semi-supervised learning random forest classifier improved the action recognition rate. The performance is compared with the existing state of the art methods.