Weakly-supervised Anomaly Detection and Classification in Untrimmed Surveillance Videos

Abstract

Anomaly detection and classification is a crucial task for ensuring public safety in real world surveillance videos. Due to the increasing number of crimes, act of terrorism, and accidents, timely detection and classification of such anomaly cases can minimize the loss. However, the task of anomaly detection and classification is challenging due to the unavailability of large annotated video data, untrimmed nature of the surveillance videos, and difficulties in obtaining discriminative feature representations for anomaly categories. This thesis aims at mitigating the mentioned challenges by adopting weakly supervised learning methods, efficient temporal dependency modeling schemes in untrimmed videos and extracting discriminative feature representations from space time convolutional neural network (3DCNN). For this, three contributions have been made in this thesis and each of the contribution mitigate one or more challenges. The first contribution focus on extracting discriminative spatiotemporal feature representation from a proposed multilevel 3DCNN for anomaly video sequences and subsequently utilizes long short term memory (LSTM) module for obtaining effective temporal dependency encoding in anomaly detection task. From result analysis, it is found that the feature representation obtained from the multilevel 3DCNN is not only superior over existing competent methods but also it is robust to challenges like illumination changes, partial occlusion. In the second contribution, the knowledge of image and video classification is used through inflated 3DCNN for obtaining enhanced feature representation. In addition, a temporal interintra clip pooling strategy is proposed for efficient temporal encoding by LSTM module in anomaly detection task. The effectiveness of this contribution has been demonstrated over existing methods and the previous contribution through exhaustive experimentation. Since the first two contributions only focus on anomaly detection task in untrimmed surveillance videos, the task of anomaly classification still remains challenging. Thus, the third contribution of this thesis proposes a framework that jointly handles the anomaly detection and classification task. In addition, the proposed framework is driven by twolevels of temporal attention mechanism which highlights the temporal saliency in the feature map as well as develops a dependency between the detection and classification task. From exhaustive experimental analysis and visualization, it is observed that the proposed method not only boosts the anomaly detection performance but also substantially improves the classification accuracy.