Experimental Study on Finding Audio-visual Association by Maximizing Mutual Information

Badgotia, Kamal Dheeriya (2015) Experimental Study on Finding Audio-visual Association by Maximizing Mutual Information. MTech thesis.



In this thesis, the fusion of audio and visual signal are done by considering one camera and one microphone. Although there are many approaches for audiovisual fusion, but still it is an open problem. Here, localizing and recognizing the speaker in a video by maximizing the mutual information occuring from audiovisual scene. Experiment is done using a method in which the arrival of audio and visual signals from a unique source are detected using a signal-level fusion technique. The information theoretic measure of audio-visual correspondence using probabilistic multi-modal generation statistical implicit model given in [1] is used here. In the used method non-parametric statistical density is able to give better result for characterizing the mutual information between signals from different domains by following some constraint for which entropy is maximized. By maximizing the above mutual information between different pairs of signals, it is possible to identify which person is speaking a given utterance, and we can verify whether the audio is associated with that visual seen or not. In doing so, no assumption is made about user appearance or speech of person. Further more this method doesn’t required any training of corpuses. In this thesis experimental results are demonstrated using CUAVE database.

Item Type:Thesis (MTech)
Uncontrolled Keywords:Non-Parametric, Mutual Information, Entropy, Information Theoretic Learning, Likelihood ratio, Audio-visual signal processing, Statistical Model, Parzen Window, Hypothesis Test, Projections
Subjects:Engineering and Technology > Electronics and Communication Engineering > Image Processing
Engineering and Technology > Electronics and Communication Engineering > Signal Processing
Divisions: Engineering and Technology > Department of Electronics and Communication Engineering
ID Code:7680
Deposited By:Mr. Sanat Kumar Behera
Deposited On:26 May 2016 10:38
Last Modified:26 May 2016 10:38
Supervisor(s):Roy, L P

Repository Staff Only: item control page