Full Frame Video Stabilization Using Motion Inpainting

Abstract

The amount of video data has increased dramatically with the advent of digital imaging. Most of the video captured these days originates from a mobile phones and handheld video cameras. Such videos are shaky compared to videos that are shot with a tripod mounted camera. Stabilizing this video to remove the shaky effect using software is called Digital video stabilization which results in a stable and visually pleasant video. In order digitally stabilize the image, we need to (1) Estimate the motion of camera, (2) Regenerate the motion of camera without the undesirable artifacts and (3) Synthesize new video frames. This dissertation is targeted at improving the last two steps of stabilizing the video. Most of the previous techniques of video stabilization produce a lower resolution stabilized video output and clip portions of frames to remove the empty area formed by transformation of the video frames. We use a Gaussian averaging filter to smoother the global motion in the video. Then the frames are transformed using the new transformation matrices obtained by subtracting the original transformation chain from the modified transformation chain. For the last step of synthesizing new video frames, we introduce an improved completion technique which can produce full frame video by using the pixel information from nearby frames to estimate the intensity of the missing pixels. This technique uses motion inpainting to ensure that the video frames are filled in both the static image area and dynamic image area with the same consistency. Additionally, the quality of the video is improved by using a deblurring algorithm which further improves the smoothness of video by eliminating undesirable motion blur. We do not estimate the PSF, in its place, we transfer and interpolate the sharper pixels from nearby frames to improve the sharpness and deblur current frame. Completing the video with motion inpainting and deblurring technique allow us to construct a full frame video stabilization system with good image quality. This is verified by implementing the technique on different video sequences.