Video Subject Inpainting: A Posture-Based Method

Despite recent advances in video inpainting techniques, reconstructing large missing regions of a moving subject while its scale changes remains an elusive goal. In this paper, we have introduced a scale-change invariant method for large missing regions to tackle this problem. Using this framework, first the moving foreground is separated from the background and its scale is equalized. Then, a nonlinear dimension reduction is performed using manifold learning. We interpolate the missing values using motion analysis based on 3D body pose changing. The motion of the subject is considered as two different time series for which values are forecasted in the missing region. Since the subject motion may be non-periodic and forecasted series being different, a single series in the missing region is interpolated using blending of two forecasted series. Finally, the interpolated points are refined using weighted signal matching. In this method, a radial basis function (RBF) network has been used for mapping from spatial to manifold space. Experimental results, for a number of videos, show that the proposed approach is better than other methods and 3D body pose changing is smooth in the inpainted sequences.