Sequential Markov random fields for human body parts tracking

Abstract

This thesis presents the sequential Markov random fields (SMRFs) for tracking human body parts in the monocular settings, which plays an important role in computer vision and pattern recognition with many potential applications in areas of entertainment, automatic surveillance, health care, virtual reality and human-computer interface. There has been increasing interests in developing an automatic system to detect, localize and track human body parts with flexibility and robustness. However, it is a challenging task to achieve high performance in terms of efficiency and accuracy because of the high degrees of freedom (DOF) of the articulated human body, background clutters, and missing body parts due to partial occlusions. We handle these problems by the hybrid strategy, where the temporal dependencies between two successive human poses are described by the sequential Monte Carlo (SMC) method, and the spatial relationships between body parts in a pose is described by the Markov random fields (MRFs). This hybrid SMRF is able to account for spatio-temporal dependencies in moving human parts, enhancing the overall tracking performance in partial occlusions and background clutters. Within the unified SMRF framework, we formulate the tracking task as a sequential labeling problem by minimizing the posterior energy function of both body parts’ temporal motion and their spatial structure. We also develop efficient inference and learning algorithms for the SMRF based on the relaxation labeling(RL). To reduce the search space, we use a labeling-by-synthesis manner to predict moving path based on MRF-based sequential importance sampling. The Experimental results from motion capture data and real walking video demonstrate that the SMRF can effectively and accurately detect and track human body parts.