Human motion sequence characterization using machine learning techniques

Abstract

Motion capture data is a digital representation of the complex spatio-temporal structure of human motion. It is widely employed in a variety of areas such as animation, visual surveillance, athletic training and advanced interface. Many of the applications require the motion dataset tailored to that specific task, which means that the data captured in the past may not be useful in the present. So methods for reuse of motion capture data attract more and more attention. Many techniques that use the prerecorded motion sequences to create new, realistic motions have been published, in which finding appropriate motions from an existing database is a basic requirement. In this thesis, we focus on machine learning based human motion sequence characterization methods for human motion sequence retrieval and categorization. The target is to obtain the expected motions from a motion database effectively and efficiently. We first propose two different content-based motion capture data retrieval approaches based on the distribution of the motion data. The search is performed by giving an example motion and looking for similar ones from a database. In the first approach, for each motion sequence in the database, the Self-Organizing Maps (SOM) clustering algorithm is first adopted to partition the frames into different classes and get the associated class reference vectors. Then given a query motion, Probabilistic Principal Component Analysis (PPCA) is applied to estimate the distribution of its data. Finally the similarity between the query example and the motion sequence in a database is measured by the average Mahalanobis distance. According to the experimental results of the first approach, we find that: though we do not take into consideration the temporal variations and correlations between the frames of a motion, the retrieval performance is not bad. The drawback of the first approach is that, it is hard to applied in large database. There fore in the second approach, we take a different way. A novel motion retrieval method based on the Hierarchical Self Organizing Map (HSOM) index structure is proposed. Specifically, we first present a new approach for extracting a histogram feature vector which represents the distribution of the human motion data from an arbitrary motion data sequence. The histogram feature is designed such that it is invariant to translation, rotation and scaling. Then, Singular Value Decomposition (SVD) is applied to reduce the dimensionality of the feature vectors. In order to search the database more efficiently, a two-level indexing scheme based on the HSOM is constructed, in which we first partition the motion sequences using the dimensionality reduced feature vectors at the first level, and then we further classify the clusters associated with a parent node at the top level into sub-clusters using the original histogram feature vectors. Finally, fuzzy hierarchical search is implemented by first traversing the top level and then the second level associated with some of the parent nodes to search for the similar motions. As a further application of the proposed features, an ensemble based human motion categorization approach is proposed, in which we first design a cluster ensemble approach to construct the consensus matrix from the feature vectors. Then, the normalized cut algorithm is applied to partition the consensus matrix and assign the motions into the corresponding clusters. To modify our approaches to obtain more local features as well as to take the temporal variations of the motion frames into account, a refinement step is constructed to extend our motion retrieval and categorization approaches. In this step, we first segment a long motion sequence into distinct behaviors and extract the local histogram features for each sub-sequence. We then define a DTW based dissimilarity measure to compute the distance between two motions when their local histogram features arranged in the original temporal order. This step will be added into the previous approaches and compared with the previous methods that take no consideration of the local features. All the above methods are tested using the motion databases obtained from CMU motion capture laboratory and the results will be demonstrated.