Trace Ratio Optimization based Semi-Supervised Multimodal Nonlinear Dimensionality Reduction for Marginal Manifold Visualization

Abstract

Visualizing similarity data of different objects by exhibiting more separate organizations with local and multimodal characteristics preserved is important in multivariate data analysis. Laplacian Eigenmaps (LAE) and Locally Linear Embedding (LLE) aim at preserving the projections of all similarity pairs in the close vicinity of the reduced output space, but they are unable to identify and separate inter-class neighbors. This paper considers the semi-supervised manifold learning problems. We apply the pairwise Cannot-Link and Must-Link constraints induced by the neighborhood graph to specify the types of neighboring pairs. More flexible regulation on supervised information is provided. Two novel multimodal nonlinear techniques, which we call trace ratio (TR) criterion based semi-supervised LAE (S2LAE) and LLE (S2LLE), are then proposed for marginal manifold visualization. Utilizing the TR optimization, the similarity based on the Euclidean distance is effectively preserved by the orthogonal projective matrix. We also present the kernelized S2LAE and S2LLE. We verify the feasibility of S2LAE and S2LLE through extensive simulations over benchmark real-world MIT CBCL, CMU PIE, MNIST and USPS datasets. Manifold visualizations show that S2LAE and S2LLE are able to deliver large margins between different clusters or classes with multimodal distributions preserved. Clustering evaluations show they can achieve comparable to or even better results than some widely used methods.