Bayesian graph-theoretic approaches to image structure modeling and matching

Abstract

Image structure modeling and matching remain two active topics in computer vision for decades, and compose the foundation for many content-based applications in media computing. Currently, most research on these two problems is usually conducted independently with different methods. In this dissertation, we generalize image structure modeling and matching with a unified paradigm, namely the Bayesian graph-theoretic approach. We present a series of novel structural modeling and matching algorithms arising from the Bayesian graph formulation. This dissertation mainly addresses the following three problems: 1) How to extract a reliable structural labeling using Bayesian decision theory, Markov random fields (MRFs) and graph formulation? 2) How to construct a compact and effective structural model to encode both global topology and salient local features? 3) How to efficiently match two structural models under moderate occlusion and clutter? Our work on the first problem focuses on integrating graph formulation and graphtheoretic methods with MRFs and Bayesian model. The MRFs and Bayesian decision theory provide a powerful tool for statistical image analysis and structural pattern modeling. However, self-validation always remains an open problem for traditional methods. Besides, the computational complexity of maximum a posteriori probability (MAP) estimation is another critical weakness, leading to a large sacrifice of accuracy for speed. Using Bayesian graph formulation and graph-theoretic methods, we proposed a general paradigm, namely graduated graph mincuts (GGC) that is self-validated and significantly reduce the computational complexity compared to flat MRF/MAP methods. Within the GGC framework, we propose three concrete algorithms: tree-structured graph cuts (TSGC), net-structured graph cuts (NSGC) and hierarchical graph cuts (HGC). We also generalize the discriminative spectral clustering to Bayesian spectral clustering by combining with generative models. We address the second problem based on the proposed structural labeling methods. A reliable structural model called Bayesian structural content abstraction (BaSCA) is proposed, which is an attributed graph representation of image structural content. The BaSCA model is robust to non-content changing operations (NCOs) and sensitive to content-changing operations (COs). In addition, to support dynamic NCO/CO partition, we further optimize the BaSCA structure model in the user-defined NCO space with an analogy mean shift algorithm, namely identical structure extension. Besides the robustness, another nice property of the BaSCA model is that it can be extended naturally by integrating salient local features. The BaSCA model is applied to region-level image authentication. Experiments show that the BaSCA signature significantly improves the false positive rate and has comparable false negative rates with previous methods. We study the third problem, structural matching, with spectral graph theory. We show how to handle the occlusion/clutter problem in the normalized eigenspace using a spectral approach, which is considered difficult for classical spectral methods. We also discuss the spectral multiplicity problem in graph matching, and propose a multiplicity-tolerant algorithm for structural matching. Finally, the proposed algorithms are applied to solving some real-world problems in computer vision and media computing, such as media content authentication, automatic/ interactive segmentation, shape matching and retrieval etc. Some related problems, e.g., multiple grouping cues extraction and shape structure abstraction, are also discussed. The algorithms developed in this dissertation are not limited to two-dimensional Cartesian spaces or any metric structure. Therefore, the Bayesian graph-theoretic approach may readily find a large variety of applications in computer vision, pattern recognition and media computing.