Function approximation for lighting property

Abstract

To relight an object under various illumination conditions, we need to model the reflectance property of its surface elements. Mathematically, the reflectance property of a surface element is a spherical function. Its input domain is the spherical surface and the output range is the set of real numbers. To facilitate the relighting process, we usually use a spherical basis, either in discrete or continuous form, to approximate the spherical functions. This thesis addresses several issues in the approximation process. Firstly, the spherical harmonic (SH) approach is the conventional method for representing the low frequency lighting effects. In this approach, a spherical function is approximated as a weighted sum of SH basis functions. However, under some conditions, the estimated SH coefficients are very sensitive to quantization noise. To reduce the noise sensitivity, we will investigate how the magnitude of SH coefficients affects the rendering results when the estimated SH coefficients are quantized. Afterwards, two fast fitting methods, namely constrained least square (CLS) and constrained eigen basis (CEB) method, for estimating the noise-resistant SH coefficients are proposed. CLS is developed based on the regularization used in neural network community while CEB is developed by considering the eigen decomposition of SH basis functions. They can effectively control the magnitude of the estimated SH coefficients, and hence suppress the rendering artifacts. Secondly, to preserve all-frequency lighting effects, we need to use the wavelet representation. However, the conventional discrete cubemap based wavelet approach has an uneven sampling problem. To ensure all sampling values are fully utilized, we need to have an efficient discrete spherical wavelet which can evenly digitize the spherical surface. This thesis proposes an icosahedron spherical wavelet (ISW) approach. We use the relighting of an illumination adjustable image (IAI) under time-varying distant lighting environment as an example to demonstrate the effectiveness of the ISW approach. The proposed ISW approach provides uniform sampling points distributed over the spherical domain, thus we can efficiently handle high-frequency variations locally. While the SH approach produces smooth and pleasant rendering effects, it fails to capture high-frequency lighting effects such as shadow. The discrete spherical wavelet approach successfully captures high-frequency lighting effects under distant environment. It may introduce visual artifact for local illumination such as a point source or equivalently when the distant environment contains a small dominant region. Therefore, at the end of this thesis, a novel multiscale spherical radial basis function (MSRBF) representation is proposed. The proposed MSRBF approach is developed based on the classical radial basis function (RBF) approach used in the pattern recognition community. In pattern recognition or neural networks, the input dimension is usually much greater than two. Therefore, it is difficult for us to extend the RBF approach to a multiscale manner. However, in the representation of a spherical function, the input domain is a 2D spherical surface. As a result, we can effectively develop MSRBF basis functions distributed over the spherical surface. The proposed MSRBF approach can capture all-frequency lighting effects without introducing noticeable visual artifact. Besides, all basis functions in the MSRBF approach have the same mathematical form. They can be efficiently evaluated and achieve the real-time rate required in time-critical applications. The proposed MSRBF approach is fully scalable. Reducing the number of basis functions can be simply achieved by dropping less important (detail) basis functions. Hence the rendering can be progressive and adaptive to the computation power of the host machine. No re-estimation of coefficients is needed. While most of the existing works mainly focus on the realistic rendering under distant environment, practical applications like games still strongly rely on local illumination. The proposed MSRBF approach supports both distant environment and the hypothetical local illumination without affecting its visual quality and computational efficiency.