Data compression for image-based relighting using radial basis functions

Abstract

In image based rendering with changing illumination, there is a set of precaptured images under fixed viewpoint but various sampling light directions. The pre-captured images are called reference images. Each reference image describes the appearance of an object under various light directions. Given a novel illumination condition, a novel image can be synthesized from reference images. The quality of the synthesized image depends on the number of reference images used. Therefore, an efficient compression algorithm for reference images is a must. This thesis proposes a compression scheme based on spherical radial basis function (SRBF) networks. The compression algorithm is divided into two levels. In the first level, we use one SRBF network to approximate the plenoptic property of each pixel. That means, the plenoptic property of every pixel is represented by a number of SRBF weights. Although a simple least square method can be used for extracting SRBF weights, the rendering image is very sensitive to the quantization noise introduced by the second level compression. So a constrained least square algorithm is proposed. Since the same SRBF weights from different pixels can be organized as a SRBF relighting map whose property is similar to an image, we then apply a wavelet-based compression technique to compress the SRBF relighting maps. Simulation results show that our proposed approach is superior to compressing the reference images with JPEG, JPEG2000 and MPEG in terms of compression ratio and reconstruction error. Compared with the spherical harmonic (SH) approach, the complexity of using the SRBF approach to render a novel image is much lower while the image quality is comparable. The way to allow users to interactively and real-time control the illumination of a scene is also discussed.