High order Parzen windows and randomized sampling

Abstract

In the thesis, high order Parzen windows are studied for understanding some algorithms in learning theory and randomized sampling in multivariate approximation. Our ideas are from Parzen window method for density estimation and sampling theory. First, we define basic window functions to construct our high order Parzen windows. We derived learning rates for the least-square regression and density estimation on bounded domains under some decay conditions near the boundary on the marginal distribution of the probability measure for sampling. These rates can be almost optimal when the marginal distribution decays fast and the order of the Parzen windows is large enough. Compared with standard Parzen windows for density estimation, the high order Parzen window estimator is not a density function when the order J is greater than 2. Then for randomized sampling in shift-invariant spaces, we investigate the approximation of functions on the whole space Rn. We consider the situation when the sampling points are neither i.i.d. nor regular, but are noised from regular grids hZn for some constant h > 0 by probability density functions. We assume some decay and regularity conditions for the noise probability function and the approximated function on Rn. Under suitable choices of the scaling parameter, the approximation orders are estimated by means of regularity of the approximated function, the density function and the order of the Parzen windows. Next we study the approximation of multivariate functions in Sobolev spaces by high order Parzen windows in a non-uniform sampling setting. Sampling points are neither i.i.d. nor regular, but are noised from regular grids by non-uniform shifts of a probability density function. Sample function values at sampling points are drawn according to probability measures with expected values being values of the approximated function. Our main result provides bounds for the approximation of the target function on Rn in Sobolev spaces. Finally, we provide an experiment example from a real application. We use second order basic window functions to construct a second order Parzen windows. The algorithm works well both in artificial data and in the real application.