CityU Institutional Repository >
3_CityU Electronic Theses and Dissertations >
ETD  Dept. of Mathematics >
MA  Doctor of Philosophy >
Please use this identifier to cite or link to this item:
http://hdl.handle.net/2031/5414

Title:  Theory of learning algorithms generated by scaling 
Other Titles:  You shen suo suan zi chan sheng de xue xi suan fa de li lun yan jiu 由伸縮算子產生的學習算法的理論研究 
Authors:  Xiang, Daohong (向道紅) 
Department:  Department of Mathematics 
Degree:  Doctor of Philosophy 
Issue Date:  2009 
Publisher:  City University of Hong Kong 
Subjects:  Computer algorithms. Machine learning  Mathematical models. 
Notes:  CityU Call Number: QA76.9.A43 X53 2009 v, 78 leaves 30 cm. Thesis (Ph.D.)City University of Hong Kong, 2009. Includes bibliographical references (leaves [71]78) 
Type:  thesis 
Abstract:  In this thesis, we study learning algorithms generated by means of the scaling operator
f ! f( ¢
¾ ) with a scaling parameter ¾ > 0: The main motivation for introducing
scaling to learning theory is to learn function features or information with different
frequency components when the scaling parameters ¾ changes, as done for signal processing
or image compression in wavelet analysis. Since ¾ varies, learning algorithms
provide richer information but the analysis becomes more involved. Such learning
algorithms include graph Laplacians, multiGaussian classification schemes, moving
leastsquare methods, Parzen windows, diffusion maps and learning the kernel parameters.
Firstly, binary classification algorithms generated from Tikhonov regularization
schemes associated with general convex loss functions and varying Gaussian kernels
are considered. Fast convergence rates are provided for the excess misclassification
error. Allowing varying Gaussian kernels in the algorithms improves learning rates
measured by regularization error and sample error. Special structures of Gaussian
kernels enable us to construct, by a nice approximation scheme with a Fourier analysis
technique, uniformly bounded regularizing functions achieving polynomial decays of
the regularization error under a Sobolev smoothness condition. The sample error is
estimated by using a projection operator and a tight bound for the covering numbers
of reproducing kernel Hilbert spaces generated by Gaussian kernels. The convexity of
the general loss function plays a very important role in our analysis.
Secondly, we consider the multiclass classification problem in learning theory. A
learning algorithm by means of Parzen windows is introduced. Under some regularity
conditions on the conditional probability for each class and some decay conditions of the marginal distribution near the boundary of the input space, we derive learning rates
in terms of the sample size, window width and the decay of the basic window. The
choice of the window width follows from bounds for the sample error and approximation
error. A novelly defined splitting function for the multiclass classification and a
comparison theorem, bounding the excess misclassification error by the norm of the
difference of function vectors, is crucial in our analysis.
Finally, the moving leastsquare method is studied for the regression problem in
learning theory. We provide a learning algorithm associated with a finite dimensional
hypothesis space of real valued functions. Mathematical analysis is conducted and
error bounds are given. 
Online Catalog Link:  http://lib.cityu.edu.hk/record=b2339800 
Appears in Collections:  MA  Doctor of Philosophy

Items in CityU IR are protected by copyright, with all rights reserved, unless otherwise indicated.
