CityU Institutional Repository >
CityU Electronic Theses and Dissertations >
ETD - Dept. of Mathematics >
MA - Doctor of Philosophy >
Please use this identifier to cite or link to this item:
|Title: ||Theory of learning algorithms generated by scaling|
|Other Titles: ||You shen suo suan zi chan sheng de xue xi suan fa de li lun yan jiu|
|Authors: ||Xiang, Daohong (向道紅)|
|Department: ||Department of Mathematics|
|Degree: ||Doctor of Philosophy|
|Issue Date: ||2009|
|Publisher: ||City University of Hong Kong|
|Subjects: ||Computer algorithms.|
Machine learning -- Mathematical models.
|Notes: ||CityU Call Number: QA76.9.A43 X53 2009|
v, 78 leaves 30 cm.
Thesis (Ph.D.)--City University of Hong Kong, 2009.
Includes bibliographical references (leaves -78)
|Abstract: ||In this thesis, we study learning algorithms generated by means of the scaling operator
f ! f( ¢
¾ ) with a scaling parameter ¾ > 0: The main motivation for introducing
scaling to learning theory is to learn function features or information with different
frequency components when the scaling parameters ¾ changes, as done for signal processing
or image compression in wavelet analysis. Since ¾ varies, learning algorithms
provide richer information but the analysis becomes more involved. Such learning
algorithms include graph Laplacians, multi-Gaussian classification schemes, moving
least-square methods, Parzen windows, diffusion maps and learning the kernel parameters.
Firstly, binary classification algorithms generated from Tikhonov regularization
schemes associated with general convex loss functions and varying Gaussian kernels
are considered. Fast convergence rates are provided for the excess misclassification
error. Allowing varying Gaussian kernels in the algorithms improves learning rates
measured by regularization error and sample error. Special structures of Gaussian
kernels enable us to construct, by a nice approximation scheme with a Fourier analysis
technique, uniformly bounded regularizing functions achieving polynomial decays of
the regularization error under a Sobolev smoothness condition. The sample error is
estimated by using a projection operator and a tight bound for the covering numbers
of reproducing kernel Hilbert spaces generated by Gaussian kernels. The convexity of
the general loss function plays a very important role in our analysis.
Secondly, we consider the multi-class classification problem in learning theory. A
learning algorithm by means of Parzen windows is introduced. Under some regularity
conditions on the conditional probability for each class and some decay conditions of the marginal distribution near the boundary of the input space, we derive learning rates
in terms of the sample size, window width and the decay of the basic window. The
choice of the window width follows from bounds for the sample error and approximation
error. A novelly defined splitting function for the multi-class classification and a
comparison theorem, bounding the excess misclassification error by the norm of the
difference of function vectors, is crucial in our analysis.
Finally, the moving least-square method is studied for the regression problem in
learning theory. We provide a learning algorithm associated with a finite dimensional
hypothesis space of real valued functions. Mathematical analysis is conducted and
error bounds are given.|
|Online Catalog Link: ||http://lib.cityu.edu.hk/record=b2339800|
|Appears in Collections:||MA - Doctor of Philosophy |
Items in CityU IR are protected by copyright, with all rights reserved, unless otherwise indicated.