City University of Hong Kong

CityU Institutional Repository >
3_CityU Electronic Theses and Dissertations >
ETD - Dept. of Mathematics  >
MA - Doctor of Philosophy  >

Please use this identifier to cite or link to this item:

Title: Theory of learning algorithms generated by scaling
Other Titles: You shen suo suan zi chan sheng de xue xi suan fa de li lun yan jiu
Authors: Xiang, Daohong (向道紅)
Department: Department of Mathematics
Degree: Doctor of Philosophy
Issue Date: 2009
Publisher: City University of Hong Kong
Subjects: Computer algorithms.
Machine learning -- Mathematical models.
Notes: CityU Call Number: QA76.9.A43 X53 2009
v, 78 leaves 30 cm.
Thesis (Ph.D.)--City University of Hong Kong, 2009.
Includes bibliographical references (leaves [71]-78)
Type: thesis
Abstract: In this thesis, we study learning algorithms generated by means of the scaling operator f ! f( ¢ ¾ ) with a scaling parameter ¾ > 0: The main motivation for introducing scaling to learning theory is to learn function features or information with different frequency components when the scaling parameters ¾ changes, as done for signal processing or image compression in wavelet analysis. Since ¾ varies, learning algorithms provide richer information but the analysis becomes more involved. Such learning algorithms include graph Laplacians, multi-Gaussian classification schemes, moving least-square methods, Parzen windows, diffusion maps and learning the kernel parameters. Firstly, binary classification algorithms generated from Tikhonov regularization schemes associated with general convex loss functions and varying Gaussian kernels are considered. Fast convergence rates are provided for the excess misclassification error. Allowing varying Gaussian kernels in the algorithms improves learning rates measured by regularization error and sample error. Special structures of Gaussian kernels enable us to construct, by a nice approximation scheme with a Fourier analysis technique, uniformly bounded regularizing functions achieving polynomial decays of the regularization error under a Sobolev smoothness condition. The sample error is estimated by using a projection operator and a tight bound for the covering numbers of reproducing kernel Hilbert spaces generated by Gaussian kernels. The convexity of the general loss function plays a very important role in our analysis. Secondly, we consider the multi-class classification problem in learning theory. A learning algorithm by means of Parzen windows is introduced. Under some regularity conditions on the conditional probability for each class and some decay conditions of the marginal distribution near the boundary of the input space, we derive learning rates in terms of the sample size, window width and the decay of the basic window. The choice of the window width follows from bounds for the sample error and approximation error. A novelly defined splitting function for the multi-class classification and a comparison theorem, bounding the excess misclassification error by the norm of the difference of function vectors, is crucial in our analysis. Finally, the moving least-square method is studied for the regression problem in learning theory. We provide a learning algorithm associated with a finite dimensional hypothesis space of real valued functions. Mathematical analysis is conducted and error bounds are given.
Online Catalog Link:
Appears in Collections:MA - Doctor of Philosophy

Files in This Item:

File Description SizeFormat
abstract.html132 BHTMLView/Open
fulltext.html132 BHTMLView/Open

Items in CityU IR are protected by copyright, with all rights reserved, unless otherwise indicated.


Valid XHTML 1.0!
DSpace Software © 2013 CityU Library - Send feedback to Library Systems
Privacy Policy · Copyright · Disclaimer