City University of Hong Kong

CityU Institutional Repository >
3_CityU Electronic Theses and Dissertations >
ETD - Dept. of Mathematics  >
MA - Doctor of Philosophy  >

Please use this identifier to cite or link to this item:

Title: Statistical learning algorithms : multi-class classification and regression with non-i.i.d. sampling
Other Titles: Tong ji xue xi suan fa : duo fen lei ji fei du li tong fen bu chou yang xia de hui gui
統計學習算法 : 多分類及非獨立同分佈抽樣下的回歸
Authors: Pan, Zhiwei (潘志偉)
Department: Department of Mathematics
Degree: Doctor of Philosophy
Issue Date: 2009
Publisher: City University of Hong Kong
Subjects: Machine learning -- Statistical methods.
Notes: CityU Call Number: Q325.5 .P36 2009
v, 75 leaves : ill. 30 cm.
Thesis (Ph.D.)--City University of Hong Kong, 2009.
Includes bibliographical references (leaves [65]-75)
Type: thesis
Abstract: Learning theory is an inter-disciplinary research field involving applied mathematics, statistics, computer science, computational biology and data mining. It aims at learning function features (such as function value and variables) or data structures from samples by learning algorithms. Main research topics include designing efficient algorithms for various purposes and theoretical analysis of learning methods. In this thesis, we consider two research problems. The first is to propose a new learning algorithm for multi-class classification by Parzen windows and to conduct both theoretical understanding and application for this algorithm. This Parzen windows classifier is better than the usual way of designing multi-class classifiers by combining binary classifiers in various ways, which is often complex and has the problems of overlapping. We give the convergence rates of the excess misclassification error, under some regularity conditions on the conditional probability distributions and some decay conditions on the marginal distribution near the boundary of the input space. In the literature of Parzen windows for density estimation and regression, the approximation error is estimated locally at points which are in the interior of the input space X away from the boundary. Our key contribution for the mathematical analysis is to show how the decay of marginal distributions near the boundary yields satisfactory bounds for errors in terms of L1 or C(X) norms taken globally on the whole input space. The second research problem considered in this thesis is the study of learning algorithms with non-i.i.d. sampling. The algorithms include least square regularized regression and binary classification. In the last few years, there have been significant developments in theoretical understanding of learning algorithms with i.i.d. sampling. But either independence or identical sampling is a rather restrictive assumption in real data analysis, such as Shannon sampling, randomized sampling and weakly dependent sampling. Our setting does not require independence or identity. Under the conditions that the sequence of marginal distributions for sampling converges exponentially fast in the dual of a H¨older space and the sampling process satisfies a polynomial strong mixing condition, we derive capacity indepedent learning rates. Our convergence rate is consistent to that of the i.i.d. setting when the mixing condition parameter tends to zero. For a binary classification learning algorithm with non-identical sampling, we also derive satisfactory capacity dependent estimates for the excess misclassification error.
Online Catalog Link:
Appears in Collections:MA - Doctor of Philosophy

Files in This Item:

File Description SizeFormat
abstract.html132 BHTMLView/Open
fulltext.html132 BHTMLView/Open

Items in CityU IR are protected by copyright, with all rights reserved, unless otherwise indicated.


Valid XHTML 1.0!
DSpace Software © 2013 CityU Library - Send feedback to Library Systems
Privacy Policy · Copyright · Disclaimer