CityU Institutional Repository >
CityU Electronic Theses and Dissertations >
ETD - Dept. of Mathematics >
MA - Master of Philosophy >
Please use this identifier to cite or link to this item:
|Title: ||Learning gradients via gradient descent method|
|Other Titles: ||Ji yu ti du xia jiang suan fa de ti du xue xi|
|Authors: ||Guo, Xin (郭昕)|
|Department: ||Department of Mathematics|
|Degree: ||Master of Philosophy|
|Issue Date: ||2008|
|Publisher: ||City University of Hong Kong|
|Subjects: ||Sobolev gradients.|
|Notes: ||CityU Call Number: QA218 .G86 2008|
iv, 55 leaves 30 cm.
Thesis (M.Phil.)--City University of Hong Kong, 2008.
Includes bibliographical references (leaves 52-55)
|Abstract: ||We discuss the early stopping algorithm for gradient descent schemes
on learning the gradient of the regression function. The motivation
is to choose \useful" or \relevant" variables by a ranking method for
the \large dimension, small sample" problem, where we do the ranking
according to the norms of partial derivatives in some function spaces.
Satisfactory learning rates are derived.
In the algorithm, we used the early stopping technique, instead of
the classical Tikhonov regularization method, to avoid over-¯tting. The
advantage is that we need no longer consider the choice of the regular-
ization coe±cient, for which no e±cient methodology is available.
Many practical problems we confront have the character of high-
dimension and small-sample, data points are well separated with con¯-
dence. We formulate this observation precisely. Then the character is
carefully and completely exploited in the analysis of the sample error.
As a result, the learning rate has been improved to O(m¡c) (where m
denotes the sample size) with c free of the dimension n of the sample
space, when n > 23. We also give some analysis of the low-dimensional
cases with 2 · n · 23.|
|Online Catalog Link: ||http://lib.cityu.edu.hk/record=b2340679|
|Appears in Collections:||MA - Master of Philosophy |
Items in CityU IR are protected by copyright, with all rights reserved, unless otherwise indicated.