Node Selection Algorithm for Fault-Tolerant Extreme Learning Machines

Abstract

Extreme Learning Machine (ELM), one of the neural network models, has been studied over 20 years. It is known for its simplicity and universal approximation ability. However, node noise problem has been neglected in the conventional ELM learning algorithms. In addition, many nodes are required during network training, which would increase learning time. Therefore, this project aims to investigate the behaviour of ELM under fault situation and propose an ADMM-based node selection algorithm for fault-tolerant ELM network. The proposed algorithm is an objective function which describes the training error of the ELM network with the consideration of node noise and node selection. Optimization technique was then applied to minimize the objective function such that the training error is at minimum. Concerning the node noise problem, which is unavoidable in network implementation, this project considers two kinds of node noise, i.e. additive noise and multiplicative noise. A noise regularization term describing the two types of noise was added to the objective function so that ELM can be trained to handle this failure. For the node selection, a ?1 norm penalty term was also added to the objective function. This penalty term is able to eliminate some insignificant nodes by setting some ? terms, i.e. the weights between hidden layer and output layer, to be zero. Then, the alternating direction method of multipliers (ADMM) framework was applied to derive an algorithm to minimize the objective function. Chapter 5 provides some simulation results to verify the effectiveness of my proposed algorithm over the conventional ones for ELM network under failure situation.