An Improved Extreme Learning Machine Using Cat Swarm Optimization

Abstract

Artificial neural networks are very useful in classification and function approximation tasks and there exists various algorithms for training a neural network. In past decade, one such decent and novel algorithm, extreme learning machine (ELM) has received attention due to its fast learning rate and better generalization ability in contrast to orthodox gradient descent learning algorithms for single-hidden-layer feed forward neural network. Despite it’s advantages, the single-hidden-layer feed forward neural network trained using ELM require more number of hidden neurons and faces poor condition problem as the input weights and hidden biases are randomly created. One way to overcome this problem is to use an optimization technique for finding an optimum set of input weights. In this work, we have proposed a hybrid learning scheme which uses one such optimisation technique called cat swarm optimization (CSO) to find an optimum set of input-hidden node weights and then this set of optimal input weights is used to analytically determine the output weights by using Moore-Penrose generalized inverse. Some optimisation techniques have been previously proposed like particle swarm optimisation but not CSO. In this work, modifications have been made to the existing CSO technique with the specific goal of further improving the generalization performance as well as the reducing the poor condition of networks trained using ELM. The proposed modifications 1) help in improving the search diversity of CSO, which allows better exploration of the solution search space and 2) improves the condition of the SLFN trained using ELM by including the norm of output weights in the evolutionary operator.
Three standard benchmark datasets have been collected from the UCI machine learning repository to validate the performance of the proposed scheme and comparative analysis has been made with other state of the art optimisation techniques. The proposed scheme, along with other evolutionary algorithms, requires a smaller number of hidden neuron compared to traditional the ELM for attaining almost similar classification accuracy. The experimental results also show that the proposed hybrid approach is able to produce well condition SLFNs with much better generalization performance.