Training a deep extreme learning machine by integrated structure and efficient learning algorithm for a deep autoencoder.

Recently, a number of Extreme Learning Machine (ELM) based training algorithms have been introduced for training deep neural network structures. ELM based Auto-Encoder (ELM-AE) is one such algorithm that has been used for making multilayer structures and tuning parameters of each layer. In a simple ELM-AE training algorithm, the weights of the first layer are initialized randomly. This issue is a leading factor in producing reconstruction error. The frequent use of ELM-AE in deep network layers results in propagating such errors through deep structures and in decreasing performance as a consequent. In this paper, we introduce a multilayer structure and a new learning algorithm to train it that prevents error propagation. In order to boost the performance of the model, the parameters in the first layer are initialized by a novel type of ELM-AE called Repeated-AE (RAE) rather than by a random selection method. This RAE-based technique determines the parameters in the first layer far better than do the other ELM-AE existed methods. Next, a single hidden layer ELM is applied for handling the classification task. Experimental results for data classification show that the proposed method outperforms some other methods in terms of the average accuracy over all datasets by amounts of 4%, 26%, 17% and 31%. Eventually, so as to verify the performance of the proposed multilayer ELM-AE in application, we used this model to reconstruct images. The reconstructed images obtained by our approach appeared visually a lot better compared to those obtained by the other methods do.