Bidirectional Layer-by-layer Pre-training Method

In this paper, a bidirectional pre-training method for initializing weights of hetero-associative deep neural network was presented. Training of deep neural networks, because of confrontation with a large number of local minima, is not often converged. This is while through proper initializing weights instead of random values at the beginning of the training; it is possible to avoid many local minima. The bidirectional layer-by-layer pre-training method pre-train weights in forward and backward manners in parallel. Afterwards, the weight values resulted from their training are applied in the deep neural network. The bidirectional layer-by-layer pre-training was applied for pre-training of the classifier deep neural network weights, and revealed that both the training speed and the recognition rate were improved in Bosphorus and CK+ databases.