A Review on Pruning Techniques in Deep Neural Networks with Emphasis on Prune at Initial

With the expansion of the use of neural networks, increase deep of NN, and increase of network parameters, as well as the limitation of computational resources, the limitation of memory, and the incomprehensibility of these networks, the compression of neural networks is necessary. Compression must be intelligent, so as not to deprive us of the benefits of deep neural networks. Pruning is one of the compression methods that eliminate unnecessary network parameters. in recent research, Pruning has always been favored by researchers as far as a step called pruning at the initial design that pruned the initial network to include the benefits compression and pruning in the training and inference. this article reviews pruning techniques in deep neural networks with emphasis on Prune at initializing. First, the basics of pruning are discussed, then the types of pruning with the mathematical definition of each discussed, and finally, a more detailed study of pruning before network training has been done.