Improving the Performance of Text Sentiment Analysis using Deep Convolutional Neural Network Integrated with Hierarchical Attention Layer

Sentiment analysis is considered as one of the most essential tasks in the field of natural language processing and cognitive science. In order to enhance the performance of sentiment analysis techniques, it is necessary to not only classify the sentences based on their sentimental labels but also to extract the informative words that contribute to the classification decision. In this regard, deep neural networks based on the attention mechanism have achieved considerable progress in recent years. However, there is still a limited number of studies on attention mechanisms for text classification and especially sentiment analysis. To fill this lacuna, a Convolution Neural Network (CNN) integrated with attention layer is presented in this paper that is able to extract informative words and assign them higher weights based on the context. In the attention layer, the proposed model employs a context vector and tries to measure the importance of a word as the similarity between the context vector and word vector. Then, by integrating the new vectors obtained from the attention layer into sentence vectors, the new generated vectors are used for classification. In order to verify the performance of the proposed model, various experiments were conducted on the Stanford datasets. Based on the results of the experiments, the proposed model not only significantly outperforms other existing studies but also is able to consider the context to extract the informative words which can be considered as a value in analysis and application.