Human Action Recognition in Still Image of Human Pose using Multi-Stream neural Network

Today, human action recognition in still images has become one of the active topics in computer vision and pattern recognition. The focus is on identifying human action or behavior in a single static image. Unlike the traditional methods that use videos or a sequence of images for human action recognition, still images do not involve temporal information. Therefore, still image-based action recognition is more challenging compared to video-based recognition. Given the importance of motion information in action recognition, the Im2flow method has been used to estimate motion information from a static image. To do this, three deep neural networks are combined together, called a three-stream neural network. The proposed structure of this paper, namely the three-stream network, stemmed from the combination of three deep neural networks. The first, second and third networks are trained based on the raw color image, the optical flow predicted by the image, and the human pose obtained in the image, respectively. In other words, in this study, in addition to the predicted spatial and temporal information, the information on human pose is also used for human action recognition due to its importance in recognition performance. Results revealed that the introduced three-stream neural network can improve the accuracy of human action recognition. The accuracy of the proposed method on Willow7 action, Pascal voc2012, and Stanford10 data sets were 91.8%, 91.02%, and 96.97%, respectively, which indicates the promising performance of the introduced method compared to state-of-the-art performance.