A New Model for Text Coherence Evaluation using Statistical Characteristics

Discourse coherence modeling evaluation becomes a critical but challenging task for all content analysis tasks in Natural Language Processing subfields, such as text summarization, question answering, text generation and machine translation. Existing methods like entitybased and graph-based models are engaging in semantic and linguistic concepts of a text and cannot solve the problem very well. Since they are only very limited to available word co-occurrence information in the sequential sentences within a short part of a text. One of the greatest challenges of the above methods is their limitation in long documents coherence evaluation and being suitable for documents with low number of sentences. Our proposed method focuses on both local and global coherence. It can also assess the local topic integrity of text at the paragraph level regardless of word meaning and handcrafted rules. The global coherence in the proposed method is evaluated by sequence paragraph dependency. Applying statistical approaches and based on recent results in word embeddings, the presented method studies how to incorporate the external word correlation knowledge into short and long stories to assess both local and global coherence, simultaneously. Using the effect of combined word2vec vectors and most likely n-grams, we show that our proposed method does not depend on the language and its semantic concepts. Results indicate that the proposed method is superior to other algorithms in terms of performance, accuracy in long documents with a high number of sentences.