Metadata Extraction from Persian Scientific Papers Using CRF Model

Metadata extraction from scientific papers is costly and time consuming. Different layouts and styles of papers increase the complexity of problem. Therefore, metadata extraction from scientific papers is a research question and different algorithms can be used to extract them. The purpose of this paper is to present a framework for metadata extraction from Persian scientific papers. CRF model has been used in this paper.
This paper is an applied research. It aims at presenting a framework for the metadata extraction. This framework includes identifying the header along with English and Persian references. CRF model has been used to extract metadata from header and references. This model can be modified by defining different features. The proposed method is tested over a set of 100 scientific papers taken from different Iranian journals. Compared to Markov in text tagging, this model has a higher accuracy than other models. On the other hand, this model is based on statistics. Extracting metadata while using statistics from papers with different layouts and styles provides better results than the rule based methods. Therefore, using this model is a good solution to this problem.
FINDING:F measure has been used to evaluate the proposed method. F measure is calculated for each token. Average F-measure is 96.89, 93.87, and 94.75 percent for header metadata, Persian references metadata, and English references metadata, respectively. The results of this paper have been compared with three similar papers in English. The results of the header author are better in English. Abstracts have better results in Persian language research. The analysis of the average references metadata extraction results shows that English researches have a higher accuracy compared to the Persian references metadata extraction results.
Reviewing the results shows that CRF model performance is good for extracting metadata. The most accurate metadata is Abstract with F measure of 99.6%. This metadata has a much larger number of tokens than the other metadata. The accuracy of the institute with the F measure is 80.95% lower than the other metadata. There are two reasons why F measure is reduced. First the number of this metadata is smaller than the other metadata in the text corpus. Second, the words used in this metadata are more diverse. In Persian references, the names of cities are used in location and institution metadata. This makes location and institution to be mistakenly identified in some cases. In Persian, the words commonly used in different metadata are more than English. For example, many Iranian names of the individuals are used with other meanings in other metadata. This issue may cause errors. Most of the errors in the metadata extraction are related to tokens that are located on the border of two metadata. Converting scientific papers in PDF format to text format is difficult in many cases and this is one of the limitations of this research. In this paper, a sample of 100 scientific articles was used. Increasing the number and variety of scientific papers for testing can have a positive effect on the results. A set of textual features are used in the CRF tagging algorithms. Changing these features can make the method better.