A Deep Neural Network-Based Information Retrieval Method for Complex Question-Answering Systems

Question-answering systems, as the next generation of search engines, have the capability to retrieve relevant answers to queries posed in natural language. These systems generally consist of three main components: question processing, information retrieval, and answer extraction, with various methods introduced for each component. One of the most important aspects is information retrieval and selecting relevant paragraphs. Nowadays, most user queries in question-answering systems are complex. To respond to such queries, it initially requires understanding the question and then retrieving various documents that are lexically and semantically related to the query. In recent years, advancements in deep neural network-based learning on one hand and the introduction of high-quality datasets on a large scale on the other have attracted researchers' attention to this field. In this research, a method for selecting relevant paragraphs for complex question-answering systems on the HotpotQA dataset is introduced. To select relevant paragraphs, the type of question is first recognized using a deep neural network. Then, using BERT language models, relevant paragraphs that show evidence of the answer are selected in several steps. This process uses keyword extraction from the question. The results obtained indicate that the outcomes are better compared to the baseline method