Content Based Image Retrieval by Fusion of Multilevel Results

Content based image retrieval (CBIR) applies machine vision techniques to extract similar images for a given query image. The main challenge of CBIR is the semantic gap between low level pixel and segment based features and high-level concepts in the image. An approach towards reducing this gaps is to use high level region and object based features. However, the low-level features describe image details and enforce between image discriminations. Accordingly, it is expected that the use of both feature types will lead to better results. This paper tries to reduce the mentioned gap by combining decision results at four granularities, namely pixel, region, object, and concept levels. Pixel level retrieval adopts SIFT features and local binary patterns. Region level subsystem partitions the image into a set of segments and extracts their color and texture features using hue descriptor and Gabor filters for subsequent processing. AlexNet convolutional neural network is employed for object based retrieval. Word2vec embedding is used for concept level retrieval that exploits conceptual relations between objects to enhance the retrieval results. Experiments over Wang and GHIM datasets confirm the feasibility of the proposed combination and conclude that it improves overall performance of the retrieval system.