scholarly journals Convolutional neural networks for relevance feedback in content based image retrieval

2020 ◽  
Vol 79 (37-38) ◽  
pp. 26995-27021
Author(s):  
Lorenzo Putzu ◽  
Luca Piras ◽  
Giorgio Giacinto
Keyword(s):  
Image Retrieval ◽  
Relevance Feedback ◽  
Image Representation ◽  
Semantic Gap ◽  
Content Based Image Retrieval ◽  
Data Sets ◽  
Great Success ◽  
Query Image ◽  
Pros And Cons ◽  
The One

Abstract Given the great success of Convolutional Neural Network (CNN) for image representation and classification tasks, we argue that Content-Based Image Retrieval (CBIR) systems could also leverage on CNN capabilities, mainly when Relevance Feedback (RF) mechanisms are employed. On the one hand, to improve the performances of CBIRs, that are strictly related to the effectiveness of the descriptors used to represent an image, as they aim at providing the user with images similar to an initial query image. On the other hand, to reduce the semantic gap between the similarity perceived by the user and the similarity computed by the machine, by exploiting an RF mechanism where the user labels the returned images as being relevant or not concerning her interests. Consequently, in this work, we propose a CBIR system based on transfer learning from a CNN trained on a vast image database, thus exploiting the generic image representation that it has already learned. Then, the pre-trained CNN is also fine-tuned exploiting the RF supplied by the user to reduce the semantic gap. In particular, after the user’s feedback, we propose to tune and then re-train the CNN according to the labelled set of relevant and non-relevant images. Then, we suggest different strategies to exploit the updated CNN for returning a novel set of images that are expected to be relevant to the user’s needs. Experimental results on different data sets show the effectiveness of the proposed mechanisms in improving the representation power of the CNN with respect to the user concept of image similarity. Moreover, the pros and cons of the different approaches can be clearly pointed out, thus providing clear guidelines for the implementation in production environments.

CONTENT-BASED IMAGE RETRIEVAL WITH RELEVANCE FEEDBACK USING ADAPTIVE PROCESSING OF TREE-STRUCTURE IMAGE REPRESENTATION

2003 ◽  
Vol 03 (01) ◽  
pp. 119-143 ◽  
Author(s):  
ZHIYONG WANG ◽  
ZHERU CHI ◽  
DAGAN FENG ◽  
AH CHUNG TSOI
Keyword(s):  
Neural Network ◽  
Image Retrieval ◽  
Network Model ◽  
Neural Network Model ◽  
Relevance Feedback ◽  
Image Representation ◽  
Tree Structure ◽  
Content Based Image Retrieval ◽  
Adaptive Processing ◽  
Image Content

Content-based image retrieval has become an essential technique in multimedia data management. However, due to the difficulties and complications involved in the various image processing tasks, a robust semantic representation of image content is still very difficult (if not impossible) to achieve. In this paper, we propose a novel content-based image retrieval approach with relevance feedback using adaptive processing of tree-structure image representation. In our approach, each image is first represented with a quad-tree, which is segmentation free. Then a neural network model with the Back-Propagation Through Structure (BPTS) learning algorithm is employed to learn the tree-structure representation of the image content. This approach that integrates image representation and similarity measure in a single framework is applied to the relevance feedback of the content-based image retrieval. In our approach, an initial ranking of the database images is first carried out based on the similarity between the query image and each of the database images according to global features. The user is then asked to categorize the top retrieved images into similar and dissimilar groups. Finally, the BPTS neural network model is used to learn the user's intention for a better retrieval result. This process continues until satisfactory retrieval results are achieved. In the refining process, a fine similarity grading scheme can also be adopted to improve the retrieval performance. Simulations on texture images and scenery pictures have demonstrated promising results which compare favorably with the other relevance feedback methods tested.

scholarly journals Interactive Content Based Image Retrieval using Multiuser Feedback

2017 ◽  
Vol 1 (4) ◽  
pp. 165
Author(s):  
M. Premkumar ◽  
R. Sowmya
Keyword(s):  
Image Retrieval ◽  
Association Rules ◽  
Relevance Feedback ◽  
Association Rule ◽  
Association Rule Mining ◽  
Human Perception ◽  
Content Based Image Retrieval ◽  
Rule Mining ◽  
Query Image ◽  
Feedback Techniques

Retrieving images from large databases becomes a difficult task. Content based image retrieval (CBIR) deals with retrieval of images based on their similarities in content (features) between the query image and the target image. But the similarities do not vary equally in all directions of feature space. Further the CBIR efforts have relatively ignored the two distinct characteristics of the CBIR systems: 1) The gap between high level concepts and low level features; 2) Subjectivity of human perception of visual content. Hence an interactive technique called the relevance feedback technique was used. These techniques used user’s feedback about the retrieved images to reformulate the query which retrieves more relevant images during next iterations. But those relevance feedback techniques are called hard relevance feedback techniques as they use only two level user annotation. It was very difficult for the user to give feedback for the retrieved images whether they are relevant to the query image or not. To better capture user’s intention soft relevance feedback technique is proposed. This technique uses multilevel user annotation. But it makes use of only single user feedback. Hence Soft association rule mining technique is also proposed to infer image relevance from the collective feedback. Feedbacks from multiple users are used to retrieve more relevant images improving the performance of the system. Here soft relevance feedback and association rule mining techniques are combined. During first iteration prior association rules about the given query image are retrieved to find out the relevant images and during next iteration the feedbacks are inserted into the database and relevance feedback techniques are activated to retrieve more relevant images. The number of association rules is kept minimum based on redundancy detection.

scholarly journals CONTENT BASED IMAGE RETRIEVAL SYSTEM BY FUSION OF COLOR, TEXTURE AND EDGE FEATURES WITH SVM CLASSIFIER AND RELEVANCE FEEDBACK

2018 ◽  
Vol 6 (9) ◽  
pp. 259-273
Author(s):  
Priyanka Saxena ◽  
Shefali
Keyword(s):  
Image Retrieval ◽  
Relevance Feedback ◽  
Retrieval System ◽  
Semantic Gap ◽  
Content Based Image Retrieval ◽  
Visual Features ◽  
Image Retrieval System ◽  
Combined Features ◽  
Edge Features ◽  
Color Texture

Content Based Image Retrieval system automatically retrieves the most relevant images to the query image by extracting the visual features instead of keywords from images. Over the years, several researches have been conducted in this field but the system still faces the challenge of semantic gap and subjectivity of human perception. This paper proposes the extraction of low-level visual features by employing color moment, Local Binary Pattern and Canny Edge Detection techniques for extracting color, texture and edge features respectively. The combination of these features is used in conjunction with Support Vector Machine to reduce the retrieval time and improve the overall precision. Also, the challenge of semantic gap between low and high level features is addressed by incorporating Relevance Feedback. Average precision value of 0.782 was obtained by combining the color, texture and edge features, 0.896 was obtained by using combined features with SVM, 0.882 was obtained by using combined features with Relevance Feedback to overcome the challenge of semantic gap. Experimental results exhibit improved performance than other state of the art techniques.

scholarly journals A State of Art on Content Based Image Retrieval Systems

2019 ◽  
Vol 8 (2S4) ◽  
pp. 297-301
Keyword(s):  
Image Retrieval ◽  
Relevance Feedback ◽  
Image Data ◽  
Distance Measures ◽  
Content Based Image Retrieval ◽  
Shape Features ◽  
Query Image ◽  
Indexing Methods ◽  
Retrieval Systems ◽  
State Of Art

The digital image data is quick expanding in capacity and heterogeneity. The customary information retrieval approaches are cannot fulfill the client's need, so there isneed to present a proficient framework for Content Based Image Retrieval(CBIR). The CBIR is an appealing wellspring of precise and quick retrieval. CBIR goes for discovering imagedatabases for explicit images that are like a given query image dependent on its features.In this paper the methodology of content based image retrieval are examined, investigated and thought about. Here, the different image substance, for example, colour, texture and shape features are mined by utilizing differentfeature extraction procedures, and furthermore extraordinary distance measures, Relevance Feedback (RF) and indexing methods are used to improve the execution of the CBIR system.The existing exploration strategies are talked about with their benefits and negative marks, so the further research works can be focused more.

Reducing the semantic gap in content-based image retrieval in mammography with relevance feedback and inclusion of expert knowledge

2008 ◽  
Vol 3 (1-2) ◽  
pp. 123-130 ◽  
Author(s):  
Paulo Mazzoncini de Azevedo-Marques ◽  
Natália Abdala Rosa ◽  
Agma Juci Machado Traina ◽  
Caetano Traina ◽  
Sérgio Koodi Kinoshita ◽  
...  
Keyword(s):  
Image Retrieval ◽  
Relevance Feedback ◽  
Expert Knowledge ◽  
Semantic Gap ◽  
Content Based Image Retrieval

scholarly journals Relevance Feedback Using RBF for Content Based Image Retrieval

2012 ◽  
Vol 6-7 ◽  
pp. 1150-1155
Author(s):  
Gui Zhi Li ◽  
Chang Sheng Zhou ◽  
Wei Wang ◽  
Ya Hui Liu
Keyword(s):  
Neural Network ◽  
Image Retrieval ◽  
Relevance Feedback ◽  
Rbf Neural Network ◽  
Feature Space ◽  
Content Based Image Retrieval ◽  
Query Image ◽  
Neural Network Learning ◽  
Multiple Queries ◽  
Semantic Relevance

Conventional approaches to image retrieval are based on the assumption that relevant images are physically near the query image in some feature space. However, semantically related images are often scattered across several visual clusters. This leads to adapting multiple queries to represent a query in the feature space. Therefore, it is necessary to handle disjunctive queries in the feature space. In this paper, a new content-based image retrieval method with relevance feedback technique using RBF neural network learning is proposed. The method transfers the process of relevance feedback into a learning problem of RBF neural network. RBFNN can describe the distribution of positive feedback sample images in feature space with a set of neighboring clusters produced through constructing neural network, for accurately reflecting their semantic relevance. The performance of the method using RBFNN is evaluated on a database of 10,000 images. Experimental results demonstrate the effectiveness of the proposed method.

scholarly journals A Content-Based Image Retrieval Method By Exploiting Cluster Shapes

2018 ◽  
Vol 14 (2) ◽  
pp. 90-102 ◽  
Author(s):  
Hanan Al-Jubouri ◽  
Hongbo Du
Keyword(s):  
Image Retrieval ◽  
Clustering Algorithm ◽  
Texture Features ◽  
Gaussian Mixture ◽  
Semantic Gap ◽  
Content Based Image Retrieval ◽  
Automatic Process ◽  
Query Image ◽  
Number Of Clusters ◽  
Leibler Divergence

Content-Based Image Retrieval (CBIR) is an automatic process of retrieving images that are the most similar to a query image based on their visual content such as colour and texture features. However, CBIR faces the technical challenge known as the semantic gap between high level conceptual meaning and the low-level image based features. This paper presents a new method that addresses the semantic gap issue by exploiting cluster shapes. The method first extracts local colours and textures using Discrete Cosine Transform (DCT) coefficients. The Expectation-Maximization Gaussian Mixture Model (EM/GMM) clustering algorithm is then applied to the local feature vectors to obtain clusters of various shapes. To compare dissimilarity between two images, the method uses a dissimilarity measure based on the principle of Kullback-Leibler divergence to compare pair-wise dissimilarity of cluster shapes. The paper further investigates two respective scenarios when the number of clusters is fixed and adaptively determined according to cluster quality. Experiments are conducted on publicly available WANG and Caltech6 databases. The results demonstrate that the proposed retrieval mechanism based on cluster shapes increases the image discrimination, and when the number of clusters is fixed to a large number, the precision of image retrieval is better than that when the relatively small number of clusters is adaptively determined.

Mosaic-Based Relevance Feedback for Image Retrieval

2011 ◽  
pp. 837-841
Author(s):  
Odej Kao ◽  
Ingo la Tendresse
Keyword(s):  
Image Retrieval ◽  
Relevance Feedback ◽  
A Priori ◽  
Standard Approach ◽  
Content Based Image Retrieval ◽  
Query Image ◽  
Starting Point ◽  
Image Ranking ◽  
Image Query ◽  
Similar Sample

A standard approach for content-based image retrieval (CBIR) is based on the extraction and comparison of features usually related to dominant colours, shapes, textures and layout (Del Bimbo, 1999). These features are a-priori defined and extracted, when the image is inserted into the database. At query time the user submits a similar sample image (query-by-sample-image) or draws a sketch (query-by-sketch) of the sought archived image. The similarity degree of the current query image and the target images is determined by calculation of a multidimensional distance between the corresponding features. The computed similarity values allow the creation of an image ranking, where the first k, usually k=32 or k=64, images are considered retrieval hits. These are chained in a list called ranking and then presented to the user. Each of these images can be used as a starting point for a refined search in order to improve the obtained results.

scholarly journals CONTENT BASED IMAGE RETRIEVAL ASSOCIATE WITH THE INTERSECTION OF MULTI KERNAL RELEVANCE VECTOR MACHINE AND HISTOGRAM JOIN

2015 ◽  
pp. 238-241
Author(s):  
K. VAISHNAVI ◽  
G.P.RAMESH KUMAR
Keyword(s):  
Active Learning ◽  
Image Retrieval ◽  
Relevance Feedback ◽  
Nearest Neighbor ◽  
Relevance Vector Machine ◽  
Content Based Image Retrieval ◽  
Training Set ◽  
Query Image ◽  
Histogram Intersection ◽  
Polynomial Kernels

Relevance Feedback is an important tool for grasping user's need in Interactive Content Based Image Retrieval (CBIR). Keeping this in mind, we have build up a framework using Relevance Vector Machine Classifier in interactive framework where user labels images as appropriate and inappropriate. The refinement of the images shown to the user is done using a few rounds of relevance feedback. This appropriate and inappropriate set then provides the training set for the RVM for each of these rounds. The method uses Histogram Intersection kernel with this interactive RVM (IKRVM). It has a retrieval component on top of this which searches for those images for retrieving which falls in the nearest neighbor set of the query image on the basis of histogram intersection based identical ranking (HIIR). The experimental results shows that the proposed framework shows better precision when compared with Active learning based RVMActive implemented with Radial Basis or Polynomial Kernels.

Sign in / Sign up

Export Citation Format

Share Document