scholarly journals Progressive Domain-Independent Feature Decomposition Network for Zero-Shot Sketch-Based Image Retrieval

2020 ◽  
Author(s):  
Xinxun Xu ◽  
Muli Yang ◽  
Yanhua Yang ◽  
Hao Wang
Keyword(s):  
Image Retrieval ◽  
Semantic Information ◽  
Semantic Space ◽  
Semantic Knowledge ◽  
Visual Features ◽  
Semantic Features ◽  
Retrieval Task ◽  
Common Space ◽  
Low Dimensional ◽  
Domain Independent

Zero-Shot Sketch-Based Image Retrieval (ZS-SBIR) is a specific cross-modal retrieval task for searching natural images given free-hand sketches under the zero-shot scenario. Most existing methods solve this problem by simultaneously projecting visual features and semantic supervision into a low-dimensional common space for efficient retrieval. However, such low-dimensional projection destroys the completeness of semantic knowledge in original semantic space, so that it is unable to transfer useful knowledge well when learning semantic features from different modalities. Moreover, the domain information and semantic information are entangled in visual features, which is not conducive for cross-modal matching since it will hinder the reduction of domain gap between sketch and image. In this paper, we propose a Progressive Domain-independent Feature Decomposition (PDFD) network for ZS-SBIR. Specifically, with the supervision of original semantic knowledge, PDFD decomposes visual features into domain features and semantic ones, and then the semantic features are projected into common space as retrieval features for ZS-SBIR. The progressive projection strategy maintains strong semantic supervision. Besides, to guarantee the retrieval features to capture clean and complete semantic information, the cross-reconstruction loss is introduced to encourage that any combinations of retrieval features and domain features can reconstruct the visual features. Extensive experiments demonstrate the superiority of our PDFD over state-of-the-art competitors.

scholarly journals Color Based Image Retrieval by Combining Various Features

2019 ◽  
Vol 9 (2) ◽  
pp. 454-460
Keyword(s):  
Image Retrieval ◽  
Retrieval System ◽  
Visual Features ◽  
Semantic Features ◽  
Retrieval Rate ◽  
Image Retrieval System ◽  
Mathematical Formulas ◽  
Image Set ◽  
Different Color ◽  
Image Collection

Content based image retrieval system retrieve the images according to the strong feature related to desire as color, texture and shape of an image. Although visual features cannot be completely determined by semantic features, but still semantic features can be integrate easily into mathematical formulas. This paper is focused on retrieval of images within a large image collection, based on color projection by applying segmentation and quantification on different color models and compared for good result. This method is applied on different categories of image set and evaluated its retrieval rate in different models

scholarly journals Deep Learning Triplet Ordinal Relation Preserving Binary Code for Remote Sensing Image Retrieval Task

2021 ◽  
Vol 13 (23) ◽  
pp. 4786
Author(s):  
Zhen Wang ◽  
Nannan Wu ◽  
Xiaohan Yang ◽  
Bingqi Yan ◽  
Pingping Liu
Keyword(s):  
Remote Sensing ◽  
Image Retrieval ◽  
Large Scale ◽  
Nearest Neighbor ◽  
Binary Code ◽  
Satellite Observation ◽  
Cross Entropy ◽  
Retrieval Task ◽  
Search Tasks ◽  
Low Dimensional

As satellite observation technology rapidly develops, the number of remote sensing (RS) images dramatically increases, and this leads RS image retrieval tasks to be more challenging in terms of speed and accuracy. Recently, an increasing number of researchers have turned their attention to this issue, as well as hashing algorithms, which map real-valued data onto a low-dimensional Hamming space and have been widely utilized to respond quickly to large-scale RS image search tasks. However, most existing hashing algorithms only emphasize preserving point-wise or pair-wise similarity, which may lead to an inferior approximate nearest neighbor (ANN) search result. To fix this problem, we propose a novel triplet ordinal cross entropy hashing (TOCEH). In TOCEH, to enhance the ability of preserving the ranking orders in different spaces, we establish a tensor graph representing the Euclidean triplet ordinal relationship among RS images and minimize the cross entropy between the probability distribution of the established Euclidean similarity graph and that of the Hamming triplet ordinal relation with the given binary code. During the training process, to avoid the non-deterministic polynomial (NP) hard problem, we utilize a continuous function instead of the discrete encoding process. Furthermore, we design a quantization objective function based on the principle of preserving triplet ordinal relation to minimize the loss caused by the continuous relaxation procedure. The comparative RS image retrieval experiments are conducted on three publicly available datasets, including UC Merced Land Use Dataset (UCMD), SAT-4 and SAT-6. The experimental results show that the proposed TOCEH algorithm outperforms many existing hashing algorithms in RS image retrieval tasks.

scholarly journals Zero-Shot Sketch-Based Image Retrieval via Graph Convolution Network

2020 ◽  
Vol 34 (07) ◽  
pp. 12943-12950
Author(s):  
Zhaolong Zhang ◽  
Yuejie Zhang ◽  
Rui Feng ◽  
Tao Zhang ◽  
Weiguo Fan
Keyword(s):  
Image Retrieval ◽  
Visual Information ◽  
Semantic Information ◽  
Visual Space ◽  
Semantic Space ◽  
Generalization Ability ◽  
Variational Autoencoder

Zero-Shot Sketch-based Image Retrieval (ZS-SBIR) has been proposed recently, putting the traditional Sketch-based Image Retrieval (SBIR) under the setting of zero-shot learning. Dealing with both the challenges in SBIR and zero-shot learning makes it become a more difficult task. Previous works mainly focus on utilizing one kind of information, i.e., the visual information or the semantic information. In this paper, we propose a SketchGCN model utilizing the graph convolution network, which simultaneously considers both the visual information and the semantic information. Thus, our model can effectively narrow the domain gap and transfer the knowledge. Furthermore, we generate the semantic information from the visual information using a Conditional Variational Autoencoder rather than only map them back from the visual space to the semantic space, which enhances the generalization ability of our model. Besides, feature loss, classification loss, and semantic loss are introduced to optimize our proposed SketchGCN model. Our model gets a good performance on the challenging Sketchy and TU-Berlin datasets.

scholarly journals Biomedical Imaging Modality Classification Using Combined Visual Features and Textual Terms

2011 ◽  
Vol 2011 ◽  
pp. 1-7 ◽  
Author(s):  
Xian-Hua Han ◽  
Yen-Wei Chen
Keyword(s):  
Image Retrieval ◽  
Imaging Modality ◽  
Kernel Functions ◽  
Visual Features ◽  
Color Intensity ◽  
Global Features ◽  
Retrieval Task ◽  
Block Based ◽  
Cross Language ◽  
Textual Feature

We describe an approach for the automatic modality classification in medical image retrieval task of the 2010 CLEF cross-language image retrieval campaign (ImageCLEF). This paper is focused on the process of feature extraction from medical images and fuses the different extracted visual features and textual feature for modality classification. To extract visual features from the images, we used histogram descriptor of edge, gray, or color intensity and block-based variation as global features and SIFT histogram as local feature. For textual feature of image representation, the binary histogram of some predefined vocabulary words from image captions is used. Then, we combine the different features using normalized kernel functions for SVM classification. Furthermore, for some easy misclassified modality pairs such as CT and MR or PET and NM modalities, a local classifier is used for distinguishing samples in the pair modality to improve performance. The proposed strategy is evaluated with the provided modality dataset by ImageCLEF 2010.

Dual discriminative auto-encoder network for zero shot image recognition

2021 ◽  
pp. 1-12
Author(s):  
Haoyue Bai ◽  
Haofeng Zhang ◽  
Qiong Wang
Keyword(s):  
Image Recognition ◽  
Feature Space ◽  
Semantic Space ◽  
High Dimensional ◽  
Visual Features ◽  
Discriminative Feature ◽  
Latent Space ◽  
Embedding Strategy ◽  
Low Dimensional ◽  
Semantic Attributes

Zero Shot learning (ZSL) aims to use the information of seen classes to recognize unseen classes, which is achieved by transferring knowledge of the seen classes from the semantic embeddings. Since the domains of the seen and unseen classes do not overlap, most ZSL algorithms often suffer from domain shift problem. In this paper, we propose a Dual Discriminative Auto-encoder Network (DDANet), in which visual features and semantic attributes are self-encoded by using the high dimensional latent space instead of the feature space or the low dimensional semantic space. In the embedded latent space, the features are projected to both preserve their original semantic meanings and have discriminative characteristics, which are realized by applying dual semantic auto-encoder and discriminative feature embedding strategy. Moreover, the cross modal reconstruction is applied to obtain interactive information. Extensive experiments are conducted on four popular datasets and the results demonstrate the superiority of this method.

scholarly journals Image Representation Using Stacked Colour Histogram

Algorithms ◽  
2021 ◽  
Vol 14 (8) ◽  
pp. 228
Author(s):  
Ezekiel Mensah Martey ◽  
Hang Lei ◽  
Xiaoyu Li ◽  
Obed Appiah
Keyword(s):  
Image Retrieval ◽  
Image Representation ◽  
Vital Role ◽  
Image Texture ◽  
Visual Features ◽  
Retrieval Task ◽  
Classification Rate ◽  
Memory Space ◽  
Image Blurring ◽  
Neighbourhood Information

Image representation plays a vital role in the realisation of Content-Based Image Retrieval (CBIR) system. The representation is performed because pixel-by-pixel matching for image retrieval is impracticable as a result of the rigid nature of such an approach. In CBIR therefore, colour, shape and texture and other visual features are used to represent images for effective retrieval task. Among these visual features, the colour and texture are pretty remarkable in defining the content of the image. However, combining these features does not necessarily guarantee better retrieval accuracy due to image transformations such rotation, scaling, and translation that an image would have gone through. More so, concerns about feature vector representation taking ample memory space affect the running time of the retrieval task. To address these problems, we propose a new colour scheme called Stack Colour Histogram (SCH) which inherently extracts colour and neighbourhood information into a descriptor for indexing images. SCH performs recurrent mean filtering of the image to be indexed. The recurrent blurring in this proposed method works by repeatedly filtering (transforming) the image. The output of a transformation serves as the input for the next transformation, and in each case a histogram is generated. The histograms are summed up bin-by-bin and the resulted vector used to index the image. The image blurring process uses pixel’s neighbourhood information, making the proposed SCH exhibit the inherent textural information of the image that has been indexed. The SCH was extensively tested on the Coil100, Outext, Batik and Corel10K datasets. The Coil100, Outext, and Batik datasets are generally used to assess image texture descriptors, while Corel10K is used for heterogeneous descriptors. The experimental results show that our proposed descriptor significantly improves retrieval and classification rate when compared with (CMTH, MTH, TCM, CTM and NRFUCTM) which are the start-of-the-art descriptors for images with textural features.

scholarly journals Computational reconstruction of mental representations using human behavior

2022 ◽  
Author(s):  
Laurent Caplette ◽  
Nicholas Turk-Browne
Keyword(s):  
Mental Representations ◽  
Semantic Space ◽  
Direct Access ◽  
Visual Features ◽  
Semantic Mapping ◽  
Semantic Features ◽  
Visual Concept ◽  
Visual Concepts ◽  
Computational Reconstruction ◽  
High Level

Revealing the contents of mental representations is a longstanding goal of cognitive science. However, there is currently no general framework for providing direct access to representations of high-level visual concepts. We asked participants to indicate what they perceived in images synthesized from random visual features in a deep neural network. We then inferred a mapping between the semantic features of their responses and the visual features of the images. This allowed us to reconstruct the mental representation of virtually any common visual concept, both those reported and others extrapolated from the same semantic space. We successfully validated 270 of these reconstructions as containing the target concept in a separate group of participants. The visual-semantic mapping uncovered with our method further generalized to new stimuli, participants, and tasks. Finally, it allowed us to reveal how the representations of individual observers differ from each other and from those of neural networks.

scholarly journals Zero-Shot Image Classification Based on a Learnable Deep Metric

Sensors ◽  
2021 ◽  
Vol 21 (9) ◽  
pp. 3241
Author(s):  
Jingyi Liu ◽  
Caijuan Shi ◽  
Dongjing Tu ◽  
Ze Shi ◽  
Yazhi Liu
Keyword(s):  
Image Classification ◽  
Euclidean Distance ◽  
Similarity Score ◽  
Visual Features ◽  
Semantic Features ◽  
Common Space ◽  
Training Samples ◽  
Mapping Process ◽  
The Common ◽  
End To End

The supervised model based on deep learning has made great achievements in the field of image classification after training with a large number of labeled samples. However, there are many categories without or only with a few labeled training samples in practice, and some categories even have no training samples at all. The proposed zero-shot learning greatly reduces the dependence on labeled training samples for image classification models. Nevertheless, there are limitations in learning the similarity of visual features and semantic features with a predefined fixed metric (e.g., as Euclidean distance), as well as the problem of semantic gap in the mapping process. To address these problems, a new zero-shot image classification method based on an end-to-end learnable deep metric is proposed in this paper. First, the common space embedding is adopted to map the visual features and semantic features into a common space. Second, an end-to-end learnable deep metric, that is, the relation network is utilized to learn the similarity of visual features and semantic features. Finally, the invisible images are classified, according to the similarity score. Extensive experiments are carried out on four datasets and the results indicate the effectiveness of the proposed method.

Bridging the Semantic Gap in Image Retrieval

2002 ◽  
pp. 14-36 ◽  
Author(s):  
Rhong Zhao ◽  
William I. Grosky
Keyword(s):  
Information Retrieval ◽  
Image Retrieval ◽  
Research Field ◽  
Visual Features ◽  
Semantic Features ◽  
Visual Data ◽  
Image Content ◽  
Low Level ◽  
Significant Research ◽  
High Level

The emergence of multimedia technology and the rapidly expanding image and video collections on the Internet have attracted significant research efforts in providing tools for effective retrieval and management of visual data. Image retrieval is based on the availability of a representation scheme of image content. Image content descriptors may be visual features such as color, texture, shape, and spatial relationships, or semantic primitives. Conventional information retrieval was based solely on text, and those approaches to textual information retrieval have been transplanted into image retrieval in a variety of ways. However, “a picture is worth a thousand words.” Image content is much more versatile compared with text, and the amount of visual data is already enormous and still expanding very rapidly. Hoping to cope with these special characteristics of visual data, content-based image retrieval methods have been introduced. It has been widely recognized that the family of image retrieval techniques should become an integration of both low-level visual features addressing the more detailed perceptual aspects and high-level semantic features underlying the more general conceptual aspects of visual data. Neither of these two types of features is sufficient to retrieve or manage visual data in an effective or efficient way (Smeulders, et al., 2000). Although efforts have been devoted to combining these two aspects of visual data, the gap between them is still a huge barrier in front of researchers. Intuitive and heuristic approaches do not provide us with satisfactory performance. Therefore, there is an urgent need of finding the latent correlation between low-level features and high-level concepts and merging them from a different perspective. How to find this new perspective and bridge the gap between visual features and semantic features has been a major challenge in this research field. Our chapter addresses these issues.

Research on High-Level Semantic Image Retrieval

2011 ◽  
Vol 268-270 ◽  
pp. 1427-1432
Author(s):  
Chang Yong Ri ◽  
Min Yao
Keyword(s):  
Image Retrieval ◽  
Image Understanding ◽  
Extraction Methods ◽  
Semantic Gap ◽  
Visual Features ◽  
Semantic Features ◽  
Image Description ◽  
Semantic Image Retrieval ◽  
Image Grammar ◽  
High Level

This paper presented the key problems to shorten “semantic gap” between low-level visual features and high-level semantic features to implement high-level semantic image retrieval. First, introduced ontology based semantic image description and semantic extraction methods based on machine learning. Then, illustrated image grammar on the high-level semantic image understanding and retrieval, and-or graph and context based methods of semantic image. Finally, we discussed the development directions and research emphases in this field.

Sign in / Sign up

Export Citation Format

Share Document