scholarly journals Knowledge-Embedded Representation Learning for Fine-Grained Image Recognition

2018 ◽  
Author(s):  
Tianshui Chen ◽  
Liang Lin ◽  
Riquan Chen ◽  
Yang Wu ◽  
Xiaonan Luo
Keyword(s):  
Neural Network ◽  
Knowledge Representation ◽  
Image Recognition ◽  
Feature Learning ◽  
Representation Learning ◽  
Feature Representation ◽  
Image Feature ◽  
Knowledge Graph ◽  
Feature Maps ◽  
Fine Grained

Humans can naturally understand an image in depth with the aid of rich knowledge accumulated from daily lives or professions. For example, to achieve fine-grained image recognition (e.g., categorizing hundreds of subordinate categories of birds) usually requires a comprehensive visual concept organization including category labels and part-level attributes. In this work, we investigate how to unify rich professional knowledge with deep neural network architectures and propose a Knowledge-Embedded Representation Learning (KERL) framework for handling the problem of fine-grained image recognition. Specifically, we organize the rich visual concepts in the form of knowledge graph and employ a Gated Graph Neural Network to propagate node message through the graph for generating the knowledge representation. By introducing a novel gated mechanism, our KERL framework incorporates this knowledge representation into the discriminative image feature learning, i.e., implicitly associating the specific attributes with the feature maps. Compared with existing methods of fine-grained image classification, our KERL framework has several appealing properties: i) The embedded high-level knowledge enhances the feature representation, thus facilitating distinguishing the subtle differences among subordinate categories. ii) Our framework can learn feature maps with a meaningful configuration that the highlighted regions finely accord with the nodes (specific attributes) of the knowledge graph. Extensive experiments on the widely used Caltech-UCSD bird dataset demonstrate the superiority of our KERL framework over existing state-of-the-art methods.

scholarly journals Aspect-level sentiment analysis merged with knowledge graph and graph convolutional neural network

2021 ◽  
Vol 2083 (4) ◽  
pp. 042044
Author(s):  
Zuhua Dai ◽  
Yuanyuan Liu ◽  
Shilong Di ◽  
Qi Fan
Keyword(s):  
Neural Network ◽  
Sentiment Analysis ◽  
Structural Information ◽  
Knowledge Graph ◽  
Convolutional Network ◽  
Text Data ◽  
Short Text ◽  
Fine Grained ◽  
Syntactic Information ◽  
Text Information

Abstract Aspect level sentiment analysis belongs to fine-grained sentiment analysis, w hich has caused extensive research in academic circles in recent years. For this task, th e recurrent neural network (RNN) model is usually used for feature extraction, but the model cannot effectively obtain the structural information of the text. Recent studies h ave begun to use the graph convolutional network (GCN) to model the syntactic depen dency tree of the text to solve this problem. For short text data, the text information is not enough to accurately determine the emotional polarity of the aspect words, and the knowledge graph is not effectively used as external knowledge that can enrich the sem antic information. In order to solve the above problems, this paper proposes a graph co nvolutional neural network (GCN) model that can process syntactic information, know ledge graphs and text semantic information. The model works on the “syntax-knowled ge” graph to extract syntactic information and common sense information at the same t ime. Compared with the latest model, the model in this paper can effectively improve t he accuracy of aspect-level sentiment classification on two datasets.

scholarly journals Semantic Relationships Guided Representation Learning for Facial Action Unit Recognition

2019 ◽  
Vol 33 ◽  
pp. 8594-8601 ◽  
Author(s):  
Guanbin Li ◽  
Xin Zhu ◽  
Yirui Zeng ◽  
Qing Wang ◽  
Liang Lin
Keyword(s):  
Neural Network ◽  
Facial Expressions ◽  
Mutual Exclusion ◽  
Representation Learning ◽  
Feature Representation ◽  
Semantic Relationship ◽  
Action Unit ◽  
Facial Action ◽  
Feature Representations ◽  
Regional Feature

Facial action unit (AU) recognition is a crucial task for facial expressions analysis and has attracted extensive attention in the field of artificial intelligence and computer vision. Existing works have either focused on designing or learning complex regional feature representations, or delved into various types of AU relationship modeling. Albeit with varying degrees of progress, it is still arduous for existing methods to handle complex situations. In this paper, we investigate how to integrate the semantic relationship propagation between AUs in a deep neural network framework to enhance the feature representation of facial regions, and propose an AU semantic relationship embedded representation learning (SRERL) framework. Specifically, by analyzing the symbiosis and mutual exclusion of AUs in various facial expressions, we organize the facial AUs in the form of structured knowledge-graph and integrate a Gated Graph Neural Network (GGNN) in a multi-scale CNN framework to propagate node information through the graph for generating enhanced AU representation. As the learned feature involves both the appearance characteristics and the AU relationship reasoning, the proposed model is more robust and can cope with more challenging cases, e.g., illumination change and partial occlusion. Extensive experiments on the two public benchmarks demonstrate that our method outperforms the previous work and achieves state of the art performance.

Fast Image Search with Pixel-Based Deep Learning Framework via Efficient Compact Binary Code and Addictive Latent Layer

2017 ◽  
Vol 32 (03) ◽  
pp. 1859004 ◽  
Author(s):  
Jun Yi Li ◽  
Jian Hua Li
Keyword(s):  
Deep Learning ◽  
Nearest Neighbor ◽  
Feature Learning ◽  
Nearest Neighbor Search ◽  
Feature Representation ◽  
Image Feature ◽  
Retrieval Algorithm ◽  
Image Search ◽  
Learning Framework ◽  
Compact Binary

As we know, the nearest neighbor search is a good and effective method for good-sized image search. This paper mainly introduced how to learn an outstanding image feature representation form and a series of compact binary Hash coding functions under deep learning framework. Our concept is that binary codes can be obtained using a hidden layer to present some latent concepts dominating the class labels with usable data labels. Our method is effective in obtaining hash codes and image representations, so it is suitable for good-sized dataset. It is demonstrated in our experiment that the performances of the proposed algorithms were then verified on three different databases, MNIST, CIFAR-10 and Caltech-101. The experimental results reveal that two-proposed image Hash retrieval algorithm based on pixel-level automatic feature learning show higher search accuracy than the other algorithms; moreover, these two algorithms were proved to be more favorable in scalability and generality.

scholarly journals Adaptive-Weighted Multiview Deep Basis Matrix Factorization for Multimedia Data Analysis

2021 ◽  
Vol 2021 ◽  
pp. 1-12
Author(s):  
Shicheng Li ◽  
Qinghua Liu ◽  
Jiangyan Dai ◽  
Wenle Wang ◽  
Xiaolin Gui ◽  
...  
Keyword(s):  
Data Analysis ◽  
Matrix Factorization ◽  
Feature Learning ◽  
Representation Learning ◽  
Feature Representation ◽  
Multimedia Data ◽  
Basis Matrix ◽  
Rich Information ◽  
Low Dimensional ◽  
Better Than

Feature representation learning is a key issue in artificial intelligence research. Multiview multimedia data can provide rich information, which makes feature representation become one of the current research hotspots in data analysis. Recently, a large number of multiview data feature representation methods have been proposed, among which matrix factorization shows the excellent performance. Therefore, we propose an adaptive-weighted multiview deep basis matrix factorization (AMDBMF) method that integrates matrix factorization, deep learning, and view fusion together. Specifically, we first perform deep basis matrix factorization on data of each view. Then, all views are integrated to complete the procedure of multiview feature learning. Finally, we propose an adaptive weighting strategy to fuse the low-dimensional features of each view so that a unified feature representation can be obtained for multiview multimedia data. We also design an iterative update algorithm to optimize the objective function and justify the convergence of the optimization algorithm through numerical experiments. We conducted clustering experiments on five multiview multimedia datasets and compare the proposed method with several excellent current methods. The experimental results demonstrate that the clustering performance of the proposed method is better than those of the other comparison methods.

scholarly journals Image-embodied Knowledge Representation Learning

2017 ◽  
Author(s):  
Ruobing Xie ◽  
Zhiyuan Liu ◽  
Huanbo Luan ◽  
Maosong Sun
Keyword(s):  
Knowledge Representation ◽  
Visual Information ◽  
Representation Learning ◽  
Learning Model ◽  
Experimental Results ◽  
Knowledge Graph ◽  
Embodied Knowledge ◽  
Knowledge Representations ◽  
Conventional Methods ◽  
Image Representations

Entity images could provide significant visual information for knowledge representation learning. Most conventional methods learn knowledge representations merely from structured triples, ignoring rich visual information extracted from entity images. In this paper, we propose a novel Image-embodied Knowledge Representation Learning model (IKRL), where knowledge representations are learned with both triple facts and images. More specifically, we first construct representations for all images of an entity with a neural image encoder. These image representations are then integrated into an aggregated image-based representation via an attention-based method. We evaluate our IKRL models on knowledge graph completion and triple classification. Experimental results demonstrate that our models outperform all baselines on both tasks, which indicates the significance of visual information for knowledge representations and the capability of our models in learning knowledge representations with images.

scholarly journals Semantics-Aligned Representation Learning for Person Re-Identification

2020 ◽  
Vol 34 (07) ◽  
pp. 11173-11180 ◽  
Author(s):  
Xin Jin ◽  
Cuiling Lan ◽  
Wenjun Zeng ◽  
Guoqiang Wei ◽  
Zhibo Chen
Keyword(s):  
State Of The Art ◽  
Representation Learning ◽  
The State ◽  
Feature Representation ◽  
Texture Image ◽  
Computationally Efficient ◽  
Feature Maps ◽  
Benchmark Datasets ◽  
Texture Generation ◽  
Base Network

Person re-identification (reID) aims to match person images to retrieve the ones with the same identity. This is a challenging task, as the images to be matched are generally semantically misaligned due to the diversity of human poses and capture viewpoints, incompleteness of the visible bodies (due to occlusion), etc. In this paper, we propose a framework that drives the reID network to learn semantics-aligned feature representation through delicate supervision designs. Specifically, we build a Semantics Aligning Network (SAN) which consists of a base network as encoder (SA-Enc) for re-ID, and a decoder (SA-Dec) for reconstructing/regressing the densely semantics aligned full texture image. We jointly train the SAN under the supervisions of person re-identification and aligned texture generation. Moreover, at the decoder, besides the reconstruction loss, we add Triplet ReID constraints over the feature maps as the perceptual losses. The decoder is discarded in the inference and thus our scheme is computationally efficient. Ablation studies demonstrate the effectiveness of our design. We achieve the state-of-the-art performances on the benchmark datasets CUHK03, Market1501, MSMT17, and the partial person reID dataset Partial REID.

scholarly journals A learning-based method for drug-target interaction prediction based on feature representation learning and deep neural network

2020 ◽  
Vol 21 (S13) ◽  
Author(s):  
Jiajie Peng ◽  
Jingyi Li ◽  
Xuequn Shang
Keyword(s):  
Neural Network ◽  
Drug Discovery ◽  
Drug Target ◽  
Deep Neural Network ◽  
Computational Prediction ◽  
Representation Learning ◽  
Feature Representation ◽  
New Drugs ◽  
Target Interaction ◽  
Interaction Prediction

Abstract Background Drug-target interaction prediction is of great significance for narrowing down the scope of candidate medications, and thus is a vital step in drug discovery. Because of the particularity of biochemical experiments, the development of new drugs is not only costly, but also time-consuming. Therefore, the computational prediction of drug target interactions has become an essential way in the process of drug discovery, aiming to greatly reducing the experimental cost and time. Results We propose a learning-based method based on feature representation learning and deep neural network named DTI-CNN to predict the drug-target interactions. We first extract the relevant features of drugs and proteins from heterogeneous networks by using the Jaccard similarity coefficient and restart random walk model. Then, we adopt a denoising autoencoder model to reduce the dimension and identify the essential features. Third, based on the features obtained from last step, we constructed a convolutional neural network model to predict the interaction between drugs and proteins. The evaluation results show that the average AUROC score and AUPR score of DTI-CNN were 0.9416 and 0.9499, which obtains better performance than the other three existing state-of-the-art methods. Conclusions All the experimental results show that the performance of DTI-CNN is better than that of the three existing methods and the proposed method is appropriately designed.

Sign in / Sign up

Export Citation Format

Share Document