Attention-Based Graph Convolutional Network for Zero-Shot Learning with Pre-Training

Zero-shot learning (ZSL) is a powerful and promising learning paradigm for classifying instances that have not been seen in training. Although graph convolutional networks (GCNs) have recently shown great potential for the ZSL tasks, these models cannot adjust the constant connection weights between the nodes in knowledge graph and the neighbor nodes contribute equally to classify the central node. In this study, we apply an attention mechanism to adjust the connection weights adaptively to learn more important information for classifying unseen target nodes. First, we propose an attention graph convolutional network for zero-shot learning (AGCNZ) by integrating the attention mechanism and GCN directly. Then, in order to prevent the dilution of knowledge from distant nodes, we apply the dense graph propagation (DGP) model for the ZSL tasks and propose an attention dense graph propagation model for zero-shot learning (ADGPZ). Finally, we propose a modified loss function with a relaxation factor to further improve the performance of the learned classifier. Experimental results under different pre-training settings verified the effectiveness of the proposed attention-based models for ZSL.

Lexical Sense Labeling and Sentiment Potential Analysis Using Corpus-Based Dependency Graph

This paper describes a graph method for labeling word senses and identifying lexical sentiment potential by integrating the corpus-based syntactic-semantic dependency graph layer, lexical semantic and sentiment dictionaries. The method, implemented as ConGraCNet application on different languages and corpora, projects a semantic function onto a particular syntactical dependency layer and constructs a seed lexeme graph with collocates of high conceptual similarity. The seed lexeme graph is clustered into subgraphs that reveal the polysemous semantic nature of a lexeme in a corpus. The construction of the WordNet hypernym graph provides a set of synset labels that generalize the senses for each lexical cluster. By integrating sentiment dictionaries, we introduce graph propagation methods for sentiment analysis. Original dictionary sentiment values are integrated into ConGraCNet lexical graph to compute sentiment values of node lexemes and lexical clusters, and identify the sentiment potential of lexemes with respect to a corpus. The method can be used to resolve sparseness of sentiment dictionaries and enrich the sentiment evaluation of lexical structures in sentiment dictionaries by revealing the relative sentiment potential of polysemous lexemes with respect to a specific corpus. The proposed approach has the potential to be used as a complementary method to other NLP resources and tasks, including word disambiguation, domain relatedness, sense structure, metaphoricity, as well as a cross- and intra-cultural discourse variations of prototypical conceptualization patterns and knowledge representations.

CosG: A Graph-Based Contrastive Learning Method for Fact Verification

Fact verification aims to verify the authenticity of a given claim based on the retrieved evidence from Wikipedia articles. Existing works mainly focus on enhancing the semantic representation of evidence, e.g., introducing the graph structure to model the evidence relation. However, previous methods can’t well distinguish semantic-similar claims and evidences with distinct authenticity labels. In addition, the performances of graph-based models are limited by the over-smoothing problem of graph neural networks. To this end, we propose a graph-based contrastive learning method for fact verification abbreviated as CosG, which introduces a contrastive label-supervised task to help the encoder learn the discriminative representations for different-label claim-evidence pairs, as well as an unsupervised graph-contrast task, to alleviate the unique node features loss in the graph propagation. We conduct experiments on FEVER, a large benchmark dataset for fact verification. Experimental results show the superiority of our proposal against comparable baselines, especially for the claims that need multiple-evidences to verify. In addition, CosG presents better model robustness on the low-resource scenario.

Connectome-based individual prediction of cognitive behaviors via the graph propagation network reveals directed brain network topology

The brain connectome supports the information flow underlying human cognitions and should reflect the individual variability in human cognitive behaviors. Various studies have utilized the brain connectome to predict individual differences in human behaviors. However, traditional studies viewed the brain connectome feature as a vector of one dimension, a method which neglects topological structures of the brain connectome. To utilize topological properties of the brain connectome, we proposed that graph neural network which combines graph theory and neural network can be adopted. Different from previous node-driven graph neural networks that parameterize on the node feature transformation, we designed an edge-driven graph neural network named graph propagation network that parameterizes on the information propagation within the brain connectome. We compared various models in predicting the individual total cognition based on the resting-state functional connectome. The edge-driven graph propagation network showed the highest prediction accuracy and outperformed the node-driven graph neural network and traditional partial least square regression. The graph propagation network also revealed a directed network topology encoding the information flow, indicating that the high-level association cortices are responsible for the information integration underlying the total cognition. These results suggest that the edge-driven graph propagation network can better explore the topological structure of the brain connectome and can serve as a new method to associate the brain connectome and human behaviors.