scholarly journals Neural Snowball for Few-Shot Relation Learning

2020 ◽  
Vol 34 (05) ◽  
pp. 7772-7779 ◽  
Author(s):  
Tianyu Gao ◽  
Xu Han ◽  
Ruobing Xie ◽  
Zhiyuan Liu ◽  
Fen Lin ◽  
...  
Keyword(s):  
Relation Extraction ◽  
Semantic Knowledge ◽  
Training Data ◽  
High Quality ◽  
Knowledge Graphs ◽  
Relation Learning ◽  
Siamese Networks

Knowledge graphs typically undergo open-ended growth of new relations. This cannot be well handled by relation extraction that focuses on pre-defined relations with sufficient training data. To address new relations with few-shot instances, we propose a novel bootstrapping approach, Neural Snowball, to learn new relations by transferring semantic knowledge about existing relations. More specifically, we use Relational Siamese Networks (RSN) to learn the metric of relational similarities between instances based on existing relations and their labeled data. Afterwards, given a new relation and its few-shot instances, we use RSN to accumulate reliable instances from unlabeled corpora; these instances are used to train a relation classifier, which can further identify new facts of the new relation. The process is conducted iteratively like a snowball. Experiments show that our model can gather high-quality instances for better few-shot relation learning and achieves significant improvement compared to baselines. Codes and datasets are released on https://github.com/thunlp/Neural-Snowball.

scholarly journals A Co-Embedding Model with Variational Auto-Encoder for Knowledge Graphs

2022 ◽  
Vol 12 (2) ◽  
pp. 715
Author(s):  
Luodi Xie ◽  
Huimin Huang ◽  
Qing Du
Keyword(s):  
State Of The Art ◽  
Relation Extraction ◽  
Semantic Space ◽  
Knowledge Graph ◽  
High Quality ◽  
Gaussian Distributions ◽  
Benchmark Datasets ◽  
Semantic Spaces ◽  
Knowledge Graphs ◽  
Low Dimensional

Knowledge graph (KG) embedding has been widely studied to obtain low-dimensional representations for entities and relations. It serves as the basis for downstream tasks, such as KG completion and relation extraction. Traditional KG embedding techniques usually represent entities/relations as vectors or tensors, mapping them in different semantic spaces and ignoring the uncertainties. The affinities between entities and relations are ambiguous when they are not embedded in the same latent spaces. In this paper, we incorporate a co-embedding model for KG embedding, which learns low-dimensional representations of both entities and relations in the same semantic space. To address the issue of neglecting uncertainty for KG components, we propose a variational auto-encoder that represents KG components as Gaussian distributions. In addition, compared with previous methods, our method has the advantages of high quality and interpretability. Our experimental results on several benchmark datasets demonstrate our model’s superiority over the state-of-the-art baselines.

One-Shot Relation Learning for Knowledge Graphs via Neighborhood Aggregation and Paths Encoding

2022 ◽  
Vol 21 (3) ◽  
pp. 1-19
Author(s):  
Jian Sun ◽  
Yu Zhou ◽  
Chengqing Zong
Keyword(s):  
Link Prediction ◽  
Auxiliary Information ◽  
Training Data ◽  
Knowledge Graph ◽  
Long Tail ◽  
Neighborhood Information ◽  
Knowledge Graphs ◽  
Relation Prediction ◽  
Relation Learning ◽  
The Impact

The relation learning between two entities is an essential task in knowledge graph (KG) completion that has received much attention recently. Previous work almost exclusively focused on relations widely seen in the original KGs, which means that enough training data are available for modeling. However, long-tail relations that only show in a few triples are actually much more common in practical KGs. Without sufficiently large training data, the performance of existing models on predicting long-tail relations drops impressively. This work aims to predict the relation under a challenging setting where only one instance is available for training. We propose a path-based one-shot relation prediction framework, which can extract neighborhood information of an entity based on the relation query attention mechanism to learn transferable knowledge among the same relation. Simultaneously, to reduce the impact of long-tail entities on relation prediction, we selectively fuse path information between entity pairs as auxiliary information of relation features. Experiments in three one-shot relation learning datasets show that our proposed framework substantially outperforms existing models on one-shot link prediction and relation prediction.

scholarly journals Experiments of Image Classification Using Dissimilarity Spaces Built with Siamese Networks

Sensors ◽  
2021 ◽  
Vol 21 (5) ◽  
pp. 1573
Author(s):  
Loris Nanni ◽  
Giovanni Minchio ◽  
Sheryl Brahnam ◽  
Gianluca Maguolo ◽  
Alessandra Lumini
Keyword(s):  
Vector Space ◽  
Image Classification ◽  
Ad Hoc ◽  
Feature Space ◽  
Medical Data ◽  
Training Data ◽  
Data Sets ◽  
Large Set ◽  
Clustering Methods ◽  
Siamese Networks

Traditionally, classifiers are trained to predict patterns within a feature space. The image classification system presented here trains classifiers to predict patterns within a vector space by combining the dissimilarity spaces generated by a large set of Siamese Neural Networks (SNNs). A set of centroids from the patterns in the training data sets is calculated with supervised k-means clustering. The centroids are used to generate the dissimilarity space via the Siamese networks. The vector space descriptors are extracted by projecting patterns onto the similarity spaces, and SVMs classify an image by its dissimilarity vector. The versatility of the proposed approach in image classification is demonstrated by evaluating the system on different types of images across two domains: two medical data sets and two animal audio data sets with vocalizations represented as images (spectrograms). Results show that the proposed system’s performance competes competitively against the best-performing methods in the literature, obtaining state-of-the-art performance on one of the medical data sets, and does so without ad-hoc optimization of the clustering methods on the tested data sets.

57 Precision neoantigen discovery using novel algorithms and expanded HLA-ligandome datasets

2020 ◽  
Vol 8 (Suppl 3) ◽  
pp. A62-A62
Author(s):  
Dattatreya Mellacheruvu ◽  
Rachel Pyke ◽  
Charles Abbott ◽  
Nick Phillips ◽  
Sejal Desai ◽  
...  
Keyword(s):  
Machine Learning ◽  
Cell Lines ◽  
Antigen Processing ◽  
Large Scale ◽  
Prediction Models ◽  
K562 Cells ◽  
Machine Learning Algorithms ◽  
Training Data ◽  
High Quality ◽  
Tissue Samples

BackgroundAccurately identified neoantigens can be effective therapeutic agents in both adjuvant and neoadjuvant settings. A key challenge for neoantigen discovery has been the availability of accurate prediction models for MHC peptide presentation. We have shown previously that our proprietary model based on (i) large-scale, in-house mono-allelic data, (ii) custom features that model antigen processing, and (iii) advanced machine learning algorithms has strong performance. We have extended upon our work by systematically integrating large quantities of high-quality, publicly available data, implementing new modelling algorithms, and rigorously testing our models. These extensions lead to substantial improvements in performance and generalizability. Our algorithm, named Systematic HLA Epitope Ranking Pan Algorithm (SHERPA™), is integrated into the ImmunoID NeXT Platform®, our immuno-genomics and transcriptomics platform specifically designed to enable the development of immunotherapies.MethodsIn-house immunopeptidomic data was generated using stably transfected HLA-null K562 cells lines that express a single HLA allele of interest, followed by immunoprecipitation using W6/32 antibody and LC-MS/MS. Public immunopeptidomics data was downloaded from repositories such as MassIVE and processed uniformly using in-house pipelines to generate peptide lists filtered at 1% false discovery rate. Other metrics (features) were either extracted from source data or generated internally by re-processing samples utilizing the ImmunoID NeXT Platform.ResultsWe have generated large-scale and high-quality immunopeptidomics data by using approximately 60 mono-allelic cell lines that unambiguously assign peptides to their presenting alleles to create our primary models. Briefly, our primary ‘binding’ algorithm models MHC-peptide binding using peptide and binding pockets while our primary ‘presentation’ model uses additional features to model antigen processing and presentation. Both primary models have significantly higher precision across all recall values in multiple test data sets, including mono-allelic cell lines and multi-allelic tissue samples. To further improve the performance of our model, we expanded the diversity of our training set using high-quality, publicly available mono-allelic immunopeptidomics data. Furthermore, multi-allelic data was integrated by resolving peptide-to-allele mappings using our primary models. We then trained a new model using the expanded training data and a new composite machine learning architecture. The resulting secondary model further improves performance and generalizability across several tissue samples.ConclusionsImproving technologies for neoantigen discovery is critical for many therapeutic applications, including personalized neoantigen vaccines, and neoantigen-based biomarkers for immunotherapies. Our new and improved algorithm (SHERPA) has significantly higher performance compared to a state-of-the-art public algorithm and furthers this objective.

scholarly journals Populating Web-Scale Knowledge Graphs Using Distantly Supervised Relation Extraction and Validation

Information ◽  
2021 ◽  
Vol 12 (8) ◽  
pp. 316
Author(s):  
Sarthak Dash ◽  
Michael R. Glass ◽  
Alfio Gliozzo ◽  
Mustafa Canim ◽  
Gaetano Rossiello
Keyword(s):  
Deep Learning ◽  
Relation Extraction ◽  
Automated System ◽  
External Information ◽  
Learning Approach ◽  
Wide Margin ◽  
Structure Information ◽  
Adaptation Cost ◽  
Knowledge Graphs ◽  
Scale Experiment

In this paper, we propose a fully automated system to extend knowledge graphs using external information from web-scale corpora. The designed system leverages a deep-learning-based technology for relation extraction that can be trained by a distantly supervised approach. In addition, the system uses a deep learning approach for knowledge base completion by utilizing the global structure information of the induced KG to further refine the confidence of the newly discovered relations. The designed system does not require any effort for adaptation to new languages and domains as it does not use any hand-labeled data, NLP analytics, and inference rules. Our experiments, performed on a popular academic benchmark, demonstrate that the suggested system boosts the performance of relation extraction by a wide margin, reporting error reductions of 50%, resulting in relative improvement of up to 100%. Furthermore, a web-scale experiment conducted to extend DBPedia with knowledge from Common Crawl shows that our system is not only scalable but also does not require any adaptation cost, while yielding a substantial accuracy gain.

Optimal Scale of Hierarchical Image Segmentation with Scribbles Guidance for Weakly Supervised Semantic Segmentation

2021 ◽  
pp. 2154026
Author(s):  
Zaid Al-Huda ◽  
Donghai Zhai ◽  
Yan Yang ◽  
Riyadh Nazar Ali Algburi
Keyword(s):  
Image Segmentation ◽  
Graphical Model ◽  
Semantic Segmentation ◽  
Saliency Map ◽  
Training Data ◽  
Deep Convolutional Neural Networks ◽  
High Quality ◽  
Optimal Scale ◽  
Supervised Segmentation ◽  
Weakly Supervised

Deep convolutional neural networks (DCNNs) trained on the pixel-level annotated images have achieved improvements in semantic segmentation. Due to the high cost of labeling training data, their applications may have great limitation. However, weakly supervised segmentation approaches can significantly reduce human labeling efforts. In this paper, we introduce a new framework to generate high-quality initial pixel-level annotations. By using a hierarchical image segmentation algorithm to predict the boundary map, we select the optimal scale of high-quality hierarchies. In the initialization step, scribble annotations and the saliency map are combined to construct a graphic model over the optimal scale segmentation. By solving the minimal cut problem, it can spread information from scribbles to unmarked regions. In the training process, the segmentation network is trained by using the initial pixel-level annotations. To iteratively optimize the segmentation, we use a graphical model to refine segmentation masks and retrain the segmentation network to get more precise pixel-level annotations. The experimental results on Pascal VOC 2012 dataset demonstrate that the proposed framework outperforms most of weakly supervised semantic segmentation methods and achieves the state-of-the-art performance, which is [Formula: see text] mIoU.

scholarly journals Using diffusion MRI data acquired with ultra-high gradients to improve tractography in routine-quality data

2021 ◽  
Author(s):  
Chiara Maffei ◽  
Christine Lee ◽  
Michael Planich ◽  
Manisha Ramprasad ◽  
Nivedita Ravi ◽  
...  
Keyword(s):  
Diffusion Mri ◽  
Region Of Interest ◽  
Training Data ◽  
Quality Data ◽  
High Quality ◽  
Scanning Time ◽  
Human Connectome Project ◽  
High Gradients ◽  
Comprehensive Comparison

The development of scanners with ultra-high gradients, spearheaded by the Human Connectome Project, has led to dramatic improvements in the spatial, angular, and diffusion resolution that is feasible for in vivo diffusion MRI acquisitions. The improved quality of the data can be exploited to achieve higher accuracy in the inference of both microstructural and macrostructural anatomy. However, such high-quality data can only be acquired on a handful of Connectom MRI scanners worldwide, while remaining prohibitive in clinical settings because of the constraints imposed by hardware and scanning time. In this study, we first update the classical protocols for tractography-based, manual annotation of major white-matter pathways, to adapt them to the much greater volume and variability of the streamlines that can be produced from today's state-of-the-art diffusion MRI data. We then use these protocols to annotate 42 major pathways manually in data from a Connectom scanner. Finally, we show that, when we use these manually annotated pathways as training data for global probabilistic tractography with anatomical neighborhood priors, we can perform highly accurate, automated reconstruction of the same pathways in much lower-quality, more widely available diffusion MRI data. The outcomes of this work include both a new, comprehensive atlas of WM pathways from Connectom data, and an updated version of our tractography toolbox, TRActs Constrained by UnderLying Anatomy (TRACULA), which is trained on data from this atlas. Both the atlas and TRACULA are distributed publicly as part of FreeSurfer. We present the first comprehensive comparison of TRACULA to the more conventional, multi-region-of-interest approach to automated tractography, and the first demonstration of training TRACULA on high-quality, Connectom data to benefit studies that use more modest acquisition protocols.

scholarly journals Attending to Entities for Better Text Understanding

2020 ◽  
Vol 34 (05) ◽  
pp. 7554-7561
Author(s):  
Pengxiang Cheng ◽  
Katrin Erk
Keyword(s):  
Large Scale ◽  
Human Performance ◽  
State Of The Art ◽  
Syntactic Structure ◽  
Semantic Knowledge ◽  
Training Data ◽  
Language Models ◽  
Long Distance ◽  
Future Directions ◽  
Text Understanding

Recent progress in NLP witnessed the development of large-scale pre-trained language models (GPT, BERT, XLNet, etc.) based on Transformer (Vaswani et al. 2017), and in a range of end tasks, such models have achieved state-of-the-art results, approaching human performance. This clearly demonstrates the power of the stacked self-attention architecture when paired with a sufficient number of layers and a large amount of pre-training data. However, on tasks that require complex and long-distance reasoning where surface-level cues are not enough, there is still a large gap between the pre-trained models and human performance. Strubell et al. (2018) recently showed that it is possible to inject knowledge of syntactic structure into a model through supervised self-attention. We conjecture that a similar injection of semantic knowledge, in particular, coreference information, into an existing model would improve performance on such complex problems. On the LAMBADA (Paperno et al. 2016) task, we show that a model trained from scratch with coreference as auxiliary supervision for self-attention outperforms the largest GPT-2 model, setting the new state-of-the-art, while only containing a tiny fraction of parameters compared to GPT-2. We also conduct a thorough analysis of different variants of model architectures and supervision configurations, suggesting future directions on applying similar techniques to other problems.

scholarly journals KG-E: Um Grafo de Conhecimento Semântico Baseado na Integração de Dados de Empresas e Sancionados

2021 ◽  
Author(s):  
Tulio Vidal Rolim ◽  
Caio Viktor S. Avila ◽  
Narciso Moura A. Junior ◽  
Francisca Jamires Costa ◽  
Roberval Gomes Mariano ◽  
...  
Keyword(s):  
A Priori ◽  
Semantic Knowledge ◽  
Knowledge Graphs

A integração de dados permite a descoberta de informações que à priori não eram possíveis em fontes distintas e isoladas. Para tanto, a utilização de Grafos de Conhecimento Semântico (Semantic Knowledge Graphs) fornece uma visão homogênea a partir de fontes heterogêneas integradas semânticamente. Este trabalho descreve o processo de construção e validação de KG-E, um Grafo de Conhecimento Semântico contendo dados de empresas cadastradas na Rede Nacional para a Simplificação do Registro e da Legalização de Empresas e Negócios (REDESIM) e de Sancionados a partir do Cadastro de Empresas Inidôneas e Suspensas (CEIS). KGE-E foi validado através de um estudo de caso com base em questões de competência, apresentando satisfatibilidade quanto às questões relativas ao domínio. Por fim, os resultados sugerem que KG-E demonstrou ser uma visão integrada e enriquecida semanticamente das bases do REDESIM e CEIS, facilitando a construção de aplicações para análises futuras sobre os dados integrados, permitindo desta forma novos insights no âmbito fiscal-empresarial.

Sign in / Sign up

Export Citation Format

Share Document