scholarly journals On Lifted Inference Using Neural Embeddings

2019 ◽  
Vol 33 ◽  
pp. 7916-7923
Author(s):  
Mohammad Maminur Islam ◽  
Somdeb Sarkhel ◽  
Deepak Venugopal
Keyword(s):  
Neural Networks ◽  
State Of The Art ◽  
Markov Logic Networks ◽  
Markov Logic ◽  
Lifted Inference ◽  
Inference Algorithms ◽  
Art Methods ◽  
Dense Representation

We present a dense representation for Markov Logic Networks (MLNs) called Obj2Vec that encodes symmetries in the MLN structure. Identifying symmetries is a key challenge for lifted inference algorithms and we leverage advances in neural networks to learn symmetries which are hard to specify using hand-crafted features. Specifically, we learn an embedding for MLN objects that predicts the context of an object, i.e., objects that appear along with it in formulas of the MLN, since common contexts indicate symmetry in the distribution. Importantly, our formulation leverages well-known skip-gram models that allow us to learn the embedding efficiently. Finally, to reduce the size of the ground MLN, we sample objects based on their learned embeddings. We integrate Obj2Vec with several inference algorithms, and show the scalability and accuracy of our approach compared to other state-of-the-art methods.

scholarly journals Lifted Message Passing for Hybrid Probabilistic Inference

2019 ◽  
Author(s):  
Yuqiao Chen ◽  
Nicholas Ruozzi ◽  
Sriraam Natarajan
Keyword(s):  
Message Passing ◽  
Computational Cost ◽  
Markov Logic Networks ◽  
Markov Logic ◽  
Lifted Inference ◽  
Particle Sampling ◽  
Variable Elimination ◽  
Inference Algorithms ◽  
Arbitrary Potential ◽  
Inference Methods

Lifted inference algorithms for first-order logic models, e.g., Markov logic networks (MLNs), have been of significant interest in recent years.  Lifted inference methods exploit model symmetries in order to reduce the size of the model and, consequently, the computational cost of inference.  In this work, we consider the problem of lifted inference in MLNs with continuous or both discrete and continuous groundings. Existing work on lifting with continuous groundings has mostly been limited to special classes of models, e.g., Gaussian models, for which variable elimination or message-passing updates can be computed exactly.  Here, we develop approximate lifted inference schemes based on particle sampling.  We demonstrate empirically that our approximate lifting schemes perform comparably to existing state-of-the-art for models for Gaussian MLNs, while having the flexibility to be applied to models with arbitrary potential functions.

scholarly journals CapsTM: capsule network for Chinese medical text matching

2021 ◽  
Vol 21 (S2) ◽  
Author(s):  
Xiaoming Yu ◽  
Yedan Shen ◽  
Yuan Ni ◽  
Xiaowei Huang ◽  
Xiaolong Wang ◽  
...  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Deep Learning ◽  
Network Architecture ◽  
State Of The Art ◽  
Interaction Matrix ◽  
Application Systems ◽  
Input Layer ◽  
Art Methods ◽  
Text Matching

Abstract Background Text Matching (TM) is a fundamental task of natural language processing widely used in many application systems such as information retrieval, automatic question answering, machine translation, dialogue system, reading comprehension, etc. In recent years, a large number of deep learning neural networks have been applied to TM, and have refreshed benchmarks of TM repeatedly. Among the deep learning neural networks, convolutional neural network (CNN) is one of the most popular networks, which suffers from difficulties in dealing with small samples and keeping relative structures of features. In this paper, we propose a novel deep learning architecture based on capsule network for TM, called CapsTM, where capsule network is a new type of neural network architecture proposed to address some of the short comings of CNN and shows great potential in many tasks. Methods CapsTM is a five-layer neural network, including an input layer, a representation layer, an aggregation layer, a capsule layer and a prediction layer. In CapsTM, two pieces of text are first individually converted into sequences of embeddings and are further transformed by a highway network in the input layer. Then, Bidirectional Long Short-Term Memory (BiLSTM) is used to represent each piece of text and attention-based interaction matrix is used to represent interactive information of the two pieces of text in the representation layer. Subsequently, the two kinds of representations are fused together by BiLSTM in the aggregation layer, and are further represented with capsules (vectors) in the capsule layer. Finally, the prediction layer is a connected network used for classification. CapsTM is an extension of ESIM by adding a capsule layer before the prediction layer. Results We construct a corpus of Chinese medical question matching, which contains 36,360 question pairs. This corpus is randomly split into three parts: a training set of 32,360 question pairs, a development set of 2000 question pairs and a test set of 2000 question pairs. On this corpus, we conduct a series of experiments to evaluate the proposed CapsTM and compare it with other state-of-the-art methods. CapsTM achieves the highest F-score of 0.8666. Conclusion The experimental results demonstrate that CapsTM is effective for Chinese medical question matching and outperforms other state-of-the-art methods for comparison.

scholarly journals Efficient Inference for Untied MLNs

2017 ◽  
Author(s):  
Somdeb Sarkhel ◽  
Deepak Venugopal ◽  
Nicholas Ruozzi ◽  
Vibhav Gogate
Keyword(s):  
Graphical Models ◽  
Scaling Up ◽  
Partition Functions ◽  
Markov Logic ◽  
Practical Applications ◽  
Lifted Inference ◽  
Hard Time ◽  
Inference Algorithms ◽  
Order Of Magnitude ◽  
Symmetric Approximation

We address the problem of scaling up local-search or sampling-based inference in Markov logic networks (MLNs) that have large shared sub-structures but no (or few) tied weights. Such untied MLNs are ubiquitous in practical applications. However, they have very few symmetries, and as a result lifted inference algorithms--the dominant approach for scaling up inference--perform poorly on them. The key idea in our approach is to reduce the hard, time-consuming sub-task in sampling algorithms, computing the sum of weights of features that satisfy a full assignment, to the problem of computing a set of partition functions of graphical models, each defined over the logical variables in a first-order formula. The importance of this reduction is that when the treewidth of all the graphical models is small, it yields an order of magnitude speedup. When the treewidth is large, we propose an over-symmetric approximation and experimentally demonstrate that it is both fast and accurate.

scholarly journals DEEPCON: protein contact prediction using dilated convolutional neural networks with dropout

2019 ◽  
Vol 36 (2) ◽  
pp. 470-477 ◽  
Author(s):  
Badri Adhikari
Keyword(s):  
Neural Networks ◽  
Long Range ◽  
Convolutional Neural Networks ◽  
High Throughput Sequencing ◽  
State Of The Art ◽  
Deep Convolutional Neural Networks ◽  
Contact Prediction ◽  
Learning Techniques ◽  
Medium Range ◽  
Art Methods

Abstract Motivation Exciting new opportunities have arisen to solve the protein contact prediction problem from the progress in neural networks and the availability of a large number of homologous sequences through high-throughput sequencing. In this work, we study how deep convolutional neural networks (ConvNets) may be best designed and developed to solve this long-standing problem. Results With publicly available datasets, we designed and trained various ConvNet architectures. We tested several recent deep learning techniques including wide residual networks, dropouts and dilated convolutions. We studied the improvements in the precision of medium-range and long-range contacts, and compared the performance of our best architectures with the ones used in existing state-of-the-art methods. The proposed ConvNet architectures predict contacts with significantly more precision than the architectures used in several state-of-the-art methods. When trained using the DeepCov dataset consisting of 3456 proteins and tested on PSICOV dataset of 150 proteins, our architectures achieve up to 15% higher precision when L/2 long-range contacts are evaluated. Similarly, when trained using the DNCON2 dataset consisting of 1426 proteins and tested on 84 protein domains in the CASP12 dataset, our single network achieves 4.8% higher precision than the ensembled DNCON2 method when top L long-range contacts are evaluated. Availability and implementation DEEPCON is available at https://github.com/badriadhikari/DEEPCON/.

scholarly journals ECG Denoising Using Artificial Neural Networks and Complete Ensemble Empirical Mode Decomposition

2021 ◽  
Vol 12 (2) ◽  
pp. 2382-2389
Author(s):  
Rajesh Birok, Et. al.
Keyword(s):  
Neural Networks ◽  
Empirical Mode Decomposition ◽  
State Of The Art ◽  
Signal To Noise Ratio ◽  
Low Frequency ◽  
Ensemble Empirical Mode Decomposition ◽  
Novel Approach ◽  
Mode Decomposition ◽  
The Neural Network ◽  
Art Methods

Electrocardiogram (ECG) is a documentation of the electrical activities of the heart. It is used to identify a number of cardiac faults such as arrhythmias, AF etc.  Quite often the ECG gets corrupted by various kinds of artifacts, thus in order to gain correct information from them, they must first be denoised. This paper presents a novel approach for the filtering of low frequency artifacts of ECG signals by using Complete Ensemble Empirical Mode Decomposition (CEED) and Neural Networks, which removes most of the constituent noise while assuring no loss of information in terms of the morphology of the ECG signal. The contribution of the method lies in the fact that it combines the advantages of both EEMD and ANN. The use of CEEMD ensures that the Neural Network does not get over fitted. It also significantly helps in building better predictors at individual frequency levels. The proposed method is compared with other state-of-the-art methods in terms of Mean Square Error (MSE), Signal to Noise Ratio (SNR) and Correlation Coefficient. The results show that the proposed method has better performance as compared to other state-of-the-art methods for low frequency artifacts removal from EEG.  

scholarly journals A New Dataset and Deep Residual Spectral Spatial Network for Hyperspectral Image Classification

Symmetry ◽  
2020 ◽  
Vol 12 (4) ◽  
pp. 561 ◽  
Author(s):  
Yiming Xue ◽  
Dan Zeng ◽  
Fansheng Chen ◽  
Yueming Wang ◽  
Zhijiang Zhang
Keyword(s):  
Neural Networks ◽  
Loss Function ◽  
Hyperspectral Image ◽  
State Of The Art ◽  
Feature Learning ◽  
Spatial Network ◽  
Hyperspectral Image Classification ◽  
Classification Framework ◽  
Prediction Confidence ◽  
Art Methods

Due to the limited varieties and sizes of existing public hyperspectral image (HSI) datasets, the classification accuracies are higher than 99% with convolutional neural networks (CNNs). In this paper, we presented a new HSI dataset named Shandong Feicheng, whose size and pixel quantity are much larger. It also has a larger intra-class variance and a smaller inter-class variance. State-of-the-art methods were compared on it to verify its diversity. Otherwise, to reduce overfitting caused by the imbalance between high dimension and small quantity of labeled HSI data, existing CNNs for HSI classification are relatively shallow and suffer from low capacity of feature learning. To solve this problem, we proposed an HSI classification framework named deep residual spectral spatial setwork (DRSSN). By using shortcut connection structure, which is an asymmetry structure, DRSSN can be deeper to extract features with better discrimination. In addition, to alleviate insufficient training caused by unbalanced sample sizes between easily and hard classified samples, we proposed a novel training loss function named sample balanced loss, which allocated weights to the losses of samples according to their prediction confidence. Experimental results on two popular datasets and our proposed dataset showed that our proposed network could provide competitive results compared with state-of-the-art methods.

scholarly journals iEnhancer-ECNN: identifying enhancers and their strength using ensembles of convolutional neural networks

BMC Genomics ◽  
2019 ◽  
Vol 20 (S9) ◽  
Author(s):  
Quang H. Nguyen ◽  
Thanh-Hoang Nguyen-Vo ◽  
Nguyen Quoc Khanh Le ◽  
Trang T.T. Do ◽  
Susanto Rahardja ◽  
...  
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Dna Sequences ◽  
State Of The Art ◽  
Characteristic Curve ◽  
Learning Models ◽  
In Silico Studies ◽  
Layer 2 ◽  
Art Methods ◽  
Enhancer Identification

Abstract Background Enhancers are non-coding DNA fragments which are crucial in gene regulation (e.g. transcription and translation). Having high locational variation and free scattering in 98% of non-encoding genomes, enhancer identification is, therefore, more complicated than other genetic factors. To address this biological issue, several in silico studies have been done to identify and classify enhancer sequences among a myriad of DNA sequences using computational advances. Although recent studies have come up with improved performance, shortfalls in these learning models still remain. To overcome limitations of existing learning models, we introduce iEnhancer-ECNN, an efficient prediction framework using one-hot encoding and k-mers for data transformation and ensembles of convolutional neural networks for model construction, to identify enhancers and classify their strength. The benchmark dataset from Liu et al.’s study was used to develop and evaluate the ensemble models. A comparative analysis between iEnhancer-ECNN and existing state-of-the-art methods was done to fairly assess the model performance. Results Our experimental results demonstrates that iEnhancer-ECNN has better performance compared to other state-of-the-art methods using the same dataset. The accuracy of the ensemble model for enhancer identification (layer 1) and enhancer classification (layer 2) are 0.769 and 0.678, respectively. Compared to other related studies, improvements in the Area Under the Receiver Operating Characteristic Curve (AUC), sensitivity, and Matthews’s correlation coefficient (MCC) of our models are remarkable, especially for the model of layer 2 with about 11.0%, 46.5%, and 65.0%, respectively. Conclusions iEnhancer-ECNN outperforms other previously proposed methods with significant improvement in most of the evaluation metrics. Strong growths in the MCC of both layers are highly meaningful in assuring the stability of our models.

scholarly journals Induction of Interpretable Possibilistic Logic Theories from Relational Data

2017 ◽  
Author(s):  
Ondrej Kuzelka ◽  
Jesse Davis ◽  
Steven Schockaert
Keyword(s):  
Neural Networks ◽  
Probabilistic Models ◽  
Relational Learning ◽  
Statistical Relational Learning ◽  
Relational Data ◽  
Markov Logic Networks ◽  
Relational Models ◽  
Markov Logic ◽  
Possibilistic Logic ◽  
Interpretable Models

The field of statistical relational learning (SRL) is concerned with learning probabilistic models from relational data. Learned SRL models are typically represented using some kind of weighted logical formulas, which makes them considerably more interpretable than those obtained by e.g. neural networks. In practice, however, these models are often still difficult to interpret correctly, as they can contain many formulas that interact in non-trivial ways and weights do not always have an intuitive meaning. To address this, we propose a new SRL method which uses possibilistic logic to encode relational models. Learned models are then essentially stratified classical theories, which explicitly encode what can be derived with a given level of certainty. Compared to Markov Logic Networks (MLNs), our method is faster and produces considerably more interpretable models.

scholarly journals BCNet: A Novel Network for Blood Cell Classification

2022 ◽  
Vol 9 ◽  
Author(s):  
Ziquan Zhu ◽  
Siyuan Lu ◽  
Shui-Hua Wang ◽  
Juan Manuel Górriz ◽  
Yu-Dong Zhang
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Blood Cell ◽  
State Of The Art ◽  
Neural Model ◽  
Convolution Neural Network ◽  
Majority Voting ◽  
Cell Classification ◽  
Art Methods ◽  
Deep Convolution Neural Network

Aims: Most blood diseases, such as chronic anemia, leukemia (commonly known as blood cancer), and hematopoietic dysfunction, are caused by environmental pollution, substandard decoration materials, radiation exposure, and long-term use certain drugs. Thus, it is imperative to classify the blood cell images. Most cell classification is based on the manual feature, machine learning classifier or the deep convolution network neural model. However, manual feature extraction is a very tedious process, and the results are usually unsatisfactory. On the other hand, the deep convolution neural network is usually composed of massive layers, and each layer has many parameters. Therefore, each deep convolution neural network needs a lot of time to get the results. Another problem is that medical data sets are relatively small, which may lead to overfitting problems.Methods: To address these problems, we propose seven models for the automatic classification of blood cells: BCARENet, BCR5RENet, BCMV2RENet, BCRRNet, BCRENet, BCRSNet, and BCNet. The BCNet model is the best model among the seven proposed models. The backbone model in our method is selected as the ResNet-18, which is pre-trained on the ImageNet set. To improve the performance of the proposed model, we replace the last four layers of the trained transferred ResNet-18 model with the three randomized neural networks (RNNs), which are RVFL, ELM, and SNN. The final outputs of our BCNet are generated by the ensemble of the predictions from the three randomized neural networks by the majority voting. We use four multi-classification indexes for the evaluation of our model.Results: The accuracy, average precision, average F1-score, and average recall are 96.78, 97.07, 96.78, and 96.77%, respectively.Conclusion: We offer the comparison of our model with state-of-the-art methods. The results of the proposed BCNet model are much better than other state-of-the-art methods.

scholarly journals DeepCapTail: A Deep Learning Framework to Predict Capsid and Tail Proteins of Phage Genomes

2018 ◽  
Author(s):  
Dhoha Abid ◽  
Liqing Zhang
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Deep Neural Networks ◽  
State Of The Art ◽  
Sequence Conservation ◽  
Structural Proteins ◽  
Genomic Sequences ◽  
Learning Framework ◽  
Art Methods ◽  
Tail Proteins

AbstractThe capsid and tail proteins are considered the main structural proteins for phages and also their footprint since they exist only in phage genomes. These proteins are known to lack sequence conservation, making them extremely diverse and thus posing a major challenge to identify and annotate them in genomic sequences. In this study, we aim to overcome this challenge and predict these proteins by using deep neural networks with composition-based features. We develop two models trained with k-mer features to predict capsid and tail proteins respectively. Evaluating the models on two different testing sets shows that they outperform state-of-the-art methods with improved F-1 scores.

Sign in / Sign up

Export Citation Format

Share Document