A novel approach using mutual information for critical nodes detecting in virtual networking environment

Machine Reading Comprehension (MRC) with multiplechoice questions requires the machine to read given passage and select the correct answer among several candidates. In this paper, we propose a novel approach called Convolutional Spatial Attention (CSA) model which can better handle the MRC with multiple-choice questions. The proposed model could fully extract the mutual information among the passage, question, and the candidates, to form the enriched representations. Furthermore, to merge various attention results, we propose to use convolutional operation to dynamically summarize the attention values within the different size of regions. Experimental results show that the proposed model could give substantial improvements over various state-of- the-art systems on both RACE and SemEval-2018 Task11 datasets.

Download Full-text

‘The ZNSL Network’: A Novel Approach to Virtual Networking

2019 International Symposium on Networks, Computers and Communications (ISNCC) ◽

10.1109/isncc.2019.8909127 ◽

2019 ◽

Author(s):

Erik Joseph Seidel ◽

Sonajie Wei

Keyword(s):

Novel Approach ◽

Virtual Networking

Download Full-text

The Image Annotation Refinement in Embedding Feature Space based on Mutual Information

International Journal of Circuits, Systems and Signal Processing ◽

10.46300/9106.2022.16.23 ◽

2022 ◽

Vol 16 ◽

pp. 191-201

Author(s):

Wei Li ◽

Haiyu Song ◽

Hongda Zhang ◽

Houjie Li ◽

Pengjie Wang

Keyword(s):

Mutual Information ◽

Image Annotation ◽

State Of The Art ◽

Feature Space ◽

Semantic Space ◽

Visual Features ◽

Novel Approach ◽

Proposed Model ◽

Information Method ◽

Available Information

The ever-increasing size of images has made automatic image annotation one of the most important tasks in the fields of machine learning and computer vision. Despite continuous efforts in inventing new annotation algorithms and new models, results of the state-of-the-art image annotation methods are often unsatisfactory. In this paper, to further improve annotation refinement performance, a novel approach based on weighted mutual information to automatically refine the original annotations of images is proposed. Unlike the traditional refinement model using only visual feature, the proposed model use semantic embedding to properly map labels and visual features to a meaningful semantic space. To accurately measure the relevance between the particular image and its original annotations, the proposed model utilize all available information including image-to-image, label-to-label and image-to-label. Experimental results conducted on three typical datasets show not only the validity of the refinement, but also the superiority of the proposed algorithm over existing ones. The improvement largely benefits from our proposed mutual information method and utilizing all available information.

Download Full-text

Critical Nodes Detecting in Virtual Networking Environment

2014 IEEE World Congress on Services ◽

10.1109/services.2014.62 ◽

2014 ◽

Cited By ~ 1

Author(s):

Rongheng Lin ◽

Budan Wu ◽

Yao Zhao ◽

Hua Zou ◽

Li Liu

Keyword(s):

Critical Nodes ◽

Virtual Networking

Download Full-text

Information-Maximization Clustering Based on Squared-Loss Mutual Information

Neural Computation ◽

10.1162/neco_a_00534 ◽

2014 ◽

Vol 26 (1) ◽

pp. 84-131 ◽

Cited By ~ 16

Author(s):

Masashi Sugiyama ◽

Gang Niu ◽

Makoto Yamada ◽

Manabu Kimura ◽

Hirotaka Hachiya

Keyword(s):

Mutual Information ◽

Optimization Problems ◽

Continuous Optimization ◽

Optimal Solution ◽

Selection Procedure ◽

Eigenvalue Decomposition ◽

Model Parameters ◽

Computationally Efficient ◽

Novel Approach ◽

Information Maximization

Information-maximization clustering learns a probabilistic classifier in an unsupervised manner so that mutual information between feature vectors and cluster assignments is maximized. A notable advantage of this approach is that it involves only continuous optimization of model parameters, which is substantially simpler than discrete optimization of cluster assignments. However, existing methods still involve nonconvex optimization problems, and therefore finding a good local optimal solution is not straightforward in practice. In this letter, we propose an alternative information-maximization clustering method based on a squared-loss variant of mutual information. This novel approach gives a clustering solution analytically in a computationally efficient way via kernel eigenvalue decomposition. Furthermore, we provide a practical model selection procedure that allows us to objectively optimize tuning parameters included in the kernel function. Through experiments, we demonstrate the usefulness of the proposed approach.

Download Full-text

Mutual Information Based Learning Rate Decay for Stochastic Gradient Descent Training of Deep Neural Networks

Entropy ◽

10.3390/e22050560 ◽

2020 ◽

Vol 22 (5) ◽

pp. 560

Author(s):

Shrihari Vasudevan

Keyword(s):

Neural Networks ◽

Mutual Information ◽

Gradient Descent ◽

Deep Neural Networks ◽

Learning Rate ◽

Stochastic Gradient ◽

Stochastic Gradient Descent ◽

Novel Approach ◽

The Neural Network ◽

Gradient Based

This paper demonstrates a novel approach to training deep neural networks using a Mutual Information (MI)-driven, decaying Learning Rate (LR), Stochastic Gradient Descent (SGD) algorithm. MI between the output of the neural network and true outcomes is used to adaptively set the LR for the network, in every epoch of the training cycle. This idea is extended to layer-wise setting of LR, as MI naturally provides a layer-wise performance metric. A LR range test determining the operating LR range is also proposed. Experiments compared this approach with popular alternatives such as gradient-based adaptive LR algorithms like Adam, RMSprop, and LARS. Competitive to better accuracy outcomes obtained in competitive to better time, demonstrate the feasibility of the metric and approach.

Download Full-text

Distant Supervision for Relation Extraction with Linear Attenuation Simulation and Non-IID Relevance Embedding

Proceedings of the AAAI Conference on Artificial Intelligence ◽

10.1609/aaai.v33i01.33017418 ◽

2019 ◽

Vol 33 ◽

pp. 7418-7425 ◽

Cited By ~ 2

Author(s):

Changsen Yuan ◽

Heyan Huang ◽

Chong Feng ◽

Xiao Liu ◽

Xiaochi Wei

Keyword(s):

Mutual Information ◽

Efficient Method ◽

Relation Extraction ◽

Benchmark Dataset ◽

Labor Costs ◽

Distant Supervision ◽

Complex Information ◽

Novel Approach ◽

Independent And Identically Distributed

Distant supervision for relation extraction is an efficient method to reduce labor costs and has been widely used to seek novel relational facts in large corpora, which can be identified as a multi-instance multi-label problem. However, existing distant supervision methods suffer from selecting important words in the sentence and extracting valid sentences in the bag. Towards this end, we propose a novel approach to address these problems in this paper. Firstly, we propose a linear attenuation simulation to reflect the importance of words in the sentence with respect to the distances between entities and words. Secondly, we propose a non-independent and identically distributed (non-IID) relevance embedding to capture the relevance of sentences in the bag. Our method can not only capture complex information of words about hidden relations, but also express the mutual information of instances in the bag. Extensive experiments on a benchmark dataset have well-validated the effectiveness of the proposed method.

Download Full-text

A novel approach of closeness centrality measure for voltage stability analysis in an electric power grid

International Journal of Emerging Electric Power Systems ◽

10.1515/ijeeps-2020-0013 ◽

2020 ◽

Vol 21 (3) ◽

Author(s):

Isaiah G. Adebayo ◽

Yanxia Sun

Keyword(s):

Power Systems ◽

Power System ◽

Voltage Stability ◽

Closeness Centrality ◽

Voltage Instability ◽

Load Demand ◽

Electric Power Grid ◽

Novel Approach ◽

Critical Nodes ◽

Enhancement Technologies

AbstractModern power systems are increasingly becoming more complex and thus become vulnerable to voltage collapse due to constant increase in load demand and introduction of new operation enhancement technologies. In this study, an approach which is based on network structural properties of a power system is proposed for the identification of critical nodes that are liable to voltage instability. The proposed Network Structurally Based Closeness Centrality (NSBCC) is formulated based on the admittance matrix between the interconnection of load to load nodes in a power system. The vertex (node) that has the highest value of NSBCC is taken as the critical node of the system. To demonstrate the significance of the concept formulated, the comparative analysis of the proposed NSBCC with the conventional techniques such as Electrical Closeness Centrality (ECC), Closeness Voltage Centrality (CVC) and Modal Analysis is performed. The effectiveness of all the approaches presented is tested on both IEEE 30 bus and the Southern Indian 10-bus power systems. Results of simulation obtained show that the proposed NSBCC could serve as valuable tool for rapid real time voltage stability assessment in a power system.

Download Full-text