scholarly journals ProNE: Fast and Scalable Network Representation Learning

2019 ◽  
Author(s):  
Jie Zhang ◽  
Yuxiao Dong ◽  
Yan Wang ◽  
Jie Tang ◽  
Ming Ding
Keyword(s):  
Large Scale ◽  
Sparse Matrix ◽  
Representation Learning ◽  
Second Step ◽  
Network Embedding ◽  
Network Mining ◽  
Large Scale Networks ◽  
Scalable Network ◽  
Relative Gains ◽  
Sparse Matrix Factorization

Recent advances in network embedding has revolutionized the field of graph and network mining. However, (pre-)training embeddings for very large-scale networks is computationally challenging for most existing methods. In this work, we present ProNE---a fast, scalable, and effective model, whose single-thread version is 10--400x faster than efficient network embedding benchmarks with 20 threads, including LINE, DeepWalk, node2vec, GraRep, and HOPE. As a concrete example, the single-version ProNE requires only 29 hours to embed a network of hundreds of millions of nodes while it takes LINE weeks and DeepWalk months by using 20 threads. To achieve this, ProNE first initializes network embeddings efficiently by formulating the task as sparse matrix factorization. The second step of ProNE is to enhance the embeddings by propagating them in the spectrally modulated space. Extensive experiments on networks of various scales and types demonstrate that ProNE achieves both effectiveness and significant efficiency superiority when compared to the aforementioned baselines. In addition, ProNE's embedding enhancement step can be also generalized for improving other models at speed, e.g., offering >10% relative gains for the used baselines. 

scholarly journals Robust Negative Sampling for Network Embedding

2019 ◽  
Vol 33 ◽  
pp. 3191-3198 ◽  
Author(s):  
Mohammadreza Armandpour ◽  
Patrick Ding ◽  
Jianhua Huang ◽  
Xia Hu
Keyword(s):  
Neural Network ◽  
Network Architecture ◽  
Large Scale ◽  
Embedding Problem ◽  
Network Embedding ◽  
Neural Network Architecture ◽  
Suitable Candidate ◽  
Computationally Expensive ◽  
Large Scale Networks ◽  
Method R

Many recent network embedding algorithms use negative sampling (NS) to approximate a variant of the computationally expensive Skip-Gram neural network architecture (SGA) objective. In this paper, we provide theoretical arguments that reveal how NS can fail to properly estimate the SGA objective, and why it is not a suitable candidate for the network embedding problem as a distinct objective. We show NS can learn undesirable embeddings, as the result of the “Popular Neighbor Problem.” We use the theory to develop a new method “R-NS” that alleviates the problems of NS by using a more intelligent negative sampling scheme and careful penalization of the embeddings. R-NS is scalable to large-scale networks, and we empirically demonstrate the superiority of R-NS over NS for multi-label classification on a variety of real-world networks including social networks and language networks.

scholarly journals SepNE: Bringing Separability to Network Embedding

2019 ◽  
Vol 33 ◽  
pp. 4261-4268 ◽  
Author(s):  
Ziyao Li ◽  
Liang Zhang ◽  
Guojie Song
Keyword(s):  
Large Scale ◽  
State Of The Art ◽  
Dynamic Networks ◽  
Distributed Learning ◽  
Network Embedding ◽  
Large Networks ◽  
Comparable Accuracy ◽  
Large Scale Networks ◽  
Low Dimensional ◽  
Almost All

Many successful methods have been proposed for learning low dimensional representations on large-scale networks, while almost all existing methods are designed in inseparable processes, learning embeddings for entire networks even when only a small proportion of nodes are of interest. This leads to great inconvenience, especially on super-large or dynamic networks, where these methods become almost impossible to implement. In this paper, we formalize the problem of separated matrix factorization, based on which we elaborate a novel objective function that preserves both local and global information. We further propose SepNE, a simple and flexible network embedding algorithm which independently learns representations for different subsets of nodes in separated processes. By implementing separability, our algorithm reduces the redundant efforts to embed irrelevant nodes, yielding scalability to super-large networks, automatic implementation in distributed learning and further adaptations. We demonstrate the effectiveness of this approach on several real-world networks with different scales and subjects. With comparable accuracy, our approach significantly outperforms state-of-the-art baselines in running times on large networks.

scholarly journals SNEQ: Semi-Supervised Attributed Network Embedding with Attention-Based Quantisation

2020 ◽  
Vol 34 (04) ◽  
pp. 4091-4098 ◽  
Author(s):  
Tao He ◽  
Lianli Gao ◽  
Jingkuan Song ◽  
Xin Wang ◽  
Kejie Huang ◽  
...  
Keyword(s):  
Link Prediction ◽  
Large Scale ◽  
State Of The Art ◽  
Network Embedding ◽  
Compression Method ◽  
Network Analytics ◽  
Attributed Network ◽  
Large Scale Networks ◽  
Low Dimensional ◽  
Embedding Methods

Learning accurate low-dimensional embeddings for a network is a crucial task as it facilitates many network analytics tasks. Moreover, the trained embeddings often require a significant amount of space to store, making storage and processing a challenge, especially as large-scale networks become more prevalent. In this paper, we present a novel semi-supervised network embedding and compression method, SNEQ, that is competitive with state-of-art embedding methods while being far more space- and time-efficient. SNEQ incorporates a novel quantisation method based on a self-attention layer that is trained in an end-to-end fashion, which is able to dramatically compress the size of the trained embeddings, thus reduces storage footprint and accelerates retrieval speed. Our evaluation on four real-world networks of diverse characteristics shows that SNEQ outperforms a number of state-of-the-art embedding methods in link prediction, node classification and node recommendation. Moreover, the quantised embedding shows a great advantage in terms of storage and time compared with continuous embeddings as well as hashing methods.

A Drug Target Interaction Prediction Based on LINE-RF Learning

2020 ◽  
Vol 15 (7) ◽  
pp. 750-757
Author(s):  
Jihong Wang ◽  
Yue Shi ◽  
Xiaodan Wang ◽  
Huiyou Chang
Keyword(s):  
Network Topology ◽  
Drug Target ◽  
Large Scale ◽  
Representation Learning ◽  
New Drugs ◽  
Combination Method ◽  
Learning Methods ◽  
Network Representation ◽  
On Line ◽  
Clinical Experiments

Background: At present, using computer methods to predict drug-target interactions (DTIs) is a very important step in the discovery of new drugs and drug relocation processes. The potential DTIs identified by machine learning methods can provide guidance in biochemical or clinical experiments. Objective: The goal of this article is to combine the latest network representation learning methods for drug-target prediction research, improve model prediction capabilities, and promote new drug development. Methods: We use large-scale information network embedding (LINE) method to extract network topology features of drugs, targets, diseases, etc., integrate features obtained from heterogeneous networks, construct binary classification samples, and use random forest (RF) method to predict DTIs. Results: The experiments in this paper compare the common classifiers of RF, LR, and SVM, as well as the typical network representation learning methods of LINE, Node2Vec, and DeepWalk. It can be seen that the combined method LINE-RF achieves the best results, reaching an AUC of 0.9349 and an AUPR of 0.9016. Conclusion: The learning method based on LINE network can effectively learn drugs, targets, diseases and other hidden features from the network topology. The combination of features learned through multiple networks can enhance the expression ability. RF is an effective method of supervised learning. Therefore, the Line-RF combination method is a widely applicable method.

scholarly journals Contribution of Ionotropic Glutamatergic Receptors to Excitability and Attentional Signals in Macaque Frontal Eye Field

2021 ◽  
Author(s):  
Miguel Dasilva ◽  
Christian Brandt ◽  
Marc Alwin Gieselmann ◽  
Claudia Distler ◽  
Alexander Thiele
Keyword(s):  
Attentional Control ◽  
Large Scale ◽  
Neuronal Excitability ◽  
Cell Types ◽  
Receptor Activation ◽  
Frontal Eye Field ◽  
Control Signals ◽  
Cognitive Operations ◽  
Eye Field ◽  
Large Scale Networks

Abstract Top-down attention, controlled by frontal cortical areas, is a key component of cognitive operations. How different neurotransmitters and neuromodulators flexibly change the cellular and network interactions with attention demands remains poorly understood. While acetylcholine and dopamine are critically involved, glutamatergic receptors have been proposed to play important roles. To understand their contribution to attentional signals, we investigated how ionotropic glutamatergic receptors in the frontal eye field (FEF) of male macaques contribute to neuronal excitability and attentional control signals in different cell types. Broad-spiking and narrow-spiking cells both required N-methyl-D-aspartic acid and α-amino-3-hydroxy-5-methyl-4-isoxazolepropionic acid receptor activation for normal excitability, thereby affecting ongoing or stimulus-driven activity. However, attentional control signals were not dependent on either glutamatergic receptor type in broad- or narrow-spiking cells. A further subdivision of cell types into different functional types using cluster-analysis based on spike waveforms and spiking characteristics did not change the conclusions. This can be explained by a model where local blockade of specific ionotropic receptors is compensated by cell embedding in large-scale networks. It sets the glutamatergic system apart from the cholinergic system in FEF and demonstrates that a reduction in excitability is not sufficient to induce a reduction in attentional control signals.

Sign in / Sign up

Export Citation Format

Share Document