scholarly journals Large-Scale Heterogeneous Feature Embedding

2019 ◽  
Vol 33 ◽  
pp. 3878-3885 ◽  
Author(s):  
Xiao Huang ◽  
Qingquan Song ◽  
Fan Yang ◽  
Xia Hu
Keyword(s):  
Real World ◽  
Large Scale ◽  
Single Type ◽  
Heterogeneous Information ◽  
Multiview Learning ◽  
Efficiency And Effectiveness ◽  
Joint Embedding ◽  
Real World Datasets ◽  
Low Dimensional ◽  
Vector Representations

Feature embedding aims to learn a low-dimensional vector representation for each instance to preserve the information in its features. These representations can benefit various offthe-shelf learning algorithms. While embedding models for a single type of features have been well-studied, real-world instances often contain multiple types of correlated features or even information within a different modality such as networks. Existing studies such as multiview learning show that it is promising to learn unified vector representations from all sources. However, high computational costs of incorporating heterogeneous information limit the applications of existing algorithms. The number of instances and dimensions of features in practice are often large. To bridge the gap, we propose a scalable framework FeatWalk, which can model and incorporate instance similarities in terms of different types of features into a unified embedding representation. To enable the scalability, FeatWalk does not directly calculate any similarity measure, but provides an alternative way to simulate the similarity-based random walks among instances to extract the local instance proximity and preserve it in a set of instance index sequences. These sequences are homogeneous with each other. A scalable word embedding algorithm is applied to them to learn a joint embedding representation of instances. Experiments on four real-world datasets demonstrate the efficiency and effectiveness of FeatWalk.

scholarly journals Relation Structure-Aware Heterogeneous Information Network Embedding

2019 ◽  
Vol 33 ◽  
pp. 4456-4463 ◽  
Author(s):  
Yuanfu Lu ◽  
Chuan Shi ◽  
Linmei Hu ◽  
Zhiyuan Liu
Keyword(s):  
Real World ◽  
Dimensional Space ◽  
Structural Characteristics ◽  
Information Network ◽  
Network Embedding ◽  
Heterogeneous Information Network ◽  
Heterogeneous Information ◽  
Real World Datasets ◽  
Low Dimensional ◽  
Embedding Methods

Heterogeneous information network (HIN) embedding aims to embed multiple types of nodes into a low-dimensional space. Although most existing HIN embedding methods consider heterogeneous relations in HINs, they usually employ one single model for all relations without distinction, which inevitably restricts the capability of network embedding. In this paper, we take the structural characteristics of heterogeneous relations into consideration and propose a novel Relation structure-aware Heterogeneous Information Network Embedding model (RHINE). By exploring the real-world networks with thorough mathematical analysis, we present two structure-related measures which can consistently distinguish heterogeneous relations into two categories: Affiliation Relations (ARs) and Interaction Relations (IRs). To respect the distinctive characteristics of relations, in our RHINE, we propose different models specifically tailored to handle ARs and IRs, which can better capture the structures and semantics of the networks. At last, we combine and optimize these models in a unified and elegant manner. Extensive experiments on three real-world datasets demonstrate that our model significantly outperforms the state-of-the-art methods in various tasks, including node clustering, link prediction, and node classification.

scholarly journals Multityped Community Discovery in Time-Evolving Heterogeneous Information Networks Based on Tensor Decomposition

Complexity ◽  
2018 ◽  
Vol 2018 ◽  
pp. 1-16 ◽  
Author(s):  
Jibing Wu ◽  
Lianfei Yu ◽  
Qun Zhang ◽  
Peiteng Shi ◽  
Lihua Liu ◽  
...  
Keyword(s):  
Real World ◽  
Tensor Decomposition ◽  
Information Networks ◽  
Community Discovery ◽  
Star Network ◽  
Heterogeneous Information ◽  
Heterogeneous Information Networks ◽  
General Network ◽  
Real World Datasets ◽  
Discovery Method

The heterogeneous information networks are omnipresent in real-world applications, which consist of multiple types of objects with various rich semantic meaningful links among them. Community discovery is an effective method to extract the hidden structures in networks. Usually, heterogeneous information networks are time-evolving, whose objects and links are dynamic and varying gradually. In such time-evolving heterogeneous information networks, community discovery is a challenging topic and quite more difficult than that in traditional static homogeneous information networks. In contrast to communities in traditional approaches, which only contain one type of objects and links, communities in heterogeneous information networks contain multiple types of dynamic objects and links. Recently, some studies focus on dynamic heterogeneous information networks and achieve some satisfactory results. However, they assume that heterogeneous information networks usually follow some simple schemas, such as bityped network and star network schema. In this paper, we propose a multityped community discovery method for time-evolving heterogeneous information networks with general network schemas. A tensor decomposition framework, which integrates tensor CP factorization with a temporal evolution regularization term, is designed to model the multityped communities and address their evolution. Experimental results on both synthetic and real-world datasets demonstrate the efficiency of our framework.

scholarly journals Attention Enhanced Serial Unet++ Network for Removing Unevenly Distributed Haze

Electronics ◽  
2021 ◽  
Vol 10 (22) ◽  
pp. 2868
Author(s):  
Wenxuan Zhao ◽  
Yaqin Zhao ◽  
Liqi Feng ◽  
Jiaxi Tang
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
Real World ◽  
Large Scale ◽  
Learning Strategy ◽  
Contextual Information ◽  
Small Scale ◽  
Image Dehazing ◽  
Atmospheric Scattering ◽  
Real World Datasets

The purpose of image dehazing is the reduction of the image degradation caused by suspended particles for supporting high-level visual tasks. Besides the atmospheric scattering model, convolutional neural network (CNN) has been used for image dehazing. However, the existing image dehazing algorithms are limited in face of unevenly distributed haze and dense haze in real-world scenes. In this paper, we propose a novel end-to-end convolutional neural network called attention enhanced serial Unet++ dehazing network (AESUnet) for single image dehazing. We attempt to build a serial Unet++ structure that adopts a serial strategy of two pruned Unet++ blocks based on residual connection. Compared with the simple Encoder–Decoder structure, the serial Unet++ module can better use the features extracted by encoders and promote contextual information fusion in different resolutions. In addition, we take some improvement measures to the Unet++ module, such as pruning, introducing the convolutional module with ResNet structure, and a residual learning strategy. Thus, the serial Unet++ module can generate more realistic images with less color distortion. Furthermore, following the serial Unet++ blocks, an attention mechanism is introduced to pay different attention to haze regions with different concentrations by learning weights in the spatial domain and channel domain. Experiments are conducted on two representative datasets: the large-scale synthetic dataset RESIDE and the small-scale real-world datasets I-HAZY and O-HAZY. The experimental results show that the proposed dehazing network is not only comparable to state-of-the-art methods for the RESIDE synthetic datasets, but also surpasses them by a very large margin for the I-HAZY and O-HAZY real-world dataset.

scholarly journals Self-Paced Robust Learning for Leveraging Clean Labels in Noisy Data

2020 ◽  
Vol 34 (04) ◽  
pp. 6853-6860
Author(s):  
Xuchao Zhang ◽  
Xian Wu ◽  
Fanglan Chen ◽  
Liang Zhao ◽  
Chang-Tien Lu
Keyword(s):  
Real World ◽  
Large Scale ◽  
Learning Algorithm ◽  
Noisy Data ◽  
Training Set ◽  
Robust Learning ◽  
Robust Model ◽  
Small Set ◽  
Real World Datasets ◽  
Theoretical Analyses

The success of training accurate models strongly depends on the availability of a sufficient collection of precisely labeled data. However, real-world datasets contain erroneously labeled data samples that substantially hinder the performance of machine learning models. Meanwhile, well-labeled data is usually expensive to obtain and only a limited amount is available for training. In this paper, we consider the problem of training a robust model by using large-scale noisy data in conjunction with a small set of clean data. To leverage the information contained via the clean labels, we propose a novel self-paced robust learning algorithm (SPRL) that trains the model in a process from more reliable (clean) data instances to less reliable (noisy) ones under the supervision of well-labeled data. The self-paced learning process hedges the risk of selecting corrupted data into the training set. Moreover, theoretical analyses on the convergence of the proposed algorithm are provided under mild assumptions. Extensive experiments on synthetic and real-world datasets demonstrate that our proposed approach can achieve a considerable improvement in effectiveness and robustness to existing methods.

scholarly journals Efficient Heterogeneous Collaborative Filtering without Negative Sampling for Recommendation

2020 ◽  
Vol 34 (01) ◽  
pp. 19-26 ◽  
Author(s):  
Chong Chen ◽  
Min Zhang ◽  
Yongfeng Zhang ◽  
Weizhi Ma ◽  
Yiqun Liu ◽  
...  
Keyword(s):  
Collaborative Filtering ◽  
Real World ◽  
Large Scale ◽  
State Of The Art ◽  
Heterogeneous Data ◽  
Model Parameters ◽  
Online Systems ◽  
Practical Applications ◽  
Real World Datasets ◽  
Primary Type

Recent studies on recommendation have largely focused on exploring state-of-the-art neural networks to improve the expressiveness of models, while typically apply the Negative Sampling (NS) strategy for efficient learning. Despite effectiveness, two important issues have not been well-considered in existing methods: 1) NS suffers from dramatic fluctuation, making sampling-based methods difficult to achieve the optimal ranking performance in practical applications; 2) although heterogeneous feedback (e.g., view, click, and purchase) is widespread in many online systems, most existing methods leverage only one primary type of user feedback such as purchase. In this work, we propose a novel non-sampling transfer learning solution, named Efficient Heterogeneous Collaborative Filtering (EHCF) for Top-N recommendation. It can not only model fine-grained user-item relations, but also efficiently learn model parameters from the whole heterogeneous data (including all unlabeled data) with a rather low time complexity. Extensive experiments on three real-world datasets show that EHCF significantly outperforms state-of-the-art recommendation methods in both traditional (single-behavior) and heterogeneous scenarios. Moreover, EHCF shows significant improvements in training efficiency, making it more applicable to real-world large-scale systems. Our implementation has been released 1 to facilitate further developments on efficient whole-data based neural methods.

An Initialization Method for Clustering Mixed Numeric and Categorical Data Based on the Density and Distance

2015 ◽  
Vol 29 (07) ◽  
pp. 1550024 ◽  
Author(s):  
Jinchao Ji ◽  
Wei Pang ◽  
Yanlin Zheng ◽  
Zhe Wang ◽  
Zhiqiang Ma
Keyword(s):  
Real World ◽  
Mixed Type ◽  
Clustering Algorithms ◽  
Numerical Data ◽  
Mixed Data ◽  
Single Type ◽  
Partitional Clustering ◽  
Initial Cluster ◽  
Real World Datasets ◽  
Type Data

Most of the initialization approaches are dedicated to the partitional clustering algorithms which process categorical or numerical data only. However, in real-world applications, data objects with both numeric and categorical features are ubiquitous. The coexistence of both categorical and numerical attributes make the initialization methods designed for single-type data inapplicable to mixed-type data. Furthermore, to the best of our knowledge, in the existing partitional clustering algorithms designed for mixed-type data, the initial cluster centers are determined randomly. In this paper, we propose a novel initialization method for mixed data clustering. In the proposed method, both the distance and density are exploited together to determine initial cluster centers. The performance of the proposed method is demonstrated by a series of experiments on three real-world datasets in comparison with that of traditional initialization methods.

scholarly journals Dictionary learning allows model-free pseudotime estimation of transcriptomic data

BMC Genomics ◽  
2022 ◽  
Vol 23 (1) ◽  
Author(s):  
Mona Rams ◽  
Tim O.F. Conrad
Keyword(s):  
Dimension Reduction ◽  
Dictionary Learning ◽  
Real World ◽  
Estimation Methods ◽  
Dynamic Processes ◽  
Dimensional Representation ◽  
Transcriptomic Data ◽  
Model Free ◽  
Real World Datasets ◽  
Low Dimensional

Abstract Background Pseudotime estimation from dynamic single-cell transcriptomic data enables characterisation and understanding of the underlying processes, for example developmental processes. Various pseudotime estimation methods have been proposed during the last years. Typically, these methods start with a dimension reduction step because the low-dimensional representation is usually easier to analyse. Approaches such as PCA, ICA or t-SNE belong to the most widely used methods for dimension reduction in pseudotime estimation methods. However, these methods usually make assumptions on the derived dimensions, which can result in important dataset properties being missed. In this paper, we suggest a new dictionary learning based approach, dynDLT, for dimension reduction and pseudotime estimation of dynamic transcriptomic data. Dictionary learning is a matrix factorisation approach that does not restrict the dependence of the derived dimensions. To evaluate the performance, we conduct a large simulation study and analyse 8 real-world datasets. Results The simulation studies reveal that firstly, dynDLT preserves the simulated patterns in low-dimension and the pseudotimes can be derived from the low-dimensional representation. Secondly, the results show that dynDLT is suitable for the detection of genes exhibiting the simulated dynamic patterns, thereby facilitating the interpretation of the compressed representation and thus the dynamic processes. For the real-world data analysis, we select datasets with samples that are taken at different time points throughout an experiment. The pseudotimes found by dynDLT have high correlations with the experimental times. We compare the results to other approaches used in pseudotime estimation, or those that are method-wise closely connected to dictionary learning: ICA, NMF, PCA, t-SNE, and UMAP. DynDLT has the best overall performance for the simulated and real-world datasets. Conclusions We introduce dynDLT, a method that is suitable for pseudotime estimation. Its main advantages are: (1) It presents a model-free approach, meaning that it does not restrict the dependence of the derived dimensions; (2) Genes that are relevant in the detected dynamic processes can be identified from the dictionary matrix; (3) By a restriction of the dictionary entries to positive values, the dictionary atoms are highly interpretable.

scholarly journals DMRAN:A Hierarchical Fine-Grained Attention-Based Network for Recommendation

2019 ◽  
Author(s):  
Huizhao Wang ◽  
Guanfeng Liu ◽  
An Liu ◽  
Zhixu Li ◽  
Kai Zheng
Keyword(s):  
Real World ◽  
State Of The Art ◽  
User Preferences ◽  
One Dimensional ◽  
Fine Grained ◽  
Item Level ◽  
Efficiency And Effectiveness ◽  
Time Encoding ◽  
Real World Datasets

The conventional methods for the next-item recommendation are generally based on RNN or one- dimensional attention with time encoding. They are either hard to preserve the long-term dependencies between different interactions, or hard to capture fine-grained user preferences. In this paper, we propose a Double Most Relevant Attention Network (DMRAN) that contains two layers, i.e., Item level Attention and Feature Level Self- attention, which are to pick out the most relevant items from the sequence of user’s historical behaviors, and extract the most relevant aspects of relevant items, respectively. Then, we can capture the fine-grained user preferences to better support the next-item recommendation. Extensive experiments on two real-world datasets illustrate that DMRAN can improve the efficiency and effectiveness of the recommendation compared with the state-of-the-art methods.

scholarly journals Quadruply Stochastic Gradient Method for Large Scale Nonlinear Semi-Supervised Ordinal Regression AUC Optimization

2020 ◽  
Vol 34 (04) ◽  
pp. 5734-5741
Author(s):  
Wanli Shi ◽  
Bin Gu ◽  
Xiang Li ◽  
Heng Huang
Keyword(s):  
Real World ◽  
Large Scale ◽  
Optimal Solution ◽  
Ordinal Regression ◽  
Data Sampling ◽  
Decomposition Approach ◽  
Scalable Algorithm ◽  
Auc Optimization ◽  
Stochastic Data ◽  
Real World Datasets

Semi-supervised ordinal regression (S2OR) problems are ubiquitous in real-world applications, where only a few ordered instances are labeled and massive instances remain unlabeled. Recent researches have shown that directly optimizing concordance index or AUC can impose a better ranking on the data than optimizing the traditional error rate in ordinal regression (OR) problems. In this paper, we propose an unbiased objective function for S2OR AUC optimization based on ordinal binary decomposition approach. Besides, to handle the large-scale kernelized learning problems, we propose a scalable algorithm called QS3ORAO using the doubly stochastic gradients (DSG) framework for functional optimization. Theoretically, we prove that our method can converge to the optimal solution at the rate of O(1/t), where t is the number of iterations for stochastic data sampling. Extensive experimental results on various benchmark and real-world datasets also demonstrate that our method is efficient and effective while retaining similar generalization performance.

scholarly journals Self-weighted Multiview Clustering with Multiple Graphs

2017 ◽  
Author(s):  
Feiping Nie ◽  
Jing Li ◽  
Xuelong Li
Keyword(s):  
Real World ◽  
Spectral Clustering ◽  
Experimental Results ◽  
Clustering Method ◽  
Elegant Method ◽  
Multiview Learning ◽  
Cluster Label ◽  
Real World Datasets ◽  
Synthetic Datasets ◽  
Multiview Clustering

In multiview learning, it is essential to assign a reasonable weight to each view according to its importance. Thus, for multiview clustering task, a wise and elegant method should achieve clustering multiview data while learning the view weights. In this paper, we address this problem by exploring a Laplacian rank constrained graph, which can be approximately as the centroid of the built graph for each view with different confidences. We start our work with a natural thought that the weights can be learned by introducing a hyperparameter. By analyzing the weakness of it, we further propose a new multiview clustering method which is totally self-weighted. Furthermore, once the target graph is obtained in our models, we can directly assign the cluster label to each data point and do not need any postprocessing such as $K$-means in standard spectral clustering. Evaluations on two synthetic datasets prove the effectiveness of our methods. Compared with several representative graph-based multiview clustering approaches on four real-world datasets, experimental results demonstrate that the proposed methods achieve the better performances and our new clustering method is more practical to use.

Sign in / Sign up

Export Citation Format

Share Document