scholarly journals Locally Linear Factorization Machines

2017 ◽  
Author(s):  
Chenghao Liu ◽  
Teng Zhang ◽  
Peilin Zhao ◽  
Jun Zhou ◽  
Jianling Sun
Keyword(s):  
Predictive Accuracy ◽  
High Order ◽  
Anchor Point ◽  
Complex Data ◽  
Coding Scheme ◽  
Feature Interactions ◽  
Label Information ◽  
Local Coding ◽  
Linear Factorization ◽  
Locally Linear

Factorization Machines (FMs) are a widely used method for efficiently using high-order feature interactions in classification and regression tasks. Unfortunately, despite increasing interests in FMs, existing work only considers high order information of the input features which limits their capacities in non-linear problems and fails to capture the underlying structures of more complex data. In this work, we present a novel Locally Linear Factorization Machines (LLFM) which overcomes this limitation by exploring local coding technique. Unlike existing local coding classifiers that involve a phase of unsupervised anchor point learning and predefined local coding scheme which is suboptimal as the class label information is not exploited in discovering the encoding and thus can result in a suboptimal encoding for prediction, we formulate a joint optimization over the anchor points, local coding coordinates and FMs variables to minimize classification or regression risk. Empirically, we demonstrate that our approach achieves much better predictive accuracy than other competitive methods which employ LLFM with unsupervised anchor point learning and predefined local coding scheme.

scholarly journals Enabling High Order SCMA Systems in Downlink Scenarios With a Serial Coding Scheme

IEEE Access ◽  
2018 ◽  
Vol 6 ◽  
pp. 33796-33809 ◽  
Author(s):  
Yuxi Han ◽  
Wuyang Zhou ◽  
Ming Zhao ◽  
Shengli Zhou
Keyword(s):  
High Order ◽  
Coding Scheme

scholarly journals Deep Neural Networks Constrained by Decision Rules

2019 ◽  
Vol 33 ◽  
pp. 2496-2505
Author(s):  
Yuzuru Okajima ◽  
Kunihiko Sadamasa
Keyword(s):  
Neural Networks ◽  
Deep Neural Networks ◽  
Predictive Accuracy ◽  
Decision Rules ◽  
Hybrid Technique ◽  
Complex Data ◽  
Rule Based ◽  
Prior Probabilities ◽  
The Neural Network ◽  
Latent Representations

Deep neural networks achieve high predictive accuracy by learning latent representations of complex data. However, the reasoning behind their decisions is difficult for humans to understand. On the other hand, rule-based approaches are able to justify the decisions by showing the decision rules leading to them, but they have relatively low accuracy. To improve the interpretability of neural networks, several techniques provide post-hoc explanations of decisions made by neural networks, but they cannot guarantee that the decisions are always explained in a simple form like decision rules because their explanations are generated after the decisions are made by neural networks.In this paper, to balance the accuracy of neural networks and the interpretability of decision rules, we propose a hybrid technique called rule-constrained networks, namely, neural networks that make decisions by selecting decision rules from a given ruleset. Because the networks are forced to make decisions based on decision rules, it is guaranteed that every decision is supported by a decision rule. Furthermore, we propose a technique to jointly optimize the neural network and the ruleset from which the network select rules. The log likelihood of correct classifications is maximized under a model with hyper parameters about the ruleset size and the prior probabilities of rules being selected. This feature makes it possible to limit the ruleset size or prioritize human-made rules over automatically acquired rules for promoting the interpretability of the output. Experiments on datasets of time-series and sentiment classification showed rule-constrained networks achieved accuracy as high as that achieved by original neural networks and significantly higher than that achieved by existing rule-based models, while presenting decision rules supporting the decisions.

scholarly journals An Attention-Based Latent Information Extraction Network (ALIEN) for High-Order Feature Interactions

2020 ◽  
Vol 10 (16) ◽  
pp. 5468
Author(s):  
Ruo Huang ◽  
Shelby McIntyre ◽  
Meina Song ◽  
Haihong E ◽  
Zhonghong Ou
Keyword(s):  
Information Extraction ◽  
Recommender Systems ◽  
Vital Role ◽  
High Order ◽  
Sequential Patterns ◽  
Sequence Information ◽  
Recommendation Algorithm ◽  
Feature Interactions ◽  
Real World Datasets ◽  
Click Through Rate

One of the primary tasks for commercial recommender systems is to predict the probabilities of users clicking items, e.g., advertisements, music and products. This is because such predictions have a decisive impact on profitability. The classic recommendation algorithm, collaborative filtering (CF), still plays a vital role in many industrial recommender systems. However, although straight CF is good at capturing similar users’ preferences for items based on their past interactions, it lacks regarding (1) modeling the influences of users’ sequential patterns from their individual history interaction sequences and (2) the relevance of users’ and items’ attributes. In this work, we developed an attention-based latent information extraction network (ALIEN) for click-through rate prediction, to integrate (1) implicit user similarity in terms of click patterns (analogous to CF), and (2) modeling the low and high-order feature interactions and (3) historical sequence information. The new model is based on the deep learning, which goes beyond the capabilities of econometric approaches, such as matrix factorization (MF) and k-means. In addition, the approach provides explainability to the recommendation by interpreting the contributions of different features and historical interactions. We have conducted experiments on real-world datasets that demonstrate considerable improvements over strong baselines.

scholarly journals Refining interaction search through signed iterative Random Forests

2018 ◽  
Author(s):  
Karl Kumbier ◽  
Sumanta Basu ◽  
James B. Brown ◽  
Susan Celniker ◽  
Bin Yu
Keyword(s):  
Gene Expression ◽  
Random Forests ◽  
Predictive Accuracy ◽  
Expression Patterns ◽  
High Order ◽  
Gene Expression Patterns ◽  
Enhancer Activity ◽  
Gap Gene ◽  
Strength Of Interaction ◽  
Interacting Features

AbstractAdvances in supervised learning have enabled accurate prediction in biological systems governed by complex interactions among biomolecules. However, state-of-the-art predictive algorithms are typically “black-boxes,” learning statistical interactions that are difficult to translate into testable hypotheses. The iterative Random Forest (iRF) algorithm took a step towards bridging this gap by providing a computationally tractable procedure to identify the stable, high-order feature interactions that drive the predictive accuracy of Random Forests (RF). Here we refine the interactions identified by iRF to explicitly map responses as a function of interacting features. Our method, signed iRF (s-iRF), describes “subsets” of rules that frequently occur on RF decision paths. We refer to these “rule subsets” as signed interactions. Signed interactions share not only the same set of interacting features but also exhibit similar thresholding behavior, and thus describe a consistent functional relationship between interacting features and responses. We describe stable and predictive importance metrics (SPIMs) to rank signed interactions in terms of their stability, predictive accuracy, and strength of interaction. For each SPIM, we define null importance metrics that characterize its expected behavior under known structure. We evaluate our proposed approach in biologically inspired simulations and two case studies: predicting enhancer activity and spatial gene expression patterns. In the case of enhancer activity, s-iRF recovers one of the few experimentally validated high-order interactions and suggests novel enhancer elements where this interaction may be active. In the case of spatial gene expression patterns, s-iRF recovers all 11 reported links in the gap gene network. By refining the process of interaction recovery, our approach has the potential to guide mechanistic inquiry into systems whose scale and complexity is beyond human comprehension.

scholarly journals PMDFI: Predicting miRNA–Disease Associations Based on High-Order Feature Interaction

2021 ◽  
Vol 12 ◽  
Author(s):  
Mingyan Tang ◽  
Chenzhe Liu ◽  
Dayun Liu ◽  
Junyi Liu ◽  
Jiaqi Liu ◽  
...  
Keyword(s):  
Computational Models ◽  
Treatment Method ◽  
High Order ◽  
Computational Method ◽  
Feature Interaction ◽  
Rna Molecules ◽  
Feature Interactions ◽  
Non Coding Rna ◽  
Disease Associations ◽  
Fold Cross Validation

MicroRNAs (miRNAs) are non-coding RNA molecules that make a significant contribution to diverse biological processes, and their mutations and dysregulations are closely related to the occurrence, development, and treatment of human diseases. Therefore, identification of potential miRNA–disease associations contributes to elucidating the pathogenesis of tumorigenesis and seeking the effective treatment method for diseases. Due to the expensive cost of traditional biological experiments of determining associations between miRNAs and diseases, increasing numbers of effective computational models are being used to compensate for this limitation. In this study, we propose a novel computational method, named PMDFI, which is an ensemble learning method to predict potential miRNA–disease associations based on high-order feature interactions. We initially use a stacked autoencoder to extract meaningful high-order features from the original similarity matrix, and then perform feature interactive learning, and finally utilize an integrated model composed of multiple random forests and logistic regression to make comprehensive predictions. The experimental results illustrate that PMDFI achieves excellent performance in predicting potential miRNA–disease associations, with the average area under the ROC curve scores of 0.9404 and 0.9415 in 5-fold and 10-fold cross-validation, respectively.

A Generalized Locally Linear Factorization Machine with Supervised Variational Encoding

2020 ◽  
Vol 32 (6) ◽  
pp. 1036-1049
Author(s):  
Xiaoshuang Chen ◽  
Yin Zheng ◽  
Peilin Zhao ◽  
Zhuxi Jiang ◽  
Wenye Ma ◽  
...  
Keyword(s):  
Factorization Machine ◽  
Linear Factorization ◽  
Locally Linear

scholarly journals LCHI: Low-Order Correlation and High-Order Interaction Integrated Model Oriented to Network Intrusion Detection

2021 ◽  
Vol 2021 ◽  
pp. 1-16
Author(s):  
Shengwei Lei ◽  
Chunhe Xia ◽  
Tianbo Wang
Keyword(s):  
Intrusion Detection ◽  
High Order ◽  
Network Intrusion Detection ◽  
Feature Interaction ◽  
Detection Accuracy ◽  
High Order Interaction ◽  
Order Interaction ◽  
Feature Interactions ◽  
Network Intrusion ◽  
Low Order

Network intrusion poses a severe threat to the Internet of Things (IoT). Thus, it is essential to study information security protection technology in IoT. Learning sophisticated feature interactions is critical in improving detection accuracy for network intrusion. Despite significant progress, existing methods seem to have a strong bias towards single low- or high-order feature interaction. Moreover, they always extract all possible low-order interactions indiscriminately, introducing too much noise. To address the above problems, we propose a low-order correlation and high-order interaction (LCHI) integrated feature extraction model. First, we selectively extract the beneficial low-order correlation between the same-type features by the multivariate correlation analysis (MCA) model and attention mechanism. Second, we extract the complicated high-order feature interaction by the deep neural network (DNN) model. Finally, we emphasize both the low- and high-order feature interactions and incorporate them. Our LCHI model seamlessly combines the linearity of MCA in modeling lower-order feature correlation and the nonlinearity of DNN in modeling higher-order feature interaction. Conceptually, our LCHI is more expressive than the previous models. We carry on a series of experiments on the public wireless and wired network intrusion detection datasets. The experimental results show that LCHI improves 1.06%, 2.46%, 3.74%, 0.25%, 1.17%, and 0.64% on the AWID, NSL-KDD, UNSW-NB15, CICIDS 2017, CICIDS 2018, and DAPT 2020 datasets, respectively.

Deep context interaction network based on attention mechanism for click-through rate prediction

2021 ◽  
pp. 1-16
Author(s):  
Ling Yuan ◽  
Zhuwen Pan ◽  
Ping Sun ◽  
Yinzhen Wei ◽  
Haiping Yu
Keyword(s):  
Online Advertising ◽  
Dimensional Space ◽  
Interaction Network ◽  
High Order ◽  
Attention Mechanism ◽  
Feature Interaction ◽  
Feature Interactions ◽  
Low Dimensional ◽  
Click Through Rate ◽  
The Relationship

Click-through rate (CTR) prediction, which aims to predict the probability of a user clicking on an ad, is a critical task in online advertising systems. The problem is very challenging since(1) an effective prediction relies on high-order combinatorial features, and(2)the relationship to auxiliary ads that may impact the CTR. In this paper, we propose Deep Context Interaction Network on Attention Mechanism(DCIN-Attention) to process feature interaction and context at the same time. The context includes other ads in the current search page, historically clicked and unclicked ads of the user. Specifically, we use the attention mechanism to learn the interactions between the target ad and each type of auxiliary ad. The residual network is used to model the feature interactions in the low-dimensional space, and with the multi-head self-attention neural network, high-order feature interactions can be modeled. Experimental results on Avito dataset show that DCIN outperform several existing methods for CTR prediction.

Sign in / Sign up

Export Citation Format

Share Document