scholarly journals Learning Feature Interactions with Lorentzian Factorization Machine

2020 ◽  
Vol 34 (04) ◽  
pp. 6470-6477
Author(s):  
Canran Xu ◽  
Ming Wu
Keyword(s):  
Deep Learning ◽  
Hyperbolic Space ◽  
Recommendation System ◽  
Triangle Inequality ◽  
State Of The Art ◽  
Learning Methods ◽  
New Model ◽  
User Behaviors ◽  
Feature Interactions ◽  
Factorization Machine

Learning representations for feature interactions to model user behaviors is critical for recommendation system and click-trough rate (CTR) predictions. Recent advances in this area are empowered by deep learning methods which could learn sophisticated feature interactions and achieve the state-of-the-art result in an end-to-end manner. These approaches require large number of training parameters integrated with the low-level representations, and thus are memory and computational inefficient. In this paper, we propose a new model named “LorentzFM” that can learn feature interactions embedded in a hyperbolic space in which the violation of triangle inequality for Lorentz distances is available. To this end, the learned representation is benefited by the peculiar geometric properties of hyperbolic triangles, and result in a significant reduction in the number of parameters (20% to 80%) because all the top deep learning layers are not required. With such a lightweight architecture, LorentzFM achieves comparable and even materially better results than the deep learning methods such as DeepFM, xDeepFM and Deep & Cross in both recommendation and CTR prediction tasks.

scholarly journals Attentional Factorization Machines: Learning the Weight of Feature Interactions via Attention Networks

2017 ◽  
Author(s):  
Jun Xiao ◽  
Hao Ye ◽  
Xiangnan He ◽  
Hanwang Zhang ◽  
Fei Wu ◽  
...  
Keyword(s):  
Deep Learning ◽  
State Of The Art ◽  
Feature Interaction ◽  
Model Parameters ◽  
Learning Approach ◽  
Attention Networks ◽  
Feature Interactions ◽  
Factorization Machine ◽  
Real World Datasets ◽  
Novel Model

Factorization Machines (FMs) are a supervised learning approach that enhances the linear regression model by incorporating the second-order feature interactions. Despite effectiveness, FM can be hindered by its modelling of all feature interactions with the same weight, as not all feature interactions are equally useful and predictive. For example, the interactions with useless features may even introduce noises and adversely degrade the performance. In this work, we improve FM by discriminating the importance of different feature interactions. We propose a novel model named Attentional Factorization Machine (AFM), which learns the importance of each feature interaction from data via a neural attention network. Extensive experiments on two real-world datasets demonstrate the effectiveness of AFM. Empirically, it is shown on regression task AFM betters FM with a 8.6% relative improvement, and consistently outperforms the state-of-the-art deep learning methods Wide&Deep [Cheng et al., 2016] and DeepCross [Shan et al., 2016] with a much simpler structure and fewer model parameters. Our implementation of AFM is publicly available at: https://github.com/hexiangnan/attentional_factorization_machine

scholarly journals Interaction-Aware Factorization Machines for Recommender Systems

2019 ◽  
Vol 33 ◽  
pp. 3804-3811 ◽  
Author(s):  
Fuxing Hong ◽  
Dongbo Huang ◽  
Ge Chen
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Interaction Effect ◽  
State Of The Art ◽  
Feature Interaction ◽  
Learning Approach ◽  
Field Interaction ◽  
Feature Interactions ◽  
Factorization Machine ◽  
Novel Model

Factorization Machine (FM) is a widely used supervised learning approach by effectively modeling of feature interactions. Despite the successful application of FM and its many deep learning variants, treating every feature interaction fairly may degrade the performance. For example, the interactions of a useless feature may introduce noises; the importance of a feature may also differ when interacting with different features. In this work, we propose a novel model named Interaction-aware Factorization Machine (IFM) by introducing Interaction-Aware Mechanism (IAM), which comprises the feature aspect and the field aspect, to learn flexible interactions on two levels. The feature aspect learns feature interaction importance via an attention network while the field aspect learns the feature interaction effect as a parametric similarity of the feature interaction vector and the corresponding field interaction prototype. IFM introduces more structured control and learns feature interaction importance in a stratified manner, which allows for more leverage in tweaking the interactions on both feature-wise and field-wise levels. Besides, we give a more generalized architecture and propose Interaction-aware Neural Network (INN) and DeepIFM to capture higher-order interactions. To further improve both the performance and efficiency of IFM, a sampling scheme is developed to select interactions based on the field aspect importance. The experimental results from two well-known datasets show the superiority of the proposed models over the state-of-the-art methods.

scholarly journals Identifying Ethnics of People through Face Recognition: A Deep CNN Approach

2020 ◽  
Vol 2020 ◽  
pp. 1-7
Author(s):  
Ahmed Jawad A. AlBdairi ◽  
Zhu Xiao ◽  
Mohammed Alghaili
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Face Recognition ◽  
Convolutional Neural Network ◽  
State Of The Art ◽  
Research Community ◽  
Facial Features ◽  
New Model ◽  
Image Dataset ◽  
Deep Cnn

The interest in face recognition studies has grown rapidly in the last decade. One of the most important problems in face recognition is the identification of ethnics of people. In this study, a new deep learning convolutional neural network is designed to create a new model that can recognize the ethnics of people through their facial features. The new dataset for ethnics of people consists of 3141 images collected from three different nationalities. To the best of our knowledge, this is the first image dataset collected for the ethnics of people and that dataset will be available for the research community. The new model was compared with two state-of-the-art models, VGG and Inception V3, and the validation accuracy was calculated for each convolutional neural network. The generated models have been tested through several images of people, and the results show that the best performance was achieved by our model with a verification accuracy of 96.9%.

scholarly journals Single-Cell Phenotype Classification Using Deep Convolutional Neural Networks

2016 ◽  
Vol 21 (9) ◽  
pp. 998-1003 ◽  
Author(s):  
Oliver Dürr ◽  
Beate Sick
Keyword(s):  
Machine Learning ◽  
Neural Networks ◽  
Deep Learning ◽  
Single Cell ◽  
Convolutional Neural Networks ◽  
State Of The Art ◽  
Misclassification Rate ◽  
Support Vector ◽  
Learning Methods ◽  
Phenotype Classification

Deep learning methods are currently outperforming traditional state-of-the-art computer vision algorithms in diverse applications and recently even surpassed human performance in object recognition. Here we demonstrate the potential of deep learning methods to high-content screening–based phenotype classification. We trained a deep learning classifier in the form of convolutional neural networks with approximately 40,000 publicly available single-cell images from samples treated with compounds from four classes known to lead to different phenotypes. The input data consisted of multichannel images. The construction of appropriate feature definitions was part of the training and carried out by the convolutional network, without the need for expert knowledge or handcrafted features. We compare our results against the recent state-of-the-art pipeline in which predefined features are extracted from each cell using specialized software and then fed into various machine learning algorithms (support vector machine, Fisher linear discriminant, random forest) for classification. The performance of all classification approaches is evaluated on an untouched test image set with known phenotype classes. Compared to the best reference machine learning algorithm, the misclassification rate is reduced from 8.9% to 6.6%.

scholarly journals Building a Real-Time 2D Lidar Using Deep Learning

2021 ◽  
Vol 2021 ◽  
pp. 1-7
Author(s):  
Nadim Arubai ◽  
Omar Hamdoun ◽  
Assef Jafar
Keyword(s):  
Deep Learning ◽  
Real Time ◽  
Obstacle Avoidance ◽  
Tilt Angle ◽  
State Of The Art ◽  
The State ◽  
Prediction Problem ◽  
Learning Methods ◽  
Depth Prediction ◽  
Avoidance Problem

Applying deep learning methods, this paper addresses depth prediction problem resulting from single monocular images. A vector of distances is predicted instead of a whole image matrix. A vector-only prediction decreases training overhead and prediction periods and requires less resources (memory, CPU). We propose a module which is more time efficient than the state-of-the-art modules ResNet, VGG, FCRN, and DORN. We enhanced the network results by training it on depth vectors from other levels (we get a new level by changing the Lidar tilt angle). The predicted results give a vector of distances around the robot, which is sufficient for the obstacle avoidance problem and many other applications.

scholarly journals PconsC4: fast, free, easy, and accurate contact predictions

2018 ◽  
Author(s):  
Mirco Michel ◽  
David Menéndez Hurtado ◽  
Arne Elofsson
Keyword(s):  
Deep Learning ◽  
State Of The Art ◽  
Prediction Methods ◽  
Coupling Analysis ◽  
Learning Methods ◽  
Contact Prediction ◽  
Residue Contact ◽  
Direct Coupling Analysis ◽  
Computationally Expensive ◽  
Contact Predictions

AbstractMotivationResidue contact prediction was revolutionized recently by the introduction of direct coupling analysis (DCA). Further improvements, in particular for small families, have been obtained by the combination of DCA and deep learning methods. However, existing deep learning contact prediction methods often rely on a number of external programs and are therefore computationally expensive.ResultsHere, we introduce a novel contact predictor, PconsC4, which performs on par with state of the art methods. PconsC4 is heavily optimized, does not use any external programs and therefore is significantly faster and easier to use than other methods.AvailabilityPconsC4 is freely available under the GPL license from https://github.com/ElofssonLab/PconsC4. Installation is easy using the pip command and works on any system with Python 3.5 or later and a modern GCC [email protected]

scholarly journals A Survey of State-of-the-art: Deep Learning Methods on Recommender System

2017 ◽  
Vol 162 (10) ◽  
pp. 17-22 ◽  
Author(s):  
Basiliyos Tilahun ◽  
Charles Awono ◽  
Bernabe Batchakui
Keyword(s):  
Deep Learning ◽  
Recommender System ◽  
State Of The Art ◽  
Learning Methods

scholarly journals Tri-net for Semi-Supervised Deep Learning

2018 ◽  
Author(s):  
Dong-Dong Chen ◽  
Wei Wang ◽  
Wei Gao ◽  
Zhi-Hua Zhou
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Deep Learning ◽  
Error Rate ◽  
Deep Neural Network ◽  
Deep Neural Networks ◽  
State Of The Art ◽  
Fine Tuning ◽  
Learning Methods ◽  
Model Initialization

Deep neural networks have witnessed great successes in various real applications, but it requires a large number of labeled data for training. In this paper, we propose tri-net, a deep neural network which is able to use massive unlabeled data to help learning with limited labeled data. We consider model initialization, diversity augmentation and pseudo-label editing simultaneously. In our work, we utilize output smearing to initialize modules, use fine-tuning on labeled data to augment diversity and eliminate unstable pseudo-labels to alleviate the influence of suspicious pseudo-labeled data. Experiments show that our method achieves the best performance in comparison with state-of-the-art semi-supervised deep learning methods. In particular, it achieves 8.30% error rate on CIFAR-10 by using only 4000 labeled examples.

scholarly journals An Input-aware Factorization Machine for Sparse Prediction

2019 ◽  
Author(s):  
Yantao Yu ◽  
Zhen Wang ◽  
Bo Yuan
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Real World ◽  
State Of The Art ◽  
Overall Performance ◽  
Factorization Machine ◽  
The Impact ◽  
Novel Model ◽  
Individual Input ◽  
Better Than

Factorization machines (FMs) are a class of general predictors working effectively with sparse data, which represents features using factorized parameters and weights. However, the accuracy of FMs can be adversely affected by the fixed representation trained for each feature, as the same feature is usually not equally predictive and useful in different instances. In fact, the inaccurate representation of features may even introduce noise and degrade the overall performance. In this work, we improve FMs by explicitly considering the impact of individual input upon the representation of features. We propose a novel model named \textit{Input-aware Factorization Machine} (IFM), which learns a unique input-aware factor for the same feature in different instances via a neural network. Comprehensive experiments on three real-world recommendation datasets are used to demonstrate the effectiveness and mechanism of IFM. Empirical results indicate that IFM is significantly better than the standard FM model and consistently outperforms four state-of-the-art deep learning based methods.

A Survey on Deep Learning for Human Activity Recognition

2022 ◽  
Vol 54 (8) ◽  
pp. 1-34
Author(s):  
Fuqiang Gu ◽  
Mu-Huan Chung ◽  
Mark Chignell ◽  
Shahrokh Valaee ◽  
Baoding Zhou ◽  
...  
Keyword(s):  
Deep Learning ◽  
Activity Recognition ◽  
Human Activity ◽  
Smart Home ◽  
State Of The Art ◽  
Human Activity Recognition ◽  
Learning Methods ◽  
Machine Learning Methods ◽  
Comprehensive Survey ◽  
Conventional Machine

Human activity recognition is a key to a lot of applications such as healthcare and smart home. In this study, we provide a comprehensive survey on recent advances and challenges in human activity recognition (HAR) with deep learning. Although there are many surveys on HAR, they focused mainly on the taxonomy of HAR and reviewed the state-of-the-art HAR systems implemented with conventional machine learning methods. Recently, several works have also been done on reviewing studies that use deep models for HAR, whereas these works cover few deep models and their variants. There is still a need for a comprehensive and in-depth survey on HAR with recently developed deep learning methods.

Sign in / Sign up

Export Citation Format

Share Document