scholarly journals An Input-aware Factorization Machine for Sparse Prediction

2019 ◽  
Author(s):  
Yantao Yu ◽  
Zhen Wang ◽  
Bo Yuan
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Real World ◽  
State Of The Art ◽  
Overall Performance ◽  
Factorization Machine ◽  
The Impact ◽  
Novel Model ◽  
Individual Input ◽  
Better Than

Factorization machines (FMs) are a class of general predictors working effectively with sparse data, which represents features using factorized parameters and weights. However, the accuracy of FMs can be adversely affected by the fixed representation trained for each feature, as the same feature is usually not equally predictive and useful in different instances. In fact, the inaccurate representation of features may even introduce noise and degrade the overall performance. In this work, we improve FMs by explicitly considering the impact of individual input upon the representation of features. We propose a novel model named \textit{Input-aware Factorization Machine} (IFM), which learns a unique input-aware factor for the same feature in different instances via a neural network. Comprehensive experiments on three real-world recommendation datasets are used to demonstrate the effectiveness and mechanism of IFM. Empirical results indicate that IFM is significantly better than the standard FM model and consistently outperforms four state-of-the-art deep learning based methods.

scholarly journals Interaction-Aware Factorization Machines for Recommender Systems

2019 ◽  
Vol 33 ◽  
pp. 3804-3811 ◽  
Author(s):  
Fuxing Hong ◽  
Dongbo Huang ◽  
Ge Chen
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Interaction Effect ◽  
State Of The Art ◽  
Feature Interaction ◽  
Learning Approach ◽  
Field Interaction ◽  
Feature Interactions ◽  
Factorization Machine ◽  
Novel Model

Factorization Machine (FM) is a widely used supervised learning approach by effectively modeling of feature interactions. Despite the successful application of FM and its many deep learning variants, treating every feature interaction fairly may degrade the performance. For example, the interactions of a useless feature may introduce noises; the importance of a feature may also differ when interacting with different features. In this work, we propose a novel model named Interaction-aware Factorization Machine (IFM) by introducing Interaction-Aware Mechanism (IAM), which comprises the feature aspect and the field aspect, to learn flexible interactions on two levels. The feature aspect learns feature interaction importance via an attention network while the field aspect learns the feature interaction effect as a parametric similarity of the feature interaction vector and the corresponding field interaction prototype. IFM introduces more structured control and learns feature interaction importance in a stratified manner, which allows for more leverage in tweaking the interactions on both feature-wise and field-wise levels. Besides, we give a more generalized architecture and propose Interaction-aware Neural Network (INN) and DeepIFM to capture higher-order interactions. To further improve both the performance and efficiency of IFM, a sampling scheme is developed to select interactions based on the field aspect importance. The experimental results from two well-known datasets show the superiority of the proposed models over the state-of-the-art methods.

scholarly journals Attentional Factorization Machines: Learning the Weight of Feature Interactions via Attention Networks

2017 ◽  
Author(s):  
Jun Xiao ◽  
Hao Ye ◽  
Xiangnan He ◽  
Hanwang Zhang ◽  
Fei Wu ◽  
...  
Keyword(s):  
Deep Learning ◽  
State Of The Art ◽  
Feature Interaction ◽  
Model Parameters ◽  
Learning Approach ◽  
Attention Networks ◽  
Feature Interactions ◽  
Factorization Machine ◽  
Real World Datasets ◽  
Novel Model

Factorization Machines (FMs) are a supervised learning approach that enhances the linear regression model by incorporating the second-order feature interactions. Despite effectiveness, FM can be hindered by its modelling of all feature interactions with the same weight, as not all feature interactions are equally useful and predictive. For example, the interactions with useless features may even introduce noises and adversely degrade the performance. In this work, we improve FM by discriminating the importance of different feature interactions. We propose a novel model named Attentional Factorization Machine (AFM), which learns the importance of each feature interaction from data via a neural attention network. Extensive experiments on two real-world datasets demonstrate the effectiveness of AFM. Empirically, it is shown on regression task AFM betters FM with a 8.6% relative improvement, and consistently outperforms the state-of-the-art deep learning methods Wide&Deep [Cheng et al., 2016] and DeepCross [Shan et al., 2016] with a much simpler structure and fewer model parameters. Our implementation of AFM is publicly available at: https://github.com/hexiangnan/attentional_factorization_machine

A Novel Hybrid Deep Multi-Criteria Model for Recommender System

2021 ◽  
Author(s):  
Nour Salim Nassar
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Recommender Systems ◽  
Recommender System ◽  
Real World ◽  
Deep Neural Network ◽  
Recommendation Systems ◽  
The Other ◽  
Learning Techniques ◽  
Novel Model

Abstract Recommender systems are everywhere books, products, movies, and more. Traditional recommender systems typically use a single criterion in the recommendation, while studies have shown that multi-criteria recommending is more accurate. Novel deep learning techniques have produced remarkable achievements in many fields. The use of such techniques in recommendation systems has started to get attention recently, and several models of recommendation have been proposed based on deep learning. However, there is still no work for using deep learning in hybrid multi-criteria recommender systems. In this work, a model for a hybrid deep multi-criteria recommender system was presented. The model mainly includes two major parts: In the first one, the model obtains the user ID, item ID, and the item metadata to be used as input to a deep neural network in order to predict the criteria ratings. In the second part, the obtained ratings act as an input to another deep neural network, where the overall rating is predicted. Our experiments were conducted on a real-world dataset. They demonstrated the superiority of the proposed novel model over the other models in all measures used to evaluate performance. This indicates the successful use of hybrid deep multi-criteria in the recommendation systems.

scholarly journals Neural Layer Bypassing Network

2021 ◽  
Author(s):  
Amogh Palasamudram
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Deep Learning ◽  
Image Classification ◽  
Network Architecture ◽  
Classification Models ◽  
Learning Models ◽  
Neural Network Architecture ◽  
Overall Performance ◽  
The Impact

<p>This research aims to introduce and evaluate a new neural network architecture to improve the speed and effectiveness of forward propagation in neural networks: the Neural Layer Bypassing Network (NLBN). The theory and workings of this architecture have been explained in this research paper, along with comparisons to other methods of increasing the efficacy of deep learning models. This research also includes code examples with 3 image classification models trained on different datasets and analyses the impact of the NLBN architecture on forward propagation. It was found that this architecture increases the speed of forward propagation and tends to slightly decrease the accuracy of the model. However, it takes longer to train and takes more memory. All in all, this architecture is a potential foundation for using deep learning to teach deep learning models to be more efficient. This includes skipping and re-propagating through layers to improve the overall performance of a model.</p><div><br></div>

scholarly journals An Interactive Model of Target and Context for Aspect-Level Sentiment Classification

2019 ◽  
Vol 2019 ◽  
pp. 1-8 ◽  
Author(s):  
Hu Han ◽  
Guoli Liu ◽  
Jianwu Dang
Keyword(s):  
Neural Network ◽  
Neural Network Model ◽  
State Of The Art ◽  
Sentiment Classification ◽  
Target Information ◽  
Context Modelling ◽  
The Right ◽  
The Impact ◽  
Better Than

Aspect-level sentiment classification aims to identify the sentiment polarity of a review expressed toward a target. In recent years, neural network-based methods have achieved success in aspect-level sentiment classification, and these methods fall into two types: the first takes the target information into account for context modelling, and the second models the context without considering the target information. It is concluded that the former is better than the latter. However, most of the target-related models just focus on the impact of the target on context modelling, while ignoring the role of context in target modelling. In this study, we introduce an interactive neural network model named LT-T-TR, which divided a review into three parts: the left context with target phrase, the target phrase, and the right context with target phrase. And the interaction between the left/right context and the target phrase is utilized by an attention mechanism to learn the representations of the left/right context and the target phrase separately. As a result, the most important words in the left/right context or in the target phrase are captured, and the results on laptop and restaurant datasets demonstrate that our model outperforms the state-of-the-art methods.

scholarly journals A Dual Input-aware Factorization Machine for CTR Prediction

2020 ◽  
Author(s):  
Wantong Lu ◽  
Yantao Yu ◽  
Yongzhe Chang ◽  
Zhen Wang ◽  
Chenhui Li ◽  
...  
Keyword(s):  
Real World ◽  
Predictive Power ◽  
State Of The Art ◽  
Model Parameters ◽  
Feature Vectors ◽  
Feature Representations ◽  
Factorization Machine ◽  
Click Through Rate ◽  
Novel Model ◽  
Original Feature

Factorization Machines (FMs) refer to a class of general predictors working with real valued feature vectors, which are well-known for their ability to estimate model parameters under significant sparsity and have found successful applications in many areas such as the click-through rate (CTR) prediction. However, standard FMs only produce a single fixed representation for each feature across different input instances, which may limit the CTR model’s expressive and predictive power. Inspired by the success of Input-aware Factorization Machines (IFMs), which aim to learn more flexible and informative representations of a given feature according to different input instances, we propose a novel model named Dual Input-aware Factorization Machines (DIFMs) that can adaptively reweight the original feature representations at the bit-wise and vector-wise levels simultaneously. Furthermore, DIFMs strategically integrate various components including Multi-Head Self-Attention, Residual Networks and DNNs into a unified end-to-end model. Comprehensive experiments on two real-world CTR prediction datasets show that the DIFM model can outperform several state-of-the-art models consistently.

scholarly journals Audio-Based Drone Detection and Identification Using Deep Learning Techniques with Dataset Enhancement through Generative Adversarial Networks

Sensors ◽  
2021 ◽  
Vol 21 (15) ◽  
pp. 4953
Author(s):  
Sara Al-Emadi ◽  
Abdulla Al-Ali ◽  
Abdulaziz Al-Ali
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Recurrent Neural Network ◽  
Learning Algorithms ◽  
Generative Adversarial Networks ◽  
Generative Adversarial Network ◽  
Adversarial Networks ◽  
Detection And Identification ◽  
Learning Techniques ◽  
The Impact

Drones are becoming increasingly popular not only for recreational purposes but in day-to-day applications in engineering, medicine, logistics, security and others. In addition to their useful applications, an alarming concern in regard to the physical infrastructure security, safety and privacy has arisen due to the potential of their use in malicious activities. To address this problem, we propose a novel solution that automates the drone detection and identification processes using a drone’s acoustic features with different deep learning algorithms. However, the lack of acoustic drone datasets hinders the ability to implement an effective solution. In this paper, we aim to fill this gap by introducing a hybrid drone acoustic dataset composed of recorded drone audio clips and artificially generated drone audio samples using a state-of-the-art deep learning technique known as the Generative Adversarial Network. Furthermore, we examine the effectiveness of using drone audio with different deep learning algorithms, namely, the Convolutional Neural Network, the Recurrent Neural Network and the Convolutional Recurrent Neural Network in drone detection and identification. Moreover, we investigate the impact of our proposed hybrid dataset in drone detection. Our findings prove the advantage of using deep learning techniques for drone detection and identification while confirming our hypothesis on the benefits of using the Generative Adversarial Networks to generate real-like drone audio clips with an aim of enhancing the detection of new and unfamiliar drones.

scholarly journals Assessing the Impact of the Loss Function, Architecture and Image Type for Deep Learning-Based Wildfire Segmentation

2021 ◽  
Vol 11 (15) ◽  
pp. 7046
Author(s):  
Jorge Francisco Ciprián-Sánchez ◽  
Gilberto Ochoa-Ruiz ◽  
Lucile Rossi ◽  
Frédéric Morandini
Keyword(s):  
Deep Learning ◽  
Loss Function ◽  
State Of The Art ◽  
Fire Detection ◽  
Loss Functions ◽  
Wildfire Spread ◽  
Combine Information ◽  
The Impact ◽  
Image Type ◽  
Segmentation Models

Wildfires stand as one of the most relevant natural disasters worldwide, particularly more so due to the effect of climate change and its impact on various societal and environmental levels. In this regard, a significant amount of research has been done in order to address this issue, deploying a wide variety of technologies and following a multi-disciplinary approach. Notably, computer vision has played a fundamental role in this regard. It can be used to extract and combine information from several imaging modalities in regard to fire detection, characterization and wildfire spread forecasting. In recent years, there has been work pertaining to Deep Learning (DL)-based fire segmentation, showing very promising results. However, it is currently unclear whether the architecture of a model, its loss function, or the image type employed (visible, infrared, or fused) has the most impact on the fire segmentation results. In the present work, we evaluate different combinations of state-of-the-art (SOTA) DL architectures, loss functions, and types of images to identify the parameters most relevant to improve the segmentation results. We benchmark them to identify the top-performing ones and compare them to traditional fire segmentation techniques. Finally, we evaluate if the addition of attention modules on the best performing architecture can further improve the segmentation results. To the best of our knowledge, this is the first work that evaluates the impact of the architecture, loss function, and image type in the performance of DL-based wildfire segmentation models.

scholarly journals A Survey on Bias and Fairness in Machine Learning

2021 ◽  
Vol 54 (6) ◽  
pp. 1-35
Author(s):  
Ninareh Mehrabi ◽  
Fred Morstatter ◽  
Nripsuta Saxena ◽  
Kristina Lerman ◽  
Aram Galstyan
Keyword(s):  
Artificial Intelligence ◽  
Machine Learning ◽  
Deep Learning ◽  
Real World ◽  
State Of The Art ◽  
Future Directions ◽  
Discriminatory Behavior ◽  
Real World Applications ◽  
Near Future ◽  
Different Sources

With the widespread use of artificial intelligence (AI) systems and applications in our everyday lives, accounting for fairness has gained significant importance in designing and engineering of such systems. AI systems can be used in many sensitive environments to make important and life-changing decisions; thus, it is crucial to ensure that these decisions do not reflect discriminatory behavior toward certain groups or populations. More recently some work has been developed in traditional machine learning and deep learning that address such challenges in different subdomains. With the commercialization of these systems, researchers are becoming more aware of the biases that these applications can contain and are attempting to address them. In this survey, we investigated different real-world applications that have shown biases in various ways, and we listed different sources of biases that can affect AI applications. We then created a taxonomy for fairness definitions that machine learning researchers have defined to avoid the existing bias in AI systems. In addition to that, we examined different domains and subdomains in AI showing what researchers have observed with regard to unfair outcomes in the state-of-the-art methods and ways they have tried to address them. There are still many future directions and solutions that can be taken to mitigate the problem of bias in AI systems. We are hoping that this survey will motivate researchers to tackle these issues in the near future by observing existing work in their respective fields.

scholarly journals Investigating the Impact of the Bit Depth of Fluorescence-Stained Images on the Performance of Deep Learning-Based Nuclei Instance Segmentation

Diagnostics ◽  
2021 ◽  
Vol 11 (6) ◽  
pp. 967
Author(s):  
Amirreza Mahbod ◽  
Gerald Schaefer ◽  
Christine Löw ◽  
Georg Dorffner ◽  
Rupert Ecker ◽  
...  
Keyword(s):  
Deep Learning ◽  
State Of The Art ◽  
Computer Assisted ◽  
Important Criterion ◽  
Histological Image ◽  
Computer Mediated ◽  
Image Patches ◽  
Processing Techniques ◽  
The Impact ◽  
Instance Segmentation

Nuclei instance segmentation can be considered as a key point in the computer-mediated analysis of histological fluorescence-stained (FS) images. Many computer-assisted approaches have been proposed for this task, and among them, supervised deep learning (DL) methods deliver the best performances. An important criterion that can affect the DL-based nuclei instance segmentation performance of FS images is the utilised image bit depth, but to our knowledge, no study has been conducted so far to investigate this impact. In this work, we released a fully annotated FS histological image dataset of nuclei at different image magnifications and from five different mouse organs. Moreover, by different pre-processing techniques and using one of the state-of-the-art DL-based methods, we investigated the impact of image bit depth (i.e., eight bits vs. sixteen bits) on the nuclei instance segmentation performance. The results obtained from our dataset and another publicly available dataset showed very competitive nuclei instance segmentation performances for the models trained with 8 bit and 16 bit images. This suggested that processing 8 bit images is sufficient for nuclei instance segmentation of FS images in most cases. The dataset including the raw image patches, as well as the corresponding segmentation masks is publicly available in the published GitHub repository.

Sign in / Sign up

Export Citation Format

Share Document