Accurate and Interpretable Factorization Machines

2019 ◽  
Vol 33 ◽  
pp. 4139-4146 ◽  
Author(s):  
Liang Lan ◽  
Yu Geng
Keyword(s):  
Scoring System ◽  
Time Complexity ◽  
Feature Representation ◽  
Feature Mapping ◽  
Feature Interactions ◽  
Proposed Model ◽  
Novel Method ◽  
Ranking Problems ◽  
New Feature ◽  
Low Dimensional

Factorization Machines (FMs), a general predictor that can efficiently model high-order feature interactions, have been widely used for regression, classification and ranking problems. However, despite many successful applications of FMs, there are two main limitations of FMs: (1) FMs consider feature interactions among input features by using only polynomial expansion which fail to capture complex nonlinear patterns in data. (2) Existing FMs do not provide interpretable prediction to users. In this paper, we present a novel method named Subspace Encoding Factorization Machines (SEFM) to overcome these two limitations by using non-parametric subspace feature mapping. Due to the high sparsity of new feature representation, our proposed method achieves the same time complexity as the standard FMs but can capture more complex nonlinear patterns. Moreover, since the prediction score of our proposed model for a sample is a sum of contribution scores of the bins and grid cells that this sample lies in low-dimensional subspaces, it works similar like a scoring system which only involves data binning and score addition. Therefore, our proposed method naturally provides interpretable prediction. Our experimental results demonstrate that our proposed method efficiently provides accurate and interpretable prediction.

A novel multi-stage ensemble model with multiple K-means-based selective undersampling: An application in credit scoring

2021 ◽  
Vol 40 (5) ◽  
pp. 9471-9484
Author(s):  
Yilun Jin ◽  
Yanan Liu ◽  
Wenyu Zhang ◽  
Shuai Zhang ◽  
Yu Lou
Keyword(s):  
Machine Learning ◽  
Predictive Accuracy ◽  
Credit Scoring ◽  
Imbalanced Data ◽  
Ensemble Model ◽  
Selective Sampling ◽  
Machine Learning Methods ◽  
Multi Stage ◽  
Proposed Model ◽  
New Feature

With the advancement of machine learning, credit scoring can be performed better. As one of the widely recognized machine learning methods, ensemble learning has demonstrated significant improvements in the predictive accuracy over individual machine learning models for credit scoring. This study proposes a novel multi-stage ensemble model with multiple K-means-based selective undersampling for credit scoring. First, a new multiple K-means-based undersampling method is proposed to deal with the imbalanced data. Then, a new selective sampling mechanism is proposed to select the better-performing base classifiers adaptively. Finally, a new feature-enhanced stacking method is proposed to construct an effective ensemble model by composing the shortlisted base classifiers. In the experiments, four datasets with four evaluation indicators are used to evaluate the performance of the proposed model, and the experimental results prove the superiority of the proposed model over other benchmark models.

scholarly journals UAV Image Multi-Labeling with Data-Efficient Transformers

2021 ◽  
Vol 11 (9) ◽  
pp. 3974
Author(s):  
Laila Bashmal ◽  
Yakoub Bazi ◽  
Mohamad Mahmoud Al Rahhal ◽  
Haikel Alhichri ◽  
Naif Al Ajlan
Keyword(s):  
Data Augmentation ◽  
Feature Representation ◽  
Aerial Image ◽  
Remote Sensing Images ◽  
Training Set ◽  
Proposed Model ◽  
Class Labels ◽  
Using Data ◽  
Uav Image

In this paper, we present an approach for the multi-label classification of remote sensing images based on data-efficient transformers. During the training phase, we generated a second view for each image from the training set using data augmentation. Then, both the image and its augmented version were reshaped into a sequence of flattened patches and then fed to the transformer encoder. The latter extracts a compact feature representation from each image with the help of a self-attention mechanism, which can handle the global dependencies between different regions of the high-resolution aerial image. On the top of the encoder, we mounted two classifiers, a token and a distiller classifier. During training, we minimized a global loss consisting of two terms, each corresponding to one of the two classifiers. In the test phase, we considered the average of the two classifiers as the final class labels. Experiments on two datasets acquired over the cities of Trento and Civezzano with a ground resolution of two-centimeter demonstrated the effectiveness of the proposed model.

scholarly journals Multiview Discriminative Geometry Preserving Projection for Image Classification

2014 ◽  
Vol 2014 ◽  
pp. 1-11 ◽  
Author(s):  
Ziqiang Wang ◽  
Xia Sun ◽  
Lijun Sun ◽  
Yuchun Huang
Keyword(s):  
Image Classification ◽  
Learning Algorithm ◽  
Subspace Learning ◽  
Visual Features ◽  
Expression Recognition ◽  
Single View ◽  
Discrimination Information ◽  
New Feature ◽  
Low Dimensional ◽  
Conventional Solution

In many image classification applications, it is common to extract multiple visual features from different views to describe an image. Since different visual features have their own specific statistical properties and discriminative powers for image classification, the conventional solution for multiple view data is to concatenate these feature vectors as a new feature vector. However, this simple concatenation strategy not only ignores the complementary nature of different views, but also ends up with “curse of dimensionality.” To address this problem, we propose a novel multiview subspace learning algorithm in this paper, named multiview discriminative geometry preserving projection (MDGPP) for feature extraction and classification. MDGPP can not only preserve the intraclass geometry and interclass discrimination information under a single view, but also explore the complementary property of different views to obtain a low-dimensional optimal consensus embedding by using an alternating-optimization-based iterative algorithm. Experimental results on face recognition and facial expression recognition demonstrate the effectiveness of the proposed algorithm.

scholarly journals A Weakly Supervised Academic Search Model Based on Knowledge-Enhanced Feature Representation

2021 ◽  
Vol 2021 ◽  
pp. 1-12
Author(s):  
Mingying Xu ◽  
Junping Du ◽  
Feifei Kou ◽  
Meiyu Liang ◽  
Xin Xu ◽  
...  
Keyword(s):  
Big Data ◽  
Internet Of Things ◽  
Feature Representation ◽  
Search Model ◽  
Model Based ◽  
Proposed Model ◽  
Cloud Server ◽  
Weakly Supervised ◽  
Internet Of Things Technology ◽  
Academic Search

Internet of Things search has great potential applications with the rapid development of Internet of Things technology. Combining Internet of Things technology and academic search to build academic search framework based on Internet of Things is an effective solution to realize massive academic resource search. Recently, the academic big data has been characterized by a large number of types and spanning many fields. The traditional web search technology is no longer suitable for the search environment of academic big data. Thus, this paper designs academic search framework based on Internet of Things Technology. In order to alleviate the pressure of the cloud server processing massive academic big data, the edge server is introduced to clean and remove the redundancy of the data to form a clean data for further analysis and processing by the cloud server. Edge computing network effectively makes up for the deficiency of cloud computing in the conditions of distributed and high concurrent access, reduces long-distance data transmission, and improves the quality of network user experience. For Academic Search, this paper proposes a novel weakly supervised academic search model based on knowledge-enhanced feature representation. The proposed model can relieve high cost of acquisition of manually labeled data by obtaining a lot of pseudolabeled data and consider word-level interactive matching and sentence-level semantic matching for more accurate matching in the process of academic search. The experimental result on academic datasets demonstrate that the performance of the proposed model is much better than that of the existing methods.

scholarly journals Unsupervised Text Feature Learning via Deep Variational Auto-encoder

2020 ◽  
Vol 49 (3) ◽  
pp. 421-437
Author(s):  
Genggeng Liu ◽  
Lin Xie ◽  
Chi-Hua Chen
Keyword(s):  
Dimensionality Reduction ◽  
High Dimensional Data ◽  
Image Data ◽  
Original Data ◽  
Feature Representation ◽  
High Dimensional ◽  
Learning To Learn ◽  
Text Feature ◽  
Reduction Methods ◽  
Low Dimensional

Dimensionality reduction plays an important role in the data processing of machine learning and data mining, which makes the processing of high-dimensional data more efficient. Dimensionality reduction can extract the low-dimensional feature representation of high-dimensional data, and an effective dimensionality reduction method can not only extract most of the useful information of the original data, but also realize the function of removing useless noise. The dimensionality reduction methods can be applied to all types of data, especially image data. Although the supervised learning method has achieved good results in the application of dimensionality reduction, its performance depends on the number of labeled training samples. With the growing of information from internet, marking the data requires more resources and is more difficult. Therefore, using unsupervised learning to learn the feature of data has extremely important research value. In this paper, an unsupervised multilayered variational auto-encoder model is studied in the text data, so that the high-dimensional feature to the low-dimensional feature becomes efficient and the low-dimensional feature can retain mainly information as much as possible. Low-dimensional feature obtained by different dimensionality reduction methods are used to compare with the dimensionality reduction results of variational auto-encoder (VAE), and the method can be significantly improved over other comparison methods.

scholarly journals A Novel Method to Detect Inner Emotion States of Human using Artificial Neural Networks

2020 ◽  
Vol 8 (6) ◽  
pp. 5820-5825
Keyword(s):  
Physiological Signals ◽  
Classification Rate ◽  
Discrete Emotion ◽  
Novel Approach ◽  
Proposed Model ◽  
Depression Detection ◽  
Novel Method ◽  
Artificial Neural ◽  
Ann Classifier ◽  
Performance Rate

Human computer interaction is a fast growing area of research where in the physiological signals are used to identify human emotion states. Identifying emotion states can be done using various approaches. One such approach which gained interest of research is through physiological signals using EEG. In the present work, a novel approach is proposed to elicit emotion states using 3-D Video-audio stimuli. Around 66 subjects were involved during data acquisition using 32 channel Enobio device. FIR filter is used to preprocess the acquired raw EEG signals. The desired frequency bands like alpha, delta, beta and theta are extracted using 8-level DWT. The statistical features, Hurst exponential, entropy, power, energy, differential entropy of each bands are computed. Artificial Neural network is implemented using Sequential Keras model and applied on the extracted features to classify in to four classes (HVLA, HVHA, LVHA and LVLA) and eight discrete emotion states like clam, relax, happy, joy, sad, fear, tensed and bored. The performance of ANN classifier found to perform better for 4- classes than 8-classes with a classification rate of 90.835% and 74.0446% respectively. The proposed model achieved better performance rate in detecting discrete emotion states. This model can be used to build applications on health like stress / depression detection and on entertainment to build emotional DJ.

Improved Privacy

2011 ◽  
pp. 295-316
Author(s):  
K. Abumani ◽  
R. Nedunchezhian
Keyword(s):  
Data Mining ◽  
Decision Making ◽  
Empirical Analysis ◽  
Time Complexity ◽  
Strategic Decision ◽  
Strategic Decision Making ◽  
Sensitive Information ◽  
Data Mining Techniques ◽  
Novel Method ◽  
Mining Tools

Data mining techniques have been widely used for extracting non-trivial information from massive amounts of data. They help in strategic decision-making as well as many more applications. However, data mining also has a few demerits apart from its usefulness. Sensitive information contained in the database may be brought out by the data mining tools. Different approaches are being utilized to hide the sensitive information. The proposed work in this article applies a novel method to access the generating transactions with minimum effort from the transactional database. It helps in reducing the time complexity of any hiding algorithm. The theoretical and empirical analysis of the algorithm shows that hiding of data using this proposed work performs association rule hiding quicker than other algorithms.

scholarly journals Augmented Context-Based Conceptual User Modeling for Personalized Recommendation System in Online Social Networks

2020 ◽  
Vol 14 (3) ◽  
pp. 1-19
Author(s):  
Ammar Alnahhas ◽  
Bassel Alkhatib
Keyword(s):  
Social Networks ◽  
Online Social Networks ◽  
Recommendation System ◽  
Contextual Information ◽  
Detailed Comparison ◽  
Personalized Recommendation ◽  
User Models ◽  
Proposed Model ◽  
Recommendation Algorithms ◽  
Novel Method

As the data on the online social networks is getting larger, it is important to build personalized recommendation systems that recommend suitable content to users, there has been much research in this field that uses conceptual representations of text to match user models with best content. This article presents a novel method to build a user model that depends on conceptual representation of text by using ConceptNet concepts that exceed the named entities to include the common-sense meaning of words and phrases. The model includes the contextual information of concepts as well, the authors also show a novel method to exploit the semantic relations of the knowledge base to extend user models, the experiment shows that the proposed model and associated recommendation algorithms outperform all previous methods as a detailed comparison shows in this article.

A versatile taxonomy of low-dimensional vortex models for unsteady aerodynamics

2018 ◽  
Vol 858 ◽  
pp. 917-948 ◽  
Author(s):  
Darwin Darakananda ◽  
Jeff D. Eldredge
Keyword(s):  
Large Scale ◽  
Bluff Body ◽  
Point Vortex ◽  
Unsteady Aerodynamics ◽  
Shear Layers ◽  
Point Vortices ◽  
The Body ◽  
Vortex Sheets ◽  
Proposed Model ◽  
Low Dimensional

Inviscid vortex models have been demonstrated to capture the essential physics of massively separated flows past aerodynamic surfaces, but they become computationally expensive as coherent vortex structures are formed and the wake is developed. In this work, we present a two-dimensional vortex model in which vortex sheets represent shear layers that separate from sharp edges of the body and point vortices represent the rolled-up cores of these shear layers and the other coherent vortices in the wake. We develop a circulation transfer procedure that enables each vortex sheet to feed its circulation into a point vortex instead of rolling up. This procedure reduces the number of computational elements required to capture the dynamics of vortex formation while eliminating the spurious force that manifests when transferring circulation between vortex elements. By tuning the rate at which the vortex sheets are siphoned into the point vortices, we can adjust the balance between the model’s dimensionality and dynamical richness, enabling it to span the entire taxonomy of inviscid vortex models. This hybrid model can capture the development and subsequent shedding of the starting vortices with insignificant wall-clock time and remain sufficiently low-dimensional to simulate long-time-horizon events such as periodic bluff-body shedding. We demonstrate the viability of the method by modelling the impulsive translation of a wing at various fixed angles of attack, pitch-up manoeuvres that linearly increase the angle of attack from $0^{\circ }$ to $90^{\circ }$, and oscillatory pitching and heaving. We show that the proposed model correctly predicts the dynamics of large-scale vortical structures in the flow by comparing the distributions of vorticity and force responses from results of the proposed model with a model using only vortex sheets and, in some cases, high-fidelity viscous simulation.

scholarly journals A Novel Method for Intelligent Fault Diagnosis of Bearing Based on Capsule Neural Network

Complexity ◽  
2019 ◽  
Vol 2019 ◽  
pp. 1-17 ◽  
Author(s):  
Zhijian Wang ◽  
Likang Zheng ◽  
Wenhua Du ◽  
Wenan Cai ◽  
Jie Zhou ◽  
...  
Keyword(s):  
Neural Network ◽  
Fault Diagnosis ◽  
Weight Coefficient ◽  
Convergence Condition ◽  
Intelligent Fault Diagnosis ◽  
Wind Turbine Gearbox ◽  
Time Frequency ◽  
Proposed Model ◽  
Novel Method ◽  
Relationship Of

In the era of big data, data-driven methods mainly based on deep learning have been widely used in the field of intelligent fault diagnosis. Traditional neural networks tend to be more subjective when classifying fault time-frequency graphs, such as pooling layer, and ignore the location relationship of features. The newly proposed neural network named capsules network takes into account the size and location of the image. Inspired by this, capsules network combined with the Xception module (XCN) is applied in intelligent fault diagnosis, so as to improve the classification accuracy of intelligent fault diagnosis. Firstly, the fault time-frequency graphs are obtained by wavelet time-frequency analysis. Then the time-frequency graphs data which are adjusted the pixel size are input into XCN for training. In order to accelerate the learning rate, the parameters which have bigger change are punished by cost function in the process of training. After the operation of dynamic routing, the length of the capsule is used to classify the types of faults and get the classification of loss. Then the longest capsule is used to reconstruct fault time-frequency graphs which are used to measure the reconstruction of loss. In order to determine the convergence condition, the three losses are combined through the weight coefficient. Finally, the proposed model and the traditional methods are, respectively, trained and tested under laboratory conditions and actual wind turbine gearbox conditions to verify the classification ability and reliable ability.

Sign in / Sign up

Export Citation Format

Share Document