scholarly journals Forecasting Economy-Related Data Utilizing Weight-Constrained Recurrent Neural Networks

Algorithms ◽  
2019 ◽  
Vol 12 (4) ◽  
pp. 85 ◽  
Author(s):  
Ioannis E. Livieris
Keyword(s):  
Neural Networks ◽  
Recurrent Neural Networks ◽  
Training Algorithm ◽  
Economic Data ◽  
Classification Problems ◽  
Useful Knowledge ◽  
Related Data ◽  
Limited Memory ◽  
Proposed Model ◽  
Projection Strategy

During the last few decades, machine learning has constituted a significant tool in extracting useful knowledge from economic data for assisting decision-making. In this work, we evaluate the performance of weight-constrained recurrent neural networks in forecasting economic classification problems. These networks are efficiently trained with a recently-proposed training algorithm, which has two major advantages. Firstly, it exploits the numerical efficiency and very low memory requirements of the limited memory BFGS matrices; secondly, it utilizes a gradient-projection strategy for handling the bounds on the weights. The reported numerical experiments present the classification accuracy of the proposed model, providing empirical evidence that the application of the bounds on the weights of the recurrent neural network provides more stable and reliable learning.

Using recurrent neural networks for probabilistic classification of the processor architecture of executable files

2021 ◽  
Author(s):  
A. A. Gladkikh ◽  
M. O. Komakhin ◽  
A. V. Simankov ◽  
D. A. Uzenkov
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Recurrent Neural Networks ◽  
Software Package ◽  
Performance Metrics ◽  
Digital Data ◽  
Processor Architecture ◽  
Classification Problems ◽  
Processor Architectures ◽  
Previous State

The main problem of applying recurrent neural networks to the problem of classifying processor architectures is that the use of a recurrent neural network is complicated by the lack of blocks that allow memorizing and taking into account the result of work at each next step. To solve this problem, the authors proposed a strategy for using a neural network based on the mechanism of controlled recurrent blocks. Each neuron of such a network has a memory cell, which stores the previous state and several filters. The update filter determines how much information will remain from the previous state and how much will be taken from the previous layer. The reset filter determines how much information about previous states is lost. The purpose of the work is to increase the efficiency of determining the processor architecture by code from executable files running on this processor by creating methods, algorithms and programs that are invariant to constant data (strings, constants, header sections, data sections, indents) contained in executable files. The paper discusses the features of the use of recurrent neural networks on the example of the problem of classifying the processor architecture by executable code from compiled executable files. The features of the machine code of various processor architectures used in modern computing have been briefly considered. The use of recurrent neural networks has been proposed, which have advantages in terms of speed and accuracy in solving classification problems. It is noted that in order to improve the classification results and practical use, it is necessary to provide a larger volume of the training sample for each of the classes, as well as to expand the number of classes. The proposed method based on a neural network with a mechanism of controlled recurrent blocks has been implemented in the software package that allows processing digital data from executable files for various processor architectures, in particular at the initial stage of security audit of embedded systems in order to determine a set of technical means that can be applied to analysis at subsequent stages. Conclusions have been drawn about the results of measuring the performance metrics of the algorithm and the possibility of expanding functionality without making changes to the architecture of the software package.

scholarly journals Building a Deep Learning Model to Generate Human Readable Text Using Recurrent Neural Networks and LSTM

2021 ◽  
Author(s):  
Anasse HANAFI ◽  
Mohammed BOUHORMA ◽  
Lotfi ELAACHAK
Keyword(s):  
Neural Networks ◽  
Recurrent Neural Networks ◽  
Binary Classification ◽  
Generative Models ◽  
Large Field ◽  
Classification Problems ◽  
Classification Tasks ◽  
Multi Class Classification ◽  
Deep Learning Model ◽  
Readable Text

Abstract Machine learning (ML) is a large field of study that overlaps with and inherits ideas from many related fields such as artificial intelligence (AI). The main focus of the field is learning from previous experiences. Classification in ML is a supervised learning method, in which the computer program learns from the data given to it and make new classifications. There are many different types of classification tasks in ML and dedicated approaches to modeling that may be used for each. For example, classification predictive modeling involves assigning a class label to input samples, binary classification refers to predicting one of two classes and multi-class classification involves predicting one of more than two categories. Recurrent Neural Networks (RNNs) are very powerful sequence models for classification problems, however, in this paper, we will use RNNs as generative models, which means they can learn the sequences of a problem and then generate entirely a new sequence for the problem domain, with the hope to better control the output of the generated text, because it is not always possible to learn the exact distribution of the data either implicitly or explicitly.

scholarly journals Relation classification via BERT with piecewise convolution and focal loss

PLoS ONE ◽  
2021 ◽  
Vol 16 (9) ◽  
pp. e0257092
Author(s):  
Jianyi Liu ◽  
Xi Duan ◽  
Ru Zhang ◽  
Youqiang Sun ◽  
Lei Guan ◽  
...  
Keyword(s):  
Neural Networks ◽  
Semantic Information ◽  
Semantic Representation ◽  
Language Model ◽  
Relation Extraction ◽  
Fine Tuning ◽  
Superior Result ◽  
Useful Knowledge ◽  
Proposed Model ◽  
Relation Classification

Recent relation extraction models’ architecture are evolved from the shallow neural networks to natural language model, such as convolutional neural networks or recurrent neural networks to Bert. However, these methods did not consider the semantic information in the sequence or the distance dependence problem, the internal semantic information may contain the useful knowledge which can help relation classification. Focus on these problems, this paper proposed a BERT-based relation classification method. Compare with the existing Bert-based architecture, the proposed model can obtain the internal semantic information between entity pair and solve the distance semantic dependence better. The pre-trained BERT model after fine tuning is used in this paper to abstract the semantic representation of sequence, then adopt the piecewise convolution to obtain semantic information which influence the extraction results. Compare with the existing methods, the proposed method can achieve a better accuracy on relational extraction task because of the internal semantic information extracted in the sequence. While, the generalization ability is still a problem that cannot be ignored, and the numbers of the relationships are difference between different categories. In this paper, the focal loss function is adopted to solve this problem by assigning a heavy weight to less number or hard classify categories. Finally, comparing with the existing methods, the F1 metric of the proposed method can reach a superior result 89.95% on the SemEval-2010 Task 8 dataset.

scholarly journals Towards Non-Saturating Recurrent Units for Modelling Long-Term Dependencies

2019 ◽  
Vol 33 ◽  
pp. 3280-3287 ◽  
Author(s):  
Sarath Chandar ◽  
Chinnadhurai Sankar ◽  
Eugene Vorontsov ◽  
Samira Ebrahimi Kahou ◽  
Yoshua Bengio
Keyword(s):  
Neural Networks ◽  
Real World ◽  
Recurrent Neural Networks ◽  
Sequence Length ◽  
Activation Functions ◽  
Memory Mechanism ◽  
Proposed Model ◽  
Recurrent Architecture ◽  
Transition Operators

Modelling long-term dependencies is a challenge for recurrent neural networks. This is primarily due to the fact that gradients vanish during training, as the sequence length increases. Gradients can be attenuated by transition operators and are attenuated or dropped by activation functions. Canonical architectures like LSTM alleviate this issue by skipping information through a memory mechanism. We propose a new recurrent architecture (Non-saturating Recurrent Unit; NRU) that relies on a memory mechanism but forgoes both saturating activation functions and saturating gates, in order to further alleviate vanishing gradients. In a series of synthetic and real world tasks, we demonstrate that the proposed model is the only model that performs among the top 2 models across all tasks with and without long-term dependencies, when compared against a range of other architectures.

scholarly journals Suggestion and invention of recipes using bi-directional LSTMs-based frameworks

2021 ◽  
Vol 3 (5) ◽  
Author(s):  
Sai Nikhil Rao Gona ◽  
Himamsu Marellapudi
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Loss Function ◽  
Recurrent Neural Networks ◽  
Network Architecture ◽  
Loss Functions ◽  
Neural Network Architecture ◽  
Calorie Content ◽  
Proposed Model ◽  
Variational Autoencoder

AbstractChoosing which recipe to eat and which recipe to avoid isn’t that simple for anyone. It takes strenuous efforts and a lot of time for people to calculate the number of calories and P.H level of the dish. In this paper, we propose an ensemble neural network architecture that suggests recipes based on the taste of the person, P.H level and calorie content of the recipes. We also propose a bi-directional LSTMs-based variational autoencoder for generating new recipes. We have ensembled three bi-directional LSTM-based recurrent neural networks which can classify the recipes based on the taste of the person, P.H level of the recipe and calorie content of the recipe. The proposed model also predicts the taste ratings of the recipes for which we proposed a custom loss function which gave better results than the standard loss functions and the model also predicts the calorie content of the recipes. The bi-directional LSTMs-based variational autoencoder after being trained with the recipes which are fit for the person generates new recipes from the existing recipes. After training and testing the recurrent neural networks and the variational autoencoder, we have tested the model with 20 new recipes and got overwhelming results in the experimentation, the variational autoencoders generated a couple of new recipes, which are healthy to the specific person and will be liked by the specific person.

Sign in / Sign up

Export Citation Format

Share Document