scholarly journals Music emotion recognition using recurrent neural networks and pretrained models

2021 ◽  
Author(s):  
Jacek Grekow
Keyword(s):  
Neural Networks ◽  
Recurrent Neural Networks ◽  
Short Term Memory ◽  
Recurrent Network ◽  
Sequential Data ◽  
Circumplex Model ◽  
Learning Networks ◽  
Audio Features ◽  
Svm Algorithm ◽  
Audio Feature Extraction

AbstractThe article presents conducted experiments using recurrent neural networks for emotion detection in musical segments. Trained regression models were used to predict the continuous values of emotions on the axes of Russell’s circumplex model. A process of audio feature extraction and creating sequential data for learning networks with long short-term memory (LSTM) units is presented. Models were implemented using the WekaDeeplearning4j package and a number of experiments were carried out with data with different sets of features and varying segmentation. The usefulness of dividing the data into sequences as well as the point of using recurrent networks to recognize emotions in music, the results of which have even exceeded the SVM algorithm for regression, were demonstrated. The author analyzed the effect of the network structure and the set of used features on the results of the regressors recognizing values on two axes of the emotion model: arousal and valence. Finally, the use of a pretrained model for processing audio features and training a recurrent network with new sequences of features is presented.

A Review of Recurrent Neural Networks: LSTM Cells and Network Architectures

2019 ◽  
Vol 31 (7) ◽  
pp. 1235-1270 ◽  
Author(s):  
Yong Yu ◽  
Xiaosheng Si ◽  
Changhua Hu ◽  
Jianxun Zhang
Keyword(s):  
Neural Networks ◽  
Recurrent Neural Networks ◽  
Short Term Memory ◽  
Cell Structure ◽  
Relevant Information ◽  
Future Research ◽  
Sequential Data ◽  
Learning Capacity ◽  
Research Areas ◽  
Almost All

Recurrent neural networks (RNNs) have been widely adopted in research areas concerned with sequential data, such as text, audio, and video. However, RNNs consisting of sigma cells or tanh cells are unable to learn the relevant information of input data when the input gap is large. By introducing gate functions into the cell structure, the long short-term memory (LSTM) could handle the problem of long-term dependencies well. Since its introduction, almost all the exciting results based on RNNs have been achieved by the LSTM. The LSTM has become the focus of deep learning. We review the LSTM cell and its variants to explore the learning capacity of the LSTM cell. Furthermore, the LSTM networks are divided into two broad categories: LSTM-dominated networks and integrated LSTM networks. In addition, their various applications are discussed. Finally, future research directions are presented for LSTM networks.

scholarly journals Compressing Recurrent Neural Networks with Tensor Ring for Action Recognition

2019 ◽  
Vol 33 ◽  
pp. 4683-4690 ◽  
Author(s):  
Yu Pan ◽  
Jing Xu ◽  
Maolin Wang ◽  
Jinmian Ye ◽  
Fei Wang ◽  
...  
Keyword(s):  
Neural Networks ◽  
Action Recognition ◽  
Recurrent Neural Networks ◽  
Input Data ◽  
Short Term Memory ◽  
Computational Cost ◽  
Tensor Decomposition ◽  
Decomposition Methods ◽  
Low Rank ◽  
Sequential Data

Recurrent Neural Networks (RNNs) and their variants, such as Long-Short Term Memory (LSTM) networks, and Gated Recurrent Unit (GRU) networks, have achieved promising performance in sequential data modeling. The hidden layers in RNNs can be regarded as the memory units, which are helpful in storing information in sequential contexts. However, when dealing with high dimensional input data, such as video and text, the input-to-hidden linear transformation in RNNs brings high memory usage and huge computational cost. This makes the training of RNNs very difficult. To address this challenge, we propose a novel compact LSTM model, named as TR-LSTM, by utilizing the low-rank tensor ring decomposition (TRD) to reformulate the input-to-hidden transformation. Compared with other tensor decomposition methods, TR-LSTM is more stable. In addition, TR-LSTM can complete an end-to-end training and also provide a fundamental building block for RNNs in handling large input data. Experiments on real-world action recognition datasets have demonstrated the promising performance of the proposed TR-LSTM compared with the tensor-train LSTM and other state-of-the-art competitors.

scholarly journals The Prediction Process Based on Deep Recurrent Neural Networks: A Review

2021 ◽  
pp. 29-45
Author(s):  
Diyar Qader Zeebaree ◽  
Adnan Mohsin Abdulazeez ◽  
Lozan M. Abdullrhman ◽  
Dathar Abas Hasan ◽  
Omar Sedqi Kareem
Keyword(s):  
Neural Networks ◽  
Recurrent Neural Networks ◽  
Short Term Memory ◽  
Recurrent Network ◽  
Current Layer ◽  
Recurrent Networks ◽  
Daily Lives ◽  
Long Short Term Memory ◽  
The Impact ◽  
Very High

Prediction is vital in our daily lives, as it is used in various ways, such as learning, adapting, predicting, and classifying. The prediction of parameters capacity of RNNs is very high; it provides more accurate results than the conventional statistical methods for prediction. The impact of a hierarchy of recurrent neural networks on Predicting process is studied in this paper. A recurrent network takes the hidden state of the previous layer as input and generates as output the hidden state of the current layer. Some of deep Learning algorithms can be utilized in as prediction tools in video analysis, musical information retrieval and time series applications. Recurrent networks may process examples simultaneously, maintaining a state or memory that recreates an arbitrarily long background window. Long Short-Term Memory (LSTM) and Bidirectional RNN (BRNN) are examples of recurrent networks. This paper aims to give a comprehensive assessment of predictions based on RNN. Additionally, each paper presents all relevant facts, such as dataset, method, architecture, and the accuracy of the predictions they deliver.

scholarly journals Attention-based hierarchical recurrent neural networks for MOOC forum posts analysis

2020 ◽  
Author(s):  
Nicola Capuano ◽  
Santi Caballé ◽  
Jordi Conesa ◽  
Antonio Greco
Keyword(s):  
Neural Networks ◽  
Recurrent Neural Networks ◽  
Text Categorization ◽  
Online Courses ◽  
Recurrent Network ◽  
Subject Area ◽  
Conversational Agents ◽  
Collaborative Tools ◽  
Experimental Findings ◽  
Productive Talk

AbstractMassive open online courses (MOOCs) allow students and instructors to discuss through messages posted on a forum. However, the instructors should limit their interaction to the most critical tasks during MOOC delivery so, teacher-led scaffolding activities, such as forum-based support, can be very limited, even impossible in such environments. In addition, students who try to clarify the concepts through such collaborative tools could not receive useful answers, and the lack of interactivity may cause a permanent abandonment of the course. The purpose of this paper is to report the experimental findings obtained evaluating the performance of a text categorization tool capable of detecting the intent, the subject area, the domain topics, the sentiment polarity, and the level of confusion and urgency of a forum post, so that the result may be exploited by instructors to carefully plan their interventions. The proposed approach is based on the application of attention-based hierarchical recurrent neural networks, in which both a recurrent network for word encoding and an attention mechanism for word aggregation at sentence and document levels are used before classification. The integration of the developed classifier inside an existing tool for conversational agents, based on the academically productive talk framework, is also presented as well as the accuracy of the proposed method in the classification of forum posts.

Fake News Detection Using Recurrent Neural Networks and Distributed Representations for the Portuguese Language

2021 ◽  
Author(s):  
Guilherme Zanini Moreira ◽  
Marcelo Romero ◽  
Manassés Ribeiro
Keyword(s):  
Neural Networks ◽  
Recurrent Neural Networks ◽  
Short Term Memory ◽  
Classification Performance ◽  
Fake News ◽  
Distributed Representations ◽  
Memory Network ◽  
Long Short Term Memory ◽  
Media Channels ◽  
Damage Rule

After the advent of Web, the number of people who abandoned traditional media channels and started receiving news only through social media has increased. However, this caused an increase of the spread of fake news due to the ease of sharing information. The consequences are various, with one of the main ones being the possible attempts to manipulate public opinion for elections or promotion of movements that can damage rule of law or the institutions that represent it. The objective of this work is to perform fake news detection using Distributed Representations and Recurrent Neural Networks (RNNs). Although fake news detection using RNNs has been already explored in the literature, there is little research on the processing of texts in Portuguese language, which is the focus of this work. For this purpose, distributed representations from texts are generated with three different algorithms (fastText, GloVe and word2vec) and used as input features for a Long Short-term Memory Network (LSTM). The approach is evaluated using a publicly available labelled news dataset. The proposed approach shows promising results for all the three distributed representation methods for feature extraction, with the combination word2vec+LSTM providing the best results. The results of the proposed approach shows a better classification performance when compared to simple architectures, while similar results are obtained when the approach is compared to deeper architectures or more complex methods.

Recurrent Neural Networks with Small Weights Implement Definite Memory Machines

2003 ◽  
Vol 15 (8) ◽  
pp. 1897-1929 ◽  
Author(s):  
Barbara Hammer ◽  
Peter Tiňo
Keyword(s):  
Neural Networks ◽  
Recurrent Neural Networks ◽  
Experimental Studies ◽  
Activation Function ◽  
Transition Function ◽  
Recurrent Network ◽  
Point Of View ◽  
Recurrent Networks ◽  
Contraction Parameter ◽  
Arbitrary Precision

Recent experimental studies indicate that recurrent neural networks initialized with “small” weights are inherently biased toward definite memory machines (Tiňno, Čerňanský, & Beňušková, 2002a, 2002b). This article establishes a theoretical counterpart: transition function of recurrent network with small weights and squashing activation function is a contraction. We prove that recurrent networks with contractive transition function can be approximated arbitrarily well on input sequences of unbounded length by a definite memory machine. Conversely, every definite memory machine can be simulated by a recurrent network with contractive transition function. Hence, initialization with small weights induces an architectural bias into learning with recurrent neural networks. This bias might have benefits from the point of view of statistical learning theory: it emphasizes one possible region of the weight space where generalization ability can be formally proved. It is well known that standard recurrent neural networks are not distribution independent learnable in the probably approximately correct (PAC) sense if arbitrary precision and inputs are considered. We prove that recurrent networks with contractive transition function with a fixed contraction parameter fulfill the so-called distribution independent uniform convergence of empirical distances property and hence, unlike general recurrent networks, are distribution independent PAC learnable.

Sign in / Sign up

Export Citation Format

Share Document