scholarly journals Speech & Song Emotion Recognition Using Multilayer Perceptron and Standard Vector Machine

2021 ◽  
Author(s):  
Behzad Javaheri
Keyword(s):  
Emotion Recognition ◽  
Multilayer Perceptron ◽  
Data Augmentation ◽  
Optimal Scaling ◽  
Dependent Manner ◽  
Audio Features ◽  
Lower Accuracy ◽  
Standard Vector ◽  
Audio Data ◽  
Increase Sample Size

herein, we have compared the performance of SVM and MLP in emotion recognition using speech and song channels of the RAVDESS dataset. We have undertaken a journey to extract various audio features, identify optimal scaling strategy and hyperparameter for our models. To increase sample size, we have performed audio data augmentation and addressed data imbalance using SMOTE. Our data indicate that optimised SVM outperforms MLP with an accuracy of 82 compared to 75%. Following data augmentation, the performance of both algorithms was identical at ~79%, however, overfitting was evident for the SVM. Our final exploration indicated that the performance of both SVM and MLP were similar in which both resulted in lower accuracy for the speech channel compared to the song channel. Our findings suggest that both SVM and MLP are powerful classifiers for emotion recognition in a vocal-dependent manner.

scholarly journals Efficient Classification of Environmental Sounds through Multiple Features Aggregation and Data Enhancement Techniques for Spectrogram Images

Symmetry ◽  
2020 ◽  
Vol 12 (11) ◽  
pp. 1822
Author(s):  
Zohaib Mushtaq ◽  
Shun-Feng Su
Keyword(s):  
Real Time ◽  
Data Augmentation ◽  
Image Features ◽  
Logarithmic Scale ◽  
Acoustic Features ◽  
Environmental Sound ◽  
Environmental Sounds ◽  
Audio Features ◽  
Audio Data

Over the past few years, the study of environmental sound classification (ESC) has become very popular due to the intricate nature of environmental sounds. This paper reports our study on employing various acoustic features aggregation and data enhancement approaches for the effective classification of environmental sounds. The proposed data augmentation techniques are mixtures of the reinforcement, aggregation, and combination of distinct acoustics features. These features are known as spectrogram image features (SIFs) and retrieved by different audio feature extraction techniques. All audio features used in this manuscript are categorized into two groups: one with general features and the other with Mel filter bank-based acoustic features. Two novel and innovative features based on the logarithmic scale of the Mel spectrogram (Mel), Log (Log-Mel) and Log (Log (Log-Mel)) denoted as L2M and L3M are introduced in this paper. In our study, three prevailing ESC benchmark datasets, ESC-10, ESC-50, and Urbansound8k (Us8k) are used. Most of the audio clips in these datasets are not fully acquired with sound and include silence parts. Therefore, silence trimming is implemented as one of the pre-processing techniques. The training is conducted by using the transfer learning model DenseNet-161, which is further fine-tuned with individual optimal learning rates based on the discriminative learning technique. The proposed methodologies attain state-of-the-art outcomes for all used ESC datasets, i.e., 99.22% for ESC-10, 98.52% for ESC-50, and 97.98% for Us8k. This work also considers real-time audio data to evaluate the performance and efficiency of the proposed techniques. The implemented approaches also have competitive results on real-time audio data.

scholarly journals Utterance Level Feature Aggregation with Deep Metric Learning for Speech Emotion Recognition

Sensors ◽  
2021 ◽  
Vol 21 (12) ◽  
pp. 4233
Author(s):  
Bogdan Mocanu ◽  
Ruxandra Tapu ◽  
Titus Zaharia
Keyword(s):  
Emotion Recognition ◽  
Loss Function ◽  
State Of The Art ◽  
Disease Diagnosis ◽  
Data Representation ◽  
Speech Emotion Recognition ◽  
Audio Features ◽  
Global Accuracy ◽  
Space Data ◽  
Art Techniques

Emotion is a form of high-level paralinguistic information that is intrinsically conveyed by human speech. Automatic speech emotion recognition is an essential challenge for various applications; including mental disease diagnosis; audio surveillance; human behavior understanding; e-learning and human–machine/robot interaction. In this paper, we introduce a novel speech emotion recognition method, based on the Squeeze and Excitation ResNet (SE-ResNet) model and fed with spectrogram inputs. In order to overcome the limitations of the state-of-the-art techniques, which fail in providing a robust feature representation at the utterance level, the CNN architecture is extended with a trainable discriminative GhostVLAD clustering layer that aggregates the audio features into compact, single-utterance vector representation. In addition, an end-to-end neural embedding approach is introduced, based on an emotionally constrained triplet loss function. The loss function integrates the relations between the various emotional patterns and thus improves the latent space data representation. The proposed methodology achieves 83.35% and 64.92% global accuracy rates on the RAVDESS and CREMA-D publicly available datasets, respectively. When compared with the results provided by human observers, the gains in global accuracy scores are superior to 24%. Finally, the objective comparative evaluation with state-of-the-art techniques demonstrates accuracy gains of more than 3%.

scholarly journals Deep Visual Attributes vs. Hand-Crafted Audio Features on Multidomain Speech Emotion Recognition

Computation ◽  
2017 ◽  
Vol 5 (4) ◽  
pp. 26 ◽  
Author(s):  
Michalis Papakostas ◽  
Evaggelos Spyrou ◽  
Theodoros Giannakopoulos ◽  
Giorgos Siantikos ◽  
Dimitrios Sgouropoulos ◽  
...  
Keyword(s):  
Emotion Recognition ◽  
Speech Emotion Recognition ◽  
Visual Attributes ◽  
Audio Features

A Method of Environmental Sound Classification Based on Residual Networks and Data Augmentation

2021 ◽  
pp. 2150018
Author(s):  
Jinfang Zeng ◽  
Youming Li ◽  
Yu Zhang ◽  
Da Chen
Keyword(s):  
Neural Networks ◽  
Data Augmentation ◽  
Machine Learning Techniques ◽  
Deep Convolutional Neural Networks ◽  
Mel Frequency Cepstral Coefficients ◽  
Environmental Sound ◽  
Sound Classification ◽  
Learning Techniques ◽  
Proposed Model ◽  
Audio Data

Environmental sound classification (ESC) is a challenging problem due to the complexity of sounds. To date, a variety of signal processing and machine learning techniques have been applied to ESC task, including matrix factorization, dictionary learning, wavelet filterbanks and deep neural networks. It is observed that features extracted from deeper networks tend to achieve higher performance than those extracted from shallow networks. However, in ESC task, only the deep convolutional neural networks (CNNs) which contain several layers are used and the residual networks are ignored, which lead to degradation in the performance. Meanwhile, a possible explanation for the limited exploration of CNNs and the difficulty to improve on simpler models is the relative scarcity of labeled data for ESC. In this paper, a residual network called EnvResNet for the ESC task is proposed. In addition, we propose to use audio data augmentation to overcome the problem of data scarcity. The experiments will be performed on the ESC-50 database. Combined with data augmentation, the proposed model outperforms baseline implementations relying on mel-frequency cepstral coefficients and achieves results comparable to other state-of-the-art approaches in terms of classification accuracy.

scholarly journals On-Road Driver Emotion Recognition Using Facial Expression

2022 ◽  
Vol 12 (2) ◽  
pp. 807
Author(s):  
Huafei Xiao ◽  
Wenbo Li ◽  
Guanzhong Zeng ◽  
Yingzhang Wu ◽  
Jiyong Xue ◽  
...  
Keyword(s):  
Facial Expression ◽  
Emotion Recognition ◽  
Data Augmentation ◽  
Recognition Task ◽  
Superior Performance ◽  
Expression Recognition ◽  
Emotion Detection ◽  
Backbone Networks ◽  
Driving Task ◽  
Machine Systems

With the development of intelligent automotive human-machine systems, driver emotion detection and recognition has become an emerging research topic. Facial expression-based emotion recognition approaches have achieved outstanding results on laboratory-controlled data. However, these studies cannot represent the environment of real driving situations. In order to address this, this paper proposes a facial expression-based on-road driver emotion recognition network called FERDERnet. This method divides the on-road driver facial expression recognition task into three modules: a face detection module that detects the driver’s face, an augmentation-based resampling module that performs data augmentation and resampling, and an emotion recognition module that adopts a deep convolutional neural network pre-trained on FER and CK+ datasets and then fine-tuned as a backbone for driver emotion recognition. This method adopts five different backbone networks as well as an ensemble method. Furthermore, to evaluate the proposed method, this paper collected an on-road driver facial expression dataset, which contains various road scenarios and the corresponding driver’s facial expression during the driving task. Experiments were performed on the on-road driver facial expression dataset that this paper collected. Based on efficiency and accuracy, the proposed FERDERnet with Xception backbone was effective in identifying on-road driver facial expressions and obtained superior performance compared to the baseline networks and some state-of-the-art networks.

scholarly journals Bimodal Emotion Recognition Model for Minnan Songs

Information ◽  
2020 ◽  
Vol 11 (3) ◽  
pp. 145 ◽  
Author(s):  
Zhenglong Xiang ◽  
Xialei Dong ◽  
Yuanxiang Li ◽  
Fei Yu ◽  
Xing Xu ◽  
...  
Keyword(s):  
Neural Network ◽  
Emotion Recognition ◽  
Short Term Memory ◽  
Music Appreciation ◽  
Research Papers ◽  
Audio Features ◽  
Analysis Theory ◽  
Proposed Model ◽  
Song Lyrics ◽  
Long Short Term Memory

Most of the existing research papers study the emotion recognition of Minnan songs from the perspectives of music analysis theory and music appreciation. However, these investigations do not explore any possibility of carrying out an automatic emotion recognition of Minnan songs. In this paper, we propose a model that consists of four main modules to classify the emotion of Minnan songs by using the bimodal data—song lyrics and audio. In the proposed model, an attention-based Long Short-Term Memory (LSTM) neural network is applied to extract lyrical features, and a Convolutional Neural Network (CNN) is used to extract the audio features from the spectrum. Then, two kinds of extracted features are concatenated by multimodal compact bilinear pooling, and finally, the concatenated features are input to the classifying module to determine the song emotion. We designed three experiment groups to investigate the classifying performance of combinations of the four main parts, the comparisons of proposed model with the current approaches and the influence of a few key parameters on the performance of emotion recognition. The results show that the proposed model exhibits better performance over all other experimental groups. The accuracy, precision and recall of the proposed model exceed 0.80 in a combination of appropriate parameters.

scholarly journals Audio Features for Music Emotion Recognition: a Survey

2020 ◽  
pp. 1-1
Author(s):  
Renato Panda ◽  
Ricardo Manuel Malheiro ◽  
Rui Pedro Paiva
Keyword(s):  
Emotion Recognition ◽  
Audio Features

scholarly journals eXnet: An Efficient Approach for Emotion Recognition in the Wild

Sensors ◽  
2020 ◽  
Vol 20 (4) ◽  
pp. 1087
Author(s):  
Muhammad Naveed Riaz ◽  
Yao Shen ◽  
Muhammad Sohail ◽  
Minyi Guo
Keyword(s):  
Facial Expression ◽  
Emotion Recognition ◽  
Facial Expression Recognition ◽  
Data Augmentation ◽  
Raspberry Pi ◽  
Expression Recognition ◽  
Extensive Evaluation ◽  
Benchmark Datasets ◽  
Augmentation Techniques ◽  
In The Wild

Facial expression recognition has been well studied for its great importance in the areas of human–computer interaction and social sciences. With the evolution of deep learning, there have been significant advances in this area that also surpass human-level accuracy. Although these methods have achieved good accuracy, they are still suffering from two constraints (high computational power and memory), which are incredibly critical for small hardware-constrained devices. To alleviate this issue, we propose a new Convolutional Neural Network (CNN) architecture eXnet (Expression Net) based on parallel feature extraction which surpasses current methods in accuracy and contains a much smaller number of parameters (eXnet: 4.57 million, VGG19: 14.72 million), making it more efficient and lightweight for real-time systems. Several modern data augmentation techniques are applied for generalization of eXnet; these techniques improve the accuracy of the network by overcoming the problem of overfitting while containing the same size. We provide an extensive evaluation of our network against key methods on Facial Expression Recognition 2013 (FER-2013), Extended Cohn-Kanade Dataset (CK+), and Real-world Affective Faces Database (RAF-DB) benchmark datasets. We also perform ablation evaluation to show the importance of different components of our architecture. To evaluate the efficiency of eXnet on embedded systems, we deploy it on Raspberry Pi 4B. All these evaluations show the superiority of eXnet for emotion recognition in the wild in terms of accuracy, the number of parameters, and size on disk.

Sign in / Sign up

Export Citation Format

Share Document