Mixing and Matching Emotion Frameworks: Investigating Cross-Framework Transfer Learning for Dutch Emotion Detection

Emotion detection has become a growing field of study, especially seeing its broad application potential. Research usually focuses on emotion classification, but performance tends to be rather low, especially when dealing with more advanced emotion categories that are tailored to specific tasks and domains. Therefore, we propose the use of the dimensional emotion representations valence, arousal and dominance (VAD), in an emotion regression task. Firstly, we hypothesize that they can improve performance of the classification task, and secondly, they might be used as a pivot mechanism to map towards any given emotion framework, which allows tailoring emotion frameworks to specific applications. In this paper, we examine three cross-framework transfer methodologies: multi-task learning, in which VAD regression and classification are learned simultaneously; meta-learning, where VAD regression and emotion classification are learned separately and predictions are jointly used as input for a meta-learner; and a pivot mechanism, which converts the predictions of the VAD model to emotion classes. We show that dimensional representations can indeed boost performance for emotion classification, especially in the meta-learning setting (up to 7% macro F1-score compared to regular emotion classification). The pivot method was not able to compete with the base model, but further inspection suggests that it could be efficient, provided that the VAD regression model is further improved.

Download Full-text

Transfer learning based convolution neural net for authentication and classification of emotions from natural and stimulated speech signals

Journal of Intelligent & Fuzzy Systems ◽

10.3233/jifs-210711 ◽

2021 ◽

pp. 1-12

Author(s):

Mukul Kumar ◽

Nipun Katyal ◽

Nersisson Ruban ◽

Elena Lyakso ◽

A. Mary Mekala ◽

...

Keyword(s):

Machine Learning ◽

Deep Learning ◽

Transfer Learning ◽

Research Work ◽

Oral Communication ◽

Neural Net ◽

Emotion Classification ◽

Style Transfer ◽

Conventional Machine ◽

Neural Network Algorithm

Over the years the need for differentiating various emotions from oral communication plays an important role in emotion based studies. There have been different algorithms to classify the kinds of emotion. Although there is no measure of fidelity of the emotion under consideration, which is primarily due to the reason that most of the readily available datasets that are annotated are produced by actors and not generated in real-world scenarios. Therefore, the predicted emotion lacks an important aspect called authenticity, which is whether an emotion is actual or stimulated. In this research work, we have developed a transfer learning and style transfer based hybrid convolutional neural network algorithm to classify the emotion as well as the fidelity of the emotion. The model is trained on features extracted from a dataset that contains stimulated as well as actual utterances. We have compared the developed algorithm with conventional machine learning and deep learning techniques by few metrics like accuracy, Precision, Recall and F1 score. The developed model performs much better than the conventional machine learning and deep learning models. The research aims to dive deeper into human emotion and make a model that understands it like humans do with precision, recall, F1 score values of 0.994, 0.996, 0.995 for speech authenticity and 0.992, 0.989, 0.99 for speech emotion classification respectively.

Download Full-text

An LIBS quantitative analysis method for alloy steel at high temperature based on transfer learning

Journal of Analytical Atomic Spectrometry ◽

10.1039/c8ja00069g ◽

2018 ◽

Vol 33 (7) ◽

pp. 1184-1195 ◽

Cited By ~ 9

Author(s):

Jianhong Yang ◽

Xiaomeng Li ◽

Huili Lu ◽

Jinwu Xu ◽

Haixia Li

Keyword(s):

High Temperature ◽

Quantitative Analysis ◽

Regression Model ◽

Transfer Learning ◽

Alloy Steel ◽

Room Temperature ◽

Analysis Method ◽

Quantitative Analysis Method

Information learnt from spectra at room temperature is transferred to assist in building a better regression model at high temperature.

Download Full-text

Application of Seq2Seq Models on Code Correction

Frontiers in Artificial Intelligence ◽

10.3389/frai.2021.590215 ◽

2021 ◽

Vol 4 ◽

Author(s):

Shan Huang ◽

Xiao Zhou ◽

Sang Chin

Keyword(s):

Transfer Learning ◽

Programming Language ◽

Computational Efficiency ◽

Test Suite ◽

Classification Task ◽

Repair Rate ◽

Error Type ◽

Memory Efficiency ◽

Type Classification ◽

Software Assurance

We apply various seq2seq models on programming language correction tasks on Juliet Test Suite for C/C++ and Java of Software Assurance Reference Datasets and achieve 75% (for C/C++) and 56% (for Java) repair rates on these tasks. We introduce pyramid encoder in these seq2seq models, which significantly increases the computational efficiency and memory efficiency, while achieving similar repair rate to their nonpyramid counterparts. We successfully carry out error type classification task on ITC benchmark examples (with only 685 code instances) using transfer learning with models pretrained on Juliet Test Suite, pointing out a novel way of processing small programming language datasets.

Download Full-text

VGG 16 Based Human Emotion Classification Using Thermal Images Through Transfer Learning

10.1109/mysurucon52639.2021.9641516 ◽

2021 ◽

Author(s):

C M Naveen Kumar ◽

G Shivakumar

Keyword(s):

Transfer Learning ◽

Emotion Classification ◽

Thermal Images ◽

Human Emotion

Download Full-text

A Meta-Learning Approach for Custom Model Training

Proceedings of the AAAI Conference on Artificial Intelligence ◽

10.1609/aaai.v33i01.33019937 ◽

2019 ◽

Vol 33 ◽

pp. 9937-9938

Author(s):

Amir Erfan Eshratifar ◽

Mohammad Saeed Abrishami ◽

David Eigen ◽

Massoud Pedram

Keyword(s):

Transfer Learning ◽

Large Data ◽

Learning Approaches ◽

Target Task ◽

Training Approach ◽

Starting Point ◽

Meta Learning ◽

Future Task ◽

Model Training ◽

Training Examples

Transfer-learning and meta-learning are two effective methods to apply knowledge learned from large data sources to new tasks. In few-class, few-shot target task settings (i.e. when there are only a few classes and training examples available in the target task), meta-learning approaches that optimize for future task learning have outperformed the typical transfer approach of initializing model weights from a pretrained starting point. But as we experimentally show, metalearning algorithms that work well in the few-class setting do not generalize well in many-shot and many-class cases. In this paper, we propose a joint training approach that combines both transfer-learning and meta-learning. Benefiting from the advantages of each, our method obtains improved generalization performance on unseen target tasks in both few- and many-class and few- and many-shot scenarios.

Download Full-text

A user-centered approach for detecting emotions with low-cost sensors

Multimedia Tools and Applications ◽

10.1007/s11042-020-09576-0 ◽

2020 ◽

Vol 79 (47-48) ◽

pp. 35885-35907

Author(s):

Rita Francese ◽

Michele Risi ◽

Genoveffa Tortora

Keyword(s):

Artificial Intelligence ◽

Detection System ◽

Low Cost ◽

Affective State ◽

Classification Performance ◽

Emotion Detection ◽

Emotion Classification ◽

User Perceptions ◽

Machine Learning Classifiers ◽

User Centered

AbstractDetecting emotions is very useful in many fields, from health-care to human-computer interaction. In this paper, we propose an iterative user-centered methodology for supporting the development of an emotion detection system based on low-cost sensors. Artificial Intelligence techniques have been adopted for emotion classification. Different kind of Machine Learning classifiers have been experimentally trained on the users’ biometrics data, such as hearth rate, movement and audio. The system has been developed in two iterations and, at the end of each of them, the performance of classifiers (MLP, CNN, LSTM, Bidirectional-LSTM and Decision Tree) has been compared. After the experiment, the SAM questionnaire is proposed to evaluate the user’s affective state when using the system. In the first experiment we gathered data from 47 participants, in the second one an improved version of the system has been trained and validated by 107 people. The emotional analysis conducted at the end of each iteration suggests that reducing the device invasiveness may affect the user perceptions and also improve the classification performance.

Download Full-text

Assessment of the performance of quantitative feature-based transfer learning LIBS analysis of chromium in high temperature alloy steel samples

Journal of Analytical Atomic Spectrometry ◽

10.1039/d0ja00334d ◽

2020 ◽

Vol 35 (11) ◽

pp. 2639-2648

Author(s):

Fu Chang ◽

Huili Lu ◽

Hao Sun ◽

Jianhong Yang

Keyword(s):

Hilbert Space ◽

High Temperature ◽

Regression Model ◽

Spectral Data ◽

Transfer Learning ◽

Alloy Steel ◽

Room Temperature ◽

High Temperature Alloy ◽

Feature Based ◽

Calibration Samples

The spectral data of calibration samples at room temperature and the spectral data of uncalibrated samples at high temperature are together transferred into a Hilbert space where an LIBS regression model is established.

Download Full-text

Emotion Mining Using Semantic Similarity

International Journal of Synthetic Emotions ◽

10.4018/ijse.2018070101 ◽

2018 ◽

Vol 9 (2) ◽

pp. 1-22 ◽

Cited By ~ 2

Author(s):

Rafiya Jan ◽

Afaq Alam Khan

Keyword(s):

Semantic Similarity ◽

Semantic Relatedness ◽

Semantic Features ◽

Emotion Detection ◽

Emotion Classification ◽

Textual Data ◽

Different Types ◽

Distributional Semantic Models ◽

Affective Information

Social networks are considered as the most abundant sources of affective information for sentiment and emotion classification. Emotion classification is the challenging task of classifying emotions into different types. Emotions being universal, the automatic exploration of emotion is considered as a difficult task to perform. A lot of the research is being conducted in the field of automatic emotion detection in textual data streams. However, very little attention is paid towards capturing semantic features of the text. In this article, the authors present the technique of semantic relatedness for automatic classification of emotion in the text using distributional semantic models. This approach uses semantic similarity for measuring the coherence between the two emotionally related entities. Before classification, data is pre-processed to remove the irrelevant fields and inconsistencies and to improve the performance. The proposed approach achieved the accuracy of 71.795%, which is competitive considering as no training or annotation of data is done.

Download Full-text