scholarly journals Multi-Source and Multi-Representation Adaptation for Cross-Domain Electroencephalography Emotion Recognition

2022 ◽  
Vol 12 ◽  
Author(s):  
Jiangsheng Cao ◽  
Xueqin He ◽  
Chenhui Yang ◽  
Sifang Chen ◽  
Zhangyu Li ◽  
...  

Due to the non-invasiveness and high precision of electroencephalography (EEG), the combination of EEG and artificial intelligence (AI) is often used for emotion recognition. However, the internal differences in EEG data have become an obstacle to classification accuracy. To solve this problem, considering labeled data from similar nature but different domains, domain adaptation usually provides an attractive option. Most of the existing researches aggregate the EEG data from different subjects and sessions as a source domain, which ignores the assumption that the source has a certain marginal distribution. Moreover, existing methods often only align the representation distributions extracted from a single structure, and may only contain partial information. Therefore, we propose the multi-source and multi-representation adaptation (MSMRA) for cross-domain EEG emotion recognition, which divides the EEG data from different subjects and sessions into multiple domains and aligns the distribution of multiple representations extracted from a hybrid structure. Two datasets, i.e., SEED and SEED IV, are used to validate the proposed method in cross-session and cross-subject transfer scenarios, experimental results demonstrate the superior performance of our model to state-of-the-art models in most settings.

2021 ◽  
Vol 11 (10) ◽  
pp. 4503
Author(s):  
Lingtong Min ◽  
Deyun Zhou ◽  
Xiaoyang Li ◽  
Qinyi Lv ◽  
Yuanjie Zhi

Distribution mismatch can be easily found in multi-sensor systems, which may be caused by different shoot angles, weather conditions and so on. Domain adaptation aims to build robust classifiers using the knowledge from a well-labeled source domain, while applied on a related but different target domain. Pseudo labeling is a prevalent technique for class-wise distribution alignment. Therefore, numerous efforts have been spent on alleviating the issue of mislabeling. In this paper, unlike existing selective hard labeling works, we propose a fuzzy labeling based graph learning framework for matching conditional distribution. Specifically, we construct the cross-domain affinity graph by considering the fuzzy label matrix of target samples. In order to solve the problem of representation shrinkage, the paradigm of sparse filtering is introduced. Finally, a unified optimization method based on gradient descent is proposed. Extensive experiments show that our method achieves comparable or superior performance when compared to state-of-the-art works.


2020 ◽  
Vol 34 (04) ◽  
pp. 4028-4035 ◽  
Author(s):  
Aditya Grover ◽  
Christopher Chute ◽  
Rui Shu ◽  
Zhangjie Cao ◽  
Stefano Ermon

Given datasets from multiple domains, a key challenge is to efficiently exploit these data sources for modeling a target domain. Variants of this problem have been studied in many contexts, such as cross-domain translation and domain adaptation. We propose AlignFlow, a generative modeling framework that models each domain via a normalizing flow. The use of normalizing flows allows for a) flexibility in specifying learning objectives via adversarial training, maximum likelihood estimation, or a hybrid of the two methods; and b) learning and exact inference of a shared representation in the latent space of the generative model. We derive a uniform set of conditions under which AlignFlow is marginally-consistent for the different learning objectives. Furthermore, we show that AlignFlow guarantees exact cycle consistency in mapping datapoints from a source domain to target and back to the source domain. Empirically, AlignFlow outperforms relevant baselines on image-to-image translation and unsupervised domain adaptation and can be used to simultaneously interpolate across the various domains using the learned representation.


2021 ◽  
Vol 15 ◽  
Author(s):  
Hao Chen ◽  
Ming Jin ◽  
Zhunan Li ◽  
Cunhang Fan ◽  
Jinpeng Li ◽  
...  

As an essential element for the diagnosis and rehabilitation of psychiatric disorders, the electroencephalogram (EEG) based emotion recognition has achieved significant progress due to its high precision and reliability. However, one obstacle to practicality lies in the variability between subjects and sessions. Although several studies have adopted domain adaptation (DA) approaches to tackle this problem, most of them treat multiple EEG data from different subjects and sessions together as a single source domain for transfer, which either fails to satisfy the assumption of domain adaptation that the source has a certain marginal distribution, or increases the difficulty of adaptation. We therefore propose the multi-source marginal distribution adaptation (MS-MDA) for EEG emotion recognition, which takes both domain-invariant and domain-specific features into consideration. First, we assume that different EEG data share the same low-level features, then we construct independent branches for multiple EEG data source domains to adopt one-to-one domain adaptation and extract domain-specific features. Finally, the inference is made by multiple branches. We evaluate our method on SEED and SEED-IV for recognizing three and four emotions, respectively. Experimental results show that the MS-MDA outperforms the comparison methods and state-of-the-art models in cross-session and cross-subject transfer scenarios in our settings. Codes at https://github.com/VoiceBeer/MS-MDA.


Sensors ◽  
2017 ◽  
Vol 17 (5) ◽  
pp. 1014 ◽  
Author(s):  
Xin Chai ◽  
Qisong Wang ◽  
Yongping Zhao ◽  
Yongqiang Li ◽  
Dan Liu ◽  
...  

2020 ◽  
Vol 34 (04) ◽  
pp. 4320-4327 ◽  
Author(s):  
Songlei Jian ◽  
Liang Hu ◽  
Longbing Cao ◽  
Kai Lu

The cross-domain representation learning plays an important role in tasks including domain adaptation and transfer learning. However, existing cross-domain representation learning focuses on building one shared space and ignores the unlabeled data in the source domain, which cannot effectively capture the distribution and structure heterogeneities in cross-domain data. To address this challenge, we propose a new cross-domain representation learning approach: MUltiple Lipschitz-constrained AligNments (MULAN) on partially-labeled cross-domain data. MULAN produces two representation spaces: a common representation space to incorporate knowledge from the source domain and a complementary representation space to complement the common representation with target local topological information by Lipschitz-constrained representation transformation. MULAN utilizes both unlabeled and labeled data in the source and target domains to address distribution heterogeneity by Lipschitz-constrained adversarial distribution alignment and structure heterogeneity by cluster assumption-based class alignment while keeping the target local topological information in complementary representation by self alignment. Moreover, MULAN is effectively equipped with a customized learning process and an iterative parameter updating process. MULAN shows its superior performance on partially-labeled semi-supervised domain adaptation and few-shot domain adaptation and outperforms the state-of-the-art visual domain adaptation models by up to 12.1%.


2020 ◽  
Vol 54 (8) ◽  
pp. 1963-1986
Author(s):  
Tilottama G. Chowdhury ◽  
Feisal Murshed

Purpose This paper proposes that categorization flexibility, operationalized as the cognitive capacity that cross-categorizes products in multiple situational categories across multiple domains, might favorably influence a consumer’s evaluation of unconventional options. Design/methodology/approach Experimental research design is used to test the theory. An exploratory study first establishes the effect of categorization flexibility in a non-food domain. Study 1 documents the moderating role of decision domain, showing that the effect works only under low- (vs high-) consequence domain. Studies 2A and 2B further refine the notion by showing that individuals can be primed in a relatively higher categorization flexibility frame of mind. Study 3 demonstrates the interactive effect of categorization flexibility and adventure priming in a high-consequence domain. Study 4 integrates the interactive effects of decisions with low- vs high-consequence, adventure priming and categorization flexibility within a single decision domain of high consequence. Findings Consumers with higher- (vs lower-) categorization flexibility tend to opt for unconventional choices when the decision domain entails low consequences, whereas such a result does not hold under decision domain of high consequences. The categorization flexibility effects in case of low-consequence decision domain holds true even when consumers are primed to be categorization flexible. Furthermore, with additional adventure priming, consumers show an increased preference for unconventional options even under a decision domain with high consequence. Research limitations/implications This study could not examine real purchase behavior as results are based on cross-sectional, behavioral intention data. In addition, it did not examine the underlying reason for presence of cross-domain categorization flexibility index. Practical implications The results suggest that stimuli may be tailored to consumers in ways that increase the salience and the perceived attractiveness of unconventional choices. Further, data reinforce the notion of cross-categorical interrelations among different domains, which could be leveraged by marketers. Originality/value This study represents the first documentation of the potential ways by which unconventional product choice might be a function of individuals’ categorization flexibility level across different types of decision domains. The findings yield implications that are novel to both categorization and consumer decision-making literature.


Sensors ◽  
2021 ◽  
Vol 21 (5) ◽  
pp. 1579 ◽  
Author(s):  
Kyoung Ju Noh ◽  
Chi Yoon Jeong ◽  
Jiyoun Lim ◽  
Seungeun Chung ◽  
Gague Kim ◽  
...  

Speech emotion recognition (SER) is a natural method of recognizing individual emotions in everyday life. To distribute SER models to real-world applications, some key challenges must be overcome, such as the lack of datasets tagged with emotion labels and the weak generalization of the SER model for an unseen target domain. This study proposes a multi-path and group-loss-based network (MPGLN) for SER to support multi-domain adaptation. The proposed model includes a bidirectional long short-term memory-based temporal feature generator and a transferred feature extractor from the pre-trained VGG-like audio classification model (VGGish), and it learns simultaneously based on multiple losses according to the association of emotion labels in the discrete and dimensional models. For the evaluation of the MPGLN SER as applied to multi-cultural domain datasets, the Korean Emotional Speech Database (KESD), including KESDy18 and KESDy19, is constructed, and the English-speaking Interactive Emotional Dyadic Motion Capture database (IEMOCAP) is used. The evaluation of multi-domain adaptation and domain generalization showed 3.7% and 3.5% improvements, respectively, of the F1 score when comparing the performance of MPGLN SER with a baseline SER model that uses a temporal feature generator. We show that the MPGLN SER efficiently supports multi-domain adaptation and reinforces model generalization.


Sensors ◽  
2021 ◽  
Vol 21 (10) ◽  
pp. 3382
Author(s):  
Zhongwei Zhang ◽  
Mingyu Shao ◽  
Liping Wang ◽  
Sujuan Shao ◽  
Chicheng Ma

As the key component to transmit power and torque, the fault diagnosis of rotating machinery is crucial to guarantee the reliable operation of mechanical equipment. Regrettably, sample class imbalance is a common phenomenon in industrial applications, which causes large cross-domain distribution discrepancies for domain adaptation (DA) and results in performance degradation for most of the existing mechanical fault diagnosis approaches. To address this issue, a novel DA approach that simultaneously reduces the cross-domain distribution difference and the geometric difference is proposed, which is defined as MRMI. This work contains three parts to improve the sample class imbalance issue: (1) A novel distance metric method (MVD) is proposed and applied to improve the performance of marginal distribution adaptation. (2) Manifold regularization is combined with instance reweighting to simultaneously explore the intrinsic manifold structure and remove irrelevant source-domain samples adaptively. (3) The ℓ2-norm regularization is applied as the data preprocessing tool to improve the model generalization performance. The gear and rolling bearing datasets with class imbalanced samples are applied to validate the reliability of MRMI. According to the fault diagnosis results, MRMI can significantly outperform competitive approaches under the condition of sample class imbalance.


Sign in / Sign up

Export Citation Format

Share Document