Improving Whole-Brain Neural Decoding of fMRI with Domain Adaptation

AbstractIn neural decoding, there has been a growing interest in machine learning on whole-brain functional magnetic resonance imaging (fMRI). However, the size discrepancy between the feature space and the training set poses serious challenges. Simply increasing the number of training examples is infeasible and costly. In this paper, we proposed a domain adaptation framework for whole-brain fMRI (DawfMRI) to improve whole-brain neural decoding on target data leveraging pre-existing source data. DawfMRI consists of three steps: 1) feature extraction from whole-brain fMRI, 2) source and target feature adaptation, and 3) source and target classifier adaptation. We evaluated its eight possible variations, including two non-adaptation and six adaptation algorithms, using a collection of seven task-based fMRI datasets (129 unique subjects and 11 cognitive tasks in total) from the OpenNeuro project. The results demonstrated that appropriate source domain can help improve neural decoding accuracy for challenging classification tasks. The best-case improvement is 8.94% (from 78.64% to 87.58%). Moreover, we discovered a plausible relationship between psychological similarity and adaptation effectiveness. Finally, visualizing and interpreting voxel weights showed that the adaptation can provide additional insights into neural decoding.

Download Full-text

Improving Whole-Brain Neural Decoding of fMRI with Domain Adaptation

Machine Learning in Medical Imaging - Lecture Notes in Computer Science ◽

10.1007/978-3-030-32692-0_31 ◽

2019 ◽

pp. 265-273

Author(s):

Shuo Zhou ◽

Christopher R. Cox ◽

Haiping Lu

Keyword(s):

Domain Adaptation ◽

Neural Decoding ◽

Whole Brain

Download Full-text

Author Correction: Simultaneous cortex-wide fluorescence Ca2+ imaging and whole-brain fMRI

Nature Methods ◽

10.1038/s41592-021-01217-0 ◽

2021 ◽

Author(s):

Evelyn M. R. Lake ◽

Xinxin Ge ◽

Xilin Shen ◽

Peter Herman ◽

Fahmeed Hyder ◽

...

Keyword(s):

Whole Brain ◽

Brain Fmri

Download Full-text

The Time Course of Changes during Motor Sequence Learning: A Whole-Brain fMRI Study

NeuroImage ◽

10.1006/nimg.1998.0349 ◽

1998 ◽

Vol 8 (1) ◽

pp. 50-61 ◽

Cited By ~ 265

Author(s):

Ivan Toni ◽

Michael Krams ◽

Robert Turner ◽

Richard E. Passingham

Keyword(s):

Sequence Learning ◽

Time Course ◽

Motor Sequence Learning ◽

Motor Sequence ◽

Whole Brain ◽

Fmri Study ◽

Brain Fmri

Download Full-text

Correction: Multiplexed Echo Planar Imaging for Sub-Second Whole Brain FMRI and Fast Diffusion Imaging

PLoS ONE ◽

10.1371/annotation/d9496d01-8c5d-4d24-8287-94449ada5064 ◽

2011 ◽

Vol 6 (9) ◽

Cited By ~ 4

Author(s):

David A. Feinberg ◽

Steen Moeller ◽

Stephen M. Smith ◽

Edward Auerbach ◽

Sudhir Ramanna ◽

...

Keyword(s):

Diffusion Imaging ◽

Echo Planar Imaging ◽

Fast Diffusion ◽

Planar Imaging ◽

Whole Brain ◽

Echo Planar ◽

Brain Fmri

Download Full-text

Joint Partial Optimal Transport for Open Set Domain Adaptation

Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence ◽

10.24963/ijcai.2020/352 ◽

2020 ◽

Author(s):

Renjun Xu ◽

Pelen Liu ◽

Yin Zhang ◽

Fang Cai ◽

Jindong Wang ◽

...

Keyword(s):

Optimal Transport ◽

Transport Model ◽

Domain Adaptation ◽

General Setting ◽

Feature Space ◽

Set Domain ◽

Target Domain ◽

Source Domain ◽

Open Set ◽

Proposed Model

Domain adaptation (DA) has achieved a resounding success to learn a good classifier by leveraging labeled data from a source domain to adapt to an unlabeled target domain. However, in a general setting when the target domain contains classes that are never observed in the source domain, namely in Open Set Domain Adaptation (OSDA), existing DA methods failed to work because of the interference of the extra unknown classes. This is a much more challenging problem, since it can easily result in negative transfer due to the mismatch between the unknown and known classes. Existing researches are susceptible to misclassification when target domain unknown samples in the feature space distributed near the decision boundary learned from the labeled source domain. To overcome this, we propose Joint Partial Optimal Transport (JPOT), fully utilizing information of not only the labeled source domain but also the discriminative representation of unknown class in the target domain. The proposed joint discriminative prototypical compactness loss can not only achieve intra-class compactness and inter-class separability, but also estimate the mean and variance of the unknown class through backpropagation, which remains intractable for previous methods due to the blindness about the structure of the unknown classes. To our best knowledge, this is the first optimal transport model for OSDA. Extensive experiments demonstrate that our proposed model can significantly boost the performance of open set domain adaptation on standard DA datasets.

Download Full-text

USING SEMANTICALLY PAIRED IMAGES TO IMPROVE DOMAIN ADAPTATION FOR THE SEMANTIC SEGMENTATION OF AERIAL IMAGES

ISPRS Annals of Photogrammetry Remote Sensing and Spatial Information Sciences ◽

10.5194/isprs-annals-v-2-2020-483-2020 ◽

2020 ◽

Vol V-2-2020 ◽

pp. 483-492

Author(s):

D. Gritzner ◽

J. Ostermann

Keyword(s):

Time Window ◽

Domain Adaptation ◽

Geographical Area ◽

Model Performance ◽

Ground Truth ◽

Semantic Segmentation ◽

Training Data ◽

Aerial Images ◽

Target Domain ◽

Training Examples

Abstract. Modern machine learning, especially deep learning, which is used in a variety of applications, requires a lot of labelled data for model training. Having an insufficient amount of training examples leads to models which do not generalize well to new input instances. This is a particular significant problem for tasks involving aerial images: often training data is only available for a limited geographical area and a narrow time window, thus leading to models which perform poorly in different regions, at different times of day, or during different seasons. Domain adaptation can mitigate this issue by using labelled source domain training examples and unlabeled target domain images to train a model which performs well on both domains. Modern adversarial domain adaptation approaches use unpaired data. We propose using pairs of semantically similar images, i.e., whose segmentations are accurate predictions of each other, for improved model performance. In this paper we show that, as an upper limit based on ground truth, using semantically paired aerial images during training almost always increases model performance with an average improvement of 4.2% accuracy and .036 mean intersection-over-union (mIoU). Using a practical estimate of semantic similarity, we still achieve improvements in more than half of all cases, with average improvements of 2.5% accuracy and .017 mIoU in those cases.

Download Full-text

Learning Descriptors Invariance through Equivalence Relations within Manifold: A New Approach to Expression Invariant 3D Face Recognition

Journal of Imaging ◽

10.3390/jimaging6110112 ◽

2020 ◽

Vol 6 (11) ◽

pp. 112

Author(s):

Faisal R. Al-Osaimi

Keyword(s):

Search Algorithm ◽

Recognition Performance ◽

Feature Space ◽

Training Data ◽

Equivalence Relations ◽

Graph Search ◽

Search Problem ◽

New Approach ◽

Unique Approach ◽

Training Examples

This paper presents a unique approach for the dichotomy between useful and adverse variations of key-point descriptors, namely the identity and the expression variations in the descriptor (feature) space. The descriptors variations are learned from training examples. Based on labels of the training data, the equivalence relations among the descriptors are established. Both types of descriptor variations are represented by a graph embedded in the descriptor manifold. Invariant recognition is then conducted as a graph search problem. A heuristic graph search algorithm suitable for the recognition under this setup was devised. The proposed approach was tested on the FRGC v2.0, the Bosphorus and the 3D TEC datasets. It has shown to enhance the recognition performance, under expression variations, by considerable margins.

Download Full-text

Multi-Source Distilling Domain Adaptation

Proceedings of the AAAI Conference on Artificial Intelligence ◽

10.1609/aaai.v34i07.6997 ◽

2020 ◽

Vol 34 (07) ◽

pp. 12975-12983

Author(s):

Sicheng Zhao ◽

Guangzhi Wang ◽

Shanghang Zhang ◽

Yang Gu ◽

Yaxian Li ◽

...

Keyword(s):

Domain Adaptation ◽

Feature Space ◽

Wasserstein Distance ◽

Training Data ◽

Source Distribution ◽

Single Source ◽

Multiple Sources ◽

Target Domain ◽

Target Feature ◽

Training Samples

Deep neural networks suffer from performance decay when there is domain shift between the labeled source domain and unlabeled target domain, which motivates the research on domain adaptation (DA). Conventional DA methods usually assume that the labeled data is sampled from a single source distribution. However, in practice, labeled data may be collected from multiple sources, while naive application of the single-source DA algorithms may lead to suboptimal solutions. In this paper, we propose a novel multi-source distilling domain adaptation (MDDA) network, which not only considers the different distances among multiple sources and the target, but also investigates the different similarities of the source samples to the target ones. Specifically, the proposed MDDA includes four stages: (1) pre-train the source classifiers separately using the training data from each source; (2) adversarially map the target into the feature space of each source respectively by minimizing the empirical Wasserstein distance between source and target; (3) select the source training samples that are closer to the target to fine-tune the source classifiers; and (4) classify each encoded target feature by corresponding source classifier, and aggregate different predictions using respective domain weight, which corresponds to the discrepancy between each source and target. Extensive experiments are conducted on public DA benchmarks, and the results demonstrate that the proposed MDDA significantly outperforms the state-of-the-art approaches. Our source code is released at: https://github.com/daoyuan98/MDDA.

Download Full-text