scholarly journals Structure-Aware Feature Fusion for Unsupervised Domain Adaptation

2020 ◽  
Vol 34 (07) ◽  
pp. 10567-10574
Author(s):  
Qingchao Chen ◽  
Yang Liu
Keyword(s):  
Spatial Structure ◽  
Feature Fusion ◽  
Domain Adaptation ◽  
Structural Information ◽  
Feature Representation ◽  
Local Feature ◽  
Global Feature ◽  
Feature Maps ◽  
Fine Grained ◽  
Unsupervised Domain Adaptation

Unsupervised domain Adaptation (UDA) aims to learn and transfer generalized features from a labelled source domain to a target domain without any annotations. Existing methods only aligning high-level representation but without exploiting the complex multi-class structure and local spatial structure. This is problematic as 1) the model is prone to negative transfer when the features from different classes are misaligned; 2) missing the local spatial structure poses a major obstacle in performing the fine-grained feature alignment. In this paper, we integrate the valuable information conveyed in classifier prediction and local feature maps into global feature representation and then perform a single mini-max game to make it domain invariant. In this way, the domain-invariant feature not only describes the holistic representation of the original image but also preserves mode-structure and fine-grained spatial structural information. The feature integration is achieved by estimating and maximizing the mutual information (MI) among the global feature, local feature and classifier prediction simultaneously. As the MI is hard to measure directly in high-dimension spaces, we adopt a new objective function that implicitly maximizes the MI via an effective sampling strategy and a discriminator design. Our STructure-Aware Feature Fusion (STAFF) network achieves the state-of-the-art performances in various UDA datasets.

scholarly journals Exploiting Local Feature Patterns for Unsupervised Domain Adaptation

2019 ◽  
Vol 33 ◽  
pp. 5401-5408 ◽  
Author(s):  
Jun Wen ◽  
Risheng Liu ◽  
Nenggan Zheng ◽  
Qian Zheng ◽  
Zhefeng Gong ◽  
...  
Keyword(s):  
Negative Transfer ◽  
Domain Adaptation ◽  
Local Feature ◽  
Fine Grained ◽  
Unsupervised Domain Adaptation ◽  
Distribution Matching ◽  
Benchmark Datasets ◽  
Invariant Representations ◽  
Multi Mode ◽  
Feature Alignment

Unsupervised domain adaptation methods aim to alleviate performance degradation caused by domain-shift by learning domain-invariant representations. Existing deep domain adaptation methods focus on holistic feature alignment by matching source and target holistic feature distributions, without considering local features and their multi-mode statistics. We show that the learned local feature patterns are more generic and transferable and a further local feature distribution matching enables fine-grained feature alignment. In this paper, we present a method for learning domain-invariant local feature patterns and jointly aligning holistic and local feature statistics. Comparisons to the state-of-the-art unsupervised domain adaptation methods on two popular benchmark datasets demonstrate the superiority of our approach and its effectiveness on alleviating negative transfer.

scholarly journals Rescaling Egocentric Vision: Collection, Pipeline and Challenges for EPIC-KITCHENS-100

2021 ◽  
Author(s):  
Dima Damen ◽  
Hazel Doughty ◽  
Giovanni Maria Farinella ◽  
Antonino Furnari ◽  
Evangelos Kazakos ◽  
...  
Keyword(s):  
Action Recognition ◽  
Domain Adaptation ◽  
Weak Supervision ◽  
Action Detection ◽  
Previous Version ◽  
Fine Grained ◽  
Unsupervised Domain Adaptation ◽  
Egocentric Vision ◽  
New Challenges

AbstractThis paper introduces the pipeline to extend the largest dataset in egocentric vision, EPIC-KITCHENS. The effort culminates in EPIC-KITCHENS-100, a collection of 100 hours, 20M frames, 90K actions in 700 variable-length videos, capturing long-term unscripted activities in 45 environments, using head-mounted cameras. Compared to its previous version (Damen in Scaling egocentric vision: ECCV, 2018), EPIC-KITCHENS-100 has been annotated using a novel pipeline that allows denser (54% more actions per minute) and more complete annotations of fine-grained actions (+128% more action segments). This collection enables new challenges such as action detection and evaluating the “test of time”—i.e. whether models trained on data collected in 2018 can generalise to new footage collected two years later. The dataset is aligned with 6 challenges: action recognition (full and weak supervision), action detection, action anticipation, cross-modal retrieval (from captions), as well as unsupervised domain adaptation for action recognition. For each challenge, we define the task, provide baselines and evaluation metrics.

Joint metric and feature representation learning for unsupervised domain adaptation

2020 ◽  
Vol 192 ◽  
pp. 105222 ◽  
Author(s):  
Yue Xie ◽  
Zhekai Du ◽  
Jingjing Li ◽  
Mengmeng Jing ◽  
Erpeng Chen ◽  
...  
Keyword(s):  
Domain Adaptation ◽  
Representation Learning ◽  
Feature Representation ◽  
Unsupervised Domain Adaptation

scholarly journals P-Norm Attention Deep CORAL: Extending Correlation Alignment Using Attention and the P-Norm Loss Function

2021 ◽  
Vol 11 (11) ◽  
pp. 5267
Author(s):  
Zhi-Yong Wang ◽  
Dae-Ki Kang
Keyword(s):  
Loss Function ◽  
Domain Adaptation ◽  
Original Data ◽  
Good Representation ◽  
Feature Maps ◽  
Target Domain ◽  
Unsupervised Domain Adaptation ◽  
Adaptation Method ◽  
Deep Coral

CORrelation ALignment (CORAL) is an unsupervised domain adaptation method that uses a linear transformation to align the covariances of source and target domains. Deep CORAL extends CORAL with a nonlinear transformation using a deep neural network and adds CORAL loss as a part of the total loss to align the covariances of source and target domains. However, there are still two problems to be solved in Deep CORAL: features extracted from AlexNet are not always a good representation of the original data, as well as joint training combined with both the classification and CORAL loss may not be efficient enough to align the distribution of the source and target domain. In this paper, we proposed two strategies: attention to improve the quality of feature maps and the p-norm loss function to align the distribution of the source and target features, further reducing the offset caused by the classification loss function. Experiments on the Office-31 dataset indicate that our proposed methodologies improved Deep CORAL in terms of performance.

scholarly journals U-shaped GAN for Semi-Supervised Learning and Unsupervised Domain Adaptation in High Resolution Chest Radiograph Segmentation

2022 ◽  
Vol 8 ◽  
Author(s):  
Hongyu Wang ◽  
Hong Gu ◽  
Pan Qin ◽  
Jia Wang
Keyword(s):  
Deep Learning ◽  
High Resolution ◽  
Supervised Learning ◽  
Chest Radiograph ◽  
Domain Adaptation ◽  
Medical Image Segmentation ◽  
Learning Approach ◽  
Feature Maps ◽  
Computational Burden ◽  
Unsupervised Domain Adaptation

Deep learning has achieved considerable success in medical image segmentation. However, applying deep learning in clinical environments often involves two problems: (1) scarcity of annotated data as data annotation is time-consuming and (2) varying attributes of different datasets due to domain shift. To address these problems, we propose an improved generative adversarial network (GAN) segmentation model, called U-shaped GAN, for limited-annotated chest radiograph datasets. The semi-supervised learning approach and unsupervised domain adaptation (UDA) approach are modeled into a unified framework for effective segmentation. We improve GAN by replacing the traditional discriminator with a U-shaped net, which predicts each pixel a label. The proposed U-shaped net is designed with high resolution radiographs (1,024 × 1,024) for effective segmentation while taking computational burden into account. The pointwise convolution is applied to U-shaped GAN for dimensionality reduction, which decreases the number of feature maps while retaining their salient features. Moreover, we design the U-shaped net with a pretrained ResNet-50 as an encoder to reduce the computational burden of training the encoder from scratch. A semi-supervised learning approach is proposed learning from limited annotated data while exploiting additional unannotated data with a pixel-level loss. U-shaped GAN is extended to UDA by taking the source and target domain data as the annotated data and the unannotated data in the semi-supervised learning approach, respectively. Compared to the previous models dealing with the aforementioned problems separately, U-shaped GAN is compatible with varying data distributions of multiple medical centers, with efficient training and optimizing performance. U-shaped GAN can be generalized to chest radiograph segmentation for clinical deployment. We evaluate U-shaped GAN with two chest radiograph datasets. U-shaped GAN is shown to significantly outperform the state-of-the-art models.

scholarly journals Non-locally Enhanced Feature Fusion Network for Aircraft Recognition in Remote Sensing Images

2020 ◽  
Vol 12 (4) ◽  
pp. 681
Author(s):  
Yunsheng Xiong ◽  
Xin Niu ◽  
Yong Dou ◽  
Hang Qie ◽  
Kang Wang
Keyword(s):  
Remote Sensing ◽  
Loss Function ◽  
Feature Fusion ◽  
Remote Sensing Images ◽  
Feature Maps ◽  
Long Distance ◽  
Test Dataset ◽  
Fine Grained ◽  
Recognition Ability ◽  
Discriminative Parts

Aircraft recognition has great application value, but aircraft in remote sensing images have some problems such as low resolution, poor contrasts, poor sharpness, and lack of details caused by the vertical view, which make the aircraft recognition very difficult. Especially when there are many kinds of aircraft and the differences between aircraft are subtle, the fine-grained recognition of aircraft is more challenging. In this paper, we propose a non-locally enhanced feature fusion network(NLFFNet) and attempt to make full use of the features from discriminative parts of aircraft. First, according to the long-distance self-correlation in aircraft images, we adopt non-locally enhanced operation and guide the network to pay more attention to the discriminating areas and enhance the features beneficial to classification. Second, we propose a part-level feature fusion mechanism(PFF), which crops 5 parts of the aircraft on the shared feature maps, then extracts the subtle features inside the parts through the part full connection layer(PFC) and fuses the features of these parts together through the combined full connection layer(CFC). In addition, by adopting the improved loss function, we can enhance the weight of hard examples in the loss function meanwhile reducing the weight of excessively hard examples, which improves the overall recognition ability of the network. The dataset includes 47 categories of aircraft, including many aircraft of the same family with slight differences in appearance, and our method can achieve 89.12% accuracy on the test dataset, which proves the effectiveness of our method.

scholarly journals An Adversarial Perturbation Oriented Domain Adaptation Approach for Semantic Segmentation

2020 ◽  
Vol 34 (07) ◽  
pp. 12613-12620 ◽  
Author(s):  
Jihan Yang ◽  
Ruijia Xu ◽  
Ruiyu Li ◽  
Xiaojuan Qi ◽  
Xiaoyong Shen ◽  
...  
Keyword(s):  
Marginal Distribution ◽  
Domain Adaptation ◽  
State Of The Art ◽  
Feature Space ◽  
Semantic Segmentation ◽  
The State ◽  
Object Size ◽  
Feature Maps ◽  
Feature Representations ◽  
Unsupervised Domain Adaptation

We focus on Unsupervised Domain Adaptation (UDA) for the task of semantic segmentation. Recently, adversarial alignment has been widely adopted to match the marginal distribution of feature representations across two domains globally. However, this strategy fails in adapting the representations of the tail classes or small objects for semantic segmentation since the alignment objective is dominated by head categories or large objects. In contrast to adversarial alignment, we propose to explicitly train a domain-invariant classifier by generating and defensing against pointwise feature space adversarial perturbations. Specifically, we firstly perturb the intermediate feature maps with several attack objectives (i.e., discriminator and classifier) on each individual position for both domains, and then the classifier is trained to be invariant to the perturbations. By perturbing each position individually, our model treats each location evenly regardless of the category or object size and thus circumvents the aforementioned issue. Moreover, the domain gap in feature space is reduced by extrapolating source and target perturbed features towards each other with attack on the domain discriminator. Our approach achieves the state-of-the-art performance on two challenging domain adaptation tasks for semantic segmentation: GTA5 → Cityscapes and SYNTHIA → Cityscapes.

scholarly journals Rich CNN Features for Water-Body Segmentation from Very High Resolution Aerial and Satellite Imagery

2021 ◽  
Vol 13 (10) ◽  
pp. 1912
Author(s):  
Zhili Zhang ◽  
Meng Lu ◽  
Shunping Ji ◽  
Huafen Yu ◽  
Chenhui Nie
Keyword(s):  
Remote Sensing ◽  
Feature Extraction ◽  
High Resolution ◽  
Satellite Imagery ◽  
Water Body ◽  
Feature Fusion ◽  
Feature Representation ◽  
Water Bodies ◽  
Feature Maps ◽  
Very High

Extracting water-bodies accurately is a great challenge from very high resolution (VHR) remote sensing imagery. The boundaries of a water body are commonly hard to identify due to the complex spectral mixtures caused by aquatic vegetation, distinct lake/river colors, silts near the bank, shadows from the surrounding tall plants, and so on. The diversity and semantic information of features need to be increased for a better extraction of water-bodies from VHR remote sensing images. In this paper, we address these problems by designing a novel multi-feature extraction and combination module. This module consists of three feature extraction sub-modules based on spatial and channel correlations in feature maps at each scale, which extract the complete target information from the local space, larger space, and between-channel relationship to achieve a rich feature representation. Simultaneously, to better predict the fine contours of water-bodies, we adopt a multi-scale prediction fusion module. Besides, to solve the semantic inconsistency of feature fusion between the encoding stage and the decoding stage, we apply an encoder-decoder semantic feature fusion module to promote fusion effects. We carry out extensive experiments in VHR aerial and satellite imagery respectively. The result shows that our method achieves state-of-the-art segmentation performance, surpassing the classic and recent methods. Moreover, our proposed method is robust in challenging water-body extraction scenarios.

Sign in / Sign up

Export Citation Format

Share Document