scholarly journals Cross-Modality Paired-Images Generation for RGB-Infrared Person Re-Identification

2020 ◽  
Vol 34 (07) ◽  
pp. 12144-12151
Author(s):  
Guan-An Wang ◽  
Tianzhu Zhang ◽  
Yang Yang ◽  
Jian Cheng ◽  
Jianlong Chang ◽  
...  
Keyword(s):  
State Of The Art ◽  
Experimental Results ◽  
Fine Grained ◽  
Invariant Features ◽  
Proposed Model ◽  
Art Methods ◽  
Conventional Methods

RGB-Infrared (IR) person re-identification is very challenging due to the large cross-modality variations between RGB and IR images. The key solution is to learn aligned features to the bridge RGB and IR modalities. However, due to the lack of correspondence labels between every pair of RGB and IR images, most methods try to alleviate the variations with set-level alignment by reducing the distance between the entire RGB and IR sets. However, this set-level alignment may lead to misalignment of some instances, which limits the performance for RGB-IR Re-ID. Different from existing methods, in this paper, we propose to generate cross-modality paired-images and perform both global set-level and fine-grained instance-level alignments. Our proposed method enjoys several merits. First, our method can perform set-level alignment by disentangling modality-specific and modality-invariant features. Compared with conventional methods, ours can explicitly remove the modality-specific features and the modality variation can be better reduced. Second, given cross-modality unpaired-images of a person, our method can generate cross-modality paired images from exchanged images. With them, we can directly perform instance-level alignment by minimizing distances of every pair of images. Extensive experimental results on two standard benchmarks demonstrate that the proposed model favourably against state-of-the-art methods. Especially, on SYSU-MM01 dataset, our model can achieve a gain of 9.2% and 7.7% in terms of Rank-1 and mAP. Code is available at https://github.com/wangguanan/JSIA-ReID.

scholarly journals Panoptic Segmentation-Based Attention for Image Captioning

2020 ◽  
Vol 10 (1) ◽  
pp. 391
Author(s):  
Wenjie Cai ◽  
Zheng Xiong ◽  
Xianfang Sun ◽  
Paul L. Rosin ◽  
Longcun Jin ◽  
...  
Keyword(s):  
Main Part ◽  
State Of The Art ◽  
Image Representation ◽  
Experimental Results ◽  
Competitive Performance ◽  
Image Captioning ◽  
Feature Vectors ◽  
Fine Grained ◽  
Art Methods

Image captioning is the task of generating textual descriptions of images. In order to obtain a better image representation, attention mechanisms have been widely adopted in image captioning. However, in existing models with detection-based attention, the rectangular attention regions are not fine-grained, as they contain irrelevant regions (e.g., background or overlapped regions) around the object, making the model generate inaccurate captions. To address this issue, we propose panoptic segmentation-based attention that performs attention at a mask-level (i.e., the shape of the main part of an instance). Our approach extracts feature vectors from the corresponding segmentation regions, which is more fine-grained than current attention mechanisms. Moreover, in order to process features of different classes independently, we propose a dual-attention module which is generic and can be applied to other frameworks. Experimental results showed that our model could recognize the overlapped objects and understand the scene better. Our approach achieved competitive performance against state-of-the-art methods. We made our code available.

scholarly journals Automatic Detection of Discrimination Actions from Social Images

Electronics ◽  
2021 ◽  
Vol 10 (3) ◽  
pp. 325
Author(s):  
Zhihao Wu ◽  
Baopeng Zhang ◽  
Tianchen Zhou ◽  
Yan Li ◽  
Jianping Fan
Keyword(s):  
Action Recognition ◽  
State Of The Art ◽  
Automatic Detection ◽  
Experimental Results ◽  
Practical Approach ◽  
Detection And Identification ◽  
Art Methods ◽  
Image Set ◽  
Social Images ◽  
Relationship Identification

In this paper, we developed a practical approach for automatic detection of discrimination actions from social images. Firstly, an image set is established, in which various discrimination actions and relations are manually labeled. To the best of our knowledge, this is the first work to create a dataset for discrimination action recognition and relationship identification. Secondly, a practical approach is developed to achieve automatic detection and identification of discrimination actions and relationships from social images. Thirdly, the task of relationship identification is seamlessly integrated with the task of discrimination action recognition into one single network called the Co-operative Visual Translation Embedding++ network (CVTransE++). We also compared our proposed method with numerous state-of-the-art methods, and our experimental results demonstrated that our proposed methods can significantly outperform state-of-the-art approaches.

scholarly journals A Deep Learning Approach to Predict Autism Spectrum Disorder Using Multisite Resting-State fMRI

2021 ◽  
Vol 11 (8) ◽  
pp. 3636
Author(s):  
Faria Zarin Subah ◽  
Kaushik Deb ◽  
Pranab Kumar Dhar ◽  
Takeshi Koshiba
Keyword(s):  
Autism Spectrum Disorder ◽  
Resting State ◽  
State Of The Art ◽  
Resting State Fmri ◽  
Autism Spectrum ◽  
Spectrum Disorder ◽  
Bootstrap Analysis ◽  
Proposed Model ◽  
Art Methods ◽  
The Mean

Autism spectrum disorder (ASD) is a complex and degenerative neuro-developmental disorder. Most of the existing methods utilize functional magnetic resonance imaging (fMRI) to detect ASD with a very limited dataset which provides high accuracy but results in poor generalization. To overcome this limitation and to enhance the performance of the automated autism diagnosis model, in this paper, we propose an ASD detection model using functional connectivity features of resting-state fMRI data. Our proposed model utilizes two commonly used brain atlases, Craddock 200 (CC200) and Automated Anatomical Labelling (AAL), and two rarely used atlases Bootstrap Analysis of Stable Clusters (BASC) and Power. A deep neural network (DNN) classifier is used to perform the classification task. Simulation results indicate that the proposed model outperforms state-of-the-art methods in terms of accuracy. The mean accuracy of the proposed model was 88%, whereas the mean accuracy of the state-of-the-art methods ranged from 67% to 85%. The sensitivity, F1-score, and area under receiver operating characteristic curve (AUC) score of the proposed model were 90%, 87%, and 96%, respectively. Comparative analysis on various scoring strategies show the superiority of BASC atlas over other aforementioned atlases in classifying ASD and control.

An Adaptive Scheme to Achieve Fine Grained Video Scaling

2017 ◽  
Vol 8 (1) ◽  
pp. 43
Author(s):  
S Safinaz ◽  
A. V. Ravi Kumar
Keyword(s):  
Reconstruction Error ◽  
Experimental Results ◽  
Adaptive Scheme ◽  
Fine Grained ◽  
Proposed Model ◽  
Reconstruction Quality ◽  
Adaptive Reconstruction ◽  
Robust Adaptive ◽  
Parallel Configuration ◽  
Very High

<p>A robust Adaptive Reconstruction Error Minimization Convolution Neural Network (<strong> ARemCNN</strong>) architecture introduced to provide high reconstruction quality from low resolution using parallel configuration. Our proposed model can easily train the bulky datasets such as YUV21 and Videoset4.Our experimental results shows that our model outperforms many existing techniques in terms of PSNR, SSIM and reconstruction quality. The experimental results shows that our average PSNR result is 39.81 considering upscale-2, 35.56 for upscale-3 and 33.77 for upscale-4 for Videoset4 dataset which is very high in contrast to other existing techniques. Similarly, the experimental results shows that our average PSNR result is 38.71 considering upscale-2, 34.58 for upscale-3 and 33.047 for upscale-4 for YUV21 dataset.</p>

scholarly journals Multimodal Summarization with Guidance of Multimodal Reference

2020 ◽  
Vol 34 (05) ◽  
pp. 9749-9756
Author(s):  
Junnan Zhu ◽  
Yu Zhou ◽  
Jiajun Zhang ◽  
Haoran Li ◽  
Chengqing Zong ◽  
...  
Keyword(s):  
Objective Function ◽  
Evaluation Method ◽  
Reference Data ◽  
State Of The Art ◽  
Semantic Space ◽  
Experimental Results ◽  
Model Output ◽  
Proposed Model ◽  
Evaluation Metric

Multimodal summarization with multimodal output (MSMO) is to generate a multimodal summary for a multimodal news report, which has been proven to effectively improve users' satisfaction. The existing MSMO methods are trained by the target of text modality, leading to the modality-bias problem that ignores the quality of model-selected image during training. To alleviate this problem, we propose a multimodal objective function with the guidance of multimodal reference to use the loss from the summary generation and the image selection. Due to the lack of multimodal reference data, we present two strategies, i.e., ROUGE-ranking and Order-ranking, to construct the multimodal reference by extending the text reference. Meanwhile, to better evaluate multimodal outputs, we propose a novel evaluation metric based on joint multimodal representation, projecting the model output and multimodal reference into a joint semantic space during evaluation. Experimental results have shown that our proposed model achieves the new state-of-the-art on both automatic and manual evaluation metrics. Besides, our proposed evaluation method can effectively improve the correlation with human judgments.

scholarly journals Semantic Locality-Aware Deformable Network for Clothing Segmentation

2018 ◽  
Author(s):  
Wei Ji ◽  
Xi Li ◽  
Yueting Zhuang ◽  
Omar El Farouk Bourahla ◽  
Yixin Ji ◽  
...  
Keyword(s):  
Learning Process ◽  
State Of The Art ◽  
Semantic Segmentation ◽  
Small Sample ◽  
Sample Problem ◽  
Fine Grained ◽  
Domain Specific ◽  
Proposed Model ◽  
Segmentation Framework ◽  
Small Sample Problem

Clothing segmentation is a challenging vision problem typically implemented within a fine-grained semantic segmentation framework. Different from conventional segmentation, clothing segmentation has some domain-specific properties such as texture richness, diverse appearance variations, non-rigid geometry deformations, and small sample learning. To deal with these points, we propose a semantic locality-aware segmentation model, which adaptively attaches an original clothing image with a semantically similar (e.g., appearance or pose) auxiliary exemplar by search. Through considering the interactions of the clothing image and its exemplar, more intrinsic knowledge about the locality manifold structures of clothing images is discovered to make the learning process of small sample problem more stable and tractable. Furthermore, we present a CNN model based on the deformable convolutions to extract the non-rigid geometry-aware features for clothing images. Experimental results demonstrate the effectiveness of the proposed model against the state-of-the-art approaches.

scholarly journals RefNet: A Reference-Aware Network for Background Based Conversation

2020 ◽  
Vol 34 (05) ◽  
pp. 8496-8503 ◽  
Author(s):  
Chuan Meng ◽  
Pengjie Ren ◽  
Zhumin Chen ◽  
Christof Monz ◽  
Jun Ma ◽  
...  
Keyword(s):  
Semantic Information ◽  
State Of The Art ◽  
Experimental Results ◽  
Background Information ◽  
Art Methods ◽  
The Right

Existing conversational systems tend to generate generic responses. Recently, Background Based Conversation (BBCs) have been introduced to address this issue. Here, the generated responses are grounded in some background information. The proposed methods for BBCs are able to generate more informative responses, however, they either cannot generate natural responses or have difficulties in locating the right background information. In this paper, we propose a Reference-aware Network (RefNet) to address both issues. Unlike existing methods that generate responses token by token, RefNet incorporates a novel reference decoder that provides an alternative way to learn to directly select a semantic unit (e.g., a span containing complete semantic information) from the background. Experimental results show that RefNet significantly outperforms state-of-the-art methods in terms of both automatic and human evaluations, indicating that RefNet can generate more appropriate and human-like responses.

scholarly journals A Topic-Aware Reinforced Model for Weakly Supervised Stance Detection

2019 ◽  
Vol 33 ◽  
pp. 7249-7256
Author(s):  
Penghui Wei ◽  
Wenji Mao ◽  
Guandan Chen
Keyword(s):  
Reinforcement Learning ◽  
Opinion Mining ◽  
State Of The Art ◽  
Public Attitudes ◽  
Representation Learning ◽  
Experimental Results ◽  
Training Data ◽  
Policy Network ◽  
Proposed Model ◽  
Weakly Supervised

Analyzing public attitudes plays an important role in opinion mining systems. Stance detection aims to determine from a text whether its author is in favor of, against, or neutral towards a given target. One challenge of this task is that a text may not explicitly express an attitude towards the target, but existing approaches utilize target content alone to build models. Moreover, although weakly supervised approaches have been proposed to ease the burden of manually annotating largescale training data, such approaches are confronted with noisy labeling problem. To address the above two issues, in this paper, we propose a Topic-Aware Reinforced Model (TARM) for weakly supervised stance detection. Our model consists of two complementary components: (1) a detection network that incorporates target-related topic information into representation learning for identifying stance effectively; (2) a policy network that learns to eliminate noisy instances from auto-labeled data based on off-policy reinforcement learning. Two networks are alternately optimized to improve each other’s performances. Experimental results demonstrate that our proposed model TARM outperforms the state-of-the-art approaches.

scholarly journals Biomedical document triage using a hierarchical attention-based capsule network

2020 ◽  
Vol 21 (S13) ◽  
Author(s):  
Jian Wang ◽  
Mengying Li ◽  
Qishuai Diao ◽  
Hongfei Lin ◽  
Zhihao Yang ◽  
...  
Keyword(s):  
Neural Networks ◽  
Information Extraction ◽  
Precision Medicine ◽  
State Of The Art ◽  
Attention Mechanism ◽  
Feature Representation ◽  
Experimental Results ◽  
Biomedical Domain ◽  
Proposed Model ◽  
Document Triage

Abstract Background Biomedical document triage is the foundation of biomedical information extraction, which is important to precision medicine. Recently, some neural networks-based methods have been proposed to classify biomedical documents automatically. In the biomedical domain, documents are often very long and often contain very complicated sentences. However, the current methods still find it difficult to capture important features across sentences. Results In this paper, we propose a hierarchical attention-based capsule model for biomedical document triage. The proposed model effectively employs hierarchical attention mechanism and capsule networks to capture valuable features across sentences and construct a final latent feature representation for a document. We evaluated our model on three public corpora. Conclusions Experimental results showed that both hierarchical attention mechanism and capsule networks are helpful in biomedical document triage task. Our method proved itself highly competitive or superior compared with other state-of-the-art methods.

A multi-grained aspect vector learning model for unsupervised aspect identification

2021 ◽  
pp. 1-11
Author(s):  
Jinglei Shi ◽  
Junjun Guo ◽  
Zhengtao Yu ◽  
Yan Xiang
Keyword(s):  
Vector Space ◽  
Sentiment Analysis ◽  
Membership Function ◽  
Topic Models ◽  
Experimental Results ◽  
Coarse Grained ◽  
Product Reviews ◽  
Fine Grained ◽  
Proposed Model ◽  
Identification Model

Unsupervised aspect identification is a challenging task in aspect-based sentiment analysis. Traditional topic models are usually used for this task, but they are not appropriate for short texts such as product reviews. In this work, we propose an aspect identification model based on aspect vector reconstruction. A key of our model is that we make connections between sentence vectors and multi-grained aspect vectors using fuzzy k-means membership function. Furthermore, to make full use of different aspect representations in vector space, we reconstruct sentence vectors based on coarse-grained aspect vectors and fine-grained aspect vectors simultaneously. The resulting model can therefore learn better aspect representations. Experimental results on two datasets from different domains show that our proposed model can outperform a few baselines in terms of aspect identification and topic coherence of the extracted aspect terms.

Sign in / Sign up

Export Citation Format

Share Document