scholarly journals Residual Augmented Attentional U-Shaped Network for Spectral Reconstruction from RGB Images

2020 ◽  
Vol 13 (1) ◽  
pp. 115
Author(s):  
Jiaojiao Li ◽  
Chaoxiong Wu ◽  
Rui Song ◽  
Yunsong Li ◽  
Weiying Xie
Keyword(s):  
Superior Performance ◽  
Feature Maps ◽  
Deep Convolutional Neural Networks ◽  
Second Order Statistics ◽  
Feature Representations ◽  
Quantitative Measurements ◽  
Spectral Reconstruction ◽  
Perceptual Comparison ◽  
Benchmark Datasets ◽  
Rgb Images

Deep convolutional neural networks (CNNs) have been successfully applied to spectral reconstruction (SR) and acquired superior performance. Nevertheless, the existing CNN-based SR approaches integrate hierarchical features from different layers indiscriminately, lacking an investigation of the relationships of intermediate feature maps, which limits the learning power of CNNs. To tackle this problem, we propose a deep residual augmented attentional u-shape network (RA2UN) with several double improved residual blocks (DIRB) instead of paired plain convolutional units. Specifically, a trainable spatial augmented attention (SAA) module is developed to bridge the encoder and decoder to emphasize the features in the informative regions. Furthermore, we present a novel channel augmented attention (CAA) module embedded in the DIRB to rescale adaptively and enhance residual learning by using first-order and second-order statistics for stronger feature representations. Finally, a boundary-aware constraint is employed to focus on the salient edge information and recover more accurate high-frequency details. Experimental results on four benchmark datasets demonstrate that the proposed RA2UN network outperforms the state-of-the-art SR methods under quantitative measurements and perceptual comparison.

scholarly journals A Deep Hyper Siamese Network for Real-Time Object Tracking

2020 ◽  
Vol 8 (1) ◽  
pp. 35-46
Author(s):  
Yongpeng Zhao ◽  
Lasheng Yu ◽  
Xiaopeng Zheng
Keyword(s):  
Object Tracking ◽  
Target Object ◽  
Visual Object ◽  
Feature Maps ◽  
Deep Convolutional Neural Networks ◽  
Backbone Networks ◽  
Feature Representations ◽  
Siamese Network ◽  
Benchmark Datasets ◽  
Siamese Networks

Siamese networks have drawn increasing interest in the field of visual object tracking due to their balance of precision and efficiency. However, Siamese trackers use relatively shallow backbone networks, such as AlexNet, and therefore do not take full advantage of the capabilities of modern deep convolutional neural networks (CNNs). Moreover, the feature representations of the target object in a Siamese tracker are extracted through the last layer of CNNs and mainly capture semantic information, which causes the tracker's precision to be relatively low and to drift easily in the presence of similar distractors. In this paper, a new nonpadding residual unit (NPRU) is designed and used to stack a 22-layer deep ResNet, referred as ResNet22. After utilizing ResNet22 as the backbone network, we can build a deep Siamese network, which can greatly enhance the tracking performance. Considering that the different levels of the feature maps of the CNN represent different aspects of the target object, we aggregated different deep convolutional layers to make use of ResNet22's multilevel feature maps, which can form hyperfeature representations of targets. The designed deep hyper Siamese network is named DHSiam. Experimental results show that DHSiam has achieved significant improvement on multiple benchmark datasets.

scholarly journals Facial Action Unit Intensity Estimation via Semantic Correspondence Learning with Dynamic Graph Convolution

2020 ◽  
Vol 34 (07) ◽  
pp. 12701-12708
Author(s):  
Yingruo Fan ◽  
Jacqueline Lam ◽  
Victor Li
Keyword(s):  
Probabilistic Models ◽  
Superior Performance ◽  
Feature Maps ◽  
Dynamic Graph ◽  
Intensity Estimation ◽  
Facial Action ◽  
Learning Framework ◽  
Benchmark Datasets ◽  
Relationship Of ◽  
Semantic Correspondence

The intensity estimation of facial action units (AUs) is challenging due to subtle changes in the person's facial appearance. Previous approaches mainly rely on probabilistic models or predefined rules for modeling co-occurrence relationships among AUs, leading to limited generalization. In contrast, we present a new learning framework that automatically learns the latent relationships of AUs via establishing semantic correspondences between feature maps. In the heatmap regression-based network, feature maps preserve rich semantic information associated with AU intensities and locations. Moreover, the AU co-occurring pattern can be reflected by activating a set of feature channels, where each channel encodes a specific visual pattern of AU. This motivates us to model the correlation among feature channels, which implicitly represents the co-occurrence relationship of AU intensity levels. Specifically, we introduce a semantic correspondence convolution (SCC) module to dynamically compute the correspondences from deep and low resolution feature maps, and thus enhancing the discriminability of features. The experimental results demonstrate the effectiveness and the superior performance of our method on two benchmark datasets.

scholarly journals Rectified Binary Convolutional Networks for Enhancing the Performance of 1-bit DCNNs

2019 ◽  
Author(s):  
Chunlei Liu ◽  
Wenrui Ding ◽  
Xin Xia ◽  
Yuan Hu ◽  
Baochang Zhang ◽  
...  
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Superior Performance ◽  
Feature Maps ◽  
Unified Framework ◽  
Deep Convolutional Neural Networks ◽  
Convolutional Networks ◽  
Computation Efficiency ◽  
Significant Performance ◽  
Binary Network

Binarized  convolutional neural networks (BCNNs) are widely used to improve memory and computation efficiency of deep convolutional neural networks (DCNNs) for mobile and AI chips based applications. However, current BCNNs are not able to fully explore their corresponding full-precision models, causing a significant performance gap between them. In this paper, we propose rectified binary convolutional networks (RBCNs), towards optimized BCNNs, by combining full-precision kernels and feature maps to rectify the binarization process in a unified framework. In particular, we use a GAN to train the 1-bit binary network with the guidance of its corresponding full-precision model, which significantly improves the performance of BCNNs. The rectified convolutional layers are generic and flexible, and can be easily incorporated into existing DCNNs such as WideResNets and ResNets. Extensive experiments demonstrate the superior performance of the proposed RBCNs over state-of-the-art BCNNs. In particular, our method shows strong generalization on the object tracking task.

Colour Fidelity in Spectral Reconstruction from RGB Images

2020 ◽  
Vol 2020 (1) ◽  
pp. 144-148
Author(s):  
Yi-Tun Lin
Keyword(s):  
Sparse Coding ◽  
State Of The Art ◽  
Linear Mapping ◽  
Superior Performance ◽  
Response Functions ◽  
Simple Linear Regression ◽  
Spectral Reconstruction ◽  
Recent Developments ◽  
Rgb Images ◽  
Spectral Recovery

Spectral reconstruction (SR) aims to recover high resolution spectra from RGB images. Recent developments - leading by Convolutional Neural Networks (CNN) - can already solve this problem with low errors. However, those leading methods do not explicitly ensure the predicted spectra will re-integrate (with the underlying camera response functions) into the same RGB colours as the ones they are recovered from, namely the 'colour fidelity' problem. The purpose of this paper is to show, visually and quantitatively, how well (or bad) the existing SR models maintain colour fidelity. Three main approaches are evaluated - regression, sparse coding and CNN. Furthermore, aiming for a more realistic setting, the evaluations are done on real RGB images and the 'end-of-pipe' images (i.e.rendered images shown to the end users) are provided for visual comparisons. It is shown that the state-of-the-art CNN-based model, despite of the superior performance in spectral recovery, introduces significant colour shifts in the final images. Interestingly, the leading sparse coding and the simple linear regression model, both of which are based on linear mapping, best preserve the colour fidelity in SR.

GreyReID: A Novel Two-stream Deep Framework with RGB-grey Information for Person Re-identification

2021 ◽  
Vol 17 (1) ◽  
pp. 1-22
Author(s):  
Lei Qi ◽  
Lei Wang ◽  
Jing Huo ◽  
Yinghuan Shi ◽  
Yang Gao
Keyword(s):  
Loss Function ◽  
Feature Representation ◽  
Query Image ◽  
Joint Learning ◽  
Ranking List ◽  
Generalization Ability ◽  
Feature Representations ◽  
Benchmark Datasets ◽  
Rgb Images ◽  
Fine Tune

In this article, we observe that most false positive images (i.e., different identities with query images) in the top ranking list usually have the similar color information with the query image in person re-identification (Re-ID). Meanwhile, when we use the greyscale images generated from RGB images to conduct the person Re-ID task, some hard query images can obtain better performance compared with using RGB images. Therefore, RGB and greyscale images seem to be complementary to each other for person Re-ID. In this article, we aim to utilize both RGB and greyscale images to improve the person Re-ID performance. To this end, we propose a novel two-stream deep neural network with RGB-grey information, which can effectively fuse RGB and greyscale feature representations to enhance the generalization ability of Re-ID. First, we convert RGB images to greyscale images in each training batch. Based on these RGB and greyscale images, we train the RGB and greyscale branches, respectively. Second, to build up connections between RGB and greyscale branches, we merge the RGB and greyscale branches into a new joint branch. Finally, we concatenate the features of all three branches as the final feature representation for Re-ID. Moreover, in the training process, we adopt the joint learning scheme to simultaneously train each branch by the independent loss function, which can enhance the generalization ability of each branch. Besides, a global loss function is utilized to further fine-tune the final concatenated feature. The extensive experiments on multiple benchmark datasets fully show that the proposed method can outperform the state-of-the-art person Re-ID methods. Furthermore, using greyscale images can indeed improve the person Re-ID performance in the proposed deep framework.

A Computer Vision-Based Yoga Pose Grading Approach Using Contrastive Skeleton Feature Representations

Healthcare ◽  
2021 ◽  
Vol 10 (1) ◽  
pp. 36
Author(s):  
Yubin Wu ◽  
Qianqian Lin ◽  
Mingrun Yang ◽  
Jing Liu ◽  
Jing Tian ◽  
...  
Keyword(s):  
Computer Vision ◽  
Quantitative Evaluation ◽  
Human Body ◽  
Superior Performance ◽  
Feature Representations ◽  
Feature Encoding ◽  
New Strategy ◽  
Benchmark Datasets

The main objective of yoga pose grading is to assess the input yoga pose and compare it to a standard pose in order to provide a quantitative evaluation as a grade. In this paper, a computer vision-based yoga pose grading approach is proposed using contrastive skeleton feature representations. First, the proposed approach extracts human body skeleton keypoints from the input yoga pose image and then feeds their coordinates into a pose feature encoder, which is trained using contrastive triplet examples; finally, a comparison of similar encoded pose features is made. Furthermore, to tackle the inherent challenge of composing contrastive examples in pose feature encoding, this paper proposes a new strategy to use both a coarse triplet example—comprised of an anchor, a positive example from the same category, and a negative example from a different category, and a fine triplet example—comprised of an anchor, a positive example, and a negative example from the same category with different pose qualities. Extensive experiments are conducted using two benchmark datasets to demonstrate the superior performance of the proposed approach.

scholarly journals Learning Instance-wise Sparsity for Accelerating Deep Models

2019 ◽  
Author(s):  
Chuanjian Liu ◽  
Yunhe Wang ◽  
Kai Han ◽  
Chunjing Xu ◽  
Chang Xu
Keyword(s):  
Neural Networks ◽  
Network Performance ◽  
High Efficiency ◽  
Feature Maps ◽  
Deep Convolutional Neural Networks ◽  
Learning Tasks ◽  
Intermediate Layers ◽  
Benchmark Datasets ◽  
The Difference ◽  
Trained Neural Network

Exploring deep convolutional neural networks of high efficiency and low memory usage is very essential for a wide variety of machine learning tasks. Most of existing approaches used to accelerate deep models by manipulating parameters or filters without data, e.g., pruning and decomposition. In contrast, we study this problem from a different perspective by respecting the difference between data. An instance-wise feature pruning is developed by identifying informative features for different instances. Specifically, by investigating a feature decay regularization, we expect intermediate feature maps of each instance in deep neural networks to be sparse while preserving the overall network performance. During online inference, subtle features of input images extracted by intermediate layers of a well-trained neural network can be eliminated to accelerate the subsequent calculations. We further take coefficient of variation as a measure to select the layers that are appropriate for acceleration. Extensive experiments conducted on benchmark datasets and networks demonstrate the effectiveness of the proposed method.

scholarly journals A Novel Upsampling and Context Convolution for Image Semantic Segmentation

Sensors ◽  
2021 ◽  
Vol 21 (6) ◽  
pp. 2170
Author(s):  
Khwaja Monib Sediqi ◽  
Hyo Jong Lee
Keyword(s):  
Spatial Information ◽  
Robot Vision ◽  
Semantic Segmentation ◽  
Autonomous Driving ◽  
Local Context ◽  
Feature Maps ◽  
Deep Convolutional Neural Networks ◽  
Object Boundary ◽  
Benchmark Datasets ◽  
Convolution Method

Semantic segmentation, which refers to pixel-wise classification of an image, is a fundamental topic in computer vision owing to its growing importance in the robot vision and autonomous driving sectors. It provides rich information about objects in the scene such as object boundary, category, and location. Recent methods for semantic segmentation often employ an encoder-decoder structure using deep convolutional neural networks. The encoder part extracts features of the image using several filters and pooling operations, whereas the decoder part gradually recovers the low-resolution feature maps of the encoder into a full input resolution feature map for pixel-wise prediction. However, the encoder-decoder variants for semantic segmentation suffer from severe spatial information loss, caused by pooling operations or stepwise convolutions, and does not consider the context in the scene. In this paper, we propose a novel dense upsampling convolution method based on a guided filter to effectively preserve the spatial information of the image in the network. We further propose a novel local context convolution method that not only covers larger-scale objects in the scene but covers them densely for precise object boundary delineation. Theoretical analyses and experimental results on several benchmark datasets verify the effectiveness of our method. Qualitatively, our approach delineates object boundaries at a level of accuracy that is beyond the current excellent methods. Quantitatively, we report a new record of 82.86% and 81.62% of pixel accuracy on ADE20K and Pascal-Context benchmark datasets, respectively. In comparison with the state-of-the-art methods, the proposed method offers promising improvements.

scholarly journals Drug-Drug Interaction Predicting by Neural Network Using Integrated Similarity

2019 ◽  
Vol 9 (1) ◽  
Author(s):  
Narjes Rohani ◽  
Changiz Eslahchi
Keyword(s):  
Neural Network ◽  
Drug Interaction ◽  
Side Effect ◽  
Network Architecture ◽  
Selection Process ◽  
Superior Performance ◽  
Multiple Drug ◽  
Interaction Prediction ◽  
Benchmark Datasets ◽  
Drug Drug Interaction

Abstract Drug-Drug Interaction (DDI) prediction is one of the most critical issues in drug development and health. Proposing appropriate computational methods for predicting unknown DDI with high precision is challenging. We proposed "NDD: Neural network-based method for drug-drug interaction prediction" for predicting unknown DDIs using various information about drugs. Multiple drug similarities based on drug substructure, target, side effect, off-label side effect, pathway, transporter, and indication data are calculated. At first, NDD uses a heuristic similarity selection process and then integrates the selected similarities with a nonlinear similarity fusion method to achieve high-level features. Afterward, it uses a neural network for interaction prediction. The similarity selection and similarity integration parts of NDD have been proposed in previous studies of other problems. Our novelty is to combine these parts with new neural network architecture and apply these approaches in the context of DDI prediction. We compared NDD with six machine learning classifiers and six state-of-the-art graph-based methods on three benchmark datasets. NDD achieved superior performance in cross-validation with AUPR ranging from 0.830 to 0.947, AUC from 0.954 to 0.994 and F-measure from 0.772 to 0.902. Moreover, cumulative evidence in case studies on numerous drug pairs, further confirm the ability of NDD to predict unknown DDIs. The evaluations corroborate that NDD is an efficient method for predicting unknown DDIs. The data and implementation of NDD are available at https://github.com/nrohani/NDD.

scholarly journals Deep convolutional neural networks for cardiovascular vulnerable plaque detection

2019 ◽  
Vol 277 ◽  
pp. 02024 ◽  
Author(s):  
Lincan Li ◽  
Tong Jia ◽  
Tianqi Meng ◽  
Yizhe Liu
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Convolutional Neural Networks ◽  
Vulnerable Plaque ◽  
Recall Rate ◽  
Superior Performance ◽  
Learning Approaches ◽  
Deep Convolutional Neural Networks ◽  
Vulnerable Plaques ◽  
Plaque Detection

In this paper, an accurate two-stage deep learning method is proposed to detect vulnerable plaques in ultrasonic images of cardiovascular. Firstly, a Fully Convonutional Neural Network (FCN) named U-Net is used to segment the original Intravascular Optical Coherence Tomography (IVOCT) cardiovascular images. We experiment on different threshold values to find the best threshold for removing noise and background in the original images. Secondly, a modified Faster RCNN is adopted to do precise detection. The modified Faster R-CNN utilize six-scale anchors (122,162,322,642,1282,2562) instead of the conventional one scale or three scale approaches. First, we present three problems in cardiovascular vulnerable plaque diagnosis, then we demonstrate how our method solve these problems. The proposed method in this paper apply deep convolutional neural networks to the whole diagnostic procedure. Test results show the Recall rate, Precision rate, IoU (Intersection-over-Union) rate and Total score are 0.94, 0.885, 0.913 and 0.913 respectively, higher than the 1st team of CCCV2017 Cardiovascular OCT Vulnerable Plaque Detection Challenge. AP of the designed Faster RCNN is 83.4%, higher than conventional approaches which use one-scale or three-scale anchors. These results demonstrate the superior performance of our proposed method and the power of deep learning approaches in diagnose cardiovascular vulnerable plaques.

Sign in / Sign up

Export Citation Format

Share Document