scholarly journals Residual Compensation Networks for Heterogeneous Face Recognition

2019 ◽  
Vol 33 ◽  
pp. 8239-8246 ◽  
Author(s):  
Zhongying Deng ◽  
Xiaojiang Peng ◽  
Yu Qiao
Keyword(s):  
Neural Networks ◽  
Face Recognition ◽  
Learning Strategies ◽  
Convolutional Neural Networks ◽  
Network Architecture ◽  
State Of The Art ◽  
The Other ◽  
Training Images ◽  
Heterogeneous Face Recognition ◽  
Cosine Distance

Heterogeneous Face Recognition (HFR) is a challenging task due to large modality discrepancy as well as insufficient training images in certain modalities. In this paper, we propose a new two-branch network architecture, termed as Residual Compensation Networks (RCN), to learn separated features for different modalities in HFR. The RCN incorporates a residual compensation (RC) module and a modality discrepancy loss (MD loss) into traditional convolutional neural networks. The RC module reduces modal discrepancy by adding compensation to one of the modalities so that its representation can be close to the other modality. The MD loss alleviates modal discrepancy by minimizing the cosine distance between different modalities. In addition, we explore different architectures and positions for the RC module, and evaluate different transfer learning strategies for HFR. Extensive experiments on IIIT-D Viewed Sketch, Forensic Sketch, CASIA NIR-VIS 2.0 and CUHK NIR-VIS show that our RCN outperforms other state-of-the-art methods significantly.

scholarly journals PlanetScope Radiometric Normalization and Sentinel-2 Super-Resolution (2.5 m): A Straightforward Spectral-Spatial Fusion of Multi-Satellite Multi-Sensor Images Using Residual Convolutional Neural Networks

2020 ◽  
Vol 12 (15) ◽  
pp. 2366
Author(s):  
Nicolas Latte ◽  
Philippe Lejeune
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Network Architecture ◽  
State Of The Art ◽  
Super Resolution ◽  
Neural Network Architecture ◽  
Research Areas ◽  
Novel Method ◽  
Spatial Fusion ◽  
Sentinel 2

Sentinel-2 (S2) imagery is used in many research areas and for diverse applications. Its spectral resolution and quality are high but its spatial resolutions, of at most 10 m, is not sufficient for fine scale analysis. A novel method was thus proposed to super-resolve S2 imagery to 2.5 m. For a given S2 tile, the 10 S2 bands (four at 10 m and six at 20 m) were fused with additional images acquired at higher spatial resolution by the PlanetScope (PS) constellation. The radiometric inconsistencies between PS microsatellites were normalized. Radiometric normalization and super-resolution were achieved simultaneously using state-of–the-art super-resolution residual convolutional neural networks adapted to the particularities of S2 and PS imageries (including masks of clouds and shadows). The method is described in detail, from image selection and downloading to neural network architecture, training, and prediction. The quality was thoroughly assessed visually (photointerpretation) and quantitatively, confirming that the proposed method is highly spatially and spectrally accurate. The method is also robust and can be applied to S2 images acquired worldwide at any date.

scholarly journals Effective Plug-Ins for Reducing Inference-Latency of Spiking Convolutional Neural Networks During Inference Phase

2021 ◽  
Vol 15 ◽  
Author(s):  
Xuan Chen ◽  
Xiaopeng Yuan ◽  
Gaoming Fu ◽  
Yuanyong Luo ◽  
Tao Yue ◽  
...  
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Network Architecture ◽  
State Of The Art ◽  
Low Cost ◽  
Training Phase ◽  
Working Mechanism ◽  
Simulation Results ◽  
New Perspective ◽  
Fully Connected

Convolutional Neural Networks (CNNs) are effective and mature in the field of classification, while Spiking Neural Networks (SNNs) are energy-saving for their sparsity of data flow and event-driven working mechanism. Previous work demonstrated that CNNs can be converted into equivalent Spiking Convolutional Neural Networks (SCNNs) without obvious accuracy loss, including different functional layers such as Convolutional (Conv), Fully Connected (FC), Avg-pooling, Max-pooling, and Batch-Normalization (BN) layers. To reduce inference-latency, existing researches mainly concentrated on the normalization of weights to increase the firing rate of neurons. There are also some approaches during training phase or altering the network architecture. However, little attention has been paid on the end of inference phase. From this new perspective, this paper presents 4 stopping criterions as low-cost plug-ins to reduce the inference-latency of SCNNs. The proposed methods are validated using MATLAB and PyTorch platforms with Spiking-AlexNet for CIFAR-10 dataset and Spiking-LeNet-5 for MNIST dataset. Simulation results reveal that, compared to the state-of-the-art methods, the proposed method can shorten the average inference-latency of Spiking-AlexNet from 892 to 267 time steps (almost 3.34 times faster) with the accuracy decline from 87.95 to 87.72%. With our methods, 4 types of Spiking-LeNet-5 only need 24–70 time steps per image with the accuracy decline not more than 0.1%, while models without our methods require 52–138 time steps, almost 1.92 to 3.21 times slower than us.

Early and Late Level Fusion of Deep Convolutional Neural Networks for Visual Concept Recognition

2016 ◽  
Vol 10 (03) ◽  
pp. 379-397 ◽  
Author(s):  
Hilal Ergun ◽  
Yusuf Caglar Akyuz ◽  
Mustafa Sert ◽  
Jianquan Liu
Keyword(s):  
Neural Networks ◽  
Best Practices ◽  
Convolutional Neural Networks ◽  
Network Architecture ◽  
Data Augmentation ◽  
State Of The Art ◽  
Great Promise ◽  
Visual Concept ◽  
Deep Convolutional Neural Networks ◽  
Concept Recognition

Visual concept recognition is an active research field in the last decade. Related to this attention, deep learning architectures are showing great promise in various computer vision domains including image classification, object detection, event detection and action recognition in videos. In this study, we investigate various aspects of convolutional neural networks for visual concept recognition. We analyze recent studies and different network architectures both in terms of running time and accuracy. In our proposed visual concept recognition system, we first discuss various important properties of popular convolutional network architecture under consideration. Then we describe our method for feature extraction at different levels of abstraction. We present extensive empirical information along with best practices for big data practitioners. Using these best practices we propose efficient fusion mechanisms both for single and multiple network models. We present state-of-the-art results on benchmark datasets while keeping computational costs at low level. Our results show that these state-of-the-art results can be reached without using extensive data augmentation techniques.

scholarly journals Face Recognition Based on Lightweight Convolutional Neural Networks

Information ◽  
2021 ◽  
Vol 12 (5) ◽  
pp. 191
Author(s):  
Wenting Liu ◽  
Li Zhou ◽  
Jie Chen
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Face Recognition ◽  
Convolutional Neural Networks ◽  
Network Architecture ◽  
Recognition Task ◽  
Training Method ◽  
The Face ◽  
And Storage ◽  
And Training

Face recognition algorithms based on deep learning methods have become increasingly popular. Most of these are based on highly precise but complex convolutional neural networks (CNNs), which require significant computing resources and storage, and are difficult to deploy on mobile devices or embedded terminals. In this paper, we propose several methods to improve the algorithms for face recognition based on a lightweight CNN, which is further optimized in terms of the network architecture and training pattern on the basis of MobileFaceNet. Regarding the network architecture, we introduce the Squeeze-and-Excitation (SE) block and propose three improved structures via a channel attention mechanism—the depthwise SE module, the depthwise separable SE module, and the linear SE module—which are able to learn the correlation of information between channels and assign them different weights. In addition, a novel training method for the face recognition task combined with an additive angular margin loss function is proposed that performs the compression and knowledge transfer of the deep network for face recognition. Finally, we obtained high-precision and lightweight face recognition models with fewer parameters and calculations that are more suitable for applications. Through extensive experiments and analysis, we demonstrate the effectiveness of the proposed methods.

Finding Facial Emotions From the Clutter Scenes Using Zernike Moments-Based Convolutional Neural Networks

2021 ◽  
pp. 241-265
Author(s):  
Wencan Zhong ◽  
Vijayalakshmi G. V. Mahesh ◽  
Alex Noel Joseph Raj ◽  
Nersisson Ruban
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Face Recognition ◽  
Facial Expression ◽  
Convolutional Neural Networks ◽  
Zernike Moments ◽  
The Other ◽  
Recognition Systems ◽  
Facial Images ◽  
Surveillance Applications

Finding faces in the clutter scenes is a challenging task in automatic face recognition systems as facial images are subjected to changes in the illumination, facial expression, orientation, and occlusions. Also, in the cluttered scenes, faces are not completely visible and detecting them is essential as it is significant in surveillance applications to study the mood of the crowd. This chapter utilizes the deep learning methods to understand the cluttered scenes to find the faces and discriminate them into partial and full faces. The work proves that MTCNN used for detecting the faces and Zernike moments-based kernels employed in CNN for classifying the faces into partial and full takes advantage in delivering a notable performance as compared to the other techniques. Considering the limitation of recognition on partial face emotions, only the full faces are preserved, and further, the KDEF dataset is modified by MTCNN to detect only faces and classify them into four emotions. PatternNet is utilized to train and test the modified dataset to improve the accuracy of the results.

IMPROVING FACE RECOGNITION MODELS USING CONVOLUTIONAL NEURAL NETWORKS, METRIC LEARNING AND OPTIMIZATION METHOD

2021 ◽  
Vol 5 ◽  
pp. 140-158
Author(s):  
Andrey Litvynchuk ◽  
◽  
Lesya Baranovska ◽  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Computer Vision ◽  
Face Recognition ◽  
Comparative Analysis ◽  
Convolutional Neural Networks ◽  
Network Architecture ◽  
Metric Learning ◽  
Stochastic Gradient Descent ◽  
Google Search

Face recognition is one of the main tasks of computer vision. It has many applications, which has led to a huge amount of research in this area. And although research in the field has been going on since the beginning of the computer vision, good results could be achieved only with the help of convolutional neural networks. In this work, a comparative analysis of facial recognition methods before convolutional neural networks was performed. A set of neural network architectures, methods of metric learning and optimization are considered. There were performed bunch of experiments and comparative analysis of the considered methods of improvement of convolutional neural networks. As a result a universal algorithm for training the face recognition model was obtained. To compare different approaches of face recognition, we chose a dataset called VGGFace2. It consists of 3,31 million images of 9131 people. It was created using images from the Google search engine. Initially, pre-trained neural networks were used to select photographs with humans. The images were then checked mannualy. For the validation sample, we set aside 50 images of 500 people, for a total of 25,000 images. Almost all experiments were performed iteratively. For example, we choose the best optimizer and then we use it to search for best arctitecture. As expected, neural networks with more parameters and more sophisticated architecture showed better results in this task. Among the considered models the best was Se-ResNet50. Metric learning is a method by which it is possible to achieve good accuracy in face recognition. Without this method it would be impossible to solve the problem. To optimize neural networks, we considered both adaptive and simple optimizers. It turned out that the stochastic gradient descent with moment is the best for this problem, and adaptive methods showed a rather poor result. In general, using different approaches, we were able to obtain an accuracy of 92 %, which is 25,5 % better than the baseline experiment. We see next ways for the further development of the research subject: improving neural network architecture, collecting more data and applying better regularization techniques.

scholarly journals SUPER-RESOLUTION OF MULTISPECTRAL SATELLITE IMAGES USING CONVOLUTIONAL NEURAL NETWORKS

2020 ◽  
Vol V-1-2020 ◽  
pp. 33-40
Author(s):  
M. U. Müller ◽  
N. Ekhtiari ◽  
R. M. Almeida ◽  
C. Rieke
Keyword(s):  
Remote Sensing ◽  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Spatial Resolution ◽  
Satellite Images ◽  
State Of The Art ◽  
Super Resolution ◽  
Image Resolution ◽  
The Other ◽  
Multispectral Satellite Images

Abstract. Super-resolution aims at increasing image resolution by algorithmic means and has progressed over the recent years due to advances in the fields of computer vision and deep learning. Convolutional Neural Networks based on a variety of architectures have been applied to the problem, e.g. autoencoders and residual networks. While most research focuses on the processing of photographs consisting only of RGB color channels, little work can be found concentrating on multi-band, analytic satellite imagery. Satellite images often include a panchromatic band, which has higher spatial resolution but lower spectral resolution than the other bands. In the field of remote sensing, there is a long tradition of applying pan-sharpening to satellite images, i.e. bringing the multispectral bands to the higher spatial resolution by merging them with the panchromatic band. To our knowledge there are so far no approaches to super-resolution which take advantage of the panchromatic band. In this paper we propose a method to train state-of-the-art CNNs using pairs of lower-resolution multispectral and high-resolution pan-sharpened image tiles in order to create super-resolved analytic images. The derived quality metrics show that the method improves information content of the processed images. We compare the results created by four CNN architectures, with RedNet30 performing best.

scholarly journals Using spatial-temporal ensembles of convolutional neural networks for lumen segmentation in ureteroscopy

2021 ◽  
Author(s):  
Jorge F. Lazo ◽  
Aldo Marzullo ◽  
Sara Moccia ◽  
Michele Catellani ◽  
Benoit Rosa ◽  
...  
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
State Of The Art ◽  
Automatic Segmentation ◽  
Temporal Information ◽  
Invasive Technique ◽  
Dice Similarity Coefficient ◽  
Specular Reflections ◽  
Lumen Segmentation ◽  
Previous State

Abstract Purpose Ureteroscopy is an efficient endoscopic minimally invasive technique for the diagnosis and treatment of upper tract urothelial carcinoma. During ureteroscopy, the automatic segmentation of the hollow lumen is of primary importance, since it indicates the path that the endoscope should follow. In order to obtain an accurate segmentation of the hollow lumen, this paper presents an automatic method based on convolutional neural networks (CNNs). Methods The proposed method is based on an ensemble of 4 parallel CNNs to simultaneously process single and multi-frame information. Of these, two architectures are taken as core-models, namely U-Net based in residual blocks ($$m_1$$ m 1 ) and Mask-RCNN ($$m_2$$ m 2 ), which are fed with single still-frames I(t). The other two models ($$M_1$$ M 1 , $$M_2$$ M 2 ) are modifications of the former ones consisting on the addition of a stage which makes use of 3D convolutions to process temporal information. $$M_1$$ M 1 , $$M_2$$ M 2 are fed with triplets of frames ($$I(t-1)$$ I ( t - 1 ) , I(t), $$I(t+1)$$ I ( t + 1 ) ) to produce the segmentation for I(t). Results The proposed method was evaluated using a custom dataset of 11 videos (2673 frames) which were collected and manually annotated from 6 patients. We obtain a Dice similarity coefficient of 0.80, outperforming previous state-of-the-art methods. Conclusion The obtained results show that spatial-temporal information can be effectively exploited by the ensemble model to improve hollow lumen segmentation in ureteroscopic images. The method is effective also in the presence of poor visibility, occasional bleeding, or specular reflections.

scholarly journals SketchGNN: Semantic Sketch Segmentation with Graph Neural Networks

2021 ◽  
Vol 40 (3) ◽  
pp. 1-13
Author(s):  
Lumin Yang ◽  
Jiajie Zhuang ◽  
Hongbo Fu ◽  
Xiangzhi Wei ◽  
Kun Zhou ◽  
...  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Network Architecture ◽  
Large Scale ◽  
State Of The Art ◽  
Semantic Segmentation ◽  
Structure Information ◽  
Graph Neural Networks ◽  
Node Labels ◽  
Point Level

We introduce SketchGNN , a convolutional graph neural network for semantic segmentation and labeling of freehand vector sketches. We treat an input stroke-based sketch as a graph with nodes representing the sampled points along input strokes and edges encoding the stroke structure information. To predict the per-node labels, our SketchGNN uses graph convolution and a static-dynamic branching network architecture to extract the features at three levels, i.e., point-level, stroke-level, and sketch-level. SketchGNN significantly improves the accuracy of the state-of-the-art methods for semantic sketch segmentation (by 11.2% in the pixel-based metric and 18.2% in the component-based metric over a large-scale challenging SPG dataset) and has magnitudes fewer parameters than both image-based and sequence-based methods.

Sign in / Sign up

Export Citation Format

Share Document