scholarly journals Learning conditional photometric stereo with high-resolution features

2021 ◽  
Vol 8 (1) ◽  
pp. 105-118
Author(s):  
Yakun Ju ◽  
Yuxin Peng ◽  
Muwei Jian ◽  
Feng Gao ◽  
Junyu Dong
Keyword(s):  
Neural Networks ◽  
High Resolution ◽  
Feature Fusion ◽  
Low Frequency ◽  
Photometric Stereo ◽  
Surface Orientation ◽  
Smooth Functions ◽  
Multi Scale ◽  
Benchmark Datasets ◽  
Deep Feature Extraction

AbstractPhotometric stereo aims to reconstruct 3D geometry by recovering the dense surface orientation of a 3D object from multiple images under differing illumination. Traditional methods normally adopt simplified reflectance models to make the surface orientation computable. However, the real reflectances of surfaces greatly limit applicability of such methods to real-world objects. While deep neural networks have been employed to handle non-Lambertian surfaces, these methods are subject to blurring and errors, especially in high-frequency regions (such as crinkles and edges), caused by spectral bias: neural networks favor low-frequency representations so exhibit a bias towards smooth functions. In this paper, therefore, we propose a self-learning conditional network with multi-scale features for photometric stereo, avoiding blurred reconstruction in such regions. Our explorations include: (i) a multi-scale feature fusion architecture, which keeps high-resolution representations and deep feature extraction, simultaneously, and (ii) an improved gradient-motivated conditionally parameterized convolution (GM-CondConv) in our photometric stereo network, with different combinations of convolution kernels for varying surfaces. Extensive experiments on public benchmark datasets show that our calibrated photometric stereo method outperforms the state-of-the-art.

scholarly journals Non-Intrusive Load Disaggregation Based on a Multi-Scale Attention Residual Network

2020 ◽  
Vol 10 (24) ◽  
pp. 9132
Author(s):  
Liguo Weng ◽  
Xiaodong Zhang ◽  
Junhao Qian ◽  
Min Xia ◽  
Yiqing Xu ◽  
...  
Keyword(s):  
Smart Grids ◽  
Recognition Rate ◽  
Low Frequency ◽  
Attention Mechanism ◽  
Learning Ability ◽  
Residual Network ◽  
Multi Scale ◽  
Energy Disaggregation ◽  
Benchmark Datasets ◽  
Load Disaggregation

Non-intrusive load disaggregation (NILD) is of great significance to the development of smart grids. Current energy disaggregation methods extract features from sequences, and this process easily leads to a loss of load features and difficulties in detecting, resulting in a low recognition rate of low-use electrical appliances. To solve this problem, a non-intrusive sequential energy disaggregation method based on a multi-scale attention residual network is proposed. Multi-scale convolutions are used to learn features, and the attention mechanism is used to enhance the learning ability of load features. The residual learning further improves the performance of the algorithm, avoids network degradation, and improves the precision of load decomposition. The experimental results on two benchmark datasets show that the proposed algorithm has more advantages than the existing algorithms in terms of load disaggregation accuracy and judgments of the on/off state, and the attention mechanism can further improve the disaggregation accuracy of low-frequency electrical appliances.

scholarly journals A Fast and Lightweight Method with Feature Fusion and Multi-Context for Face Detection

2018 ◽  
Vol 10 (8) ◽  
pp. 80
Author(s):  
Lei Zhang ◽  
Xiaoli Zhi
Keyword(s):  
Face Detection ◽  
Graphics Processing Units ◽  
High Performance ◽  
Feature Fusion ◽  
Local Context ◽  
Data Set ◽  
Global Context ◽  
Detection Algorithms ◽  
Multi Scale ◽  
Benchmark Datasets

Convolutional neural networks (CNN for short) have made great progress in face detection. They mostly take computation intensive networks as the backbone in order to obtain high precision, and they cannot get a good detection speed without the support of high-performance GPUs (Graphics Processing Units). This limits CNN-based face detection algorithms in real applications, especially in some speed dependent ones. To alleviate this problem, we propose a lightweight face detector in this paper, which takes a fast residual network as backbone. Our method can run fast even on cheap and ordinary GPUs. To guarantee its detection precision, multi-scale features and multi-context are fully exploited in efficient ways. Specifically, feature fusion is used to obtain semantic strongly multi-scale features firstly. Then multi-context including both local and global context is added to these multi-scale features without extra computational burden. The local context is added through a depthwise separable convolution based approach, and the global context by a simple global average pooling way. Experimental results show that our method can run at about 110 fps on VGA (Video Graphics Array)-resolution images, while still maintaining competitive precision on WIDER FACE and FDDB (Face Detection Data Set and Benchmark) datasets as compared with its state-of-the-art counterparts.

scholarly journals High-Resolution SAR Image Classification Using Multi-Scale Deep Feature Fusion and Covariance Pooling Manifold Network

2021 ◽  
Vol 13 (2) ◽  
pp. 328
Author(s):  
Wenkai Liang ◽  
Yan Wu ◽  
Ming Li ◽  
Yice Cao ◽  
Xin Hu
Keyword(s):  
High Resolution ◽  
Image Classification ◽  
Feature Fusion ◽  
Representation Learning ◽  
Sar Image ◽  
Gabor Filtering ◽  
Feature Maps ◽  
Sar Images ◽  
Multi Scale ◽  
Deep Feature

The classification of high-resolution (HR) synthetic aperture radar (SAR) images is of great importance for SAR scene interpretation and application. However, the presence of intricate spatial structural patterns and complex statistical nature makes SAR image classification a challenging task, especially in the case of limited labeled SAR data. This paper proposes a novel HR SAR image classification method, using a multi-scale deep feature fusion network and covariance pooling manifold network (MFFN-CPMN). MFFN-CPMN combines the advantages of local spatial features and global statistical properties and considers the multi-feature information fusion of SAR images in representation learning. First, we propose a Gabor-filtering-based multi-scale feature fusion network (MFFN) to capture the spatial pattern and get the discriminative features of SAR images. The MFFN belongs to a deep convolutional neural network (CNN). To make full use of a large amount of unlabeled data, the weights of each layer of MFFN are optimized by unsupervised denoising dual-sparse encoder. Moreover, the feature fusion strategy in MFFN can effectively exploit the complementary information between different levels and different scales. Second, we utilize a covariance pooling manifold network to extract further the global second-order statistics of SAR images over the fusional feature maps. Finally, the obtained covariance descriptor is more distinct for various land covers. Experimental results on four HR SAR images demonstrate the effectiveness of the proposed method and achieve promising results over other related algorithms.

scholarly journals DCPNet: A Densely Connected Pyramid Network for Monocular Depth Estimation

Sensors ◽  
2021 ◽  
Vol 21 (20) ◽  
pp. 6780
Author(s):  
Zhitong Lai ◽  
Rui Tian ◽  
Zhiguo Wu ◽  
Nannan Ding ◽  
Linjian Sun ◽  
...  
Keyword(s):  
Multiple Scales ◽  
Feature Fusion ◽  
State Of The Art ◽  
Depth Estimation ◽  
Multi Scale ◽  
Pyramid Structure ◽  
Benchmark Datasets ◽  
The Common ◽  
Monocular Depth ◽  
Multiple Stages

Pyramid architecture is a useful strategy to fuse multi-scale features in deep monocular depth estimation approaches. However, most pyramid networks fuse features only within the adjacent stages in a pyramid structure. To take full advantage of the pyramid structure, inspired by the success of DenseNet, this paper presents DCPNet, a densely connected pyramid network that fuses multi-scale features from multiple stages of the pyramid structure. DCPNet not only performs feature fusion between the adjacent stages, but also non-adjacent stages. To fuse these features, we design a simple and effective dense connection module (DCM). In addition, we offer a new consideration of the common upscale operation in our approach. We believe DCPNet offers a more efficient way to fuse features from multiple scales in a pyramid-like network. We perform extensive experiments using both outdoor and indoor benchmark datasets (i.e., the KITTI and the NYU Depth V2 datasets) and DCPNet achieves the state-of-the-art results.

scholarly journals HR-MPF: high-resolution representation network with multi-scale progressive fusion for pulmonary nodule segmentation and classification

2021 ◽  
Vol 2021 (1) ◽  
Author(s):  
Ling Zhu ◽  
Hongqing Zhu ◽  
Suyi Yang ◽  
Pengyu Wang ◽  
Yang Yu
Keyword(s):  
High Resolution ◽  
Loss Function ◽  
Pulmonary Nodule ◽  
Feature Fusion ◽  
Back Propagation ◽  
Pulmonary Nodules ◽  
Improve Patient Survival ◽  
Multi Scale ◽  
Training Scheme ◽  
Adversarial Training

AbstractAccurate segmentation and classification of pulmonary nodules are of great significance to early detection and diagnosis of lung diseases, which can reduce the risk of developing lung cancer and improve patient survival rate. In this paper, we propose an effective network for pulmonary nodule segmentation and classification at one time based on adversarial training scheme. The segmentation network consists of a High-Resolution network with Multi-scale Progressive Fusion (HR-MPF) and a proposed Progressive Decoding Module (PDM) recovering final pixel-wise prediction results. Specifically, the proposed HR-MPF firstly incorporates boosted module to High-Resolution Network (HRNet) in a progressive feature fusion manner. In this case, feature communication is augmented among all levels in this high-resolution network. Then, downstream classification module would identify benign and malignant pulmonary nodules based on feature map from PDM. In the adversarial training scheme, a discriminator is set to optimize HR-MPF and PDM through back propagation. Meanwhile, a reasonably designed multi-task loss function optimizes performance of segmentation and classification overall. To improve the accuracy of boundary prediction crucial to nodule segmentation, a boundary consistency constraint is designed and incorporated in the segmentation loss function. Experiments on publicly available LUNA16 dataset show that the framework outperforms relevant advanced methods in quantitative evaluation and visual perception.

Deep neural networks to downscale ocean climate models

2020 ◽  
Author(s):  
Marie Déchelle-Marquet ◽  
Marina Levy ◽  
Patrick Gallinari ◽  
Michel Crepon ◽  
Sylvie Thiria
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
High Resolution ◽  
Deep Neural Networks ◽  
Climate Models ◽  
Ocean Currents ◽  
Sea Surface ◽  
Low Resolution ◽  
Ocean Climate ◽  
Multi Scale

<p>Ocean currents are a major source of impact on climate variability, through the heat transport they induce for instance. Ocean climate models have quite low resolution of about 50 km. Several dynamical processes such as instabilities and filaments which have a scale of 1km have a strong influence on the ocean state. We propose to observe and model these fine scale effects by a combination of satellite high resolution SST observations (1km resolution, daily observations) and mesoscale resolution altimetry observations (10km resolution, weekly observations) with deep neural networks. Whereas the downscaling of climate models has been commonly addressed with assimilation approaches, in the last few years neural networks emerged as powerful multi-scale analysis method. Besides, the large amount of available oceanic data makes attractive the use of deep learning to bridge the gap between scales variability.</p><p>This study aims at reconstructing the multi-scale variability of oceanic fields, based on the high resolution NATL60 model of ocean observations at different spatial resolutions: low-resolution sea surface height (SSH) and high resolution SST. As the link between residual neural networks and dynamical systems has recently been established, such a network is trained in a supervised way to reconstruct the high variability of SSH and ocean currents at submesoscale (a few kilometers). To ensure the conservation of physical aspects in the model outputs, physical knowledge is incorporated into the deep learning models training. Different validation methods are investigated and the model outputs are tested with regards to their physical plausibility. The method performance is discussed and compared to other baselines (namely convolutional neural network). The generalization of the proposed method on different ocean variables such as sea surface chlorophyll or sea surface salinity is also examined.</p>

scholarly journals Unsupervised phenotypic analysis of cellular images with multi-scale convolutional neural networks

2018 ◽  
Author(s):  
William J. Godinez ◽  
Imtiaz Hossain ◽  
Xian Zhang
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Large Scale ◽  
Cellular Imaging ◽  
Phenotypic Analysis ◽  
Learning Methods ◽  
Multi Scale ◽  
Full Resolution ◽  
Benchmark Datasets ◽  
Microscopy Images

AbstractLarge-scale cellular imaging and phenotyping is a widely adopted strategy for understanding biological systems and chemical perturbations. Quantitative analysis of cellular images for identifying phenotypic changes is a key challenge within this strategy, and has recently seen promising progress with approaches based on deep neural networks. However, studies so far require either pre-segmented images as input or manual phenotype annotations for training, or both. To address these limitations, we have developed an unsupervised approach that exploits the inherent groupings within cellular imaging datasets to define surrogate classes that are used to train a multi-scale convolutional neural network. The trained network takes as input full-resolution microscopy images, and, without the need for segmentation, yields as output feature vectors that support phenotypic profiling. Benchmarked on two diverse benchmark datasets, the proposed approach yields accurate phenotypic predictions as well as compound potency estimates comparable to the state-of-the-art. More importantly, we show that the approach identifies novel cellular phenotypes not included in the manual annotation nor detected by previous studies.Author summaryCellular microscopy images provide detailed information about how cells respond to genetic or chemical treatments, and have been widely and successfully used in basic research and drug discovery. The recent breakthrough of deep learning methods for natural imaging recognition tasks has triggered the development and application of deep learning methods to cellular images to understand how cells change upon perturbation. Although successful, deep learning studies so far either can only take images of individual cells as input or require human experts to label a large amount of images. In this paper, we present an unsupervised deep learning approach that, without any human annotation, analyzes directly full-resolution microscopy images displaying typically hundreds of cells. We apply the approach to two benchmark datasets, and show that the approach identifies novel visual phenotypes not detected by previous studies.

scholarly journals Battlefield Target Aggregation Behavior Recognition Model Based on Multi-Scale Feature Fusion

Symmetry ◽  
2019 ◽  
Vol 11 (6) ◽  
pp. 761 ◽  
Author(s):  
Haiyang Jiang ◽  
Yaozong Pan ◽  
Jian Zhang ◽  
Haitao Yang
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Feature Fusion ◽  
Computational Cost ◽  
Aggregation Behavior ◽  
Behavior Recognition ◽  
Scale Feature ◽  
Multi Scale ◽  
Model Based ◽  
Spatio Temporal

In this paper, our goal is to improve the recognition accuracy of battlefield target aggregation behavior while maintaining the low computational cost of spatio-temporal depth neural networks. To this end, we propose a novel 3D-CNN (3D Convolutional Neural Networks) model, which extends the idea of multi-scale feature fusion to the spatio-temporal domain, and enhances the feature extraction ability of the network by combining feature maps of different convolutional layers. In order to reduce the computational complexity of the network, we further improved the multi-fiber network, and finally established an architecture—3D convolution Two-Stream model based on multi-scale feature fusion. Extensive experimental results on the simulation data show that our network significantly boosts the efficiency of existing convolutional neural networks in the aggregation behavior recognition, achieving the most advanced performance on the dataset constructed in this paper.

scholarly journals Feature Extraction and Analysis of Microseismic Signal Based on Convolutional Neural Network

2021 ◽  
Vol 2143 (1) ◽  
pp. 012017
Author(s):  
Hui Zhang ◽  
Hao Zhai ◽  
Ke Zhang ◽  
Lujun Wang ◽  
Xing Zhao ◽  
...  
Keyword(s):  
Neural Networks ◽  
Feature Extraction ◽  
Big Data ◽  
Convolutional Neural Networks ◽  
Feature Fusion ◽  
Practical Significance ◽  
Scale Feature ◽  
Multi Scale ◽  
Seismic Detection ◽  
Microseismic Signal

Abstract Seismic detection technology has been widely used in safety detection of engineering construction abroad. Although it has just started in the field of engineering in our country, its role is becoming more and more important. Through computer technology, micro-seismic detection can provide accurate data for the construction safety detection of large-scale projects, which has important practical significance for the rapid and effective identification of micro-seismic signals. Based on this, the purpose of this article is to study the feature extraction and classification of microseismic signals based on neural games. This article first summarizes the development status of microseismic monitoring technology. Using traditional convolutional neural networks for analysis, a multi-scale feature fusion network is proposed on the basis of convolutional neural networks and big data, the multi-scale feature fusion network is used to research and analyze microseismic feature extraction and classification. This article systematically explains The principle of microseismic signal acquisition and the construction of multi-scale feature fusion network. And use big data, comparative analysis method, observation method and other research methods to study the theme of this article. Experimental research shows that the db7 wavelet base has little effect on the Megatron signal.

Sign in / Sign up

Export Citation Format

Share Document