Electromagnetic Signal Feature Fusion and Recognition based on Multi-Modal Deep Learning

2020 ◽  
Vol 16 (6) ◽  
pp. 941
Author(s):  
Changbo Hou ◽  
Xiao Zhang ◽  
Xiang Chen
Keyword(s):  
Deep Learning ◽  
Feature Fusion ◽  
Electromagnetic Signal

scholarly journals Explicit-implicit dual stream network for image quality assessment

2020 ◽  
Vol 2020 (1) ◽  
Author(s):  
Guangyi Yang ◽  
Xingyu Ding ◽  
Tian Huang ◽  
Kun Cheng ◽  
Weizheng Jin
Keyword(s):  
Deep Learning ◽  
Image Quality ◽  
Quality Assessment ◽  
Frequency Domain ◽  
Image Quality Assessment ◽  
Feature Fusion ◽  
Learning Model ◽  
Stream Network ◽  
Perception System ◽  
Deep Learning Model

Abstract Communications industry has remarkably changed with the development of fifth-generation cellular networks. Image, as an indispensable component of communication, has attracted wide attention. Thus, finding a suitable approach to assess image quality is important. Therefore, we propose a deep learning model for image quality assessment (IQA) based on explicit-implicit dual stream network. We use frequency domain features of kurtosis based on wavelet transform to represent explicit features and spatial features extracted by convolutional neural network (CNN) to represent implicit features. Thus, we constructed an explicit-implicit (EI) parallel deep learning model, namely, EI-IQA model. The EI-IQA model is based on the VGGNet that extracts the spatial domain features. On this basis, the number of network layers of VGGNet is reduced by adding the parallel wavelet kurtosis value frequency domain features. Thus, the training parameters and the sample requirements decline. We verified, by cross-validation of different databases, that the wavelet kurtosis feature fusion method based on deep learning has a more complete feature extraction effect and a better generalisation ability. Thus, the method can simulate the human visual perception system better, and subjective feelings become closer to the human eye. The source code about the proposed EI-IQA model is available on github https://github.com/jacob6/EI-IQA.

Sewer pipe defect detection via deep learning with local and global feature fusion

2021 ◽  
Vol 129 ◽  
pp. 103823
Author(s):  
Dawei Li ◽  
Qian Xie ◽  
Zhenghao Yu ◽  
Qiaoyun Wu ◽  
Jun Zhou ◽  
...  
Keyword(s):  
Deep Learning ◽  
Defect Detection ◽  
Feature Fusion ◽  
Global Feature ◽  
Sewer Pipe

MUFFIN: multi-scale feature fusion for drug–drug interaction prediction

2021 ◽  
Author(s):  
Yujie Chen ◽  
Tengfei Ma ◽  
Xixi Yang ◽  
Jianmin Wang ◽  
Bosheng Song ◽  
...  
Keyword(s):  
Molecular Structure ◽  
Deep Learning ◽  
Medical Information ◽  
Feature Fusion ◽  
Molecular Graph ◽  
Knowledge Graph ◽  
Sequence Information ◽  
Learning Models ◽  
Scale Feature ◽  
Multi Scale

Abstract Motivation Adverse drug–drug interactions (DDIs) are crucial for drug research and mainly cause morbidity and mortality. Thus, the identification of potential DDIs is essential for doctors, patients and the society. Existing traditional machine learning models rely heavily on handcraft features and lack generalization. Recently, the deep learning approaches that can automatically learn drug features from the molecular graph or drug-related network have improved the ability of computational models to predict unknown DDIs. However, previous works utilized large labeled data and merely considered the structure or sequence information of drugs without considering the relations or topological information between drug and other biomedical objects (e.g. gene, disease and pathway), or considered knowledge graph (KG) without considering the information from the drug molecular structure. Results Accordingly, to effectively explore the joint effect of drug molecular structure and semantic information of drugs in knowledge graph for DDI prediction, we propose a multi-scale feature fusion deep learning model named MUFFIN. MUFFIN can jointly learn the drug representation based on both the drug-self structure information and the KG with rich bio-medical information. In MUFFIN, we designed a bi-level cross strategy that includes cross- and scalar-level components to fuse multi-modal features well. MUFFIN can alleviate the restriction of limited labeled data on deep learning models by crossing the features learned from large-scale KG and drug molecular graph. We evaluated our approach on three datasets and three different tasks including binary-class, multi-class and multi-label DDI prediction tasks. The results showed that MUFFIN outperformed other state-of-the-art baselines. Availability and implementation The source code and data are available at https://github.com/xzenglab/MUFFIN.

scholarly journals A Multimodal Feature Fusion-Based Deep Learning Method for Online Fault Diagnosis of Rotating Machinery

Sensors ◽  
2018 ◽  
Vol 18 (10) ◽  
pp. 3521 ◽  
Author(s):  
Funa Zhou ◽  
Po Hu ◽  
Shuai Yang ◽  
Chenglin Wen
Keyword(s):  
Deep Learning ◽  
Fault Diagnosis ◽  
Real Time ◽  
Time Domain ◽  
Feature Fusion ◽  
Rotating Machinery ◽  
Time Domain Data ◽  
Diagnosis Method ◽  
The Time Domain ◽  
Potential Frequency

Rotating machinery usually suffers from a type of fault, where the fault feature extracted in the frequency domain is significant, while the fault feature extracted in the time domain is insignificant. For this type of fault, a deep learning-based fault diagnosis method developed in the frequency domain can reach high accuracy performance without real-time performance, whereas a deep learning-based fault diagnosis method developed in the time domain obtains real-time diagnosis with lower diagnosis accuracy. In this paper, a multimodal feature fusion-based deep learning method for accurate and real-time online diagnosis of rotating machinery is proposed. The proposed method can directly extract the potential frequency of abnormal features involved in the time domain data. Firstly, multimodal features corresponding to the original data, the slope data, and the curvature data are firstly extracted by three separate deep neural networks. Then, a multimodal feature fusion is developed to obtain a new fused feature that can characterize the potential frequency feature involved in the time domain data. Lastly, the fused new feature is used as the input of the Softmax classifier to achieve a real-time online diagnosis result from the frequency-type fault data. A simulation experiment and a case study of the bearing fault diagnosis confirm the high efficiency of the method proposed in this paper.

scholarly journals MR-InpaintNet: Toward Deep Multi-Resolution Learning for Progressive Image Inpainting

2021 ◽  
Author(s):  
Huan Zhang ◽  
Zhao Zhang ◽  
Haijun Zhang ◽  
Yi Yang ◽  
Shuicheng Yan ◽  
...  
Keyword(s):  
Deep Learning ◽  
High Resolution ◽  
Semantic Information ◽  
Feature Fusion ◽  
Image Inpainting ◽  
Feature Learning ◽  
Low Resolution ◽  
Resolution Image ◽  
Texture Information ◽  
Multiple Resolutions

<div>Deep learning based image inpainting methods have improved the performance greatly due to powerful representation ability of deep learning. However, current deep inpainting methods still tend to produce unreasonable structure and blurry texture, implying that image inpainting is still a challenging topic due to the ill-posed property of the task. To address these issues, we propose a novel deep multi-resolution learning-based progressive image inpainting method, termed MR-InpaintNet, which takes the damaged images of different resolutions as input and then fuses the multi-resolution features for repairing the damaged images. The idea is motivated by the fact that images of different resolutions can provide different levels of feature information. Specifically, the low-resolution image provides strong semantic information and the high-resolution image offers detailed texture information. The middle-resolution image can be used to reduce the gap between low-resolution and high-resolution images, which can further refine the inpainting result. To fuse and improve the multi-resolution features, a novel multi-resolution feature learning (MRFL) process is designed, which is consisted of a multi-resolution feature fusion (MRFF) module, an adaptive feature enhancement (AFE) module and a memory enhanced mechanism (MEM) module for information preservation. Then, the refined multi-resolution features contain both rich semantic information and detailed texture information from multiple resolutions. We further handle the refined multiresolution features by the decoder to obtain the recovered image. Extensive experiments on the Paris Street View, Places2 and CelebA-HQ datasets demonstrate that our proposed MRInpaintNet can effectively recover the textures and structures, and performs favorably against state-of-the-art methods.</div>

Sign in / Sign up

Export Citation Format

Share Document