A New Deep Learning Algorithm for SAR Scene Classification Based on Spatial Statistical Modeling and Features Re-Calibration

Synthetic Aperture Radar (SAR) scene classification is challenging but widely applied, in which deep learning can play a pivotal role because of its hierarchical feature learning ability. In the paper, we propose a new scene classification framework, named Feature Recalibration Network with Multi-scale Spatial Features (FRN-MSF), to achieve high accuracy in SAR-based scene classification. First, a Multi-Scale Omnidirectional Gaussian Derivative Filter (MSOGDF) is constructed. Then, Multi-scale Spatial Features (MSF) of SAR scenes are generated by weighting MSOGDF, a Gray Level Gradient Co-occurrence Matrix (GLGCM) and Gabor transformation. These features were processed by the Feature Recalibration Network (FRN) to learn high-level features. In the network, the Depthwise Separable Convolution (DSC), Squeeze-and-Excitation (SE) Block and Convolution Neural Network (CNN) are integrated. Finally, these learned features will be classified by the Softmax function. Eleven types of SAR scenes obtained from four systems combining different bands and resolutions were trained and tested, and a mean accuracy of 98.18% was obtained. To validate the generality of FRN-MSF, five types of SAR scenes sampled from two additional large-scale Gaofen-3 and TerraSAR-X images were evaluated for classification. The mean accuracy of the five types reached 94.56%; while the mean accuracy for the same five types of the former tested 11 types of scene was 96%. The high accuracy indicates that the FRN-MSF is promising for SAR scene classification without losing generality.

Download Full-text

Research on Efficient Deep Learning Algorithm Based on ShuffleGhost in the Field of Virtual Reality

Wireless Communications and Mobile Computing ◽

10.1155/2021/1382781 ◽

2021 ◽

Vol 2021 ◽

pp. 1-11

Author(s):

Bangtong Huang ◽

Hongquan Zhang ◽

Zihong Chen ◽

Lingling Li ◽

Lihua Shi

Keyword(s):

Virtual Reality ◽

Deep Learning ◽

Large Scale ◽

Learning Algorithm ◽

Feature Maps ◽

Embedded Devices ◽

Feature Map ◽

Deep Learning Algorithm ◽

Proper Design ◽

The Cost

Deep learning algorithms are facing the limitation in virtual reality application due to the cost of memory, computation, and real-time computation problem. Models with rigorous performance might suffer from enormous parameters and large-scale structure, and it would be hard to replant them onto embedded devices. In this paper, with the inspiration of GhostNet, we proposed an efficient structure ShuffleGhost to make use of the redundancy in feature maps to alleviate the cost of computations, as well as tackling some drawbacks of GhostNet. Since GhostNet suffers from high computation of convolution in Ghost module and shortcut, the restriction of downsampling would make it more difficult to apply Ghost module and Ghost bottleneck to other backbone. This paper proposes three new kinds of ShuffleGhost structure to tackle the drawbacks of GhostNet. The ShuffleGhost module and ShuffleGhost bottlenecks are utilized by the shuffle layer and group convolution from ShuffleNet, and they are designed to redistribute the feature maps concatenated from Ghost Feature Map and Primary Feature Map. Besides, they eliminate the gap of them and extract the features. Then, SENet layer is adopted to reduce the computation cost of group convolution, as well as evaluating the importance of the feature maps which concatenated from Ghost Feature Maps and Primary Feature Maps and giving proper weights for the feature maps. This paper conducted some experiments and proved that the ShuffleGhostV3 has smaller trainable parameters and FLOPs with the ensurance of accuracy. And with proper design, it could be more efficient in both GPU and CPU side.

Download Full-text

MS2CNN: predicting MS/MS spectrum based on protein sequence using deep convolutional neural networks

BMC Genomics ◽

10.1186/s12864-019-6297-6 ◽

2019 ◽

Vol 20 (S9) ◽

Cited By ~ 1

Author(s):

Yang-Ming Lin ◽

Ching-Tai Chen ◽

Jia-Ming Chang

Keyword(s):

Neural Networks ◽

Deep Learning ◽

Convolutional Neural Networks ◽

Large Scale ◽

Learning Algorithm ◽

Pearson Correlation ◽

Tandem Mass ◽

Spectral Library ◽

Deep Convolutional Neural Networks ◽

Library Search

Abstract Background Tandem mass spectrometry allows biologists to identify and quantify protein samples in the form of digested peptide sequences. When performing peptide identification, spectral library search is more sensitive than traditional database search but is limited to peptides that have been previously identified. An accurate tandem mass spectrum prediction tool is thus crucial in expanding the peptide space and increasing the coverage of spectral library search. Results We propose MS2CNN, a non-linear regression model based on deep convolutional neural networks, a deep learning algorithm. The features for our model are amino acid composition, predicted secondary structure, and physical-chemical features such as isoelectric point, aromaticity, helicity, hydrophobicity, and basicity. MS2CNN was trained with five-fold cross validation on a three-way data split on the large-scale human HCD MS2 dataset of Orbitrap LC-MS/MS downloaded from the National Institute of Standards and Technology. It was then evaluated on a publicly available independent test dataset of human HeLa cell lysate from LC-MS experiments. On average, our model shows better cosine similarity and Pearson correlation coefficient (0.690 and 0.632) than MS2PIP (0.647 and 0.601) and is comparable with pDeep (0.692 and 0.642). Notably, for the more complex MS2 spectra of 3+ peptides, MS2PIP is significantly better than both MS2PIP and pDeep. Conclusions We showed that MS2CNN outperforms MS2PIP for 2+ and 3+ peptides and pDeep for 3+ peptides. This implies that MS2CNN, the proposed convolutional neural network model, generates highly accurate MS2 spectra for LC-MS/MS experiments using Orbitrap machines, which can be of great help in protein and peptide identifications. The results suggest that incorporating more data for deep learning model may improve performance.

Download Full-text

Sequential Network with Residual Neural Network for Rotatory Machine Remaining Useful Life Prediction Using Deep Transfer Learning

Shock and Vibration ◽

10.1155/2020/8888627 ◽

2020 ◽

Vol 2020 ◽

pp. 1-16

Author(s):

Hao Zhang ◽

Qiang Zhang ◽

Siyu Shao ◽

Tianlin Niu ◽

Xinyu Yang ◽

...

Keyword(s):

Deep Learning ◽

Transfer Learning ◽

Life Prediction ◽

Feature Learning ◽

High Accuracy ◽

Remaining Useful Life ◽

Learning Ability ◽

Model Parameters ◽

Deep Model ◽

Useful Life

Deep learning has a strong feature learning ability, which has proved its effectiveness in fault prediction and remaining useful life prediction of rotatory machine. However, training a deep network from scratch requires a large amount of training data and is time-consuming. In the practical model training process, it is difficult for the deep model to converge when the parameter initialization is inappropriate, which results in poor prediction performance. In this paper, a novel deep learning framework is proposed to predict the remaining useful life of rotatory machine with high accuracy. Firstly, model parameters and feature learning ability of the pretrained model are transferred to the new network by means of transfer learning to achieve reasonable initialization. Then, the specific sensor signals are converted to RGB image as the specific task data to fine-tune the parameters of the high-level network structure. The features extracted from the pretrained network are the input into the Bidirectional Long Short-Term Memory to obtain the RUL prediction results. The ability of LSTM to model sequence signals and the dynamic learning ability of bidirectional propagation to time information contribute to accurate RUL prediction. Finally, the deep model proposed in this paper is tested on the sensor signal dataset of bearing and gearbox. The high accuracy prediction results show the superiority of the transfer learning-based sequential network in RUL prediction.

Download Full-text

A deep learning view of the census of galaxy clusters in IllustrisTNG

Monthly Notices of the Royal Astronomical Society ◽

10.1093/mnras/staa2690 ◽

2020 ◽

Vol 498 (4) ◽

pp. 5620-5628

Author(s):

Y Su ◽

Y Zhang ◽

G Liang ◽

J A ZuHone ◽

D J Barnes ◽

...

Keyword(s):

Deep Learning ◽

Galaxy Clusters ◽

Large Scale ◽

Learning Algorithm ◽

X Ray ◽

Deep Learning Algorithm ◽

Cluster Evolution ◽

Novel Method ◽

Massive Clusters ◽

Activation Mapping

ABSTRACT The origin of the diverse population of galaxy clusters remains an unexplained aspect of large-scale structure formation and cluster evolution. We present a novel method of using X-ray images to identify cool core (CC), weak cool core (WCC), and non-cool core (NCC) clusters of galaxies that are defined by their central cooling times. We employ a convolutional neural network, ResNet-18, which is commonly used for image analysis, to classify clusters. We produce mock Chandra X-ray observations for a sample of 318 massive clusters drawn from the IllustrisTNG simulations. The network is trained and tested with low-resolution mock Chandra images covering a central 1 Mpc square for the clusters in our sample. Without any spectral information, the deep learning algorithm is able to identify CC, WCC, and NCC clusters, achieving balanced accuracies (BAcc) of 92 per cent, 81 per cent, and 83 per cent, respectively. The performance is superior to classification by conventional methods using central gas densities, with an average ${\rm BAcc}=81{{\ \rm per\ cent}}$, or surface brightness concentrations, giving ${\rm BAcc}=73{{\ \rm per\ cent}}$. We use class activation mapping to localize discriminative regions for the classification decision. From this analysis, we observe that the network has utilized regions from cluster centres out to r ≈ 300 kpc and r ≈ 500 kpc to identify CC and NCC clusters, respectively. It may have recognized features in the intracluster medium that are associated with AGN feedback and disruptive major mergers.

Download Full-text

Statistical Approaches Based on Deep Learning Regression for Verification of Normality of Blood Pressure Estimates

Sensors ◽

10.3390/s19092137 ◽

2019 ◽

Vol 19 (9) ◽

pp. 2137 ◽

Cited By ~ 1

Author(s):

Soojeong Lee ◽

Gangseong Lee ◽

Gwanggil Jeon

Keyword(s):

Blood Pressure ◽

Deep Learning ◽

Learning Algorithm ◽

Statistical Tests ◽

Single Point ◽

Absolute Error ◽

Standard Normal Distribution ◽

Rank Test ◽

The Mean ◽

Statistical Approaches

Oscillometric blood pressure (BP) monitors currently estimate a single point but do not identify variations in response to physiological characteristics. In this paper, to analyze BP’s normality based on oscillometric measurements, we use statistical approaches including kurtosis, skewness, Kolmogorov-Smirnov, and correlation tests. Then, to mitigate uncertainties, we use a deep learning method to determine the confidence limits (CLs) of BP measurements based on their normality. The proposed deep learning regression model decreases the standard deviation of error (SDE) of the mean error and the mean absolute error and reduces the uncertainties of the CLs and SDEs of the proposed technique. We validate the normality of the distribution of the BP estimation which fits the standard normal distribution very well. We use a rank test in the deep learning technique to demonstrate the independence of the artificial systolic BP and diastolic BP estimations. We perform statistical tests to verify the normality of the BP measurements for individual subjects. The proposed methodology provides accurate BP estimations and reduces the uncertainties associated with the CLs and SDEs using the deep learning algorithm.

Download Full-text

Application of deep learning methods in biological networks

Briefings in Bioinformatics ◽

10.1093/bib/bbaa043 ◽

2020 ◽

Cited By ~ 1

Author(s):

Shuting Jin ◽

Xiangxiang Zeng ◽

Feng Xia ◽

Wei Huang ◽

Xiangrong Liu

Keyword(s):

Deep Learning ◽

Biological Networks ◽

Learning Algorithm ◽

Biological Data ◽

Training Data ◽

Learning Ability ◽

Network Data ◽

Complex Data ◽

Layer By Layer ◽

Information Layer

Abstract The increase in biological data and the formation of various biomolecule interaction databases enable us to obtain diverse biological networks. These biological networks provide a wealth of raw materials for further understanding of biological systems, the discovery of complex diseases and the search for therapeutic drugs. However, the increase in data also increases the difficulty of biological networks analysis. Therefore, algorithms that can handle large, heterogeneous and complex data are needed to better analyze the data of these network structures and mine their useful information. Deep learning is a branch of machine learning that extracts more abstract features from a larger set of training data. Through the establishment of an artificial neural network with a network hierarchy structure, deep learning can extract and screen the input information layer by layer and has representation learning ability. The improved deep learning algorithm can be used to process complex and heterogeneous graph data structures and is increasingly being applied to the mining of network data information. In this paper, we first introduce the used network data deep learning models. After words, we summarize the application of deep learning on biological networks. Finally, we discuss the future development prospects of this field.

Download Full-text

Active Learning Plus Deep Learning Can Establish Cost-Effective and Robust Model for Multichannel Image: A Case on Hyperspectral Image Classification

Sensors ◽

10.3390/s20174975 ◽

2020 ◽

Vol 20 (17) ◽

pp. 4975

Author(s):

Fangyu Shi ◽

Zhaodi Wang ◽

Menghan Hu ◽

Guangtao Zhai

Keyword(s):

Deep Learning ◽

Active Learning ◽

Image Classification ◽

Large Scale ◽

Hyperspectral Image ◽

Image Annotation ◽

Learning Algorithm ◽

Magnetic Resonance Images ◽

Biological Engineering ◽

Hyperspectral Image Classification

Relying on large scale labeled datasets, deep learning has achieved good performance in image classification tasks. In agricultural and biological engineering, image annotation is time-consuming and expensive. It also requires annotators to have technical skills in specific areas. Obtaining the ground truth is difficult because natural images are expensive. In addition, images in these areas are usually stored as multichannel images, such as computed tomography (CT) images, magnetic resonance images (MRI), and hyperspectral images (HSI). In this paper, we present a framework using active learning and deep learning for multichannel image classification. We use three active learning algorithms, including least confidence, margin sampling, and entropy, as the selection criteria. Based on this framework, we further introduce an “image pool” to make full advantage of images generated by data augmentation. To prove the availability of the proposed framework, we present a case study on agricultural hyperspectral image classification. The results show that the proposed framework achieves better performance compared with the deep learning model. Manual annotation of all the training sets achieves an encouraging accuracy. In comparison, using active learning algorithm of entropy and image pool achieves a similar accuracy with only part of the whole training set manually annotated. In practical application, the proposed framework can remarkably reduce labeling effort during the model development and upadting processes, and can be applied to multichannel image classification in agricultural and biological engineering.

Download Full-text

Evaluation and Analysis of Cardiovascular Function in Intensive Care Unit Patients by Ultrasound Image Segmentation Based on Deep Learning

Journal of Medical Imaging and Health Informatics ◽

10.1166/jmihi.2020.3119 ◽

2020 ◽

Vol 10 (8) ◽

pp. 1892-1898

Author(s):

Jiaqi Shen ◽

Fangfang Huang ◽

Myers Ulrich

Keyword(s):

Image Segmentation ◽

Deep Learning ◽

Carotid Artery ◽

Blood Vessel ◽

Nerve Cell ◽

Clinical Application ◽

Learning Algorithm ◽

Linear Filter ◽

Multi Scale ◽

Deep Learning Algorithm

Many studies have shown that cardiovascular disease has become one of the major diseases leading to death in the world. Therefore, it is a very meaningful topic to use image segmentation technology to segment blood vessels for clinical application. In order to automatically extract the features of blood vessel images in the process of segmentation, the deep learning algorithm is combined with image segmentation technology to segment the nerve cell membrane and carotid artery images of ICU patients, and to segment the blood vessel images from a multi-dimensional perspective. The relevant data are collected to observe the effect of this model. The results show that the three-dimensional multi-scale linear filter has a good effect on carotid artery segmentation in the image segmentation of nerve cell membranes and carotid artery. When analyzing the accuracy of vascular image segmentation from network parameters and training parameters, it is found that the accuracy of the threedimensional multi-scale linear filter can reach about 85%. Therefore, it can be found that the combination of deep learning algorithm and image segmentation technology has a good segmentation effect, and the segmentation accuracy is also high. The experiment achieves the desired effect, which provides experimental basis for the clinical application of the vascular image segmentation technology.

Download Full-text

Incremental Regressive Learning Algorithm of Support Vector Machine and its Application

Advanced Materials Research ◽

10.4028/www.scientific.net/amr.216.301 ◽

2011 ◽

Vol 216 ◽

pp. 301-306

Author(s):

Shi Hua Zhang ◽

Xi Long Qu ◽

Xue Ye Wang

Keyword(s):

Support Vector Machine ◽

Transition Temperature ◽

Incremental Learning ◽

Large Scale ◽

Learning Algorithm ◽

Learning Ability ◽

Support Vector ◽

Learning Method ◽

Essential Characteristic ◽

Incremental Learning Method

There is no incremental learning ability for the traditional support vector machine (SVM) and there are all kind of merits and flaws for usually used incremental learning method. Normal SVM is unable to train in large-scale samples, while the computer’s memory is limited. In order to resolve this problem and improve training speed of the SVM, we analyze essential characteristic of SVM and bring up the incremental learning algorithm of SVM based on regression of SVM related to SV (support vectors). The algorithm increases the speed of training and can be able to learning with large-scale samples while its regressive precision loses fewer. The experiments show that SVM performs effectively and practically. Its application to prediction of the transition temperature (Tg) for high molecular polymers show that this model (R2=0.9427) proved to be considerably more accurate compared to a ANNs regression model (R2=0.9269).

Download Full-text

Scale-Covariant and Scale-Invariant Gaussian Derivative Networks

Journal of Mathematical Imaging and Vision ◽

10.1007/s10851-021-01057-9 ◽

2021 ◽

Author(s):

Tony Lindeberg

Keyword(s):

Deep Learning ◽

Network Architecture ◽

Large Scale ◽

Hybrid Approach ◽

Multiple Scale ◽

Scale Space ◽

Training Data ◽

Scale Invariant ◽

Gaussian Derivative ◽

Space Operations

AbstractThis paper presents a hybrid approach between scale-space theory and deep learning, where a deep learning architecture is constructed by coupling parameterized scale-space operations in cascade. By sharing the learnt parameters between multiple scale channels, and by using the transformation properties of the scale-space primitives under scaling transformations, the resulting network becomes provably scale covariant. By in addition performing max pooling over the multiple scale channels, or other permutation-invariant pooling over scales, a resulting network architecture for image classification also becomes provably scale invariant. We investigate the performance of such networks on the MNIST Large Scale dataset, which contains rescaled images from the original MNIST dataset over a factor of 4 concerning training data and over a factor of 16 concerning testing data. It is demonstrated that the resulting approach allows for scale generalization, enabling good performance for classifying patterns at scales not spanned by the training data.

Download Full-text