scholarly journals Automatic Deep Feature Learning via Patch-Based Deep Belief Network for Vertebrae Segmentation in CT Images

2018 ◽  
Vol 9 (1) ◽  
pp. 69 ◽  
Author(s):  
Syed Furqan Qadri ◽  
Danni Ai ◽  
Guoyu Hu ◽  
Mubashir Ahmad ◽  
Yong Huang ◽  
...  
Keyword(s):  
Computational Cost ◽  
Feature Learning ◽  
Region Of Interest ◽  
Ct Images ◽  
Feature Reduction ◽  
Fine Tuning ◽  
Deep Feature ◽  
Image Patches ◽  
Contrastive Divergence ◽  
Vertebrae Segmentation

Precise automatic vertebra segmentation in computed tomography (CT) images is important for the quantitative analysis of vertebrae-related diseases but remains a challenging task due to high variation in spinal anatomy among patients. In this paper, we propose a deep learning approach for automatic CT vertebra segmentation named patch-based deep belief networks (PaDBNs). Our proposed PaDBN model automatically selects the features from image patches and then measures the differences between classes and investigates performance. The region of interest (ROI) is obtained from CT images. Unsupervised feature reduction contrastive divergence algorithm is applied for weight initialization, and the weights are optimized by layers in a supervised fine-tuning procedure. The discriminative learning features obtained from the steps above are used as input of a classifier to obtain the likelihood of the vertebrae. Experimental results demonstrate that the proposed PaDBN model can considerably reduce computational cost and produce an excellent performance in vertebra segmentation in terms of accuracy compared with state-of-the-art methods.

A Novel Method for Classification of ECG Arrhythmias Using Deep Belief Networks

2016 ◽  
Vol 15 (04) ◽  
pp. 1650021 ◽  
Author(s):  
Zhiyong Wu ◽  
Xiangqian Ding ◽  
Guangrui Zhang
Keyword(s):  
Feature Learning ◽  
Feature Representation ◽  
Fine Tuning ◽  
Classification Model ◽  
Training Algorithms ◽  
Belief Networks ◽  
Deep Belief Networks ◽  
Ecg Signals ◽  
Contrastive Divergence

In this paper, a novel approach based on deep belief networks (DBN) for electrocardiograph (ECG) arrhythmias classification is proposed. The construction process of ECG classification model consists of two steps: features learning for ECG signals and supervised fine-tuning. In order to deeply extract features from continuous ECG signals, two types of restricted Boltzmann machine (RBM) including Gaussian–Bernoulli and Bernoulli–Bernoulli are stacked to form DBN. The parameters of RBM can be learned by two training algorithms such as contrastive divergence and persistent contrastive divergence. A suitable feature representation from the raw ECG data can therefore be extracted in an unsupervised way. In order to enhance the performance of DBN, a fine-tuning process is carried out, which uses backpropagation by adding a softmax regression layer on the top of the resulting hidden representation layer to perform multiclass classification. The method is then validated by experiments on the well-known MIT-BIH arrhythmia database. Considering the real clinical application, the inter-patient heartbeat dataset is divided into two sets and grouped into four classes (N, S, V, F) following the recommendations of AAMI. The experiment results show our approach achieves better performance with less feature learning time than traditional hand-designed methods on the classification of ECG arrhythmias.

scholarly journals Simple Yet Effective Fine-Tuning of Deep CNNs Using an Auxiliary Classification Loss for Remote Sensing Scene Classification

2019 ◽  
Vol 11 (24) ◽  
pp. 2908 ◽  
Author(s):  
Yakoub Bazi ◽  
Mohamad M. Al Rahhal ◽  
Haikel Alhichri ◽  
Naif Alajlan
Keyword(s):  
Remote Sensing ◽  
State Of The Art ◽  
Computational Cost ◽  
Feature Learning ◽  
Extraction Methods ◽  
Fine Tuning ◽  
Scene Classification ◽  
Negative Effect ◽  
Benchmark Datasets ◽  
Low Computational Cost

The current literature of remote sensing (RS) scene classification shows that state-of-the-art results are achieved using feature extraction methods, where convolutional neural networks (CNNs) (mostly VGG16 with 138.36 M parameters) are used as feature extractors and then simple to complex handcrafted modules are added for additional feature learning and classification, thus coming back to feature engineering. In this paper, we revisit the fine-tuning approach for deeper networks (GoogLeNet and Beyond) and show that it has not been well exploited due to the negative effect of the vanishing gradient problem encountered when transferring knowledge to small datasets. The aim of this work is two-fold. Firstly, we provide best practices for fine-tuning pre-trained CNNs using the root-mean-square propagation (RMSprop) method. Secondly, we propose a simple yet effective solution for tackling the vanishing gradient problem by injecting gradients at an earlier layer of the network using an auxiliary classification loss function. Then, we fine-tune the resulting regularized network by optimizing both the primary and auxiliary losses. As for pre-trained CNNs, we consider in this work inception-based networks and EfficientNets with small weights: GoogLeNet (7 M) and EfficientNet-B0 (5.3 M) and their deeper versions Inception-v3 (23.83 M) and EfficientNet-B3 (12 M), respectively. The former networks have been used previously in the context of RS and yielded low accuracies compared to VGG16, while the latter are new state-of-the-art models. Extensive experimental results on several benchmark datasets reveal clearly that if fine-tuning is done in an appropriate way, it can settle new state-of-the-art results with low computational cost.

TLBO-FLN: Teaching-Learning Based Optimization of Functional Link Neural Networks for Stock Closing Price Prediction

2020 ◽  
Vol 10 (4) ◽  
pp. 522-532 ◽  
Author(s):  
Sarat Chandra Nayak ◽  
Subhranginee Das ◽  
Mohammad Dilsad Ansari
Keyword(s):  
Neural Networks ◽  
Computational Cost ◽  
Optimization Techniques ◽  
Fine Tuning ◽  
Functional Link ◽  
Price Prediction ◽  
Closing Price ◽  
Teaching Learning Based Optimization ◽  
Artificial Neural ◽  
Teaching Learning

Background and Objective: Stock closing price prediction is enormously complicated. Artificial Neural Networks (ANN) are excellent approximation algorithms applied to this area. Several nature-inspired evolutionary optimization techniques are proposed and used in the literature to search the optimum parameters of ANN based forecasting models. However, most of them need fine-tuning of several control parameters as well as algorithm specific parameters to achieve optimal performance. Improper tuning of such parameters either leads toward additional computational cost or local optima. Methods: Teaching Learning Based Optimization (TLBO) is a newly proposed algorithm which does not necessitate any parameters specific to it. The intrinsic capability of Functional Link Artificial Neural Network (FLANN) to recognize the multifaceted nonlinear relationship present in the historical stock data made it popular and got wide applications in the stock market prediction. This article presents a hybrid model termed as Teaching Learning Based Optimization of Functional Neural Networks (TLBO-FLN) by combining the advantages of both TLBO and FLANN. Results and Conclusion: The model is evaluated by predicting the short, medium, and long-term closing prices of four emerging stock markets. The performance of the TLBO-FLN model is measured through Mean Absolute Percentage of Error (MAPE), Average Relative Variance (ARV), and coefficient of determination (R2); compared with that of few other state-of-the-art models similarly trained and found superior.

Construction of a deep sparse filtering network for rotating machinery fault diagnosis

2021 ◽  
pp. 095440702110148
Author(s):  
Chun Cheng ◽  
Wei Zou ◽  
Weiping Wang ◽  
Michael Pecht
Keyword(s):  
Fault Diagnosis ◽  
Back Propagation ◽  
Feature Learning ◽  
Back Propagation Neural Network ◽  
Rotating Machinery ◽  
Fine Tuning ◽  
Local Optimum ◽  
Back Propagation Algorithm ◽  
Intelligent Fault Diagnosis ◽  
Sparse Filtering

Deep neural networks (DNNs) have shown potential in intelligent fault diagnosis of rotating machinery. However, traditional DNNs such as the back-propagation neural network are highly sensitive to the initial weights and easily fall into the local optimum, which restricts the feature learning capability and diagnostic performance. To overcome the above problems, a deep sparse filtering network (DSFN) constructed by stacked sparse filtering is developed in this paper and applied to fault diagnosis. The developed DSFN is pre-trained by sparse filtering in an unsupervised way. The back-propagation algorithm is employed to optimize the DSFN after pre-training. Then, the DSFN-based intelligent fault diagnosis method is validated using two experiments. The results show that pre-training with sparse filtering and fine-tuning can help the DSFN search for the optimal network parameters, and the DSFN can learn discriminative features adaptively from rotating machinery datasets. Compared with classical methods, the developed diagnostic method can diagnose rotating machinery faults with higher accuracy using fewer training samples.

scholarly journals Updated Kriging-Assisted Shape Optimization of a Gravity Dam

Water ◽  
2021 ◽  
Vol 13 (1) ◽  
pp. 87
Author(s):  
Yongqiang Wang ◽  
Ye Liu ◽  
Xiaoyi Ma
Keyword(s):  
Surrogate Model ◽  
Computational Cost ◽  
Region Of Interest ◽  
Optimization Procedure ◽  
Optimal Shape ◽  
Small Sample ◽  
Gravity Dam ◽  
Kriging Surrogate Model ◽  
Gravity Dams ◽  
Sample Set

The numerical simulation of the optimal design of gravity dams is computationally expensive. Therefore, a new optimization procedure is presented in this study to reduce the computational cost for determining the optimal shape of a gravity dam. Optimization was performed using a combination of the genetic algorithm (GA) and an updated Kriging surrogate model (UKSM). First, a Kriging surrogate model (KSM) was constructed with a small sample set. Second, the minimizing the predictor strategy was used to add samples in the region of interest to update the KSM in each updating cycle until the optimization process converged. Third, an existing gravity dam was used to demonstrate the effectiveness of the GA–UKSM. The solution obtained with the GA–UKSM was compared with that obtained using the GA–KSM. The results revealed that the GA–UKSM required only 7.53% of the total number of numerical simulations required by the GA–KSM to achieve similar optimization results. Thus, the GA–UKSM can significantly improve the computational efficiency. The method adopted in this study can be used as a reference for the optimization of the design of gravity dams.

scholarly journals Data-Informed Decomposition for Localized Uncertainty Quantification of Dynamical Systems

Vibration ◽  
2020 ◽  
Vol 4 (1) ◽  
pp. 49-63
Author(s):  
Waad Subber ◽  
Sayan Ghosh ◽  
Piyush Pandita ◽  
Yiming Zhang ◽  
Liping Wang
Keyword(s):  
Dynamical Systems ◽  
Computational Cost ◽  
Three Dimensional ◽  
Region Of Interest ◽  
System Response ◽  
Computational Domain ◽  
User Interest ◽  
Numerical Resolution ◽  
Multi Scale ◽  
Operation Conditions

Industrial dynamical systems often exhibit multi-scale responses due to material heterogeneity and complex operation conditions. The smallest length-scale of the systems dynamics controls the numerical resolution required to resolve the embedded physics. In practice however, high numerical resolution is only required in a confined region of the domain where fast dynamics or localized material variability is exhibited, whereas a coarser discretization can be sufficient in the rest majority of the domain. Partitioning the complex dynamical system into smaller easier-to-solve problems based on the localized dynamics and material variability can reduce the overall computational cost. The region of interest can be specified based on the localized features of the solution, user interest, and correlation length of the material properties. For problems where a region of interest is not evident, Bayesian inference can provide a feasible solution. In this work, we employ a Bayesian framework to update the prior knowledge of the localized region of interest using measurements of the system response. Once, the region of interest is identified, the localized uncertainty is propagate forward through the computational domain. We demonstrate our framework using numerical experiments on a three-dimensional elastodynamic problem.

scholarly journals Multiview Deep Feature Learning Network for SAR Automatic Target Recognition

2021 ◽  
Vol 13 (8) ◽  
pp. 1455
Author(s):  
Jifang Pei ◽  
Weibo Huo ◽  
Chenwei Wang ◽  
Yulin Huang ◽  
Yin Zhang ◽  
...  
Keyword(s):  
Target Recognition ◽  
Feature Learning ◽  
Automatic Target Recognition ◽  
Operating Conditions ◽  
Sar Images ◽  
Single View ◽  
Learning Network ◽  
Deep Feature ◽  
Deep Feature Learning ◽  
Classification Information

Multiview synthetic aperture radar (SAR) images contain much richer information for automatic target recognition (ATR) than a single-view one. It is desirable to establish a reasonable multiview ATR scheme and design effective ATR algorithm to thoroughly learn and extract that classification information, so that superior SAR ATR performance can be achieved. Hence, a general processing framework applicable for a multiview SAR ATR pattern is first given in this paper, which can provide an effective approach to ATR system design. Then, a new ATR method using a multiview deep feature learning network is designed based on the proposed multiview ATR framework. The proposed neural network is with a multiple input parallel topology and some distinct deep feature learning modules, with which significant classification features, the intra-view and inter-view features existing in the input multiview SAR images, will be learned simultaneously and thoroughly. Therefore, the proposed multiview deep feature learning network can achieve an excellent SAR ATR performance. Experimental results have shown the superiorities of the proposed multiview SAR ATR method under various operating conditions.

scholarly journals Plant Leaf Disease Recognition Using Depth-Wise Separable Convolution-Based Models

Symmetry ◽  
2021 ◽  
Vol 13 (3) ◽  
pp. 511
Author(s):  
Syed Mohammad Minhaz Hossain ◽  
Kaushik Deb ◽  
Pranab Kumar Dhar ◽  
Takeshi Koshiba
Keyword(s):  
State Of The Art ◽  
Computational Cost ◽  
Region Of Interest ◽  
Number Of Clusters ◽  
Plant Leaf ◽  
Leaf Disease ◽  
Automatic Initialization ◽  
Adequate Accuracy ◽  
Model Size ◽  
High Computational Cost

Proper plant leaf disease (PLD) detection is challenging in complex backgrounds and under different capture conditions. For this reason, initially, modified adaptive centroid-based segmentation (ACS) is used to trace the proper region of interest (ROI). Automatic initialization of the number of clusters (K) using modified ACS before recognition increases tracing ROI’s scalability even for symmetrical features in various plants. Besides, convolutional neural network (CNN)-based PLD recognition models achieve adequate accuracy to some extent. However, memory requirements (large-scaled parameters) and the high computational cost of CNN-based PLD models are burning issues for the memory restricted mobile and IoT-based devices. Therefore, after tracing ROIs, three proposed depth-wise separable convolutional PLD (DSCPLD) models, such as segmented modified DSCPLD (S-modified MobileNet), segmented reduced DSCPLD (S-reduced MobileNet), and segmented extended DSCPLD (S-extended MobileNet), are utilized to represent the constructive trade-off among accuracy, model size, and computational latency. Moreover, we have compared our proposed DSCPLD recognition models with state-of-the-art models, such as MobileNet, VGG16, VGG19, and AlexNet. Among segmented-based DSCPLD models, S-modified MobileNet achieves the best accuracy of 99.55% and F1-sore of 97.07%. Besides, we have simulated our DSCPLD models using both full plant leaf images and segmented plant leaf images and conclude that, after using modified ACS, all models increase their accuracy and F1-score. Furthermore, a new plant leaf dataset containing 6580 images of eight plants was used to experiment with several depth-wise separable convolution models.

Sign in / Sign up

Export Citation Format

Share Document