scholarly journals Beyond Network Pruning: a Joint Search-and-Training Approach

2020 ◽  
Author(s):  
Xiaotong Lu ◽  
Han Huang ◽  
Weisheng Dong ◽  
Xin Li ◽  
Guangming Shi
Keyword(s):  
Random Perturbation ◽  
Search Space ◽  
Fine Tuning ◽  
Superior Performance ◽  
Network Pruning ◽  
Training Approach ◽  
Target Network ◽  
Tuning Strategy ◽  
Coarse To Fine ◽  
And Training

Network pruning has been proposed as a remedy for alleviating the over-parameterization problem of deep neural networks. However, its value has been recently challenged especially from the perspective of neural architecture search (NAS). We challenge the conventional wisdom of pruning-after-training by proposing a joint search-and-training approach that directly learns a compact network from the scratch. By treating pruning as a search strategy, we present two new insights in this paper: 1) it is possible to expand the search space of networking pruning by associating each filter with a learnable weight; 2) joint search-and-training can be conducted iteratively to maximize the learning efficiency. More specifically, we propose a coarse-to-fine tuning strategy to iteratively sample and update compact sub-network to approximate the target network. The weights associated with network filters will be accordingly updated by joint search-and-training to reflect learned knowledge in NAS space. Moreover, we introduce strategies of random perturbation (inspired by Monte Carlo) and flexible thresholding (inspired by Reinforcement Learning) to adjust the weight and size of each layer. Extensive experiments on ResNet and VGGNet demonstrate the superior performance of our proposed method on popular datasets including CIFAR10, CIFAR100 and ImageNet.

scholarly journals Fine-Tuning and Training of DenseNet for Histopathology Image Representation Using TCGA Diagnostic Slides

2021 ◽  
pp. 102032
Author(s):  
Abtin Riasatian ◽  
Morteza Babaie ◽  
Danial Maleki ◽  
Shivam Kalra ◽  
Mojtaba Valipour ◽  
...  
Keyword(s):  
Image Representation ◽  
Fine Tuning ◽  
And Training

scholarly journals PCDRN: Progressive Cascade Deep Residual Network for Pansharpening

2020 ◽  
Vol 12 (4) ◽  
pp. 676 ◽  
Author(s):  
Yong Yang ◽  
Wei Tu ◽  
Shuying Huang ◽  
Hangyuan Lu
Keyword(s):  
High Resolution ◽  
Loss Function ◽  
High Frequency ◽  
Experimental Results ◽  
Superior Performance ◽  
Residual Network ◽  
High Quality ◽  
Convolution Approach ◽  
Visual Assessments ◽  
Coarse To Fine

Pansharpening is the process of fusing a low-resolution multispectral (LRMS) image with a high-resolution panchromatic (PAN) image. In the process of pansharpening, the LRMS image is often directly upsampled by a scale of 4, which may result in the loss of high-frequency details in the fused high-resolution multispectral (HRMS) image. To solve this problem, we put forward a novel progressive cascade deep residual network (PCDRN) with two residual subnetworks for pansharpening. The network adjusts the size of an MS image to the size of a PAN image twice and gradually fuses the LRMS image with the PAN image in a coarse-to-fine manner. To prevent an overly-smooth phenomenon and achieve high-quality fusion results, a multitask loss function is defined to train our network. Furthermore, to eliminate checkerboard artifacts in the fusion results, we employ a resize-convolution approach instead of transposed convolution for upsampling LRMS images. Experimental results on the Pléiades and WorldView-3 datasets prove that PCDRN exhibits superior performance compared to other popular pansharpening methods in terms of quantitative and visual assessments.

scholarly journals Two-Stage Mask-RCNN Approach for Detecting and Segmenting the Optic Nerve Head, Optic Disc, and Optic Cup in Fundus Images

2020 ◽  
Vol 10 (11) ◽  
pp. 3833 ◽  
Author(s):  
Haidar Almubarak ◽  
Yakoub Bazi ◽  
Naif Alajlan
Keyword(s):  
Optic Nerve ◽  
Optic Disc ◽  
Optic Nerve Head ◽  
Fine Tuning ◽  
Fundus Images ◽  
Two Stage ◽  
Second Stage ◽  
Retinal Fundus Images ◽  
Tuning Strategy ◽  
Retinal Fundus

In this paper, we propose a method for localizing the optic nerve head and segmenting the optic disc/cup in retinal fundus images. The approach is based on a simple two-stage Mask-RCNN compared to sophisticated methods that represent the state-of-the-art in the literature. In the first stage, we detect and crop around the optic nerve head then feed the cropped image as input for the second stage. The second stage network is trained using a weighted loss to produce the final segmentation. To further improve the detection in the first stage, we propose a new fine-tuning strategy by combining the cropping output of the first stage with the original training image to train a new detection network using different scales for the region proposal network anchors. We evaluate the method on Retinal Fundus Images for Glaucoma Analysis (REFUGE), Magrabi, and MESSIDOR datasets. We used the REFUGE training subset to train the models in the proposed method. Our method achieved 0.0430 mean absolute error in the vertical cup-to-disc ratio (MAE vCDR) on the REFUGE test set compared to 0.0414 obtained using complex and multiple ensemble networks methods. The models trained with the proposed method transfer well to datasets outside REFUGE, achieving a MAE vCDR of 0.0785 and 0.077 on MESSIDOR and Magrabi datasets, respectively, without being retrained. In terms of detection accuracy, the proposed new fine-tuning strategy improved the detection rate from 96.7% to 98.04% on MESSIDOR and from 93.6% to 100% on Magrabi datasets compared to the reported detection rates in the literature.

Global Nonlinear Modelling of Gas Turbine Dynamics Using NARMAX Structures

2001 ◽  
Author(s):  
Neophytos Chiras ◽  
Ceri Evans ◽  
David Rees
Keyword(s):  
Gas Turbine ◽  
Nonlinear Model ◽  
Linear Models ◽  
Search Space ◽  
Frequency Domain Analysis ◽  
Superior Performance ◽  
Large Signal ◽  
Nonlinear Modelling ◽  
Frequency Domains ◽  
Identification Techniques

This paper examines the estimation of a global nonlinear gas turbine model using NARMAX techniques. Linear models estimated on small-signal data are first examined and the need for a global nonlinear model is established. A nonparametric analysis of the engine nonlinearity is then performed in the time and frequency domains. The information obtained from the linear modelling and nonlinear analysis is used to restrict the search space for nonlinear modelling. The nonlinear model is then validated using large-signal data and its superior performance illustrated by comparison with a linear model. This paper illustrates how periodic test signals, frequency domain analysis and identification techniques, and time-domain NARMAX modelling can be effectively combined to enhance the modelling of an aircraft gas turbine.

On-device Prior Knowledge Incorporated Learning for Personalized Atrial Fibrillation Detection

2021 ◽  
Vol 20 (5s) ◽  
pp. 1-25
Author(s):  
Zhenge Jia ◽  
Yiyu Shi ◽  
Samir Saba ◽  
Jingtong Hu
Keyword(s):  
Atrial Fibrillation ◽  
Prior Knowledge ◽  
Domain Knowledge ◽  
Fine Tuning ◽  
Cardiac Monitoring ◽  
Patient Specific ◽  
Detection Accuracy ◽  
Specific Patient ◽  
Deep Model ◽  
Tuning Strategy

Atrial Fibrillation (AF), one of the most prevalent arrhythmias, is an irregular heart-rate rhythm causing serious health problems such as stroke and heart failure. Deep learning based methods have been exploited to provide an end-to-end AF detection by automatically extracting features from Electrocardiogram (ECG) signal and achieve state-of-the-art results. However, the pre-trained models cannot adapt to each patient’s rhythm due to the high variability of rhythm characteristics among different patients. Furthermore, the deep models are prone to overfitting when fine-tuned on the limited ECG of the specific patient for personalization. In this work, we propose a prior knowledge incorporated learning method to effectively personalize the model for patient-specific AF detection and alleviate the overfitting problems. To be more specific, a prior-incorporated portion importance mechanism is proposed to enforce the network to learn to focus on the targeted portion of the ECG, following the cardiologists’ domain knowledge in recognizing AF. A prior-incorporated regularization mechanism is further devised to alleviate model overfitting during personalization by regularizing the fine-tuning process with feature priors on typical AF rhythms of the general population. The proposed personalization method embeds the well-defined prior knowledge in diagnosing AF rhythm into the personalization procedure, which improves the personalized deep model and eliminates the workload of manually adjusting parameters in conventional AF detection method. The prior knowledge incorporated personalization is feasibly and semi-automatically conducted on the edge, device of the cardiac monitoring system. We report an average AF detection accuracy of 95.3% of three deep models over patients, surpassing the pre-trained model by a large margin of 11.5% and the fine-tuning strategy by 8.6%.

scholarly journals Recognizing human activities from smartphone sensors using hierarchical continuous hidden Markov models

2017 ◽  
Vol 13 (1) ◽  
pp. 155014771668368 ◽  
Author(s):  
Charissa Ann Ronao ◽  
Sung-Bae Cho
Keyword(s):  
Hierarchical Structure ◽  
Markov Models ◽  
Hidden Markov ◽  
Classification Performance ◽  
Fine Tuning ◽  
Sensor Data ◽  
Model Framework ◽  
The Hierarchical Structure ◽  
Smartphone Sensors ◽  
And Training

Human activity recognition has been gaining more and more attention from researchers in recent years, particularly with the use of widespread and commercially available devices such as smartphones. However, most of the existing works focus on discriminative classifiers while neglecting the inherent time-series and continuous characteristics of sensor data. To address this, we propose a two-stage continuous hidden Markov model framework, which also takes advantage of the innate hierarchical structure of basic activities. This kind of system architecture not only enables the use of different feature subsets on different subclasses, which effectively reduces feature computation overhead, but also allows for varying number of states and iterations. Experiments show that the hierarchical structure dramatically increases classification performance. We analyze the behavior of the accelerometer and gyroscope signals for each activity through graphs, and with added fine tuning of states and training iterations, the proposed method is able to achieve an overall accuracy of up to 93.18%, which is the best performance among the state-of-the-art classifiers for the problem at hand.

scholarly journals Ls-II: An Improved Locust Search Algorithm for Solving Optimization Problems

2018 ◽  
Vol 2018 ◽  
pp. 1-15 ◽  
Author(s):  
Octavio Camarena ◽  
Erik Cuevas ◽  
Marco Pérez-Cisneros ◽  
Fernando Fausto ◽  
Adrián González ◽  
...  
Keyword(s):  
Optimization Problems ◽  
Search Algorithm ◽  
Search Space ◽  
Optimization Method ◽  
Superior Performance ◽  
Solution Quality ◽  
Original Algorithm ◽  
Motion Models ◽  
Search Mechanism ◽  
Solitary Phase

The Locust Search (LS) algorithm is a swarm-based optimization method inspired in the natural behavior of the desert locust. LS considers the inclusion of two distinctive nature-inspired search mechanism, namely, their solitary phase and social phase operators. These interesting search schemes allow LS to overcome some of the difficulties that commonly affect other similar methods, such as premature convergence and the lack of diversity on solutions. Recently, computer vision experiments in insect tracking methods have conducted to the development of more accurate locust motion models than those produced by simple behavior observations. The most distinctive characteristic of such new models is the use of probabilities to emulate the locust decision process. In this paper, a modification to the original LS algorithm, referred to as LS-II, is proposed to better handle global optimization problems. In LS-II, the locust motion model of the original algorithm is modified incorporating the main characteristics of the new biological formulations. As a result, LS-II improves its original capacities of exploration and exploitation of the search space. In order to test its performance, the proposed LS-II method is compared against several the state-of-the-art evolutionary methods considering a set of benchmark functions and engineering problems. Experimental results demonstrate the superior performance of the proposed approach in terms of solution quality and robustness.

scholarly journals Scanning electron microscope fine tuning using four-bar piezoelectric actuated mechanism

2018 ◽  
Vol 69 (1) ◽  
pp. 24-31
Author(s):  
Khaled S. Hatamleh ◽  
Qais A. Khasawneh ◽  
Adnan Al-Ghasem ◽  
Mohammad A. Jaradat ◽  
Laith Sawaqed ◽  
...  
Keyword(s):  
Electron Microscope ◽  
Scanning Electron Microscope ◽  
Inverse Kinematic ◽  
Fine Tuning ◽  
Electron Microscopes ◽  
Tuning Strategy ◽  
Scanning Electron Microscopes ◽  
Fine Tune ◽  
Kinematic Solution ◽  
Scanning Electron

Abstract Scanning Electron Microscopes are extensively used for accurate micro/nano images exploring. Several strategies have been proposed to fine tune those microscopes in the past few years. This work presents a new fine tuning strategy of a scanning electron microscope sample table using four bar piezoelectric actuated mechanisms. The introduced paper presents an algorithm to find all possible inverse kinematics solutions of the proposed mechanism. In addition, another algorithm is presented to search for the optimal inverse kinematic solution. Both algorithms are used simultaneously by means of a simulation study to fine tune a scanning electron microscope sample table through a pre-specified circular or linear path of motion. Results of the study shows that, proposed algorithms were able to minimize the power required to drive the piezoelectric actuated mechanism by a ratio of 97.5% for all simulated paths of motion when compared to general non-optimized solution.

Inhibition of Hill Reaction by 2-Azido-s-triazine Derivatives: QSAR Study with Molecular Connectivity Indices

1989 ◽  
Vol 44 (3-4) ◽  
pp. 255-261 ◽  
Author(s):  
Milan Š Šoškić ◽  
Aleksandar Sabljić
Keyword(s):  
Structural Features ◽  
Second Order ◽  
Fine Tuning ◽  
Superior Performance ◽  
Hill Reaction ◽  
Retention Data ◽  
Molecular Connectivity ◽  
Inhibitory Potency ◽  
Molecular Connectivity Indices ◽  
Connectivity Indices

Abstract 2-A zido-s-T riazines, Connectivity Indices. Inhibition of Hill Reaction. Photosystem II, OSAR Modelling This study was undertaken to find a simple and accurate structural parameters for the quantita­tive description of inhibitory potency of 2-azido-s-triazines in Hill reaction and to gain more information about the mechanism of inhibition on molecular level. A very good correlation (r = 0.946) was obtained between the pl50 values (the negative logarithm of the molar concentra­tion that causes 50% inhibition) and the valence zero-order and the difference between the second-order and the valence second-order molecular connectivity indices. This model, when com pared with the empirical models based on the 1-octano/water partition coefficients and the chromatographic retention data, shows superior performance in accuracy and range of applicabili­ty. In addition, the direct correspondence between molecular structure and above connectivity indices makes it possible to locate structural features responsible for the inhibitory potency of 2-azido-s-triazines in Hill reaction. From our OSAR analysis, the interaction between the chloro­plast receptor site and 2-azido-s-triazines, which causes inhibition of Hill reaction, is primarily influenced by the size of alkylamino substituents and it accounts for the most variation in the pl50 data. The structural features of secondary importance that control the magnitude of pl50’s are the polarity of alkylamino chains and the degree of branching on alpha carbon atom of R:alkylamino substituent. Com pared with the main factor, the size of alkylamino substituents, they can be viewed as a fine tuning elements for the inhibitory potency of 2-azido-s-triazines.

Sign in / Sign up

Export Citation Format

Share Document