scholarly journals SuperPruner: Automatic Neural Network Pruning via Super Network

2021 ◽  
Vol 2021 ◽  
pp. 1-11
Author(s):  
Yu Liu ◽  
Yong Wang ◽  
Haojin Qi ◽  
Xiaoming Ju
Keyword(s):  
Network Structure ◽  
Network Performance ◽  
Particle Swarm Optimization Algorithm ◽  
State Of The Art ◽  
Semantic Segmentation ◽  
Fine Tuning ◽  
Swarm Optimization ◽  
Network Pruning ◽  
Optimal Network ◽  
Network Performance Evaluation

Most network pruning methods rely on rule-of-thumb for human experts to prune the unimportant channels. This is time-consuming and can lead to suboptimal pruning. In this paper, we propose an effective SuperPruner algorithm, which aims to find optimal pruned structure instead of pruning unimportant channels. We first train a VerifyNet, a kind of super network, which is able to roughly evaluate the performance of any given network structure. The particle swarm optimization algorithm is then used to search for optimal network structure. Lastly, the weights in the VerifyNet are used as the initial weights of the optimal pruned structure to make fine-tuning. VerifyNet is a network performance evaluation; our algorithm can quickly prune the network under any hardware constraints. Our algorithm can be applied in multiple fields such as object recognition and semantic segmentation. Extensive experiment results demonstrate the effectiveness of SuperPruner. For example, on CIFAR-10, the pruned VGG16 achieves 93.18% Top-1 accuracy and reduces 74.19% of FLOPs and 89.25% of parameters. Compared with state-of-the-art methods, our algorithm can achieve higher pruned ratio with less accuracy cost.

Geometry-Based PSO for Mechatronic System Design Optimization

2013 ◽  
Author(s):  
Wenqiang Yuan ◽  
Yusheng Liu
Keyword(s):  
Pareto Front ◽  
Particle Swarm Optimization Algorithm ◽  
State Of The Art ◽  
Mechatronic System ◽  
Swarm Optimization ◽  
Multi Objective ◽  
Data Points ◽  
System Design Optimization ◽  
Geometry Analysis ◽  
Parameter Values

In this work, we present a new multi-objective particle swarm optimization algorithm (PSO) characterized by the use of the geometrization analysis of the particles. The proposed method, called geometry analysis PSO (GAPSO), firstly parameterize the data points of the optimization model of mechatronic system to obtain their parameter values, then one curve or one surface is adopted to fit these points and the tangent value and normal value for each point are acquired, eventually the particles are guided by the use of its derivative value and tangent value to approximate the true Pareto front and get a uniform distribution. Our proposed method is compared with respect to two multi-objective metaheuristics representative of the state-of-the-art in this area. The experiments carried out indicate that GAPSO obtains remarkable results in terms of both accuracy and distribution.

scholarly journals A Review of Automatic Music Generation Based on Performance RNN

2020 ◽  
Author(s):  
Jiyanbo Cao ◽  
Jinan Fiaidhi ◽  
Maolin Qi
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Recurrent Neural Network ◽  
Network Structure ◽  
Network Performance ◽  
State Of The Art ◽  
Learning Techniques ◽  
History Of ◽  
Music Generation ◽  
Automatic Music Generation

This paper has reviewed the deep learning techniques which used in music generation. The research was based on <i>Sageev Oore's</i> proposed LSTM based recurrent neural network (Performance RNN). We have study the history of automatic music generation, and now we are using a state of the art techniques to achieve this mission. We have conclude the process of making a MIDI file to a structure as input of Performance RNN and the network structure of it.

Efficient Risk Profiling Using Bayesian Networks and Particle Swarm Optimization Algorithm

2016 ◽  
pp. 1580-1612
Author(s):  
Goran Klepac ◽  
Leo Mrsic ◽  
Robert Kopal
Keyword(s):  
Particle Swarm Optimization ◽  
Bayesian Network ◽  
Network Structure ◽  
Optimization Algorithm ◽  
Particle Swarm Optimization Algorithm ◽  
Expert Knowledge ◽  
Particle Swarm ◽  
Swarm Optimization ◽  
Bayesian Network Structure ◽  
Risk Profiling

Chapter introduce usage of particle swarm optimization algorithm and explained methodology, as a tool for discovering customer profiles based on previously developed Bayesian network (BN). Bayesian network usage is common known method for risk modelling although BN's are not pure statistical predictive models (like neural networks or logistic regression, for example) because their structure could also depend on expert knowledge. Bayesian network structure could be trained using algorithm but, from perspective of businesses requirements model efficiency and overall performance, it is recommended that domain expert modify Bayesian network structure using expert knowledge and experience. Chapter will also explain methodology of using particle swarm optimization algorithm as a tool for finding most riskiness profiles based on previously developed Bayesian network. Presented methodology has significant practical value in all phases of decision support in business environment (especially for complex environments).

scholarly journals A Review of Automatic Music Generation Based on Performance RNN

2020 ◽  
Author(s):  
Jiyanbo Cao ◽  
Jinan Fiaidhi ◽  
Maolin Qi
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Recurrent Neural Network ◽  
Network Structure ◽  
Network Performance ◽  
State Of The Art ◽  
Learning Techniques ◽  
History Of ◽  
Music Generation ◽  
Automatic Music Generation

This paper has reviewed the deep learning techniques which used in music generation. The research was based on <i>Sageev Oore's</i> proposed LSTM based recurrent neural network (Performance RNN). We have study the history of automatic music generation, and now we are using a state of the art techniques to achieve this mission. We have conclude the process of making a MIDI file to a structure as input of Performance RNN and the network structure of it.

scholarly journals Fitness Function Based Particle Swarm Optimization Algorithm for Mobile Adhoc Networks

2018 ◽  
Vol 7 (3.1) ◽  
pp. 31
Author(s):  
Rohan Gupta ◽  
Gurpreet Singh ◽  
Amanpreet Kaur ◽  
Aashdeep Singh
Keyword(s):  
Particle Swarm Optimization ◽  
Optimization Algorithm ◽  
Network Performance ◽  
Particle Swarm Optimization Algorithm ◽  
Fitness Function ◽  
Particle Swarm ◽  
Delivery Ratio ◽  
Average Delay ◽  
Swarm Optimization ◽  
Mobile Adhoc Network

Mobile adhoc network is a network which carries out discussion between nodes in the absence of infrastructure. The fitness function based Particle Swarm Optimization Algorithm has been projected for improving the network performance. The effect of changing the number of nodes, communication range and transmission range is investigated on various qualities of service metrics namely packet delivery ratio, throughput and average delay. The investigation has been carried out using NS-2 simulator.  

An Improved Swarm Optimizer for RFID Network Scheduling

2013 ◽  
Vol 427-429 ◽  
pp. 600-605
Author(s):  
Shi Lei Lu ◽  
Shun Zheng Yu
Keyword(s):  
Radio Frequency Identification ◽  
Network Performance ◽  
Particle Swarm Optimization Algorithm ◽  
Foraging Strategy ◽  
Work Status ◽  
Network Scheduling ◽  
Swarm Optimization ◽  
Network Topologies ◽  
Frequency Identification ◽  
High Diversity

Optimization of network scheduling is a significant way to improve the performance of the radio frequency identification (RFID) networks. This paper proposes an improved particle swarm optimization algorithm (PSO). It uses an animal foraging strategy to maintain a high diversity of swarms, which can protect them from premature convergence. The proposed algorithm is used to optimize the network performance by determining the optimal work status of readers. It has been tested in two different RFID network topologies to evaluate the effectivenesss. The simulation results reveal that the proposed algorithm outperforms the other algorithms in terms of optimization precision.

scholarly journals A Local and Global Search Combined Particle Swarm Optimization Algorithm and Its Convergence Analysis

2014 ◽  
Vol 2014 ◽  
pp. 1-11 ◽  
Author(s):  
Weitian Lin ◽  
Zhigang Lian ◽  
Xingsheng Gu ◽  
Bin Jiao
Keyword(s):  
Particle Swarm Optimization ◽  
Optimization Algorithm ◽  
Particle Swarm Optimization Algorithm ◽  
Particle Swarm ◽  
Global Search ◽  
Fine Tuning ◽  
Particle Swarm Algorithm ◽  
Swarm Optimization ◽  
Large Size ◽  
Swarm Algorithm

Particle swarm optimization algorithm (PSOA) is an advantage optimization tool. However, it has a tendency to get stuck in a near optimal solution especially for middle and large size problems and it is difficult to improve solution accuracy by fine-tuning parameters. According to the insufficiency, this paper researches the local and global search combine particle swarm algorithm (LGSCPSOA), and its convergence and obtains its convergence qualification. At the same time, it is tested with a set of 8 benchmark continuous functions and compared their optimization results with original particle swarm algorithm (OPSOA). Experimental results indicate that the LGSCPSOA improves the search performance especially on the middle and large size benchmark functions significantly.

Sign in / Sign up

Export Citation Format

Share Document