Most network pruning methods rely on rule-of-thumb for human experts to prune the unimportant channels. This is time-consuming and can lead to suboptimal pruning. In this paper, we propose an effective SuperPruner algorithm, which aims to find optimal pruned structure instead of pruning unimportant channels. We first train a VerifyNet, a kind of super network, which is able to roughly evaluate the performance of any given network structure. The particle swarm optimization algorithm is then used to search for optimal network structure. Lastly, the weights in the VerifyNet are used as the initial weights of the optimal pruned structure to make fine-tuning. VerifyNet is a network performance evaluation; our algorithm can quickly prune the network under any hardware constraints. Our algorithm can be applied in multiple fields such as object recognition and semantic segmentation. Extensive experiment results demonstrate the effectiveness of SuperPruner. For example, on CIFAR-10, the pruned VGG16 achieves 93.18% Top-1 accuracy and reduces 74.19% of FLOPs and 89.25% of parameters. Compared with state-of-the-art methods, our algorithm can achieve higher pruned ratio with less accuracy cost.
SuperPruner: Automatic Neural Network Pruning via Super Network
