A Dynamically Adjusted Subspace Gradient Method and Its Application in Image Restoration

In this paper, a new subspace gradient method is proposed in which the search direction is determined by solving an approximate quadratic model in which a simple symmetric matrix is used to estimate the Hessian matrix in a three-dimensional subspace. The obtained algorithm has the ability to automatically adjust the search direction according to the feedback from experiments. Under some mild assumptions, we use the generalized line search with non-monotonicity to obtain remarkable results, which not only establishes the global convergence of the algorithm for general functions, but also R-linear convergence for uniformly convex functions is further proved. The numerical performance for both the traditional test functions and image restoration problems show that the proposed algorithm is efficient.

A Tristage Adaptive Biased Learning for Artificial Bee Colony

In recent ten years, artificial bee colony (ABC) has attracted more and more attention, and many state-of-the-art ABC variants (ABCs) have been developed by introducing different biased information to the search equations. However, the same biased information is employed in employed bee and onlooker bee phases, which will cause over exploitation and lead to premature convergence. To overcome this limit, an effective framework with tristage adaptive biased learning is proposed for existing ABCs (TABL + ABCs). In TABL + ABCs, the search direction in the employed bee stage is guided by learning the ranking biased information of the parent food sources, while in the onlooker bee stage, the search direction is determined by extracting the biased information of population distribution. Moreover, a deletion-restart learning strategy is designed in scout bee stage to prevent the potential risk of population stagnation. Systematic experiment results conducted on CEC2014 competition benchmark suite show that proposed TABL + ABCs perform better than recently published AEL + ABCs and ACoS + ABCs.

A Modified PRP-CG Type Derivative-Free Algorithm with Optimal Choices for Solving Large-Scale Nonlinear Symmetric Equations

Inspired by the large number of applications for symmetric nonlinear equations, this article will suggest two optimal choices for the modified Polak–Ribiére–Polyak (PRP) conjugate gradient (CG) method by minimizing the measure function of the search direction matrix and combining the proposed direction with the default Newton direction. In addition, the corresponding PRP parameters are incorporated with the Li and Fukushima approximate gradient to propose two robust CG-type algorithms for finding solutions for large-scale systems of symmetric nonlinear equations. We have also demonstrated the global convergence of the suggested algorithms using some classical assumptions. Finally, we demonstrated the numerical advantages of the proposed algorithms compared to some of the existing methods for nonlinear symmetric equations.

Efficient Resource-Aware Convolutional Neural Architecture Search for Edge Computing with Pareto-Bayesian Optimization

With the development of deep learning technologies and edge computing, the combination of them can make artificial intelligence ubiquitous. Due to the constrained computation resources of the edge device, the research in the field of on-device deep learning not only focuses on the model accuracy but also on the model efficiency, for example, inference latency. There are many attempts to optimize the existing deep learning models for the purpose of deploying them on the edge devices that meet specific application requirements while maintaining high accuracy. Such work not only requires professional knowledge but also needs a lot of experiments, which limits the customization of neural networks for varied devices and application scenarios. In order to reduce the human intervention in designing and optimizing the neural network structure, multi-objective neural architecture search methods that can automatically search for neural networks featured with high accuracy and can satisfy certain hardware performance requirements are proposed. However, the current methods commonly set accuracy and inference latency as the performance indicator during the search process, and sample numerous network structures to obtain the required neural network. Lacking regulation to the search direction with the search objectives will generate a large number of useless networks during the search process, which influences the search efficiency to a great extent. Therefore, in this paper, an efficient resource-aware search method is proposed. Firstly, the network inference consumption profiling model for any specific device is established, and it can help us directly obtain the resource consumption of each operation in the network structure and the inference latency of the entire sampled network. Next, on the basis of the Bayesian search, a resource-aware Pareto Bayesian search is proposed. Accuracy and inference latency are set as the constraints to regulate the search direction. With a clearer search direction, the overall search efficiency will be improved. Furthermore, cell-based structure and lightweight operation are applied to optimize the search space for further enhancing the search efficiency. The experimental results demonstrate that with our method, the inference latency of the searched network structure reduced 94.71% without scarifying the accuracy. At the same time, the search efficiency increased by 18.18%.

A Three-Term Gradient Descent Method with Subspace Techniques

We proposed a three-term gradient descent method that can be well applied to address the optimization problems in this article. The search direction of the obtained method is generated in a specific subspace. Specifically, a quadratic approximation model is applied in the process of generating the search direction. In order to reduce the amount of calculation and make the best use of the existing information, the subspace was made up of the gradient of the current and prior iteration point and the previous search direction. By using the subspace-based optimization technology, the global convergence result is established under Wolfe line search. The results of numerical experiments show that the new method is effective and robust.

Analysis of the behavior of search engine optimization algorithms on transient modes of non-stationary simulation processes

The article deals with the analysis of the behavior of controlled simulation models for solving the choice of extreme values of the functional, which it determines on the basis of the average integral estimate. It is assumed that the search engine optimization algorithm is directly included in the model. Of interest is the problem of estimating the duration of the control interval, i.e. system simulation time with different parameters to select the search direction. The smaller the control interval, the lower the accuracy of the estimates of the functional and, accordingly, the lower the probability of choosing the correct search direction. However, with a general limitation on the simulation time, the search algorithm performs a larger number of steps, which increases the rate of convergence to the extreme value. Thus, the choice of the duration of the control interval raises a question. The aim of the work is to build a model of a controlled process, i.e. the process of changing the controlled parameters, to estimate the rate of convergence of the optimization algorithm depending on the duration of the control interval. The analysis of the convergence of the optimization process directly on the simulation model is practically impossible due to the nonstationary nature of all ongoing processes. In this regard, the article introduces a class of conditionally non-stationary Gaussian processes, on which the efficiency of a controlled simulation model is evaluated. It is assumed that a symmetric design is used to choose the direction, and all realizations of the nonstationary process at the current point have the same initial state. As a result of the analysis of such a model, analytical expressions were obtained for estimating the accuracy of the position of the extremum depending on the duration of the control interval. The results obtained make it possible, with a general limitation of the time for conducting experiments with a simulation model, to construct a sequential analysis plan, which improves the accuracy of solving the optimization problem.