scholarly journals A First-Order Optimization Algorithm for Statistical Learning with Hierarchical Sparsity Structure

2021 ◽  
Author(s):  
Dewei Zhang ◽  
Yin Liu ◽  
Sam Davanloo Tajbakhsh
Keyword(s):  
Numerical Simulation ◽  
Statistical Learning ◽  
Optimization Algorithm ◽  
Large Scale ◽  
Optimal Solution ◽  
Linear Convergence ◽  
Simulation Studies ◽  
Speed Of Convergence ◽  
Proximal Operator ◽  
Sparsity Structure

In many statistical learning problems, it is desired that the optimal solution conform to an a priori known sparsity structure represented by a directed acyclic graph. Inducing such structures by means of convex regularizers requires nonsmooth penalty functions that exploit group overlapping. Our study focuses on evaluating the proximal operator of the latent overlapping group lasso developed by Jacob et al. in 2009. We implemented an alternating direction method of multiplier with a sharing scheme to solve large-scale instances of the underlying optimization problem efficiently. In the absence of strong convexity, global linear convergence of the algorithm is established using the error bound theory. More specifically, the paper contributes to establishing primal and dual error bounds when the nonsmooth component in the objective function does not have a polyhedral epigraph. We also investigate the effect of the graph structure on the speed of convergence of the algorithm. Detailed numerical simulation studies over different graph structures supporting the proposed algorithm and two applications in learning are provided. Summary of Contribution: The paper proposes a computationally efficient optimization algorithm to evaluate the proximal operator of a nonsmooth hierarchical sparsity-inducing regularizer and establishes its convergence properties. The computationally intensive subproblem of the proposed algorithm can be fully parallelized, which allows solving large-scale instances of the underlying problem. Comprehensive numerical simulation studies benchmarking the proposed algorithm against five other methods on the speed of convergence to optimality are provided. Furthermore, performance of the algorithm is demonstrated on two statistical learning applications related to topic modeling and breast cancer classification. The code along with the simulation studies and benchmarks are available on the corresponding author’s GitHub website for evaluation and future use.

scholarly journals Domain decomposition for entropy regularized optimal transport

2021 ◽  
Author(s):  
Mauro Bonafini ◽  
Bernhard Schmitzer
Keyword(s):  
Domain Decomposition ◽  
Large Scale ◽  
Optimal Transport ◽  
Optimal Solution ◽  
Linear Convergence ◽  
Efficient Implementation ◽  
Computationally Efficient ◽  
Solution Quality ◽  
Leibler Divergence ◽  
Transport Problems

AbstractWe study Benamou’s domain decomposition algorithm for optimal transport in the entropy regularized setting. The key observation is that the regularized variant converges to the globally optimal solution under very mild assumptions. We prove linear convergence of the algorithm with respect to the Kullback–Leibler divergence and illustrate the (potentially very slow) rates with numerical examples. On problems with sufficient geometric structure (such as Wasserstein distances between images) we expect much faster convergence. We then discuss important aspects of a computationally efficient implementation, such as adaptive sparsity, a coarse-to-fine scheme and parallelization, paving the way to numerically solving large-scale optimal transport problems. We demonstrate efficient numerical performance for computing the Wasserstein-2 distance between 2D images and observe that, even without parallelization, domain decomposition compares favorably to applying a single efficient implementation of the Sinkhorn algorithm in terms of runtime, memory and solution quality.

scholarly journals Optimal Scheduling of Non-Convex Cogeneration Units Using Exponentially Varying Whale Optimization Algorithm

Energies ◽  
2021 ◽  
Vol 14 (4) ◽  
pp. 1008
Author(s):  
Vinay Kumar Jadoun ◽  
G. Rahul Prashanth ◽  
Siddharth Suhas Joshi ◽  
Anshul Agarwal ◽  
Hasmat Malik ◽  
...  
Keyword(s):  
Optimization Algorithm ◽  
Large Scale ◽  
Optimization Technique ◽  
Optimal Solution ◽  
Whale Optimization Algorithm ◽  
Small Scale ◽  
Optimal Output ◽  
Whale Optimization ◽  
Fuel Costs ◽  
Fine Tune

This paper proposes an Exponentially Varying Whale Optimization Algorithm (EVWOA) to solve the single-objective non-convex Cogeneration Units problem. This problem seeks to evaluate the optimal output of the generator unit to minimize a CHP system’s fuel costs. The nonlinear and non-convex characteristics of the objective function demands a powerful optimization technique. The traditional Whale Optimization Algorithm (WOA) is improved by incorporating four different acceleration functions to fine-tune its performance during exploration and exploitation phases. Among the four variants of the proposed WOA, the emphasis is laid on the EVWOA which uses the exponentially varying acceleration function (EVAF). The proposed EVWOA is tested on six different small-scale to large-scale systems. The results obtained for these six test systems, followed by a statistical study highlight the supremacy of EVWOA for finding the best optimal solution and the convergence traits.

AO-BBO: A Novel Optimization Algorithm and Its Application in Plant Drug Extraction

2019 ◽  
Vol 19 (2) ◽  
pp. 139-145 ◽  
Author(s):  
Bote Lv ◽  
Juan Chen ◽  
Boyan Liu ◽  
Cuiying Dong
Keyword(s):  
Optimization Algorithm ◽  
Learning Strategy ◽  
Optimal Solution ◽  
Population Diversity ◽  
Global Optimal Solution ◽  
Plant Medicine ◽  
Suitability Index ◽  
Sensing Model ◽  
Local Optimal Solution ◽  
Global Optimal

<P>Introduction: It is well-known that the biogeography-based optimization (BBO) algorithm lacks searching power in some circumstances. </P><P> Material & Methods: In order to address this issue, an adaptive opposition-based biogeography-based optimization algorithm (AO-BBO) is proposed. Based on the BBO algorithm and opposite learning strategy, this algorithm chooses different opposite learning probabilities for each individual according to the habitat suitability index (HSI), so as to avoid elite individuals from returning to local optimal solution. Meanwhile, the proposed method is tested in 9 benchmark functions respectively. </P><P> Result: The results show that the improved AO-BBO algorithm can improve the population diversity better and enhance the search ability of the global optimal solution. The global exploration capability, convergence rate and convergence accuracy have been significantly improved. Eventually, the algorithm is applied to the parameter optimization of soft-sensing model in plant medicine extraction rate. Conclusion: The simulation results show that the model obtained by this method has higher prediction accuracy and generalization ability.</P>

Research on Power Load Forecasting Model Based on Hybrid Algorithm Optimizing BP Neural Network

2014 ◽  
Vol 8 (1) ◽  
pp. 723-728 ◽  
Author(s):  
Chenhao Niu ◽  
Xiaomin Xu ◽  
Yan Lu ◽  
Mian Xing
Keyword(s):  
Neural Network ◽  
Optimization Algorithm ◽  
Load Forecasting ◽  
Optimal Solution ◽  
Electric Power System ◽  
Local Optimum ◽  
Good Prediction ◽  
Ant Colony Optimization Algorithm ◽  
The Neural Network ◽  
Power Load

Short time load forecasting is essential for daily planning and operation of electric power system. It is the important basis for economic dispatching, scheduling and safe operation. Neural network, which has strong nonlinear fitting capability, is widely used in the load forecasting and obtains good prediction effect in nonlinear chaotic time series forecasting. However, the neural network is easy to fall in local optimum, unable to find the global optimal solution. This paper will integrate the traditional optimization algorithm and propose the hybrid intelligent optimization algorithm based on particle swarm optimization algorithm and ant colony optimization algorithm (ACO-PSO) to improve the generalization of the neural network. In the empirical analysis, we select electricity consumption in a certain area for validation. Compared with the traditional BP neutral network and statistical methods, the experimental results demonstrate that the performance of the improved model with more precise results and stronger generalization ability is much better than the traditional methods.

Multi Disease-Prediction Framework Using Hybrid Deep Learning: An Optimal Prediction Model (Preprint)

2020 ◽  
Author(s):  
Anusha Ampavathi ◽  
Vijaya Saradhi T
Keyword(s):  
Feature Extraction ◽  
Big Data ◽  
Deep Learning ◽  
Weight Function ◽  
Optimization Algorithm ◽  
Large Scale ◽  
Heuristic Algorithms ◽  
Disease Prediction ◽  
Health Care Decisions ◽  
Proposed Model

UNSTRUCTURED Big data and its approaches are generally helpful for healthcare and biomedical sectors for predicting the disease. For trivial symptoms, the difficulty is to meet the doctors at any time in the hospital. Thus, big data provides essential data regarding the diseases on the basis of the patient’s symptoms. For several medical organizations, disease prediction is important for making the best feasible health care decisions. Conversely, the conventional medical care model offers input as structured that requires more accurate and consistent prediction. This paper is planned to develop the multi-disease prediction using the improvised deep learning concept. Here, the different datasets pertain to “Diabetes, Hepatitis, lung cancer, liver tumor, heart disease, Parkinson’s disease, and Alzheimer’s disease”, from the benchmark UCI repository is gathered for conducting the experiment. The proposed model involves three phases (a) Data normalization (b) Weighted normalized feature extraction, and (c) prediction. Initially, the dataset is normalized in order to make the attribute's range at a certain level. Further, weighted feature extraction is performed, in which a weight function is multiplied with each attribute value for making large scale deviation. Here, the weight function is optimized using the combination of two meta-heuristic algorithms termed as Jaya Algorithm-based Multi-Verse Optimization algorithm (JA-MVO). The optimally extracted features are subjected to the hybrid deep learning algorithms like “Deep Belief Network (DBN) and Recurrent Neural Network (RNN)”. As a modification to hybrid deep learning architecture, the weight of both DBN and RNN is optimized using the same hybrid optimization algorithm. Further, the comparative evaluation of the proposed prediction over the existing models certifies its effectiveness through various performance measures.

Whale optimization algorithm based on lateral inhibition for image matching and vision-guided AUV docking

2020 ◽  
pp. 1-12
Author(s):  
Zheping Yan ◽  
Jinzhong Zhang ◽  
Jialing Tang
Keyword(s):  
Lateral Inhibition ◽  
Optimization Algorithm ◽  
Image Matching ◽  
Autonomous Underwater Vehicle ◽  
Optimal Solution ◽  
Search Space ◽  
Whale Optimization Algorithm ◽  
Inhibition Mechanism ◽  
Whale Optimization ◽  
Accuracy And Stability

The accuracy and stability of relative pose estimation of an autonomous underwater vehicle (AUV) and a target depend on whether the characteristics of the underwater image can be accurately and quickly extracted. In this paper, a whale optimization algorithm (WOA) based on lateral inhibition (LI) is proposed to solve the image matching and vision-guided AUV docking problem. The proposed method is named the LI-WOA. The WOA is motivated by the behavior of humpback whales, and it mainly imitates encircling prey, bubble-net attacking and searching for prey to obtain the globally optimal solution in the search space. The WOA not only balances exploration and exploitation but also has a faster convergence speed, higher calculation accuracy and stronger robustness than other approaches. The lateral inhibition mechanism can effectively perform image enhancement and image edge extraction to improve the accuracy and stability of image matching. The LI-WOA combines the optimization efficiency of the WOA and the matching accuracy of the LI mechanism to improve convergence accuracy and the correct matching rate. To verify its effectiveness and feasibility, the WOA is compared with other algorithms by maximizing the similarity between the original image and the template image. The experimental results show that the LI-WOA has a better average value, a higher correct rate, less execution time and stronger robustness than other algorithms. The LI-WOA is an effective and stable method for solving the image matching and vision-guided AUV docking problem.

scholarly journals Particle Swarm Optimization Combined with Inertia-Free Velocity and Direction Search

Electronics ◽  
2021 ◽  
Vol 10 (5) ◽  
pp. 597
Author(s):  
Kun Miao ◽  
Qian Feng ◽  
Wei Kuang
Keyword(s):  
Particle Swarm Optimization ◽  
Optimization Algorithm ◽  
Particle Swarm ◽  
Optimal Solution ◽  
Small Region ◽  
Benchmark Problems ◽  
Swarm Optimization ◽  
Local Searches ◽  
Engineering Design Problems ◽  
Direction Search

The particle swarm optimization algorithm (PSO) is a widely used swarm-based natural inspired optimization algorithm. However, it suffers search stagnation from being trapped into a sub-optimal solution in an optimization problem. This paper proposes a novel hybrid algorithm (SDPSO) to improve its performance on local searches. The algorithm merges two strategies, the static exploitation (SE, a velocity updating strategy considering inertia-free velocity), and the direction search (DS) of Rosenbrock method, into the original PSO. With this hybrid, on the one hand, extensive exploration is still maintained by PSO; on the other hand, the SE is responsible for locating a small region, and then the DS further intensifies the search. The SDPSO algorithm was implemented and tested on unconstrained benchmark problems (CEC2014) and some constrained engineering design problems. The performance of SDPSO is compared with that of other optimization algorithms, and the results show that SDPSO has a competitive performance.

Numerical simulation studies of Cs3Bi2I9 perovskite solar device with optimal selection of electron and hole transport layers

Optik ◽  
2021 ◽  
Vol 231 ◽  
pp. 166417 ◽  
Author(s):  
Md Tohidul Islam ◽  
Md Rafsun Jani ◽  
Kazi Md Shorowordi ◽  
Zameer Hoque ◽  
Ali Mucteba Gokcek ◽  
...  
Keyword(s):  
Numerical Simulation ◽  
Hole Transport ◽  
Optimal Selection ◽  
Simulation Studies ◽  
Hole Transport Layers ◽  
Selection Of

scholarly journals Temporal concatenation for Markov decision processes

2021 ◽  
pp. 1-28
Author(s):  
Ruiyang Song ◽  
Kuang Xu
Keyword(s):  
Markov Decision Processes ◽  
Large Scale ◽  
Optimal Solution ◽  
Upper Bounds ◽  
Black Box ◽  
Decision Processes ◽  
Optimal Solutions ◽  
Wide Range ◽  
Markov Decision ◽  
Speed Up

We propose and analyze a temporal concatenation heuristic for solving large-scale finite-horizon Markov decision processes (MDP), which divides the MDP into smaller sub-problems along the time horizon and generates an overall solution by simply concatenating the optimal solutions from these sub-problems. As a “black box” architecture, temporal concatenation works with a wide range of existing MDP algorithms. Our main results characterize the regret of temporal concatenation compared to the optimal solution. We provide upper bounds for general MDP instances, as well as a family of MDP instances in which the upper bounds are shown to be tight. Together, our results demonstrate temporal concatenation's potential of substantial speed-up at the expense of some performance degradation.

Sign in / Sign up

Export Citation Format

Share Document