EA Multi-Model Selection for SVM

2011 ◽  
pp. 520-525
Author(s):  
Gilles Lebrun ◽  
Olivier Lezoray ◽  
Christopher Charrier ◽  
Hubert Cardot
Keyword(s):  
Model Selection ◽  
Efficient Solution ◽  
Optimization Problems ◽  
Fitness Function ◽  
Optimal Solution ◽  
Efficient Solutions ◽  
Support Vector ◽  
Natural Processes ◽  
Open Question ◽  
Global Selection

Evolutionary algorithms (EA) (Rechenberg, 1965) belong to a family of stochastic search algorithms inspired by natural evolution. In the last years, EA were used successfully to produce efficient solutions for a great number of hard optimization problems (Beasley, 1997). These algorithms operate on a population of potential solutions and apply a survival principle according to a fitness measure associated to each solution to produce better approximations of the optimal solution. At each iteration, a new set of solutions is created by selecting individuals according to their level of fitness and by applying to them several operators. These operators model natural processes, such as selection, recombination, mutation, migration, locality and neighborhood. Although the basic idea of EA is straightforward, solutions coding, size of population, fitness function and operators must be defined in compliance with the kind of problem to optimize. Multi-class problems with binary SVM (Support Vector Machine) classifiers are commonly treated as a decomposition in several binary sub-problems. An open question is how to properly choose all models for these sub-problems in order to have the lowest error rate for a specific SVM multi-class scheme. In this paper, we propose a new approach to optimize the generalization capacity of such SVM multi-class schemes. This approach consists in a global selection of models for sub-problems altogether and is denoted as multi-model selection. A multi-model selection can outperform the classical individual model selection used until now in the literature, but this type of selection defines a hard optimisation problem, because it corresponds to a search a efficient solution into a huge space. Therefore, we propose an adapted EA to achieve that multi-model selection by defining specific fitness function and recombination operator.

A Novel Approach to Designing Surrogate-assisted Genetic Algorithms by Combining Efficient Learning of Walsh Coefficients and Dependencies

2021 ◽  
Vol 1 (2) ◽  
pp. 1-23
Author(s):  
Arkadiy Dushatskiy ◽  
Tanja Alderliesten ◽  
Peter A. N. Bosman
Keyword(s):  
Boolean Functions ◽  
Surrogate Model ◽  
Optimization Problems ◽  
State Of The Art ◽  
Fitness Function ◽  
Optimal Solution ◽  
Problem Structure ◽  
Novel Approach ◽  
Efficient Learning ◽  
Nk Landscapes

Surrogate-assisted evolutionary algorithms have the potential to be of high value for real-world optimization problems when fitness evaluations are expensive, limiting the number of evaluations that can be performed. In this article, we consider the domain of pseudo-Boolean functions in a black-box setting. Moreover, instead of using a surrogate model as an approximation of a fitness function, we propose to precisely learn the coefficients of the Walsh decomposition of a fitness function and use the Walsh decomposition as a surrogate. If the coefficients are learned correctly, then the Walsh decomposition values perfectly match with the fitness function, and, thus, the optimal solution to the problem can be found by optimizing the surrogate without any additional evaluations of the original fitness function. It is known that the Walsh coefficients can be efficiently learned for pseudo-Boolean functions with k -bounded epistasis and known problem structure. We propose to learn dependencies between variables first and, therefore, substantially reduce the number of Walsh coefficients to be calculated. After the accurate Walsh decomposition is obtained, the surrogate model is optimized using GOMEA, which is considered to be a state-of-the-art binary optimization algorithm. We compare the proposed approach with standard GOMEA and two other Walsh decomposition-based algorithms. The benchmark functions in the experiments are well-known trap functions, NK-landscapes, MaxCut, and MAX3SAT problems. The experimental results demonstrate that the proposed approach is scalable at the supposed complexity of O (ℓ log ℓ) function evaluations when the number of subfunctions is O (ℓ) and all subfunctions are k -bounded, outperforming all considered algorithms.

scholarly journals Support Vector Ordinal Regression

2007 ◽  
Vol 19 (3) ◽  
pp. 792-815 ◽  
Author(s):  
Wei Chu ◽  
S. Sathiya Keerthi
Keyword(s):  
Optimization Problems ◽  
Optimal Solution ◽  
Quadratic Function ◽  
Ordinal Regression ◽  
Support Vector ◽  
Data Sets ◽  
Sequential Minimal Optimization ◽  
Real World Data ◽  
Training Samples ◽  
Multiple Thresholds

In this letter, we propose two new support vector approaches for ordinal regression, which optimize multiple thresholds to define parallel discriminant hyperplanes for the ordinal scales. Both approaches guarantee that the thresholds are properly ordered at the optimal solution. The size of these optimization problems is linear in the number of training samples. The sequential minimal optimization algorithm is adapted for the resulting optimization problems; it is extremely easy to implement and scales efficiently as a quadratic function of the number of examples. The results of numerical experiments on some benchmark and real-world data sets, including applications of ordinal regression to information retrieval, verify the usefulness of these approaches.

scholarly journals A certified RB method for PDE-constrained parametric optimization problems

2019 ◽  
Vol 10 (1) ◽  
pp. 123-152
Author(s):  
Andrea Manzoni ◽  
Stefano Pagani
Keyword(s):  
Optimal Control ◽  
Efficient Solution ◽  
Parametric Optimization ◽  
Optimization Problems ◽  
A Posteriori Error Estimates ◽  
Optimal Solution ◽  
Posteriori Error ◽  
Elliptic Pdes ◽  
Reduced Basis ◽  
Cost Functionals

Abstract We present a certified reduced basis (RB) framework for the efficient solution of PDE-constrained parametric optimization problems. We consider optimization problems (such as optimal control and optimal design) governed by elliptic PDEs and involving possibly non-convex cost functionals, assuming that the control functions are described in terms of a parameter vector. At each optimization step, the high-fidelity approximation of state and adjoint problems is replaced by a certified RB approximation, thus yielding a very efficient solution through an “optimize-then-reduce” approach. We develop a posteriori error estimates for the solutions of state and adjoint problems, the cost functional, its gradient and the optimal solution. We confirm our theoretical results in the case of optimal control/design problems dealing with potential and thermal flows.

scholarly journals Generate random rectangular maps

2018 ◽  
Author(s):  
I. V. Kozin ◽  
S. E. Batovskiy
Keyword(s):  
Optimization Problems ◽  
Optimal Solution ◽  
Approximate Solutions ◽  
Test Problems ◽  
Complex Shapes ◽  
Applied Optimization ◽  
Open Question ◽  
Complex Algorithm ◽  
Guillotine Cutting

It is known that a large number of applied optimization problems can’t be exactly solved nowadays, because their computational complexity is related to the NP-hard class. In many cases metaheuristics of various types are used to search for approximate solutions, but the choice of the concrete metaheuristic has open question of the quality of the chosen method. There are several possible solutions to this problem, one of which is the verification of metaheuristic algorithms using examples from known test libraries with known records. Another approach to solving the problem of evaluating the quality of algorithms is to compare the "new" algorithm with other algorithms, the work of which has already been investigated. The construction a generator of random problems with a known optimal solution can solve the problem of obtaining "average" estimates of the accuracy for used algorithm in comparison with other methods. The article considers the construction of generators of random non-waste maps of rec-tangular cutting with restrictions on the rectangles of limited sizes. The existence of sets of such cards forms the basis of test problems for checking the quality of approximate algorithms for searching for optimal solution. Rectangular cutting, which is considered in the article, is also the basis for building cuts using more complex shapes. As the simplest method of generating random rectangular non-waste maps, considered a method that uses guillotine cutting. Also, a more complex algorithm for generating a random rectangular cut is given, whose job is to generate a random dot grid and remove some random points from this grid. Much attention is paid to the implementation of the above methods, since the main purpose of the article is to simplify using of generators in practice. All the above algorithms are already used in the software system for testing evolution-aryfragmentary algorithms for various classes of optimization problems on the graphs

scholarly journals An Efficient Fitness Function in Genetic Algorithm Classifier for Landuse Recognition on Satellite Images

2014 ◽  
Vol 2014 ◽  
pp. 1-12 ◽  
Author(s):  
Ming-Der Yang ◽  
Yeh-Fen Yang ◽  
Tung-Ching Su ◽  
Kai-Siang Huang
Keyword(s):  
Genetic Algorithm ◽  
Classification Accuracy ◽  
Satellite Images ◽  
Optimization Problems ◽  
Fitness Function ◽  
Satellite Image ◽  
Optimal Solution ◽  
Unsupervised Classification ◽  
Shihmen Reservoir ◽  
Spot 5

Genetic algorithm (GA) is designed to search the optimal solution via weeding out the worse gene strings based on a fitness function. GA had demonstrated effectiveness in solving the problems of unsupervised image classification, one of the optimization problems in a large domain. Many indices or hybrid algorithms as a fitness function in a GA classifier are built to improve the classification accuracy. This paper proposes a new index, DBFCMI, by integrating two common indices, DBI and FCMI, in a GA classifier to improve the accuracy and robustness of classification. For the purpose of testing and verifying DBFCMI, well-known indices such as DBI, FCMI, and PASI are employed as well for comparison. A SPOT-5 satellite image in a partial watershed of Shihmen reservoir is adopted as the examined material for landuse classification. As a result, DBFCMI acquires higher overall accuracy and robustness than the rest indices in unsupervised classification.

Cuckoo Search Algorithm for Hydrothermal Scheduling Problem

2016 ◽  
pp. 367-400 ◽  
Author(s):  
Thang Trung Nguyen ◽  
Dieu Ngoc Vo
Keyword(s):  
Optimization Problems ◽  
Search Algorithm ◽  
Fitness Function ◽  
Optimal Solution ◽  
Cuckoo Search ◽  
Cuckoo Search Algorithm ◽  
Scheduling Problems ◽  
Short Term ◽  
Step Size ◽  
Hydrothermal Scheduling

This chapter proposes a Cuckoo Search Algorithm (CSA) and a Modified Cuckoo Search Algorithm (MCSA) for solving short-term hydrothermal scheduling (ST-HTS) problem. The CSA method is a new meta-heuristic algorithm inspired from the obligate brood parasitism of some cuckoo species by laying their eggs in the nests of other host birds of other species for solving optimization problems. In the MCSA method, the eggs are first classified into two groups in which ones with low fitness function are put in top group whereas others with higher fitness function are put in abandoned group. In addition, an updated step size in the MCSA changes and tends to decrease as the iteration increases leading to near global optimal solution. The robustness and effectiveness of the CSA and MCSA are tested on several systems with different objective functions of thermal units. The results obtained by the CSA and MCSA are analyzed and compared have shown that the two methods are favorable for solving short-term hydrothermal scheduling problems.

scholarly journals Masi Entropy for Satellite Color Image Segmentation Using Tournament-Based Lévy Multiverse Optimization Algorithm

2019 ◽  
Vol 11 (8) ◽  
pp. 942 ◽  
Author(s):  
Jia ◽  
Peng ◽  
Song ◽  
Oliva ◽  
Lang ◽  
...  
Keyword(s):  
Image Segmentation ◽  
Optimization Algorithm ◽  
Optimization Problems ◽  
Color Image ◽  
Fitness Function ◽  
Similarity Index ◽  
Optimal Solution ◽  
Population Diversity ◽  
Color Image Segmentation ◽  
Masi Entropy

A novel multilevel threshold segmentation method for color satellite images based on Masi entropy is proposed in this paper. Lévy multiverse optimization algorithm (LMVO) has a strong advantage over the traditional multiverse optimization algorithm (MVO) in finding the optimal solution for the segmentation in the three channels of an RGB image. As the work advancement introduces a Lévy multiverse optimization algorithm which uses tournament selection instead of roulette wheel selection, and updates some formulas in the algorithm with mutation factor. Then, the proposal is called TLMVO, and another advantage is that the population diversity of the algorithm in the latest iterations is maintained. The Masi entropy is used as an application and combined with the improved TLMVO algorithm for satellite color image segmentation. Masi entropy combines the additivity of Renyi entropy and the non-extensibility of Tsallis entropy. By increasing the number of thesholds, the quality of segmenttion becomes better, then the dimensionality of the problem also increases. Fitness function value, average CPU running time, Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index (SSIM) and Feature Similarity Index (FSIM) were used to evaluate the segmentation results. Further statistical evaluation was given by Wilcoxon's rank sum test and Friedman test. The experimental results show that the TLMVO algorithm has wide adaptability to high-dimensional optimization problems, and has obvious advantages in objective function value, image quality detection, convergence performance and robustness.

Resampling Methods for Meta-Model Validation with Recommendations for Evolutionary Computation

2012 ◽  
Vol 20 (2) ◽  
pp. 249-275 ◽  
Author(s):  
B. Bischl ◽  
O. Mersmann ◽  
H. Trautmann ◽  
C. Weihs
Keyword(s):  
Model Validation ◽  
Optimization Problems ◽  
Fitness Function ◽  
Support Vector ◽  
Modeling Framework ◽  
Meta Modeling ◽  
Expensive Optimization Problems ◽  
And Performance ◽  
Classical Linear Regression ◽  
Expensive Optimization

Meta-modeling has become a crucial tool in solving expensive optimization problems. Much of the work in the past has focused on finding a good regression method to model the fitness function. Examples include classical linear regression, splines, neural networks, Kriging and support vector regression. This paper specifically draws attention to the fact that assessing model accuracy is a crucial aspect in the meta-modeling framework. Resampling strategies such as cross-validation, subsampling, bootstrapping, and nested resampling are prominent methods for model validation and are systematically discussed with respect to possible pitfalls, shortcomings, and specific features. A survey of meta-modeling techniques within evolutionary optimization is provided. In addition, practical examples illustrating some of the pitfalls associated with model selection and performance assessment are presented. Finally, recommendations are given for choosing a model validation technique for a particular setting.

High equilibrium optimizer for global optimization

2020 ◽  
pp. 1-12
Author(s):  
Heming Jia ◽  
Xiaoxu Peng
Keyword(s):  
Engineering Design ◽  
Heuristic Algorithms ◽  
Optimization Problems ◽  
Fitness Function ◽  
Algorithm Design ◽  
Optimal Solution ◽  
Rolling Bearing ◽  
Experimental Results ◽  
Thermal Exchange ◽  
Engineering Design Problems

With the advent of the information age, people have higher requirements for basic algorithms. Meta-heuristic algorithms have received wide attention as a high-level strategy to study and generate fully optimized solutions to data-driven optimization problems. Using the advantage of equilibrium optimizer (EO) with better balance mode, combined with the strategy of memetic algorithm, different proportion of temperature is introduced in different stages. That is, EO and thermal exchange optimization (TEO) are fused to obtain a new highly balanced optimizer (HEO). While keeping the guiding strategy and memory mode unchanged of EO, the accuracy of optimization is greatly improved. 14 well-known benchmark functions and 7 selective algorithms were used for HEO evaluation comparison experiments. On the basis of the fitness function curve, the optimal solution and other experimental data are tested statistically. The experimental results show that the improved algorithm has high accuracy and stability, but at the cost of running a little more time. Application testing of complex engineering problems is also one of the main purposes of algorithm design. In this paper, three typical engineering design problems (three truss, welded beam and rolling bearing design) are tested and the experimental results show that this algorithm has certain competitiveness and superiority in classical engineering design.

∊-STRICTLY EFFICIENT SOLUTIONS OF VECTOR OPTIMIZATION PROBLEMS WITH SET-VALUED MAPS

2007 ◽  
Vol 24 (06) ◽  
pp. 841-854 ◽  
Author(s):  
TAIYONG LI ◽  
YIHONG XU ◽  
CHUANXI ZHU
Keyword(s):  
Saddle Point ◽  
Vector Optimization ◽  
Efficient Solution ◽  
Optimization Problems ◽  
Efficient Solutions ◽  
Vector Optimization Problems ◽  
Lagrangian Multiplier ◽  
Multiplier Theorem ◽  
Lagrangian Multiplier Theorem

In this paper, the notion of ∊-strictly efficient solution for vector optimization with set-valued maps is introduced. Under the assumption of the ic-cone-convexlikeness for set-valued maps, the scalarization theorem, ∊-Lagrangian multiplier theorem, ∊-saddle point theorems and ∊-duality assertions are established for ∊-strictly efficient solution.

Sign in / Sign up

Export Citation Format

Share Document