Evolving Multimodal Robot Behavior via Many Stepping Stones with the Combinatorial Multi-Objective Evolutionary Algorithm

An important challenge in reinforcement learning is to solve multimodal problems, where agents have to act in qualitatively different ways depending on the circumstances. Because multimodal problems are often too difficult to solve directly, it is often helpful to define a curriculum, which is an ordered set of sub-tasks that can serve as the stepping stones for solving the overall problem. Unfortunately, choosing an effective ordering for these subtasks is difficult, and a poor ordering can reduce the performance of the learning process. Here, we provide a thorough introduction and investigation of the Combinatorial Multi-Objective Evolutionary Algorithm (CMOEA), which allows all combinations of subtasks to be explored simultaneously. We compare CMOEA against three algorithms that can similarly optimize on multiple subtasks simultaneously: NSGA-II, NSGA-III and ϵ-Lexicase Selection. The algorithms are tested on a function-optimization problem with two subtasks, a simulated multimodal robot locomotion problem with six subtasks and a simulated robot maze navigation problem where a hundred random mazes are treated as subtasks. On these problems, CMOEA either outperforms or is competitive with the controls. As a separate contribution, we show that adding a linear combination over all objectives can improve the ability of the control algorithms to solve these multimodal problems. Lastly, we show that CMOEA can leverage auxiliary objectives more effectively than the controls on the multimodal locomotion task. In general, our experiments suggest that CMOEA is a promising algorithm for solving multimodal problems.

A new method for global optimization

This paper presents a new method for global optimization. We use exact quadratic regularization for the transformation of the multimodal problems to a problem of a maximum norm vector on a convex set. Quadratic regularization often allows you to convert a multimodal problem into a unimodal problem. For this, we use the shift of the feasible region along the bisector of the positive orthant. We use only local search (primal-dual interior point method) and a dichotomy method for search of a global extremum in the multimodal problems. The comparative numerical experiments have shown that this method is very efficient and promising.

Generalized Oppositional Moth Flame Optimization with Crossover Strategy: An Approach for Medical Diagnosis

In the original Moth-Flame Optimization (MFO), the search behavior of the moth depends on the corresponding flame and the interaction between the moth and its corresponding flame, so it will get stuck in the local optimum easily when facing the multi-dimensional and high-dimensional optimization problems. Therefore, in this work, a generalized oppositional MFO with crossover strategy, named GCMFO, is presented to overcome the mentioned defects. In the proposed GCMFO, GOBL is employed to increase the population diversity and expand the search range in the initialization and iteration jump phase based on the jump rate; crisscross search (CC) is adopted to promote the exploitation and/or exploration ability of MFO. The proposed algorithm’s performance is estimated by organizing a series of experiments; firstly, the CEC2017 benchmark set is adopted to evaluate the performance of GCMFO in tackling high-dimensional and multimodal problems. Secondly, GCMFO is applied to handle multilevel thresholding image segmentation problems. At last, GCMFO is integrated into kernel extreme learning machine classifier to deal with three medical diagnosis cases, including the appendicitis diagnosis, overweight statuses diagnosis, and thyroid cancer diagnosis. Experimental results and discussions show that the proposed approach outperforms the original MFO and other state-of-the-art algorithms on both convergence speed and accuracy. It also indicates that the presented GCMFO has a promising potential for application.

A Novel Crow Swarm Optimization Algorithm (CSO) Coupling Particle Swarm Optimization (PSO) and Crow Search Algorithm (CSA)

The balance between exploitation and exploration essentially determines the performance of a population-based optimization algorithm, which is also a big challenge in algorithm design. Particle swarm optimization (PSO) has strong ability in exploitation, but is relatively weak in exploration, while crow search algorithm (CSA) is characterized by simplicity and more randomness. This study proposes a new crow swarm optimization algorithm coupling PSO and CSA, which provides the individuals the possibility of exploring the unknown regions under the guidance of another random individual. The proposed CSO algorithm is tested on several benchmark functions, including both unimodal and multimodal problems with different variable dimensions. The performance of the proposed CSO is evaluated by the optimization efficiency, the global search ability, and the robustness to parameter settings, all of which are improved to a great extent compared with either PSO and CSA, as the proposed CSO combines the advantages of PSO in exploitation and that of CSA in exploration, especially for complex high-dimensional problems.

Worldvolume approach to the tempered Lefschetz thimble method

As a solution towards the numerical sign problem, we propose a novel Hybrid Monte Carlo algorithm, in which molecular dynamics is performed on a continuum set of integration surfaces foliated by the antiholomorphic gradient flow (“the worldvolume of an integration surface”). This is an extension of the tempered Lefschetz thimble method (TLTM), and solves the sign and multimodal problems simultaneously as the original TLTM does. Furthermore, in this new algorithm, one no longer needs to compute the Jacobian of the gradient flow in generating a configuration, and only needs to evaluate its phase upon measurement. To demonstrate that this algorithm works correctly, we apply the algorithm to a chiral random matrix model, for which the complex Langevin method is known not to work.

Diversity Measures for Niching Algorithms

Multimodal problems are single objective optimisation problems with multiple local and global optima. The objective of multimodal optimisation is to locate all or most of the optima. Niching algorithms are the techniques utilised to locate these optima. A critical factor in determining the success of niching algorithms is how well the search space is covered by the candidate solutions. For niching algorithms, high diversity during the exploration phase will facilitate location and identification of many solutions while a low diversity means that the candidate solutions are clustered at optima. This paper provides a review of measures used to quantify diversity, and how they can be utilised to quantify the dispersion of both the candidate solutions and the solutions of niching algorithms (i.e., found optima). The investigated diversity measures are then used to evaluate the distribution of candidate solutions and solutions when the enhanced species-based particle swarm optimisation (ESPSO) algorithm is utilised to optimise a selected set of multimodal problems.