Metaheuristics are nondeterministic optimization algorithms used to solve complex problems for which classic approaches are unsuitable. Despite their effectiveness, metaheuristics require considerable computational power and cannot easily be used in time critical applications. Fortunately, those algorithms are intrinsically parallel and have been implemented on shared memory systems and more recently on graphics processing units (GPUs). In this paper, we present highly efficient parallel implementations of the particle swarm optimization (PSO), the genetic algorithm (GA) and the simulated annealing (SA) algorithm on GPU using CUDA. Our approach exploits the parallelism at the solution level, follows an island model and allows for speedup up to 346× for different benchmark functions. Most importantly, we also present a strategy that uses the generalized island model to integrate multiple metaheuristics into a parallel hybrid solution adapted to the GPU. Our proposed solution uses OpenMP to heavily exploit the concurrent kernel execution feature of recent NVIDIA GPUs, allowing for the parallel execution of the different metaheuristics in an asynchronous manner. Asynchronous hybrid metaheuristics has been developed for multicore CPU, but never for GPU. The speedup offered by the GPU is far superior and key to the optimization of solutions to complex engineering problems.
Parallel Hybrid Metaheuristics with Distributed Intensification and Diversification for Large-scale Optimization in Big Data Statistical Analysis