Generalised Pattern Search Based on Covariance Matrix Diagonalisation

Pattern Search is a family of gradient-free direct search methods for numerical optimisation problems. The characterising feature of pattern search methods is the use of multiple directions spanning the problem domain to sample new candidate solutions. These directions compose a matrix of potential search moves, that is the pattern. Although some fundamental studies theoretically indicate that various directions can be used, the selection of the search directions remains an unaddressed problem. The present article proposes a procedure for selecting the directions that guarantee high convergence/high performance of pattern search. The proposed procedure consists of a fitness landscape analysis to characterise the geometry of the problem by sampling points and selecting those whose objective function values are below a threshold. The eigenvectors of the covariance matrix of this distribution are then used as search directions for the pattern search. Numerical results show that the proposed method systematically outperforms its standard counterpart and is competitive with modern complex direct search and metaheuristic methods.