This paper presents a GPU-based parallel Population Based Incremental Learning (PBIL) algorithm with a local search on bound constrained optimization problems. The genotype of an entire population is evolved in PBIL, which was derived from Genetic Algorithms. Graphics Processing Units (GPU) is an emerging technology for desktop parallel computing. In this research, the classical PBIL is adapted in the data-parallel GPU computing platform. The global optimal search of the PBIL is enhanced by a local Pattern Search method. The hybrid PBIL method is implemented in the GPU environment, and compared to a similar implementation in the common computing environment with a Central Processing Unit (CPU). Computational results indicate that GPU-accelerated PBIL method is effective and faster than the corresponding CPU implementation.