The main aim of this paper is to study the dynamics of a recurrent neural networks with different input currents in terms of asymptotic point. Under certain circumstances, we studied the existence, the uniqueness of bounded solutions and their homoclinic and heteroclinic motions of the considered system with rectangular currents input. Moreover, we studied the unpredictable behavior of the continuous high-order recurrent neural networks and the discrete high-order recurrent neural networks. Our method was primarily based on Banach’s fixed-point theorem, topology of uniform convergence on compact sets and Gronwall inequality. For the demonstration of theoretical results, we give examples and their numerical simulations.