In this paper, a novel efficient optimization method based on reinforcement learning automata (RLA) for optimum parameters setting of conventional proportional-integral-derivative (PID) controller for AVR system of power synchronous generator is proposed. The proposed method is Combinatorial Discrete and Continuous Action Reinforcement Learning Automata (CDCARLA) which is able to explore and learn to improve control performance without the knowledge of the analytical system model. This paper demonstrates the full details of the CDCARLA technique and compares its performance with Particle Swarm Optimization (PSO) as an efficient evolutionary optimization method. The proposed method has been applied to PID controller design. The simulation results show the superior efficiency and robustness of the proposed method.