In this article, a composite controller is proposed for the manipulator with the flexible joint and link under uncertainties and time-varying disturbances. The dynamic of the system is developed by the Euler–Lagrange and assumed mode method, which is a nonlinear, strong coupling, and underacted system. Therefore, based on the singular perturbation theory, the dynamic is decomposed into a slow and fast subsystem. For the slow dynamic, a novel adaptive-gain super-twisting sliding mode controller is designed to guarantee joint tracking under the uncertainties and disturbances. For the fast dynamics, adaptive dynamic programming is used to deal with the uncertainty. The simulation result shows that the proposed composite controller can effectively track the trajectory and suppress the vibration simultaneously.