To comprehensively analyze the modal characteristics of the planetary reducer with small tooth number difference, based on shaft–bearing–gear–shell coupling, a finite element modal analysis model was established in ABAQUS. The teeth meshing sites were constrained by binding, bearings were simulated by spring elements, and then the natural frequencies and corresponding vibration modes of the reducer were obtained by applying the Lanczos method. Further, a hammering modal experiment on the reducer was carried out utilizing LMS Test.Lab. The modal data were analyzed using a modal identification method, and the modal frequencies and damping ratios were achieved, also the experimental modal parameters were validated according to the modal assurance criterion. The research results indicate that the lowest-order natural frequency of the reducer is 148.53 Hz, which is much higher than the rotation frequency of the eccentric shaft, double gear, and output gear. Also, the two-stage gear mesh frequencies are away from the natural frequencies, therefore the reducer under normal operating conditions will not cause coupling resonance. This research provides a theoretical basis and experimental reference for the dynamic structure optimization of the planetary reducer.