The coupling interaction between nonlinear solitary waves in one-dimensional granular chains and damaged composite material plates is considered. Based on Hertz contact law and meso-mechanical model of stiffness reduction of composite material plates when the fiber breakage is the main damage mode, the coupled differential equations of particle chains and damaged composite material plates are derived. By solving the differential equations with Runge–Kutta method to get the velocity and displacement curves of particles and analyzing the delays and amplitude ratios of reflected waves, it is found that the damage quantity, fiber volume fraction, and thickness of damaged composite material plates as well as gravity have an effect on solitary waves. The preliminary research results provide a theoretical basis for nondestructive testing of damaged composite material plates by using solitary waves.