Carbon aerogels are a promising candidate for vibration insulation due to their three-dimensional networked structures interconnected with carbon nanoparticles. However, the effect of adding carbon aerogels to polymer-based composites on their dynamic properties remains unclear. In this study, an epoxy polymer matrix was modified with carbon aerogels, and this modified matrix was used to manufacture nanocomposite plates and carbon fiber-reinforced polymer composite laminates to investigate its dynamic properties. Force vibration tests were performed on cantilever beams of the composite beams. The frequency responses of the composite beams were measured experimentally and analytically; the half-power method was used to calculate the damping ratio for each vibration mode. According to the experimental results, the presence of carbon aerogel in the nanocomposites and laminates steadily increased the natural frequencies. Differences within 10% of the natural frequencies were obtained between the experimental and numerically. Furthermore, the damping ratios of the nanocomposite and laminate beams increased significantly with the increase in aerogel loading. For a nanocomposite with 0.3 wt% aerogel, a damping ratio approximately 44% greater than that of unmodified nanocomposite was obtained. The maximum damping ratio was 4.682% for the laminate with 0.5 wt% aerogel—an 88% increase compared with the unmodified laminate.