Ultrasonic echo intensity (EI), an easy-to-use measure of intramuscular fat and fibrous tissues, is known to increase with aging. However, age-related changes in EI have not been examined in a longitudinal design. The objective of this study was to investigate 4-yr longitudinal changes in the EI of the quadriceps femoris in older adults, based on difference in physical activity (PA). This study included 131 community-dwelling older adults with a mean age of 72.9 ± 5.2 yr. Subcutaneous fat thickness (FT), muscle thickness (MT), and EI of the quadriceps femoris were measured by ultrasound. Isometric knee extensor strength was also measured. PA was assessed using a questionnaire at baseline, and participants were classified into the high or low PA groups. In 4 yr, a significant decrease in FT, MT, and strength was observed in both groups ( P < 0.05), whereas a significant decrease in EI was observed only in the high PA group ( P < 0.05). Multiple linear regression analyses revealed that the difference in PA was a significant predictor of 4-yr changes in MT (β = 0.189, P = 0.031) and EI (β = −3.145, P = 0.045) but not in the body mass index, FT, or strength adjusted for potential confounders. The present findings suggest that greater PA has a positive effect on longitudinal changes in the MT and EI of the quadriceps femoris in older adults. In addition, greater PA may contribute to a future decrease in EI, and an increase in EI may not occur in 4 yr, even in older adults with lesser PA. NEW & NOTEWORTHY Our results suggest that greater physical activity (PA) may mitigate future changes in muscle thickness and echo intensity (EI). A decrease in EI over 4 yr was observed in older adults with greater PA, and an increase in EI was not observed, even in older adults with smaller PA. Several cross-sectional studies demonstrated an increase in EI with aging. Additionally, the results of our longitudinal study suggest that an age-related increase in EI may be moderated after the old-age period.