In recent decades, global climate change, especially human activities, has led to profound changes in the hydrological cycle and hydrological processes in watersheds. Taking the Yue River watershed in the Qinling Mountains in China as the study area, the Mann–Kendall test and Pettitt mutation test method were used to analyze the various characteristics of hydrological and climatic elements from 1960 to 2018. Then, the elastic coefficient method based on the Budyko framework was used to estimate the elastic coefficient of runoff change on each influencing factor. The results showed that the annual runoff decreased at a rate of 0.038 × 108 m3/a (
P
>
0.05
), and a significant abrupt change occurred in 1990. The annual precipitation and potential evapotranspiration (ET0) increased and decreased, with change rates of 0.614 mm/a and −0.811 mm/a (
P
>
0.05
), respectively. The elasticity coefficients of precipitation, ET0, and the underlying surface were 1.95, −0.95, and −0.85, respectively, indicating that annual runoff was most sensitive to the change in precipitation, followed by the change in ET0, and had the lowest sensitivity to the change in the underlying surface. Underlying surface change is the main factor of runoff decrease; the contribution is 89.07%. The total contribution of climate change to runoff change is 10.93%, in which the contributions of precipitation and ET0 are 17.59% and −6.66%, respectively. The NDVI reflecting underlying surface change has been increasing since 1990, which is an important reason for the runoff decrease.