This article focuses on the output-only recursive identification of time-varying systems by using parametric time-domain methods. A novel multivariate recursive Bayesian linear regression method is proposed based on the vector time-dependent autoregressive moving average model. The standard setup of univariate batch Bayesian linear regression is first extended to the multivariate case for multiple response signal modeling and further extended to the recursive case to meet the output-only recursive identification requirement of practical systems. A sliding window mechanism is finally applied to deemphasize data from the remote past and fix the computational complexity for each consecutive update, allowing the proposed method to be capable of tracking the time-varying dynamics online. The proposed multivariate recursive Bayesian linear regression method is first validated by a simple numerical system and subsequently applied to identify two mechanical systems with typical time-varying dynamics. Comparative identification results via Monte Carlo tests numerically and experimentally demonstrate the superior achievable accuracy and time-varying tracking capability of the proposed method.