AbstractThis paper considers singular systems that involve both continuous dynamics and discrete events with the coefficients being modulated by a continuous-time Markov chain. The underlying systems have two distinct characteristics. First, the systems are singular, that is, characterized by a singular coefficient matrix. Second, the Markov chain of the modulating force has a large state space. We focus on stability of such hybrid singular systems. To carry out the analysis, we use a two-time-scale formulation, which is based on the rationale that, in a large-scale system, not all components or subsystems change at the same speed. To highlight the different rates of variation, we introduce a small parameter ε>0. Under suitable conditions, the system has a limit. We then use a perturbed Lyapunov function argument to show that if the limit system is stable then so is the original system in a suitable sense for ε small enough. This result presents a perspective on reduction of complexity from a stability point of view.
On Solving MDPs With Large State Space: Exploitation of Policy Structures and Spectral Properties
