Controlled Discrete-Time Semi-Markov Random Evolutions and Their Applications

In this paper, we introduced controlled discrete-time semi-Markov random evolutions. These processes are random evolutions of discrete-time semi-Markov processes where we consider a control. applied to the values of random evolution. The main results concern time-rescaled weak convergence limit theorems in a Banach space of the above stochastic systems as averaging and diffusion approximation. The applications are given to the controlled additive functionals, controlled geometric Markov renewal processes, and controlled dynamical systems. We provide dynamical principles for discrete-time dynamical systems such as controlled additive functionals and controlled geometric Markov renewal processes. We also produce dynamic programming equations (Hamilton–Jacobi–Bellman equations) for the limiting processes in diffusion approximation such as controlled additive functionals, controlled geometric Markov renewal processes and controlled dynamical systems. As an example, we consider the solution of portfolio optimization problem by Merton for the limiting controlled geometric Markov renewal processes in diffusion approximation scheme. The rates of convergence in the limit theorems are also presented.

Non-Homogeneous Semi-Markov and Markov Renewal Processes and Change of Measure in Credit Risk

For a G-inhomogeneous semi-Markov chain and G-inhomogeneous Markov renewal processes, we study the change from real probability measure into a forward probability measure. We find the values of risky bonds using the forward probabilities that the bond will not default up to maturity time for both processes. It is established in the form of a theorem that the forward probability measure does not alter the semi Markov structure. In addition, foundation of a G-inhohomogeneous Markov renewal process is done and a theorem is provided where it is proved that the Markov renewal process is maintained under the forward probability measure. We show that for an inhomogeneous semi-Markov there are martingales that characterize it. We show that the same is true for a Markov renewal processes. We discuss in depth the calibration of the G-inhomogeneous semi-Markov chain model and propose an algorithm for it. We conclude with an application for risky bonds.

Fano Factor: A Potentially Useful Information

The Fano factor, defined as the variance-to-mean ratio of spike counts in a time window, is often used to measure the variability of neuronal spike trains. However, despite its transparent definition, careless use of the Fano factor can easily lead to distorted or even wrong results. One of the problems is the unclear dependence of the Fano factor on the spiking rate, which is often neglected or handled insufficiently. In this paper we aim to explore this problem in more detail and to study the possible solution, which is to evaluate the Fano factor in the operational time. We use equilibrium renewal and Markov renewal processes as spike train models to describe the method in detail, and we provide an illustration on experimental data.

Reliability Analysis of a System with Multiple-Stage Policy Considering Type I and II Errors

This paper formulates a stochastic model for a system with illegal access. The server has the function of IDS, and illegal access is checked in multiple stages which consist of simple check and detailed check. In this model, we consider type I and II errors of simple check and a type I error of detailed check. There are two cases where IDS judges the occurrence of illegal access erroneously. One is when illegal access does not occur, and the other is when illegal access occurs. We apply the theory of Markov renewal processes to a system with illegal access, and derive the mean time and the expected checking number until a server system becomes faulty. Further, an optimal policy which minimizes the expected cost is discussed. Finally, numerical examples are given.

Multiple-Stage Policies for a Server System with Illegal Access

As the Internet technology has developed, the demands for the improvement of the reliability and security of the system connected with the Internet have increased. Although various services are performed on the Internet, illegal access on the Internet has become a problem in recent years. This paper formulates stochastic models for a system with illegal access. The server has the function of IDS, and illegal access is checked in multiple stages which consist of simple check, detailed check and dynamic check. We apply the theory of Markov renewal processes to a system with illegal access, and derive the mean time and the expected checking number until a server system becomes faulty. Further, optimal policies which minimize the expected cost are discussed. Finally, numerical examples are given.

Large deviations for the empirical measure of heavy-tailed Markov renewal processes

A large deviations principle is established for the joint law of the empirical measure and the flow measure of a Markov renewal process on a finite graph. We do not assume any bound on the arrival times, allowing heavy-tailed distributions. In particular, the rate function is in general degenerate (it has a nontrivial set of zeros) and not strictly convex. These features show a behaviour highly different from what one may guess with a heuristic Donsker‒Varadhan analysis of the problem.