Multivariate Markov Chain Model for Sales Demand Estimation in a Company

Estimation of the number of demands for a product must be done correctly, so that the company can get maximum profit. Therefore, this study discusses how to estimate the amount of sales demand in a company correctly. The model that will be used to estimate sales demand is the Multivariate Markov Chain Model. This model can estimate the future state by observing the present state. The model requires parameter estimation values ​​first, namely the transition probability matrix and the weighted Markov chain, where in previous studies an estimation of the transition probability matrix has been carried out, so that in this study we will continue to estimate the weighted Markov chain parameters. This model is compatible with 5 data sequences (product types) defined as product 1, product 2, product 3, product 4, and product 5, with 6 conditions (no sales volume, very slow-moving, slow-moving, standard, fast moving, and very fast moving). As the result, the state probability for product 1, product 2 and product 3 in company 1 are stationary at state 6 (very fast moving), product 4 and product 5 are stationary at state 2 (very slow moving).

A Predictive Energy Management Strategy for Multi-Energy Source Vehicles Based on Full-Factor Trip Information

To achieve the real-time application of a dynamic programming (DP) control strategy, we propose a predictive energy management strategy (PEMS) based on full-factor trip information, including vehicle speed, slip ratio and slope. Firstly, the prediction model of the full-factor trip information is proposed, which provides an information basis for global optimization energy management. To improve the prediction’s accuracy, the vehicle speed is predicted based on the state transition probability matrix generated in the same driving scene. The characteristic parameters are extracted by a feature selection method taken as the basis for the driving condition’s identification. Similar to speed prediction, regarding the uncertain route at an intersection, the slope prediction is modelled as a Markov model. On the basis of the predicted speed and the identified maximum adhesion coefficient, the slip ratio is predicted based on a neural network. Then, a predictive energy management strategy is developed based on the predictive full-factor trip information. According to the statistical rules of DP results under multiple standard driving cycles, the reference SOC trajectory is generated to ensure global sub-optimality, which determines the feasible state domain at each prediction horizon. Simulations are performed under different types of driving conditions (Urban Dynamometer Driving Schedule, UDDS and World Light Vehicle Test Cycle, WLTC) to verify the effectiveness of the proposed strategy.

Recognizing duration effects in multistate population models

The risk of many demographic events varies by both current state and duration in that state. However, the use of such semi-Markov models has been substantially constrained by data limitations. Here, a new specification of the semi-Markov transition probability matrix in terms of the underlying rates is provided, and a general procedure is developed to estimate semi-Markov probabilities and rates from adjacent population data.Multistate models recognizing marriage and divorce by duration in state are constructed for United States Females, 1995. The results show that recognizing duration in the married and divorced states adds significantly to the model’s analytical value. Extending the constant-α method to semi-Markov models, 2000–2005 U.S. population data and 1995 cross-product ratios are employed to estimate 2000–2005 duration-dependent transfer probabilities and rates.The present analyses provide new relationships between probabilities and rates in semi-Markov models. Extending the constant cross-product ratio estimation approach opens new sources of data and expands the range of data susceptible to state-duration analyses.

A Characteristic Polynomial for the Transition Probability Matrix of Correlated Random Walks on a Graph

We define a correlated random walk (CRW) induced from the time evolution matrix (the Grover matrix) of the Grover walk on a graph $G$, and present a formula for the characteristic polynomial of the transition probability matrix of this CRW by using a determinant expression for the generalized weighted zeta function of $G$. As an application, we give the spectrum of the transition probability matrices for the CRWs induced from the Grover matrices of regular graphs and semiregular bipartite graphs. Furthermore, we consider another type of the CRW on a graph.

dynaPhenoM: Dynamic Phenotype Modeling from Longitudinal Patient Records Using Machine Learning

Identification of clinically meaningful subphenotypes of disease progression can facilitate better understanding of disease heterogeneity and underlying pathophysiology. We propose a machine learning algorithm, termed dynaPhenoM, to achieve this goal based on longitudinal patient records such as electronic health records (EHR) or insurance claims. Specifically, dynaPhenoM first learns a set of coherent clinical topics from the events across different patient visits within the records along with the topic transition probability matrix, and then employs the time-aware latent class analysis (T-LCA) procedure to characterize each subphenotype as the evolution of these learned topics over time. The patients in the same subphenotype have similar such topic evolution patterns. We demonstrate the effectiveness and robustness of dynaPhenoM on the case of mild cognitive impairment (MCI) to Alzheimer's disease (AD) progression on three patient cohorts, and five informative subphenotypes were identified which suggest the different clinical trajectories for disease progression from MCI to AD.

Variational Beta Process Hidden Markov Models with Shared Hidden States for Trajectory Recognition

Hidden Markov model (HMM) is a vital model for trajectory recognition. As the number of hidden states in HMM is important and hard to be determined, many nonparametric methods like hierarchical Dirichlet process HMMs and Beta process HMMs (BP-HMMs) have been proposed to determine it automatically. Among these methods, the sampled BP-HMM models the shared information among different classes, which has been proved to be effective in several trajectory recognition scenes. However, the existing BP-HMM maintains a state transition probability matrix for each trajectory, which is inconvenient for classification. Furthermore, the approximate inference of the BP-HMM is based on sampling methods, which usually takes a long time to converge. To develop an efficient nonparametric sequential model that can capture cross-class shared information for trajectory recognition, we propose a novel variational BP-HMM model, in which the hidden states can be shared among different classes and each class chooses its own hidden states and maintains a unified transition probability matrix. In addition, we derive a variational inference method for the proposed model, which is more efficient than sampling-based methods. Experimental results on a synthetic dataset and two real-world datasets show that compared with the sampled BP-HMM and other related models, the variational BP-HMM has better performance in trajectory recognition.

Dynamics of Area Substitution of Pulses in India

Background: Pulses have been very curial in many aspects like; rich source of protein, economy aspect and contribute to agricultural and environmental sustainability. In this investigation an attempt has been made to evaluate the dynamics of area substitution between pulses and other crops, extent of spatial shift among pulses producing states and some policy measures have been suggested to stabilize the area under pulses across states. Methods: Secondary data on area of principal crops for the period 1966-2016 was used in this article. By computing quartile values, all states and groups of states were clubbed into four different quartiles for each decade. Area substitution among principal field crops including pulses has been analyzed using first order Markov transition probability matrix (TPM). Result: This TPM was further used to evaluate the mobility of membership status of states under pulses production. It was found that period 1986-2006 happened to be the golden period for area under pulses in India. Mean area under pulses had increased for first three decades and in the subsequent two decades mean area (quartile values) had declined substantially.

Concentration optimization of combinatorial drugs using Markov chain-based models

Background Combinatorial drug therapy for complex diseases, such as HSV infection and cancers, has a more significant efficacy than single-drug treatment. However, one key challenge is how to effectively and efficiently determine the optimal concentrations of combinatorial drugs because the number of drug combinations increases exponentially with the types of drugs. Results In this study, a searching method based on Markov chain is presented to optimize the combinatorial drug concentrations. In this method, the searching process of the optimal drug concentrations is converted into a Markov chain process with state variables representing all possible combinations of discretized drug concentrations. The transition probability matrix is updated by comparing the drug responses of the adjacent states in the network of the Markov chain and the drug concentration optimization is turned to seek the state with maximum value in the stationary distribution vector. Its performance is compared with five stochastic optimization algorithms as benchmark methods by simulation and biological experiments. Both simulation results and experimental data demonstrate that the Markov chain-based approach is more reliable and efficient in seeking global optimum than the benchmark algorithms. Furthermore, the Markov chain-based approach allows parallel implementation of all drug testing experiments, and largely reduces the times in the biological experiments. Conclusion This article provides a versatile method for combinatorial drug screening, which is of great significance for clinical drug combination therapy.

Model-Based Reinforcement Learning for Infinite-Horizon Discounted Constrained Markov Decision Processes

In many real-world reinforcement learning (RL) problems, in addition to maximizing the objective, the learning agent has to maintain some necessary safety constraints. We formulate the problem of learning a safe policy as an infinite-horizon discounted Constrained Markov Decision Process (CMDP) with an unknown transition probability matrix, where the safety requirements are modeled as constraints on expected cumulative costs. We propose two model-based constrained reinforcement learning (CRL) algorithms for learning a safe policy, namely, (i) GM-CRL algorithm, where the algorithm has access to a generative model, and (ii) UC-CRL algorithm, where the algorithm learns the model using an upper confidence style online exploration method. We characterize the sample complexity of these algorithms, i.e., the the number of samples needed to ensure a desired level of accuracy with high probability, both with respect to objective maximization and constraint satisfaction.

Intergenerational educational mobility in Bangladesh

Introduction Social mobility is considered as an important indicator of the economic development of a country. However, it varies widely across geographical regions and social groups in developing countries like Bangladesh. This paper intends to evaluate the intergenerational mobility in Bangladesh across generations. Methods and materials This paper considers a nationally representative sample survey of 8,403 respondents (rural: 5,436 and urban: 2,967). The male and female respondents aged 23 years and above were included in the sample. The education attainment of a son or daughter as compared to their father’s education level was considered as the measure of intergenerational mobility. Transition probability matrix and different social mobility indices were used to find out the intergenerational education mobility in Bangladesh. Results The findings reveal that approximately three-fourth (74.5%) of the respondents attained formal education, while more than half (58.3%) of the respondents’ father was illiterate. The educational status of the respondents and their father who lived in urban areas was relatively better than who lived in rural areas. It is also observed that 91.2% and 81.6% of the intergenerational class movement was upward among sons and daughters respectively. The probability of a higher educated father will have a higher educated child is higher in urban areas than in rural areas of Bangladesh. The intergenerational mobility is higher in the primary, secondary, and higher secondary educational levels, though the illiterate and higher education levels are the least mobile classes. In addition, the limiting probabilities reveal that the chance of sending sons to schools by an illiterate father is less as compared to their daughters. Such difference is more obvious in the urban areas, i.e., it is highly likely that sons of the illiterate father are also illiterate. Conclusion Bangladesh has been progressing remarkably in recent years. To keep the pace of the ongoing economic development in the country, it is necessary to give more attention to the illiterate people especially the girls who live in rural areas. The authors anticipate that the findings will be helpful for the policymakers as the relationship between inequality and intergenerational mobility is vital for several aspects of the economic development of a country.