scholarly journals PAC-Bayes Bounds on Variational Tempered Posteriors for Markov Models

Entropy ◽  
2021 ◽  
Vol 23 (3) ◽  
pp. 313
Author(s):  
Imon Banerjee ◽  
Vinayak A. Rao ◽  
Harsha Honnappa
Keyword(s):  
Markov Model ◽  
Markov Models ◽  
Model Misspecification ◽  
Dependence Structure ◽  
Posterior Distributions ◽  
Ergodic Properties ◽  
Variational Approximations ◽  
Popular View ◽  
Probably Approximately Correct ◽  
Risk Bounds

Datasets displaying temporal dependencies abound in science and engineering applications, with Markov models representing a simplified and popular view of the temporal dependence structure. In this paper, we consider Bayesian settings that place prior distributions over the parameters of the transition kernel of a Markov model, and seek to characterize the resulting, typically intractable, posterior distributions. We present a Probably Approximately Correct (PAC)-Bayesian analysis of variational Bayes (VB) approximations to tempered Bayesian posterior distributions, bounding the model risk of the VB approximations. Tempered posteriors are known to be robust to model misspecification, and their variational approximations do not suffer the usual problems of over confident approximations. Our results tie the risk bounds to the mixing and ergodic properties of the Markov data generating model. We illustrate the PAC-Bayes bounds through a number of example Markov models, and also consider the situation where the Markov model is misspecified.

scholarly journals A copula-based multivariate hidden Markov model for modelling momentum in football

2021 ◽  
Author(s):  
Marius Ötting ◽  
Roland Langrock ◽  
Antonello Maruotti
Keyword(s):  
Markov Model ◽  
Hidden Markov Models ◽  
Markov Models ◽  
Hidden Markov ◽  
State Dependence ◽  
Change Points ◽  
Sports Fans ◽  
Modelling Framework ◽  
State Dependent ◽  
Different Levels

AbstractWe investigate the potential occurrence of change points—commonly referred to as “momentum shifts”—in the dynamics of football matches. For that purpose, we model minute-by-minute in-game statistics of Bundesliga matches using hidden Markov models (HMMs). To allow for within-state dependence of the variables, we formulate multivariate state-dependent distributions using copulas. For the Bundesliga data considered, we find that the fitted HMMs comprise states which can be interpreted as a team showing different levels of control over a match. Our modelling framework enables inference related to causes of momentum shifts and team tactics, which is of much interest to managers, bookmakers, and sports fans.

Projecting demands for renal replacement therapy in the Northern Territory: a stochastic Markov model

2018 ◽  
Vol 42 (4) ◽  
pp. 380 ◽  
Author(s):  
Jiqiong You ◽  
Yuejen Zhao ◽  
Paul Lawton ◽  
Steven Guthridge ◽  
Stephen P. McDonald ◽  
...  
Keyword(s):  
Renal Replacement Therapy ◽  
Markov Model ◽  
Replacement Therapy ◽  
Management Practices ◽  
Markov Models ◽  
Self Care ◽  
Potential Effect ◽  
Northern Territory ◽  
Baseline Scenario ◽  
On Demand

Objective The aim of the present study was to evaluate the potential effects of different health intervention strategies on demand for renal replacement therapy (RRT) services in the Northern Territory (NT). Methods A Markov chain simulation model was developed to estimate demand for haemodialysis (HD) and kidney transplantation (Tx) over the next 10 years, based on RRT registry data between 2002 and 2013. Four policy-relevant scenarios were evaluated: (1) increased Tx; (2) increased self-care dialysis; (3) reduced incidence of end-stage kidney disease (ESKD); and (4) reduced mortality. Results There were 957 new cases of ESKD during the study period, with most patients being Indigenous people (85%). The median age was 50 years at onset and 57 years at death, 12 and 13 years younger respectively than Australian medians. The prevalence of RRT increased 5.6% annually, 20% higher than the national rate (4.7%). If current trends continue (baseline scenario), the demand for facility-based HD (FHD) would approach 100 000 treatments (95% confidence interval 75 000–121 000) in 2023, a 5% annual increase. Increasing Tx (0.3%), increasing self-care (5%) and reducing incidence (5%) each attenuate demand for FHD to ~70 000 annually by 2023. Conclusions The present study demonstrates the effects of changing service patterns to increase Tx, self-care and prevention, all of which will substantially attenuate the growth in FHD requirements in the NT. What is known about the topic? The burden of ESKD is projected to increase in the NT, with demand for FHD doubling every 15 years. Little is known about the potential effect of changes in health policy and clinical practice on demand. What does this paper add? This study assessed the usefulness of a stochastic Markov model to evaluate the effects of potential policy changes on FHD demand. What are the implications for practitioners? The scenarios simulated by the stochastic Markov models suggest that changes in current ESKD management practices would have a large effect on future demand for FHD.

scholarly journals Computing Optimal Properties of Drugs Using Mathematical Models of Single Channel Dynamics

2018 ◽  
Vol 6 (1) ◽  
pp. 41-64 ◽  
Author(s):  
Aslak Tveito ◽  
Mary M. Maleckar ◽  
Glenn T. Lines
Keyword(s):  
Differential Equations ◽  
Markov Model ◽  
Stochastic Models ◽  
Markov Models ◽  
Single Channel ◽  
Probability Density Functions ◽  
Density Functions ◽  
Mathematical Framework ◽  
Wild Type ◽  
Channel Dynamics

AbstractSingle channel dynamics can be modeled using stochastic differential equations, and the dynamics of the state of the channel (e.g. open, closed, inactivated) can be represented using Markov models. Such models can also be used to represent the effect of mutations as well as the effect of drugs used to alleviate deleterious effects of mutations. Based on the Markov model and the stochastic models of the single channel, it is possible to derive deterministic partial differential equations (PDEs) giving the probability density functions (PDFs) of the states of the Markov model. In this study, we have analyzed PDEs modeling wild type (WT) channels, mutant channels (MT) and mutant channels for which a drug has been applied (MTD). Our aim is to show that it is possible to optimize the parameters of a given drug such that the solution of theMTD model is very close to that of the WT: the mutation’s effect is, theoretically, reduced significantly.We will present the mathematical framework underpinning this methodology and apply it to several examples. In particular, we will show that it is possible to use the method to, theoretically, improve the properties of some well-known existing drugs.

scholarly journals The Application of Baum-Welch Algorithm in Multistep Attack

2014 ◽  
Vol 2014 ◽  
pp. 1-7 ◽  
Author(s):  
Yanxue Zhang ◽  
Dongmei Zhao ◽  
Jinxing Liu
Keyword(s):  
Markov Model ◽  
Hidden Markov Model ◽  
Hidden Markov Models ◽  
Viterbi Algorithm ◽  
Markov Models ◽  
Hidden Markov ◽  
Simulation Experiments ◽  
Forward Algorithm ◽  
Better Than

The biggest difficulty of hidden Markov model applied to multistep attack is the determination of observations. Now the research of the determination of observations is still lacking, and it shows a certain degree of subjectivity. In this regard, we integrate the attack intentions and hidden Markov model (HMM) and support a method to forecasting multistep attack based on hidden Markov model. Firstly, we train the existing hidden Markov model(s) by the Baum-Welch algorithm of HMM. Then we recognize the alert belonging to attack scenarios with the Forward algorithm of HMM. Finally, we forecast the next possible attack sequence with the Viterbi algorithm of HMM. The results of simulation experiments show that the hidden Markov models which have been trained are better than the untrained in recognition and prediction.

scholarly journals A Markov model of urban evolution: Neighbourhood change as a complex process

PLoS ONE ◽  
2021 ◽  
Vol 16 (1) ◽  
pp. e0245357
Author(s):  
Daniel Silver ◽  
Thiago H. Silva
Keyword(s):  
Markov Model ◽  
Markov Models ◽  
Past Research ◽  
Hierarchical Approach ◽  
Evolutionary Trajectory ◽  
Complex Process ◽  
Trajectories Of Change ◽  
Neighbourhood Change ◽  
The Impact ◽  
Complexity Theories

This paper seeks to advance neighbourhood change research and complexity theories of cities by developing and exploring a Markov model of socio-spatial neighbourhood evolution in Toronto, Canada. First, we classify Toronto neighbourhoods into distinct groups using established geodemographic segmentation techniques, a relatively novel application in this geographic setting. Extending previous studies, we pursue a hierarchical approach to classifying neighbourhoods that situates many neighbourhood types within the city’s broader structure. Our hierarchical approach is able to incorporate a richer set of types than most past research and allows us to study how neighbourhoods’ positions within this hierarchy shape their trajectories of change. Second, we use Markov models to identify generative processes that produce patterns of change in the city’s distribution of neighbourhood types. Moreover, we add a spatial component to the Markov process to uncover the extent to which change in one type of neighbourhood depends on the character of nearby neighbourhoods. In contrast to the few studies that have explored Markov models in this research tradition, we validate the model’s predictive power. Third, we demonstrate how to use such models in theoretical scenarios considering the impact on the city’s predicted evolutionary trajectory when existing probabilities of neighbourhood transitions or distributions of neighbourhood types would hypothetically change. Markov models of transition patterns prove to be highly accurate in predicting the final distribution of neighbourhood types. Counterfactual scenarios empirically demonstrate urban complexity: small initial changes reverberate throughout the system, and unfold differently depending on their initial geographic distribution. These scenarios show the value of complexity as a framework for interpreting data and guiding scenario-based planning exercises.

scholarly journals Utilizing Gaze Behavior for Inferring Task Transitions Using Abstract Hidden Markov Models

2016 ◽  
Vol 19 (58) ◽  
pp. 1
Author(s):  
Daniel Fernando Tello Gamarra
Keyword(s):  
Markov Model ◽  
Hidden Markov Model ◽  
Hidden Markov Models ◽  
Markov Models ◽  
Hidden Markov ◽  
Hand Movement ◽  
Gaze Behavior ◽  
Improved Method ◽  
The Relationship

We demonstrate an improved method for utilizing observed gaze behavior and show that it is useful in inferring hand movement intent during goal directed tasks. The task dynamics and the relationship between hand and gaze behavior are learned using an Abstract Hidden Markov Model (AHMM). We show that the predicted hand movement transitions occur consistently earlier in AHMM models with gaze than those models that do not include gaze observations.

scholarly journals Modelling persistence in annual Australia point rainfall

2003 ◽  
Vol 7 (2) ◽  
pp. 197-211 ◽  
Author(s):  
J. P. Whiting ◽  
M. F. Lambert ◽  
A. V. Metcalfe
Keyword(s):  
Markov Model ◽  
Strong Evidence ◽  
Markov Models ◽  
Southern Oscillation ◽  
Mixture Distribution ◽  
Simulation Models ◽  
Annual Rainfall ◽  
Climate Indices ◽  
Rainfall Time Series ◽  
Fractional Differencing

Abstract. Annual rainfall time series for Sydney from 1859 to 1999 is analysed. Clear evidence of nonstationarity is presented, but substantial evidence for persistence or hidden states is more elusive. A test of the hypothesis that a hidden state Markov model reduces to a mixture distribution is presented. There is strong evidence of a correlation between the annual rainfall and climate indices. Strong evidence of persistence of one of these indices, the Pacific Decadal Oscillation (PDO), is presented together with a demonstration that this is better modelled by fractional differencing than by a hidden state Markov model. It is shown that conditioning the logarithm of rainfall on PDO, the Southern Oscillation index (SOI), and their interaction provides realistic simulation of rainfall that matches observed statistics. Similar simulation models are presented for Brisbane, Melbourne and Perth. Keywords: Hydrological persistence,hidden state Markov models, fractional differencing, PDO, SOI, Australian rainfall

A semi-markov model for clinical trials

1965 ◽  
Vol 2 (02) ◽  
pp. 269-285 ◽  
Author(s):  
George H. Weiss ◽  
Marvin Zelen
Keyword(s):  
Clinical Trials ◽  
Probability Distribution ◽  
Acute Leukemia ◽  
Markov Model ◽  
Markov Models ◽  
Transition Probabilities ◽  
State Transitions ◽  
Semi Markov Processes ◽  
Absorbing States ◽  
Epidemic Diseases

This paper applies the theory of semi-Markov processes to the construction of a stochastic model for interpreting data obtained from clinical trials. The model characterizes the patient as being in one of a finite number of states at any given time with an arbitrary probability distribution to describe the length of stay in a state. Transitions between states are assumed to be chosen according to a stationary finite Markov chain.Other attempts have been made to develop stochastic models of clinical trials. However, these have all been essentially Markovian with constant transition probabilities which implies that the distribution of time spent during a visit to a state is exponential (or geometric for discrete Markov chains). Markov models need also to assume that the transitions in the state of a patient depend only on absolute time whereas the semi-Markov model assumes that transitions depend on time relative to a patient. Thus the models are applicable to degenerative diseases (cancer, acute leukemia), while Markov models with time dependent transition probabilities are applicable to colds and epidemic diseases. In this paper the Laplace transforms are obtained for (i) probability of being in a state at timet, (ii) probability distribution to reach absorption state and (iii) the probability distribution of the first passage times to go from initial states to transient or absorbing states, transient to transient, and transient to absorbing. The model is applied to a clinical study of acute leukemia in which patients have been treated with methotrexate and 6-mercaptopurine. The agreement between the data and the model is very good.

scholarly journals A Hidden Markov Model for the Linguistic Analysis of the Voynich Manuscript

2019 ◽  
Vol 24 (1) ◽  
pp. 14 ◽  
Author(s):  
Luis Acedo
Keyword(s):  
Time Series ◽  
Markov Model ◽  
Hidden Markov Model ◽  
Hidden Markov Models ◽  
Markov Models ◽  
Hidden Markov ◽  
Mathematical Linguistics ◽  
Underlying Structure ◽  
Internal States ◽  
Consonant Sounds

Hidden Markov models are a very useful tool in the modeling of time series and any sequence of data. In particular, they have been successfully applied to the field of mathematical linguistics. In this paper, we apply a hidden Markov model to analyze the underlying structure of an ancient and complex manuscript, known as the Voynich manuscript, which remains undeciphered. By assuming a certain number of internal states representations for the symbols of the manuscripts, we train the network by means of the α and β -pass algorithms to optimize the model. By this procedure, we are able to obtain the so-called transition and observation matrices to compare with known languages concerning the frequency of consonant andvowel sounds. From this analysis, we conclude that transitions occur between the two states with similar frequencies to other languages. Moreover, the identification of the vowel and consonant sounds matches some previous tentative bottom-up approaches to decode the manuscript.

A COMPARISON OF SCORING METRICS FOR PREDICTING THE NEXT NAVIGATION STEP WITH MARKOV MODEL-BASED SYSTEMS

2010 ◽  
Vol 09 (04) ◽  
pp. 547-573 ◽  
Author(s):  
JOSÉ BORGES ◽  
MARK LEVENE
Keyword(s):  
Markov Model ◽  
Prediction Accuracy ◽  
Prediction Models ◽  
Markov Models ◽  
Real Data ◽  
Absolute Error ◽  
Brier Score ◽  
Data Sets ◽  
Extensive Evaluation ◽  
The Impact

The problem of predicting the next request during a user's navigation session has been extensively studied. In this context, higher-order Markov models have been widely used to model navigation sessions and to predict the next navigation step, while prediction accuracy has been mainly evaluated with the hit and miss score. We claim that this score, although useful, is not sufficient for evaluating next link prediction models with the aim of finding a sufficient order of the model, the size of a recommendation set, and assessing the impact of unexpected events on the prediction accuracy. Herein, we make use of a variable length Markov model to compare the usefulness of three alternatives to the hit and miss score: the Mean Absolute Error, the Ignorance Score, and the Brier score. We present an extensive evaluation of the methods on real data sets and a comprehensive comparison of the scoring methods.

Sign in / Sign up

Export Citation Format

Share Document