Abstraction and Counterexample-Guided Refinement in Model Checking of Hybrid Systems

2003 ◽  
Vol 14 (04) ◽  
pp. 583-604 ◽  
Author(s):  
Edmund Clarke ◽  
Ansgar Fehnker ◽  
Zhi Han ◽  
Bruce Krogh ◽  
Joël Ouaknine ◽  
...  
Keyword(s):  
Model Checking ◽  
State Space ◽  
Hybrid Systems ◽  
State Variables ◽  
Discrete State ◽  
Continuous State Space ◽  
Continuous State ◽  
Finite State ◽  
Refinement Procedure ◽  
State Abstraction

Hybrid dynamic systems include both continuous and discrete state variables. Properties of hybrid systems, which have an infinite state space, can often be verified using ordinary model checking together with a finite-state abstraction. Model checking can be inconclusive, however, in which case the abstraction must be refined. This paper presents a new procedure to perform this refinement operation for abstractions of hybrid systems. Following an approach originally developed for finite-state systems [11, 25], the refinement procedure constructs a new abstraction that eliminates a counterexample generated by the model checker. For hybrid systems, analysis of the counterexample requires the computation of sets of reachable states in the continuous state space. We show how such reachability computations with varying degrees of complexity can be used to refine hybrid system abstractions efficiently. Examples illustrate our counterexample-guided refinement procedure. Experimental results for a prototype implementation indicate significant advantages over existing methods.

Asymptotic behaviour of continuous time, continuous state-space branching processes

1974 ◽  
Vol 11 (04) ◽  
pp. 669-677 ◽  
Author(s):  
D. R. Grey
Keyword(s):  
Asymptotic Behaviour ◽  
State Space ◽  
Discrete Time ◽  
Continuous Time ◽  
Branching Processes ◽  
Discrete State ◽  
Space And Time ◽  
Continuous State Space ◽  
Continuous State ◽  
Discrete State Space

Results on the behaviour of Markov branching processes as time goes to infinity, hitherto obtained for models which assume a discrete state-space or discrete time or both, are here generalised to a model with both state-space and time continuous. The results are similar but the methods not always so.

A STATISTICAL METHOD FOR DETECTING CYCLES IN DISCRETE DYNAMICAL SYSTEMS

1996 ◽  
Vol 06 (12a) ◽  
pp. 2375-2388 ◽  
Author(s):  
MARKUS LOHMANN ◽  
JAN WENZELBURGER
Keyword(s):  
Dynamical Systems ◽  
State Space ◽  
Statistical Method ◽  
Basins Of Attraction ◽  
Discrete State ◽  
Continuous State Space ◽  
Continuous State ◽  
Long Term Behavior ◽  
Discrete Time Dynamical Systems

This paper introduces a statistical method for detecting cycles in discrete time dynamical systems. The continuous state space is replaced by a discrete one consisting of cells. Hashing is used to represent the cells in the computer’s memory. An algorithm for a two-parameter bifurcation analysis is presented which uses the statistical method to detect cycles in the discrete state space. The output of this analysis is a colored cartogram where parameter regions are marked according to the long-term behavior of the system. Moreover, the algorithm allows the computation of basins of attraction of cycles.

Asymptotic behaviour of continuous time, continuous state-space branching processes

1974 ◽  
Vol 11 (4) ◽  
pp. 669-677 ◽  
Author(s):  
D. R. Grey
Keyword(s):  
Asymptotic Behaviour ◽  
State Space ◽  
Discrete Time ◽  
Continuous Time ◽  
Branching Processes ◽  
Discrete State ◽  
Space And Time ◽  
Continuous State Space ◽  
Continuous State ◽  
Discrete State Space

Results on the behaviour of Markov branching processes as time goes to infinity, hitherto obtained for models which assume a discrete state-space or discrete time or both, are here generalised to a model with both state-space and time continuous. The results are similar but the methods not always so.

scholarly journals Multi-Agent Reinforcement Learning Using Linear Fuzzy Model Applied to Cooperative Mobile Robots

Symmetry ◽  
2018 ◽  
Vol 10 (10) ◽  
pp. 461 ◽  
Author(s):  
David Luviano-Cruz ◽  
Francesco Garcia-Luna ◽  
Luis Pérez-Domínguez ◽  
S. Gadi
Keyword(s):  
Reinforcement Learning ◽  
Mobile Robots ◽  
State Space ◽  
Fuzzy Model ◽  
Discrete State ◽  
Continuous State Space ◽  
Continuous State ◽  
Multi Agent ◽  
Successful Approach ◽  
Q Function

A multi-agent system (MAS) is suitable for addressing tasks in a variety of domains without any programmed behaviors, which makes it ideal for the problems associated with the mobile robots. Reinforcement learning (RL) is a successful approach used in the MASs to acquire new behaviors; most of these select exact Q-values in small discrete state space and action space. This article presents a joint Q-function linearly fuzzified for a MAS’ continuous state space, which overcomes the dimensionality problem. Also, this article gives a proof for the convergence and existence of the solution proposed by the algorithm presented. This article also discusses the numerical simulations and experimental results that were carried out to validate the proposed algorithm.

Construction of Semi-Markov Decision Process Models of Continuous State Space Environments Using Growing Cell Structures and Multiagentk-Certainty Exploration Method

2009 ◽  
Vol 13 (6) ◽  
pp. 608-614
Author(s):  
Takeshi Tateyama ◽  
◽  
Seiichi Kawata ◽  
Yoshiki Shimomura ◽  
◽  
...  
Keyword(s):  
State Space ◽  
Markov Decision Process ◽  
Decision Process ◽  
Discrete State ◽  
Cell Structures ◽  
Growing Cell ◽  
Continuous State Space ◽  
Continuous State ◽  
Markov Decision ◽  
Growing Cell Structures

k-certainty exploration method, an efficient reinforcement learning algorithm, is not applied to environments whose state space is continuous because continuous state space must be changed to discrete state space. Our purpose is to construct discrete semi-Markov decision process (SMDP) models of such environments using growing cell structures to autonomously divide continuous state space then usingk-certainty exploration method to construct SMDP models. Multiagentk-certainty exploration method is then used to improve exploration efficiency. Mobile robot simulation demonstrated our proposal's usefulness and efficiency.

scholarly journals Reinforcement learning versus swarm intelligence for autonomous multi-HAPS coordination

2021 ◽  
Vol 3 (6) ◽  
Author(s):  
Ogbonnaya Anicho ◽  
Philip B. Charlesworth ◽  
Gurvinder S. Baicher ◽  
Atulya K. Nagar
Keyword(s):  
Reinforcement Learning ◽  
State Space ◽  
Swarm Intelligence ◽  
Performance Indicators ◽  
Convergence Rates ◽  
Tuning Parameters ◽  
Continuous State Space ◽  
Continuous State ◽  
User Coverage ◽  
Better Than

AbstractThis work analyses the performance of Reinforcement Learning (RL) versus Swarm Intelligence (SI) for coordinating multiple unmanned High Altitude Platform Stations (HAPS) for communications area coverage. It builds upon previous work which looked at various elements of both algorithms. The main aim of this paper is to address the continuous state-space challenge within this work by using partitioning to manage the high dimensionality problem. This enabled comparing the performance of the classical cases of both RL and SI establishing a baseline for future comparisons of improved versions. From previous work, SI was observed to perform better across various key performance indicators. However, after tuning parameters and empirically choosing suitable partitioning ratio for the RL state space, it was observed that the SI algorithm still maintained superior coordination capability by achieving higher mean overall user coverage (about 20% better than the RL algorithm), in addition to faster convergence rates. Though the RL technique showed better average peak user coverage, the unpredictable coverage dip was a key weakness, making SI a more suitable algorithm within the context of this work.

scholarly journals Instantaneous brain dynamics mapped to a continuous state space

NeuroImage ◽  
2017 ◽  
Vol 162 ◽  
pp. 344-352 ◽  
Author(s):  
Jacob C.W. Billings ◽  
Alessio Medda ◽  
Sadia Shakil ◽  
Xiaohong Shen ◽  
Amrit Kashyap ◽  
...  
Keyword(s):  
State Space ◽  
Brain Dynamics ◽  
Continuous State Space ◽  
Continuous State

scholarly journals Uncertainty quantification for quantum chemical models of complex reaction networks

2016 ◽  
Vol 195 ◽  
pp. 497-520 ◽  
Author(s):  
Jonny Proppe ◽  
Tamara Husch ◽  
Gregor N. Simm ◽  
Markus Reiher
Keyword(s):  
Free Energy ◽  
Electronic Structure ◽  
State Space ◽  
Time Scales ◽  
Reaction Network ◽  
Reaction Networks ◽  
Multiple Time Scales ◽  
Complex Reaction ◽  
Discrete State ◽  
Continuous State

For the quantitative understanding of complex chemical reaction mechanisms, it is, in general, necessary to accurately determine the corresponding free energy surface and to solve the resulting continuous-time reaction rate equations for a continuous state space. For a general (complex) reaction network, it is computationally hard to fulfill these two requirements. However, it is possible to approximately address these challenges in a physically consistent way. On the one hand, it may be sufficient to consider approximate free energies if a reliable uncertainty measure can be provided. On the other hand, a highly resolved time evolution may not be necessary to still determine quantitative fluxes in a reaction network if one is interested in specific time scales. In this paper, we present discrete-time kinetic simulations in discrete state space taking free energy uncertainties into account. The method builds upon thermo-chemical data obtained from electronic structure calculations in a condensed-phase model. Our kinetic approach supports the analysis of general reaction networks spanning multiple time scales, which is here demonstrated for the example of the formose reaction. An important application of our approach is the detection of regions in a reaction network which require further investigation, given the uncertainties introduced by both approximate electronic structure methods and kinetic models. Such cases can then be studied in greater detail with more sophisticated first-principles calculations and kinetic simulations.

scholarly journals Improved estimations of stochastic chemical kinetics by finite-state expansion

2021 ◽  
Vol 477 (2251) ◽  
pp. 20200964
Author(s):  
Tabea Waizmann ◽  
Luca Bortolussi ◽  
Andrea Vandin ◽  
Mirco Tribastone
Keyword(s):  
Master Equation ◽  
State Space ◽  
Stochastic Dynamics ◽  
Reaction Network ◽  
Discrete State ◽  
State Expansion ◽  
Analytical Approximations ◽  
Finite State ◽  
Stochastic Reaction ◽  
Discrete State Space

Stochastic reaction networks are a fundamental model to describe interactions between species where random fluctuations are relevant. The master equation provides the evolution of the probability distribution across the discrete state space consisting of vectors of population counts for each species. However, since its exact solution is often elusive, several analytical approximations have been proposed. The deterministic rate equation (DRE) gives a macroscopic approximation as a compact system of differential equations that estimate the average populations for each species, but it may be inaccurate in the case of nonlinear interaction dynamics. Here we propose finite-state expansion (FSE), an analytical method mediating between the microscopic and the macroscopic interpretations of a stochastic reaction network by coupling the master equation dynamics of a chosen subset of the discrete state space with the mean population dynamics of the DRE. An algorithm translates a network into an expanded one where each discrete state is represented as a further distinct species. This translation exactly preserves the stochastic dynamics, but the DRE of the expanded network can be interpreted as a correction to the original one. The effectiveness of FSE is demonstrated in models that challenge state-of-the-art techniques due to intrinsic noise, multi-scale populations and multi-stability.

Sign in / Sign up

Export Citation Format

Share Document