scholarly journals Learning and Solving Regular Decision Processes

2020 ◽  
Author(s):  
Eden Abadi ◽  
Ronen I. Brafman
Keyword(s):  
Dynamic Logic ◽  
Decision Processes ◽  
Regular Expressions ◽  
Challenging Problem ◽  
State Action ◽  
Linear Dynamic ◽  
Markovian Dynamics ◽  
Learning Techniques ◽  
Automata Learning ◽  
Mealy Machine

Regular Decision Processes (RDPs) are a recently introduced model that extends MDPs with non-Markovian dynamics and rewards. The non-Markovian behavior is restricted to depend on regular properties of the history. These can be specified using regular expressions or formulas in linear dynamic logic over finite traces. Fully specified RDPs can be solved by compiling them into an appropriate MDP. Learning RDPs from data is a challenging problem that has yet to be addressed, on which we focus in this paper. Our approach rests on a new representation for RDPs using Mealy Machines that emit a distribution and an expected reward for each state-action pair. Building on this representation, we combine automata learning techniques with history clustering to learn such a Mealy machine and solve it by adapting MCTS to it. We empirically evaluate this approach, demonstrating its feasibility.

scholarly journals Regular Decision Processes: A Model for Non-Markovian Domains

2019 ◽  
Author(s):  
Ronen I. Brafman ◽  
Giuseppe De Giacomo
Keyword(s):  
Hidden Variables ◽  
Dynamic Logic ◽  
Expressive Power ◽  
Decision Processes ◽  
Regular Expressions ◽  
Linear Dynamic ◽  
The Past ◽  
Markovian Dynamics ◽  
Complex Dependence ◽  
Reward Functions

We introduce and study Regular Decision Processes (RDPs), a new, compact, factored model for domains with non-Markovian dynamics and rewards. In RDPs, transition and reward functions are specified using formulas in linear dynamic logic over finite traces, a language with the expressive power of regular expressions. This allows specifying complex dependence on the past using intuitive and compact formulas, and provides a model that generalizes MDPs and k-order MDPs. RDPs can also approximate POMDPs without having to postulate the existence of hidden variables, and, in principle, can be learned from observations only. 

scholarly journals Prediction of Caption and Emoji of an Image using Deep Learning

2019 ◽  
Vol 8 (2S11) ◽  
pp. 3721-3724
Keyword(s):  
Deep Learning ◽  
Social Networking ◽  
Web Application ◽  
Automatic Generation ◽  
Application Framework ◽  
Challenging Problem ◽  
Artificial Intelligence Research ◽  
Learning Techniques ◽  
User Friendly ◽  
Visually Impaired Persons

With the invention of deep learning, there is a good progress in image classification. But automatic generation of captions for images is still a challenging problem and is in the initial stages of artificial intelligence research. Automatic description of images has applications in social networking and will be useful to visually impaired persons. This paper concentrates on designing a user-friendly web application framework which can predict the caption of an image using deep learning techniques. The verbs and objects present in the caption are used for forming the emoji and for predicting the major color of the image

scholarly journals Weighted Linear Dynamic Logic

2016 ◽  
Vol 226 ◽  
pp. 149-163 ◽  
Author(s):  
Manfred Droste ◽  
George Rahonis
Keyword(s):  
Dynamic Logic ◽  
Linear Dynamic

scholarly journals Representation and Reasoning about Strategic Abilities with ω-Regular Properties

Mathematics ◽  
2021 ◽  
Vol 9 (23) ◽  
pp. 3052
Author(s):  
Liping Xiong ◽  
Sumei Guo
Keyword(s):  
Temporal Logic ◽  
Linear Time ◽  
Dynamic Logic ◽  
Research Area ◽  
Regular Expressions ◽  
Multi Agent Systems ◽  
Practical Applications ◽  
Multi Agent ◽  
Strategy Logic ◽  
Active Research

Specification and verification of coalitional strategic abilities have been an active research area in multi-agent systems, artificial intelligence, and game theory. Recently, many strategic logics, e.g., Strategy Logic (SL) and alternating-time temporal logic (ATL*), have been proposed based on classical temporal logics, e.g., linear-time temporal logic (LTL) and computational tree logic (CTL*), respectively. However, these logics cannot express general ω-regular properties, the need for which are considered compelling from practical applications, especially in industry. To remedy this problem, in this paper, based on linear dynamic logic (LDL), proposed by Moshe Y. Vardi, we propose LDL-based Strategy Logic (LDL-SL). Interpreted on concurrent game structures, LDL-SL extends SL, which contains existential/universal quantification operators about regular expressions. Here we adopt a branching-time version. This logic can express general ω-regular properties and describe more programmed constraints about individual/group strategies. Then we study three types of fragments (i.e., one-goal, ATL-like, star-free) of LDL-SL. Furthermore, we show that prevalent strategic logics based on LTL/CTL*, such as SL/ATL*, are exactly equivalent with those corresponding star-free strategic logics, where only star-free regular expressions are considered. Moreover, results show that reasoning complexity about the model-checking problems for these new logics, including one-goal and ATL-like fragments, is not harder than those of corresponding SL or ATL*.

Cloud Load Balancing and Reinforcement Learning

2018 ◽  
pp. 266-291
Author(s):  
Abdelghafour Harraz ◽  
Mostapha Zbakh
Keyword(s):  
Artificial Intelligence ◽  
Reinforcement Learning ◽  
Load Balancing ◽  
Decision Process ◽  
Cloud System ◽  
Human Intervention ◽  
Q Learning ◽  
State Action ◽  
Learning Techniques ◽  
Markov Decision

Artificial Intelligence allows to create engines that are able to explore, learn environments and therefore create policies that permit to control them in real time with no human intervention. It can be applied, through its Reinforcement Learning techniques component, using frameworks such as temporal differences, State-Action-Reward-State-Action (SARSA), Q Learning to name a few, to systems that are be perceived as a Markov Decision Process, this opens door in front of applying Reinforcement Learning to Cloud Load Balancing to be able to dispatch load dynamically to a given Cloud System. The authors will describe different techniques that can used to implement a Reinforcement Learning based engine in a cloud system.

scholarly journals Parametric Linear Dynamic Logic

2017 ◽  
Vol 253 ◽  
pp. 237-256 ◽  
Author(s):  
Peter Faymonville ◽  
Martin Zimmermann
Keyword(s):  
Dynamic Logic ◽  
Linear Dynamic

Alternative Method to Simulate a Sub-idle Engine Operation in Order to Synthesize Its Control System

2016 ◽  
Vol 33 (3) ◽  
Author(s):  
Sergii I. Sukhovii ◽  
Feliks F. Sirenko ◽  
Sergiy V. Yepifanov ◽  
Igor Loboda
Keyword(s):  
Steady State ◽  
Combustion Chamber ◽  
Control Systems ◽  
Static Model ◽  
Control Algorithms ◽  
Challenging Problem ◽  
Engine Operation ◽  
Linear Dynamic ◽  
Lack Of Information ◽  
Linear Thermodynamic

AbstractThe steady-state and transient engine performances in control systems are usually evaluated by applying thermodynamic engine models. Most models operate between the idle and maximum power points, only recently, they sometimes address a sub-idle operating range. The lack of information about the component maps at the sub-idle modes presents a challenging problem. A common method to cope with the problem is to extrapolate the component performances to the sub-idle range. Precise extrapolation is also a challenge. As a rule, many scientists concern only particular aspects of the problem such as the lighting combustion chamber or the turbine operation under the turned-off conditions of the combustion chamber. However, there are no reports about a model that considers all of these aspects and simulates the engine starting. The proposed paper addresses a new method to simulate the starting. The method substitutes the non-linear thermodynamic model with a linear dynamic model, which is supplemented with a simplified static model. The latter model is the set of direct relations between parameters that are used in the control algorithms instead of commonly used component performances. Specifically, this model consists of simplified relations between the gas path parameters and the corrected rotational speed.

scholarly journals Editorial

2021 ◽  
Vol 27 (4) ◽  
pp. 323-323
Author(s):  
Christian Gütl
Keyword(s):  
Financial Support ◽  
Detection System ◽  
Heterogeneous Data ◽  
Spam Detection ◽  
Regular Expressions ◽  
Challenging Problem ◽  
Crucial Issue ◽  
High Quality ◽  
Distributed Information ◽  
Great Support

I am pleased to announce the fourth issue of 2021. As always, I would like to express my sincere appreciation for the great support that makes the continued publication of novel and high quality articles possible. Thus, I would like to thank all authors for their sound research contributions, the reviewers for their very helpful suggestions and the consortium members for their financial support. I would also like to report on further achievements regarding our new platform. We have successfully migrated all the information of the Board of Editors and we have also started to use the new review module. Due to the cooperation with Pensoft Inc., our new platform provider, we will also be able to offer review acknowledgment on the Publons portal in the future. In this regular issue, I am very pleased to introduce four accepted papers from three different countries and 14 involved authors. Martin Berglund, Brink van der Merwe, and Steyn van Litsenborgh from South Africa investigate in their article regular expressions which contain lookaheads in addition to the standard operators of union, concatenation, and Kleene star. Fairouz Fakhfakh, Slim Kallel and Saoussen Cheikhrouhou from Tunisia research and discuss in their work a crucial issue in modern distributed information systems, i.e. how to verify the correctness of Cloud and Fog systems based on formal verification. Marcia Henke, Eulanda Santos, Eduardo Souto, and Altair O. Santin from Brazil introduce their enhanced spam detection system which is based on analyzing the evolution of features. And finally, also from Brazil, Marcelo Aires Vieira, Elivaldo Lozer Fracalossi Ribeiro, Daniela Barreiro Claro, and Babacar Mane investigate the challenging problem of integrating heterogeneous DaaS and DBaaS sources and explore the Data Join (DJ) method for integrating heterogeneous data.

Sign in / Sign up

Export Citation Format

Share Document