scholarly journals Bridging Commonsense Reasoning and Probabilistic Planning via a Probabilistic Action Language

2019 ◽  
Vol 19 (5-6) ◽  
pp. 1090-1106
Author(s):  
YI WANG ◽  
SHIQI ZHANG ◽  
JOOHYUNG LEE
Keyword(s):  
Planning Under Uncertainty ◽  
Commonsense Reasoning ◽  
Sequential Decision ◽  
Unified Framework ◽  
Probabilistic Planning ◽  
Commonsense Knowledge ◽  
Action Languages ◽  
Action Language ◽  
Markov Decision ◽  
Probabilistic Action

AbstractTo be responsive to dynamically changing real-world environments, an intelligent agent needs to perform complex sequential decision-making tasks that are often guided by commonsense knowledge. The previous work on this line of research led to the framework called interleaved commonsense reasoning and probabilistic planning (icorpp), which used P-log for representing commmonsense knowledge and Markov Decision Processes (MDPs) or Partially Observable MDPs (POMDPs) for planning under uncertainty. A main limitation of icorpp is that its implementation requires non-trivial engineering efforts to bridge the commonsense reasoning and probabilistic planning formalisms. In this paper, we present a unified framework to integrate icorpp’s reasoning and planning components. In particular, we extend probabilistic action language pBC+ to express utility, belief states, and observation as in POMDP models. Inheriting the advantages of action languages, the new action language provides an elaboration tolerant representation of POMDP that reflects commonsense knowledge. The idea led to the design of the system pbcplus2pomdp, which compiles a pBC+ action description into a POMDP model that can be directly processed by off-the-shelf POMDP solvers to compute an optimal policy of the pBC+ action description. Our experiments show that it retains the advantages of icorpp while avoiding the manual efforts in bridging the commonsense reasoner and the probabilistic planner.

Elaboration Tolerant Representation of Markov Decision Process via Decision-Theoretic Extension of Probabilistic Action Language +

2020 ◽  
pp. 1-24
Author(s):  
YI WANG ◽  
JOOHYUNG LEE
Keyword(s):  
Decision Theory ◽  
Programming Language ◽  
Markov Decision Process ◽  
Optimal Policy ◽  
Decision Process ◽  
Answer Set Programming ◽  
Shorthand Notation ◽  
Action Language ◽  
Markov Decision ◽  
Probabilistic Action

Abstract We extend probabilistic action language $p{\cal BC}$ + with the notion of utility in decision theory. The semantics of the extended $p{\cal BC}$ + can be defined as a shorthand notation for a decision-theoretic extension of the probabilistic answer set programming language LPMLN. Alternatively, the semantics of $p{\cal BC}$ + can also be defined in terms of Markov decision process (MDP), which in turn allows for representing MDP in a succinct and elaboration tolerant way as well as leveraging an MDP solver to compute a $p{\cal BC}$ + action description. The idea led to the design of the system pbcplus2mdp, which can find an optimal policy of a $p{\cal BC}$ + action description using an MDP solver.

scholarly journals Decision-Theoretic Planning: Structural Assumptions and Computational Leverage

1999 ◽  
Vol 11 ◽  
pp. 1-94 ◽  
Author(s):  
C. Boutilier ◽  
T. Dean ◽  
S. Hanks
Keyword(s):  
Operations Research ◽  
Performance Criteria ◽  
State Transitions ◽  
Value Functions ◽  
Planning Under Uncertainty ◽  
Ai Planning ◽  
Sequential Decision ◽  
Markov Decision ◽  
Planning Decision ◽  
Planning Problems

Planning under uncertainty is a central problem in the study of automated sequential decision making, and has been addressed by researchers in many different fields, including AI planning, decision analysis, operations research, control theory and economics. While the assumptions and perspectives adopted in these areas often differ in substantial ways, many planning problems of interest to researchers in these fields can be modeled as Markov decision processes (MDPs) and analyzed using the techniques of decision theory. This paper presents an overview and synthesis of MDP-related methods, showing how they provide a unifying framework for modeling many classes of planning problems studied in AI. It also describes structural properties of MDPs that, when exhibited by particular classes of problems, can be exploited in the construction of optimal or approximately optimal policies or plans. Planning problems commonly possess structure in the reward and value functions used to describe performance criteria, in the functions used to describe state transitions and observations, and in the relationships among features used to describe states, actions, rewards, and observations. Specialized representations, and algorithms employing these representations, can achieve computational leverage by exploiting these various forms of structure. Certain AI techniques -- in particular those based on the use of structured, intensional representations -- can be viewed in this way. This paper surveys several types of representations for both classical and decision-theoretic planning problems, and planning algorithms that exploit these representations in a number of different ways to ease the computational burden of constructing policies or plans. It focuses primarily on abstraction, aggregation and decomposition techniques based on AI-style representations.

scholarly journals Optimal Policies for Quantum Markov Decision Processes

2021 ◽  
Author(s):  
Ming-Sheng Ying ◽  
Yuan Feng ◽  
Sheng-Gang Ying
Keyword(s):  
Decision Making ◽  
Reinforcement Learning ◽  
Quantum Systems ◽  
Sequential Decision Making ◽  
Mathematical Framework ◽  
Sequential Decision ◽  
Learning Techniques ◽  
Optimal Policies ◽  
Markov Decision ◽  
Programming Algorithms

AbstractMarkov decision process (MDP) offers a general framework for modelling sequential decision making where outcomes are random. In particular, it serves as a mathematical framework for reinforcement learning. This paper introduces an extension of MDP, namely quantum MDP (qMDP), that can serve as a mathematical model of decision making about quantum systems. We develop dynamic programming algorithms for policy evaluation and finding optimal policies for qMDPs in the case of finite-horizon. The results obtained in this paper provide some useful mathematical tools for reinforcement learning techniques applied to the quantum world.

POMDP-Based Coding of Child–Robot Interaction within a Robot-Assisted ASD Diagnostic Protocol

2018 ◽  
Vol 15 (02) ◽  
pp. 1850011 ◽  
Author(s):  
Frano Petric ◽  
Damjan Miklić ◽  
Zdenko Kovačić
Keyword(s):  
Humanoid Robots ◽  
Autism Diagnostic Observation Schedule ◽  
Autism Spectrum ◽  
Diagnostic Process ◽  
Sequential Decision ◽  
Robot Assisted ◽  
Robot Interaction ◽  
Fine Grained ◽  
Markov Decision ◽  
Partially Observable

The existing procedures for autism spectrum disorder (ASD) diagnosis are often time consuming and tiresome both for highly-trained human evaluators and children, which may be alleviated by using humanoid robots in the diagnostic process. Hence, this paper proposes a framework for robot-assisted ASD evaluation based on partially observable Markov decision process (POMDP) modeling, specifically POMDPs with mixed observability (MOMDPs). POMDP is broadly used for modeling optimal sequential decision making tasks under uncertainty. Spurred by the widely accepted autism diagnostic observation schedule (ADOS), we emulate ADOS through four tasks, whose models incorporate observations of multiple social cues such as eye contact, gestures and utterances. Relying only on those observations, the robot provides an assessment of the child’s ASD-relevant functioning level (which is partially observable) within a particular task and provides human evaluators with readable information by partitioning its belief space. Finally, we evaluate the proposed MOMDP task models and demonstrate that chaining the tasks provides fine-grained outcome quantification, which could also increase the appeal of robot-assisted diagnostic protocols in the future.

Deterministic policies based on maximum regrets in MDPs with imprecise rewards

2021 ◽  
pp. 1-16
Author(s):  
Pegah Alizadeh ◽  
Emiliano Traversi ◽  
Aomar Osmani
Keyword(s):  
Decision Making ◽  
Decision Process ◽  
Process Models ◽  
Sequential Decision Making ◽  
Sequential Decision ◽  
Exact Procedure ◽  
Markov Decision ◽  
Intuitive Idea ◽  
First Time ◽  
Maximum Regret

Markov Decision Process Models (MDPs) are a powerful tool for planning tasks and sequential decision-making issues. In this work we deal with MDPs with imprecise rewards, often used when dealing with situations where the data is uncertain. In this context, we provide algorithms for finding the policy that minimizes the maximum regret. To the best of our knowledge, all the regret-based methods proposed in the literature focus on providing an optimal stochastic policy. We introduce for the first time a method to calculate an optimal deterministic policy using optimization approaches. Deterministic policies are easily interpretable for users because for a given state they provide a unique choice. To better motivate the use of an exact procedure for finding a deterministic policy, we show some (theoretical and experimental) cases where the intuitive idea of using a deterministic policy obtained after “determinizing” the optimal stochastic policy leads to a policy far from the exact deterministic policy.

CoreALMlib: An library translated from the Component Library

2016 ◽  
Vol 16 (5-6) ◽  
pp. 800-816 ◽  
Author(s):  
DANIELA INCLEZAN
Keyword(s):  
Natural Language ◽  
Answer Set Programming ◽  
Natural Language Understanding ◽  
Inference Engine ◽  
Lexical Resources ◽  
Language Understanding ◽  
Commonsense Knowledge ◽  
Component Library ◽  
Action Language ◽  
Answer Set

AbstractThis paper presents CoreALMlib, an $\mathscr{ALM}$ library of commonsense knowledge about dynamic domains. The library was obtained by translating part of the Component Library (CLib) into the modular action language $\mathscr{ALM}$. CLib consists of general reusable and composable commonsense concepts, selected based on a thorough study of ontological and lexical resources. Our translation targets CLibstates (i.e., fluents) and actions. The resulting $\mathscr{ALM}$ library contains the descriptions of 123 action classes grouped into 43 reusable modules that are organized into a hierarchy. It is made available online and of interest to researchers in the action language, answer-set programming, and natural language understanding communities. We believe that our translation has two main advantages over its CLib counterpart: (i) it specifies axioms about actions in a more elaboration tolerant and readable way, and (ii) it can be seamlessly integrated with ASP reasoning algorithms (e.g., for planning and postdiction). In contrast, axioms are described in CLib using STRIPS-like operators, and CLib's inference engine cannot handle planning nor postdiction.

scholarly journals Modular action language

2015 ◽  
Vol 16 (2) ◽  
pp. 189-235 ◽  
Author(s):  
DANIELA INCLEZAN ◽  
MICHAEL GELFOND
Keyword(s):  
Logic Programming ◽  
Medium Size ◽  
Programming System ◽  
Problem Structuring ◽  
System Description ◽  
Action Languages ◽  
Special Cases ◽  
Action Language ◽  
Stepwise Development ◽  
High Level

AbstractThe paper introduces a new modular action language,${\mathcal ALM}$, and illustrates the methodology of its use. It is based on the approach of Gelfond and Lifschitz (1993,Journal of Logic Programming 17, 2–4, 301–321; 1998,Electronic Transactions on AI 3, 16, 193–210) in which a high-level action language is used as a front end for a logic programming system description. The resulting logic programming representation is used to perform various computational tasks. The methodology based on existing action languages works well for small and even medium size systems, but is not meant to deal with larger systems that requirestructuring of knowledge.$\mathcal{ALM}$is meant to remedy this problem. Structuring of knowledge in${\mathcal ALM}$is supported by the concepts ofmodule(a formal description of a specific piece of knowledge packaged as a unit),module hierarchy, andlibrary, and by the division of a system description of${\mathcal ALM}$into two parts:theoryandstructure. Atheoryconsists of one or more modules with a common theme, possibly organized into a module hierarchy based on adependency relation. It contains declarations of sorts, attributes, and properties of the domain together with axioms describing them.Structuresare used to describe the domain's objects. These features, together with the means for defining classes of a domain as special cases of previously defined ones, facilitate the stepwise development, testing, and readability of a knowledge base, as well as the creation of knowledge representation libraries.

Action language ℬ𝒞+

2015 ◽  
Vol 30 (4) ◽  
pp. 899-922 ◽  
Author(s):  
Joseph Babb ◽  
Joohyung Lee
Keyword(s):  
Main Idea ◽  
Answer Set Programming ◽  
Stable Model ◽  
Formal Models ◽  
Transition Systems ◽  
Action Languages ◽  
Action Language ◽  
Language Constructs ◽  
High Level ◽  
Answer Set

Abstract Action languages are formal models of parts of natural language that are designed to describe effects of actions. Many of these languages can be viewed as high-level notations of answer set programs structured to represent transition systems. However, the form of answer set programs considered in the earlier work is quite limited in comparison with the modern Answer Set Programming (ASP) language, which allows several useful constructs for knowledge representation, such as choice rules, aggregates and abstract constraint atoms. We propose a new action language called BC +, which closes the gap between action languages and the modern ASP language. The main idea is to define the semantics of BC + in terms of general stable model semantics for propositional formulas, under which many modern ASP language constructs can be identified with shorthands for propositional formulas. Language BC  + turns out to be sufficiently expressive to encompass the best features of other action languages, such as languages B , C , C + and BC . Computational methods available in ASP solvers are readily applicable to compute BC +, which led to an implementation of the language by extending system cplus2asp .

Declarative Planning and Knowledge Representation in an Action Language

2011 ◽  
pp. 192-221 ◽  
Author(s):  
Thomas Eiter ◽  
Wolfgang Faber ◽  
Gerald Pfeifer
Keyword(s):  
Knowledge Representation ◽  
Logic Programming ◽  
Planning System ◽  
Epistemic State ◽  
Action Languages ◽  
Knowledge States ◽  
Action Language ◽  
High Level ◽  
Planning Problems ◽  
Complex Planning

This chapter introduces planning and knowledge representation in the declarative action language K. Rooted in the area of Knowledge Representation & Reasoning, action languages like K allow the formalization of complex planning problems involving non-determinism and incomplete knowledge in a very flexible manner. By giving an overview of existing planning languages and comparing these against our language, we aim on further promoting the applicability and usefulness of high-level action languages in the area of planning. As opposed to previously existing languages for modeling actions and change, K adopts a logic programming view where fluents representing the epistemic state of an agent might be true, false or undefined in each state. We will show that this view of knowledge states can be fruitfully applied to several well-known planning domains from the literature as well as novel planning domains. Remarkably, K often allows to model problems more concisely than previous action languages. All the examples given can be tested in an available implementation, the DLVK planning system.

Sign in / Sign up

Export Citation Format

Share Document