Partially Observable Markov Decision Process Approximations for Adaptive Sensing

2009 ◽  
Vol 19 (3) ◽  
pp. 377-422 ◽  
Author(s):  
Edwin K. P. Chong ◽  
Christopher M. Kreucher ◽  
Alfred O. Hero
Keyword(s):  
Markov Decision Process ◽  
Decision Process ◽  
Adaptive Sensing ◽  
Markov Decision ◽  
Partially Observable Markov ◽  
Partially Observable

scholarly journals Partially observable Markov Decision Process to prioritize software defects

2021 ◽  
Author(s):  
Shirin Akbarinasaji
Keyword(s):  
Markov Decision Process ◽  
Decision Process ◽  
Tracking System ◽  
Dependency Graph ◽  
Relative Importance ◽  
Bug Reports ◽  
Markov Decision ◽  
Partially Observable Markov ◽  
Partially Observable ◽  
Issue Tracking System

Background: Bug tracking systems receive many bug reports daily. Although the software quality team aims to identify and resolve these bugs, they are never able to fix all of the reported bugs in the issue tracking system before the release deadline. However, postponing the bug fixing may have some consequences. Prioritization of bug reports will help the software manager decide which bugs to fix and which bugs to postpone. Typically, bug reports are prioritized based on the severity, priority, time and effort for fixing, customer pressure, etc. Aim: Previous studies have shown that these factors may not be appropriate for prioritization. Therefore, relying on them to automate bug prioritization might be misleading. In this dissertation, we aim to prioritize bug reports with respect to the consequence of not fixing the bugs in terms of their relative importance in the issue tracking system. Method: In order to measure the relative importance of bugs in the issue tracking system, we propose the construction of a dependency graph based on the reported dependency-blocking information in the issue tracking system. Two metrics, namely depth and degree, are used to measure the relative importance of the bugs. However, there is uncertainty in the dependency graph structure as the dependency information is discovered manually and gradually. Owing to this uncertainty, prioritization of bugs in the descending order of depth and degree may be misleading. To handle the uncertainty, we propose a novel approach of a partially observable Markov decision process (POMDP) and partially observable Monte Carlo planning (POMCP). Result: To check the feasibility of the proposed approach, we analyzed seven years of data from an open source project, Firefox, and a commercial project. We compared the proposed policy with the developer policy, maximum policy, and random policy. Conclusion: The results suggest that software practitioners do not consider the relative importance of bugs in their current practice. The proposed framework can be combined with practitioners’ expertise to prioritize bugs more effectively and take the depth and degree of bugs into account. In practice, the POMDP framework with the POMCP planner can help practitioners sequentially select bugs to minimize the connectivity of the dependency graph.

scholarly journals COG-DICE: An Algorithm for Solving Continuous-Observation Dec-POMDPs

2017 ◽  
Author(s):  
Madison Clark-Turner ◽  
Christopher Amato
Keyword(s):  
Markov Decision Process ◽  
Real World ◽  
Decision Process ◽  
Extended Version ◽  
Continuous Observation ◽  
Solution Methods ◽  
Markov Decision ◽  
Multi Agent ◽  
Partially Observable Markov ◽  
Partially Observable

The decentralized partially observable Markov decision process (Dec-POMDP) is a powerful model for representing multi-agent problems with decentralized behavior. Unfortunately, current Dec-POMDP solution methods cannot solve problems with continuous observations, which are common in many real-world domains. To that end, we present a framework for representing and generating Dec-POMDP policies that explicitly include continuous observations. We apply our algorithm to a novel tagging problem and an extended version of a common benchmark, where it generates policies that meet or exceed the values of equivalent discretized domains without the need for finding an adequate discretization.

scholarly journals State Estimation for Mobile Robot using Partially Observable Markov Decision Process

1996 ◽  
Vol 8 (3) ◽  
pp. 272-277
Author(s):  
Daehee Kang ◽  
◽  
Hideki Hashimoto ◽  
Fumio Harashima
Keyword(s):  
Mobile Robot ◽  
Markov Decision Process ◽  
Decision Process ◽  
Dead Reckoning ◽  
Position Estimation ◽  
Estimation Errors ◽  
Markov Decision ◽  
Partially Observable Markov ◽  
Partially Observable ◽  
Robot Position

Dead Reckoning has been commonly used for position estimation. However, this method has inherent problems, one of the biggest being it always cumulates estimation errors. In this paper, we propose a new method to estimate a current mobile robot state using Partially Observable Markov Decision Process (POMDP). POMDP generalizes the Markov Decision Process (MDP) framework to the case where the agent must make its decisions in partial ignorance of its current situation. Here, the robot state means the robot position or current subgoal at which the mobile robot is located. It is shown that we will be able to estimate the mobile robot state precisely and robustly, even if the environment is changed slightly, through a case study.

scholarly journals Decision Making in Complex Multiagent Contexts: A Tale of Two Frameworks

AI Magazine ◽  
2012 ◽  
Vol 33 (4) ◽  
pp. 82 ◽  
Author(s):  
Prashant J. Doshi
Keyword(s):  
Decision Making ◽  
Markov Decision Process ◽  
Decision Process ◽  
Partial Information ◽  
Autonomous Systems ◽  
Relevant Research ◽  
Physical Context ◽  
Markov Decision ◽  
Partially Observable Markov ◽  
Partially Observable

Decision making is a key feature of autonomous systems. It involves choosing optimally between different lines of action in various information contexts that range from perfectly knowing all aspects of the decision problem to having just partial knowledge about it. The physical context often includes other interacting autonomous systems, typically called agents. In this article, I focus on decision making in a multiagent context with partial information about the problem. Relevant research in this complex but realistic setting has converged around two complementary, general frameworks and also introduced myriad specializations on its way. I put the two frameworks, decentralized partially observable Markov decision process (Dec-POMDP) and the interactive partially observable Markov decision process (I-POMDP), in context and review the foundational algorithms for these frameworks, while briefly discussing the advances in their specializations. I conclude by examining the avenues that research pertaining to these frameworks is pursuing.

Sign in / Sign up

Export Citation Format

Share Document