scholarly journals A Decision-Theoretic Model of Assistance

2014 ◽  
Vol 50 ◽  
pp. 71-104 ◽  
Author(s):  
A. Fern ◽  
S. Natarajan ◽  
K. Judah ◽  
P. Tadepalli
Keyword(s):  
Real World ◽  
Human Subjects ◽  
Theoretic Model ◽  
Empirical Results ◽  
Novel Approach ◽  
Myopic Policy ◽  
Optimal Action ◽  
Selection For ◽  
Intelligent Assistance ◽  
Real World Problems

There is a growing interest in intelligent assistants for a variety of applications from sorting email to helping people with disabilities to do their daily chores. In this paper, we formulate the problem of intelligent assistance in a decision-theoretic framework, and present both theoretical and empirical results. We first introduce a class of POMDPs called hidden-goal MDPs (HGMDPs), which formalizes the problem of interactively assisting an agent whose goal is hidden and whose actions are observable. In spite of its restricted nature, we show that optimal action selection for HGMDPs is PSPACE-complete even for deterministic dynamics. We then introduce a more restricted model called helper action MDPs (HAMDPs), which are sufficient for modeling many real-world problems. We show classes of HAMDPs for which efficient algorithms are possible. More interestingly, for general HAMDPs we show that a simple myopic policy achieves a near optimal regret, compared to an oracle assistant that knows the agent's goal. We then introduce more sophisticated versions of this policy for the general case of HGMDPs that we combine with a novel approach for quickly learning about the agent being assisted. We evaluate our approach in two game-like computer environments where human subjects perform tasks, and in a real-world domain of providing assistance during folder navigation in a computer desktop environment. The results show that in all three domains the framework results in an assistant that substantially reduces user effort with only modest computation.

Computer Vision Based Asset Surveillance for Smart Buildings

2020 ◽  
Vol 17 (1) ◽  
pp. 456-463
Author(s):  
K. S. Gautam ◽  
Latha Parameswaran ◽  
Senthil Kumar Thangavel
Keyword(s):  
Computer Vision ◽  
Correlation Coefficient ◽  
Real World ◽  
Vision System ◽  
Region Of Interest ◽  
Detection And Tracking ◽  
Novel Approach ◽  
Ip Camera ◽  
Pros And Cons ◽  
Real World Problems

Unraveling meaningful pattern form the video offers a solution to many real-world problems, especially surveillance and security. Detecting and tracking an object under the area of video surveillance, not only automates the security but also leverages smart nature of the buildings. The objective of the manuscript is to detect and track assets inside the building using vision system. In this manuscript, the strategies involved in asset detection and tracking are discussed with their pros and cons. In addition to it, a novel approach has been proposed that detects and tracks the object of interest across all the frames using correlation coefficient. The proposed approach is said to be significant since the user has an option to select the object of interest from any two frames in the video and correlation coefficient is calculated for the region of interest. Based on the arrived correlation coefficient the object of interest is tracked across the rest of the frames. Experimentation is carried out using the 10 videos acquired from IP camera inside the building.

scholarly journals Learning to Cooperate via an Attention-Based Communication Neural Network in Decentralized Multi-Robot Exploration

Entropy ◽  
2019 ◽  
Vol 21 (3) ◽  
pp. 294 ◽  
Author(s):  
Mingyang Geng ◽  
Kele Xu ◽  
Xing Zhou ◽  
Bo Ding ◽  
Huaimin Wang ◽  
...  
Keyword(s):  
Neural Network ◽  
Real World ◽  
Attention Mechanism ◽  
Empirical Results ◽  
The Past ◽  
Novel Approach ◽  
Exploration Task ◽  
Explicit Communication ◽  
Centralized Controller ◽  
Multi Robot

In a decentralized multi-robot exploration problem, the robots have to cooperate effectively to map a strange environment as soon as possible without a centralized controller. In the past few decades, a set of “human-designed” cooperation strategies have been proposed to address this problem, such as the well-known frontier-based approach. However, many real-world settings, especially the ones that are constantly changing, are too complex for humans to design efficient and decentralized strategies. This paper presents a novel approach, the Attention-based Communication neural network (CommAttn), to “learn” the cooperation strategies automatically in the decentralized multi-robot exploration problem. The communication neural network enables the robots to learn the cooperation strategies with explicit communication. Moreover, the attention mechanism we introduced additionally can precisely calculate whether the communication is necessary for each pair of agents by considering the relevance of each received message, which enables the robots to communicate only with the necessary partners. The empirical results on a simulated multi-robot disaster exploration scenario demonstrate that our proposal outperforms the traditional “human-designed” methods, as well as other competing “learning-based” methods in the exploration task.

scholarly journals Managing caching strategies for stream reasoning with reinforcement learning

2020 ◽  
Vol 20 (5) ◽  
pp. 625-640
Author(s):  
CARMINE DODARO ◽  
THOMAS EITER ◽  
PAUL OGRIS ◽  
KONSTANTIN SCHEKOTIHIN
Keyword(s):  
Reinforcement Learning ◽  
Real World ◽  
Performance Improvements ◽  
Novel Approach ◽  
Stream Reasoning ◽  
Significant Performance ◽  
Intelligent Management ◽  
Truth Maintenance ◽  
Constraint Learning ◽  
Real World Problems

AbstractEfficient decision-making over continuously changing data is essential for many application domains such as cyber-physical systems, industry digitalization, etc. Modern stream reasoning frameworks allow one to model and solve various real-world problems using incremental and continuous evaluation of programs as new data arrives in the stream. Applied techniques use, e.g., Datalog-like materialization or truth maintenance algorithms to avoid costly re-computations, thus ensuring low latency and high throughput of a stream reasoner. However, the expressiveness of existing approaches is quite limited and, e.g., they cannot be used to encode problems with constraints, which often appear in practice. In this paper, we suggest a novel approach that uses the Conflict-Driven Constraint Learning (CDCL) to efficiently update legacy solutions by using intelligent management of learned constraints. In particular, we study the applicability of reinforcement learning to continuously assess the utility of learned constraints computed in previous invocations of the solving algorithm for the current one. Evaluations conducted on real-world reconfiguration problems show that providing a CDCL algorithm with relevant learned constraints from previous iterations results in significant performance improvements of the algorithm in stream reasoning scenarios.

scholarly journals Leveraging Human Knowledge in Tabular Reinforcement Learning: A Study of Human Subjects

2017 ◽  
Author(s):  
Ariel Rosenfeld ◽  
Matthew E. Taylor ◽  
Sarit Kraus
Keyword(s):  
Reinforcement Learning ◽  
Human Factors ◽  
Real World ◽  
Human Subjects ◽  
Human Knowledge ◽  
Human Psychology ◽  
Novel Method ◽  
Extensive Effort ◽  
Psychology Literature ◽  
Real World Problems

Reinforcement Learning (RL) can be extremely effective in solving complex, real-world problems. However, injecting human knowledge into an RL agent may require extensive effort on the human designer's part. To date, human factors are generally not considered in the development and evaluation of possible approaches. In this paper, we propose and evaluate a novel method, based on human psychology literature, which we show to be both effective and efficient, for both expert and non-expert designers, in injecting human knowledge for speeding up tabular RL.

scholarly journals Handrails through the Swamp? A Pilot to Test the Integration and Implementation Science Framework in Complex Real-World Research

2021 ◽  
Vol 13 (10) ◽  
pp. 5491
Author(s):  
Melissa Robson-Williams ◽  
Bruce Small ◽  
Roger Robson-Williams ◽  
Nick Kirk
Keyword(s):  
Pilot Study ◽  
Case Studies ◽  
Implementation Science ◽  
Real World ◽  
Environmental Problems ◽  
Environmental Research ◽  
Integrative Framework ◽  
The World ◽  
Science Framework ◽  
Real World Problems

The socio-environmental challenges the world faces are ‘swamps’: situations that are messy, complex, and uncertain. The aim of this paper is to help disciplinary scientists navigate these swamps. To achieve this, the paper evaluates an integrative framework designed for researching complex real-world problems, the Integration and Implementation Science (i2S) framework. As a pilot study, we examine seven inter and transdisciplinary agri-environmental case studies against the concepts presented in the i2S framework, and we hypothesise that considering concepts in the i2S framework during the planning and delivery of agri-environmental research will increase the usefulness of the research for next users. We found that for the types of complex, real-world research done in the case studies, increasing attention to the i2S dimensions correlated with increased usefulness for the end users. We conclude that using the i2S framework could provide handrails for researchers, to help them navigate the swamps when engaging with the complexity of socio-environmental problems.

scholarly journals Combined Games with Randomly Delayed Beginnings

Mathematics ◽  
2021 ◽  
Vol 9 (5) ◽  
pp. 534
Author(s):  
F. Thomas Bruss
Keyword(s):  
Discrete Time ◽  
Real World ◽  
Optimal Stopping ◽  
Random Variables ◽  
Approximate Solutions ◽  
Real World Problems

This paper presents two-person games involving optimal stopping. As far as we are aware, the type of problems we study are new. We confine our interest to such games in discrete time. Two players are to chose, with randomised choice-priority, between two games G1 and G2. Each game consists of two parts with well-defined targets. Each part consists of a sequence of random variables which determines when the decisive part of the game will begin. In each game, the horizon is bounded, and if the two parts are not finished within the horizon, the game is lost by definition. Otherwise the decisive part begins, on which each player is entitled to apply their or her strategy to reach the second target. If only one player achieves the two targets, this player is the winner. If both win or both lose, the outcome is seen as “deuce”. We motivate the interest of such problems in the context of real-world problems. A few representative problems are solved in detail. The main objective of this article is to serve as a preliminary manual to guide through possible approaches and to discuss under which circumstances we can obtain solutions, or approximate solutions.

Review of Network Flow Algorithms David P. Williamson

2021 ◽  
Vol 52 (1) ◽  
pp. 12-15
Author(s):  
S.V. Nagaraj
Keyword(s):  
Graduate Students ◽  
Real World ◽  
Network Flow ◽  
Network Flows ◽  
Optimization Problems ◽  
Capacity Constraints ◽  
Incoming Flow ◽  
Network Flow Problems ◽  
Flow Problems ◽  
Real World Problems

This book is on algorithms for network flows. Network flow problems are optimization problems where given a flow network, the aim is to construct a flow that respects the capacity constraints of the edges of the network, so that incoming flow equals the outgoing flow for all vertices of the network except designated vertices known as the source and the sink. Network flow algorithms solve many real-world problems. This book is intended to serve graduate students and as a reference. The book is also available in eBook (ISBN 9781316952894/US$ 32.00), and hardback (ISBN 9781107185890/US$99.99) formats. The book has a companion web site www.networkflowalgs.com where a pre-publication version of the book can be downloaded gratis.

Sign in / Sign up

Export Citation Format

Share Document