scholarly journals Managing caching strategies for stream reasoning with reinforcement learning

2020 ◽  
Vol 20 (5) ◽  
pp. 625-640
Author(s):  
CARMINE DODARO ◽  
THOMAS EITER ◽  
PAUL OGRIS ◽  
KONSTANTIN SCHEKOTIHIN
Keyword(s):  
Reinforcement Learning ◽  
Real World ◽  
Performance Improvements ◽  
Novel Approach ◽  
Stream Reasoning ◽  
Significant Performance ◽  
Intelligent Management ◽  
Truth Maintenance ◽  
Constraint Learning ◽  
Real World Problems

AbstractEfficient decision-making over continuously changing data is essential for many application domains such as cyber-physical systems, industry digitalization, etc. Modern stream reasoning frameworks allow one to model and solve various real-world problems using incremental and continuous evaluation of programs as new data arrives in the stream. Applied techniques use, e.g., Datalog-like materialization or truth maintenance algorithms to avoid costly re-computations, thus ensuring low latency and high throughput of a stream reasoner. However, the expressiveness of existing approaches is quite limited and, e.g., they cannot be used to encode problems with constraints, which often appear in practice. In this paper, we suggest a novel approach that uses the Conflict-Driven Constraint Learning (CDCL) to efficiently update legacy solutions by using intelligent management of learned constraints. In particular, we study the applicability of reinforcement learning to continuously assess the utility of learned constraints computed in previous invocations of the solving algorithm for the current one. Evaluations conducted on real-world reconfiguration problems show that providing a CDCL algorithm with relevant learned constraints from previous iterations results in significant performance improvements of the algorithm in stream reasoning scenarios.

Computer Vision Based Asset Surveillance for Smart Buildings

2020 ◽  
Vol 17 (1) ◽  
pp. 456-463
Author(s):  
K. S. Gautam ◽  
Latha Parameswaran ◽  
Senthil Kumar Thangavel
Keyword(s):  
Computer Vision ◽  
Correlation Coefficient ◽  
Real World ◽  
Vision System ◽  
Region Of Interest ◽  
Detection And Tracking ◽  
Novel Approach ◽  
Ip Camera ◽  
Pros And Cons ◽  
Real World Problems

Unraveling meaningful pattern form the video offers a solution to many real-world problems, especially surveillance and security. Detecting and tracking an object under the area of video surveillance, not only automates the security but also leverages smart nature of the buildings. The objective of the manuscript is to detect and track assets inside the building using vision system. In this manuscript, the strategies involved in asset detection and tracking are discussed with their pros and cons. In addition to it, a novel approach has been proposed that detects and tracks the object of interest across all the frames using correlation coefficient. The proposed approach is said to be significant since the user has an option to select the object of interest from any two frames in the video and correlation coefficient is calculated for the region of interest. Based on the arrived correlation coefficient the object of interest is tracked across the rest of the frames. Experimentation is carried out using the 10 videos acquired from IP camera inside the building.

scholarly journals Aspect-Based Sentiment Analysis Using Smart Company and Hotel Aspect Review Data

2020 ◽  
pp. 112-118
Author(s):  
C. Selvi ◽  
Niveda. C. P
Keyword(s):  
Knowledge Acquisition ◽  
Sentiment Analysis ◽  
Language Processing ◽  
Real World ◽  
Performance Improvements ◽  
Information Navigation ◽  
Real World Applications ◽  
Significant Performance ◽  
Processing Techniques ◽  
Online Feedback

Digital sources such as smart applications opinions and online feedback statistics are crucial resources to be seeking for customers’ remarks and input. However, the reviews are often disorganized, leading to difficulties in information navigation and knowledge acquisition. The aforementioned problem is overcome by generating aspect-sentiment based embedding for the hotels and companies by looking into reliable reviews of them. The important product aspects are identified based on two observations: 1) the important aspects are usually commented on by a large number of consumers and 2) consumer opinions on the important aspects greatly influence their overall opinions. Aspect frequency and the influence of consumer opinions given to each aspect over their overall opinions are identified for hotel reviews whereas for company reviews approach adopts language processing techniques, policies, and lexicons to address several sentiment evaluation challenges, and convey summarized results. Moreover, aspect ranking achieve significant performance improvements, which demonstrate the capacity of aspect ranking in facilitating real-world applications.

scholarly journals Improving Neural Relation Extraction with Positive and Unlabeled Learning

2020 ◽  
Vol 34 (05) ◽  
pp. 7927-7934
Author(s):  
Zhengqiu He ◽  
Wenliang Chen ◽  
Yuyi Wang ◽  
Wei Zhang ◽  
Guanchun Wang ◽  
...  
Keyword(s):  
Reinforcement Learning ◽  
Real World ◽  
Relation Extraction ◽  
Experimental Results ◽  
New Approach ◽  
Distant Supervision ◽  
Novel Approach ◽  
Pu Learning ◽  
Positive And Unlabeled Learning

We present a novel approach to improve the performance of distant supervision relation extraction with Positive and Unlabeled (PU) Learning. This approach first applies reinforcement learning to decide whether a sentence is positive to a given relation, and then positive and unlabeled bags are constructed. In contrast to most previous studies, which mainly use selected positive instances only, we make full use of unlabeled instances and propose two new representations for positive and unlabeled bags. These two representations are then combined in an appropriate way to make bag-level prediction. Experimental results on a widely used real-world dataset demonstrate that this new approach indeed achieves significant and consistent improvements as compared to several competitive baselines.

Investigating the streamability of XProc pipelines

2009 ◽  
Author(s):  
Norman Walsh
Keyword(s):  
Real World ◽  
Performance Improvements ◽  
Significant Performance ◽  
Using Data ◽  
Open Question

XProc: An XML Pipeline Language is a specification being developed by the W3C for describing a sequence of XML operations performed over a set of input documents. Not all of the steps in XProc are known to streamable and consequently, the XProc specification does not require implementations to support streaming. It's an open question whether or not a streaming implementation would be likely to achieve significant performance improvements over a similar non-streaming implementation. Using data collected from real-world pipelines, this paper examines that question.

Chapter 24. Maximum Satisfiabiliy

2021 ◽  
Author(s):  
Fahiem Bacchus ◽  
Matti Järvisalo ◽  
Ruben Martins
Keyword(s):  
Objective Function ◽  
Real World ◽  
Optimization Problems ◽  
Truth Assignment ◽  
Maximum Satisfiability ◽  
Performance Improvements ◽  
Algorithm Portfolios ◽  
Recent Developments ◽  
Significant Performance ◽  
Objective Value

Maximum satisfiability (MaxSAT) is an optimization version of SAT that is solved by finding an optimal truth assignment instead of just a satisfying one. In MaxSAT the objective function to be optimized is specified by a set of weighted soft clauses: the objective value of a truth assignment is the sum of the weights of the soft clauses it satisfies. In addition, the MaxSAT problem can have hard clauses that the truth assignment must satisfy. Many optimization problems can be naturally encoded into MaxSAT and this, along with significant performance improvements in MaxSAT solvers, has led to MaxSAT being used in a number of different application areas. This chapter provides a detailed overview of the approaches to MaxSAT solving that have in recent years been most successful in solving real-world optimization problems. Further recent developments in MaxSAT research are also overviewed, including encodings, applications, preprocessing, incomplete solving, algorithm portfolios, partitioning-based solving, and parallel solving.

scholarly journals A Decision-Theoretic Model of Assistance

2014 ◽  
Vol 50 ◽  
pp. 71-104 ◽  
Author(s):  
A. Fern ◽  
S. Natarajan ◽  
K. Judah ◽  
P. Tadepalli
Keyword(s):  
Real World ◽  
Human Subjects ◽  
Theoretic Model ◽  
Empirical Results ◽  
Novel Approach ◽  
Myopic Policy ◽  
Optimal Action ◽  
Selection For ◽  
Intelligent Assistance ◽  
Real World Problems

There is a growing interest in intelligent assistants for a variety of applications from sorting email to helping people with disabilities to do their daily chores. In this paper, we formulate the problem of intelligent assistance in a decision-theoretic framework, and present both theoretical and empirical results. We first introduce a class of POMDPs called hidden-goal MDPs (HGMDPs), which formalizes the problem of interactively assisting an agent whose goal is hidden and whose actions are observable. In spite of its restricted nature, we show that optimal action selection for HGMDPs is PSPACE-complete even for deterministic dynamics. We then introduce a more restricted model called helper action MDPs (HAMDPs), which are sufficient for modeling many real-world problems. We show classes of HAMDPs for which efficient algorithms are possible. More interestingly, for general HAMDPs we show that a simple myopic policy achieves a near optimal regret, compared to an oracle assistant that knows the agent's goal. We then introduce more sophisticated versions of this policy for the general case of HGMDPs that we combine with a novel approach for quickly learning about the agent being assisted. We evaluate our approach in two game-like computer environments where human subjects perform tasks, and in a real-world domain of providing assistance during folder navigation in a computer desktop environment. The results show that in all three domains the framework results in an assistant that substantially reduces user effort with only modest computation.

scholarly journals Multi-Agent Systems of Inverse Reinforcement Learners in Complex Games

2017 ◽  
Author(s):  
Dave Mobley
Keyword(s):  
Reinforcement Learning ◽  
Real World ◽  
Computer Games ◽  
Computer Game ◽  
Role Playing ◽  
Multi Agent Systems ◽  
Role Playing Games ◽  
Starting Point ◽  
Multi Agent ◽  
Real World Problems

Real-world problems exhibit a few defining criteria that make them hard for computers to solve. Problems such as driving a car or flying a helicopter have primary goals of reaching a destination as well as doing it safely and timely. These problems must each manage many resources and tasks to achieve their primary goals. The tasks themselves are made up of states that are represented by variables or features. As the feature set grows, the problems become intractable. Computer games are smaller problems but also are representative of real-world problems of this type. In my research, I will look at a particular class of computer game, namely computer role-playing games (RPGs), which are made up of a collection of overarching goals such as improving the player avatar, navigating a virtual world, and keeping the avatar alive. While playing there are also subtasks such as combatting other characters and managing inventory which are not primary, but yet important to overall game play. I will be exploring tiered Reinforcement Learning techniques coupled with training from expert policies using Inverse Reinforcement Learning as a starting point on learning how to play a complex game while attempting to extrapolate ideal goals and rewards.

scholarly journals Leveraging Human Knowledge in Tabular Reinforcement Learning: A Study of Human Subjects

2017 ◽  
Author(s):  
Ariel Rosenfeld ◽  
Matthew E. Taylor ◽  
Sarit Kraus
Keyword(s):  
Reinforcement Learning ◽  
Human Factors ◽  
Real World ◽  
Human Subjects ◽  
Human Knowledge ◽  
Human Psychology ◽  
Novel Method ◽  
Extensive Effort ◽  
Psychology Literature ◽  
Real World Problems

Reinforcement Learning (RL) can be extremely effective in solving complex, real-world problems. However, injecting human knowledge into an RL agent may require extensive effort on the human designer's part. To date, human factors are generally not considered in the development and evaluation of possible approaches. In this paper, we propose and evaluate a novel method, based on human psychology literature, which we show to be both effective and efficient, for both expert and non-expert designers, in injecting human knowledge for speeding up tabular RL.

Sign in / Sign up

Export Citation Format

Share Document