scholarly journals Planning and Acting with Non-Deterministic Events: Navigating between Safe States

2020 ◽  
Vol 34 (06) ◽  
pp. 9802-9809
Author(s):  
Lukas Chrpa ◽  
Jakub Gemrot ◽  
Martin Pilat
Keyword(s):  
Real World ◽  
Automated Planning ◽  
Dynamic Environments ◽  
Initial State ◽  
Goal State

Automated Planning addresses the problem of finding a sequence of actions, a plan, transforming the environment from its initial state to some goal state. In real-world environments, exogenous events might occur and might modify the environment without agent's consent. Besides disrupting agent's plan, events might hinder agent's pursuit towards its goals and even cause damage (e.g. destroying the robot).In this paper, we leverage the notion of Safe States in dynamic environments under presence of non-deterministic exogenous events that might eventually cause dead-ends (e.g. “damage” the agent) if the agent is not careful while executing its plan. We introduce a technique for generating plans that constrains the number of consecutive “unsafe” actions in a plan and a technique for generating “robust” plans that effectively evade event effects. Combination of both approaches plans and executes robust plans between safe states. We empirically show that such an approach effectively navigates the agent towards its goals in spite of presence of dead-ends.

scholarly journals Effective grounding for hybrid planning problems represented in PDDL+

2021 ◽  
Vol 36 ◽  
Author(s):  
Enrico Scala ◽  
Mauro Vallati
Keyword(s):  
Real World ◽  
Automated Planning ◽  
Search Space ◽  
Language Models ◽  
State Variables ◽  
Initial State ◽  
Discrete State ◽  
Goal State ◽  
Search Spaces ◽  
Planning Problems

Abstract Automated planning is the field of Artificial Intelligence (AI) that focuses on identifying sequences of actions allowing to reach a goal state from a given initial state. The need of using such techniques in real-world applications has brought popular languages for expressing automated planning problems to provide direct support for continuous and discrete state variables, along with changes that can be either instantaneous or durative. PDDL+ (Planning Domain Definition Language +) models support the encoding of such representations, but the resulting planning problems are notoriously difficult for AI planners to cope with due to non-linear dependencies arising from the variables and infinite search spaces. This difficulty is exacerbated by the potentially huge fully ground representations used by modern planners in order to effectively explore the search space, which can make some problems impossible to tackle. This paper investigates two grounding techniques for PDDL+ problems, both aimed at reducing the size of the full ground representation by reasoning over the lifted, more abstract problem structure. The first method extends the simple mechanism of invariant analysis to limit the groundings of operators upfront. The second method proposes to tackle the grounding process through a PDDL+ to classical planning abstraction; this allows us to leverage the amount of research done in the classical planning area. Our empirical analysis studies the effect of these novel approaches over both real-world hybrid applications and synthetic PDDL+ problems took from standard benchmarks of the planning community; our results reveal that not only the techniques improve the running time of previous grounding mechanisms but also let the planner extend the reach to problems that were not solvable before.

Robust and Accurate Monocular Vision-Based Localization in Outdoor Environments of Real-World Robot Challenge

2017 ◽  
Vol 29 (4) ◽  
pp. 685-696 ◽  
Author(s):  
Adi Sujiwo ◽  
Eijiro Takeuchi ◽  
Luis Yoichi Morales ◽  
Naoki Akai ◽  
Hatem Darweesh ◽  
...  
Keyword(s):  
Real World ◽  
Dynamic Environments ◽  
Monocular Vision ◽  
Experimental Results ◽  
Data Sets ◽  
Place Recognition ◽  
Global Localization ◽  
Vision Based Navigation ◽  
Outdoor Environments ◽  
Vision Based Localization

This paper describes our approach to perform robust monocular camera metric localization in the dynamic environments of Tsukuba Challenge 2016. We address two issues related to vision-based navigation. First, we improved the coverage by building a custom vocabulary out of the scene and improving upon place recognition routine which is key for global localization. Second, we established possibility of lifelong localization by using previous year’s map. Experimental results show that localization coverage was higher than 90% for six different data sets taken in different years, while localization average errors were under 0.2 m. Finally, the average of coverage for data sets tested with maps taken in different years was of 75%.

CMCABC: Clustering and Memory-Based Chaotic Artificial Bee Colony Dynamic Optimization Algorithm

2018 ◽  
Vol 17 (04) ◽  
pp. 1007-1046 ◽  
Author(s):  
Mohsen Moradi ◽  
Samad Nejatian ◽  
Hamid Parvin ◽  
Vahideh Rezaie
Keyword(s):  
Dynamic Optimization ◽  
Real World ◽  
Artificial Bee Colony ◽  
Optimization Problems ◽  
Optimization Algorithms ◽  
Dynamic Environments ◽  
Dynamic Optimization Problems ◽  
The Real ◽  
Bee Colony ◽  
Swarm Intelligence Optimization

The swarm intelligence optimization algorithms are used widely in static purposes and applications. They solve the static optimization problems successfully. However, most of the recent optimization problems in the real world have a dynamic nature. Thus, an optimization algorithm is required to solve the problems in dynamic environments as well. The dynamic optimization problems indicate the ones whose solutions change over time. The artificial bee colony algorithm is one of the swarm intelligence optimization algorithms. In this study, a clustering and memory-based chaotic artificial bee colony algorithm, denoted by CMCABC, has been proposed for solving the dynamic optimization problems. A chaotic system has a more accurate prediction for future in the real-world applications compared to a random system, because in the real-world chaotic behaviors have emerged, but random behaviors havenot been observed. In the proposed CMCABC method, explicit memory has been used to save the previous good solutions which are not very old. Maintaining diversity in the dynamic environments is one of the fundamental challenges while solving the dynamic optimization problems. Using clustering technique in the proposed method can well maintain the diversity of the problem environment. The proposed CMCABC method has been tested on the moving peaks benchmark (MPB). The MPB is a good simulator to evaluate the efficiency of the optimization algorithms in dynamic environments. The experimental results on the MPB reveal the appropriate efficiency of the proposed CMCABC method compared to the other state-of-the-art methods in solving dynamic optimization problems.

Learning-Based Planning

2011 ◽  
pp. 1024-1028
Author(s):  
Sergio Jiménez Celorrio ◽  
Tomás de la Rosa Turbides
Keyword(s):  
Automated Planning ◽  
Transition Function ◽  
Initial State ◽  
Main Research ◽  
Complete Problem ◽  
Current Trends ◽  
Action Sequences ◽  
The World ◽  
State Transition Function ◽  
Domain Independent

Automated Planning (AP) studies the generation of action sequences for problem solving. A problem in AP is defined by a state-transition function describing the dynamics of the world, the initial state of the world and the goals to be achieved. According to this definition, AP problems seem to be easily tackled by searching for a path in a graph, which is a well-studied problem. However, the graphs resulting from AP problems are so large that explicitly specifying them is not feasible. Thus, different approaches have been tried to address AP problems. Since the mid 90’s, new planning algorithms have enabled the solution of practical-size AP problems. Nevertheless, domain-independent planners still fail in solving complex AP problems, as solving planning tasks is a PSPACE-Complete problem (Bylander, 94). How do humans cope with this planning-inherent complexity? One answer is that our experience allows us to solve problems more quickly; we are endowed with learning skills that help us plan when problems are selected from a stable population. Inspire by this idea, the field of learning-based planning studies the development of AP systems able to modify their performance according to previous experiences. Since the first days, Artificial Intelligence (AI) has been concerned with the problem of Machine Learning (ML). As early as 1959, Arthur L. Samuel developed a prominent program that learned to improve its play in the game of checkers (Samuel, 1959). It is hardly surprising that ML has often been used to make changes in systems that perform tasks associated with AI, such as perception, robot control or AP. This article analyses the diverse ways ML can be used to improve AP processes. First, we review the major AP concepts and summarize the main research done in learning-based planning. Second, we describe current trends in applying ML to AP. Finally, we comment on the next avenues for combining AP and ML and conclude.

Applications in Noisy and Dynamic Environments

2010 ◽  
pp. 222-244
Author(s):  
E. Parsopoulos Konstantinos ◽  
N. Vrahatis Michael
Keyword(s):  
Real World ◽  
Particle Filtering ◽  
Dynamic Environments ◽  
Experimental Results ◽  
Particle Identification ◽  
Special Importance ◽  
Extensive Literature ◽  
Research Fields ◽  
System States ◽  
Problem Formulations

This chapter discusses the workings of PSO in two research fields with special importance in real-world applications, namely noisy and dynamic environments. Noise simulation schemes are presented and experimental results on benchmark problems are reported. In addition, we present the application of PSO on a simulated real world problem, namely the particle identification by light scattering. Moreover, a hybrid scheme that incorporates PSO in particle filtering methods to estimate system states online is analyzed, and representative experimental results are reported. Finally, the combination of noisy and continuously changing environments is shortly discussed, providing illustrative graphical representations of performance for different PSO variants. The text focuses on providing the basic concepts and problem formulations, and suggesting experimental settings reported in literature, rather than on the bibliographical presentation of the (prohibitively extensive) literature.

scholarly journals Emotion Learning: Solving a Shortest Path Problem in an Arbitrary Deterministic Environment in Linear Time with an Emotional Agent

2008 ◽  
Vol 18 (3) ◽  
pp. 409-421 ◽  
Author(s):  
Silvana Petruseva
Keyword(s):  
Shortest Path ◽  
Emotional State ◽  
Linear Time ◽  
Decisive Role ◽  
Time Algorithm ◽  
Shortest Path Problem ◽  
Adaptive Array ◽  
Agent Architecture ◽  
Initial State ◽  
Goal State

Emotion Learning: Solving a Shortest Path Problem in an Arbitrary Deterministic Environment in Linear Time with an Emotional AgentThe paper presents an algorithm which solves the shortest path problem in an arbitrary deterministic environment withnstates with an emotional agent in linear time. The algorithm originates from an algorithm which in exponential time solves the same problem, and the agent architecture used for solving the problem is an NN-CAA architecture (neural network crossbar adaptive array). By implementing emotion learning, the linear time algorithm is obtained and the agent architecture is modified. The complexity of the algorithm without operations for initiation in general does not depend on the number of statesn, but only on the length of the shortest path. Depending on the position of the goal state, the complexity can be at mostO (n).It can be concluded that the choice of the function which evaluates the emotional state of the agent plays a decisive role in solving the problem efficiently. That function should give as detailed information as possible about the consequences of the agent's actions, starting even from the initial state. In this way the function implements properties of human emotions.

Generation of Probabilistically Optimal Paths for Nonholonomic Needle Insertion

2016 ◽  
Author(s):  
Jianping Lin ◽  
Wooram Park
Keyword(s):  
Cost Function ◽  
Optimal Path ◽  
Target Position ◽  
Nonholonomic Constraints ◽  
Needle Insertion ◽  
Initial State ◽  
Goal State ◽  
Medical Needles ◽  
The Cost ◽  
Planning Problems

Rapidly-exploring Random Tree (RRT) is a sampling-based algorithm which is designed for path planning problems. It is efficient to handle high-dimensional configuration space (C-space) and nonholonomic constraints. Under the nonholonomic constraints, the RRT can generate paths between an initial state and a goal state while avoiding obstacles. Since this framework assumes that a system is deterministic, more improvement should be added when the method is applied to a system with uncertainty. In robotic systems with motion uncertainty, probability for successful targeting and obstacle avoidance are more suitable measurement than the deterministic distance between the robot system and the target position. In this paper, the probabilistic targeting error is defined as a root-mean-square (RMS) distance between the system to the desired target. The proximity of the obstacle to the system is also defined as an averaged distance of obstacles to the robotic system. Then, we consider a cost function that is a sum of the targeting error and the obstacle proximity. By numerically minimizing the cost, we can obtain the optimal path. In this paper, a method for efficient evaluation and minimization of this cost function is proposed and the proposed method is applied to nonholonomic flexible medical needles for performance tests.

scholarly journals Deep reinforcement learning for map-less goal-driven robot navigation

2021 ◽  
Vol 18 (1) ◽  
pp. 172988142199262
Author(s):  
Matej Dobrevski ◽  
Danijel Skočaj
Keyword(s):  
Reinforcement Learning ◽  
Path Planning ◽  
Mobile Robots ◽  
Real World ◽  
Robot Navigation ◽  
Dynamic Environments ◽  
Learning Framework ◽  
Local Navigation ◽  
Real Robot ◽  
Navigation Method

Mobile robots that operate in real-world environments need to be able to safely navigate their surroundings. Obstacle avoidance and path planning are crucial capabilities for achieving autonomy of such systems. However, for new or dynamic environments, navigation methods that rely on an explicit map of the environment can be impractical or even impossible to use. We present a new local navigation method for steering the robot to global goals without relying on an explicit map of the environment. The proposed navigation model is trained in a deep reinforcement learning framework based on Advantage Actor–Critic method and is able to directly translate robot observations to movement commands. We evaluate and compare the proposed navigation method with standard map-based approaches on several navigation scenarios in simulation and demonstrate that our method is able to navigate the robot also without the map or when the map gets corrupted, while the standard approaches fail. We also show that our method can be directly transferred to a real robot.

scholarly journals A Novel Model-Based Reinforcement Learning Attitude Control Method for Virtual Reality Satellite

2021 ◽  
Vol 2021 ◽  
pp. 1-11
Author(s):  
Jian Zhang ◽  
Fengge Wu
Keyword(s):  
Virtual Reality ◽  
Reinforcement Learning ◽  
Real World ◽  
Attitude Control ◽  
Mixed Model ◽  
Control Method ◽  
Sensor Data ◽  
Dynamics Model ◽  
Goal State ◽  
The Impact

Observing the universe with virtual reality satellite is an amazing experience. An intelligent method of attitude control is the core object of research to achieve this goal. Attitude control is essentially one of the goal-state reaching tasks under constraints. Using reinforcement learning methods in real-world systems faces many challenges, such as insufficient samples, exploration safety issues, unknown actuator delays, and noise in the raw sensor data. In this work, a mixed model with different input sizes was proposed to represent the environmental dynamics model. The predication accuracy of the environmental dynamics model and the performance of the policy trained in this paper were gradually improved. Our method reduces the impact of noisy data on the model’s accuracy and improves the sampling efficiency. The experiments showed that the agent trained with our method completed a goal-state reaching task in a real-world system under wireless circumstances whose actuators were reaction wheels, whereas the soft actor-critic method failed in the same training process. The method’s effectiveness is ensured theoretically under given conditions.

scholarly journals Penerapan Metode Best First Search Pada Permainan Tic Tac Toe

2017 ◽  
pp. 1-6
Author(s):  
Harvei Desmon Hutahaean
Keyword(s):  
Problem Solving ◽  
State Space ◽  
Directed Graph ◽  
The State ◽  
Initial State ◽  
Problem Space ◽  
Goal State ◽  
Best First Search ◽  
Problem Solving Behavior

Search is the process of finding solutions in a problem until a solution or goal is found, or a movement in the state-space to search for trajectories from initial-state to goal-state. In a TIC TAC Toe game the process of finding a space situation is not enough to automate problem-solving behavior, in each of these situations there are only a limited number of choices that a player can make. The problems that will be faced can be solved by searching from the choices available, supported by the usual way of resolving. Best First Search works by searching for a directed graph which each node represents a point in a problem space.

Sign in / Sign up

Export Citation Format

Share Document