Chapter 2. Symbolic Reasoning in Latent Space: Classical Planning as an Example

Symbolic systems require hand-coded symbolic representation as input, resulting in a knowledge acquisition bottleneck. Meanwhile, although deep learning has achieved significant success in many fields, the knowledge is encoded in a subsymbolic representation which is incompatible with symbolic systems. To address the gap between the two fields, one has to solve Symbol Grounding problem: The question of how a machine can generate symbols automatically. We discuss our recent work called Latplan, an unsupervised architecture combining deep learning and classical planning. Given only an unlabeled set of image pairs showing a subset of transitions allowed in the environment (training inputs), Latplan learns a complete propositional PDDL action model of the environment. Later, when a pair of images representing the initial and the goal states (planning inputs) is given, Latplan finds a plan to the goal state in a symbolic latent space and returns a visualized plan execution. We discuss several key ideas that made Latplan possible which would hopefully extend to many other symbolic paradigms outside classical planning.

A new gap-based obstacle avoidance approach: follow the obstacle circle method

One of the most challenging tasks for autonomous robots is avoiding unexpected obstacles during their path following operation. Follow the gap method (FGM) is one of the most popular obstacle avoidance algorithms that recursively guides the robot to the goal state by considering the angle to the goal point and the distance to the closest obstacles. It selects the largest gap around the robot, where the gap angle is calculated by the vector to the midpoint of the largest gap. In this paper, a novel obstacle avoidance procedure is developed and applied to a real fully autonomous wheelchair. This proposed algorithm improves the FGM’s travel safety and brings a new solution to the obstacle avoidance task. In the proposed algorithm, the largest gap is selected based on gap width. Moreover, the avoidance angle (similar to the gap center angle of FGM) is calculated considering the locus of the equidistant points from obstacles that create obstacle circles. Monte Carlo simulations are used to test the proposed algorithm, and according to the results, the new procedure guides the robot to safer trajectories compared with classical FGM. The real experimental test results are in parallel to the simulations and show the real-time performance of the proposed approach.

The Path to Contentless Experience in Meditation: An Evidence Synthesis Based on Expert Texts

In contentless experience (sometimes termed pure consciousness) there is an absence of mental content such as thoughts, perceptions, and mental images. The path to contentless experience in meditation can be taken to comprise the meditation technique, and the experiences (“interim-states”) on the way to the contentless “goal-state/s”. Shamatha, Transcendental, and Stillness Meditation are each said to access contentless experience, but the path to that experience in each practice is not yet well understood from a scientific perspective. We have employed evidence synthesis to select and review 135 expert texts from those traditions. In this paper we describe the techniques and interim-states based on the expert texts and compare them across the practices on key dimensions. Superficially, Shamatha and Transcendental Meditation appear very different to Stillness Meditation in that they require bringing awareness to a meditation object. The more detailed and systematic approach taken in this paper indicates that posturally Shamatha is closer to Stillness Meditation, and that on several other dimensions Shamatha is quite different to both other practices. In particular, Shamatha involves greater measures to cultivate attentional stability and vividness on an object, greater focusing, less tolerance of mind-wandering, more monitoring, and more deliberate doing/control. Achieving contentless experience in Shamatha is much slower, more difficult, and less frequent. The findings have important implications for consciousness, neuroscientific, and clinical research and practice.

Learning Generalized Unsolvability Heuristics for Classical Planning

Recent work in classical planning has introduced dedicated techniques for detecting unsolvable states, i.e., states from which no goal state can be reached. We approach the problem from a generalized planning perspective and learn first-order-like formulas that characterize unsolvability for entire planning domains. We show how to cast the problem as a self-supervised classification task. Our training data is automatically generated and labeled by exhaustive exploration of small instances of each domain, and candidate features are automatically computed from the predicates used to define the domain. We investigate three learning algorithms with different properties and compare them to heuristics from the literature. Our empirical results show that our approach often captures important classes of unsolvable states with high classification accuracy. Additionally, the logical form of our heuristics makes them easy to interpret and reason about, and can be used to show that the characterizations learned in some domains capture exactly all unsolvable states of the domain.

Stochastic Shortest Path with Adversarially Changing Costs

Stochastic shortest path (SSP) is a well-known problem in planning and control, in which an agent has to reach a goal state in minimum total expected cost. In this paper we present the adversarial SSP model that also accounts for adversarial changes in the costs over time, while the underlying transition function remains unchanged. Formally, an agent interacts with an SSP environment for K episodes, the cost function changes arbitrarily between episodes, and the transitions are unknown to the agent. We develop the first algorithms for adversarial SSPs and prove high probability regret bounds of square-root K assuming all costs are strictly positive, and sub-linear regret in the general case. We are the first to consider this natural setting of adversarial SSP and obtain sub-linear regret for it.

A Method of Offline Reinforcement Learning Virtual Reality Satellite Attitude Control Based on Generative Adversarial Network

Virtual reality satellites give people an immersive experience of exploring space. The intelligent attitude control method using reinforcement learning to achieve multiaxis synchronous control is one of the important tasks of virtual reality satellites. In real-world systems, methods based on reinforcement learning face safety issues during exploration, unknown actuator delays, and noise in the raw sensor data. To improve the sample efficiency and avoid safety issues during exploration, this paper proposes a new offline reinforcement learning method to make full use of samples. This method learns a policy set with imitation learning and a policy selector using a generative adversarial network (GAN). The performance of the proposed method was verified in a real-world system (reaction-wheel-based inverted pendulum). The results showed that the agent trained with our method reached and maintained a stable goal state in 10,000 steps, whereas the behavior cloning method only remained stable for 500 steps.

A Novel Model-Based Reinforcement Learning Attitude Control Method for Virtual Reality Satellite

Observing the universe with virtual reality satellite is an amazing experience. An intelligent method of attitude control is the core object of research to achieve this goal. Attitude control is essentially one of the goal-state reaching tasks under constraints. Using reinforcement learning methods in real-world systems faces many challenges, such as insufficient samples, exploration safety issues, unknown actuator delays, and noise in the raw sensor data. In this work, a mixed model with different input sizes was proposed to represent the environmental dynamics model. The predication accuracy of the environmental dynamics model and the performance of the policy trained in this paper were gradually improved. Our method reduces the impact of noisy data on the model’s accuracy and improves the sampling efficiency. The experiments showed that the agent trained with our method completed a goal-state reaching task in a real-world system under wireless circumstances whose actuators were reaction wheels, whereas the soft actor-critic method failed in the same training process. The method’s effectiveness is ensured theoretically under given conditions.

Emergence of Content-Agnostic Information Processing by a Robot Using Active Inference, Visual Attention, Working Memory, and Planning

Generalization by learning is an essential cognitive competency for humans. For example, we can manipulate even unfamiliar objects and can generate mental images before enacting a preplan. How is this possible? Our study investigated this problem by revisiting our previous study (Jung, Matsumoto, & Tani, 2019), which examined the problem of vision-based, goal-directed planning by robots performing a task of block stacking. By extending the previous study, our work introduces a large network comprising dynamically interacting submodules, including visual working memory (VWMs), a visual attention module, and an executive network. The executive network predicts motor signals, visual images, and various controls for attention, as well as masking of visual information. The most significant difference from the previous study is that our current model contains an additional VWM. The entire network is trained by using predictive coding and an optimal visuomotor plan to achieve a given goal state is inferred using active inference. Results indicate that our current model performs significantly better than that used in Jung et al. (2019), especially when manipulating blocks with unlearned colors and textures. Simulation results revealed that the observed generalization was achieved because content-agnostic information processing developed through synergistic interaction between the second VWM and other modules during the course of learning, in which memorizing image contents and transforming them are dissociated. This letter verifies this claim by conducting both qualitative and quantitative analysis of simulation results.

Nature-Based Urbanization: Scan Opportunities, Determine Directions and Create Inspiring Ecologies

To base urbanization on nature, inspiring ecologies are necessary. The concept of nature-based solutions (NBS) could be helpful in achieving this goal. State of the art urban planning starts from the aim to realize a (part of) a city, not to improve natural quality or increase biodiversity. The aim of this article is to introduce a planning approach that puts the ecological landscape first, before embedding urban development. This ambition is explored using three NBS frameworks as the input for a series of design workshops, which conceived a regional plan for the Western Sydney Parklands in Australia. From these frameworks, elements were derived at three abstraction levels as the input for the design process: envisioning a long-term future (scanning the opportunities), evaluating the benefits and disadvantages, and identifying a common direction for the design (determining directions), and implementing concrete spatial cross-cutting solutions (creating inspiring ecologies), ultimately resulting in a regional landscape-based plan. The findings of this research demonstrate that, at every abstraction, a specific outcome is found: a mapped ecological landscape showing the options for urbanization, formulating a food-forest strategy as the commonly found direction for the design, and a regional plan that builds from the landscape ecologies adding layers of productive ecologies and urban synergies. By using NBS-frameworks, the potentials of putting the ecological landscape first in the planning process is illuminated, and urbanization can become resilient and nature-inclusive. Future research should emphasize the balance that should be established between the NBS-frameworks and the design approach, as an overly technocratic and all-encompassing framework prevents the freedom of thought that is needed to come to fruitful design propositions.

Superconductor qubits hamiltonian approximations effect on quantum state evolution and control

Microwave IQ-mixer controllers are designed for the three approximated Hamiltonians of charge, phase and flux qubits and the controllers are exerted both on approximate and precise quantum system models. The controlled qubits are for the implementation of the two quantum-gates with these three fundamental types of qubits, Quantum NOT-gate and Hadamard-gate. In the charge-qubit, for implementation of both gates, in the approximated and precise model, we observed different controlled trajectories. But fortunately, applying the controller designed for the approximated system over the precise system leads to the passing of the quantum state from the desired state sooner that the expected time. Phase-qubit and flux qubit have similar behaviour under the control system action. In both of them, the implementation of NOT-gate operation led to same trajectories which arrive at final goal state at different times. But in both of those two qubits for implementation of Hadamard-gate, desired trajectory and precise trajectory have some angle of deviation, then by exerting the approximated design controller to precise system, it caused the quantum state to approach the goal state for Hadamard gate implementation, and since the quantum state does not completely reach the goal state, we can not obtain very high gate fidelity.