BI-DIRECTIONAL MONTE CARLO TREE SEARCH

This paper describes a new algorithm called Bi-Directional Monte Carlo Tree Search. The essential idea of Bi-directional Monte Carlo Tree Search is to run an MCTS forwards from the start state, and simultaneously run an MCTS backwards from the goal state, and stop when the two searches meet. Bi-Directional MCTS is tested on 8-Puzzle and Pancakes Problem, two single-agent search problems, which allow control over the optimal solution length d and average branching factor b respectively. Preliminary results indicate that enhancing Monte Carlo Tree Search by making it Bi-Directional speeds up the search. The speedup of Bi-directional MCTS grows with increasing the problem size, in terms of both optimal solution length d and also branching factor b. Furthermore, Bi-Directional Search has been applied to a Reinforcement Learning algorithm. It is hoped that the speed enhancement of Bi-directional Monte Carlo Tree Search will also apply to other planning problems.

Significance of infrared spectroscopic branching factor for investigation of structural characteristics of alkanes, geochemical properties and viscosity of oils

A detailed investigation of significance of the infrared (IR) spectroscopic branching factor (??2/??3; the ratio of methylene and methyl group peak heights at 2917-2921 and 2951-2954 cm-1, respectively in the IR spectra) for characterization of alkane structure, geochemical properties and viscosity of 76 oil samples was performed. These oils, originating from 13 Serbian oil fields in SE Pannonian Basin, differ according to source and depositional environment of organic matter (OM), as well as by thermal maturity and biodegradation stage. Methylene and methyl asymmetric stretching peak absorbances were used for the branching factor calculation. CH2 peak positions exhibited 3-4 cm-1 red shift with increasing the CH2/CH3 ratio, due to a greater contribution of trans vs. gauche rotamers in aliphatic chains. Comparing IR spectra of the oils and model n-alkanes, it was established that the average (CH2)n methylene chain length per ??3 group varied from n = 3.5 to 6.5. The CH2/CH3 ratio showed significant concordance with geochemical parameters, enabling clear distinction of the oils according to source and depositional environment of OM. At the same time, dependence of the CH2/CH3 ratio on oil maturity in the range from immature to mature was not observed, allowing for an accurate determination of oil genetic types irrespective of maturity. The CH2/CH3 ratio showed good accordance with oil biodegradation scale and oil viscosity.

Synthesizing Designs With Interpart Dependencies Using Hierarchical Generative Adversarial Networks

Real-world designs usually consist of parts with interpart dependencies, i.e., the geometry of one part is dependent on one or multiple other parts. We can represent such dependency in a part dependency graph. This paper presents a method for synthesizing these types of hierarchical designs using generative models learned from examples. It decomposes the problem of synthesizing the whole design into synthesizing each part separately but keeping the interpart dependencies satisfied. Specifically, this method constructs multiple generative models, the interaction of which is based on the part dependency graph. We then use the trained generative models to synthesize or explore each part design separately via a low-dimensional latent representation, conditioned on the corresponding parent part(s). We verify our model on multiple design examples with different interpart dependencies. We evaluate our model by analyzing the constraint satisfaction performance, the synthesis quality, the latent space quality, and the effects of part dependency depth and branching factor. This paper’s techniques for capturing dependencies among parts lay the foundation for learned generative models to extend to more realistic engineering systems where such relationships are widespread.

Regular Boardgames

We propose a new General Game Playing (GGP) language called Regular Boardgames (RBG), which is based on the theory of regular languages. The objective of RBG is to join key properties as expressiveness, efficiency, and naturalness of the description in one GGP formalism, compensating certain drawbacks of the existing languages. This often makes RBG more suitable for various research and practical developments in GGP. While dedicated mostly for describing board games, RBG is universal for the class of all finite deterministic turn-based games with perfect information. We establish foundations of RBG, and analyze it theoretically and experimentally, focusing on the efficiency of reasoning. Regular Boardgames is the first GGP language that allows efficient encoding and playing games with complex rules and with large branching factor (e.g. amazons, arimaa, large chess variants, go, international checkers, paper soccer).

Benchmark Generator for Stratified Controllers of Tank Networks

We present a new model of a tank network used to transfer liquid. Tanks are connected by channels. The throughput velocity of every particular channel is governed by the controller. We consider a special class of stratified controllers which are organized in several phases. Every phase can be further partitioned into multiple options. This structure makes it easy to generate a variety of benchmark instances ranging in the size, branching factor and generally analysis complexity. We provide a flexible benchmark generator for this class of benchmarks and a sample benchmark suite built by the generator. Finally, we use the Hyst model transformation framework to convert the original model in a format compatible with several reachability tools.

Q-Table compression for reinforcement learning

Reinforcement learning (RL) algorithms are often used to compute agents capable of acting in environments without prior knowledge of the environment dynamics. However, these algorithms struggle to converge in environments with large branching factors and their large resulting state-spaces. In this work, we develop an approach to compress the number of entries in a Q-value table using a deep auto-encoder. We develop a set of techniques to mitigate the large branching factor problem. We present the application of such techniques in the scenario of a real-time strategy (RTS) game, where both state space and branching factor are a problem. We empirically evaluate an implementation of the technique to control agents in an RTS game scenario where classical RL fails and provide a number of possible avenues of further work on this problem.

Focused Depth-first Proof Number Search using Convolutional Neural Networks for the Game of Hex

Proof Number search (PNS) is an effective algorithm for searching theoretical values on games with non-uniform branching factors. Focused depth-first proof number search (FDFPN) with dynamic widening was proposed for Hex where the branching factor is nearly uniform. However, FDFPN is fragile to its heuristic move ordering function. The recent advances of Convolutional Neural Networks (CNNs) have led to considerable progress in game playing. We investigate how to incorporate the strength of CNNs into solving, with application to the game of Hex. We describe FDFPN-CNN, a new focused DFPN search that uses convolutional neural networks. FDFPN-CNN integrates two CNNs trained from games played by expert players. The value approximation CNN provides reliable information for defining the widening size by estimating the value of the node to expand, while the policy CNN selects promising children nodes to the search. On 8x8 Hex, experimental results show FDFPN-CNN performs notably better than FDFPN, suggesting a promising direction for better solving Hex positions where learning from strong players is possible.

Combinatorial Multi-armed Bandits for Real-Time Strategy Games

Games with large branching factors pose a significant challenge for game tree search algorithms. In this paper, we address this problem with a sampling strategy for Monte Carlo Tree Search (MCTS) algorithms called "naive sampling", based on a variant of the Multi-armed Bandit problem called "Combinatorial Multi-armed Bandits" (CMAB). We analyze the theoretical properties of several variants of naive sampling, and empirically compare it against the other existing strategies in the literature for CMABs. We then evaluate these strategies in the context of real-time strategy (RTS) games, a genre of computer games characterized by their very large branching factors. Our results show that as the branching factor grows, naive sampling outperforms the other sampling strategies.