Online Maintenance Prioritization via Monte Carlo Tree Search and Case-based Reasoning

When maintenance resources in a manufacturing system are limited, a challenge arises in determining how to allocate these resources among multiple competing maintenance jobs. We formulate this problem as an online prioritization problem using a Markov decision process (MDP) to model the system behavior and Monte Carlo tree search (MCTS) to seek optimal maintenance actions in various states of the system. Further, we use Case-based Reasoning (CBR) to retain and reuse search experience gathered from MCTS to reduce the computational effort needed over time and to improve decision-making efficiency. We demonstrate that our proposed method results in increased system throughput when compared to existing methods of maintenance prioritization while also reducing the time needed to identify optimal maintenance actions as more experience is gathered. This is especially beneficial in manufacturing settings where maintenance decisions must be made quickly.

Spectra to Structure: Deep Reinforcement Learning for Molecular Inverse Problem

Spectroscopy is the study of how matter interacts with electromagnetic radiations of specific frequencies that has led to several monumental discoveries in science. The spectra of any particular molecule is highly information-rich, yet the inverse relation from the spectra to the molecular structure is still an unsolved problem. Nuclear Magnetic Resonance (NMR) spectroscopy is one such critical tool in the tool-set for scientists to characterise any chemical sample. In this work, a novel framework is proposed that attempts to solve this inverse problem by navigating the chemical space to find the correct structure that resulted in the target spectra. The proposed framework uses a combination of online Monte- Carlo-Tree-Search (MCTS) and a set of offline trained Graph Convolution Networks to build a molecule iteratively from scratch. Our method is able to predict the correct structure of the molecule ∼80% of the time in its top 3 guesses. We believe that the proposed framework is a significant step in solving the inverse design problem of NMR spectra to molecule.

MERMAID: an open source automated hit-to-lead method based on deep reinforcement learning

The hit-to-lead process makes the physicochemical properties of the hit molecules that show the desired type of activity obtained in the screening assay more drug-like. Deep learning-based molecular generative models are expected to contribute to the hit-to-lead process. The simplified molecular input line entry system (SMILES), which is a string of alphanumeric characters representing the chemical structure of a molecule, is one of the most commonly used representations of molecules, and molecular generative models based on SMILES have achieved significant success. However, in contrast to molecular graphs, during the process of generation, SMILES are not considered as valid SMILES. Further, it is quite difficult to generate molecules starting from a certain molecule, thus making it difficult to apply SMILES to the hit-to-lead process. In this study, we have developed a SMILES-based generative model that can be generated starting from a certain molecule. This method generates partial SMILES and inserts it into the original SMILES using Monte Carlo Tree Search and a Recurrent Neural Network. We validated our method using a molecule dataset obtained from the ZINC database and successfully generated molecules that were both well optimized for the objectives of the quantitative estimate of drug-likeness (QED) and penalized octanol-water partition coefficient (PLogP) optimization. The source code is available at https://github.com/sekijima-lab/mermaid.

Generate plane quad mesh with neural networks and tree search

The quality of mesh generation has long been considered a vital aspect in providing engineers with reliable simulation results throughout the history of the Finite Element Method (FEM). The element extraction method, which is currently the most robust method, is used in business software. However, in order to speed up extraction, the approach is done by finding the next element that optimizes a target function, which can result in local mesh of bad quality after many time steps. We provide TreeMesh, a method that uses this method in conjunction with reinforcement learning (also possible with supervised learning) and a novel Monte-Carlo tree search (MCTS) (Coulom(2006), Kocsis and Szepesvári(2006), Browne et~al.(2012)). The algorithm is based on a previously proposed approach (Pan et~al.(2021)). After making many improvements on DRL (algorithm, state-action-reward setting) and adding a MCTS, it outperforms the former work on the same boundary. Furthermore, using tree search, our program reveals much preponderance on seed-density-changing boundaries, which is common on thin-film materials.

Analysis of the Impact of Randomization of Search-Control Parameters in Monte-Carlo Tree Search

Monte-Carlo Tree Search (MCTS) has been applied successfully in many domains, including games. However, its performance is not uniform on all domains, and it also depends on how parameters that control the search are set. Parameter values that are optimal for a task might be sub-optimal for another. In a domain that tackles many games with different characteristics, like general game playing (GGP), selecting appropriate parameter settings is not a trivial task. Games are unknown to the player, thus, finding optimal parameters for a given game in advance is not feasible. Previous work has looked into tuning parameter values online, while the game is being played, showing some promising results. This tuning approach looks for optimal parameter values, balancing exploitation of values that performed well so far in the search and exploration of less sampled values. Continuously changing parameter values while performing the search, combined also with exploration of multiple values, introduces some randomization in the process. In addition, previous research indicates that adding randomization to certain components of MCTS might increase the diversification of the search and improve the performance. Therefore, this article investigates the effect of randomly selecting values for MCTS search-control parameters online among predefined sets of reasonable values. For the GGP domain, this article evaluates four different online parameter randomization strategies by comparing them with other methods to set parameter values: online parameter tuning, offline parameter tuning and sub-optimal parameter choices. Results on a set of 14 heterogeneous abstract games show that randomizing parameter values before each simulation has a positive effect on the search in some of the tested games, with respect to using fixed offline-tuned parameters. Moreover, results show a clear distinction between games for which online parameter tuning works best and games for which online randomization works best. In addition, the overall performance of online parameter randomization is closer to the one of online parameter turning than the one of sub-optimal parameter values, showing that online randomization is a reasonable parameter selection strategy. When analyzing the structure of the search trees generated by agents that use the different parameters selection strategies, it is clear that randomization causes MCTS to become more explorative, which is helpful for alignment games that present many winning paths in their trees. Online parameter tuning, instead, seems more suitable for games that present narrow winning paths and many losing paths.

Monte caro tree search algorithm for the receiving containers intelligently problem among container shipping terminals

With the development of economic globalization, shipping exchanges between countries are rapidly increasing and the container throughput of shipping ports has increased quickly, and the operation of most seaport terminals in China has reached a bottleneck which puts forward new requirements for the efficiency of container shipping terminal operations. Therefore, locating for export containers intelligently is of great significance for the development of terminals. This paper focuses on receiving containers intelligently, and establishes the export container locating model based on the principle of actual shipping terminal operations. The Monte Carlo Tree Search algorithm for export container locating problem was proposed and constructed. After examples, the algorithm can effectively solve the problem that meet the constraints, which further proves the practicability of the algorithm and the correctness of the model. The research shows that the method of locating for export container based on Monte Carlo Tree Search algorithm can effectively solve the problem and maintain the green, energy-saving and sustainable development of the shipping terminal. The ideas and methods have certain academic value and reference significance for other NP-Hard problems.