scholarly journals Analysis of the Impact of Randomization of Search-Control Parameters in Monte-Carlo Tree Search

2021 ◽  
Vol 72 ◽  
pp. 717-757
Author(s):  
Chiara F. Sironi ◽  
Mark H. M. Winands
Keyword(s):  
Monte Carlo ◽  
Optimal Parameter ◽  
Parameter Tuning ◽  
Tree Search ◽  
Control Parameters ◽  
Monte Carlo Tree Search ◽  
The One ◽  
Parameter Values ◽  
Search Control ◽  
The Impact

Monte-Carlo Tree Search (MCTS) has been applied successfully in many domains, including games. However, its performance is not uniform on all domains, and it also depends on how parameters that control the search are set. Parameter values that are optimal for a task might be sub-optimal for another. In a domain that tackles many games with different characteristics, like general game playing (GGP), selecting appropriate parameter settings is not a trivial task. Games are unknown to the player, thus, finding optimal parameters for a given game in advance is not feasible. Previous work has looked into tuning parameter values online, while the game is being played, showing some promising results. This tuning approach looks for optimal parameter values, balancing exploitation of values that performed well so far in the search and exploration of less sampled values. Continuously changing parameter values while performing the search, combined also with exploration of multiple values, introduces some randomization in the process. In addition, previous research indicates that adding randomization to certain components of MCTS might increase the diversification of the search and improve the performance. Therefore, this article investigates the effect of randomly selecting values for MCTS search-control parameters online among predefined sets of reasonable values. For the GGP domain, this article evaluates four different online parameter randomization strategies by comparing them with other methods to set parameter values: online parameter tuning, offline parameter tuning and sub-optimal parameter choices. Results on a set of 14 heterogeneous abstract games show that randomizing parameter values before each simulation has a positive effect on the search in some of the tested games, with respect to using fixed offline-tuned parameters. Moreover, results show a clear distinction between games for which online parameter tuning works best and games for which online randomization works best. In addition, the overall performance of online parameter randomization is closer to the one of online parameter turning than the one of sub-optimal parameter values, showing that online randomization is a reasonable parameter selection strategy. When analyzing the structure of the search trees generated by agents that use the different parameters selection strategies, it is clear that randomization causes MCTS to become more explorative, which is helpful for alignment games that present many winning paths in their trees. Online parameter tuning, instead, seems more suitable for games that present narrow winning paths and many losing paths.

scholarly journals Monte Carlo Tree Search-Based Recursive Algorithm for Feature Selection in High-Dimensional Datasets

Entropy ◽  
2020 ◽  
Vol 22 (10) ◽  
pp. 1093
Author(s):  
Muhammad Umar Chaudhry ◽  
Muhammad Yasir ◽  
Muhammad Nabeel Asghar ◽  
Jee-Hyong Lee
Keyword(s):  
Monte Carlo ◽  
Feature Selection ◽  
Classification Accuracy ◽  
Recursive Algorithm ◽  
Feature Subset Selection ◽  
Great Success ◽  
Feature Subset ◽  
Tree Search ◽  
Monte Carlo Tree Search ◽  
The Impact

The complexity and high dimensionality are the inherent concerns of big data. The role of feature selection has gained prime importance to cope with the issue by reducing dimensionality of datasets. The compromise between the maximum classification accuracy and the minimum dimensions is as yet an unsolved puzzle. Recently, Monte Carlo Tree Search (MCTS)-based techniques have been invented that have attained great success in feature selection by constructing a binary feature selection tree and efficiently focusing on the most valuable features in the features space. However, one challenging problem associated with such approaches is a tradeoff between the tree search and the number of simulations. In a limited number of simulations, the tree might not meet the sufficient depth, thus inducing biasness towards randomness in feature subset selection. In this paper, a new algorithm for feature selection is proposed where multiple feature selection trees are built iteratively in a recursive fashion. The state space of every successor feature selection tree is less than its predecessor, thus increasing the impact of tree search in selecting best features, keeping the MCTS simulations fixed. In this study, experiments are performed on 16 benchmark datasets for validation purposes. We also compare the performance with state-of-the-art methods in literature both in terms of classification accuracy and the feature selection ratio.

Sign in / Sign up

Export Citation Format

Share Document