Forecast of complex financial big data using model tree optimized by bilevel evolution strategy
AbstractIf a decision tree is constructed through a series of locally optimal solutions, such as the Greedy method, overfitting to the data is likely to occur. In order to avoid overfitting, many previous research have attempted to collectively optimize the structure of a decision tree by using evolutionary computation. However, if attributes of each split and their thresholds are searched simultaneously, the evaluation function becomes intermittent; thus, optimization methods assuming continuous distribution cannot be used. In this study, in order to enable efficient search assuming continuous distribution even for complicated data that contains a lot of noise and extraordinary values, such as financial time series data, the inner level search that optimizes each threshold value collectively given a specific attribute for each split in a model tree and the outer level search that optimizes the attributes of each split were performed by separate evolutionary computing. As a result, we obtained high prediction accuracy that far exceeded the performance of the conventional method.