scholarly journals Instance-Specific Selection of AOS Methods for Solving Combinatorial Optimisation Problems via Neural Networks

2018 ◽  
pp. 98-114
Author(s):  
Teck-Hou Teng ◽  
Hoong Chuin Lau ◽  
Aldy Gunawan
Keyword(s):  
Neural Networks ◽  
Combinatorial Optimisation ◽  
Selection Of

scholarly journals A New Hyper-Parameter Optimization Method for Power Load Forecast Based on Recurrent Neural Networks

Algorithms ◽  
2021 ◽  
Vol 14 (6) ◽  
pp. 163
Author(s):  
Yaru Li ◽  
Yulai Zhang ◽  
Yongping Cai
Keyword(s):  
Neural Networks ◽  
Parameter Optimization ◽  
Recurrent Neural Networks ◽  
Critical Role ◽  
Optimization Method ◽  
Bayesian Optimization ◽  
Power Load ◽  
Gradient Calculation ◽  
Truncated Newton ◽  
Selection Of

The selection of the hyper-parameters plays a critical role in the task of prediction based on the recurrent neural networks (RNN). Traditionally, the hyper-parameters of the machine learning models are selected by simulations as well as human experiences. In recent years, multiple algorithms based on Bayesian optimization (BO) are developed to determine the optimal values of the hyper-parameters. In most of these methods, gradients are required to be calculated. In this work, the particle swarm optimization (PSO) is used under the BO framework to develop a new method for hyper-parameter optimization. The proposed algorithm (BO-PSO) is free of gradient calculation and the particles can be optimized in parallel naturally. So the computational complexity can be effectively reduced which means better hyper-parameters can be obtained under the same amount of calculation. Experiments are done on real world power load data,where the proposed method outperforms the existing state-of-the-art algorithms,BO with limit-BFGS-bound (BO-L-BFGS-B) and BO with truncated-newton (BO-TNC),in terms of the prediction accuracy. The errors of the prediction result in different models show that BO-PSO is an effective hyper-parameter optimization method.

scholarly journals Dewpoint Temperature Prediction Using Artificial Neural Networks

2008 ◽  
Vol 47 (6) ◽  
pp. 1757-1769 ◽  
Author(s):  
D. B. Shank ◽  
G. Hoogenboom ◽  
R. W. McClendon
Keyword(s):  
Neural Networks ◽  
Artificial Neural Networks ◽  
Air Temperature ◽  
Heat Waves ◽  
Weather Data ◽  
Temperature Prediction ◽  
Related Data ◽  
Dewpoint Temperature ◽  
Artificial Neural ◽  
Selection Of

Abstract Dewpoint temperature, the temperature at which water vapor in the air will condense into liquid, can be useful in estimating frost, fog, snow, dew, evapotranspiration, and other meteorological variables. The goal of this study was to use artificial neural networks (ANNs) to predict dewpoint temperature from 1 to 12 h ahead using prior weather data as inputs. This study explores using three-layer backpropagation ANNs and weather data combined for three years from 20 locations in Georgia, United States, to develop general models for dewpoint temperature prediction anywhere within Georgia. Specific objectives included the selection of the important weather-related inputs, the setting of ANN parameters, and the selection of the duration of prior input data. An iterative search found that, in addition to dewpoint temperature, important weather-related ANN inputs included relative humidity, solar radiation, air temperature, wind speed, and vapor pressure. Experiments also showed that the best models included 60 nodes in the ANN hidden layer, a ±0.15 initial range for the ANN weights, a 0.35 ANN learning rate, and a duration of prior weather-related data used as inputs ranging from 6 to 30 h based on the lead time. The evaluation of the final models with weather data from 20 separate locations and for a different year showed that the 1-, 4-, 8-, and 12-h predictions had mean absolute errors (MAEs) of 0.550°, 1.234°, 1.799°, and 2.280°C, respectively. These final models predicted dewpoint temperature adequately using previously unseen weather data, including difficult freeze and heat stress extremes. These predictions are useful for decisions in agriculture because dewpoint temperature along with air temperature affects the intensity of freezes and heat waves, which can damage crops, equipment, and structures and can cause injury or death to animals and humans.

scholarly journals Data Size Requirement for Forecasting Daily Crude Oil Price with Neural Networks

2019 ◽  
Vol 66 (3) ◽  
pp. 363-388
Author(s):  
Serkan Aras ◽  
Manel Hamdi
Keyword(s):  
Neural Networks ◽  
Crude Oil ◽  
Significant Interaction ◽  
Statistical Tests ◽  
Oil Price ◽  
Training Data ◽  
Forecasting Model ◽  
Crude Oil Price ◽  
Crude Oil Prices ◽  
Selection Of

When the literature regarding applications of neural networks is investigated, it appears that a substantial issue is what size the training data should be when modelling a time series through neural networks. The aim of this paper is to determine the size of training data to be used to construct a forecasting model via a multiple-breakpoint test and compare its performance with two general methods, namely, using all available data and using just two years of data. Furthermore, the importance of the selection of the final neural network model is investigated in detail. The results obtained from daily crude oil prices indicate that the data from the last structural change lead to simpler architectures of neural networks and have an advantage in reaching more accurate forecasts in terms of MAE value. In addition, the statistical tests show that there is a statistically significant interaction between data size and stopping rule.

scholarly journals Soft Threshold Ternary Networks

2020 ◽  
Author(s):  
Weixiang Xu ◽  
Xiangyu He ◽  
Tianli Zhao ◽  
Qinghao Hu ◽  
Peisong Wang ◽  
...  
Keyword(s):  
Neural Networks ◽  
Mobile Devices ◽  
State Of The Art ◽  
Performance Gap ◽  
Training Time ◽  
Current State ◽  
The Arts ◽  
Soft Threshold ◽  
And Storage ◽  
Selection Of

Large neural networks are difficult to deploy on mobile devices because of intensive computation and storage. To alleviate it, we study ternarization, a balance between efficiency and accuracy that quantizes both weights and activations into ternary values. In previous ternarized neural networks, a hard threshold Δ is introduced to determine quantization intervals. Although the selection of Δ greatly affects the training results, previous works estimate Δ via an approximation or treat it as a hyper-parameter, which is suboptimal. In this paper, we present the Soft Threshold Ternary Networks (STTN), which enables the model to automatically determine quantization intervals instead of depending on a hard threshold. Concretely, we replace the original ternary kernel with the addition of two binary kernels at training time, where ternary values are determined by the combination of two corresponding binary values. At inference time, we add up the two binary kernels to obtain a single ternary kernel. Our method dramatically outperforms current state-of-the-arts, lowering the performance gap between full-precision networks and extreme low bit networks. Experiments on ImageNet with AlexNet (Top-1 55.6%), ResNet-18 (Top-1 66.2%) achieves new state-of-the-art.

Prediction of slip in cable-drum systems using structured neural networks

2013 ◽  
Vol 228 (3) ◽  
pp. 441-456 ◽  
Author(s):  
Ergin Kilic ◽  
Melik Dolen
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Network Size ◽  
Network Architectures ◽  
Network Development ◽  
Low Velocity ◽  
Error Band ◽  
Artificial Neural ◽  
Recurrent Type ◽  
Selection Of

This study focuses on the slip prediction in a cable-drum system using artificial neural networks for the prospect of developing linear motion sensing scheme for such mechanisms. Both feed-forward and recurrent-type artificial neural network architectures are considered to capture the slip dynamics of cable-drum mechanisms. In the article, the network development is presented in a progressive (step-by-step) fashion for the purpose of not only making the design process transparent to the readers but also highlighting the corresponding challenges associated with the design phase (i.e. selection of architecture, network size, training process parameters, etc.). Prediction performances of the devised networks are evaluated rigorously via an experimental study. Finally, a structured neural network, which embodies the network with the best prediction performance, is further developed to overcome the drift observed at low velocity. The study illustrates that the resulting structured neural network could predict the slip in the mechanism within an error band of 100 µm when an absolute reference is utilized.

Sign in / Sign up

Export Citation Format

Share Document