Multiple IoT based Network Attacks Discrimination by Multilayer Feedforward Neural Networks

In this paper a new neural model for detection of multiple network IoT-based attacks, such as DDoS TCP, UDP, and HHTP flood, is presented. It consists of feedforward multilayer network with back propagation. A general algorithm for its optimization during training is proposed, leading to proper number of neurons in the hidden layers. The Scaled Gradient Descent algorithm and the Adam optimization are studied with better classification results, obtained by the developed classifiers, using the latter. Tangent hyperbolic function appears to be proper selection for the hidden neurons. Two sets of features, gathered from aggregated records of the network traffic, are tested, containing 8 and 10 components. While more accurate results are obtained for the 10-feature set, the 8-feature set offers twice lower training time and seems applicable for real-world applications. The detection rate for 7 of 10 different network attacks, primarily various types of floods, is higher than 90% and for 3 of them – mainly reconnaissance and keylogging activities with low intensity of the generated traffic, deviates between 57% and 68%. The classifier is considered applicable for industrial implementation.

Adaptive Deep Incremental Learning — Assisted Missing Data Imputation for Streaming Data

With the rapid explosion of the data streams from the applications, ensuring accurate data analysis is essential for effective real-time decision making. Nowadays, data stream applications often confront the missing values that affect the performance of the classification models. Several imputation models have adopted the deep learning algorithms for estimating the missing values; however, the lack of parameter and structure tuning in classification, degrade the performance for data imputation. This work presents the missing data imputation model using the adaptive deep incremental learning algorithm for streaming applications. The proposed approach incorporates two main processes: enhancing the deep incremental learning algorithm and enhancing deep incremental learning-based imputation. Initially, the proposed approach focuses on tuning the learning rate with both the Adaptive Moment Estimation (Adam) along with Stochastic Gradient Descent (SGD) optimizers and tuning the hidden neurons. Secondly, the proposed approach applies the enhanced deep incremental learning algorithm to estimate the imputed values in two steps: (i) imputation process to predict the missing values based on the temporal-proximity and (ii) generation of complete IoT dataset by imputing the missing values from both the predicted values. The experimental outcomes illustrate that the proposed imputation model effectively transforms the incomplete dataset into a complete dataset with minimal error.

Interpreting and Comparing Convolutional Neural Networks: A Quantitative Approach

A convolutional neural network (CNN) is sometimes understood as a black box in the sense that while it can approximate any function, studying its structure will not give us any insights into the nature of the function being approximated. In other terms, the discriminative ability does not reveal much about the latent representation of a network. This research aims to establish a framework for interpreting the CNNs by profiling them in terms of interpretable visual concepts and verifying them by means of Integrated Gradient. We also ask the question, "Do different input classes have a relationship or are they unrelated?" For instance, could there be an overlapping set of highly active neurons to identify different classes? Could there be a set of neurons that are useful for one input class whereas misleading for a different one? Intuition answers these questions positively, implying the existence of a structured set of neurons inclined to a particular class. Knowing this structure has significant values; it provides a principled way for identifying redundancies across the classes. Here the interpretability profiling has been done by evaluating the correspondence between individual hidden neurons and a set of human-understandable visual semantic concepts. We also propose an integrated gradient-based class-specific relevance mapping approach that takes the spatial position of the region of interest in the input image. Our relevance score verifies the interpretability scores in terms of neurons tuned to a particular concept/class. Further, we perform network ablation and measure the performance of the network based on our approach.

Rent index forecasting through neural networks

PurposeChinese housing market has been growing fast during the past decade, and price-related forecasting has turned to be an important issue to various market participants, including the people, investors and policy makers. Here, the authors approach this issue by researching neural networks for rent index forecasting from 10 major cities for March 2012 to May 2020. The authors aim at building simple and accurate neural networks to contribute to pure technical forecasting of the Chinese rental housing market.Design/methodology/approachTo facilitate the analysis, the authors examine different model settings over the algorithm, delay, hidden neuron and data spitting ratio.FindingsThe authors reach a rather simple neural network with six delays and two hidden neurons, which leads to stable performance of 1.4% average relative root mean square error across the ten cities for the training, validation and testing phases.Originality/valueThe results might be used on a standalone basis or combined with fundamental forecasting to form perspectives of rent price trends and conduct policy analysis.

Automatic time series forecasting using nonlinear autoregressive neural network model with exogenous input

This study aims to determine an automatic forecasting method of univariate time series, using the nonlinear autoregressive neural network model with exogenous input (NARX). In this automatic setting, users only need to supply the input of time series. Then, an automatic forecasting algorithm sets up the appropriate features, estimate the parameters in the model, and calculate forecasts, without the users’ intervention. The algorithm method used include preprocessing, tests for trends, and the application of first differences. The time series were tested for seasonality, and seasonal differences were obtained from a successful analysis. These series were also linearly scaled to [−1, +1]. The autoregressive lags and hidden neurons were further selected through the stepwise and optimization algorithms, respectively. The 20 NARX models were fitted with different random starting weights, and the forecasts were combined using the ensemble operator, in order to obtain the final product. This proposed method was applied to real data, and its performance was compared with several available automatic models in the literature. The forecasting accuracy was also measured by mean squared error (MSE) and mean absolute percent error (MAPE), and the results showed that the proposed method outperformed the other automatic models.

Determining the number of hidden layer and hidden neuron of neural network for wind speed prediction

Artificial neural network (ANN) is one of the techniques in artificial intelligence, which has been widely applied in many fields for prediction purposes, including wind speed prediction. The aims of this research is to determine the topology of neural network that are used to predict wind speed. Topology determination means finding the hidden layers number and the hidden neurons number for corresponding hidden layer in the neural network. The difference between this research and previous research is that the objective function of this research is regression, while the objective function of previous research is classification. Determination of the topology of the neural network using principal component analysis (PCA) and K-means clustering. PCA is used to determine the hidden layers number, while clustering is used to determine the hidden neurons number for corresponding hidden layer. The selected topology is then used to predict wind speed. Then the performance of topology determination using PCA and clustering is then compared with several other methods. The results of the experiment show that the performance of the neural network topology determined using PCA and clustering has better performance than the other methods being compared. Performance is determined based on the RMSE value, the smaller the RMSE value, the better the neural network performance. In future research, it is necessary to apply a correlation or relationship between input attribute and output attribute and then analyzed, prior to conducting PCA and clustering analysis.

Efficient Crude Oil Pricing Using a Machine Learning Approach

Crude oil, a base for more than 6000 products that we use on a daily basis, accounts for 33% of global energy consumption. However, the outbreak and transmission of COVID-19 had significant implications for the entire value chain in the oil industry. The price crash and the fluctuations in price is known to have far reaching effect on global economies, with Nigeria hard. It has therefore become imperative to develop a tool for forecasting the price of crude oil in order to minimise the risks associated with volatility in oil prices and also be able to do proper planning. Hence, this article proposed a hybrid forecasting model involving a classical and machine learning techniques – autoregressive neural network, in determining the prices of crude oil. The monthly data used were obtained from the Central Bank of Nigeria website, spanning January 2006 to October 2020. Statistical efficiency was computed for the hybrid, and the models from which the proposed hybrid was built, using the percent relative efficiency. Analyses showed that the efficiency of the hybrid model, at 20 and 100 hidden neurons, was higher than that of the individual models, the latter being the best performing. The study recommends urgent diversification of the economy in order not for the nation to be plunged into a seemingly unending recession.

Evolution of Activation Functions: An Empirical Investigation

The hyper-parameters of a neural network are traditionally designed through a time-consuming process of trial and error that requires substantial expert knowledge. Neural Architecture Search algorithms aim to take the human out of the loop by automatically finding a good set of hyper-parameters for the problem at hand. These algorithms have mostly focused on hyper-parameters such as the architectural configurations of the hidden layers and the connectivity of the hidden neurons, but there has been relatively little work on automating the search for completely new activation functions, which are one of the most crucial hyperparameters to choose. There are some widely used activation functions nowadays that are simple and work well, but nonetheless, there has been some interest in finding better activation functions. The work in the literature has mostly focused on designing new activation functions by hand or choosing from a set of predefined functions while this work presents an evolutionary algorithm to automate the search for completely new activation functions. We compare these new evolved activation functions to other existing and commonly used activation functions. The results are favorable and are obtained from averaging the performance of the activation functions found over 30 runs, with experiments being conducted on 10 different datasets and architectures to ensure the statistical robustness of the study.

Prognostication of Shortwave Radiation Using an Improved No-Tuned Fast Machine Learning

Shortwave radiation density flux (SRDF) modeling can be key in estimating actual evapotranspiration in plants. SRDF is the result of the specific and scattered reflection of shortwave radiation by the underlying surface. SRDF can have profound effects on some plant biophysical processes such as photosynthesis and land surface energy budgets. Since it is the main energy source for most atmospheric phenomena, SRDF is also widely used in numerical weather forecasting. In the current study, an improved version of the extreme learning machine was developed for SRDF forecasting using the historical value of this variable. To do that, the SRDF through 1981–2019 was extracted by developing JavaScript-based coding in the Google Earth Engine. The most important lags were found using the auto-correlation function and defined fifteen input combinations to model SRDF using the improved extreme learning machine (IELM). The performance of the developed model is evaluated based on the correlation coefficient (R), root mean square error (RMSE), mean absolute percentage error (MAPE), and Nash–Sutcliffe efficiency (NSE). The shortwave radiation was developed for two time ahead forecasting (R = 0.986, RMSE = 21.11, MAPE = 8.68%, NSE = 0.97). Additionally, the estimation uncertainty of the developed improved extreme learning machine is quantified and compared with classical ELM and found to be the least with a value of ±3.64 compared to ±6.9 for the classical extreme learning machine. IELM not only overcomes the limitation of the classical extreme learning machine in random adjusting of bias of hidden neurons and input weights but also provides a simple matrix-based method for practical tasks so that there is no need to have any knowledge of the improved extreme learning machine to use it.

Artificial neurovascular network (ANVN) to study the accuracy vs. efficiency trade-off in an energy dependent neural network

Artificial feedforward neural networks perform a wide variety of classification and function approximation tasks with high accuracy. Unlike their artificial counterparts, biological neural networks require a supply of adequate energy delivered to single neurons by a network of cerebral microvessels. Since energy is a limited resource, a natural question is whether the cerebrovascular network is capable of ensuring maximum performance of the neural network while consuming minimum energy? Should the cerebrovascular network also be trained, along with the neural network, to achieve such an optimum? In order to answer the above questions in a simplified modeling setting, we constructed an Artificial Neurovascular Network (ANVN) comprising a multilayered perceptron (MLP) connected to a vascular tree structure. The root node of the vascular tree structure is connected to an energy source, and the terminal nodes of the vascular tree supply energy to the hidden neurons of the MLP. The energy delivered by the terminal vascular nodes to the hidden neurons determines the biases of the hidden neurons. The “weights” on the branches of the vascular tree depict the energy distribution from the parent node to the child nodes. The vascular weights are updated by a kind of “backpropagation” of the energy demand error generated by the hidden neurons. We observed that higher performance was achieved at lower energy levels when the vascular network was also trained along with the neural network. This indicates that the vascular network needs to be trained to ensure efficient neural performance. We observed that below a certain network size, the energetic dynamics of the network in the per capita energy consumption vs. classification accuracy space approaches a fixed-point attractor for various initial conditions. Once the number of hidden neurons increases beyond a threshold, the fixed point appears to vanish, giving place to a line of attractors. The model also showed that when there is a limited resource, the energy consumption of neurons is strongly correlated to their individual contribution to the network’s performance.