Wavelet Neural Networks and Equalization of Nonlinear Satellite Communication Channel

2021 ◽  
pp. 207-226
Author(s):  
Saikat Majumder
Keyword(s):  
Neural Networks ◽  
Communication Channel ◽  
Satellite Communication ◽  
Nonlinear Function ◽  
Activation Functions ◽  
Wavelet Neural Networks ◽  
Transmitter Power ◽  
Saturation Region ◽  
Hidden Layer ◽  
Nonlinear Function Approximation

Wavelet neural networks are a class of single hidden layer neural networks consisting of wavelets as activation functions. Wavelet neural networks (WNN) are an alternative to the classical multilayer perceptron neural networks for arbitrary nonlinear function approximation and can provide compact network representation. In this chapter, a tutorial introduction to different types of WNNs and their architecture is given, along with its training algorithm. Subsequently, a novel application of WNN for equalization of nonlinear satellite communication channel is presented. Nonlinearity in a satellite communication channel is mainly caused due to use of transmitter power amplifiers near its saturation region to improve efficiency. Two models describing amplitude and phase distortion caused in a power amplifier are explained. Performance of the proposed equalizer is evaluated and compared to an existing equalizer in literature.

Analysis of Non-Linear Activation Functions for Classification Tasks Using Convolutional Neural Networks

2019 ◽  
Vol 12 (3) ◽  
pp. 156-161 ◽  
Author(s):  
Aman Dureja ◽  
Payal Pahwa
Keyword(s):  
Neural Networks ◽  
Deep Neural Networks ◽  
Activation Function ◽  
Primary Objective ◽  
Experimental Comparison ◽  
Activation Functions ◽  
Practical Applications ◽  
Network Activation ◽  
Non Linear ◽  
Hidden Layer

Background: In making the deep neural network, activation functions play an important role. But the choice of activation functions also affects the network in term of optimization and to retrieve the better results. Several activation functions have been introduced in machine learning for many practical applications. But which activation function should use at hidden layer of deep neural networks was not identified. Objective: The primary objective of this analysis was to describe which activation function must be used at hidden layers for deep neural networks to solve complex non-linear problems. Methods: The configuration for this comparative model was used by using the datasets of 2 classes (Cat/Dog). The number of Convolutional layer used in this network was 3 and the pooling layer was also introduced after each layer of CNN layer. The total of the dataset was divided into the two parts. The first 8000 images were mainly used for training the network and the next 2000 images were used for testing the network. Results: The experimental comparison was done by analyzing the network by taking different activation functions on each layer of CNN network. The validation error and accuracy on Cat/Dog dataset were analyzed using activation functions (ReLU, Tanh, Selu, PRelu, Elu) at number of hidden layers. Overall the Relu gave best performance with the validation loss at 25th Epoch 0.3912 and validation accuracy at 25th Epoch 0.8320. Conclusion: It is found that a CNN model with ReLU hidden layers (3 hidden layers here) gives best results and improve overall performance better in term of accuracy and speed. These advantages of ReLU in CNN at number of hidden layers are helpful to effectively and fast retrieval of images from the databases.

scholarly journals Hardware implementation of radial-basis neural networks with Gaussian activation functions on FPGA

2021 ◽  
Author(s):  
Volodymyr Shymkovych ◽  
Sergii Telenyk ◽  
Petro Kravets
Keyword(s):  
Neural Networks ◽  
Hardware Implementation ◽  
Gaussian Function ◽  
Activation Function ◽  
Rbf Neural Networks ◽  
Activation Functions ◽  
Rbf Network ◽  
Combination Scheme ◽  
Radial Basis ◽  
Hidden Layer

AbstractThis article introduces a method for realizing the Gaussian activation function of radial-basis (RBF) neural networks with their hardware implementation on field-programmable gaits area (FPGAs). The results of modeling of the Gaussian function on FPGA chips of different families have been presented. RBF neural networks of various topologies have been synthesized and investigated. The hardware component implemented by this algorithm is an RBF neural network with four neurons of the latent layer and one neuron with a sigmoid activation function on an FPGA using 16-bit numbers with a fixed point, which took 1193 logic matrix gate (LUTs—LookUpTable). Each hidden layer neuron of the RBF network is designed on an FPGA as a separate computing unit. The speed as a total delay of the combination scheme of the block RBF network was 101.579 ns. The implementation of the Gaussian activation functions of the hidden layer of the RBF network occupies 106 LUTs, and the speed of the Gaussian activation functions is 29.33 ns. The absolute error is ± 0.005. The Spartan 3 family of chips for modeling has been used to get these results. Modeling on chips of other series has been also introduced in the article. RBF neural networks of various topologies have been synthesized and investigated. Hardware implementation of RBF neural networks with such speed allows them to be used in real-time control systems for high-speed objects.

Analysis and Design of Associative Memories Based on Recurrent Neural Networks with Linear Saturation Activation Functions and Time-Varying Delays

2007 ◽  
Vol 19 (8) ◽  
pp. 2149-2182 ◽  
Author(s):  
Zhigang Zeng ◽  
Jun Wang
Keyword(s):  
Neural Networks ◽  
Recurrent Neural Networks ◽  
Sufficient Conditions ◽  
Time Varying ◽  
Activation Functions ◽  
Analysis And Design ◽  
Design Procedures ◽  
The Neural Network ◽  
Saturation Region ◽  
The Neural Networks

In this letter, some sufficient conditions are obtained to guarantee recurrent neural networks with linear saturation activation functions, and time-varying delays have multiequilibria located in the saturation region and the boundaries of the saturation region. These results on pattern characterization are used to analyze and design autoassociative memories, which are directly based on the parameters of the neural networks. Moreover, a formula for the numbers of spurious equilibria is also derived. Four design procedures for recurrent neural networks with linear saturation activation functions and time-varying delays are developed based on stability results. Two of these procedures allow the neural network to be capable of learning and forgetting. Finally, simulation results demonstrate the validity and characteristics of the proposed approach.

scholarly journals MULTILAYER PERCEPTRON ARTIFICIAL NEURAL NETWORKS: AN APPROACH FOR LEARNING THROUGH THE BAYESIAN FRAMEWORK

2021 ◽  
Vol 39 (1) ◽  
pp. 45
Author(s):  
Suellen Teixeira Zavadzki de PAULI ◽  
Mariana KLEINA ◽  
Wagner Hugo BONAT
Keyword(s):  
Neural Networks ◽  
Multilayer Perceptron ◽  
Bayesian Method ◽  
Simulated Data ◽  
Activation Functions ◽  
Mlp Neural Network ◽  
Network Learning ◽  
Learning Capacity ◽  
Input Layer ◽  
Hidden Layer

The machine learning area has recently gained prominence and articial neural networks are among the most popular techniques in this eld. Such techniques have the learning capacity that occurs during an iterative process of model tting. Multilayer perceptron (MLP) is one of the rst networks that emerged and, for thisarchitecture, backpropagation and its modications are widely used learning algorithms. In this article, the learning of the MLP neural network was approached from the Bayesian perspective by using Monte Carlo via Markov Chains (MCMC) simulations. The MLP architecture consists of the input, hidden and output layers. In the structure, there are several weights that connect each neuron in each layer. The input layer is composedof the covariates of the model. In the hidden layer there are activation functions. In the output layer, there are the result which is compared with the observed value and the loss function is calculated. We analyzed the network learning through simulated data of known weights in order to understand the estimation by the Bayesian method. Subsequently, we predicted the price of WTI oil and obtained a credibility interval for theforecasts. We provide an R implementation and the datasets as supplementary materials.

scholarly journals Application of artificial neural networks for predicting the physical composition of municipal solid waste: An assessment of the impact of seasonal variation

2021 ◽  
pp. 0734242X2199164 ◽  
Author(s):  
Oluwatobi Adeleke ◽  
Stephen A Akinlabi ◽  
Tien-Chien Jen ◽  
Israel Dunmade
Keyword(s):  
Neural Networks ◽  
Artificial Neural Networks ◽  
Waste Management ◽  
Maximum Temperature ◽  
Activation Functions ◽  
Waste Streams ◽  
Textile Waste ◽  
Physical Composition ◽  
Artificial Neural ◽  
Hidden Layer

Sustainable planning of waste management is contingent on reliable data on waste characteristics and their variation across the seasons owing to the consequential environmental impact of such variation. Traditional waste characterization techniques in most developing countries are time-consuming and expensive; hence the need to address the issue from a modelling approach arises. In modelling the complexity within the system, a paradigm shift from the classical models to the intelligent models has been observed. The application of artificial intelligence models in waste management is gaining traction; however its application in predicting the physical composition of waste is still lacking. This study aims at investigating the optimal combinations of network architecture, training algorithm and activation functions that accurately predict the fraction of physical waste streams from meteorological parameters using artificial neural networks. The city of Johannesburg was used as a case study. Maximum temperature, minimum temperature, wind speed and humidity were used as input variables to predict the percentage composition of organic, paper, plastics and textile waste streams. Several sub-models were stimulated with combination of nine training algorithms and four activation functions in each single hidden layer topology with a range of 1–15 neurons. Performance metrics used to evaluate the accuracy of the system are, root mean square error, mean absolute deviation, mean absolute percentage error and correlation coefficient (R). Optimal architectures in the order of input layer-number of neurons in the hidden layer-output layer for predicting organic, paper, plastics and textile waste were 4-10-1, 4-14-1, 4-5-1 and 4-8-1 with R-values of 0.916, 0.862, 0.834 and 0.826, respectively at the testing phase. The result of the study verifies that waste composition prediction can be done in a single hidden-layer satisfactorily.

scholarly journals Overview of Configuring Adaptive Activation Functions for Deep Neural Networks - A Comparative Study

2021 ◽  
Vol 3 (1) ◽  
pp. 10-22
Author(s):  
Wang Haoxiang ◽  
Smys S
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Learning Process ◽  
Deep Neural Networks ◽  
Research Work ◽  
Nonlinear Function ◽  
Activation Function ◽  
Classification Error ◽  
Activation Functions ◽  
Research Article

Recently, the deep neural networks (DNN) have demonstrated many performances in the pattern recognition paradigm. The research studies on DNN include depth layer networks, filters, training and testing datasets. Deep neural network is providing many solutions for nonlinear partial differential equations (PDE). This research article comprises of many activation functions for each neuron. Besides, these activation networks are allowing many neurons within the neuron networks. In this network, the multitude of the functions will be selected between node by node to minimize the classification error. This is the reason for selecting the adaptive activation function for deep neural networks. Therefore, the activation functions are adapted with every neuron on the network, which is used to reduce the classification error during the process. This research article discusses the scaling factor for activation function that provides better optimization for the process in the dynamic changes of procedure. The proposed adaptive activation function has better learning capability than fixed activation function in any neural network. The research articles compare the convergence rate, early training function, and accuracy between existing methods. Besides, this research work provides improvements in debt ideas of the learning process of various neural networks. This learning process works and tests the solution available in the domain of various frequency bands. In addition to that, both forward and inverse problems of the parameters in the overriding equation will be identified. The proposed method is very simple architecture and efficiency, robustness, and accuracy will be high when considering the nonlinear function. The overall classification performance will be improved in the resulting networks, which have been trained with common datasets. The proposed work is compared with the recent findings in neuroscience research and proved better performance.

Sign in / Sign up

Export Citation Format

Share Document