scholarly journals A Memory-Efficient Learning Framework for Symbol Level Precoding with Quantized NN Weights

2021 ◽  
Author(s):  
Abdullahi Mohammad ◽  
Christos Masouros ◽  
Yiannis Andreopoulos
Keyword(s):  
Learning Algorithm ◽  
Activation Function ◽  
Stochastic Gradient Descent ◽  
Modulation Scheme ◽  
Training Samples ◽  
Input Dataset ◽  
Parametric Rectified Linear Unit ◽  
Efficient Learning ◽  
Factor Α ◽  
Fully Connected

We consider a downlink situation where the BS is equipped with four antennas (M = 4) that serve single users; and assume a single cell. We obtain the dataset from the channel realizations randomly generated from a normal distribution with zero mean and unit variance. The dataset is reshaped and converted to real number domain.<div>The input dataset is normalized by the transmit data symbol so that data entries are within the nominal range, potentially aiding the training. We generate 50,000 training samples and 2000 test samples, respectively. The transmit data symbols are modulated using a QPSK modulation scheme. The training SINR is obtained randomly from uniform distribution Γtrain∼U(Γlow, Γhigh). Stochastic gradient descent is used with the Lagrangian function as a loss metric. A parametric rectified linear unit (PReLu) activation function is used for convolutional and fully connected layers in a full-precision model and the low-bit activation function for the quantized model. After every iteration, the learning rate is reduced by a factor α= 0.65 to help the learning algorithm converge faster. <br></div>

scholarly journals A Memory-Efficient Learning Framework for Symbol Level Precoding with Quantized NN Weights

2021 ◽  
Author(s):  
Abdullahi Mohammad ◽  
Christos Masouros ◽  
Yiannis Andreopoulos
Keyword(s):  
Learning Algorithm ◽  
Activation Function ◽  
Stochastic Gradient Descent ◽  
Modulation Scheme ◽  
Training Samples ◽  
Input Dataset ◽  
Parametric Rectified Linear Unit ◽  
Efficient Learning ◽  
Factor Α ◽  
Fully Connected

We consider a downlink situation where the BS is equipped with four antennas (M = 4) that serve single users; and assume a single cell. We obtain the dataset from the channel realizations randomly generated from a normal distribution with zero mean and unit variance. The dataset is reshaped and converted to real number domain.<div>The input dataset is normalized by the transmit data symbol so that data entries are within the nominal range, potentially aiding the training. We generate 50,000 training samples and 2000 test samples, respectively. The transmit data symbols are modulated using a QPSK modulation scheme. The training SINR is obtained randomly from uniform distribution Γtrain∼U(Γlow, Γhigh). Stochastic gradient descent is used with the Lagrangian function as a loss metric. A parametric rectified linear unit (PReLu) activation function is used for convolutional and fully connected layers in a full-precision model and the low-bit activation function for the quantized model. After every iteration, the learning rate is reduced by a factor α= 0.65 to help the learning algorithm converge faster. <br></div>

Adaptive Online Learning Algorithms for Blind Separation: Maximum Entropy and Minimum Mutual Information

1997 ◽  
Vol 9 (7) ◽  
pp. 1457-1482 ◽  
Author(s):  
Howard Hua Yang ◽  
Shun-ichi Amari
Keyword(s):  
Mutual Information ◽  
Maximum Entropy ◽  
Learning Algorithm ◽  
Adaptive Method ◽  
Descent Method ◽  
Stochastic Gradient Descent ◽  
Gradient Descent Method ◽  
Natural Gradient ◽  
Blind Separation ◽  
Efficient Learning

There are two major approaches for blind separation: maximum entropy (ME) and minimum mutual information (MMI). Both can be implemented by the stochastic gradient descent method for obtaining the demixing matrix. The MI is the contrast function for blind separation; the entropy is not. To justify the ME, the relation between ME and MMI is first elucidated by calculating the first derivative of the entropy and proving that the mean subtraction is necessary in applying the ME and at the solution points determined by the MI, the ME will not update the demixing matrix in the directions of increasing the cross-talking. Second, the natural gradient instead of the ordinary gradient is introduced to obtain efficient algorithms, because the parameter space is a Riemannian space consisting of matrices. The mutual information is calculated by applying the Gram-Charlier expansion to approximate probability density functions of the outputs. Finally, we propose an efficient learning algorithm that incorporates with an adaptive method of estimating the unknown cumulants. It is shown by computer simulation that the convergence of the stochastic descent algorithms is improved by using the natural gradient and the adaptively estimated cumulants.

scholarly journals Application of Artificial Neural Networks for Mangrove Mapping Using Multi-Temporal and Multi-Source Remote Sensing Imagery

Water ◽  
2022 ◽  
Vol 14 (2) ◽  
pp. 244
Author(s):  
Arsalan Ghorbanian ◽  
Seyed Ali Ahmadi ◽  
Meisam Amani ◽  
Ali Mohammadzadeh ◽  
Sadegh Jamali
Keyword(s):  
Remote Sensing ◽  
Learning Algorithm ◽  
Remote Sensing Data ◽  
Mangrove Ecosystem ◽  
Activation Function ◽  
Ann Model ◽  
Sensing Data ◽  
Training Samples ◽  
Multi Temporal ◽  
Ann Models

Mangroves, as unique coastal wetlands with numerous benefits, are endangered mainly due to the coupled effects of anthropogenic activities and climate change. Therefore, acquiring reliable and up-to-date information about these ecosystems is vital for their conservation and sustainable blue carbon development. In this regard, the joint use of remote sensing data and machine learning algorithms can assist in producing accurate mangrove ecosystem maps. This study investigated the potential of artificial neural networks (ANNs) with different topologies and specifications for mangrove classification in Iran. To this end, multi-temporal synthetic aperture radar (SAR) and multi-spectral remote sensing data from Sentinel-1 and Sentinel-2 were processed in the Google Earth Engine (GEE) cloud computing platform. Afterward, the ANN topologies and specifications considering the number of layers and neurons, learning algorithm, type of activation function, and learning rate were examined for mangrove ecosystem mapping. The results indicated that an ANN model with four hidden layers, 36 neurons in each layer, adaptive moment estimation (Adam) learning algorithm, rectified linear unit (Relu) activation function, and the learning rate of 0.001 produced the most accurate mangrove ecosystem map (F-score = 0.97). Further analysis revealed that although ANN models were subjected to accuracy decline when a limited number of training samples were used, they still resulted in satisfactory results. Additionally, it was observed that ANN models had a high resistance when training samples included wrong labels, and only the ANN model with the Adam learning algorithm produced an accurate mangrove ecosystem map when no data standardization was performed. Moreover, further investigations showed the higher potential of multi-temporal and multi-source remote sensing data compared to single-source and mono-temporal (e.g., single season) for accurate mangrove ecosystem mapping. Overall, the high potential of the proposed method, along with utilizing open-access satellite images and big-geo data processing platforms (i.e., GEE, Google Colab, and scikit-learn), made the proposed approach efficient and applicable over other study areas for all interested users.

Share Market Data Prediction Strategies using Deep Learning Algorithm

2019 ◽  
Vol 13 ◽  
Author(s):  
A John. ◽  
D. Praveen Dominic ◽  
M. Adimoolam ◽  
N. M. Balamurugan
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Stock Market ◽  
Predictive Analytics ◽  
Learning Algorithm ◽  
Market Price ◽  
Stochastic Gradient ◽  
Stochastic Gradient Descent ◽  
Mining Machine ◽  
Gradient Descent Algorithm

Background:: Predictive analytics has a multiplicity of statistical schemes from predictive modelling, data mining, machine learning. It scrutinizes present and chronological data to make predictions about expectations or if not unexplained measures. Most predictive models are used for business analytics to overcome loses and profit gaining. Predictive analytics is used to exploit the pattern in old and historical data. Objective: People used to follow some strategies for predicting stock value to invest in the more profit-gaining stocks and those strategies to search the stock market prices which are incorporated in some intelligent methods and tools. Such strategies will increase the investor’s profits and also minimize their risks. So prediction plays a vital role in stock market gaining and is also a very intricate and challenging process. Method: The proposed optimized strategies are the Deep Neural Network with Stochastic Gradient for stock prediction. The Neural Network is trained using Back-propagation neural networks algorithm and stochastic gradient descent algorithm as optimal strategies. Results: The experiment is conducted for stock market price prediction using python language with the visual package. In this experiment RELIANCE.NS, TATAMOTORS.NS, and TATAGLOBAL.NS dataset are taken as input dataset and it is downloaded from National Stock Exchange site. The artificial neural network component including Deep Learning model is most effective for more than 100,000 data points to train this model. This proposed model is developed on daily prices of stock market price to understand how to build model with better performance than existing national exchange method.

scholarly journals Real-Time AI-Based Informational Decision-Making Support System Utilizing Dynamic Text Sources

2021 ◽  
Vol 11 (13) ◽  
pp. 6237
Author(s):  
Azharul Islam ◽  
KyungHi Chang
Keyword(s):  
Machine Learning ◽  
Decision Making ◽  
Random Forest ◽  
Support System ◽  
Classification Accuracy ◽  
Short Term Memory ◽  
Learning Algorithm ◽  
Unstructured Data ◽  
Stochastic Gradient Descent ◽  
Decision Making Support

Unstructured data from the internet constitute large sources of information, which need to be formatted in a user-friendly way. This research develops a model that classifies unstructured data from data mining into labeled data, and builds an informational and decision-making support system (DMSS). We often have assortments of information collected by mining data from various sources, where the key challenge is to extract valuable information. We observe substantial classification accuracy enhancement for our datasets with both machine learning and deep learning algorithms. The highest classification accuracy (99% in training, 96% in testing) was achieved from a Covid corpus which is processed by using a long short-term memory (LSTM). Furthermore, we conducted tests on large datasets relevant to the Disaster corpus, with an LSTM classification accuracy of 98%. In addition, random forest (RF), a machine learning algorithm, provides a reasonable 84% accuracy. This research’s main objective is to increase the application’s robustness by integrating intelligence into the developed DMSS, which provides insight into the user’s intent, despite dealing with a noisy dataset. Our designed model selects the random forest and stochastic gradient descent (SGD) algorithms’ F1 score, where the RF method outperforms by improving accuracy by 2% (to 83% from 81%) compared with a conventional method.

scholarly journals AIRBORNE HYPERSPECTRAL REMOTE SENSING FOR IDENTIFICATION GRASSLAND VEGETATION

2015 ◽  
Vol XL-3/W3 ◽  
pp. 427-431 ◽  
Author(s):  
P. Burai ◽  
T. Tomor ◽  
L. Bekő ◽  
B. Deák
Keyword(s):  
Image Classification ◽  
Learning Algorithm ◽  
Training Sample ◽  
Hyperspectral Data ◽  
Training Dataset ◽  
Classification Methods ◽  
Grassland Vegetation ◽  
Training Samples ◽  
Almost All ◽  
Noise Fraction

In our study we classified grassland vegetation types of an alkali landscape (Eastern Hungary), using different image classification methods for hyperspectral data. Our aim was to test the applicability of hyperspectral data in this complex system using various image classification methods. To reach the highest classification accuracy, we compared the performance of traditional image classifiers, machine learning algorithm, feature extraction (MNF-transformation) and various sizes of training dataset. Hyperspectral images were acquired by an AISA EAGLE II hyperspectral sensor of 128 contiguous bands (400–1000 nm), a spectral sampling of 5 nm bandwidth and a ground pixel size of 1 m. We used twenty vegetation classes which were compiled based on the characteristic dominant species, canopy height, and total vegetation cover. Image classification was applied to the original and MNF (minimum noise fraction) transformed dataset using various training sample sizes between 10 and 30 pixels. In the case of the original bands, both SVM and RF classifiers provided high accuracy for almost all classes irrespectively of the number of the training pixels. We found that SVM and RF produced the best accuracy with the first nine MNF transformed bands. Our results suggest that in complex open landscapes, application of SVM can be a feasible solution, as this method provides higher accuracies compared to RF and MLC. SVM was not sensitive for the size of the training samples, which makes it an adequate tool for cases when the available number of training pixels are limited for some classes.

scholarly journals Deep Learning-Based Beamforming for Millimeter-Wave Systems Using Parametric ReLU Activation Function

2021 ◽  
Author(s):  
Alshimaa Hamdy ◽  
Tarek Abed Soliman ◽  
Mohamed Rihan ◽  
Moawad I. Dessouky
Keyword(s):  
Deep Learning ◽  
Performance Improvement ◽  
Millimeter Wave ◽  
Antenna Arrays ◽  
Activation Function ◽  
Model Accuracy ◽  
Learning Network ◽  
Parametric Rectified Linear Unit ◽  
Complexity Cost ◽  
Deep Learning Network

Abstract Beamforming design is a crucial stage in millimeter-wave systems with massive antenna arrays. We propose a deep learning network for the design of the precoder and combiner in hybrid architectures. The proposed network employs a parametric rectified linear unit (PReLU) activation function which improves model accuracy with almost no complexity cost compared to other functions. The proposed network accepts practical channel estimation input and can be trained to enhance spectral efficiency considering the hardware limitation of the hybrid design. Simulation shows that the proposed network achieves small performance improvement when compared to the same network with the ReLU activation function.

Sign in / Sign up

Export Citation Format

Share Document