Comparison of Adaptive Gradient Descent Optimization Algorithms for Estimating Wavenumbers K in 2.5D Electrical Resistivity Modeling

2021 ◽  
Author(s):  
U. Mishra ◽  
A. Bansal ◽  
A. Mandal
Keyword(s):  
Electrical Resistivity ◽  
Gradient Descent ◽  
Optimization Algorithms ◽  
Resistivity Modeling

FW2_5D: A MATLAB 2.5-D electrical resistivity modeling code

2008 ◽  
Vol 34 (12) ◽  
pp. 1645-1654 ◽  
Author(s):  
Adam Pidlisecky ◽  
Rosemary Knight
Keyword(s):  
Electrical Resistivity ◽  
Modeling Code ◽  
Resistivity Modeling

scholarly journals Efficient Private ERM for Smooth Objectives

2017 ◽  
Author(s):  
Jiaqi Zhang ◽  
Kai Zheng ◽  
Wenlong Mou ◽  
Liwei Wang
Keyword(s):  
Gradient Descent ◽  
Optimization Algorithms ◽  
Stochastic Gradient Descent ◽  
Empirical Risk Minimization ◽  
Risk Minimization ◽  
Running Time ◽  
Strongly Convex ◽  
Empirical Risk ◽  
Gradient Descent Algorithm ◽  
Previous State

In this paper, we consider efficient differentially private empirical risk minimization from the viewpoint of optimization algorithms. For strongly convex and smooth objectives, we prove that gradient descent with output perturbation not only achieves nearly optimal utility, but also significantly improves the running time of previous state-of-the-art private optimization algorithms, for both $\epsilon$-DP and $(\epsilon, \delta)$-DP. For non-convex but smooth objectives, we propose an RRPSGD (Random Round Private Stochastic Gradient Descent) algorithm, which provably converges to a stationary point with privacy guarantee. Besides the expected utility bounds, we also provide guarantees in high probability form. Experiments demonstrate that our algorithm consistently outperforms existing method in both utility and running time.

scholarly journals Strong error analysis for stochastic gradient descent optimization algorithms

2020 ◽  
Author(s):  
Arnulf Jentzen ◽  
Benno Kuckuck ◽  
Ariel Neufeld ◽  
Philippe von Wurstemberger
Keyword(s):  
Error Analysis ◽  
Objective Function ◽  
Optimization Algorithm ◽  
Gradient Descent ◽  
Convergence Rates ◽  
Optimization Algorithms ◽  
Stochastic Gradient ◽  
Stochastic Gradient Descent ◽  
Induction Argument ◽  
Machine Learning Applications

Abstract Stochastic gradient descent (SGD) optimization algorithms are key ingredients in a series of machine learning applications. In this article we perform a rigorous strong error analysis for SGD optimization algorithms. In particular, we prove for every arbitrarily small $\varepsilon \in (0,\infty )$ and every arbitrarily large $p{\,\in\,} (0,\infty )$ that the considered SGD optimization algorithm converges in the strong $L^p$-sense with order $1/2-\varepsilon $ to the global minimum of the objective function of the considered stochastic optimization problem under standard convexity-type assumptions on the objective function and relaxed assumptions on the moments of the stochastic errors appearing in the employed SGD optimization algorithm. The key ideas in our convergence proof are, first, to employ techniques from the theory of Lyapunov-type functions for dynamical systems to develop a general convergence machinery for SGD optimization algorithms based on such functions, then, to apply this general machinery to concrete Lyapunov-type functions with polynomial structures and, thereafter, to perform an induction argument along the powers appearing in the Lyapunov-type functions in order to achieve for every arbitrarily large $ p \in (0,\infty ) $ strong $ L^p $-convergence rates.

Gradient Descent Optimization Algorithms for Decoding SCMA Signals

2020 ◽  
pp. 2150002
Author(s):  
Sergio Vidal-Beltrán ◽  
José Luis López Bonilla ◽  
Fernando Martínez Piñón ◽  
Jesús Yalja-Montiel
Keyword(s):  
Machine Learning ◽  
Neural Networks ◽  
Deep Learning ◽  
Wireless Communications ◽  
Performance Analysis ◽  
Multiple Access ◽  
Gradient Descent ◽  
Optimization Algorithms ◽  
Sparse Code ◽  
Simulation Results

Recently, technologies based on neural networks (NNs) and deep learning have improved in different areas of Science such as wireless communications. This study demonstrates the applicability of NN-based receivers for detecting and decoding sparse code multiple access (SCMA) codewords. The simulation results reveal that the proposed receiver provides highly accurate predictions based on new data. Moreover, the performance analysis results of the primary optimization algorithms used in machine learning are presented in this study.

scholarly journals Multilayer perceptron for face recognition

2017 ◽  
Vol 58 ◽  
Author(s):  
Ričardas Toliušis ◽  
Olga Kurasova
Keyword(s):  
Neural Network ◽  
Face Recognition ◽  
Multilayer Perceptron ◽  
Gradient Descent ◽  
Optimization Algorithms ◽  
Effective Model ◽  
Activation Functions ◽  
Facial Landmarks ◽  
Network Parameters ◽  
Euclidean Distances

In this paper, an algorithm is proposed which uses facial landmarks to calculate normalized Euclidean distances between different facial parts and performs faces recognition by using Multilayer Perceptron. In order to determine the most effective model, different neural network parameters have been changed in an experimental way, such as hidden layers and the number of neurons, gradient descent optimization algorithms, error and activation functions, and different sets of distances.

scholarly journals A Hybrid Deep Convolutional Neural Network Approach for Predicting the Traffic Congestion Index

2021 ◽  
Vol 33 (3) ◽  
pp. 373-385
Author(s):  
Duy Tran Quang ◽  
Sang Hoon Bae
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
Traffic Congestion ◽  
Gradient Descent ◽  
Optimization Algorithms ◽  
Deep Convolutional Neural Network ◽  
Short Term ◽  
Probe Vehicles ◽  
Proposed Model ◽  
Congestion Index

Traffic congestion is one of the most important issues in large cities, and the overall travel speed is an important factor that reflects the traffic status on road networks. This study proposes a hybrid deep convolutional neural network (CNN) method that uses gradient descent optimization algorithms and pooling operations for predicting the short-term traffic congestion index in urban networks based on probe vehicles. First, the input data are collected by the probe vehicles to calculate the traffic congestion index (output label). Then, a CNN that uses gradient descent optimization algorithms and pooling operations is applied to enhance its performance. Finally, the proposed model is chosen on the basis of the R-squared (R2) and root mean square error (RMSE) values. In the best-case scenario, the proposed model achieved an R2 value of 98.7%. In addition, the experiments showed that the proposed model significantly outperforms other algorithms, namely the ordinary least squares (OLS), k-nearest neighbors (KNN), random forest (RF), recurrent neural network (RNN), artificial neural network (ANN), and convolutional long short-term memory (ConvLSTM), in predicting traffic congestion index. Furthermore, using the proposed method, the time-series changes in the traffic congestion status can be reliably visualized for the entire urban network.

Sign in / Sign up

Export Citation Format

Share Document