scholarly journals An Optical Flow Based Left-Invariant Metric for Natural Gradient Descent in Affine Image Registration

2021 ◽  
Vol 7 ◽  
Author(s):  
Daniel Jacob Tward
Keyword(s):  
Optical Flow ◽  
Gradient Descent ◽  
Computation Time ◽  
Natural Gradient ◽  
Invariant Metric ◽  
Matrix Groups ◽  
Spatial Alignment ◽  
Gradient Descent Algorithm ◽  
Affine Image ◽  
Gradient Based

Accurate spatial alignment is essential for any population neuroimaging study, and affine (12 parameter linear/translation) or rigid (6 parameter rotation/translation) alignments play a major role. Here we consider intensity based alignment of neuroimages using gradient based optimization, which is a problem that continues to be important in many other areas of medical imaging and computer vision in general. A key challenge is robustness. Optimization often fails when transformations have components with different characteristic scales, such as linear versus translation parameters. Hand tuning or other scaling approaches have been used, but efficient automatic methods are essential for generalizing to new imaging modalities, to specimens of different sizes, and to big datasets where manual approaches are not feasible. To address this we develop a left invariant metric on these two matrix groups, based on the norm squared of optical flow induced on a template image. This metric is used in a natural gradient descent algorithm, where gradients (covectors) are converted to perturbations (vectors) by applying the inverse of the metric to define a search direction in which to update parameters. Using a publicly available magnetic resonance neuroimage database, we show that this approach outperforms several other gradient descent optimization strategies. Due to left invariance, our metric needs to only be computed once during optimization, and can therefore be implemented with negligible computation time.

scholarly journals Implicit Stochastic Gradient Descent Method for Cross-Domain Recommendation System

Sensors ◽  
2020 ◽  
Vol 20 (9) ◽  
pp. 2510
Author(s):  
Nam D. Vo ◽  
Minsung Hong ◽  
Jason J. Jung
Keyword(s):  
Gradient Descent ◽  
Recommendation System ◽  
Computation Time ◽  
Descent Method ◽  
Stochastic Gradient ◽  
Stochastic Gradient Descent ◽  
Gradient Descent Method ◽  
Target Domain ◽  
Cross Domain ◽  
Gradient Descent Algorithm

The previous recommendation system applied the matrix factorization collaborative filtering (MFCF) technique to only single domains. Due to data sparsity, this approach has a limitation in overcoming the cold-start problem. Thus, in this study, we focus on discovering latent features from domains to understand the relationships between domains (called domain coherence). This approach uses potential knowledge of the source domain to improve the quality of the target domain recommendation. In this paper, we consider applying MFCF to multiple domains. Mainly, by adopting the implicit stochastic gradient descent algorithm to optimize the objective function for prediction, multiple matrices from different domains are consolidated inside the cross-domain recommendation system (CDRS). Additionally, we design a conceptual framework for CDRS, which applies to different industrial scenarios for recommenders across domains. Moreover, an experiment is devised to validate the proposed method. By using a real-world dataset gathered from Amazon Food and MovieLens, experimental results show that the proposed method improves 15.2% and 19.7% in terms of computation time and MSE over other methods on a utility matrix. Notably, a much lower convergence value of the loss function has been obtained from the experiment. Furthermore, a critical analysis of the obtained results shows that there is a dynamic balance between prediction accuracy and computational complexity.

scholarly journals An Efficient Riemannian Gradient based Algorithm for Max-Cut Problems

2021 ◽  
Author(s):  
Mohamad Mahdi Mohades ◽  
Mohammad Hossein Kahaei
Keyword(s):  
Optimization Algorithm ◽  
Gradient Descent ◽  
Riemannian Optimization ◽  
Gradient Descent Algorithm ◽  
Max Cut Problem ◽  
Gradient Based ◽  
Simulation Results ◽  
Cut Problems ◽  
Suboptimal Algorithms ◽  
Do So

<p>The max-cut problem addresses the problem of finding a cut for a graph that splits the graph into two subsets of vertices so that the number of edges between these two subsets is as large as possible. However, this problem is NP-Hard, which may be solved by suboptimal algorithms. In this paper, we propose a fast and accurate Riemannian optimization algorithm for solving the max-cut problem. To do so, we develop a gradient descent algorithm and prove its convergence. Our simulation results show that the proposed method is extremely efficient on some already-investigated graphs. Specifically, our method is on average 50 times faster than the best well-known techniques with slightly losing the performance, which is on average 0.9729 of the max-cut value of the others.</p> <p></p>

scholarly journals Boosting Adversarial Attacks on Neural Networks with Better Optimizer

2021 ◽  
Vol 2021 ◽  
pp. 1-9
Author(s):  
Heng Yin ◽  
Hengwei Zhang ◽  
Jindong Wang ◽  
Ruiyu Dou
Keyword(s):  
Neural Networks ◽  
Success Rate ◽  
Gradient Descent ◽  
State Of The Art ◽  
Black Box ◽  
Security Threats ◽  
Gradient Descent Algorithm ◽  
Gradient Based ◽  
Adversarial Examples ◽  
Fast Gradient

Convolutional neural networks have outperformed humans in image recognition tasks, but they remain vulnerable to attacks from adversarial examples. Since these data are crafted by adding imperceptible noise to normal images, their existence poses potential security threats to deep learning systems. Sophisticated adversarial examples with strong attack performance can also be used as a tool to evaluate the robustness of a model. However, the success rate of adversarial attacks can be further improved in black-box environments. Therefore, this study combines a modified Adam gradient descent algorithm with the iterative gradient-based attack method. The proposed Adam iterative fast gradient method is then used to improve the transferability of adversarial examples. Extensive experiments on ImageNet showed that the proposed method offers a higher attack success rate than existing iterative methods. By extending our method, we achieved a state-of-the-art attack success rate of 95.0% on defense models.

scholarly journals An Efficient Riemannian Gradient based Algorithm for Max-Cut Problems

2021 ◽  
Author(s):  
Mohamad Mahdi Mohades ◽  
Mohammad Hossein Kahaei
Keyword(s):  
Optimization Algorithm ◽  
Gradient Descent ◽  
Riemannian Optimization ◽  
Gradient Descent Algorithm ◽  
Max Cut Problem ◽  
Gradient Based ◽  
Simulation Results ◽  
Cut Problems ◽  
Suboptimal Algorithms ◽  
Do So

<p>The max-cut problem addresses the problem of finding a cut for a graph that splits the graph into two subsets of vertices so that the number of edges between these two subsets is as large as possible. However, this problem is NP-Hard, which may be solved by suboptimal algorithms. In this paper, we propose a fast and accurate Riemannian optimization algorithm for solving the max-cut problem. To do so, we develop a gradient descent algorithm and prove its convergence. Our simulation results show that the proposed method is extremely efficient on some already-investigated graphs. Specifically, our method is on average 50 times faster than the best well-known techniques with slightly losing the performance, which is on average 0.9729 of the max-cut value of the others.</p> <p></p>

scholarly journals Revisiting the dynamic interactions between economic growth and environmental pollution in Italy: evidence from a gradient descent algorithm

2021 ◽  
Author(s):  
Marco Mele ◽  
Cosimo Magazzino ◽  
Nicolas Schneider ◽  
Floriana Nicolai
Keyword(s):  
Economic Growth ◽  
Gradient Descent ◽  
Statistical Data ◽  
Stochastic Gradient Descent ◽  
Policy Recommendations ◽  
Dynamic Interactions ◽  
Gradient Descent Algorithm ◽  
The Relationship ◽  
Main Strand ◽  
Yearly Data

AbstractAlthough the literature on the relationship between economic growth and CO2 emissions is extensive, the use of machine learning (ML) tools remains seminal. In this paper, we assess this nexus for Italy using innovative algorithms, with yearly data for the 1960–2017 period. We develop three distinct models: the batch gradient descent (BGD), the stochastic gradient descent (SGD), and the multilayer perceptron (MLP). Despite the phase of low Italian economic growth, results reveal that CO2 emissions increased in the predicting model. Compared to the observed statistical data, the algorithm shows a correlation between low growth and higher CO2 increase, which contradicts the main strand of literature. Based on this outcome, adequate policy recommendations are provided.

Sign in / Sign up

Export Citation Format

Share Document