scholarly journals Quasi-Newton Optimization Methods for Deep Learning Applications

2020 ◽  
pp. 9-38 ◽  
Author(s):  
Jacob Rafati ◽  
Roummel F. Marica
Keyword(s):  
Deep Learning ◽  
Optimization Methods ◽  
Quasi Newton

A REVIEW OF OPTIMIZATION METHODS IN DEEP LEARNING

2018 ◽  
Vol 6 (4) ◽  
pp. 440-447
Author(s):  
Amita Khatana ◽  
◽  
◽  
◽  
V.K Narang ◽  
...  
Keyword(s):  
Deep Learning ◽  
Optimization Methods

scholarly journals Thermal-based early breast cancer detection using inception V3, inception V4 and modified inception MV4

2021 ◽  
Author(s):  
Mohammed Abdulla Salim Al Husaini ◽  
Mohamed Hadi Habaebi ◽  
Teddy Surya Gunawan ◽  
Md Rafiqul Islam ◽  
Elfatih A. A. Elsheikh ◽  
...  
Keyword(s):  
Breast Cancer ◽  
Neural Networks ◽  
Deep Learning ◽  
Convolutional Neural Networks ◽  
Early Stage ◽  
Computational Cost ◽  
Optimization Methods ◽  
Optimization Method ◽  
Deep Convolutional Neural Networks ◽  
Breast Thermography

AbstractBreast cancer is one of the most significant causes of death for women around the world. Breast thermography supported by deep convolutional neural networks is expected to contribute significantly to early detection and facilitate treatment at an early stage. The goal of this study is to investigate the behavior of different recent deep learning methods for identifying breast disorders. To evaluate our proposal, we built classifiers based on deep convolutional neural networks modelling inception V3, inception V4, and a modified version of the latter called inception MV4. MV4 was introduced to maintain the computational cost across all layers by making the resultant number of features and the number of pixel positions equal. DMR database was used for these deep learning models in classifying thermal images of healthy and sick patients. A set of epochs 3–30 were used in conjunction with learning rates 1 × 10–3, 1 × 10–4 and 1 × 10–5, Minibatch 10 and different optimization methods. The training results showed that inception V4 and MV4 with color images, a learning rate of 1 × 10–4, and SGDM optimization method, reached very high accuracy, verified through several experimental repetitions. With grayscale images, inception V3 outperforms V4 and MV4 by a considerable accuracy margin, for any optimization methods. In fact, the inception V3 (grayscale) performance is almost comparable to inception V4 and MV4 (color) performance but only after 20–30 epochs. inception MV4 achieved 7% faster classification response time compared to V4. The use of MV4 model is found to contribute to saving energy consumed and fluidity in arithmetic operations for the graphic processor. The results also indicate that increasing the number of layers may not necessarily be useful in improving the performance.

scholarly journals Neural networks for topology optimization

2019 ◽  
Vol 34 (4) ◽  
pp. 215-223 ◽  
Author(s):  
Ivan Sosnovik ◽  
Ivan Oseledets
Keyword(s):  
Deep Learning ◽  
Topology Optimization ◽  
High Performance ◽  
Current Method ◽  
Optimization Methods ◽  
Proposed Model ◽  
Significant Acceleration ◽  
Decoder Architecture ◽  
Segmentation Task ◽  
Convolutional Encoder

Abstract In this research, we propose a deep learning based approach for speeding up the topology optimization methods. The problem we seek to solve is the layout problem. The main novelty of this work is to state the problem as an image segmentation task. We leverage the power of deep learning methods as the efficient pixel-wise image labeling technique to perform the topology optimization. We introduce convolutional encoder-decoder architecture and the overall approach of solving the above-described problem with high performance. The conducted experiments demonstrate the significant acceleration of the optimization process. The proposed approach has excellent generalization properties. We demonstrate the ability of the application of the proposed model to other problems. The successful results, as well as the drawbacks of the current method, are discussed.

scholarly journals An Application of Optimal Control to Sugarcane Harvesting in Thailand

Computation ◽  
2021 ◽  
Vol 9 (3) ◽  
pp. 36
Author(s):  
Wisanlaya Pornprakun ◽  
Surattana Sungnul ◽  
Chanakarn Kiataramkul ◽  
Elvin James Moore
Keyword(s):  
Optimal Control ◽  
Sugar Industry ◽  
Sugar Content ◽  
Optimization Methods ◽  
North Central ◽  
North East ◽  
Thai Government ◽  
The Government ◽  
Quasi Newton ◽  
Optimal Control Methods

The sugar industry is of great importance to the Thai economy. In general, the government sets sugarcane prices at the beginning of each harvesting season based on type (fresh or fired), sweetness (sugar content) and gross weight. The main aim of the present research is to use optimal control to find optimal sugarcane harvesting policies for fresh and fired sugarcane for the four sugarcane producing regions of Thailand, namely North, Central, East and North-east, for harvesting seasons 2012/13, 2013/14, 2014/15, 2017/18 and 2018/19. The optimality problem is to determine the harvesting policy which gives maximum profit to the farmers subject to constraints on the maximum amount that can be cut in each day, where a harvesting policy is defined as the amount of each type of sugarcane harvested and delivered to the sugar factories during each day of a harvesting season. The results from the optimal control methods are also compared with results from three optimization methods, namely bi-objective, linear programming and quasi-Newton. The results suggest that discrete optimal control is the most effective of the five methods considered. The data used in this paper were obtained from the Ministry of Industry and the Ministry of Agriculture and Co-operatives of the Royal Thai government.

scholarly journals Determining optimal policies for sugarcane harvesting in Thailand using bi-objective and quasi-Newton optimization methods

2019 ◽  
Vol 2019 (1) ◽  
Author(s):  
Wisanlaya Pornprakun ◽  
Surattana Sungnul ◽  
Chanakarn Kiataramkul ◽  
Elvin J. Moore
Keyword(s):  
Optimization Methods ◽  
Optimal Policies ◽  
Quasi Newton

scholarly journals Numerical Experience with Damped Quasi-Newton Optimization Methods when the Objective Function is Quadratic

2012 ◽  
Vol 16 ◽  
pp. 1 ◽  
Author(s):  
Mehiddin Al-Baali ◽  
Anton Purnama
Keyword(s):  
Convergence Property ◽  
Quadratic Function ◽  
Optimization Methods ◽  
Newton Methods ◽  
Objective Functions ◽  
Numerical Experience ◽  
Damped Technique ◽  
Quasi Newton ◽  
Broyden Family

A class of damped quasi-Newton methods for nonlinear optimization has recently been proposed by extending the damped-technique of Powell for the BFGS method to the Broyden family of quasi-Newton methods. It has been shown that this damped class has the global and superlinear convergence property that a restricted class of 'undamped' methods has for convex objective functions in unconstrained optimization. To test this result, we applied several members of the Broyden family and their corresponding damped methods to a simple quadratic function and observed several useful features of the damped-technique. These observations and other numerical experiences are described in this paper. The important role of the damped-technique is shown not only for enforcing the above convergence property, but also for improving the performance of efficient, inefficient and divergent undamped methods substantially (significantly in the latter case). Thus, some appropriate ways for employing the damped-technique are suggested. 

scholarly journals The general design of the automation for multiple fields using reinforcement learning algorithm

2022 ◽  
Vol 25 (1) ◽  
pp. 481
Author(s):  
Vijaya Kumar Reddy Radha ◽  
Anantha N. Lakshmipathi ◽  
Ravi Kumar Tirandasu ◽  
Paruchuri Ravi Prakash
Keyword(s):  
Machine Learning ◽  
Deep Learning ◽  
Reinforcement Learning ◽  
Learning Algorithm ◽  
Optimization Methods ◽  
Graph Representation ◽  
Machine Learning Technique ◽  
Learning Society ◽  
Learning Technique ◽  
Learning Concept

<p>Reinforcement learning is considered as a machine learning technique that is anxious with software agents should behave in particular environment. Reinforcement learning (RL) is a division of deep learning concept that assists you to make best use of some part of the collective return. In this paper evolving reinforcement learning algorithms shows possible to learn a fresh and understable concept by using a graph representation and applying optimization methods from the auto machine learning society. In this observe, we stand for the loss function, it is used to optimize an agent’s parameter in excess of its knowledge, as an imputational graph, and use traditional evolution to develop a population of the imputational graphs over a set of uncomplicated guidance environments. These outcomes in gradually better RL algorithms and the exposed algorithms simplify to more multifaceted environments, even though with visual annotations.</p>

Sign in / Sign up

Export Citation Format

Share Document