scholarly journals Modification of the Marquardt method for training a neural network predictor in eddy viscosity models

2021 ◽  
pp. 27-34
Author(s):  
Vladimir Viktorovich Pekunov
Keyword(s):  
Neural Network ◽  
Turbulent Flows ◽  
Eddy Viscosity ◽  
Random Search ◽  
Optimization Techniques ◽  
Marquardt Method ◽  
Viscosity Models ◽  
The Neural Network ◽  
Limited Series ◽  
Speed Up

The subject of this article is the numerical optimization techniques used in training neural networks that serve as predicate components in certain modern eddy viscosity models. Qualitative solution to the problem of training (minimization of the functional of neural network offsets) often requires significant computational costs, which necessitates to increase the speed of such training based on combination of numerical methods and parallelization of calculations. The Marquardt method draws particular interest, as it contains  the parameter that allows speeding up the solution by switching the method from the descent away from the solution to the Newton’s method of approximate solution. The article offers modification of the Marquardt method, which uses the limited series of random samples for improving the current point and calculate the parameter of the method. The author demonstrate descent characteristics of the method in numerical experiments, both on the test functions of Himmelblau and Rosenbrock, as well as the actual task of training the neural network predictor applies in modeling of the turbulent flows. The use of this method may significantly speed up the training of neural network predictor in corrective models of eddy viscosity. The method is less time-consuming in comparison with random search, namely in terms of a small amount of compute kernels; however, it provides solution that is close to the result of random search and is better than the original Marquardt method.

scholarly journals A Scaled Conjugate Gradient Algorithm for Fast Supervised Learning

1990 ◽  
Vol 19 (339) ◽  
Author(s):  
Martin F. Møller
Keyword(s):  
Neural Network ◽  
Supervised Learning ◽  
Conjugate Gradient ◽  
Gradient Methods ◽  
Optimization Techniques ◽  
Gradient Algorithm ◽  
Convergence Criterion ◽  
Scaled Conjugate Gradient ◽  
The Neural Network ◽  
Speed Up

<p>A supervised learning algorithm (Scaled Conjugate Gradient, SCG) with superlinear convergence rate is introduced. The algorithm is based upon a class of optimization techniques well known in numerical analysis as the Conjugate Gradient Methods. SCG uses second order information from the neural network but requires only O(N) memory usage, where N is the number of weights in the network. The performance of SCG is benchmarked against the performance of the standard backpropagation algorithm (BP), the conjugate gradient backpropagation (CGB) and the one-step Broyden-Fletcher-Goldfarb-Shanno memoryless quasi-Newton algorithm (BFGS). SCG yields a speed-up of at least an order of magnitude relative to BP. The speed-up depends on the convergence criterion, i.e., the bigger demand for reduction in error the bigger the speed-up. SCG is fully automated including no user dependent parameters and avoids a time consuming line-search, which CGB and BFGS use in each iteration in order to determine an appropriate step size.</p><p> </p><p>Incorporating problem dependent structural information in the architecture of a neural network often lowers the overall complexity. The smaller the complexity of the neural network relative to the problem domain, the bigger the possibility that the weight space contains long ravines characterized by sharp curvature. While BP is inefficient on these ravine phenomena, it is shown that SCG handles them effectively.</p>

Numerical Investigation of Low-Reynolds Number Airfoil Flows Using Transition-Sensitive and Fully Turbulent RANS Models

2014 ◽  
Author(s):  
Varun Chitta ◽  
Tausif Jamal ◽  
D. Keith Walters
Keyword(s):  
Boundary Layer ◽  
Turbulent Flows ◽  
Eddy Viscosity ◽  
Boundary Layer Transition ◽  
Transition Flow ◽  
Layer Transition ◽  
Laminar Separation ◽  
Viscosity Models ◽  
Separation Bubbles ◽  
Laminar Separation Bubbles

A numerical analysis is performed to study the pre-stall and post-stall aerodynamic characteristics over a group of six airfoils using commercially available transition-sensitive and fully turbulent eddy-viscosity models. The study is focused on a range of Reynolds numbers from 6 × 104 to 2 × 106, wherein the flow around the airfoil is characterized by complex phenomena such as boundary layer transition, flow separation and reattachment, and formation of laminar separation bubbles on either the suction, pressure or both surfaces of airfoil. The predictive capability of the transition-sensitive k-kL-ω model versus the fully turbulent SST k-ω model is investigated for all airfoils. The transition-sensitive k-kL-ω model used in this study is capable of predicting both attached and separated turbulent flows over the surface of an airfoil without the need for an external linear stability solver to predict transition. The comparison between experimental data and results obtained from the numerical simulations is presented, which shows that the boundary layer transition and laminar separation bubbles that appear on the suction and pressure surfaces of the airfoil can be captured accurately by the use of a transition-sensitive model. The fully turbulent SST k-ω model predicts a turbulent boundary layer on both surfaces of the airfoil for all angles of attack and fails to predict boundary layer transition or separation bubbles. Discrepancies are observed in the predictions of airfoil stall by both the models. Reasons for the discrepancies between computational and experimental results, and also possible improvements in eddy-viscosity models, are discussed.

scholarly journals TRAINING NEURAL NETWORK FOR TAXI PASSENGER DEMAND FORECASTING USING GRAPHICS PROCESSING UNITS

2020 ◽  
Vol 2 (1) ◽  
pp. 29-36
Author(s):  
M. I. Zghoba ◽  
◽  
Yu. I. Hrytsiuk ◽  
Keyword(s):  
Neural Network ◽  
Graphics Processing Units ◽  
Neural Network Training ◽  
Training Time ◽  
Passenger Demand ◽  
Network Training ◽  
The Neural Network ◽  
Input Dataset ◽  
Speed Up ◽  
Graphics Processing

The peculiarities of neural network training for forecasting taxi passenger demand using graphics processing units are considered, which allowed to speed up the training procedure for different sets of input data, hardware configurations, and its power. It has been found that taxi services are becoming more accessible to a wide range of people. The most important task for any transportation company and taxi driver is to minimize the waiting time for new orders and to minimize the distance from drivers to passengers on order receiving. Understanding and assessing the geographical passenger demand that depends on many factors is crucial to achieve this goal. This paper describes an example of neural network training for predicting taxi passenger demand. It shows the importance of a large input dataset for the accuracy of the neural network. Since the training of a neural network is a lengthy process, parallel training was used to speed up the training. The neural network for forecasting taxi passenger demand was trained using different hardware configurations, such as one CPU, one GPU, and two GPUs. The training times of one epoch were compared along with these configurations. The impact of different hardware configurations on training time was analyzed in this work. The network was trained using a dataset containing 4.5 million trips within one city. The results of this study show that the training with GPU accelerators doesn't necessarily improve the training time. The training time depends on many factors, such as input dataset size, splitting of the entire dataset into smaller subsets, as well as hardware and power characteristics.

scholarly journals Efficient Neural Network Modeling for Flight and Space Dynamics Simulation

2011 ◽  
Vol 2011 ◽  
pp. 1-7 ◽  
Author(s):  
Ayman Hamdy Kassem
Keyword(s):  
Neural Network ◽  
Dynamic Systems ◽  
Dynamic Models ◽  
Network Modeling ◽  
Linear Equations ◽  
Dynamics Simulation ◽  
Neural Network Modeling ◽  
The Neural Network ◽  
Speed Up ◽  
Space Dynamics

This paper represents an efficient technique for neural network modeling of flight and space dynamics simulation. The technique will free the neural network designer from guessing the size and structure for the required neural network model and will help to minimize the number of neurons. For linear flight/space dynamics systems, the technique can find the network weights and biases directly by solving a system of linear equations without the need for training. Nonlinear flight dynamic systems can be easily modeled by training its linearized models keeping the same network structure. The training is fast, as it uses the linear system knowledge to speed up the training process. The technique is tested on different flight/space dynamic models and showed promising results.

Research on the Application of the Prediction of the Expressway Traffic Flow Based on the Neural Network with Genetic Algorithm

2011 ◽  
Vol 189-193 ◽  
pp. 4400-4404 ◽  
Author(s):  
Chun Mei Zhu ◽  
Chang Peng Yan ◽  
Xiao Li Xu ◽  
Guo Xin Wu
Keyword(s):  
Neural Network ◽  
Genetic Algorithm ◽  
Traffic Flow ◽  
Bp Neural Network ◽  
Mixed Model ◽  
The Neural Network ◽  
Neural Network Prediction ◽  
Speed Up ◽  
Neural Network Algorithm ◽  
Genetic Neural Network

In order to improve the efficiency and accuracy of the prediction of expressway traffic flow, this paper, based on the characteristics of the data of the expressway traffic flow, focuses on an optimized method of prediction with the application of the neural network with genetic algorithm. Applying genetic algorithm, optimizing BP neural network structure and establishing a new mixed model, this algorithm speed up the slow convergence velocity of traditional BP neural network prediction and increases the possibility to escape local minima. This algorithm based on the optimized genetic neural network predicts the actual data of the expressway traffic flow, the result of which shows that the application of the optimized method of prediction with the genetic neural network algorithm is effective and that it improves the rate and the accuracy of the prediction of the expressway traffic flow.

Review: Adapting Scalar Turbulence Closure Models for Rotation and Curvature

2011 ◽  
Vol 133 (6) ◽  
Author(s):  
Paul Durbin
Keyword(s):  
Growth Rate ◽  
Turbulent Flows ◽  
Equilibrium Solution ◽  
Eddy Viscosity ◽  
Turbulent Energy ◽  
Streamline Curvature ◽  
Viscosity Models ◽  
Scalar Turbulence ◽  
Convective Derivative ◽  
Effects Of Rotation

Scalar, eddy viscosity models are widely used for predicting engineering turbulent flows. System rotation, or streamline curvature, can enhance or reduce the intensity of turbulence. Methods to incorporate the effects of rotation and streamline curvature consist of introducing parametric variation of model coefficients, such that either the growth rate of turbulent energy is altered; or such that the equilibrium solution bifurcates from healthy to decaying solution branches. For general use, parameters must be developed in coordinate invariant forms. Effects of rotation and of curvature can be unified by introducing the convective derivative of the rate of strain eigenvectors as their measure.

scholarly journals Development of a Neural Network for the Synthesis of Freeform Optical Elements with a Flat Wavefront

2020 ◽  
pp. short45-1-short45-9
Author(s):  
Iana Mazur ◽  
Anna Voznesenskaya ◽  
Alexander Trifanov ◽  
Mikhail Svintsov
Keyword(s):  
Neural Network ◽  
Inverse Problem ◽  
Optical Systems ◽  
Direct Algorithm ◽  
Further Training ◽  
Optical Elements ◽  
The Neural Network ◽  
Speed Up ◽  
Tracing Algorithm ◽  
The Given

In this work, a direct algorithm for modeling optical systems using freeform surfaces is considered, which allows you to form a given illumination distribution of illuminating image systems of diffraction quality. Using the proposed ray tracing algorithm based on the laws of geometric optics, a database of optical systems for further training of the neural network is formed. To increase efficiency, the algorithm is tested on a sample of 10,000 pairs of various optical systems. Using a neural network, the inverse problem of calculating optical systems is solved - according to the given parameters of the object and image, the neural network generates a result in the form of a design of freeform optical elements. Further training of the neural network will speed up the design of new optical systems, and the potential for its learning opens up new opportunities for the development of better and more efficient optical systems.

scholarly journals Synthetic Data Generation to Speed-up the Object Recognition Pipeline

2021 ◽  
Author(s):  
Damiano Perri ◽  
Marco Simonetti ◽  
Osvaldo Gervasi
Keyword(s):  
Neural Network ◽  
Data Augmentation ◽  
Synthetic Data ◽  
Ease Of Use ◽  
Full Potential ◽  
Data Generation ◽  
Synthetic Data Generation ◽  
The Neural Network ◽  
Speed Up ◽  
Synthetic Images

This paper provides a methodology for the production of synthetic images for training neural networks to recognise shapes and objects. There are many scenarios in which it is difficult, expensive and even dangerous to produce a set of images that is satisfactory for the training of a neural network. The development of 3D modelling software has nowadays reached such a level of realism and ease of use that it seemed natural to explore this innovative path and to give an answer regarding the reliability of this method that bases the training of the neural network on synthetic images. The results obtained in the two proposed use cases, that of the recognition of a pictorial style and that of the recognition of migrants at sea, leads us to support the validity of the approach, provided that the work is conducted in a very scrupulous and rigorous manner, exploiting the full potential of the modelling software. The code produced, which automatically generates the transformations necessary for the data augmentation of each image, and the generation of random environmental conditions in the case of Blender and Unity3D software, is available under the GPL licence on GitHub. The results obtained lead us to affirm that through the good practices presented in the article, we have defined a simple, reliable, economic and safe method to feed the training phase of a neural network dedicated to the recognition of objects and features, to be applied to various contexts.

Sign in / Sign up

Export Citation Format

Share Document