scholarly journals Investigation of the Batch Size Influence on the Quality of Text Generation by the SeqGAN Neural Network

2021 ◽  
Author(s):  
Nikolay Krivosheev ◽  
Ksenia Vik ◽  
Yulia Ivanova ◽  
Vladimir Spitsyn
Keyword(s):  
Neural Network ◽  
Monte Carlo Algorithm ◽  
Batch Size ◽  
Preliminary Training ◽  
Text Generation ◽  
Further Training ◽  
Adversarial Learning ◽  
Network Training ◽  
Testing Algorithms

One of the problems of text generation using the LSTM neural network is a decrease in the quality of generation with an increase in the length of the generated text. There are various solutions to improve the quality of text generation based on generative adversarial neural networks. This work uses preliminary training of the LSTM neural network based on the MLE approach and further training based on the SeqGAN neural network. Based on the presented results, we can conclude that the SeqGAN-based approach allows to increase the quality of text generation according to the NLL and BLEU metrics. The study of the influence of the batch size, in the process of competitive training of the SeqGAN neural network, on the quality of text generation has been carried out. It is shown that with an increase in the batch size, in the process of adversarial learning, the quality of LSTM neural network training increases. In this work, the Monte Carlo algorithm is not used in the training process of the SeqGAN neural network. For training and testing algorithms, image captions from the COCO Image Captions data sample are used. The quality of text generation based on the NLL and BLEU metrics has been assessed. Examples of the results of generating texts with an assessment of the quality of examples according to the BLEU metric are given,

The Application Process of Wavelet Neural Network in Intelligent Evaluation of the Quality of Engineering Anchor Poles

2011 ◽  
Vol 243-249 ◽  
pp. 2969-2972
Author(s):  
Rui Jun Li ◽  
Ya Qing Shi ◽  
Jian Suo Ma ◽  
Xi Yan Jiang
Keyword(s):  
Neural Network ◽  
Wavelet Neural Network ◽  
Application Process ◽  
Destructive Testing ◽  
Network Training ◽  
Quality Detection ◽  
Non Destructive ◽  
The Ideal

Most detection means on the anchorage integrity today still remain on the destructive testing level, which can hardly meet the actual needs of quality detection on large volumes of anchor poles in the anchorage engineering. This paper presents the application process of wavelet neural network in the non-destructive intelligent testing on the quality of engineering anchor poles. Taking the project of "Management Buildings and Museum of China Marine Sports School" in Qingdao as an example, this paper uses neural toolbox of MATLAB to do the network training by selecting training and simulation samples. The ideal training results indicate that with the help of neural toolbox of MATLAB, the application process of wavelet neural network can not only make intelligent evaluation of the quality of engineering anchor poles, but also make up traditional means, which can not detect large volumes of anchor poles.

Compact Disc Visual Inspection Using Neural Network

2012 ◽  
Vol 433-440 ◽  
pp. 727-732
Author(s):  
Anton Satria Prabuwono ◽  
Siti Rahayu Zulkipli ◽  
Doli Anggia Harahap ◽  
Wendi Usino ◽  
A. Hasniaty
Keyword(s):  
Neural Network ◽  
Image Processing ◽  
Visual Inspection ◽  
Processing Technique ◽  
Compact Disc ◽  
Image Processing Technique ◽  
Accuracy Rate ◽  
Network Training ◽  
Fields Of Study

Image processing is widely used in various fields of study including manufacturing as product inspection. In compact disc manufacturing, image processing has been implemented to recognize defect products. In this research, we implemented image processing technique as pre-processing processes. The aim is to acquire simple image to be processed and analyzed. In order to express the object from the image, the features were extracted using Invariant Moment (IM). Afterward, neural network was used to train the input from IM’s results. Thus, decision can be made whether the compact disc is accepted or rejected based on the training. Two experiments have been done in this research to evaluate 40 datasets of good and defective images of compact discs. The result shows that accuracy rate increased and can identify the quality of compact discs based on neural network training.

Noise Injection: Theoretical Prospects

1997 ◽  
Vol 9 (5) ◽  
pp. 1093-1108 ◽  
Author(s):  
Yves Grandvalet ◽  
Stéphane Canu ◽  
Stéphane Boucheron
Keyword(s):  
Neural Network ◽  
Cost Function ◽  
Heat Kernel ◽  
Smoothness Assumption ◽  
Network Training ◽  
Gaussian Analysis ◽  
Noise Injection ◽  
Global Smoothness ◽  
Input Perturbation

Noise injection consists of adding noise to the inputs during neural network training. Experimental results suggest that it might improve the generalization ability of the resulting neural network. A justification of this improvement remains elusive: describing analytically the average perturbed cost function is difficult, and controlling the fluctuations of the random perturbed cost function is hard. Hence, recent papers suggest replacing the random perturbed cost by a (deterministic) Taylor approximation of the average perturbed cost function. This article takes a different stance: when the injected noise is gaussian, noise injection is naturally connected to the action of the heat kernel. This provides indications on the relevance domain of traditional Taylor expansions and shows the dependence of the quality of Taylor approximations on global smoothness properties of neural networks under consideration. The connection between noise injection and heat kernel also enables controlling the fluctuations of the random perturbed cost function. Under the global smoothness assumption, tools from gaussian analysis provide bounds on the tail behavior of the perturbed cost. This finally suggests that mixing input perturbation with smoothness-based penalization might be profitable.

High-resolution soil moisture retrieval using a Neural Network approach from Sentinel-1 SAR data

2020 ◽  
Author(s):  
Qi Gao ◽  
Maria Jose Escorihuela ◽  
Nemesio Rodriguez-Fernandez ◽  
Olivier Merlin ◽  
Mehrez Zribi
Keyword(s):  
Neural Network ◽  
Soil Moisture ◽  
High Resolution ◽  
Neural Network Training ◽  
Neural Network Approach ◽  
Auxiliary Data ◽  
Network Training ◽  
The Neural Network ◽  
Sar Data

<p>High-resolution soil moisture product is important for agriculture-related managements including irrigation. We have investigated the Change Detection (CD) method using Sentinel-1 data for 100 m resolution soil moisture retrieval and got a Root Mean Square Error (RMSE) about 0.6 m<sup>3</sup>/m<sup>3</sup>. However, the result of this approach is not accurate enough for high-density crops like corn. Another approach needs to be studied to get better accuracy over all types of crops. The artificial neural network (NN) technique, which involves nonlinear parameterized mapping from an input vector to an output vector, is an appropriate tool for retrieving geophysical parameters from remote sensing data. Many studies have explored the NN approach for processing remotely sensed data, including retrieving soil moisture, however, only a few studies [Notarnicola et al., 2010; Paloscia et al., 2013, etc.] had investigated NN for soil moisture estimation over vegetation-covered areas, especially in a large scale.</p><p>The objective of this study is to develop an approach based on neural networks to estimate soil moisture at high resolution over vegetation-covered areas from Sentinel-1 C-band SAR data. The quality of the output results depends directly on the quality of the input data used to train the NN and the reference data for the training, therefore, we performed our study over Catalonia, where we have many auxiliary data. The study is performed using both VV and VH polarization over the whole Catalonia. Apart from Sentinel-1 SAR data, auxiliary data including Sentinel-2 NDVI, SMAP soil moisture, CCI (ESA Climate Change Initiative) land cover, SIGPAC (Sistema de Información Geográfica de Parcelas Agrícolas) land cover, irrigation index and crop type information from SIGPAC, and DEM (Digital elevation model) are also used for approach development. DISPATCH (Disaggregation based on Physical and Theoretical scale Change) soil moisture product at 1 km resolution is considered as the target in the Neural Network training, adding great value to our study. To prepare the Neural Network training, all data sets are co-registered at 1 km resolution within the same size and resampled for the same dates within one year (2017). Two indexes describing the normalized backscatter difference and soil moisture are introduced as equation (1) and (2):</p><table><tbody><tr><td>Index<sub>1 </sub>= (σ<sup>0</sup><sub>i </sub>- σ<sup>0</sup><sub>min</sub>) / (σ<sup>0</sup><sub>max </sub>- σ<sup>0</sup><sub>min</sub>)</td> <td>(1)</td> </tr><tr><td>Index<sub>2 </sub>= SM<sub>min</sub> + (SM<sub>max </sub>- SM<sub>min</sub>) * Index<sub>1</sub></td> <td>(2)</td> </tr></tbody></table><p> </p><p>Different parameters were tested to train the Neural Network approach, the preliminary results show a correlation value compared with DISPATCH product about 0.71 over croplands, 0.73 over irrigated fields, and 0.65 over forests, considering Index1, Index2 and SMAP soil moisture. Works are still on-going to try to improve the results by better analyzing the SAR data performance over different fields and conditions. The final goal of the study is to produce 100 m resolution soil moisture product. After 1 km resolution study, we will apply the approach at 100m resolution, and the in-situ soil moisture will be used for validation.</p><p>This work is inscribed within the Water4Ever project, which is funded by the European Commission under the framework of the ERA-NET COFUND WATERWORKS 2015 Programme. </p>

scholarly journals APPLICATION OF COMPUTATIONAL TECHNOLOGIES FOR SOLUTION OF PROBLEM OF STOCK MARKET

2020 ◽  
Vol 70 (2) ◽  
pp. 14-20
Author(s):  
D.B. Amirkhan ◽  
◽  
A.B. Shansharkhanov ◽  
Keyword(s):  
Neural Network ◽  
Exchange Rate ◽  
Single Layer ◽  
Prediction Algorithm ◽  
Program Code ◽  
Delta Rule ◽  
Network Training ◽  
Numerical Testing ◽  
The Exchange Rate

The article discusses a method for forecasting the exchange rate. Artificial neural networks act as a forecasting tool. As a currency for the numerical testing of the proposed approach, the oil price in dollars, USD (value in rubles and tenge) was chosen as the most common currency in the world. The data will be processed from 2000 to 2019. In the course of the study, the indicators of the General exchange rate were identified with each other by day. When determining the dollar exchange rate using a single-layer neural network, the Adeline algorithm and the generalized Delta rule were used. Based on the prediction algorithm, the program code is written in Python. It is obvious that the quality of neural network training can be used to further predict the dynamics of the exchange rate.

scholarly journals Specificities of data set preparation and neural network training for object recognition

2021 ◽  
pp. 146-151
Author(s):  
Dmytro Kyrychuk ◽  
Andriy Segin
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Object Recognition ◽  
Neural Network Training ◽  
Data Set ◽  
Network Training ◽  
Object Based ◽  
Recognition Ability ◽  
Trained Network

The paper presents the results of the research on the expediency of training a neural network on images of different clarity and brightness using unevenly distributed lighting on a working area with statically positioned system elements. The use of transfer learning for neural networks to improve the accuracy of object recognition was justified. The object recognition ability of a convolutional neural network while scaling the object relatively to the original was researched. The results of the research on the influence of lighting on the quality of object recognition by a trained network and the influence of background choice for a working area on the quality of object-based feature selection are presented. Based on the results obtained, recommendations for the preparation of individual datasets to improve the quality of training and further object recognition of convolutional neural networks through the elimination of unnecessary variables in images were provided.

scholarly journals Neural network training in SCILAB for classifying mango (Mangifera indica) according to maturity level using the RGB color model

2021 ◽  
Vol 1 (3) ◽  
pp. 186
Author(s):  
Eduardo Castillo-Castaneda
Keyword(s):  
Neural Network ◽  
Artificial Intelligence ◽  
Raw Materials ◽  
Mangifera Indica ◽  
Color Model ◽  
Maturity Level ◽  
Network Training ◽  
The Neural Network ◽  
Blue Background

<p style='text-indent:20px;'>Industries that use fruits as raw materials must, at some point in the process, classify them to discard the unsuitable ones and thus ensure the quality of the final product. To produce mango nectar, it is necessary to ensure that the mango is mature enough to start the extraction of the nectar; however, sorting thousands of mangoes may require many people, who can easily lose attention and reduce the accuracy of the result. Such kind of decision can be supported by current Artificial Intelligence techniques. The theoretical details of the processing are presented, as well as the programming code of the neural network using SCILAB as a computer language; the code includes the color extraction from mango images. SCILAB programming is simple, efficient and does not require computers with large processing capacity. The classification was validated with 30 images (TIF format) of Manila variety mango; the mangoes were placed on a blue background to easily separate the background from the object of interest. Four and six mangoes were used to train the neural network. This application of neural networks is part of an undergraduate course on artificial intelligence, which shows the potential of these techniques for solving real and concrete problems.</p>

scholarly journals Streaming Batch Gradient Tracking for Neural Network Training (Student Abstract)

2020 ◽  
Vol 34 (10) ◽  
pp. 13813-13814
Author(s):  
Siyuan Huang ◽  
Brian D. Hoskins ◽  
Matthew W. Daniels ◽  
Mark D. Stiles ◽  
Gina C. Adam
Keyword(s):  
Neural Network ◽  
Matrix Multiplication ◽  
Network Models ◽  
Principal Component ◽  
Batch Size ◽  
Neural Network Models ◽  
Network Training ◽  
Very Large Datasets ◽  
Vector Matrix ◽  
Batch Data

Faster and more energy efficient hardware accelerators are critical for machine learning on very large datasets. The energy cost of performing vector-matrix multiplication and repeatedly moving neural network models in and out of memory motivates a search for alternative hardware and algorithms. We propose to use streaming batch principal component analysis (SBPCA) to compress batch data during training by using a rank-k approximation of the total batch update. This approach yields comparable training performance to minibatch gradient descent (MBGD) at the same batch size while reducing overall memory and compute requirements.

Model Research of Supercritical CO2 Extraction Tillering Onion Essential Oil Based on Differential Mass Conservation Equation and BP Neural Network

2013 ◽  
Vol 325-326 ◽  
pp. 111-113
Author(s):  
Xiao Mei Lin ◽  
Li Sheng Zhang ◽  
Ye Tian
Keyword(s):  
Neural Network ◽  
Essential Oil ◽  
Bp Neural Network ◽  
Mass Conservation ◽  
Conservation Equation ◽  
Further Training ◽  
Network Training ◽  
Model Research ◽  
Conservation Model ◽  
Mass Conservation Equation

According to supercritical CO2Extraction and characteristics of tiller onion crush the processing, we have established a differential mass conservation model, Extraction rate was accurately obtained in the differential mass conservation model based on the characteristics of the BP neural network. Through further training, we got the network training and simulation figures. Finally, the differential mass conservation model was solved.

Sign in / Sign up

Export Citation Format

Share Document