Applying Machine Learning to the Task of Generating Search Queries

In this paper we research two modifications of recurrent neural networks – Long Short-Term Memory networks and networks with Gated Recurrent Unit with the addition of an attention mechanism to both networks, as well as the Transformer model in the task of generating queries to search engines. GPT-2 by OpenAI was used as the Transformer, which was trained on user queries. Latent-semantic analysis was carried out to identify semantic similarities between the corpus of user queries and queries generated by neural networks. The corpus was convert-ed into a bag of words format, the TFIDF model was applied to it, and a singular value decomposition was performed. Semantic similarity was calculated based on the cosine measure. Also, for a more complete evaluation of the applicability of the models to the task, an expert analysis was carried out to assess the coherence of words in artificially created queries.

Download Full-text

An Autoencoder Gated Recurrent Unit for Remaining Useful Life Prediction

Processes ◽

10.3390/pr8091155 ◽

2020 ◽

Vol 8 (9) ◽

pp. 1155

Author(s):

Yi-Wei Lu ◽

Chia-Yu Hsu ◽

Kuang-Chieh Huang

Keyword(s):

Neural Networks ◽

Recurrent Neural Networks ◽

Short Term Memory ◽

Development Trend ◽

Production Equipment ◽

Remaining Useful Life ◽

Sensor Data ◽

Smart Manufacturing ◽

Useful Life ◽

Gated Recurrent Unit

With the development of smart manufacturing, in order to detect abnormal conditions of the equipment, a large number of sensors have been used to record the variables associated with production equipment. This study focuses on the prediction of Remaining Useful Life (RUL). RUL prediction is part of predictive maintenance, which uses the development trend of the machine to predict when the machine will malfunction. High accuracy of RUL prediction not only reduces the consumption of manpower and materials, but also reduces the need for future maintenance. This study focuses on detecting faults as early as possible, before the machine needs to be replaced or repaired, to ensure the reliability of the system. It is difficult to extract meaningful features from sensor data directly. This study proposes a model based on an Autoencoder Gated Recurrent Unit (AE-GRU), in which the Autoencoder (AE) extracts the important features from the raw data and the Gated Recurrent Unit (GRU) selects the information from the sequences to forecast RUL. To evaluate the performance of the proposed AE-GRU model, an aircraft turbofan engine degradation simulation dataset provided by NASA was used and a comparison made of different recurrent neural networks. The results demonstrate that the AE-GRU is better than other recurrent neural networks, such as Long Short-Term Memory (LSTM) and GRU.

Download Full-text

A comparative study on long short-term memory and gated recurrent unit neural networks in fault diagnosis for chemical processes using visualization

Journal of the Taiwan Institute of Chemical Engineers ◽

10.1016/j.jtice.2021.08.016 ◽

2021 ◽

Author(s):

Somayeh Mirzaei ◽

Jia-Lin Kang ◽

Kuang-Yi Chu

Keyword(s):

Neural Networks ◽

Fault Diagnosis ◽

Comparative Study ◽

Short Term Memory ◽

Chemical Processes ◽

Short Term ◽

Term Memory ◽

Long Short Term Memory ◽

Gated Recurrent Unit

Download Full-text

Deep Neural Networks for the Classification of Pure and Impure Strawberry Purees

Sensors ◽

10.3390/s20041223 ◽

2020 ◽

Vol 20 (4) ◽

pp. 1223 ◽

Cited By ~ 1

Author(s):

Zhong Zheng ◽

Xin Zhang ◽

Jinxing Yu ◽

Rui Guo ◽

Lili Zhangzhong

Keyword(s):

Neural Networks ◽

Time Series ◽

Classification Accuracy ◽

Deep Neural Networks ◽

Short Term Memory ◽

Convolutional Network ◽

Different Types ◽

Long Short Term Memory ◽

Gated Recurrent Unit

In this paper, a comparative study of the effectiveness of deep neural networks (DNNs) in the classification of pure and impure purees is conducted. Three different types of deep neural networks (DNNs)—the Gated Recurrent Unit (GRU), the Long Short Term Memory (LSTM), and the temporal convolutional network (TCN)—are employed for the detection of adulteration of strawberry purees. The Strawberry dataset, a time series spectroscopy dataset from the UCR time series classification repository, is utilized to evaluate the performance of different DNNs. Experimental results demonstrate that the TCN is able to obtain a higher classification accuracy than the GRU and LSTM. Moreover, the TCN achieves a new state-of-the-art classification accuracy on the Strawberry dataset. These results indicates the great potential of using the TCN for the detection of adulteration of fruit purees in the future.

Download Full-text

LSTM and GRU Neural Networks as Models of Dynamical Processes Used in Predictive Control: A Comparison of Models Developed for Two Chemical Reactors

Sensors ◽

10.3390/s21165625 ◽

2021 ◽

Vol 21 (16) ◽

pp. 5625

Author(s):

Krzysztof Zarzycki ◽

Maciej Ławryńczuk

Keyword(s):

Neural Networks ◽

Predictive Control ◽

Short Term Memory ◽

Model Accuracy ◽

Dynamical Processes ◽

Significant Deterioration ◽

Control Quality ◽

Ph Process ◽

Long Short Term Memory ◽

Gated Recurrent Unit

This work thoroughly compares the efficiency of Long Short-Term Memory Networks (LSTMs) and Gated Recurrent Unit (GRU) neural networks as models of the dynamical processes used in Model Predictive Control (MPC). Two simulated industrial processes were considered: a polymerisation reactor and a neutralisation (pH) process. First, MPC prediction equations for both types of models were derived. Next, the efficiency of the LSTM and GRU models was compared for a number of model configurations. The influence of the order of dynamics and the number of neurons on the model accuracy was analysed. Finally, the efficiency of the considered models when used in MPC was assessed. The influence of the model structure on different control quality indicators and the calculation time was discussed. It was found that the GRU network, although it had a lower number of parameters than the LSTM one, may be successfully used in MPC without any significant deterioration of control quality.

Download Full-text

Evaluation of Mixed Deep Neural Networks for Reverberant Speech Enhancement

Biomimetics ◽

10.3390/biomimetics5010001 ◽

2019 ◽

Vol 5 (1) ◽

pp. 1 ◽

Cited By ~ 1

Author(s):

Michelle Gutiérrez-Muñoz ◽

Astryd González-Salazar ◽

Marvin Coto-Jiménez

Keyword(s):

Neural Networks ◽

Short Term Memory ◽

Computational Cost ◽

Real Life ◽

Fixed Number ◽

Training Procedure ◽

Statistical Validation ◽

Significant Drop ◽

Training Time ◽

Important Solution

Speech signals are degraded in real-life environments, as a product of background noise or other factors. The processing of such signals for voice recognition and voice analysis systems presents important challenges. One of the conditions that make adverse quality difficult to handle in those systems is reverberation, produced by sound wave reflections that travel from the source to the microphone in multiple directions. To enhance signals in such adverse conditions, several deep learning-based methods have been proposed and proven to be effective. Recently, recurrent neural networks, especially those with long short-term memory (LSTM), have presented surprising results in tasks related to time-dependent processing of signals, such as speech. One of the most challenging aspects of LSTM networks is the high computational cost of the training procedure, which has limited extended experimentation in several cases. In this work, we present a proposal to evaluate the hybrid models of neural networks to learn different reverberation conditions without any previous information. The results show that some combinations of LSTM and perceptron layers produce good results in comparison to those from pure LSTM networks, given a fixed number of layers. The evaluation was made based on quality measurements of the signal’s spectrum, the training time of the networks, and statistical validation of results. In total, 120 artificial neural networks of eight different types were trained and compared. The results help to affirm the fact that hybrid networks represent an important solution for speech signal enhancement, given that reduction in training time is on the order of 30%, in processes that can normally take several days or weeks, depending on the amount of data. The results also present advantages in efficiency, but without a significant drop in quality.

Download Full-text