scholarly journals Deep Learning Approach for Prediction of Critical Temperature of Superconductor Materials Described by Chemical Formulas

2021 ◽  
Vol 8 ◽  
Author(s):  
Dmitry Viatkin ◽  
Begonya Garcia-Zapirain ◽  
Amaia Méndez-Zorrilla ◽  
Maxim Zakharov
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Critical Temperature ◽  
Short Term Memory ◽  
Learning Rate ◽  
Coefficient Of Determination ◽  
Chemical Formula ◽  
Short Term ◽  
Term Memory ◽  
Long Short Term Memory

This paper proposes a novel neural network architecture and its ensembles to predict the critical superconductivity temperature of materials based on their chemical formula. The research describes the methods and processes of extracting data from the chemical formula and preparing these extracted data for use in neural network training using TensorFlow. In our approach, recurrent neural networks are used including long short-term memory layers and neural networks based on one-dimensional convolution layers for data analysis. The proposed model is an ensemble of pre-trained neural network architectures for the prediction of the critical temperature of superconductors based on their chemical formula. The architecture of seven pre-trained neural networks is based on the long short-term memory layers and convolution layers. In the final ensemble, six neural networks are used: one network based on LSTM and four based on convolutional neural networks, and one embedding ensemble of convolution neural networks. LSTM neural network and convolution neural network were trained in 300 epochs. Ensembles of models were trained in 20 epochs. All neural networks are trained in two stages. At both stages, the optimizer Adam was used. In the first stage, training was carried out by the function of losses Mean Absolute Error (MAE) with the value of optimizer learning rate equal to 0.001. In the second stage, the previously trained model was trained by the function of losses Mean Squared Error (MSE) with a learning rate equal to 0.0001. The final ensemble is trained with a learning rate equal to 0.00001. The final ensemble model has the following accuracy values: MAE is 4.068, MSE is 67.272, and the coefficient of determination (R2) is 0.923. The final model can predict the critical temperature for the chemistry formula with an accuracy of 4.068°.

scholarly journals MULTI-TEMPORAL LAND COVER CLASSIFICATION WITH LONG SHORT-TERM MEMORY NEURAL NETWORKS

2017 ◽  
Vol XLII-1/W1 ◽  
pp. 551-558 ◽  
Author(s):  
M. Rußwurm ◽  
M. Körner
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Land Cover ◽  
Short Term Memory ◽  
Land Cover Classification ◽  
Support Vector ◽  
Short Term ◽  
Term Memory ◽  
Classification Techniques ◽  
Long Short Term Memory

<i>Land cover classification (LCC)</i> is a central and wide field of research in earth observation and has already put forth a variety of classification techniques. Many approaches are based on classification techniques considering observation at certain points in time. However, some land cover classes, such as crops, change their spectral characteristics due to environmental influences and can thus not be monitored effectively with classical mono-temporal approaches. Nevertheless, these temporal observations should be utilized to benefit the classification process. After extensive research has been conducted on modeling temporal dynamics by spectro-temporal profiles using vegetation indices, we propose a deep learning approach to utilize these temporal characteristics for classification tasks. In this work, we show how <i>long short-term memory</i> (LSTM) neural networks can be employed for crop identification purposes with SENTINEL 2A observations from large study areas and label information provided by local authorities. We compare these temporal neural network models, <i>i.e.</i>, LSTM and <i>recurrent neural network</i> (RNN), with a classical non-temporal <i>convolutional neural network</i> (CNN) model and an additional <i>support vector machine</i> (SVM) baseline. With our rather straightforward LSTM variant, we exceeded state-of-the-art classification performance, thus opening promising potential for further research.

scholarly journals PM<sub>2.5</sub> ∕ PM<sub>10</sub> ratio prediction based on a long short-term memory neural network in Wuhan, China

2020 ◽  
Vol 13 (3) ◽  
pp. 1499-1511 ◽  
Author(s):  
Xueling Wu ◽  
Ying Wang ◽  
Siyuan He ◽  
Zhongfang Wu
Keyword(s):  
Neural Network ◽  
Air Pollution ◽  
Short Term Memory ◽  
Coefficient Of Determination ◽  
Short Term ◽  
Current Output ◽  
Term Memory ◽  
Time Space ◽  
Average Maximum ◽  
Long Short Term Memory

Abstract. Air pollution is a serious problem in China that urgently needs to be addressed. Air pollution has a great impact on the lives of citizens and on urban development. The particulate matter (PM) value is usually used to indicate the degree of air pollution. In addition to that of PM2.5 and PM10, the use of the PM2.5 ∕ PM10 ratio as an indicator and assessor of air pollution has also become more widespread. This ratio reflects the air pollution conditions and pollution sources. In this paper, a better composite prediction system aimed at improving the accuracy and spatiotemporal applicability of PM2.5 ∕ PM10 was proposed. First, the aerosol optical depth (AOD) in 2017 in Wuhan was obtained based on Moderate Resolution Imaging Spectroradiometer (MODIS) images, with a 1 km spatial resolution, by using the dense dark vegetation (DDV) method. Second, the AOD was corrected by calculating the planetary boundary layer height (PBLH) and relative humidity (RH). Third, the coefficient of determination of the optimal subset selection was used to select the factor with the highest correlation with PM2.5 ∕ PM10 from meteorological factors and gaseous pollutants. Then, PM2.5 ∕ PM10 predictions based on time, space, and random patterns were obtained by using nine factors (the corrected AOD, meteorological data, and gaseous pollutant data) with the long short-term memory (LSTM) neural network method, which is a dynamic model that remembers historical information and applies it to the current output. Finally, the LSTM model prediction results were compared and analyzed with the results of other intelligent models. The results showed that the LSTM model had significant advantages in the average, maximum, and minimum accuracy and the stability of PM2.5 ∕ PM10 prediction.

scholarly journals Structured Sparsification of Gated Recurrent Neural Networks

2020 ◽  
Vol 34 (04) ◽  
pp. 4989-4996
Author(s):  
Ekaterina Lobacheva ◽  
Nadezhda Chirkova ◽  
Alexander Markovich ◽  
Dmitry Vetrov
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Text Classification ◽  
Recurrent Neural Networks ◽  
Short Term Memory ◽  
Language Modeling ◽  
Short Term ◽  
Term Memory ◽  
Long Short Term Memory ◽  
Network Compression

One of the most popular approaches for neural network compression is sparsification — learning sparse weight matrices. In structured sparsification, weights are set to zero by groups corresponding to structure units, e. g. neurons. We further develop the structured sparsification approach for the gated recurrent neural networks, e. g. Long Short-Term Memory (LSTM). Specifically, in addition to the sparsification of individual weights and neurons, we propose sparsifying the preactivations of gates. This makes some gates constant and simplifies an LSTM structure. We test our approach on the text classification and language modeling tasks. Our method improves the neuron-wise compression of the model in most of the tasks. We also observe that the resulting structure of gate sparsity depends on the task and connect the learned structures to the specifics of the particular tasks.

scholarly journals PM2.5 / PM10 Ratio Prediction Based on a Long Short-term Memory Neural Network in Wuhan, China

2019 ◽  
Author(s):  
Xueling Wu ◽  
Ying Wang ◽  
Siyuan He ◽  
Zhongfang Wu
Keyword(s):  
Neural Network ◽  
Air Pollution ◽  
Short Term Memory ◽  
Coefficient Of Determination ◽  
Short Term ◽  
Current Output ◽  
Term Memory ◽  
Time Space ◽  
Average Maximum ◽  
Long Short Term Memory

Abstract. Air pollution is a serious and urgent problem in China, and it has a great impact on the lives of residents and urban development. The particulate matter (PM) value is usually used to indicate the degree of air pollution. In addition to PM2.5 and PM10, the use of the PM2.5 / PM10 ratio as an indicator and assessor of air pollution has also become more widespread. This ratio reflects the air pollution conditions and pollution sources. In this paper, a better composite prediction system was proposed that aimed at improving the accuracy and spatio-temporal applicability of PM2.5 / PM10. First, the aerosol optical depth (AOD) in 2017 in Wuhan was obtained based on Moderate Resolution Imaging Spectroradiometer images, with a 1 km spatial resolution, by using the Dense Dark Vegetation method. Second, the AOD was corrected by calculating the planetary boundary layer height and relative humidity. Third, the coefficient of determination of the optimal subset selection was used to select the factor with the highest correlation with PM2.5 / PM10 from meteorological factors and gaseous pollutants. Then, PM2.5 / PM10 predictions based on time, space, and random patterns were obtained by using 9 factors (the corrected AOD, meteorological data and gaseous pollutant data) with the long short-term memory (LSTM) neural network method, which is a dynamic model that remembers historical information and applies it to the current output. Finally, the LSTM model prediction results were compared and analysed with the results of other intelligent models. The results showed that the LSTM model had significant advantages in the average, maximum and minimum accuracies and the stability of PM2.5 / PM10 prediction.

scholarly journals Using Machine Learning Methods in the Financial Market for Technical Analysis Based on Hybrid Models

2020 ◽  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Hybrid Model ◽  
Deep Neural Network ◽  
Deep Neural Networks ◽  
Short Term Memory ◽  
Short Term ◽  
Term Memory ◽  
Input Structure ◽  
Long Short Term Memory

In this study, it is presented a new hybrid model based on deep neural networks to predict the direction and magnitude of the Forex market movement in the short term. The overall model presented is based on the scalping strategy and is provided for high frequency transactions. The proposed hybrid model is based on a combination of three models based on deep neural networks. The first model is a deep neural network with a multi-input structure consisting of a combination of Long Short Term Memory layers. The second model is a deep neural network with a multi-input structure made of a combination of one-dimensional Convolutional Neural network layers. The third model has a simpler structure and is a multi-input model of the Multi-Layer Perceptron layers. The overall model was also a model based on the majority vote of three top models. This study showed that models based on Long Short-Term Memory layers provided better results than the other models and even hybrid models with more than 70% accurate.

scholarly journals Neural Reversible Steganography with Long Short-Term Memory

2021 ◽  
Vol 2021 ◽  
pp. 1-14
Author(s):  
Ching-Chun Chang
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Deep Learning ◽  
Network Model ◽  
Neural Network Model ◽  
Short Term Memory ◽  
Short Term ◽  
Term Memory ◽  
Low Level ◽  
Long Short Term Memory

Deep learning has brought about a phenomenal paradigm shift in digital steganography. However, there is as yet no consensus on the use of deep neural networks in reversible steganography, a class of steganographic methods that permits the distortion caused by message embedding to be removed. The underdevelopment of the field of reversible steganography with deep learning can be attributed to the perception that perfect reversal of steganographic distortion seems scarcely achievable, due to the lack of transparency and interpretability of neural networks. Rather than employing neural networks in the coding module of a reversible steganographic scheme, we instead apply them to an analytics module that exploits data redundancy to maximise steganographic capacity. State-of-the-art reversible steganographic schemes for digital images are based primarily on a histogram-shifting method in which the analytics module is often modelled as a pixel intensity predictor. In this paper, we propose to refine the prior estimation from a conventional linear predictor through a neural network model. The refinement can be to some extent viewed as a low-level vision task (e.g., noise reduction and super-resolution imaging). In this way, we explore a leading-edge neuroscience-inspired low-level vision model based on long short-term memory with a brief discussion of its biological plausibility. Experimental results demonstrated a significant boost contributed by the neural network model in terms of prediction accuracy and steganographic rate-distortion performance.

scholarly journals Stock Index Prices Prediction via Temporal Pattern Attention and Long-Short-Term Memory

2020 ◽  
Vol 2020 ◽  
pp. 1-7
Author(s):  
Xiaolu Wei ◽  
Binbin Lei ◽  
Hongbing Ouyang ◽  
Qiufeng Wu
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Time Series ◽  
Temporal Pattern ◽  
Short Term Memory ◽  
Stock Index ◽  
Support Vector ◽  
Short Term ◽  
Term Memory ◽  
Long Short Term Memory

This study attempts to predict stock index prices using multivariate time series analysis. The study’s motivation is based on the notion that datasets of stock index prices involve weak periodic patterns, long-term and short-term information, for which traditional approaches and current neural networks such as Autoregressive models and Support Vector Machine (SVM) may fail. This study applied Temporal Pattern Attention and Long-Short-Term Memory (TPA-LSTM) for prediction to overcome the issue. The results show that stock index prices prediction through the TPA-LSTM algorithm could achieve better prediction performance over traditional deep neural networks, such as recurrent neural network (RNN), convolutional neural network (CNN), and long and short-term time series network (LSTNet).

Sign in / Sign up

Export Citation Format

Share Document