An Advanced Image Captioning using combination of CNN and LSTM

The Captioning of Image now a days is gaining a lot of interest which generates an automated simple and short sentence describing the image content. Machines indeed are trained in a way that they can understand the Image content and generate captions which are almost accurate at a human level of knowledge is a very tedious and interesting task. There are various solutions used to solve this tedious task and generate simple sentences known as captions using neural network which still comes with problems such as inaccurate captions, generating captions only for the seen images, etc. In this paper, the proposed system model was able to generate more precise captions using a two staged model which consists of a combination of Deep Neural Network algorithms (Convolutional and Long Short-Term Memory). The proposed model was able to overcome the problems arise using Traditional CNN and RNN algorithms. The model is trained and tested using the Flicker8k Data set.

Download Full-text

Roll Motion Prediction of Unmanned Surface Vehicle Based on Coupled CNN and LSTM

Future Internet ◽

10.3390/fi11110243 ◽

2019 ◽

Vol 11 (11) ◽

pp. 243 ◽

Cited By ~ 1

Author(s):

Wenjie Zhang ◽

Pin Wu ◽

Yan Peng ◽

Dongke Liu

Keyword(s):

Neural Network ◽

Short Term Memory ◽

Real Data ◽

Sensor Data ◽

Motion Prediction ◽

Roll Motion ◽

Time Varying ◽

Motion Modeling ◽

Data Set ◽

Proposed Model

The prediction of roll motion in unmanned surface vehicles (USVs) is vital for marine safety and the efficiency of USV operations. However, the USV roll motion at sea is a complex time-varying nonlinear and non-stationary dynamic system, which varies with time-varying environmental disturbances as well as various sailing conditions. The conventional methods have the disadvantages of low accuracy, poor robustness, and insufficient practical application ability. The rise of deep learning provides new opportunities for USV motion modeling and prediction. In this paper, a data-driven neural network model is constructed by combining a convolution neural network (CNN) with long short-term memory (LSTM) for USV roll motion prediction. The CNN is used to extract spatially relevant and local time series features of the USV sensor data. The LSTM layer is exploited to reflect the long-term movement process of the USV and predict roll motion for the next moment. The fully connected layer is utilized to decode the LSTM output and calculate the final prediction results. The effectiveness of the proposed model was proved using USV roll motion prediction experiments based on two case studies from “JingHai-VI” and “JingHai-III” USVS of Shanghai University. Experimental results on a real data set indicated that our proposed model obviously outperformed the state-of-the-art methods.

Download Full-text

Research on PM2.5 Spatiotemporal Forecasting Model Based on LSTM Neural Network

Computational Intelligence and Neuroscience ◽

10.1155/2021/1616806 ◽

2021 ◽

Vol 2021 ◽

pp. 1-10

Author(s):

Fang Zhao ◽

Ziyi Liang ◽

Qiyan Zhang ◽

Dewen Seng ◽

Xiyuan Chen

Keyword(s):

Neural Network ◽

Air Quality ◽

Short Term Memory ◽

Environmental Data ◽

Weather Data ◽

Short Term ◽

Concentration Data ◽

Data Set ◽

Term Memory ◽

Proposed Model

Accurate monitoring of air quality can no longer meet people’s needs. People hope to predict air quality in advance and make timely warnings and defenses to minimize the threat to life. This paper proposed a new air quality spatiotemporal prediction model to predict future air quality and is based on a large number of environmental data and a long short-term memory (LSTM) neural network. In order to capture the spatial and temporal characteristics of the pollutant concentration data, the data of the five sites with the highest correlation of time-series concentration of PM2.5 (particles with aerodynamic diameter ≤2.5 mm) at the experimental site were first extracted, and the weather data and other pollutant data at the same time were merged in the next step, extracting advanced spatiotemporal features through long- and short-term memory neural networks. The model presented in this paper was compared with other baseline models on the hourly PM2.5 concentration data set collected at 35 air quality monitoring sites in Beijing from January 1, 2016, to December 31, 2017. The experimental results show that the performance of the proposed model is better than other baseline models.

Download Full-text

Human Activity Recognition using Fourier Transform Inspired Deep Learning Combination Model

International Journal of Sensors Wireless Communications and Control ◽

10.2174/2210327908666180727123657 ◽

2019 ◽

Vol 9 (1) ◽

pp. 16-31

Author(s):

Kyungkoo Jun

Keyword(s):

Fourier Transform ◽

Deep Learning ◽

Short Term Memory ◽

Window Size ◽

Sensor Data ◽

Data Sets ◽

Data Set ◽

Proposed Model ◽

Testing Data ◽

Labeling Scheme

Background & Objective: This paper proposes a Fourier transform inspired method to classify human activities from time series sensor data. Methods: Our method begins by decomposing 1D input signal into 2D patterns, which is motivated by the Fourier conversion. The decomposition is helped by Long Short-Term Memory (LSTM) which captures the temporal dependency from the signal and then produces encoded sequences. The sequences, once arranged into the 2D array, can represent the fingerprints of the signals. The benefit of such transformation is that we can exploit the recent advances of the deep learning models for the image classification such as Convolutional Neural Network (CNN). Results: The proposed model, as a result, is the combination of LSTM and CNN. We evaluate the model over two data sets. For the first data set, which is more standardized than the other, our model outperforms previous works or at least equal. In the case of the second data set, we devise the schemes to generate training and testing data by changing the parameters of the window size, the sliding size, and the labeling scheme. Conclusion: The evaluation results show that the accuracy is over 95% for some cases. We also analyze the effect of the parameters on the performance.

Download Full-text

Sentence similarity evaluation using Sent2Vec and siamese neural network with parallel structure

Journal of Intelligent & Fuzzy Systems ◽

10.3233/jifs-189593 ◽

2021 ◽

pp. 1-10

Author(s):

Hye-Jeong Song ◽

Tak-Sung Heo ◽

Jong-Dae Kim ◽

Chan-Young Park ◽

Yu-Seop Kim

Keyword(s):

Neural Network ◽

Language Processing ◽

Short Term Memory ◽

Parallel Structure ◽

Short Term ◽

Similarity Estimation ◽

Accurate Judgment ◽

Proposed Model ◽

Sentence Similarity ◽

Long Short Term Memory

Sentence similarity evaluation is a significant task used in machine translation, classification, and information extraction in the field of natural language processing. When two sentences are given, an accurate judgment should be made whether the meaning of the sentences is equivalent even if the words and contexts of the sentences are different. To this end, existing studies have measured the similarity of sentences by focusing on the analysis of words, morphemes, and letters. To measure sentence similarity, this study uses Sent2Vec, a sentence embedding, as well as morpheme word embedding. Vectors representing words are input to the 1-dimension convolutional neural network (1D-CNN) with various sizes of kernels and bidirectional long short-term memory (Bi-LSTM). Self-attention is applied to the features transformed through Bi-LSTM. Subsequently, vectors undergoing 1D-CNN and self-attention are converted through global max pooling and global average pooling to extract specific values, respectively. The vectors generated through the above process are concatenated to the vector generated through Sent2Vec and are represented as a single vector. The vector is input to softmax layer, and finally, the similarity between the two sentences is determined. The proposed model can improve the accuracy by up to 5.42% point compared with the conventional sentence similarity estimation models.

Download Full-text

A Proposed Heuristic Optimization Algorithm for Detecting Network Attacks

The Academic Research Community Publication ◽

10.21625/archive.v2i4.397 ◽

2019 ◽

Vol 2 (4) ◽

pp. 530

Author(s):

Amr Hassan Yassin ◽

Hany Hamdy Hussien

Keyword(s):

Neural Network ◽

Heuristic Algorithms ◽

Feed Forward Neural Network ◽

Data Set ◽

Network Attacks ◽

Security Issues ◽

Proposed Model ◽

Network Intrusions ◽

Attack Patterns ◽

Selection Of

Due to the exponential growth of E-Business and computing capabilities over the web for a pay-for-use groundwork, the risk factors regarding security issues also increase rapidly. As the usage increases, it becomes very difficult to identify malicious attacks since the attack patterns change. Therefore, host machines in the network must continually be monitored for intrusions since they are the final endpoint of any network. The purpose of this work is to introduce a generalized neural network model that has the ability to detect network intrusions. Two recent heuristic algorithms inspired by the behavior of natural phenomena, namely, the particle swarm optimization (PSO) and gravitational search (GSA) algorithms are introduced. These algorithms are combined together to train a feed forward neural network (FNN) for the purpose of utilizing the effectiveness of these algorithms to reduce the problems of getting stuck in local minima and the time-consuming convergence rate. Dimension reduction focuses on using information obtained from NSL-KDD Cup 99 data set for the selection of some features to discover the type of attacks. Detecting the network attacks and the performance of the proposed model are evaluated under different patterns of network data.

Download Full-text

The Application of Probabilistic Neural Network in Speech Recognition Based on Partition Clustering

Applied Mechanics and Materials ◽

10.4028/www.scientific.net/amm.263-266.2173 ◽

2012 ◽

Vol 263-266 ◽

pp. 2173-2178

Author(s):

Xin Guang Li ◽

Min Feng Yao ◽

Li Rui Jian ◽

Zhen Jiang Li

Keyword(s):

Neural Network ◽

Speech Recognition ◽

Clustering Algorithm ◽

Probabilistic Neural Network ◽

Back Propagation ◽

Back Propagation Neural Network ◽

Data Sets ◽

Data Set ◽

Proposed Model ◽

Partition Clustering

A probabilistic neural network (PNN) speech recognition model based on the partition clustering algorithm is proposed in this paper. The most important advantage of PNN is that training is easy and instantaneous. Therefore, PNN is capable of dealing with real time speech recognition. Besides, in order to increase the performance of PNN, the selection of data set is one of the most important issues. In this paper, using the partition clustering algorithm to select data is proposed. The proposed model is tested on two data sets from the field of spoken Arabic numbers, with promising results. The performance of the proposed model is compared to single back propagation neural network and integrated back propagation neural network. The final comparison result shows that the proposed model performs better than the other two neural networks, and has an accuracy rate of 92.41%.

Download Full-text

Production Forecasting with the Interwell Interference by Integrating Graph Convolutional and Long Short-Term Memory Neural Network

SPE Reservoir Evaluation & Engineering ◽

10.2118/208596-pa ◽

2021 ◽

pp. 1-17

Author(s):

Enda Du ◽

Yuetian Liu ◽

Ziyan Cheng ◽

Liang Xue ◽

Jing Ma ◽

...

Keyword(s):

Neural Network ◽

Short Term Memory ◽

Short Term ◽

Term Memory ◽

Production Forecasting ◽

Temporal Correlations ◽

Proposed Model ◽

The Mean ◽

Long Short Term Memory ◽

The Impact

Summary Accurate production forecasting is an essential task and accompanies the entire process of reservoir development. With the limitation of prediction principles and processes, the traditional approaches are difficult to make rapid predictions. With the development of artificial intelligence, the data-driven model provides an alternative approach for production forecasting. To fully take the impact of interwell interference on production into account, this paper proposes a deep learning-based hybrid model (GCN-LSTM), where graph convolutional network (GCN) is used to capture complicated spatial patterns between each well, and long short-term memory (LSTM) neural network is adopted to extract intricate temporal correlations from historical production data. To implement the proposed model more efficiently, two data preprocessing procedures are performed: Outliers in the data set are removed by using a box plot visualization, and measurement noise is reduced by a wavelet transform. The robustness and applicability of the proposed model are evaluated in two scenarios of different data types with the root mean square error (RMSE), the mean absolute error (MAE), and the mean absolute percentage error (MAPE). The results show that the proposed model can effectively capture spatial and temporal correlations to make a rapid and accurate oil production forecast.

Download Full-text

Multi-Regional Online Car-Hailing Order Quantity Forecasting Based on the Convolutional Neural Network

Information ◽

10.3390/info10060193 ◽

2019 ◽

Vol 10 (6) ◽

pp. 193 ◽

Cited By ~ 1

Author(s):

Zihao Huang ◽

Gang Huang ◽

Zhijun Chen ◽

Chaozhong Wu ◽

Xiaofeng Ma ◽

...

Keyword(s):

Neural Network ◽

Convolutional Neural Network ◽

Travel Demand ◽

Short Term Memory ◽

Demand Forecasting ◽

Image Feature ◽

Support Vector ◽

Data Set ◽

Demand Distribution ◽

Demand Forecasting Model

With the development of online cars, the demand for travel prediction is increasing in order to reduce the information asymmetry between passengers and drivers of online car-hailing. This paper proposes a travel demand forecasting model named OC-CNN based on the convolutional neural network to forecast the travel demand. In order to make full use of the spatial characteristics of the travel demand distribution, this paper meshes the prediction area and creates a travel demand data set of the graphical structure to preserve its spatial properties. Taking advantage of the convolutional neural network in image feature extraction, the historical demand data of the first twenty-five minutes of the entire region are used as a model input to predict the travel demand for the next five minutes. In order to verify the performance of the proposed method, one-month data from online car-hailing of the Chengdu Fourth Ring Road are used. The results show that the model successfully extracts the spatiotemporal features of the data, and the prediction accuracies of the proposed method are superior to those of the representative methods, including the Bayesian Ridge Model, Linear Regression, Support Vector Regression, and Long Short-Term Memory networks.

Download Full-text

Bimodal Emotion Recognition Model for Minnan Songs

Information ◽

10.3390/info11030145 ◽

2020 ◽

Vol 11 (3) ◽

pp. 145 ◽

Cited By ~ 1

Author(s):

Zhenglong Xiang ◽

Xialei Dong ◽

Yuanxiang Li ◽

Fei Yu ◽

Xing Xu ◽

...

Keyword(s):

Neural Network ◽

Emotion Recognition ◽

Short Term Memory ◽

Music Appreciation ◽

Research Papers ◽

Audio Features ◽

Analysis Theory ◽

Proposed Model ◽

Song Lyrics ◽

Long Short Term Memory

Most of the existing research papers study the emotion recognition of Minnan songs from the perspectives of music analysis theory and music appreciation. However, these investigations do not explore any possibility of carrying out an automatic emotion recognition of Minnan songs. In this paper, we propose a model that consists of four main modules to classify the emotion of Minnan songs by using the bimodal data—song lyrics and audio. In the proposed model, an attention-based Long Short-Term Memory (LSTM) neural network is applied to extract lyrical features, and a Convolutional Neural Network (CNN) is used to extract the audio features from the spectrum. Then, two kinds of extracted features are concatenated by multimodal compact bilinear pooling, and finally, the concatenated features are input to the classifying module to determine the song emotion. We designed three experiment groups to investigate the classifying performance of combinations of the four main parts, the comparisons of proposed model with the current approaches and the influence of a few key parameters on the performance of emotion recognition. The results show that the proposed model exhibits better performance over all other experimental groups. The accuracy, precision and recall of the proposed model exceed 0.80 in a combination of appropriate parameters.

Download Full-text

Trajectory-Based Air-Writing Recognition Using Deep Neural Network and Depth Sensor

Sensors ◽

10.3390/s20020376 ◽

2020 ◽

Vol 20 (2) ◽

pp. 376 ◽

Cited By ~ 6

Author(s):

Md. Shahinur Alam ◽

Ki-Chul Kwon ◽

Md. Ashraful Alam ◽

Mohammed Y. Abbass ◽

Shariar Md Imtiaz ◽

...

Keyword(s):

Neural Network ◽

Nearest Neighbor ◽

Short Term Memory ◽

Three Dimensional ◽

Recognition System ◽

Writing Systems ◽

Depth Sensor ◽

Handheld Device ◽

Proposed Model ◽

Long Short Term Memory

Trajectory-based writing system refers to writing a linguistic character or word in free space by moving a finger, marker, or handheld device. It is widely applicable where traditional pen-up and pen-down writing systems are troublesome. Due to the simple writing style, it has a great advantage over the gesture-based system. However, it is a challenging task because of the non-uniform characters and different writing styles. In this research, we developed an air-writing recognition system using three-dimensional (3D) trajectories collected by a depth camera that tracks the fingertip. For better feature selection, the nearest neighbor and root point translation was used to normalize the trajectory. We employed the long short-term memory (LSTM) and a convolutional neural network (CNN) as a recognizer. The model was tested and verified by the self-collected dataset. To evaluate the robustness of our model, we also employed the 6D motion gesture (6DMG) alphanumeric character dataset and achieved 99.32% accuracy which is the highest to date. Hence, it verifies that the proposed model is invariant for digits and characters. Moreover, we publish a dataset containing 21,000 digits; which solves the lack of dataset in the current research.

Download Full-text