scholarly journals Learning Object Context for Dense Captioning

2019 ◽  
Vol 33 ◽  
pp. 8650-8657 ◽  
Author(s):  
Xiangyang Li ◽  
Shuqiang Jiang ◽  
Jungong Han
Keyword(s):  
Short Term Memory ◽  
State Of The Art ◽  
Short Term ◽  
Visual Elements ◽  
Context Learning ◽  
Learning Procedure ◽  
Benchmark Datasets ◽  
Long Short Term Memory ◽  
Lstm Network ◽  
Context Features

Dense captioning is a challenging task which not only detects visual elements in images but also generates natural language sentences to describe them. Previous approaches do not leverage object information in images for this task. However, objects provide valuable cues to help predict the locations of caption regions as caption regions often highly overlap with objects (i.e. caption regions are usually parts of objects or combinations of them). Meanwhile, objects also provide important information for describing a target caption region as the corresponding description not only depicts its properties, but also involves its interactions with objects in the image. In this work, we propose a novel scheme with an object context encoding Long Short-Term Memory (LSTM) network to automatically learn complementary object context for each caption region, transferring knowledge from objects to caption regions. All contextual objects are arranged as a sequence and progressively fed into the context encoding module to obtain context features. Then both the learned object context features and region features are used to predict the bounding box offsets and generate the descriptions. The context learning procedure is in conjunction with the optimization of both location prediction and caption generation, thus enabling the object context encoding LSTM to capture and aggregate useful object context. Experiments on benchmark datasets demonstrate the superiority of our proposed approach over the state-of-the-art methods.

JAZZ MELODY GENERATION USING RECURRENT NETWORKS AND REINFORCEMENT LEARNING

2006 ◽  
Vol 15 (04) ◽  
pp. 623-650
Author(s):  
JUDY A. FRANKLIN
Keyword(s):  
Reinforcement Learning ◽  
Dynamic Systems ◽  
Recurrent Neural Networks ◽  
Short Term Memory ◽  
State Of The Art ◽  
Recurrent Network ◽  
Recurrent Networks ◽  
Short Term ◽  
Long Short Term Memory ◽  
Lstm Network

Recurrent (neural) networks have been deployed as models for learning musical processes, by computational scientists who study processes such as dynamic systems. Over time, more intricate music has been learned as the state of the art in recurrent networks improves. One particular recurrent network, the Long Short-Term Memory (LSTM) network shows promise for learning long songs, and generating new songs. We are experimenting with a module containing two inter-recurrent LSTM networks to cooperatively learn several human melodies, based on the songs' harmonic structures, and on the feedback inherent in the network. We show that these networks can learn to reproduce four human melodies. We then present as input new harmonizations, so as to generate new songs. We describe the reharmonizations, and show the new melodies that result. We also present a hierarchical structure for using reinforcement learning to choose LSTM modules during the course of melody generation.

scholarly journals AraSenCorpus: A Semi-Supervised Approach for Sentiment Annotation of a Large Arabic Text Corpus

2021 ◽  
Vol 11 (5) ◽  
pp. 2434
Author(s):  
Ali Al-Laith ◽  
Muhammad Shahbaz ◽  
Hind F. Alaskar ◽  
Asim Rehmat
Keyword(s):  
Short Term Memory ◽  
State Of The Art ◽  
Arabic Text ◽  
Short Term ◽  
Learning Classifier ◽  
Learning Technique ◽  
Benchmark Datasets ◽  
Long Short Term Memory ◽  
Self Learning ◽  
Modern Standard

At a time when research in the field of sentiment analysis tends to study advanced topics in languages, such as English, other languages such as Arabic still suffer from basic problems and challenges, most notably the availability of large corpora. Furthermore, manual annotation is time-consuming and difficult when the corpus is too large. This paper presents a semi-supervised self-learning technique, to extend an Arabic sentiment annotated corpus with unlabeled data, named AraSenCorpus. We use a neural network to train a set of models on a manually labeled dataset containing 15,000 tweets. We used these models to extend the corpus to a large Arabic sentiment corpus called “AraSenCorpus”. AraSenCorpus contains 4.5 million tweets and covers both modern standard Arabic and some of the Arabic dialects. The long-short term memory (LSTM) deep learning classifier is used to train and test the final corpus. We evaluate our proposed framework on two external benchmark datasets to ensure the improvement of the Arabic sentiment classification. The experimental results show that our corpus outperforms the existing state-of-the-art systems.

scholarly journals BJBN： BERT-JOIN-BiLSTM Networks for Medical Auxiliary Diagnostic

2022 ◽  
Vol 2022 ◽  
pp. 1-7
Author(s):  
Chuanjie Xu ◽  
Feng Yuan ◽  
Shouqiang Chen
Keyword(s):  
Short Term Memory ◽  
State Of The Art ◽  
Local Features ◽  
Global Information ◽  
Short Term ◽  
Baseline Model ◽  
Proposed Model ◽  
Global Representation ◽  
Long Short Term Memory ◽  
Lstm Network

This study proposed a medicine auxiliary diagnosis model based on neural network. The model combines a bidirectional long short-term memory（Bi-LSTM）network and bidirectional encoder representations from transformers (BERT), which can well complete the extraction of local features of Chinese medicine texts. BERT can learn the global information of the text, so use BERT to get the global representation of medical text and then use Bi-LSTM to extract local features. We conducted a large number of comparative experiments on datasets. The results show that the proposed model has significant advantages over the state-of-the-art baseline model. The accuracy of the proposed model is 0.75.

scholarly journals Leveraging Title-Abstract Attentive Semantics for Paper Recommendation

2020 ◽  
Vol 34 (01) ◽  
pp. 67-74
Author(s):  
Guibing Guo ◽  
Bowei Chen ◽  
Xiaoyan Zhang ◽  
Zhirong Liu ◽  
Zhenhua Dong ◽  
...  
Keyword(s):  
Short Term Memory ◽  
State Of The Art ◽  
Semantic Relationship ◽  
Short Term ◽  
Sentence Patterns ◽  
Memory Network ◽  
Long Short Term Memory ◽  
Lstm Network ◽  
Gated Recurrent Unit

Paper recommendation is a research topic to provide users with personalized papers of interest. However, most existing approaches equally treat title and abstract as the input to learn the representation of a paper, ignoring their semantic relationship. In this paper, we regard the abstract as a sequence of sentences, and propose a two-level attentive neural network to capture: (1) the ability of each word within a sentence to reflect if it is semantically close to the words within the title. (2) the extent of each sentence in the abstract relative to the title, which is often a good summarization of the abstract document. Specifically, we propose a Long-Short Term Memory (LSTM) network with attention to learn the representation of sentences, and integrate a Gated Recurrent Unit (GRU) network with a memory network to learn the long-term sequential sentence patterns of interacted papers for both user and item (paper) modeling. We conduct extensive experiments on two real datasets, and show that our approach outperforms other state-of-the-art approaches in terms of accuracy.

scholarly journals A Combined Method for MEMS Gyroscope Error Compensation Using a Long Short-Term Memory Network and Kalman Filter in Random Vibration Environments

Sensors ◽  
2021 ◽  
Vol 21 (4) ◽  
pp. 1181
Author(s):  
Chenhao Zhu ◽  
Sheng Cai ◽  
Yifan Yang ◽  
Wei Xu ◽  
Honghai Shen ◽  
...  
Keyword(s):  
Kalman Filter ◽  
Standard Deviation ◽  
Error Compensation ◽  
Random Vibration ◽  
Short Term Memory ◽  
Combined Method ◽  
Short Term ◽  
Mems Gyroscope ◽  
Long Short Term Memory ◽  
Lstm Network

In applications such as carrier attitude control and mobile device navigation, a micro-electro-mechanical-system (MEMS) gyroscope will inevitably be affected by random vibration, which significantly affects the performance of the MEMS gyroscope. In order to solve the degradation of MEMS gyroscope performance in random vibration environments, in this paper, a combined method of a long short-term memory (LSTM) network and Kalman filter (KF) is proposed for error compensation, where Kalman filter parameters are iteratively optimized using the Kalman smoother and expectation-maximization (EM) algorithm. In order to verify the effectiveness of the proposed method, we performed a linear random vibration test to acquire MEMS gyroscope data. Subsequently, an analysis of the effects of input data step size and network topology on gyroscope error compensation performance is presented. Furthermore, the autoregressive moving average-Kalman filter (ARMA-KF) model, which is commonly used in gyroscope error compensation, was also combined with the LSTM network as a comparison method. The results show that, for the x-axis data, the proposed combined method reduces the standard deviation (STD) by 51.58% and 31.92% compared to the bidirectional LSTM (BiLSTM) network, and EM-KF method, respectively. For the z-axis data, the proposed combined method reduces the standard deviation by 29.19% and 12.75% compared to the BiLSTM network and EM-KF method, respectively. Furthermore, for x-axis data and z-axis data, the proposed combined method reduces the standard deviation by 46.54% and 22.30% compared to the BiLSTM-ARMA-KF method, respectively, and the output is smoother, proving the effectiveness of the proposed method.

scholarly journals Extraction of local and global features by a convolutional neural network–long short-term memory network for diagnosing bearing faults

2021 ◽  
pp. 095440622110165
Author(s):  
Zhang Chao ◽  
Wang Wei-zhi ◽  
Zhang Chen ◽  
Fan Bin ◽  
Wang Jian-guo ◽  
...  
Keyword(s):  
Neural Network ◽  
Fault Diagnosis ◽  
Condition Monitoring ◽  
Short Term Memory ◽  
Vibration Signal ◽  
Short Term ◽  
Global Features ◽  
Term Memory ◽  
Long Short Term Memory ◽  
Lstm Network

Accurate and reliable fault diagnosis is one of the key and difficult issues in mechanical condition monitoring. In recent years, Convolutional Neural Network (CNN) has been widely used in mechanical condition monitoring, which is also a great breakthrough in the field of bearing fault diagnosis. However, CNN can only extract local features of signals. The model accuracy and generalization of the original vibration signals are very low in the process of vibration signal processing only by CNN. Based on the above problems, this paper improves the traditional convolution layer of CNN, and builds the learning module (local feature learning block, LFLB) of the local characteristics. At the same time, the Long Short-Term Memory (LSTM) is introduced into the network, which is used to extract the global features. This paper proposes the new neural network—improved CNN-LSTM network. The extracted deep feature is used for fault classification. The improved CNN-LSTM network is applied to the processing of the vibration signal of the faulty bearing collected by the bearing failure laboratory of Inner Mongolia University of science and technology. The results show that the accuracy of the improved CNN-LSTM network on the same batch test set is 98.75%, which is about 24% higher than that of the traditional CNN. The proposed network is applied to the bearing data collection of Western Reserve University under the condition that the network parameters remain unchanged. The experiment shows that the improved CNN-LSTM network has better generalization than the traditional CNN.

A Two-Layer Long Short-Term Memory Network for Bottleneck Prediction in Multi-Job Manufacturing Systems

2018 ◽  
Author(s):  
Xingjian Lai ◽  
Huanyi Shui ◽  
Jun Ni
Keyword(s):  
Manufacturing Systems ◽  
Short Term Memory ◽  
Complex Dynamics ◽  
State Of The Art ◽  
Short Term ◽  
Term Memory ◽  
Future Production ◽  
Effective Manner ◽  
Long Short Term Memory ◽  
Factory Floor

Throughput bottlenecks define and constrain the productivity of a production line. Prediction of future bottlenecks provides a great support for decision-making on the factory floor, which can help to foresee and formulate appropriate actions before production to improve the system throughput in a cost-effective manner. Bottleneck prediction remains a challenging task in literature. The difficulty lies in the complex dynamics of manufacturing systems. There are multiple factors collaboratively affecting bottleneck conditions, such as machine performance, machine degradation, line structure, operator skill level, and product release schedules. These factors impact on one another in a nonlinear manner and exhibit long-term temporal dependencies. State-of-the-art research utilizes various assumptions to simplify the modeling by reducing the input dimensionality. As a result, those models cannot accurately reflect complex dynamics of the bottleneck in a manufacturing system. To tackle this problem, this paper will propose a systematic framework to design a two-layer Long Short-Term Memory (LSTM) network tailored to the dynamic bottleneck prediction problem in multi-job manufacturing systems. This neural network based approach takes advantage of historical high dimensional factory floor data to predict system bottlenecks dynamically considering the future production planning inputs. The model is demonstrated with data from an automotive underbody assembly line. The result shows that the proposed method can achieve higher prediction accuracy compared with current state-of-the-art approaches.

scholarly journals Arabic dialect sentiment analysis with ZERO effort. \\ Case study: Algerian dialect

2020 ◽  
Vol 23 (65) ◽  
pp. 124-135
Author(s):  
Imane Guellil ◽  
Marcelo Mendoza ◽  
Faical Azouaou
Keyword(s):  
Sentiment Analysis ◽  
Short Term Memory ◽  
State Of The Art ◽  
Short Term ◽  
Term Memory ◽  
Ongoing Work ◽  
Long Short Term Memory ◽  
Large Corpus ◽  
Unique Condition

This paper presents an analytic study showing that it is entirely possible to analyze the sentiment of an Arabic dialect without constructing any resources. The idea of this work is to use the resources dedicated to a given dialect \textit{X} for analyzing the sentiment of another dialect \textit{Y}. The unique condition is to have \textit{X} and \textit{Y} in the same category of dialects. We apply this idea on Algerian dialect, which is a Maghrebi Arabic dialect that suffers from limited available tools and other handling resources required for automatic sentiment analysis. To do this analysis, we rely on Maghrebi dialect resources and two manually annotated sentiment corpus for respectively Tunisian and Moroccan dialect. We also use a large corpus for Maghrebi dialect. We use a state-of-the-art system and propose a new deep learning architecture for automatically classify the sentiment of Arabic dialect (Algerian dialect). Experimental results show that F1-score is up to 83% and it is achieved by Multilayer Perceptron (MLP) with Tunisian corpus and with Long short-term memory (LSTM) with the combination of Tunisian and Moroccan. An improvement of 15% compared to its closest competitor was observed through this study. Ongoing work is aimed at manually constructing an annotated sentiment corpus for Algerian dialect and comparing the results

scholarly journals Intelligent Islanding Detection of Microgrids Using Long Short-Term Memory Networks

Energies ◽  
2021 ◽  
Vol 14 (18) ◽  
pp. 5762
Author(s):  
Syed Basit Ali Bukhari ◽  
Khawaja Khalid Mehmood ◽  
Abdul Wadood ◽  
Herie Park
Keyword(s):  
Short Term Memory ◽  
Computational Time ◽  
Islanding Detection ◽  
Phase Voltage ◽  
Short Term ◽  
Term Memory ◽  
Three Phase ◽  
Empirical Wavelet Transform ◽  
Long Short Term Memory ◽  
Lstm Network

This paper presents a new intelligent islanding detection scheme (IIDS) based on empirical wavelet transform (EWT) and long short-term memory (LSTM) network to identify islanding events in microgrids. The concept of EWT is extended to extract features from three-phase signals. First, the three-phase voltage signals sampled at the terminal of targeted distributed energy resource (DER) or point of common coupling (PCC) are decomposed into empirical modes/frequency subbands using EWT. Then, instantaneous amplitudes and instantaneous frequencies of the three-phases at different frequency subbands are combined, and various statistical features are calculated. Finally, the EWT-based features along with the three-phase voltage signals are input to the LSTM network to differentiate between non-islanding and islanding events. To assess the efficacy of the proposed IIDS, extensive simulations are performed on an IEC microgrid and an IEEE 34-node system. The simulation results verify the effectiveness of the proposed IIDS in terms of non-detection zone (NDZ), computational time, detection accuracy, and robustness against noisy measurement. Furthermore, comparisons with existing intelligent methods and different LSTM architectures demonstrate that the proposed IIDS offers higher reliability by significantly reducing the NDZ and stands robust against measurements uncertainty.

scholarly journals Natural language description of images using hybrid recurrent neural network

2019 ◽  
Vol 9 (4) ◽  
pp. 2932
Author(s):  
Md. Asifuzzaman Jishan ◽  
Khan Raqib Mahmud ◽  
Abul Kalam Al Azad
Keyword(s):  
Neural Network ◽  
Natural Language ◽  
Recurrent Neural Network ◽  
Short Term Memory ◽  
Text Line ◽  
Short Term ◽  
Word Representation ◽  
Benchmark Datasets ◽  
Long Short Term Memory ◽  
Language Description

We presented a learning model that generated natural language description of images. The model utilized the connections between natural language and visual data by produced text line based contents from a given image. Our Hybrid Recurrent Neural Network model is based on the intricacies of Convolutional Neural Network (CNN), Long Short-Term Memory (LSTM), and Bi-directional Recurrent Neural Network (BRNN) models. We conducted experiments on three benchmark datasets, e.g., Flickr8K, Flickr30K, and MS COCO. Our hybrid model utilized LSTM model to encode text line or sentences independent of the object location and BRNN for word representation, this reduced the computational complexities without compromising the accuracy of the descriptor. The model produced better accuracy in retrieving natural language based description on the dataset.

Sign in / Sign up

Export Citation Format

Share Document