scholarly journals BJBN： BERT-JOIN-BiLSTM Networks for Medical Auxiliary Diagnostic

2022 ◽  
Vol 2022 ◽  
pp. 1-7
Author(s):  
Chuanjie Xu ◽  
Feng Yuan ◽  
Shouqiang Chen
Keyword(s):  
Short Term Memory ◽  
State Of The Art ◽  
Local Features ◽  
Global Information ◽  
Short Term ◽  
Baseline Model ◽  
Proposed Model ◽  
Global Representation ◽  
Long Short Term Memory ◽  
Lstm Network

This study proposed a medicine auxiliary diagnosis model based on neural network. The model combines a bidirectional long short-term memory（Bi-LSTM）network and bidirectional encoder representations from transformers (BERT), which can well complete the extraction of local features of Chinese medicine texts. BERT can learn the global information of the text, so use BERT to get the global representation of medical text and then use Bi-LSTM to extract local features. We conducted a large number of comparative experiments on datasets. The results show that the proposed model has significant advantages over the state-of-the-art baseline model. The accuracy of the proposed model is 0.75.

JAZZ MELODY GENERATION USING RECURRENT NETWORKS AND REINFORCEMENT LEARNING

2006 ◽  
Vol 15 (04) ◽  
pp. 623-650
Author(s):  
JUDY A. FRANKLIN
Keyword(s):  
Reinforcement Learning ◽  
Dynamic Systems ◽  
Recurrent Neural Networks ◽  
Short Term Memory ◽  
State Of The Art ◽  
Recurrent Network ◽  
Recurrent Networks ◽  
Short Term ◽  
Long Short Term Memory ◽  
Lstm Network

Recurrent (neural) networks have been deployed as models for learning musical processes, by computational scientists who study processes such as dynamic systems. Over time, more intricate music has been learned as the state of the art in recurrent networks improves. One particular recurrent network, the Long Short-Term Memory (LSTM) network shows promise for learning long songs, and generating new songs. We are experimenting with a module containing two inter-recurrent LSTM networks to cooperatively learn several human melodies, based on the songs' harmonic structures, and on the feedback inherent in the network. We show that these networks can learn to reproduce four human melodies. We then present as input new harmonizations, so as to generate new songs. We describe the reharmonizations, and show the new melodies that result. We also present a hierarchical structure for using reinforcement learning to choose LSTM modules during the course of melody generation.

scholarly journals SWIM: A Simple Word Interaction Model for Implicit Discourse Relation Recognition

2017 ◽  
Author(s):  
Wenqiang Lei ◽  
Xuancong Wang ◽  
Meichun Liu ◽  
Ilija Ilievski ◽  
Xiangnan He ◽  
...  
Keyword(s):  
Short Term Memory ◽  
State Of The Art ◽  
Interaction Model ◽  
Weighted Averaging ◽  
Short Term ◽  
Current State ◽  
Proposed Model ◽  
Order Of Magnitude ◽  
Long Short Term Memory ◽  
Semantic Interaction

Capturing the semantic interaction of pairs of words across arguments and proper argument representation are both crucial issues in implicit discourse relation recognition. The current state-of-the-art represents arguments as distributional vectors that are computed via bi-directional Long Short-Term Memory networks (BiLSTMs), known to have significant model complexity.In contrast, we demonstrate that word-weighted averaging can encode argument representation which can incorporate word pair information efficiently. By saving an order of magnitude in parameters, our proposed model achieves equivalent performance, but trains seven times faster.

scholarly journals Malicious Traffic classification using Long Short-Term Memory (LSTM) model

2021 ◽  
Author(s):  
Naresh Kumar Thapa K ◽  
N. Duraipandian
Keyword(s):  
Short Term Memory ◽  
State Of The Art ◽  
Detection System ◽  
Classification Systems ◽  
Traffic Classification ◽  
Short Term ◽  
Fixed Sequence ◽  
Term Memory ◽  
Proposed Model ◽  
Long Short Term Memory

Abstract Malicious traffic classification is the initial and primary step for any network-based security systems. This traffic classification systems include behavior-based anomaly detection system and Intrusion Detection System. Existing methods always relies on the conventional techniques and process the data in the fixed sequence, which may leads to performance issues. Furthermore, conventional techniques require proper annotation to process the volumetric data. Relying on the data annotation for efficient traffic classification may leads to network loops and bandwidth issues within the network. To address the above-mentioned issues, this paper presents a novel solution based on artificial intelligence perspective. The key idea of this paper is to propose a novel malicious classification system using Long Short-Term Memory (LSTM) model. To validate the efficiency of the proposed model, an experimental setup along with experimental validation is carried out. From the experimental results, it is proven that the proposed model is better in terms of accuracy, throughput when compared to the state-of-the-art models. Further, the accuracy of the proposed model outperforms the existing state of the art models with increase in 5% and overall 99.5% in accuracy.

scholarly journals Learning Object Context for Dense Captioning

2019 ◽  
Vol 33 ◽  
pp. 8650-8657 ◽  
Author(s):  
Xiangyang Li ◽  
Shuqiang Jiang ◽  
Jungong Han
Keyword(s):  
Short Term Memory ◽  
State Of The Art ◽  
Short Term ◽  
Visual Elements ◽  
Context Learning ◽  
Learning Procedure ◽  
Benchmark Datasets ◽  
Long Short Term Memory ◽  
Lstm Network ◽  
Context Features

Dense captioning is a challenging task which not only detects visual elements in images but also generates natural language sentences to describe them. Previous approaches do not leverage object information in images for this task. However, objects provide valuable cues to help predict the locations of caption regions as caption regions often highly overlap with objects (i.e. caption regions are usually parts of objects or combinations of them). Meanwhile, objects also provide important information for describing a target caption region as the corresponding description not only depicts its properties, but also involves its interactions with objects in the image. In this work, we propose a novel scheme with an object context encoding Long Short-Term Memory (LSTM) network to automatically learn complementary object context for each caption region, transferring knowledge from objects to caption regions. All contextual objects are arranged as a sequence and progressively fed into the context encoding module to obtain context features. Then both the learned object context features and region features are used to predict the bounding box offsets and generate the descriptions. The context learning procedure is in conjunction with the optimization of both location prediction and caption generation, thus enabling the object context encoding LSTM to capture and aggregate useful object context. Experiments on benchmark datasets demonstrate the superiority of our proposed approach over the state-of-the-art methods.

scholarly journals Dynamic Displacement Forecasting of Dashuitian Landslide in China Using Variational Mode Decomposition and Stack Long Short-Term Memory Network

2019 ◽  
Vol 9 (15) ◽  
pp. 2951 ◽  
Author(s):  
Yin Xing ◽  
Jianping Yue ◽  
Chuang Chen ◽  
Kanglin Cong ◽  
Shaolin Zhu ◽  
...  
Keyword(s):  
Short Term Memory ◽  
Forecast Accuracy ◽  
Short Term ◽  
Dynamic Displacement ◽  
Term Memory ◽  
Mode Decomposition ◽  
Proposed Model ◽  
Memory Network ◽  
Long Short Term Memory ◽  
Lstm Network

In recent decades, landslide displacement forecasting has received increasing attention due to its ability to reduce landslide hazards. To improve the forecast accuracy of landslide displacement, a dynamic forecasting model based on variational mode decomposition (VMD) and a stack long short-term memory network (SLSTM) is proposed. VMD is used to decompose landslide displacement into different displacement subsequences, and the SLSTM network is used to forecast each displacement subsequence. Then, the forecast values of landslide displacement are obtained by reconstructing the forecast values of all displacement subsequences. On the other hand, the SLSTM networks are updated by adding the forecast values into the training set, realizing the dynamic displacement forecasting. The proposed model was verified on the Dashuitian landslide in China. The results show that compared with the two advanced forecasting models, long short-term memory (LSTM) network, and empirical mode decomposition (EMD)–LSTM network, the proposed model has higher forecast accuracy.

scholarly journals Leveraging Title-Abstract Attentive Semantics for Paper Recommendation

2020 ◽  
Vol 34 (01) ◽  
pp. 67-74
Author(s):  
Guibing Guo ◽  
Bowei Chen ◽  
Xiaoyan Zhang ◽  
Zhirong Liu ◽  
Zhenhua Dong ◽  
...  
Keyword(s):  
Short Term Memory ◽  
State Of The Art ◽  
Semantic Relationship ◽  
Short Term ◽  
Sentence Patterns ◽  
Memory Network ◽  
Long Short Term Memory ◽  
Lstm Network ◽  
Gated Recurrent Unit

Paper recommendation is a research topic to provide users with personalized papers of interest. However, most existing approaches equally treat title and abstract as the input to learn the representation of a paper, ignoring their semantic relationship. In this paper, we regard the abstract as a sequence of sentences, and propose a two-level attentive neural network to capture: (1) the ability of each word within a sentence to reflect if it is semantically close to the words within the title. (2) the extent of each sentence in the abstract relative to the title, which is often a good summarization of the abstract document. Specifically, we propose a Long-Short Term Memory (LSTM) network with attention to learn the representation of sentences, and integrate a Gated Recurrent Unit (GRU) network with a memory network to learn the long-term sequential sentence patterns of interacted papers for both user and item (paper) modeling. We conduct extensive experiments on two real datasets, and show that our approach outperforms other state-of-the-art approaches in terms of accuracy.

scholarly journals A Combined Method for MEMS Gyroscope Error Compensation Using a Long Short-Term Memory Network and Kalman Filter in Random Vibration Environments

Sensors ◽  
2021 ◽  
Vol 21 (4) ◽  
pp. 1181
Author(s):  
Chenhao Zhu ◽  
Sheng Cai ◽  
Yifan Yang ◽  
Wei Xu ◽  
Honghai Shen ◽  
...  
Keyword(s):  
Kalman Filter ◽  
Standard Deviation ◽  
Error Compensation ◽  
Random Vibration ◽  
Short Term Memory ◽  
Combined Method ◽  
Short Term ◽  
Mems Gyroscope ◽  
Long Short Term Memory ◽  
Lstm Network

In applications such as carrier attitude control and mobile device navigation, a micro-electro-mechanical-system (MEMS) gyroscope will inevitably be affected by random vibration, which significantly affects the performance of the MEMS gyroscope. In order to solve the degradation of MEMS gyroscope performance in random vibration environments, in this paper, a combined method of a long short-term memory (LSTM) network and Kalman filter (KF) is proposed for error compensation, where Kalman filter parameters are iteratively optimized using the Kalman smoother and expectation-maximization (EM) algorithm. In order to verify the effectiveness of the proposed method, we performed a linear random vibration test to acquire MEMS gyroscope data. Subsequently, an analysis of the effects of input data step size and network topology on gyroscope error compensation performance is presented. Furthermore, the autoregressive moving average-Kalman filter (ARMA-KF) model, which is commonly used in gyroscope error compensation, was also combined with the LSTM network as a comparison method. The results show that, for the x-axis data, the proposed combined method reduces the standard deviation (STD) by 51.58% and 31.92% compared to the bidirectional LSTM (BiLSTM) network, and EM-KF method, respectively. For the z-axis data, the proposed combined method reduces the standard deviation by 29.19% and 12.75% compared to the BiLSTM network and EM-KF method, respectively. Furthermore, for x-axis data and z-axis data, the proposed combined method reduces the standard deviation by 46.54% and 22.30% compared to the BiLSTM-ARMA-KF method, respectively, and the output is smoother, proving the effectiveness of the proposed method.

scholarly journals Air pollution forecasting application based on deep learning model and optimization algorithm

2021 ◽  
Author(s):  
Azim Heydari ◽  
Meysam Majidi Nezhad ◽  
Davide Astiaso Garcia ◽  
Farshid Keynia ◽  
Livio De Santoli
Keyword(s):  
Air Pollution ◽  
Wind Speed ◽  
Power Plant ◽  
Air Temperature ◽  
Short Term Memory ◽  
Combined Cycle ◽  
Short Term ◽  
Term Memory ◽  
Proposed Model ◽  
Long Short Term Memory

AbstractAir pollution monitoring is constantly increasing, giving more and more attention to its consequences on human health. Since Nitrogen dioxide (NO2) and sulfur dioxide (SO2) are the major pollutants, various models have been developed on predicting their potential damages. Nevertheless, providing precise predictions is almost impossible. In this study, a new hybrid intelligent model based on long short-term memory (LSTM) and multi-verse optimization algorithm (MVO) has been developed to predict and analysis the air pollution obtained from Combined Cycle Power Plants. In the proposed model, long short-term memory model is a forecaster engine to predict the amount of produced NO2 and SO2 by the Combined Cycle Power Plant, where the MVO algorithm is used to optimize the LSTM parameters in order to achieve a lower forecasting error. In addition, in order to evaluate the proposed model performance, the model has been applied using real data from a Combined Cycle Power Plant in Kerman, Iran. The datasets include wind speed, air temperature, NO2, and SO2 for five months (May–September 2019) with a time step of 3-h. In addition, the model has been tested based on two different types of input parameters: type (1) includes wind speed, air temperature, and different lagged values of the output variables (NO2 and SO2); type (2) includes just lagged values of the output variables (NO2 and SO2). The obtained results show that the proposed model has higher accuracy than other combined forecasting benchmark models (ENN-PSO, ENN-MVO, and LSTM-PSO) considering different network input variables. Graphic abstract

scholarly journals Extraction of local and global features by a convolutional neural network–long short-term memory network for diagnosing bearing faults

2021 ◽  
pp. 095440622110165
Author(s):  
Zhang Chao ◽  
Wang Wei-zhi ◽  
Zhang Chen ◽  
Fan Bin ◽  
Wang Jian-guo ◽  
...  
Keyword(s):  
Neural Network ◽  
Fault Diagnosis ◽  
Condition Monitoring ◽  
Short Term Memory ◽  
Vibration Signal ◽  
Short Term ◽  
Global Features ◽  
Term Memory ◽  
Long Short Term Memory ◽  
Lstm Network

Accurate and reliable fault diagnosis is one of the key and difficult issues in mechanical condition monitoring. In recent years, Convolutional Neural Network (CNN) has been widely used in mechanical condition monitoring, which is also a great breakthrough in the field of bearing fault diagnosis. However, CNN can only extract local features of signals. The model accuracy and generalization of the original vibration signals are very low in the process of vibration signal processing only by CNN. Based on the above problems, this paper improves the traditional convolution layer of CNN, and builds the learning module (local feature learning block, LFLB) of the local characteristics. At the same time, the Long Short-Term Memory (LSTM) is introduced into the network, which is used to extract the global features. This paper proposes the new neural network—improved CNN-LSTM network. The extracted deep feature is used for fault classification. The improved CNN-LSTM network is applied to the processing of the vibration signal of the faulty bearing collected by the bearing failure laboratory of Inner Mongolia University of science and technology. The results show that the accuracy of the improved CNN-LSTM network on the same batch test set is 98.75%, which is about 24% higher than that of the traditional CNN. The proposed network is applied to the bearing data collection of Western Reserve University under the condition that the network parameters remain unchanged. The experiment shows that the improved CNN-LSTM network has better generalization than the traditional CNN.

Sentence similarity evaluation using Sent2Vec and siamese neural network with parallel structure

2021 ◽  
pp. 1-10
Author(s):  
Hye-Jeong Song ◽  
Tak-Sung Heo ◽  
Jong-Dae Kim ◽  
Chan-Young Park ◽  
Yu-Seop Kim
Keyword(s):  
Neural Network ◽  
Language Processing ◽  
Short Term Memory ◽  
Parallel Structure ◽  
Short Term ◽  
Similarity Estimation ◽  
Accurate Judgment ◽  
Proposed Model ◽  
Sentence Similarity ◽  
Long Short Term Memory

Sentence similarity evaluation is a significant task used in machine translation, classification, and information extraction in the field of natural language processing. When two sentences are given, an accurate judgment should be made whether the meaning of the sentences is equivalent even if the words and contexts of the sentences are different. To this end, existing studies have measured the similarity of sentences by focusing on the analysis of words, morphemes, and letters. To measure sentence similarity, this study uses Sent2Vec, a sentence embedding, as well as morpheme word embedding. Vectors representing words are input to the 1-dimension convolutional neural network (1D-CNN) with various sizes of kernels and bidirectional long short-term memory (Bi-LSTM). Self-attention is applied to the features transformed through Bi-LSTM. Subsequently, vectors undergoing 1D-CNN and self-attention are converted through global max pooling and global average pooling to extract specific values, respectively. The vectors generated through the above process are concatenated to the vector generated through Sent2Vec and are represented as a single vector. The vector is input to softmax layer, and finally, the similarity between the two sentences is determined. The proposed model can improve the accuracy by up to 5.42% point compared with the conventional sentence similarity estimation models.

Sign in / Sign up

Export Citation Format

Share Document