scholarly journals Hybrid Distance-based, CNN and Bi-LSTM System for Dictionary Expansion

2020 ◽  
pp. 6-13
Author(s):  
Béla Benedek Szakács ◽  
Tamás Mészáros
Keyword(s):  
Natural Language ◽  
Language Processing ◽  
Short Term Memory ◽  
Morphological Data ◽  
Short Term ◽  
Convolutional Networks ◽  
Word Forms ◽  
Long Short Term Memory ◽  
Primary Focus ◽  
Hybrid Distance

Dictionaries like Wordnet can help in a variety of Natural Language Processing applications by providing additional morphological data. They can be used in Digital Humanities research, building knowledge graphs and other applications. Creating dictionaries from large corpora of texts written in a natural language is a task that has not been a primary focus of research, as other tasks have dominated the field (such as chat-bots), but it can be a very useful tool in analysing texts. Even in the case of contemporary texts, categorizing the words according to their dictionary entry is a complex task, and for less conventional texts (in old or less researched languages) it is even harder to solve this problem automatically. Our task was to create a software that helps in expanding a dictionary containing word forms and tagging unprocessed text. We used a manually created corpus for training and testing the model. We created a combination of Bidirectional Long-Short Term Memory networks, convolutional networks and a distancebased solution that outperformed other existing solutions. While manual post-processing for the tagged text is still needed, it significantly reduces the amount of it.

scholarly journals COVID-19 ChatBot

2021 ◽  
Vol 9 (11) ◽  
pp. 44-49
Author(s):  
Satish Tirumalapudi
Keyword(s):  
Deep Learning ◽  
Natural Language Processing ◽  
Natural Language ◽  
Language Processing ◽  
Short Term Memory ◽  
Short Term ◽  
Term Memory ◽  
Long Short Term Memory ◽  
And Control ◽  
Prediction Problems

Abstract: Chat bots are software applications that help users to communicate with the machine and get the required result, this is where Natural Language Processing (NLP) comes into the picture. Natural language processing is based on deep learning that enables computers to acquire meaning from inputs given by the users. Natural language processing techniques can make possible the use of natural language to express ideas, thus drastically increasing accessibility. NLP engines rely on the elements of intent, utterance, entity, context, and session. Here in this project, we will be using Deep learning techniques which will be trained on the dataset which contains categories, patterns, and responses. Long Short-Term Memory (LSTM) is a Recurrent Neural Network that is capable of learning order dependence in sequence prediction problems. One of the most popular RNN approaches is LSTM to identify and control a dynamic system. We use an RNN to classify the category user’s message belongs to and then will give a response from the list of responses. Keywords: NLP – Natural Language Processing, LSTM – Long Short Term Memory, RNN – Recurrent Neural Networks.

scholarly journals Penerapan Convolutional Long Short-Term Memory untuk Klasifikasi Teks Berita Bahasa Indonesia

2021 ◽  
Vol 10 (4) ◽  
pp. 354-361
Author(s):  
Yudi Widhiyasana ◽  
Transmissia Semiawan ◽  
Ilham Gibran Achmad Mudzakir ◽  
Muhammad Randi Noor
Keyword(s):  
Deep Learning ◽  
Natural Language Processing ◽  
Natural Language ◽  
Language Processing ◽  
Short Term Memory ◽  
Learning Rate ◽  
Short Term ◽  
Term Memory ◽  
Long Short Term Memory ◽  
Bahasa Indonesia

Klasifikasi teks saat ini telah menjadi sebuah bidang yang banyak diteliti, khususnya terkait Natural Language Processing (NLP). Terdapat banyak metode yang dapat dimanfaatkan untuk melakukan klasifikasi teks, salah satunya adalah metode deep learning. RNN, CNN, dan LSTM merupakan beberapa metode deep learning yang umum digunakan untuk mengklasifikasikan teks. Makalah ini bertujuan menganalisis penerapan kombinasi dua buah metode deep learning, yaitu CNN dan LSTM (C-LSTM). Kombinasi kedua metode tersebut dimanfaatkan untuk melakukan klasifikasi teks berita bahasa Indonesia. Data yang digunakan adalah teks berita bahasa Indonesia yang dikumpulkan dari portal-portal berita berbahasa Indonesia. Data yang dikumpulkan dikelompokkan menjadi tiga kategori berita berdasarkan lingkupnya, yaitu “Nasional”, “Internasional”, dan “Regional”. Dalam makalah ini dilakukan eksperimen pada tiga buah variabel penelitian, yaitu jumlah dokumen, ukuran batch, dan nilai learning rate dari C-LSTM yang dibangun. Hasil eksperimen menunjukkan bahwa nilai F1-score yang diperoleh dari hasil klasifikasi menggunakan metode C-LSTM adalah sebesar 93,27%. Nilai F1-score yang dihasilkan oleh metode C-LSTM lebih besar dibandingkan dengan CNN, dengan nilai 89,85%, dan LSTM, dengan nilai 90,87%. Dengan demikian, dapat disimpulkan bahwa kombinasi dua metode deep learning, yaitu CNN dan LSTM (C-LSTM),memiliki kinerja yang lebih baik dibandingkan dengan CNN dan LSTM.

DLRCNeg: Deep Learning based Reading Comprehension by handling Negation

2019 ◽  
Author(s):  
Felicia Lilian J. ◽  
Sundarakantham K ◽  
Mercy Shalinie S.
Keyword(s):  
Reading Comprehension ◽  
Natural Language ◽  
Language Processing ◽  
Short Term Memory ◽  
Activation Function ◽  
Attention Mechanism ◽  
Question Type ◽  
Short Term ◽  
Novel Approach ◽  
Long Short Term Memory

Question Answer (QA) System for Reading Comprehension (RC) is a computerized approach to retrieve relevant response to the query posted by the users. The underlined concept in developing such a system is to build a human computer interaction. The interactions will be in natural language and we tend to use negation words as a part of our expressions. During the pre-processing stage in Natural Language Processing (NLP) task these negation words gets removed and hence the semantics gets changed. This remains to be an unsolved problem in QA system. In order to maintain the semantics we have proposed a novel approach Hybrid NLP based Bi-directional Long Short Term Memory (Bi-LSTM) with attention mechanism. It deals with the negation words and maintains the semantics of the sentence. We also focus on answering any factoid query (i.e. ’what’, ’when’, ’where’, ’who’) that is raised by the user. For this purpose, the use of attention mechanism with softmax activation function has obtained superior results that matches the question type and process the context information effectively. The experimental results are performed over the SQuAD dataset for reading comprehension and the Stanford Negation dataset is used to perform the negation in the RC sentence. The accuracy of the system over negation is obtained as 93.9% and over the QA system is 87%.

scholarly journals Hoax analyzer for Indonesian news using RNNs with fasttext and glove embeddings

2021 ◽  
Vol 10 (4) ◽  
pp. 2130-2136
Author(s):  
Ryan Adipradana ◽  
Bagas Pradipabista Nayoga ◽  
Ryan Suryadi ◽  
Derwin Suhartono
Keyword(s):  
Natural Language Processing ◽  
Natural Language ◽  
Language Processing ◽  
Short Term Memory ◽  
Short Term ◽  
Term Memory ◽  
High Resource ◽  
Long Short Term Memory ◽  
Linguistic Approach ◽  
Gated Recurrent Unit

Misinformation has become an innocuous yet potentially harmful problem ever since the development of internet. Numbers of efforts are done to prevent the consumption of misinformation, including the use of artificial intelligence (AI), mainly natural language processing (NLP). Unfortunately, most of natural language processing use English as its linguistic approach since English is a high resource language. On the contrary, Indonesia language is considered a low resource language thus the amount of effort to diminish consumption of misinformation is low compared to English-based natural language processing. This experiment is intended to compare fastText and GloVe embeddings for four deep neural networks (DNN) models: long short-term memory (LSTM), bidirectional long short-term memory (BI-LSTM), gated recurrent unit (GRU) and bidirectional gated recurrent unit (BI-GRU) in terms of metrics score when classifying news between three classes: fake, valid, and satire. The latter results show that fastText embedding is better than GloVe embedding in supervised text classification, along with BI-GRU + fastText yielding the best result.

scholarly journals Automatic Image Captioning Methods

2020 ◽  
Vol 9 (7S) ◽  
pp. 93-97
Keyword(s):  
Natural Language ◽  
Language Processing ◽  
Short Term Memory ◽  
Image Understanding ◽  
Image Captioning ◽  
Short Term ◽  
Natural Languages ◽  
Term Memory ◽  
Long Short Term Memory ◽  
The Given

A language known to humans is a natural language. In computer science it is the most challenging task to make the computers understand the natural languages and generating caption automatically from the given image. While a lot of work has been done, the total solution to this problem has been demonstrated daunting so far. Image captioning is a crucial job involving linguistic image understanding and the ability to generate interpretation of sentences with proper and accurate structure. It requires expertise in Image processing and natural language processing. The publishers suggest in this practice a system using the multilayer Convolutional Neural Network (CNN) to generate language describing the images and Long Short Term Memory (LSTM) to concisely frame relevant phrases using the driven keywords. We aim in this article to provide a brief overview of current methods and algorithms of image captioning using deep learning. We also address datasets and measurement criteria widely used for the same.

Sentence similarity evaluation using Sent2Vec and siamese neural network with parallel structure

2021 ◽  
pp. 1-10
Author(s):  
Hye-Jeong Song ◽  
Tak-Sung Heo ◽  
Jong-Dae Kim ◽  
Chan-Young Park ◽  
Yu-Seop Kim
Keyword(s):  
Neural Network ◽  
Language Processing ◽  
Short Term Memory ◽  
Parallel Structure ◽  
Short Term ◽  
Similarity Estimation ◽  
Accurate Judgment ◽  
Proposed Model ◽  
Sentence Similarity ◽  
Long Short Term Memory

Sentence similarity evaluation is a significant task used in machine translation, classification, and information extraction in the field of natural language processing. When two sentences are given, an accurate judgment should be made whether the meaning of the sentences is equivalent even if the words and contexts of the sentences are different. To this end, existing studies have measured the similarity of sentences by focusing on the analysis of words, morphemes, and letters. To measure sentence similarity, this study uses Sent2Vec, a sentence embedding, as well as morpheme word embedding. Vectors representing words are input to the 1-dimension convolutional neural network (1D-CNN) with various sizes of kernels and bidirectional long short-term memory (Bi-LSTM). Self-attention is applied to the features transformed through Bi-LSTM. Subsequently, vectors undergoing 1D-CNN and self-attention are converted through global max pooling and global average pooling to extract specific values, respectively. The vectors generated through the above process are concatenated to the vector generated through Sent2Vec and are represented as a single vector. The vector is input to softmax layer, and finally, the similarity between the two sentences is determined. The proposed model can improve the accuracy by up to 5.42% point compared with the conventional sentence similarity estimation models.

scholarly journals Efficient natural language classification algorithm for detecting duplicate unsupervised features

2021 ◽  
Vol 20 (3) ◽  
pp. 623-653
Author(s):  
Saud Altaf ◽  
Sofia Iqbal ◽  
Muhammad Waseem Soomro
Keyword(s):  
Natural Language ◽  
Short Term Memory ◽  
Short Term ◽  
Vocabulary Size ◽  
Language Understanding ◽  
Inverse Document Frequency ◽  
Classification Technique ◽  
Document Frequency ◽  
Text Features ◽  
Long Short Term Memory

This paper focuses on capturing the meaning of Natural Language Understanding (NLU) text features to detect the duplicate unsupervised features. The NLU features are compared with lexical approaches to prove the suitable classification technique. The transfer-learning approach is utilized to train the extraction of features on the Semantic Textual Similarity (STS) task. All features are evaluated with two types of datasets that belong to Bosch bug and Wikipedia article reports. This study aims to structure the recent research efforts by comparing NLU concepts for featuring semantics of text and applying it to IR. The main contribution of this paper is a comparative study of semantic similarity measurements. The experimental results demonstrate the Term Frequency–Inverse Document Frequency (TF-IDF) feature results on both datasets with reasonable vocabulary size. It indicates that the Bidirectional Long Short Term Memory (BiLSTM) can learn the structure of a sentence to improve the classification.

scholarly journals Deep Learning-Based Sentiment Analysis of COVID-19 Vaccination Responses from Twitter Data

2021 ◽  
Vol 2021 ◽  
pp. 1-15
Author(s):  
Kazi Nabiul Alam ◽  
Md Shakib Khan ◽  
Abdur Rab Dhruba ◽  
Mohammad Monirujjaman Khan ◽  
Jehad F. Al-Amri ◽  
...  
Keyword(s):  
Deep Learning ◽  
Language Processing ◽  
Performance Metrics ◽  
Short Term Memory ◽  
Confusion Matrix ◽  
Short Term ◽  
Learning Techniques ◽  
The World ◽  
Long Short Term Memory ◽  
Severe Anxiety

The COVID-19 pandemic has had a devastating effect on many people, creating severe anxiety, fear, and complicated feelings or emotions. After the initiation of vaccinations against coronavirus, people’s feelings have become more diverse and complex. Our aim is to understand and unravel their sentiments in this research using deep learning techniques. Social media is currently the best way to express feelings and emotions, and with the help of Twitter, one can have a better idea of what is trending and going on in people’s minds. Our motivation for this research was to understand the diverse sentiments of people regarding the vaccination process. In this research, the timeline of the collected tweets was from December 21 to July21. The tweets contained information about the most common vaccines available recently from across the world. The sentiments of people regarding vaccines of all sorts were assessed using the natural language processing (NLP) tool, Valence Aware Dictionary for sEntiment Reasoner (VADER). Initializing the polarities of the obtained sentiments into three groups (positive, negative, and neutral) helped us visualize the overall scenario; our findings included 33.96% positive, 17.55% negative, and 48.49% neutral responses. In addition, we included our analysis of the timeline of the tweets in this research, as sentiments fluctuated over time. A recurrent neural network- (RNN-) oriented architecture, including long short-term memory (LSTM) and bidirectional LSTM (Bi-LSTM), was used to assess the performance of the predictive models, with LSTM achieving an accuracy of 90.59% and Bi-LSTM achieving 90.83%. Other performance metrics such as precision,, F1-score, and a confusion matrix were also used to validate our models and findings more effectively. This study improves understanding of the public’s opinion on COVID-19 vaccines and supports the aim of eradicating coronavirus from the world.

Sign in / Sign up

Export Citation Format

Share Document