scholarly journals DisKnow: A Social-Driven Disaster Support Knowledge Extraction System

2020 ◽  
Vol 10 (17) ◽  
pp. 6083
Author(s):  
João Boné ◽  
Mariana Dias ◽  
João C. Ferreira ◽  
Ricardo Ribeiro
Keyword(s):  
Neural Networks ◽  
Crisis Management ◽  
Language Processing ◽  
Visual Information ◽  
State Of The Art ◽  
Text Processing ◽  
Data Extraction ◽  
Extraction System ◽  
Management Process ◽  
Knowledge Graphs

This research is aimed at creating and presenting DisKnow, a data extraction system with the capability of filtering and abstracting tweets, to improve community resilience and decision-making in disaster scenarios. Nowadays most people act as human sensors, exposing detailed information regarding occurring disasters, in social media. Through a pipeline of natural language processing (NLP) tools for text processing, convolutional neural networks (CNNs) for classifying and extracting disasters, and knowledge graphs (KG) for presenting connected insights, it is possible to generate real-time visual information about such disasters and affected stakeholders, to better the crisis management process, by disseminating such information to both relevant authorities and population alike. DisKnow has proved to be on par with the state-of-the-art Disaster Extraction systems, and it contributes with a way to easily manage and present such happenings.

Advances in Computational Linguistics and Text Processing Frameworks

2020 ◽  
pp. 217-244
Author(s):  
Ayush Srivastav ◽  
Hera Khan ◽  
Amit Kumar Mishra
Keyword(s):  
Neural Networks ◽  
Natural Language Processing ◽  
Natural Language ◽  
Computational Linguistics ◽  
Language Processing ◽  
Text Processing ◽  
Named Entity Recognition ◽  
Entity Recognition ◽  
Named Entity ◽  
Part Of Speech

The chapter provides an eloquent account of the major methodologies and advances in the field of Natural Language Processing. The most popular models that have been used over time for the task of Natural Language Processing have been discussed along with their applications in their specific tasks. The chapter begins with the fundamental concepts of regex and tokenization. It provides an insight to text preprocessing and its methodologies such as Stemming and Lemmatization, Stop Word Removal, followed by Part-of-Speech tagging and Named Entity Recognition. Further, this chapter elaborates the concept of Word Embedding, its various types, and some common frameworks such as word2vec, GloVe, and fastText. A brief description of classification algorithms used in Natural Language Processing is provided next, followed by Neural Networks and its advanced forms such as Recursive Neural Networks and Seq2seq models that are used in Computational Linguistics. A brief description of chatbots and Memory Networks concludes the chapter.

scholarly journals Convolutional Neural Networks Inference Memory Optimization with Receptive Field-Based InputTiling

2021 ◽  
Author(s):  
Weihao Zhuang ◽  
Tristan Hascoet ◽  
Xunquan Chen ◽  
Ryoichi Takashima ◽  
Tetsuya Takiguchi ◽  
...  
Keyword(s):  
Neural Networks ◽  
Computer Vision ◽  
Convolutional Neural Networks ◽  
Language Processing ◽  
State Of The Art ◽  
Input Image ◽  
Memory Consumption ◽  
Excellent Performance ◽  
Conceptual Approach ◽  
Recent Developments

Abstract Currently, deep learning plays an indispensable role in many fields, including computer vision, natural language processing, and speech recognition. Convolutional Neural Networks (CNNs) have demonstrated excellent performance in computer vision tasks thanks to their powerful feature extraction capability. However, as the larger models have shown higher accuracy, recent developments have led to state-of-the-art CNN models with increasing resource consumption. This paper investigates a conceptual approach to reduce the memory consumption of CNN inference. Our method consists of processing the input image in a sequence of carefully designed tiles within the lower subnetwork of the CNN, so as to minimize its peak memory consumption, while keeping the end-to-end computation unchanged. This method introduces a trade-off between memory consumption and computations, which is particularly suitable for high-resolution inputs. Our experimental results show that MobileNetV2 memory consumption can be reduced by up to 5.3 times with our proposed method. For ResNet50, one of the most commonly used CNN models in computer vision tasks, memory can be optimized by up to 2.3 times.

scholarly journals Grapheme-to-Phoneme Conversion with Convolutional Neural Networks

2019 ◽  
Vol 9 (6) ◽  
pp. 1143 ◽  
Author(s):  
Sevinj Yolchuyeva ◽  
Géza Németh ◽  
Bálint Gyires-Tóth
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Convolutional Neural Network ◽  
Convolutional Neural Networks ◽  
Language Processing ◽  
Speech Synthesis ◽  
State Of The Art ◽  
Error Rates ◽  
Written Form ◽  
G2p Conversion

Grapheme-to-phoneme (G2P) conversion is the process of generating pronunciation for words based on their written form. It has a highly essential role for natural language processing, text-to-speech synthesis and automatic speech recognition systems. In this paper, we investigate convolutional neural networks (CNN) for G2P conversion. We propose a novel CNN-based sequence-to-sequence (seq2seq) architecture for G2P conversion. Our approach includes an end-to-end CNN G2P conversion with residual connections and, furthermore, a model that utilizes a convolutional neural network (with and without residual connections) as encoder and Bi-LSTM as a decoder. We compare our approach with state-of-the-art methods, including Encoder-Decoder LSTM and Encoder-Decoder Bi-LSTM. Training and inference times, phoneme and word error rates were evaluated on the public CMUDict dataset for US English, and the best performing convolutional neural network-based architecture was also evaluated on the NetTalk dataset. Our method approaches the accuracy of previous state-of-the-art results in terms of phoneme error rate.

scholarly journals Sentence embeddings in NLI with iterative refinement encoders

2019 ◽  
Vol 25 (4) ◽  
pp. 467-482 ◽  
Author(s):  
Aarne Talman ◽  
Anssi Yli-Jyrä ◽  
Jörg Tiedemann
Keyword(s):  
Neural Networks ◽  
Natural Language Processing ◽  
Natural Language ◽  
Language Processing ◽  
Recurrent Neural Networks ◽  
State Of The Art ◽  
Iterative Refinement ◽  
Learning Tasks ◽  
Sentence Level ◽  
Refinement Strategy

AbstractSentence-level representations are necessary for various natural language processing tasks. Recurrent neural networks have proven to be very effective in learning distributed representations and can be trained efficiently on natural language inference tasks. We build on top of one such model and propose a hierarchy of bidirectional LSTM and max pooling layers that implements an iterative refinement strategy and yields state of the art results on the SciTail dataset as well as strong results for Stanford Natural Language Inference and Multi-Genre Natural Language Inference. We can show that the sentence embeddings learned in this way can be utilized in a wide variety of transfer learning tasks, outperforming InferSent on 7 out of 10 and SkipThought on 8 out of 9 SentEval sentence embedding evaluation tasks. Furthermore, our model beats the InferSent model in 8 out of 10 recently published SentEval probing tasks designed to evaluate sentence embeddings’ ability to capture some of the important linguistic properties of sentences.

scholarly journals Visual Weather Property Prediction by Multi-Task Learning and Two-Dimensional RNNs

Atmosphere ◽  
2021 ◽  
Vol 12 (5) ◽  
pp. 584
Author(s):  
Wei-Ta Chu ◽  
Yu-Hsuan Liang ◽  
Kai-Chia Ho
Keyword(s):  
Neural Networks ◽  
Visual Information ◽  
Temporal Evolution ◽  
State Of The Art ◽  
Image Data ◽  
Visual Features ◽  
Two Dimensional ◽  
Estimation Model ◽  
Property Estimation ◽  
Task Learning

We attempted to employ convolutional neural networks to extract visual features and developed recurrent neural networks for weather property estimation using only image data. Four common weather properties are estimated, i.e., temperature, humidity, visibility, and wind speed. Based on the success of previous works on temperature prediction, we extended them in terms of two aspects. First, by considering the effectiveness of deep multi-task learning, we jointly estimated four weather properties on the basis of the same visual information. Second, we propose that weather property estimations considering temporal evolution can be conducted from two perspectives, i.e., day-wise or hour-wise. A two-dimensional recurrent neural network is thus proposed to unify the two perspectives. In the evaluation, we show that better prediction accuracy can be obtained compared to the state-of-the-art models. We believe that the proposed approach is the first visual weather property estimation model trained based on multi-task learning.

scholarly journals Review of neural approaches for conditional text generation

2021 ◽  
pp. 102-107
Author(s):  
O. H. Skurzhanskyi ◽  
A. A. Marchenko
Keyword(s):  
Artificial Intelligence ◽  
Neural Networks ◽  
Language Processing ◽  
Test Generation ◽  
State Of The Art ◽  
Text Generation ◽  
Text Simplification ◽  
Local Sequence ◽  
Key Aspects ◽  
Paraphrase Generation

The article is devoted to the review of conditional test generation, one of the most promising fields of natural language processing and artificial intelligence. Specifically, we explore monolingual local sequence transduction tasks: paraphrase generation, grammatical and spelling errors correction, text simplification. To give a better understanding of the considered tasks, we show examples of good rewrites. Then we take a deep look at such key aspects as publicly available datasets with the splits (training, validation, and testing), quality metrics for proper evaluation, and modern solutions based primarily on modern neural networks. For each task, we analyze its main characteristics and how they influence the state-of-the-art models. Eventually, we investigate the most significant shared features for the whole group of tasks in general and for approaches that provide solutions for them.

scholarly journals Graph Convolutional Networks for Text Classification

2019 ◽  
Vol 33 ◽  
pp. 7370-7377 ◽  
Author(s):  
Liang Yao ◽  
Chengsheng Mao ◽  
Yuan Luo
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Language Processing ◽  
Text Classification ◽  
State Of The Art ◽  
Classical Problem ◽  
Experimental Results ◽  
Training Data ◽  
Convolutional Networks ◽  
Single Text

Text classification is an important and classical problem in natural language processing. There have been a number of studies that applied convolutional neural networks (convolution on regular grid, e.g., sequence) to classification. However, only a limited number of studies have explored the more flexible graph convolutional neural networks (convolution on non-grid, e.g., arbitrary graph) for the task. In this work, we propose to use graph convolutional networks for text classification. We build a single text graph for a corpus based on word co-occurrence and document word relations, then learn a Text Graph Convolutional Network (Text GCN) for the corpus. Our Text GCN is initialized with one-hot representation for word and document, it then jointly learns the embeddings for both words and documents, as supervised by the known class labels for documents. Our experimental results on multiple benchmark datasets demonstrate that a vanilla Text GCN without any external word embeddings or knowledge outperforms state-of-the-art methods for text classification. On the other hand, Text GCN also learns predictive word and document embeddings. In addition, experimental results show that the improvement of Text GCN over state-of-the-art comparison methods become more prominent as we lower the percentage of training data, suggesting the robustness of Text GCN to less training data in text classification.

scholarly journals Preliminary Results on Different Text Processing Tasks Using Encoder-Decoder Networks and the Causal Feature Extractor

2020 ◽  
Vol 10 (17) ◽  
pp. 5772
Author(s):  
Adrián Javaloy ◽  
Ginés García-Mateos
Keyword(s):  
Language Processing ◽  
Network Architecture ◽  
State Of The Art ◽  
Text Processing ◽  
Short Term ◽  
Training Time ◽  
Preliminary Results ◽  
Feature Extractor ◽  
Decoder Architecture ◽  
Different Types

Deep learning methods are gaining popularity in different application domains, and especially in natural language processing. It is commonly believed that using a large enough dataset and an adequate network architecture, almost any processing problem can be solved. A frequent and widely used typology is the encoder-decoder architecture, where the input data is transformed into an intermediate code by means of an encoder, and then a decoder takes this code to produce its output. Different types of networks can be used in the encoder and the decoder, depending on the problem of interest, such as convolutional neural networks (CNN) or long-short term memories (LSTM). This paper uses for the encoder a method recently proposed, called Causal Feature Extractor (CFE). It is based on causal convolutions (i.e., convolutions that depend only on one direction of the input), dilatation (i.e., increasing the aperture size of the convolutions) and bidirectionality (i.e., independent networks in both directions). Some preliminary results are presented on three different tasks and compared with state-of-the-art methods: bilingual translation, LaTeX decompilation and audio transcription. The proposed method achieves promising results, showing its ubiquity to work with text, audio and images. Moreover, it has a shorter training time, requiring less time per iteration, and a good use of the attention mechanisms based on attention matrices.

scholarly journals Innovative Topologies and Algorithms for Neural Networks

2020 ◽  
Vol 12 (7) ◽  
pp. 117
Author(s):  
Salvatore Graziani ◽  
Maria Gabriella Xibilia
Keyword(s):  
Neural Networks ◽  
Network Architecture ◽  
Deep Neural Networks ◽  
State Of The Art ◽  
Text Processing ◽  
Successful Outcome ◽  
Training Algorithm ◽  
Special Issue ◽  
And Training ◽  
Selection Of

The introduction of new topologies and training procedures to deep neural networks has solicited a renewed interest in the field of neural computation. The use of deep structures has significantly improved the state of the art in many applications, such as computer vision, speech and text processing, medical applications, and IoT (Internet of Things). The probability of a successful outcome from a neural network is linked to selection of an appropriate network architecture and training algorithm. Accordingly, much of the recent research on neural networks is devoted to the study and proposal of novel architectures, including solutions tailored to specific problems. The papers of this Special Issue make significant contributions to the above-mentioned fields by merging theoretical aspects and relevant applications. Twelve papers are collected in the issue, addressing many relevant aspects of the topic.

scholarly journals Few-Shot Knowledge Graph Completion

2020 ◽  
Vol 34 (03) ◽  
pp. 3041-3048 ◽  
Author(s):  
Chuxu Zhang ◽  
Huaxiu Yao ◽  
Chao Huang ◽  
Meng Jiang ◽  
Zhenhui Li ◽  
...  
Keyword(s):  
Natural Language Processing ◽  
Natural Language ◽  
Language Processing ◽  
State Of The Art ◽  
Knowledge Graph ◽  
Graph Structure ◽  
Reference Set ◽  
Knowledge Graphs ◽  
Public Datasets ◽  
Relation Learning

Knowledge graphs (KGs) serve as useful resources for various natural language processing applications. Previous KG completion approaches require a large number of training instances (i.e., head-tail entity pairs) for every relation. The real case is that for most of the relations, very few entity pairs are available. Existing work of one-shot learning limits method generalizability for few-shot scenarios and does not fully use the supervisory information; however, few-shot KG completion has not been well studied yet. In this work, we propose a novel few-shot relation learning model (FSRL) that aims at discovering facts of new relations with few-shot references. FSRL can effectively capture knowledge from heterogeneous graph structure, aggregate representations of few-shot references, and match similar entity pairs of reference set for every relation. Extensive experiments on two public datasets demonstrate that FSRL outperforms the state-of-the-art.

Sign in / Sign up

Export Citation Format

Share Document