Attention-Guided Image Captioning through Word Information

Image captioning generates written descriptions of an image. In recent image captioning research, attention regions seldom cover all objects, and generated captions may lack the details of objects and may remain far from reality. In this paper, we propose a word guided attention (WGA) method for image captioning. First, WGA extracts word information using the embedded word and memory cell by applying transformation and multiplication. Then, WGA applies word information to the attention results and obtains the attended feature vectors via elementwise multiplication. Finally, we apply WGA with the words from different time steps to obtain previous word guided attention (PW) and current word attention (CW) in the decoder. Experiments on the MSCOCO dataset show that our proposed WGA can achieve competitive performance against state-of-the-art methods, with PW results of a 39.1 Bilingual Evaluation Understudy score (BLEU-4) and a 127.6 Consensus-Based Image Description Evaluation score (CIDEr-D); and CW results of a 39.1 BLEU-4 score and a 127.2 CIDER-D score on a Karpathy test split.

Comparing written Indian Englishes with the new Corpus of Regional Indian Newspaper Englishes (CORINNE)

This article introduces the new Corpus of Regional Indian Newspaper Englishes (CORINNE). The current version of CORINNE contains news and other text types from regional Indian newspapers published between 2015 and 2020, covering 13 states and regions so far. The corpus complements previous corpora, such as the Indian component of the International Corpus of English (ICE) as well as the Indian section of the South Asian Varieties of English (SAVE) corpus, by giving researchers the opportunity to analyse and compare regional (written) Englishes in India. In the first sections of the paper we discuss the rationale for creating CORINNE as well as the development of the corpus. We stress the potential of CORINNE and go into detail about selection criteria for the inclusion of newspapers as well as corpus compilation and the current word count. In order to show the potential of the corpus, the paper presents a case study of ‘intrusive as’, a syntactic feature that has made its way into formal registers of Indian English. Based on two subcorpora covering newspapers from Tamil Nadu and Uttarakhand, we compare frequencies and usage patterns of call (as) and term (as). The case study lends further weight to the hypothesis that the presence or absence of a quotative in the majority language spoken in an Indian state has an impact on the frequency of ‘intrusive as’. Finally, we foreshadow the next steps in the development of CORINNE as well as potential studies that can be carried out using the corpus.

Optimization of argon injection into shroud of tundish during continuous casting: a numerical study

In order to prevent molten steel reoxidation in the tundish during continuous casting, argon injection into the shroud is applied. The injected argon bubbles in molten steel may change the molten steel flow pattern in the tundish. Consequently, the ratios of dead zone, plunger zone and well-mixed zone would change. Also, the motion of inclusions in the molten steel of tundish would change resulting from argon injection. The current word developed a coupling multiphase flow mathematical model. Basing on the developed model, the present work has researched the influences of argon injection on the molten steel flow pattern, inclusion motion characteristics in the tundish.

Multidimensional Self-Attention for Aspect Term Extraction and Biomedical Named Entity Recognition

Wide attention has been paid to named entity recognition (NER) in specific fields. Among the representative tasks are the aspect term extraction (ATE) in user online comments and the biomedical named entity recognition (BioNER) in medical documents. Existing methods only perform well in a particular field, and it is difficult to maintain an advantage in other fields. In this article, we propose a supervised learning method that can be used for much special domain NER tasks. The model consists of two parts, a multidimensional self-attention (MDSA) network and a CNN-based model. The multidimensional self-attention mechanism can calculate the importance of the context to the current word, select the relevance according to the importance, and complete the update of the word vector. This update mechanism allows the subsequent CNN model to have variable-length memory of sentence context. We conduct experiments on benchmark datasets of ATE and BioNER tasks. The results show that our model surpasses most baseline methods.

Learn and Tell: Learning Priors for Image Caption Generation

In this work, we propose a novel priors-based attention neural network (PANN) for image captioning, which aims at incorporating two kinds of priors, i.e., the probabilities being mentioned for local region proposals (PBM priors) and part-of-speech clues for caption words (POS priors), into a visual information extraction process at each word prediction. This work was inspired by the intuitions that region proposals have different inherent probabilities for image captioning, and that the POS clues bridge the word class (part-of-speech tag) with the categories of visual features. We propose new methods to extract these two priors, in which the PBM priors are obtained by computing the similarities between the caption feature vector and local feature vectors, while the POS priors are predicated at each step of word generation by taking the hidden state of the decoder as input. After that, these two kinds of priors are further incorporated into the PANN module of the decoder to help the decoder extract more accurate visual information for the current word generation. In our experiments, we qualitatively analyzed the proposed approach and quantitatively evaluated several captioning schemes with our PANN on the MS-COCO dataset. Experimental results demonstrate that our proposed method could achieve better performance as well as the effectiveness of the proposed network for image captioning.

APPLICATION OF INFORMATION TECHNOLOGIES FOR SEMANTIC TEXT PROCESSING

An expert system for text analysis based on the heuristic knowledge of an expert linguist is proposed. Methods of linguistic analysis of the text through the use of computer technology have been further developed. Data verification was performed on the example of the Germanic language group. The algorithm of the system operation is given. The sequence of actions of the text analysis process is described. Research relates to the subject of computational linguistics and helps to automate text analysis processes. The main purpose of the research is to improve the machine's understanding of the semantic structure of the text by finding current connections between the main members of the sentence, current connections between secondary members of the sentence, the best concept of the current word and the function of the current word in the sentence. Semantic networks are used in the software solution. The Java programming shell, such as NetBeans IDE 8.1, and the CLIPS shell, were used to create the software product. The main logical connections and structure of the program are described in the article. Methods and relations are considered on the example of the Germanic group of languages. All languages of the Germanic group are similar because they have a direct line of words, which makes them even more similar: subject + predicate + subordinate clauses. Thus, to reflect the structure of the Germanic group of languages, it is sufficient to consider one of them. Namely, English, as it is the most common (1.5 billion people), international, has the largest vocabulary among the group (500 thousand words) and, in our opinion, the most complex.

Exploiting Electrophysiological Measures of Semantic Processing for Auditory Attention Decoding

ABSTRACTIn Auditory Attention Decoding, a user’s electrophysiological brain responses to certain features of speech are modelled and subsequently used to distinguish attended from unattended speech in multi-speaker contexts. Such approaches are frequently based on acoustic features of speech, such as the auditory envelope. A recent paper shows that the brain’s response to a semantic description (i.e., semantic dissimilarity) of narrative speech can also be modelled using such an approach. Here we use the (publicly available) data accompanying that study, in order to investigate whether combining this semantic dissimilarity feature with an auditory envelope approach improves decoding performance over using the envelope alone. We analyse data from their ‘Cocktail Party’ experiment in which 33 subjects attended to one of two simultaneously presented audiobook narrations, for 30 1-minute fragments. We find that the addition of the dissimilarity feature to an envelope-based approach significantly increases accuracy, though the increase is marginal (85.4% to 86.6%). However, we subsequently show that this dissimilarity feature, in which the degree of dissimilarity of the current word with regard to the previous context is tagged to the onsets of each content word, can be replaced with a binary content-word-onset feature, without significantly affecting the results (i.e., modelled responses or accuracy), putting in question the added value of the dissimilarity information for the approach introduced in this recent paper.

Recurrent Neural Network based Models for Word Prediction

Globally, people are spending a cumulative amount of time on their mobile device, laptop, tab, desktop, etc,. for messaging, sending emails, banking, interaction through social media, and all other activities. It is necessary to cut down the time spend on typing through these devices. It can be achieved when the device can provide the user more options for what the next word might be for the current typed word. It also increases the speed of typing. In this paper, we suggest and presented a comparative study on various models like Recurrent Neural Network, Stacked Recurrent Neural Network, Long Short Term Memory network (LSTM) and Bi-directional LSTM that gives solution for the above said problem. Our primary goal is to suggest the best model among the four models to predict the next word for the given current word in English Language. Our survey says that for predicting next word RNN provide accuracy 60% and loss 40%, Stacked RNN provide accuracy 62% and loss 38%, LSTM provide accuracy 64% and loss 36% and Bidirectional LSTM provide accuracy 72% and loss 28%.

Leap-LSTM: Enhancing Long Short-Term Memory for Text Categorization

Recurrent Neural Networks (RNNs) are widely used in the field of natural language processing (NLP), ranging from text categorization to question answering and machine translation. However, RNNs generally read the whole text from beginning to end or vice versa sometimes, which makes it inefficient to process long texts. When reading a long document for a categorization task, such as topic categorization, large quantities of words are irrelevant and can be skipped. To this end, we propose Leap-LSTM, an LSTM-enhanced model which dynamically leaps between words while reading texts. At each step, we utilize several feature encoders to extract messages from preceding texts, following texts and the current word, and then determine whether to skip the current word. We evaluate Leap-LSTM on several text categorization tasks: sentiment analysis, news categorization, ontology classification and topic classification, with five benchmark data sets. The experimental results show that our model reads faster and predicts better than standard LSTM. Compared to previous models which can also skip words, our model achieves better trade-offs between performance and efficiency.

A Unified Model for Opinion Target Extraction and Target Sentiment Prediction

Target-based sentiment analysis involves opinion target extraction and target sentiment classification. However, most of the existing works usually studied one of these two sub-tasks alone, which hinders their practical use. This paper aims to solve the complete task of target-based sentiment analysis in an end-to-end fashion, and presents a novel unified model which applies a unified tagging scheme. Our framework involves two stacked recurrent neural networks: The upper one predicts the unified tags to produce the final output results of the primary target-based sentiment analysis; The lower one performs an auxiliary target boundary prediction aiming at guiding the upper network to improve the performance of the primary task. To explore the inter-task dependency, we propose to explicitly model the constrained transitions from target boundaries to target sentiment polarities. We also propose to maintain the sentiment consistency within an opinion target via a gate mechanism which models the relation between the features for the current word and the previous word. We conduct extensive experiments on three benchmark datasets and our framework achieves consistently superior results.