Multi-Self-Attention for Aspect Category Detection and Biomedical Multilabel Text Classification with BERT

Multilabel classification is one of the most challenging tasks in natural language processing, posing greater technical difficulties than single-label classification. At the same time, multilabel classification has more natural applications. For individual labels, the whole piece of text has different focuses or component distributions, which require full use of local information of the sentence. As a widely adopted mechanism in natural language processing, attention becomes a natural choice for the issue. This paper proposes a multilayer self-attention model to deal with aspect category and word attention at different granularities. Combined with the BERT pretraining model, it achieves competitive performance in aspect category detection and electronic medical records’ classification.

Attention-Guided Image Captioning through Word Information

Image captioning generates written descriptions of an image. In recent image captioning research, attention regions seldom cover all objects, and generated captions may lack the details of objects and may remain far from reality. In this paper, we propose a word guided attention (WGA) method for image captioning. First, WGA extracts word information using the embedded word and memory cell by applying transformation and multiplication. Then, WGA applies word information to the attention results and obtains the attended feature vectors via elementwise multiplication. Finally, we apply WGA with the words from different time steps to obtain previous word guided attention (PW) and current word attention (CW) in the decoder. Experiments on the MSCOCO dataset show that our proposed WGA can achieve competitive performance against state-of-the-art methods, with PW results of a 39.1 Bilingual Evaluation Understudy score (BLEU-4) and a 127.6 Consensus-Based Image Description Evaluation score (CIDEr-D); and CW results of a 39.1 BLEU-4 score and a 127.2 CIDER-D score on a Karpathy test split.

Interpreting BERT-based stance classification: a case study about the Brazilian COVID vaccination

The actions to control the COVID-19 pandemics should be based on scientific facts. However, Brazil is facing a politically polarized scenario that has influenced the population’s behavior regarding social distance or vaccination issues. This paper addresses this subject by proposing a BERT-based stance classification model and an attention-based mechanism to identify the influential words for stance classification. The interpretation mechanism traces tokens’ attentions back to words, assigning word attention scores (absolute and relative). We use these metrics to assess if words with high attention weights correspond to domain intrinsic properties and contribute to the correct classification of stances. Our experiments revealed good results for stance classification (F1=0.752), and that 74% of the top-100 words with the highest absolute attention are representative of the arguments that support the investigated stances.

Miss Marjoribanks’s Pronouns; or, the General, the Particular, and the Novel

Erik Gray, “Miss Marjoribanks’s Pronouns; or, the General, the Particular, and the Novel” (pp. 223–252) The novel as a genre is always concerned with questions of the general and the particular: it details the particulars of everyday lives as representatives of general truths and characteristics. Margaret Oliphant’s Miss Marjoribanks (1866) not only reflects on this familiar binary but also reveals how easily the distinction between its two terms collapses. The tendency of the heroine, Lucilla Marjoribanks, to refer to all men as “They” illustrates this phenomenon. She uses the pronoun, with no antecedent, to refer either to a particular group of men or to men in general; her doing so both demeans men, by grouping them into an indiscriminate mass, and exalts them, by treating them as so significant as to need no introduction. By the same token, Lucilla’s various suitors are at the same time generalized—they appear as nearly interchangeable functions of the marriage plot—and particularized, since marriage itself involves a form of “particular” (Oliphant’s word) attention. And in the election plot that dominates the final volume of the novel, Lucilla’s chosen candidate, Mr. Ashburton, is singled out precisely for being so typical. Miss Marjoribanks thus demonstrates how the very building blocks of narrative, like those of language, effectively confound the distinction between general and particular. In its elucidation of this tendency of the novel genre, and of art in general, lies the genius and importance of Oliphant’s novel.

A Topic Detection Method Based on Word-attention Networks

Purpose We proposed a method to represent scientific papers by a complex network, which combines the approaches of neural and complex networks. Design/methodology/approach Its novelty is representing a paper by a word branch, which carries the sequential structure of words in sentences. The branches are generated by the attention mechanism in deep learning models. We connected those branches at the positions of their common words to generate networks, called word-attention networks, and then detect their communities, defined as topics. Findings Those detected topics can carry the sequential structure of words in sentences, represent the intra- and inter-sentential dependencies among words, and reveal the roles of words playing in them by network indexes. Research limitations The parameter setting of our method may depend on practical data. Thus it needs human experience to find proper settings. Practical implications Our method is applied to the papers of the PNAS, where the discipline designations provided by authors are used as the golden labels of papers’ topics. Originality/value This empirical study shows that the proposed method outperforms the Latent Dirichlet Allocation and is more stable.

Perceptions and Attitudes towards Mules in a Group of Sodiers

Mules are essential for pack work in mountainous areas, but there is a lack of research on this species. This study intends to assess the perceptions, attitudes, empathy and pain perception of soldiers about mules, to understand the type of human–mule relationship. For this, a survey was applied with closed-ended questions where the empathy and pain perception tools were included and later analyzed through correlations. Open-ended questions were analyzed through text mining. A total of 73 soldiers were surveyed. They had a wide range of ages and years of experience working with equids. Significant positive correlations were found between human empathy, animal empathy and pain perception. Soldiers show a preference for working with mules over donkeys and horses. Text mining analysis shows three clusters associated with the mules’ nutritional, environmental and health needs. In the same line, relevant relations were found for the word “attention” with “load”, “food”, and “harness”. When asked what mules signify for them, two clusters were found, associated with mules’ working capacity and their role in the army. Relevant relations were found between the terms “mountain”, “support”, and “logistics”, and also between “intelligent” and “noble”. To secure mules’ behavioral and emotional needs, future training strategies should include behavior and welfare concepts.

Articulatory-to-Acoustic Conversion Using BiLSTM-CNN Word-Attention-Based Method

In the recent years, along with the development of artificial intelligence (AI) and man-machine interaction technology, speech recognition and production have been asked to adapt to the rapid development of AI and man-machine technology, which need to improve recognition accuracy through adding novel features, fusing the feature, and improving recognition methods. Aiming at developing novel recognition feature and application to speech recognition, this paper presents a new method for articulatory-to-acoustic conversion. In the study, we have converted articulatory features (i.e., velocities of tongue and motion of lips) into acoustic features (i.e., the second formant and Mel-Cepstra). By considering the graphical representation of the articulators’ motion, this study combined Bidirectional Long Short-Term Memory (BiLSTM) with convolution neural network (CNN) and adopted the idea of word attention in Mandarin to extract semantic features. In this paper, we used the electromagnetic articulography (EMA) database designed by Taiyuan University of Technology, which contains ten speakers’ 299 disyllables and sentences of Mandarin, and extracted 8-dimensional articulatory features and 1-dimensional semantic feature relying on the word-attention layer; we then trained 200 samples and tested 99 samples for the articulatory-to-acoustic conversion. Finally, Root Mean Square Error (RMSE), Mean Mel-Cepstral Distortion (MMCD), and correlation coefficient have been used to evaluate the conversion effect and for comparison with Gaussian Mixture Model (GMM) and BiLSTM of recurrent neural network (BiLSTM-RNN). The results illustrated that the MMCD of Mel-Frequency Cepstrum Coefficient (MFCC) was 1.467 dB, and the RMSE of F2 was 22.10 Hz. The research results of this study can be used in the features fusion and speech recognition to improve the accuracy of recognition.