scholarly journals The Use of Transformer Model in Opinion Summarisation

Webology ◽  
2021 ◽  
Vol 18 (Special Issue 05) ◽  
pp. 1084-1095
Author(s):  
Kethan Pabbi ◽  
C. Sindhu ◽  
Isukapalli Sainath Reddy ◽  
Bhumireddy Naga Sai Abhijit

We live in an age of information, therefore collected data and documentation are practically treasure resources. All about a business and its development can be estimated with clarity via statistics. Any machine that could really analyse information to predict a projected outcome is known for being extremely vital for the business. It is critical for the system to provide accurate and useful knowledge of the products in order to conduct accurate assessment. Summarisation is a technique for obtaining a rundown from series of sentences in a study or observation that facilitates us with understanding the basic content of the knowledge expressed within. Simple and brief summaries of just a product will assist the system in performing prospective product research and development. In our paper, we use a deep learning framework that provides to extract clean, relevant, brief summaries from comprehensive customer feedback. Strategies of abstractive text summarisation is used. The method of extracting the primary keyword from a statement and using them in the summary is defined as extractive text summarisation. We utilise abstractive summarisation in this case, which evolves from sample information and provides the best feasible description. Utilising Transformer with Depth Scaling MultiHeaded Attention as well as GloVe word embedding with positional encoding, we illustrate an abstractive approach to extract summaries from the Amazon fine food reviews dataset. Transformer aids in the parallelisation of workloads in order to process data more quickly. We have used an Attention layer which boost the model's quality and enables it to become more effective. The BLUE rating is used to quantify the model's potency.

2020 ◽  
Vol 34 (05) ◽  
pp. 8139-8146
Author(s):  
Duong Le ◽  
My Thai ◽  
Thien Nguyen

The current deep learning works on metaphor detection have only considered this task independently, ignoring the useful knowledge from the related tasks and knowledge resources. In this work, we introduce two novel mechanisms to improve the performance of the deep learning models for metaphor detection. The first mechanism employs graph convolutional neural networks (GCN) with dependency parse trees to directly connect the words of interest with their important context words for metaphor detection. The GCN networks in this work also present a novel control mechanism to filter the learned representation vectors to retain the most important information for metaphor detection. The second mechanism, on the other hand, features a multi-task learning framework that exploits the similarity between word sense disambiguation and metaphor detection to transfer the knowledge between the two tasks. The extensive experiments demonstrate the effectiveness of the proposed techniques, yielding the state-of-the-art performance over several datasets.


2021 ◽  
Vol 15 ◽  
Author(s):  
Zhengning Wang ◽  
Dawei Peng ◽  
Yongbin Shang ◽  
Jingjing Gao

Autism spectrum disorder (ASD) is a range of neurodevelopmental disorders, which brings enormous burdens to the families of patients and society. However, due to the lack of representation of variance for diseases and the absence of biomarkers for diagnosis, the early detection and intervention of ASD are remarkably challenging. In this study, we proposed a self-attention deep learning framework based on the transformer model on structural MR images from the ABIDE consortium to classify ASD patients from normal controls and simultaneously identify the structural biomarkers. In our work, the individual structural covariance networks are used to perform ASD/NC classification via a self-attention deep learning framework, instead of the original structural MR data, to take full advantage of the coordination patterns of morphological features between brain regions. The self-attention deep learning framework based on the transformer model can extract both local and global information from the input data, making it more suitable for the brain network data than the CNN- structural model. Meanwhile, the potential diagnosis structural biomarkers are identified by the self-attention coefficients map. The experimental results showed that our proposed method outperforms most of the current methods for classifying ASD patients with the ABIDE data and achieves a classification accuracy of 72.5% across different sites. Furthermore, the potential diagnosis biomarkers were found mainly located in the prefrontal cortex, temporal cortex, and cerebellum, which may be treated as the early biomarkers for the ASD diagnosis. Our study demonstrated that the self-attention deep learning framework is an effective way to diagnose ASD and establish the potential biomarkers for ASD.


2021 ◽  
Vol 4 (2) ◽  
pp. 127-139
Author(s):  
Ivan M. Lobachev ◽  
Svitlana G. Antoshchuk ◽  
Mykola A. Hodovychenko

This work is devoted to the development of a distributed framework based on deep learning for processing data from various sensors that are generated by transducer networks that are used in the field of smart buildings. The proposed framework allows you to process data that comes from sensors of various types to solve classification and regression problems. The framework architecture consists of several subnets: particular convolutional net that handle input from the same type of sensors, a single convolutional fusion net that processes multiple outputs of particular convolutional nets. Further, the result of a single convolutional fusion net is fed to the input of a recurrent net, which allows extracting meaningful features from time sequences. The result of the recurrent net opera- tion is fed to the output layer, which generates the framework output based on the type of problem being solved. For the experimental evaluation of the developed framework, two tasks were taken: the task of recognizing human actions and the task of identifying a person by movement. The dataset contained data from two sensors (accelerometer and gyroscope), which were collected from 9 users who performed 6 actions. A mobile device was used as the hardware platforms, as well as the Edison Compute Module hardware device. To compare the results of the work, variations of the proposed framework with different architectures were used, as well as third-party approaches based on various methods of machine learning, including support machines of vectors, a random forest, lim- ited Boltzmann machines, and so on. As a result, the proposed framework, on average, surpassed other algorithms by about 8% in three metrics in the task of recognizing human actions and turned out to be about 13% more efficient in the task of identifying a per- son by movement. We also measured the power consumption and operating time of the proposed framework and its analogues. It was found that the proposed framework consumes a moderate amount of energy, and the operating time can be estimated as acceptable.


2020 ◽  
Author(s):  
Raniyaharini R ◽  
Madhumitha K ◽  
Mishaa S ◽  
Virajaravi R

2021 ◽  
Vol 15 (8) ◽  
pp. 898-911
Author(s):  
Yongqing Zhang ◽  
Jianrong Yan ◽  
Siyu Chen ◽  
Meiqin Gong ◽  
Dongrui Gao ◽  
...  

Rapid advances in biological research over recent years have significantly enriched biological and medical data resources. Deep learning-based techniques have been successfully utilized to process data in this field, and they have exhibited state-of-the-art performances even on high-dimensional, nonstructural, and black-box biological data. The aim of the current study is to provide an overview of the deep learning-based techniques used in biology and medicine and their state-of-the-art applications. In particular, we introduce the fundamentals of deep learning and then review the success of applying such methods to bioinformatics, biomedical imaging, biomedicine, and drug discovery. We also discuss the challenges and limitations of this field, and outline possible directions for further research.


2020 ◽  
Author(s):  
Jinseok Lee

BACKGROUND The coronavirus disease (COVID-19) has explosively spread worldwide since the beginning of 2020. According to a multinational consensus statement from the Fleischner Society, computed tomography (CT) can be used as a relevant screening tool owing to its higher sensitivity for detecting early pneumonic changes. However, physicians are extremely busy fighting COVID-19 in this era of worldwide crisis. Thus, it is crucial to accelerate the development of an artificial intelligence (AI) diagnostic tool to support physicians. OBJECTIVE We aimed to quickly develop an AI technique to diagnose COVID-19 pneumonia and differentiate it from non-COVID pneumonia and non-pneumonia diseases on CT. METHODS A simple 2D deep learning framework, named fast-track COVID-19 classification network (FCONet), was developed to diagnose COVID-19 pneumonia based on a single chest CT image. FCONet was developed by transfer learning, using one of the four state-of-art pre-trained deep learning models (VGG16, ResNet50, InceptionV3, or Xception) as a backbone. For training and testing of FCONet, we collected 3,993 chest CT images of patients with COVID-19 pneumonia, other pneumonia, and non-pneumonia diseases from Wonkwang University Hospital, Chonnam National University Hospital, and the Italian Society of Medical and Interventional Radiology public database. These CT images were split into a training and a testing set at a ratio of 8:2. For the test dataset, the diagnostic performance to diagnose COVID-19 pneumonia was compared among the four pre-trained FCONet models. In addition, we tested the FCONet models on an additional external testing dataset extracted from the embedded low-quality chest CT images of COVID-19 pneumonia in recently published papers. RESULTS Of the four pre-trained models of FCONet, the ResNet50 showed excellent diagnostic performance (sensitivity 99.58%, specificity 100%, and accuracy 99.87%) and outperformed the other three pre-trained models in testing dataset. In additional external test dataset using low-quality CT images, the detection accuracy of the ResNet50 model was the highest (96.97%), followed by Xception, InceptionV3, and VGG16 (90.71%, 89.38%, and 87.12%, respectively). CONCLUSIONS The FCONet, a simple 2D deep learning framework based on a single chest CT image, provides excellent diagnostic performance in detecting COVID-19 pneumonia. Based on our testing dataset, the ResNet50-based FCONet might be the best model, as it outperformed other FCONet models based on VGG16, Xception, and InceptionV3.


Sign in / Sign up

Export Citation Format

Share Document