scholarly journals A Video Captioning Method Based on Multi-Representation Switching for Sustainable Computing

2021 ◽  
Vol 13 (4) ◽  
pp. 2250
Author(s):  
Heechan Kim ◽  
Soowon Lee
Keyword(s):  
Natural Language ◽  
Language Processing ◽  
Entity Extraction ◽  
Video Captioning ◽  
Motion Extraction ◽  
Video Description ◽  
Switching Method ◽  
Deep Learning Model ◽  
Additional Loss ◽  
Three Components

Video captioning is a problem that generates a natural language sentence as a video’s description. A video description includes not only words that express the objects in the video but also words that express the relationships between the objects, or grammatically necessary words. To reflect this characteristic explicitly using a deep learning model, we propose a multi-representation switching method. The proposed method consists of three components: entity extraction, motion extraction, and textual feature extraction. The proposed multi-representation switching method makes it possible for the three components to extract important information for a given video and description pair efficiently. In experiments conducted on the Microsoft Research Video Description dataset, the proposed method recorded scores that exceeded the performance of most existing video captioning methods. This result was achieved without any preprocessing based on computer vision and natural language processing, nor any additional loss function. Consequently, the proposed method has a high generality that can be extended to various domains in terms of sustainable computing.

scholarly journals A Natural Language Processing System for Extracting Evidence of Drug Repurposing from Scientific Publications

2020 ◽  
Vol 34 (08) ◽  
pp. 13369-13381
Author(s):  
Shivashankar Subramanian ◽  
Ioana Baldini ◽  
Sushma Ravichandran ◽  
Dmitriy A. Katz-Rogozhnikov ◽  
Karthikeyan Natesan Ramamurthy ◽  
...  
Keyword(s):  
Natural Language Processing ◽  
Natural Language ◽  
Language Processing ◽  
Generic Drugs ◽  
Low Cost ◽  
Processing System ◽  
Drug Repurposing ◽  
Cancer Type ◽  
Entity Extraction ◽  
Scientific Publications

More than 200 generic drugs approved by the U.S. Food and Drug Administration for non-cancer indications have shown promise for treating cancer. Due to their long history of safe patient use, low cost, and widespread availability, repurposing of these drugs represents a major opportunity to rapidly improve outcomes for cancer patients and reduce healthcare costs. In many cases, there is already evidence of efficacy for cancer, but trying to manually extract such evidence from the scientific literature is intractable. In this emerging applications paper, we introduce a system to automate non-cancer generic drug evidence extraction from PubMed abstracts. Our primary contribution is to define the natural language processing pipeline required to obtain such evidence, comprising the following modules: querying, filtering, cancer type entity extraction, therapeutic association classification, and study type classification. Using the subject matter expertise on our team, we create our own datasets for these specialized domain-specific tasks. We obtain promising performance in each of the modules by utilizing modern language processing techniques and plan to treat them as baseline approaches for future improvement of individual components.

scholarly journals Video Captioning Based on Both Egocentric and Exocentric Views of Robot Vision for Human-Robot Interaction

2021 ◽  
Author(s):  
Soo-Han Kang ◽  
Ji-Hyeong Han
Keyword(s):  
Deep Learning ◽  
Natural Language ◽  
Vision System ◽  
Robot Vision ◽  
Learning Model ◽  
Human Robot Interaction ◽  
Robot Interaction ◽  
Video Captioning ◽  
Global Action ◽  
Deep Learning Model

AbstractRobot vision provides the most important information to robots so that they can read the context and interact with human partners successfully. Moreover, to allow humans recognize the robot’s visual understanding during human-robot interaction (HRI), the best way is for the robot to provide an explanation of its understanding in natural language. In this paper, we propose a new approach by which to interpret robot vision from an egocentric standpoint and generate descriptions to explain egocentric videos particularly for HRI. Because robot vision equals to egocentric video on the robot’s side, it contains as much egocentric view information as exocentric view information. Thus, we propose a new dataset, referred to as the global, action, and interaction (GAI) dataset, which consists of egocentric video clips and GAI descriptions in natural language to represent both egocentric and exocentric information. The encoder-decoder based deep learning model is trained based on the GAI dataset and its performance on description generation assessments is evaluated. We also conduct experiments in actual environments to verify whether the GAI dataset and the trained deep learning model can improve a robot vision system

scholarly journals An artificial intelligence approach to COVID-19 infection risk assessment in virtual visits: A case report

2020 ◽  
Vol 27 (8) ◽  
pp. 1321-1325 ◽  
Author(s):  
Jihad S Obeid ◽  
Matthew Davis ◽  
Matthew Turner ◽  
Stephane M Meystre ◽  
Paul M Heider ◽  
...  
Keyword(s):  
Artificial Intelligence ◽  
Natural Language Processing ◽  
Natural Language ◽  
Language Processing ◽  
Characteristic Curve ◽  
Computer Algorithms ◽  
Outbreak Response ◽  
Virtual Visits ◽  
Informatics Tools ◽  
Deep Learning Model

Abstract Objective In an effort to improve the efficiency of computer algorithms applied to screening for coronavirus disease 2019 (COVID-19) testing, we used natural language processing and artificial intelligence–based methods with unstructured patient data collected through telehealth visits. Materials and Methods After segmenting and parsing documents, we conducted analysis of overrepresented words in patient symptoms. We then developed a word embedding–based convolutional neural network for predicting COVID-19 test results based on patients’ self-reported symptoms. Results Text analytics revealed that concepts such as smell and taste were more prevalent than expected in patients testing positive. As a result, screening algorithms were adapted to include these symptoms. The deep learning model yielded an area under the receiver-operating characteristic curve of 0.729 for predicting positive results and was subsequently applied to prioritize testing appointment scheduling. Conclusions Informatics tools such as natural language processing and artificial intelligence methods can have significant clinical impacts when applied to data streams early in the development of clinical systems for outbreak response.

scholarly journals Attention as Relation: Learning Supervised Multi-head Self-Attention for Relation Extraction

2020 ◽  
Author(s):  
Jie Liu ◽  
Shaowei Chen ◽  
Bingquan Wang ◽  
Jiaxin Zhang ◽  
Na Li ◽  
...  
Keyword(s):  
Natural Language Processing ◽  
Natural Language ◽  
Language Processing ◽  
State Of The Art ◽  
Relation Extraction ◽  
Attention Mechanism ◽  
Entity Extraction ◽  
Relation Type ◽  
Benchmark Datasets ◽  
Relation Learning

Joint entity and relation extraction is critical for many natural language processing (NLP) tasks, which has attracted increasing research interest. However, it is still faced with the challenges of identifying the overlapping relation triplets along with the entire entity boundary and detecting the multi-type relations. In this paper, we propose an attention-based joint model, which mainly contains an entity extraction module and a relation detection module, to address the challenges. The key of our model is devising a supervised multi-head self-attention mechanism as the relation detection module to learn the token-level correlation for each relation type separately. With the attention mechanism, our model can effectively identify overlapping relations and flexibly predict the relation type with its corresponding intensity. To verify the effectiveness of our model, we conduct comprehensive experiments on two benchmark datasets. The experimental results demonstrate that our model achieves state-of-the-art performances.

scholarly journals Automatic speech recognition: can you understand me?

2021 ◽  
pp. 121-126
Author(s):  
Susana Pérez Castillejo
Keyword(s):  
Speech Recognition ◽  
Natural Language ◽  
Language Processing ◽  
Automatic Speech Recognition ◽  
Text Messaging ◽  
Digital Communication ◽  
Spoken Discourse ◽  
Video Captioning ◽  
Written Text ◽  
Communication Method

What is it? Automatic Speech Recognition (ASR) is a digital communication method that transforms spoken discourse into written text. This rapidly evolving technology is used in email, text messaging, or live video captioning. Current ASR systems operate in conjunction with Natural Language Processing (NLP) technology to transform speech into text that people – and machines – can read. NLP refers to the methodologies and computational tools that analyze data produced in a natural language, such as English.

scholarly journals POP-ON: Prediction of Process Using One-Way Language Model Based on NLP Approach

2021 ◽  
Vol 11 (2) ◽  
pp. 864
Author(s):  
Junhyung Moon ◽  
Gyuyoung Park ◽  
Jongpil Jeong
Keyword(s):  
Natural Language ◽  
Language Processing ◽  
Business Process ◽  
Process Management ◽  
Language Model ◽  
Superior Performance ◽  
Event Prediction ◽  
Process Prediction ◽  
Monitoring Service ◽  
Deep Learning Model

In business process management, the monitoring service is an important element that can prevent various problems in advance from before they occur in companies and industries. Execution log is created in an information system that is aware of the enterprise process, which helps predict the process. The ultimate goal of the proposed method is to predict the process following the running process instance and predict events based on previously completed event log data. Companies can flexibly respond to unwanted deviations in their workflow. When solving the next event prediction problem, we use a fully attention-based transformer, which has performed well in recent natural language processing approaches. After recognizing the name attribute of the event in the natural language and predicting the next event, several necessary elements were applied. It is trained using the proposed deep learning model according to specific pre-processing steps. Experiments using various business process log datasets demonstrate the superior performance of the proposed method. The name of the process prediction model we propose is “POP-ON”.

scholarly journals Deep Learning for Video Captioning: A Review

2019 ◽  
Author(s):  
Shaoxiang Chen ◽  
Ting Yao ◽  
Yu-Gang Jiang
Keyword(s):  
Deep Learning ◽  
Natural Language ◽  
Language Processing ◽  
State Of The Art ◽  
Future Research ◽  
Research Directions ◽  
Video Captioning ◽  
Future Research Directions ◽  
Review State ◽  
Vision And Language

Deep learning has achieved great successes in solving specific artificial intelligence problems recently. Substantial progresses are made on Computer Vision (CV) and Natural Language Processing (NLP). As a connection between the two worlds of vision and language, video captioning is the task of producing a natural-language utterance (usually a sentence) that describes the visual content of a video. The task is naturally decomposed into two sub-tasks. One is to encode a video via a thorough understanding and learn visual representation. The other is caption generation, which decodes the learned representation into a sequential sentence, word by word. In this survey, we first formulate the problem of video captioning, then review state-of-the-art methods categorized by their emphasis on vision or language, and followed by a summary of standard datasets and representative approaches. Finally, we highlight the challenges which are not yet fully understood in this task and present future research directions.

Sign in / Sign up

Export Citation Format

Share Document