scholarly journals DIBERT: Dependency Injected Bidirectional Encoder Representations from Transformers

2021 ◽  
Author(s):  
Abdul Wahab ◽  
Rafet Sifa
Keyword(s):  
Natural Language ◽  
Sentiment Analysis ◽  
Semantic Similarity ◽  
Syntactic Structure ◽  
Language Modeling ◽  
Benchmark Dataset ◽  
Fine Tuning ◽  
New Model ◽  
Dependency Tree ◽  
Better Than

<div> <div> <div> <p> </p><div> <div> <div> <p>In this paper, we propose a new model named DIBERT which stands for Dependency Injected Bidirectional Encoder Representations from Transformers. DIBERT is a variation of the BERT and has an additional third objective called Parent Prediction (PP) apart from Masked Language Modeling (MLM) and Next Sentence Prediction (NSP). PP injects the syntactic structure of a dependency tree while pre-training the DIBERT which generates syntax-aware generic representations. We use the WikiText-103 benchmark dataset to pre-train both BERT- Base and DIBERT. After fine-tuning, we observe that DIBERT performs better than BERT-Base on various downstream tasks including Semantic Similarity, Natural Language Inference and Sentiment Analysis. </p> </div> </div> </div> </div> </div> </div>

scholarly journals DIBERT: Dependency Injected Bidirectional Encoder Representations from Transformers

2021 ◽  
Author(s):  
Abdul Wahab ◽  
Rafet Sifa
Keyword(s):  
Natural Language ◽  
Sentiment Analysis ◽  
Semantic Similarity ◽  
Syntactic Structure ◽  
Language Modeling ◽  
Benchmark Dataset ◽  
Fine Tuning ◽  
New Model ◽  
Dependency Tree ◽  
Better Than

<div> <div> <div> <p> </p><div> <div> <div> <p>In this paper, we propose a new model named DIBERT which stands for Dependency Injected Bidirectional Encoder Representations from Transformers. DIBERT is a variation of the BERT and has an additional third objective called Parent Prediction (PP) apart from Masked Language Modeling (MLM) and Next Sentence Prediction (NSP). PP injects the syntactic structure of a dependency tree while pre-training the DIBERT which generates syntax-aware generic representations. We use the WikiText-103 benchmark dataset to pre-train both BERT- Base and DIBERT. After fine-tuning, we observe that DIBERT performs better than BERT-Base on various downstream tasks including Semantic Similarity, Natural Language Inference and Sentiment Analysis. </p> </div> </div> </div> </div> </div> </div>

scholarly journals DIBERT: Dependency Injected Bidirectional Encoder Representations from Transformers

2021 ◽  
Author(s):  
Abdul Wahab ◽  
Rafet Sifa
Keyword(s):  
Natural Language ◽  
Sentiment Analysis ◽  
Semantic Similarity ◽  
Syntactic Structure ◽  
Language Modeling ◽  
Benchmark Dataset ◽  
Fine Tuning ◽  
New Model ◽  
Dependency Tree ◽  
Better Than

<div> <div> <div> <p> </p><div> <div> <div> <p>In this paper, we propose a new model named DIBERT which stands for Dependency Injected Bidirectional Encoder Representations from Transformers. DIBERT is a variation of the BERT and has an additional third objective called Parent Prediction (PP) apart from Masked Language Modeling (MLM) and Next Sentence Prediction (NSP). PP injects the syntactic structure of a dependency tree while pre-training the DIBERT which generates syntax-aware generic representations. We use the WikiText-103 benchmark dataset to pre-train both BERT- Base and DIBERT. After fine-tuning, we observe that DIBERT performs better than BERT-Base on various downstream tasks including Semantic Similarity, Natural Language Inference and Sentiment Analysis. </p> </div> </div> </div> </div> </div> </div>

Natural language modeling with syntactic structure dependency

2020 ◽  
Vol 523 ◽  
pp. 220-233
Author(s):  
Kai Shuang ◽  
Yijia Tan ◽  
Zhun Cai ◽  
Yue Sun
Keyword(s):  
Natural Language ◽  
Syntactic Structure ◽  
Language Modeling

scholarly journals Semantic Similarity in a Taxonomy: An Information-Based Measure and its Application to Problems of Ambiguity in Natural Language

1999 ◽  
Vol 11 ◽  
pp. 95-130 ◽  
Author(s):  
P. Resnik
Keyword(s):  
Natural Language ◽  
Information Content ◽  
Semantic Similarity ◽  
Experimental Evaluation ◽  
Experimental Results ◽  
Semantic Ambiguity ◽  
Counting Approach ◽  
Shared Information ◽  
Similarity Judgments ◽  
Better Than

This article presents a measure of semantic similarity in an IS-A taxonomy based on the notion of shared information content. Experimental evaluation against a benchmark set of human similarity judgments demonstrates that the measure performs better than the traditional edge-counting approach. The article presents algorithms that take advantage of taxonomic similarity in resolving syntactic and semantic ambiguity, along with experimental results demonstrating their effectiveness.

A Review on Sentiment Classification: Natural Language Understanding

2019 ◽  
Vol 13 (1) ◽  
pp. 20-27 ◽  
Author(s):  
Srishty Jindal ◽  
Kamlesh Sharma
Keyword(s):  
Natural Language ◽  
Sentiment Analysis ◽  
Social Networking Sites ◽  
Natural Language Understanding ◽  
Business Analytics ◽  
Language Understanding ◽  
Text Data ◽  
Data Set ◽  
Market Positioning ◽  
Illegal Activities

Background: With the tremendous increase in the use of social networking sites for sharing the emotions, views, preferences etc. a huge volume of data and text is available on the internet, there comes the need for understanding the text and analysing the data to determine the exact intent behind the same for a greater good. This process of understanding the text and data involves loads of analytical methods, several phases and multiple techniques. Efficient use of these techniques is important for an effective and relevant understanding of the text/data. This analysis can in turn be very helpful in ecommerce for targeting audience, social media monitoring for anticipating the foul elements from society and take proactive actions to avoid unethical and illegal activities, business analytics, market positioning etc. Method: The goal is to understand the basic steps involved in analysing the text data which can be helpful in determining sentiments behind them. This review provides detailed description of steps involved in sentiment analysis with the recent research done. Patents related to sentiment analysis and classification are reviewed to throw some light in the work done related to the field. Results: Sentiment analysis determines the polarity behind the text data/review. This analysis helps in increasing the business revenue, e-health, or determining the behaviour of a person. Conclusion: This study helps in understanding the basic steps involved in natural language understanding. At each step there are multiple techniques that can be applied on data. Different classifiers provide variable accuracy depending upon the data set and classification technique used.

scholarly journals Sentiment Analysis Techniques Applied to Raw-Text Data from a Csq-8 Questionnaire about Mindfulness in Times of COVID-19 to Improve Strategy Generation

2021 ◽  
Vol 18 (12) ◽  
pp. 6408
Author(s):  
Mario Jojoa Acosta ◽  
Gema Castillo-Sánchez ◽  
Begonya Garcia-Zapirain ◽  
Isabel de la Torre Díez ◽  
Manuel Franco-Martín
Keyword(s):  
Health Care ◽  
Natural Language Processing ◽  
Natural Language ◽  
Sentiment Analysis ◽  
Transfer Learning ◽  
Language Processing ◽  
Health Care Professionals ◽  
Ground Truth ◽  
Relevant Information ◽  
Free Text

The use of artificial intelligence in health care has grown quickly. In this sense, we present our work related to the application of Natural Language Processing techniques, as a tool to analyze the sentiment perception of users who answered two questions from the CSQ-8 questionnaires with raw Spanish free-text. Their responses are related to mindfulness, which is a novel technique used to control stress and anxiety caused by different factors in daily life. As such, we proposed an online course where this method was applied in order to improve the quality of life of health care professionals in COVID 19 pandemic times. We also carried out an evaluation of the satisfaction level of the participants involved, with a view to establishing strategies to improve future experiences. To automatically perform this task, we used Natural Language Processing (NLP) models such as swivel embedding, neural networks, and transfer learning, so as to classify the inputs into the following three categories: negative, neutral, and positive. Due to the limited amount of data available—86 registers for the first and 68 for the second—transfer learning techniques were required. The length of the text had no limit from the user’s standpoint, and our approach attained a maximum accuracy of 93.02% and 90.53%, respectively, based on ground truth labeled by three experts. Finally, we proposed a complementary analysis, using computer graphic text representation based on word frequency, to help researchers identify relevant information about the opinions with an objective approach to sentiment. The main conclusion drawn from this work is that the application of NLP techniques in small amounts of data using transfer learning is able to obtain enough accuracy in sentiment analysis and text classification stages.

scholarly journals Technology Enhanced Learning Using Humanoid Robots

2021 ◽  
Vol 13 (2) ◽  
pp. 32
Author(s):  
Diego Reforgiato Recupero
Keyword(s):  
Natural Language ◽  
Sentiment Analysis ◽  
Architectural Design ◽  
Humanoid Robots ◽  
Human Robot Interaction ◽  
Technology Enhanced Learning ◽  
Rehabilitation Centers ◽  
Learning Domains ◽  
E Learning ◽  
Analysis Engine

In this paper we present a mixture of technologies tailored for e-learning related to the Deep Learning, Sentiment Analysis, and Semantic Web domains, which we have employed to show four different use cases that we have validated in the field of Human-Robot Interaction. The approach has been designed using Zora, a humanoid robot that can be easily extended with new software behaviors. The goal is to make the robot able to engage users through natural language for different tasks. Using our software the robot can (i) talk to the user and understand their sentiments through a dedicated Semantic Sentiment Analysis engine; (ii) answer to open-dialog natural language utterances by means of a Generative Conversational Agent; (iii) perform action commands leveraging a defined Robot Action ontology and open-dialog natural language utterances; and (iv) detect which objects the user is handing by using convolutional neural networks trained on a huge collection of annotated objects. Each module can be extended with more data and information and the overall architectural design is general, flexible, and scalable and can be expanded with other components, thus enriching the interaction with the human. Different applications within the e-learning domains are foreseen: The robot can either be a trainer and autonomously perform physical actions (e.g., in rehabilitation centers) or it can interact with the users (performing simple tests or even identifying emotions) according to the program developed by the teachers.

scholarly journals A Natural Language Processing Approach to Measuring Treatment Adherence and Consistency Using Semantic Similarity

AERA Open ◽  
2021 ◽  
Vol 7 ◽  
pp. 233285842110286
Author(s):  
Kylie L. Anglin ◽  
Vivian C. Wong ◽  
Arielle Boguslav
Keyword(s):  
Natural Language Processing ◽  
Natural Language ◽  
Semantic Similarity ◽  
Language Processing ◽  
Intervention Implementation ◽  
Proof Of Concept ◽  
Coaching Intervention ◽  
Processing Techniques ◽  
Teacher Coaching ◽  
The Impact

Though there is widespread recognition of the importance of implementation research, evaluators often face intense logistical, budgetary, and methodological challenges in their efforts to assess intervention implementation in the field. This article proposes a set of natural language processing techniques called semantic similarity as an innovative and scalable method of measuring implementation constructs. Semantic similarity methods are an automated approach to quantifying the similarity between texts. By applying semantic similarity to transcripts of intervention sessions, researchers can use the method to determine whether an intervention was delivered with adherence to a structured protocol, and the extent to which an intervention was replicated with consistency across sessions, sites, and studies. This article provides an overview of semantic similarity methods, describes their application within the context of educational evaluations, and provides a proof of concept using an experimental study of the impact of a standardized teacher coaching intervention.

Sign in / Sign up

Export Citation Format

Share Document