scholarly journals Development of Dialogue Management System for Banking Services

2021 ◽  
Vol 11 (22) ◽  
pp. 10995
Author(s):  
Samir Rustamov ◽  
Aygul Bayramova ◽  
Emin Alasgarov
Keyword(s):  
Neural Networks ◽  
Natural Language ◽  
Conditional Random Fields ◽  
Natural Language Understanding ◽  
Text Messages ◽  
Entity Extraction ◽  
Language Understanding ◽  
User Intention ◽  
Dialogue Management ◽  
Dialogue Manager

Rapid increase in conversational AI and user chat data lead to intensive development of dialogue management systems (DMS) for various industries. Yet, for low-resource languages, such as Azerbaijani, very little research has been conducted. The main purpose of this work is to experiment with various DMS pipeline set-ups to decide on the most appropriate natural language understanding and dialogue manager settings. In our project, we designed and evaluated different DMS pipelines with respect to the conversational text data obtained from one of the leading retail banks in Azerbaijan. In the work, the main two components of DMS—Natural language Understanding (NLU) and Dialogue Manager—have been investigated. In the first step of NLU, we utilized a language identification (LI) component for language detection. We investigated both built-in LI methods such as fastText and custom machine learning (ML) models trained on the domain-based dataset. The second step of the work was a comparison of the classic ML classifiers (logistic regression, neural networks, and SVM) and Dual Intent and Entity Transformer (DIET) architecture for user intention detection. In these experiments we used different combinations of feature extractors such as CountVectorizer, Term Frequency-Inverse Document Frequency (TF-IDF) Vectorizer, and word embeddings for both word and character n-gram based tokens. To extract important information from the text messages, Named Entity Extraction (NER) component was added to the pipeline. The best NER model was chosen among conditional random fields (CRF) tagger, deep neural networks (DNN), models and build in entity extraction component inside DIET architecture. Obtained entity tags fed to the Dialogue Management module as features. All NLU set-ups were followed by the Dialogue Management module that contains a Rule-based Policy to handle FAQs and chitchats as well as a Transformer Embedding Dialogue (TED) Policy to handle more complex and unexpected dialogue inputs. As a result, we suggest a DMS pipeline for a financial assistant, which is capable of identifying intentions, named entities, and a language of text followed by policies that allow generating a proper response (based on the designed dialogues) and suggesting the best next action.

scholarly journals Semi-Supervised Learning of Statistical Models for Natural Language Understanding

2014 ◽  
Vol 2014 ◽  
pp. 1-11 ◽  
Author(s):  
Deyu Zhou ◽  
Yulan He
Keyword(s):  
Natural Language ◽  
Statistical Models ◽  
Conditional Random Fields ◽  
Reduction Rate ◽  
Natural Language Understanding ◽  
Superior Performance ◽  
Support Vector ◽  
Language Understanding ◽  
Learning Framework ◽  
Semantic Annotations

Natural language understanding is to specify a computational model that maps sentences to their semantic mean representation. In this paper, we propose a novel framework to train the statistical models without using expensive fully annotated data. In particular, the input of our framework is a set of sentences labeled with abstract semantic annotations. These annotations encode the underlying embedded semantic structural relations without explicit word/semantic tag alignment. The proposed framework can automatically induce derivation rules that map sentences to their semantic meaning representations. The learning framework is applied on two statistical models, the conditional random fields (CRFs) and the hidden Markov support vector machines (HM-SVMs). Our experimental results on the DARPA communicator data show that both CRFs and HM-SVMs outperform the baseline approach, previously proposed hidden vector state (HVS) model which is also trained on abstract semantic annotations. In addition, the proposed framework shows superior performance than two other baseline approaches, a hybrid framework combining HVS and HM-SVMs and discriminative training of HVS, with a relative error reduction rate of about 25% and 15% being achieved inF-measure.

scholarly journals Exploring and Improving Robustness of Multi Task Deep Neural Networks via Domain Agnostic Defenses

2019 ◽  
Author(s):  
Kashyap Coimbatore Murali
Keyword(s):  
Neural Networks ◽  
Natural Language ◽  
Deep Neural Networks ◽  
Natural Language Understanding ◽  
General Purpose ◽  
Absolute Difference ◽  
Language Understanding ◽  
Spell Checker

In this paper I explore the robustness of the Multi-Task Deep Neural Networks (MT-DNN) againstnon-targeted adversarial attacks across Natural Language Understanding (NLU) tasks as well assome possible ways to defend against them. Liu et al., have shown that the Multi-Task Deep NeuralNetwork, due to the regularization effect produced when training as a result of it’s cross task data, ismore robust than a vanilla BERT model trained only on one task (1.1%-1.5% absolute difference).I then show that although the MT-DNN has generalized better, making it easily transferable acrossdomains and tasks, it can still be compromised as after only 2 attacks (1-character and 2-character)the accuracy drops by 42.05% and 32.24% for the SNLI and SciTail tasks. Finally I propose a domainadaptable defense which restores the model’s accuracy (36.75% and 25.94% respectively) as opposedto a general purpose defense or an off-the-shelf spell checker.

scholarly journals Designing and Implementing Conversational Intelligent Chat-bot Using Natural Language Processing

2021 ◽  
pp. 262-266
Author(s):  
Asoke Nath ◽  
Rupamita Sarkar ◽  
Swastik Mitra ◽  
Rohitaswa Pradhan
Keyword(s):  
Neural Networks ◽  
Natural Language ◽  
Language Processing ◽  
Deep Neural Networks ◽  
Natural Language Understanding ◽  
Natural Language Generation ◽  
Representation Learning ◽  
Language Understanding ◽  
Language Generation ◽  
User Input

In the early days of Artificial Intelligence, it was observed that tasks which humans consider ‘natural’ and ‘commonplace’, such as Natural Language Understanding, Natural Language Generation and Vision were the most difficult task to carry over to computers. Nevertheless, attempts to crack the proverbial NLP nut were made, initially with methods that fall under ‘Symbolic NLP’. One of the products of this era was ELIZA. At present the most promising forays into the world of NLP are provided by ‘Neural NLP’, which uses Representation Learning and Deep Neural networks to model, understand and generate natural language. In the present paper the authors tried to develop a Conversational Intelligent Chatbot, a program that can chat with a user about any conceivable topic, without having domain-specific knowledge programmed into it. This is a challenging task, as it involves both ‘Natural Language Understanding’ (the task of converting natural language user input into representations that a machine can understand) and subsequently ‘Natural Language Generation’ (the task of generating an appropriate response to the user input in natural language). Several approaches exist for building conversational chatbots. In the present paper, two models have been used and their performance has been compared and contrasted. The first model is purely generative and uses a Transformer-based architecture. The second model is retrieval-based, and uses Deep Neural Networks.

Sign in / Sign up

Export Citation Format

Share Document