scholarly journals Intent Detection and Slot Filling with Capsule Net Architectures for a Romanian Home Assistant

Sensors ◽  
2021 ◽  
Vol 21 (4) ◽  
pp. 1230
Author(s):  
Anda Stoica ◽  
Tibor Kadar ◽  
Camelia Lemnaru ◽  
Rodica Potolea ◽  
Mihaela Dînşoreanu
Keyword(s):  
Neural Network ◽  
Error Analysis ◽  
Natural Language ◽  
Network Model ◽  
Network Architecture ◽  
Natural Language Understanding ◽  
Wide Spread ◽  
Neural Network Architecture ◽  
Language Understanding ◽  
Slot Filling

As virtual home assistants are becoming more popular, there is an emerging need for supporting languages other than English. While more wide-spread or popular languages such as Spanish, French or Hindi are already integrated into existing home assistants like Google Home or Alexa, integration of other less-known languages such as Romanian is still missing. This paper explores the problem of Natural Language Understanding (NLU) applied to a Romanian home assistant. We propose a customized capsule neural network architecture that performs intent detection and slot filling in a joint manner and we evaluate how well it handles utterances containing various levels of complexity. The capsule network model shows a significant improvement in intent detection when compared to models built using the well-known Rasa NLU tool. Through error analysis, we observe clear error patterns that occur systematically. Variability in language when expressing one intent proves to be the biggest challenge encountered by the model.

scholarly journals Sensitivity Direction Learning with Neural Networks Using Domain Knowledge as Soft Shape Constraints

2021 ◽  
Author(s):  
Kazuyuki Wakasugi
Keyword(s):  
Neural Network ◽  
Network Model ◽  
Neural Network Model ◽  
Network Architecture ◽  
Domain Knowledge ◽  
Neural Network Architecture ◽  
Shape Constraints ◽  
Direction Error ◽  
The Neural Network ◽  
Strong Candidate

If domain knowledge can be integrated as an appropriate constraint, it is highly possible that the generalization performance of a neural network model can be improved. We propose Sensitivity Direction Learning (SDL) for learning about the neural network model with user-specified relationships (e.g., monotonicity, convexity) between each input feature and the output of the model by imposing soft shape constraints which represent domain knowledge. To impose soft shape constraints, SDL uses a novel penalty function, Sensitivity Direction Error (SDE) function, which returns the squared error between coefficients of the approximation curve for each Individual Conditional Expectation plot and coefficient constraints which represent domain knowledge. The effectiveness of our concept was verified by simple experiments. Similar to those such as L2 regularization and dropout, SDL and SDE can be used without changing neural network architecture. We believe our algorithm can be a strong candidate for neural network users who want to incorporate domain knowledge.

A NEW NEURAL NETWORK ARCHITECTURE WITH ASSOCIATIVE MEMORY, PRUNING AND ORDER-SENSITIVE LEARNING

1999 ◽  
Vol 09 (04) ◽  
pp. 351-370 ◽  
Author(s):  
M. SREENIVASA RAO ◽  
ARUN K. PUJARI
Keyword(s):  
Neural Network ◽  
Associative Memory ◽  
Network Architecture ◽  
Hopfield Network ◽  
New Paradigm ◽  
Stable States ◽  
Neural Network Architecture ◽  
Language Understanding ◽  
Structure Database ◽  
Learning Technique

A new paradigm of neural network architecture is proposed that works as associative memory along with capabilities of pruning and order-sensitive learning. The network has a composite structure wherein each node of the network is a Hopfield network by itself. The Hopfield network employs an order-sensitive learning technique and converges to user-specified stable states without having any spurious states. This is based on geometrical structure of the network and of the energy function. The network is so designed that it allows pruning in binary order as it progressively carries out associative memory retrieval. The capacity of the network is 2n, where n is the number of basic nodes in the network. The capabilities of the network are demonstrated by experimenting on three different application areas, namely a Library Database, a Protein Structure Database and Natural Language Understanding.

scholarly journals The Singleton Fallacy: Why Current Critiques of Language Models Miss the Point

2021 ◽  
Vol 4 ◽  
Author(s):  
Magnus Sahlgren ◽  
Fredrik Carlsson
Keyword(s):  
Neural Network ◽  
Natural Language ◽  
Computational Linguistics ◽  
Natural Language Understanding ◽  
Language Models ◽  
Current Debate ◽  
Language Understanding ◽  
Original Motivation ◽  
Language Meaning

This paper discusses the current critique against neural network-based Natural Language Understanding solutions known as language models. We argue that much of the current debate revolves around an argumentation error that we refer to as the singleton fallacy: the assumption that a concept (in this case, language, meaning, and understanding) refers to a single and uniform phenomenon, which in the current debate is assumed to be unobtainable by (current) language models. By contrast, we argue that positing some form of (mental) “unobtanium” as definiens for understanding inevitably leads to a dualistic position, and that such a position is precisely the original motivation for developing distributional methods in computational linguistics. As such, we argue that language models present a theoretically (and practically) sound approach that is our current best bet for computers to achieve language understanding. This understanding must however be understood as a computational means to an end.

Vector Symbolic Spiking Neural Network Model of Hippocampal Subarea CA1 Novelty Detection Functionality

2016 ◽  
Vol 28 (4) ◽  
pp. 613-628 ◽  
Author(s):  
Claus Agerskov
Keyword(s):  
Neural Network ◽  
Network Model ◽  
Neural Network Model ◽  
Network Architecture ◽  
Hippocampal Formation ◽  
Novelty Detection ◽  
Behavioral Data ◽  
Long Term Memory ◽  
Spiking Neural Network ◽  
Neural Network Architecture

A neural network model is presented of novelty detection in the CA1 subdomain of the hippocampal formation from the perspective of information flow. This computational model is restricted on several levels by both anatomical information about hippocampal circuitry and behavioral data from studies done in rats. Several studies report that the CA1 area broadcasts a generalized novelty signal in response to changes in the environment. Using the neural engineering framework developed by Eliasmith et al., a spiking neural network architecture is created that is able to compare high-dimensional vectors, symbolizing semantic information, according to the semantic pointer hypothesis. This model then computes the similarity between the vectors, as both direct inputs and a recalled memory from a long-term memory network by performing the dot-product operation in a novelty neural network architecture. The developed CA1 model agrees with available neuroanatomical data, as well as the presented behavioral data, and so it is a biologically realistic model of novelty detection in the hippocampus, which can provide a feasible explanation for experimentally observed dynamics.

scholarly journals Residual Neural Network Model for Detecting Waste Disposing Action in Images

2021 ◽  
Vol 5 (2) ◽  
pp. 52
Author(s):  
I Made Arsa Suyadnya ◽  
Duman Care Khrisne
Keyword(s):  
Neural Network ◽  
Network Model ◽  
Neural Network Model ◽  
Network Architecture ◽  
Input Image ◽  
Scene Recognition ◽  
Learning Approaches ◽  
Neural Network Architecture ◽  
The World ◽  
Disposal Behavior

Waste in general has become a major problem for people around the world. Evidence internationally shows that everyone, or nearly everyone, admits to polluting at some point, with the majority of people littering at least occasionally. This research wants to overcome these problems, by utilizing computer vision and deep learning approaches. This research was conducted to detect the actions carried out by humans in the activities/actions of disposing of waste in an image. This is useful to provide better information for research on better waste disposal behavior than before. We use a Convolutional Neural Network model with a Residual Neural Network architecture to detect the types of activities that objects perform in an image. The result is an artificial neural network model that can label the activities that occur in the input image (scene recognition). This model has been able to carry out the recognition process with an accuracy of 88% with an F1-Score of 0.87.

Sign in / Sign up

Export Citation Format

Share Document