scholarly journals Neural Stochastic Differential Equations with Neural Processes Family Members for Uncertainty Estimation in Deep Learning

Sensors ◽  
2021 ◽  
Vol 21 (11) ◽  
pp. 3708
Author(s):  
Yongguang Wang ◽  
Shuzhen Yao
Keyword(s):  
Model Uncertainty ◽  
Deep Neural Networks ◽  
Predictive Accuracy ◽  
Uncertainty Estimation ◽  
General Situation ◽  
System Perspective ◽  
One Dimensional ◽  
Differential Equation Models ◽  
Neural Processes ◽  
Classification Tasks

Existing neural stochastic differential equation models, such as SDE-Net, can quantify the uncertainties of deep neural networks (DNNs) from a dynamical system perspective. SDE-Net is either dominated by its drift net with in-distribution (ID) data to achieve good predictive accuracy, or dominated by its diffusion net with out-of-distribution (OOD) data to generate high diffusion for characterizing model uncertainty. However, it does not consider the general situation in a wider field, such as ID data with noise or high missing rates in practice. In order to effectively deal with noisy ID data for credible uncertainty estimation, we propose a vNPs-SDE model, which firstly applies variants of neural processes (NPs) to deal with the noisy ID data, following which the completed ID data can be processed more effectively by SDE-Net. Experimental results show that the proposed vNPs-SDE model can be implemented with convolutional conditional neural processes (ConvCNPs), which have the property of translation equivariance, and can effectively handle the ID data with missing rates for one-dimensional (1D) regression and two-dimensional (2D) image classification tasks. Alternatively, vNPs-SDE can be implemented with conditional neural processes (CNPs) or attentive neural processes (ANPs), which have the property of permutation invariance, and exceeds vanilla SDE-Net in multidimensional regression tasks.

scholarly journals Deep Scattering Spectra with Deep Neural Networks for Acoustic Scene Classification Tasks

2019 ◽  
Vol 28 (6) ◽  
pp. 1177-1183
Author(s):  
Pengyuan Zhang ◽  
Hangting Chen ◽  
Haichuan Bai ◽  
Qingsheng Yuan
Keyword(s):  
Neural Networks ◽  
Deep Neural Networks ◽  
Scene Classification ◽  
Scattering Spectra ◽  
Classification Tasks

scholarly journals Analytical results in transient brush tyre models: theory for large camber angles and classic solutions with limited friction

Meccanica ◽  
2021 ◽  
Author(s):  
Luigi Romano ◽  
Francesco Timpone ◽  
Fredrik Bruzelius ◽  
Bengt Jacobson
Keyword(s):  
Rolling Direction ◽  
State Theory ◽  
General Situation ◽  
Linear Dynamical Systems ◽  
Transient Conditions ◽  
One Dimensional ◽  
Analytical Results ◽  
Classic Theory ◽  
Uniqueness Of The Solution ◽  
The One

AbstractThis paper establishes new analytical results in the mathematical theory of brush tyre models. In the first part, the exact problem which considers large camber angles is analysed from the perspective of linear dynamical systems. Under the assumption of vanishing sliding, the most salient properties of the model are discussed with some insights on concepts as existence and uniqueness of the solution. A comparison against the classic steady-state theory suggests that the latter represents a very good approximation even in case of large camber angles. Furthermore, in respect to the classic theory, the more general situation of limited friction is explored. It is demonstrated that, in transient conditions, exact sliding solutions can be determined for all the one-dimensional problems. For the case of pure lateral slip, the investigation is conducted under the assumption of a strictly concave pressure distribution in the rolling direction.

scholarly journals Explicit Interaction Model towards Text Classification

2019 ◽  
Vol 33 ◽  
pp. 6359-6366 ◽  
Author(s):  
Cunxiao Du ◽  
Zhaozheng Chen ◽  
Fuli Feng ◽  
Lei Zhu ◽  
Tian Gan ◽  
...  
Keyword(s):  
Language Processing ◽  
Text Classification ◽  
Deep Neural Networks ◽  
Interaction Mechanism ◽  
Interaction Model ◽  
Classification Task ◽  
Fine Grained ◽  
Word Level ◽  
Benchmark Datasets ◽  
Classification Tasks

Text classification is one of the fundamental tasks in natural language processing. Recently, deep neural networks have achieved promising performance in the text classification task compared to shallow models. Despite of the significance of deep models, they ignore the fine-grained (matching signals between words and classes) classification clues since their classifications mainly rely on the text-level representations. To address this problem, we introduce the interaction mechanism to incorporate word-level matching signals into the text classification task. In particular, we design a novel framework, EXplicit interAction Model (dubbed as EXAM), equipped with the interaction mechanism. We justified the proposed approach on several benchmark datasets including both multilabel and multi-class text classification tasks. Extensive experimental results demonstrate the superiority of the proposed method. As a byproduct, we have released the codes and parameter settings to facilitate other researches.

scholarly journals Towards Robust Text Classification with Semantics-Aware Recurrent Neural Architecture

2019 ◽  
Vol 1 (2) ◽  
pp. 575-589 ◽  
Author(s):  
Blaž Škrlj ◽  
Jan Kralj ◽  
Nada Lavrač ◽  
Senja Pollak
Keyword(s):  
Text Mining ◽  
Language Processing ◽  
Text Classification ◽  
Deep Neural Networks ◽  
Semantic Knowledge ◽  
Text Documents ◽  
Neural Architecture ◽  
Classification Tasks ◽  
And Gender ◽  
Semantic Resources

Deep neural networks are becoming ubiquitous in text mining and natural language processing, but semantic resources, such as taxonomies and ontologies, are yet to be fully exploited in a deep learning setting. This paper presents an efficient semantic text mining approach, which converts semantic information related to a given set of documents into a set of novel features that are used for learning. The proposed Semantics-aware Recurrent deep Neural Architecture (SRNA) enables the system to learn simultaneously from the semantic vectors and from the raw text documents. We test the effectiveness of the approach on three text classification tasks: news topic categorization, sentiment analysis and gender profiling. The experiments show that the proposed approach outperforms the approach without semantic knowledge, with highest accuracy gain (up to 10%) achieved on short document fragments.

scholarly journals Deep Neural Networks Constrained by Decision Rules

2019 ◽  
Vol 33 ◽  
pp. 2496-2505
Author(s):  
Yuzuru Okajima ◽  
Kunihiko Sadamasa
Keyword(s):  
Neural Networks ◽  
Deep Neural Networks ◽  
Predictive Accuracy ◽  
Decision Rules ◽  
Hybrid Technique ◽  
Complex Data ◽  
Rule Based ◽  
Prior Probabilities ◽  
The Neural Network ◽  
Latent Representations

Deep neural networks achieve high predictive accuracy by learning latent representations of complex data. However, the reasoning behind their decisions is difficult for humans to understand. On the other hand, rule-based approaches are able to justify the decisions by showing the decision rules leading to them, but they have relatively low accuracy. To improve the interpretability of neural networks, several techniques provide post-hoc explanations of decisions made by neural networks, but they cannot guarantee that the decisions are always explained in a simple form like decision rules because their explanations are generated after the decisions are made by neural networks.In this paper, to balance the accuracy of neural networks and the interpretability of decision rules, we propose a hybrid technique called rule-constrained networks, namely, neural networks that make decisions by selecting decision rules from a given ruleset. Because the networks are forced to make decisions based on decision rules, it is guaranteed that every decision is supported by a decision rule. Furthermore, we propose a technique to jointly optimize the neural network and the ruleset from which the network select rules. The log likelihood of correct classifications is maximized under a model with hyper parameters about the ruleset size and the prior probabilities of rules being selected. This feature makes it possible to limit the ruleset size or prioritize human-made rules over automatically acquired rules for promoting the interpretability of the output. Experiments on datasets of time-series and sentiment classification showed rule-constrained networks achieved accuracy as high as that achieved by original neural networks and significantly higher than that achieved by existing rule-based models, while presenting decision rules supporting the decisions.

scholarly journals Inter-Class Angular Loss for Convolutional Neural Networks

2019 ◽  
Vol 33 ◽  
pp. 3894-3901 ◽  
Author(s):  
Le Hui ◽  
Xiang Li ◽  
Chen Gong ◽  
Meng Fang ◽  
Joey Tianyi Zhou ◽  
...  
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Deep Neural Networks ◽  
Learning Difficulties ◽  
Feature Space ◽  
Superior Performance ◽  
Strongly Correlated ◽  
Discriminative Ability ◽  
Practical Applications ◽  
Classification Tasks

Convolutional Neural Networks (CNNs) have shown great power in various classification tasks and have achieved remarkable results in practical applications. However, the distinct learning difficulties in discriminating different pairs of classes are largely ignored by the existing networks. For instance, in CIFAR-10 dataset, distinguishing cats from dogs is usually harder than distinguishing horses from ships. By carefully studying the behavior of CNN models in the training process, we observe that the confusion level of two classes is strongly correlated with their angular separability in the feature space. That is, the larger the inter-class angle is, the lower the confusion will be. Based on this observation, we propose a novel loss function dubbed “Inter-Class Angular Loss” (ICAL), which explicitly models the class correlation and can be directly applied to many existing deep networks. By minimizing the proposed ICAL, the networks can effectively discriminate the examples in similar classes by enlarging the angle between their corresponding class vectors. Thorough experimental results on a series of vision and nonvision datasets confirm that ICAL critically improves the discriminative ability of various representative deep neural networks and generates superior performance to the original networks with conventional softmax loss.

Sign in / Sign up

Export Citation Format

Share Document