scholarly journals Sensitivity Direction Learning with Neural Networks Using Domain Knowledge as Soft Shape Constraints

2021 ◽  
Author(s):  
Kazuyuki Wakasugi
Keyword(s):  
Neural Network ◽  
Network Model ◽  
Neural Network Model ◽  
Network Architecture ◽  
Domain Knowledge ◽  
Neural Network Architecture ◽  
Shape Constraints ◽  
Direction Error ◽  
The Neural Network ◽  
Strong Candidate

If domain knowledge can be integrated as an appropriate constraint, it is highly possible that the generalization performance of a neural network model can be improved. We propose Sensitivity Direction Learning (SDL) for learning about the neural network model with user-specified relationships (e.g., monotonicity, convexity) between each input feature and the output of the model by imposing soft shape constraints which represent domain knowledge. To impose soft shape constraints, SDL uses a novel penalty function, Sensitivity Direction Error (SDE) function, which returns the squared error between coefficients of the approximation curve for each Individual Conditional Expectation plot and coefficient constraints which represent domain knowledge. The effectiveness of our concept was verified by simple experiments. Similar to those such as L2 regularization and dropout, SDL and SDE can be used without changing neural network architecture. We believe our algorithm can be a strong candidate for neural network users who want to incorporate domain knowledge.

Fracture Design Parameters of Middle Carbon Steel in Extra-Low Cycle Bend Torsion Loading

2014 ◽  
Vol 574 ◽  
pp. 342-346
Author(s):  
Hong Yan Duan ◽  
Huan Rong Zhang ◽  
Ming Zheng ◽  
Xiao Hong Wang
Keyword(s):  
Neural Network ◽  
Carbon Steel ◽  
Network Model ◽  
Neural Network Model ◽  
Network Architecture ◽  
Training Data ◽  
Design Parameters ◽  
Ann Model ◽  
Approximate Methods ◽  
The Neural Network

The fracture problems of medium carbon steel under extra-low cycle bend torsion fatigue loading were studied using artificial neural networks (ANN) in this paper. The ANN model exhibited excellent comparison with the experimental results. It was concluded that predicted fracture design parameters by the trained neural network model seem more reasonable compared to approximate methods. It is possible to claim that, ANN is fairly promising prediction technique if properly used. Training ANN model was introduced at first. And then the Training data for the development of the neural network model was obtained from the experiments. The input parameters, the presetting deflection and notch open angle, and the output, the cycle times of fracture were used during the network training. The neural network architecture is designed. The ANN model was developed using back propagation architecture with three layers jump connections, where every layer was connected or linked to every previous layer. The number of hidden neurons was determined according to special formula. The performance of system is summarized at last. In order to facilitate the comparisons of predicted values, the error evaluation and mean relative error are obtained. The result show that the training model has good performance, and the experimental data and predicted data from ANN are in good coherence.

Application of ANN Back-Propagation for Fracture Design Parameters of Middle Carbon Steel in Extra-Low Cycle Bend Torsion Loading

2013 ◽  
Vol 345 ◽  
pp. 272-276 ◽  
Author(s):  
Hong Yan Duan ◽  
You Tang Li ◽  
Zhi Jia Sun ◽  
Yang Yang Zhang
Keyword(s):  
Neural Network ◽  
Carbon Steel ◽  
Network Model ◽  
Neural Network Model ◽  
Network Architecture ◽  
Back Propagation ◽  
Training Data ◽  
Design Parameters ◽  
Ann Model ◽  
The Neural Network

The fracture problems of medium carbon steel (MCS) under extra-low cycle bend torsion loading were studied using artificial neural networks (ANN) in this paper. The training data were used in the formation of training set of ANN. The ANN model exhibited excellent comparison with the experimental results. It was concluded that predicted fracture design parameters by the trained neural network model seem more reasonable compared to approximate methods. It is possible to claim that, ANN is fairly promising prediction technique if properly used. Training ANN model was introduced at first. And then the Training data for the development of the neural network model was obtained from the experiments. The input parameters, notch depth and tip radius of the notch, and the output, the cycle times of fracture were used during the network training. The neural network architecture is designed. The ANN model was developed using back propagation architecture with three layers jump connections, where every layer was connected or linked to every previous layer. The number of hidden neurons was determined according to special formula. The performance of system is summarized at last. In order to facilitate the comparisons of predicted values, the error evaluation and mean relative error are obtained. The result show that the training model has good performance, and the experimental data and predicted data from ANN are in good coherence.

Vector Symbolic Spiking Neural Network Model of Hippocampal Subarea CA1 Novelty Detection Functionality

2016 ◽  
Vol 28 (4) ◽  
pp. 613-628 ◽  
Author(s):  
Claus Agerskov
Keyword(s):  
Neural Network ◽  
Network Model ◽  
Neural Network Model ◽  
Network Architecture ◽  
Hippocampal Formation ◽  
Novelty Detection ◽  
Behavioral Data ◽  
Long Term Memory ◽  
Spiking Neural Network ◽  
Neural Network Architecture

A neural network model is presented of novelty detection in the CA1 subdomain of the hippocampal formation from the perspective of information flow. This computational model is restricted on several levels by both anatomical information about hippocampal circuitry and behavioral data from studies done in rats. Several studies report that the CA1 area broadcasts a generalized novelty signal in response to changes in the environment. Using the neural engineering framework developed by Eliasmith et al., a spiking neural network architecture is created that is able to compare high-dimensional vectors, symbolizing semantic information, according to the semantic pointer hypothesis. This model then computes the similarity between the vectors, as both direct inputs and a recalled memory from a long-term memory network by performing the dot-product operation in a novelty neural network architecture. The developed CA1 model agrees with available neuroanatomical data, as well as the presented behavioral data, and so it is a biologically realistic model of novelty detection in the hippocampus, which can provide a feasible explanation for experimentally observed dynamics.

scholarly journals Residual Neural Network Model for Detecting Waste Disposing Action in Images

2021 ◽  
Vol 5 (2) ◽  
pp. 52
Author(s):  
I Made Arsa Suyadnya ◽  
Duman Care Khrisne
Keyword(s):  
Neural Network ◽  
Network Model ◽  
Neural Network Model ◽  
Network Architecture ◽  
Input Image ◽  
Scene Recognition ◽  
Learning Approaches ◽  
Neural Network Architecture ◽  
The World ◽  
Disposal Behavior

Waste in general has become a major problem for people around the world. Evidence internationally shows that everyone, or nearly everyone, admits to polluting at some point, with the majority of people littering at least occasionally. This research wants to overcome these problems, by utilizing computer vision and deep learning approaches. This research was conducted to detect the actions carried out by humans in the activities/actions of disposing of waste in an image. This is useful to provide better information for research on better waste disposal behavior than before. We use a Convolutional Neural Network model with a Residual Neural Network architecture to detect the types of activities that objects perform in an image. The result is an artificial neural network model that can label the activities that occur in the input image (scene recognition). This model has been able to carry out the recognition process with an accuracy of 88% with an F1-Score of 0.87.

scholarly journals Using Models of Collective Neural Networks for Classification of the Input Data Applying Simple Voting

2018 ◽  
pp. 333-339
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Network Model ◽  
Neural Network Model ◽  
Network Architecture ◽  
Input Data ◽  
Bootstrap Method ◽  
Classification Models ◽  
The Neural Network ◽  
Voting Method

This paper deals with the use of neural networks in binary classification problems based on the simple voting method. It specifies that the accuracy of the neural network classification depends both on the choice of the network architecture and on the partitioning of data into training and test sets. It is noted that the process of building a neural network model is probabilistic in nature. To eliminate this drawback and improve the accuracy of classification, the need to combine several models in the form of a collective of neural networks is actualized. To build such a model, it is proposed to use the 0.632-bootstrap method. To aggregate individual solutions formed at the output of each neural network, it is proposed to use a single-choice simple voting. The choice of the model structure in the form of a single-layer Perceptron is justified, and its mathematical model is presented. Using the evaluation data of the functional state of a drunk human as an example, the results of an experimental assessment of the bootstrap error and the accuracy of the neural network model are presented. It is concluded that it is possible to achieve a higher accuracy of classification based on the neural network model when aggregating the results of all bootstrap models using the simple voting method. The accuracy of the constructed model is compared with the accuracy of other classification models. The accuracy of the constructed model was 96.7%, which on average exceeded the accuracy of other classification models by 6.6%. Thus, the neural network collective model is an effective tool for classifying input data using the simple voting method.

A NEURAL NETWORK MODEL FOR MINIMUM SPANNING CIRCLE: ITS CONVERGENCE, ARCHITECTURE DESIGN AND APPLICATIONS

2002 ◽  
Vol 16 (07) ◽  
pp. 797-815
Author(s):  
AMITAVA DATTA ◽  
S. K. PARUI
Keyword(s):  
Neural Network ◽  
Network Model ◽  
Neural Network Model ◽  
Network Architecture ◽  
Parallel Implementation ◽  
Higher Dimensions ◽  
Neural Network Architecture ◽  
Rigorous Analysis ◽  
Proposed Model ◽  
Finite Set

A self-organizing neural network model that computes the smallest circle (also called minimum spanning circle) enclosing a finite set of given points was proposed by Datta.3 In the article,3 the algorithm is stated and it is demonstrated by simulation that the center of the smallest circle can be achieved with a given level of accuracy. No rigorous proof was given in support of the simulation results. In this paper, we make a rigorous analysis of the model and mathematically prove that the model converges to the desired center of the minimum spanning circle. A suitable neural network architecture is also designed for parallel implementation of the proposed model. Time complexity of the algorithm is worked out under the proposed architecture. Extension of the proposed model to higher dimensions is discussed and demonstrated with some applications.

SCORING MODELING BASED ON NEURAL NETWORKS FOR DETERMINING A BANK BORROWER'S RATING

2020 ◽  
Vol 2020 (10) ◽  
pp. 54-62
Author(s):  
Oleksii VASYLIEV ◽  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Network Architecture ◽  
Statistical Data ◽  
Activation Function ◽  
Decision Making Process ◽  
Neural Network Architecture ◽  
Acceptable Accuracy ◽  
The Neural Network ◽  
Sigmoid Activation Function

The problem of applying neural networks to calculate ratings used in banking in the decision-making process on granting or not granting loans to borrowers is considered. The task is to determine the rating function of the borrower based on a set of statistical data on the effectiveness of loans provided by the bank. When constructing a regression model to calculate the rating function, it is necessary to know its general form. If so, the task is to calculate the parameters that are included in the expression for the rating function. In contrast to this approach, in the case of using neural networks, there is no need to specify the general form for the rating function. Instead, certain neural network architecture is chosen and parameters are calculated for it on the basis of statistical data. Importantly, the same neural network architecture can be used to process different sets of statistical data. The disadvantages of using neural networks include the need to calculate a large number of parameters. There is also no universal algorithm that would determine the optimal neural network architecture. As an example of the use of neural networks to determine the borrower's rating, a model system is considered, in which the borrower's rating is determined by a known non-analytical rating function. A neural network with two inner layers, which contain, respectively, three and two neurons and have a sigmoid activation function, is used for modeling. It is shown that the use of the neural network allows restoring the borrower's rating function with quite acceptable accuracy.

scholarly journals Intent Detection and Slot Filling with Capsule Net Architectures for a Romanian Home Assistant

Sensors ◽  
2021 ◽  
Vol 21 (4) ◽  
pp. 1230
Author(s):  
Anda Stoica ◽  
Tibor Kadar ◽  
Camelia Lemnaru ◽  
Rodica Potolea ◽  
Mihaela Dînşoreanu
Keyword(s):  
Neural Network ◽  
Error Analysis ◽  
Natural Language ◽  
Network Model ◽  
Network Architecture ◽  
Natural Language Understanding ◽  
Wide Spread ◽  
Neural Network Architecture ◽  
Language Understanding ◽  
Slot Filling

As virtual home assistants are becoming more popular, there is an emerging need for supporting languages other than English. While more wide-spread or popular languages such as Spanish, French or Hindi are already integrated into existing home assistants like Google Home or Alexa, integration of other less-known languages such as Romanian is still missing. This paper explores the problem of Natural Language Understanding (NLU) applied to a Romanian home assistant. We propose a customized capsule neural network architecture that performs intent detection and slot filling in a joint manner and we evaluate how well it handles utterances containing various levels of complexity. The capsule network model shows a significant improvement in intent detection when compared to models built using the well-known Rasa NLU tool. Through error analysis, we observe clear error patterns that occur systematically. Variability in language when expressing one intent proves to be the biggest challenge encountered by the model.

Estimating Traffic Volume on Minor Roads at Rural Stop-Controlled Intersections using Deep Learning

2019 ◽  
Vol 2673 (4) ◽  
pp. 108-116 ◽  
Author(s):  
Mostafa H. Tawfeek ◽  
Karim El-Basyouny
Keyword(s):  
Neural Network ◽  
Linear Regression ◽  
Network Model ◽  
Neural Network Model ◽  
Deep Neural Network ◽  
Safety Management ◽  
Estimation Method ◽  
Estimation Model ◽  
The Neural Network ◽  
Annual Average Daily Traffic

Safety Performance Functions (SPFs) are regression models used to predict the expected number of collisions as a function of various traffic and geometric characteristics. One of the integral components in developing SPFs is the availability of accurate exposure factors, that is, annual average daily traffic (AADT). However, AADTs are not often available for minor roads at rural intersections. This study aims to develop a robust AADT estimation model using a deep neural network. A total of 1,350 rural four-legged, stop-controlled intersections from the Province of Alberta, Canada, were used to train the neural network. The results of the deep neural network model were compared with the traditional estimation method, which uses linear regression. The results indicated that the deep neural network model improved the estimation of minor roads’ AADT by 35% when compared with the traditional method. Furthermore, SPFs developed using linear regression resulted in models with statistically insignificant AADTs on minor roads. Conversely, the SPF developed using the neural network provided a better fit to the data with both AADTs on minor and major roads being statistically significant variables. The findings indicated that the proposed model could enhance the predictive power of the SPF and therefore improve the decision-making process since SPFs are used in all parts of the safety management process.

Sign in / Sign up

Export Citation Format

Share Document