Equation-based domain knowledge utilization into neural network structure and learning

2018 ◽  
Vol 232 (10) ◽  
pp. 1275-1291 ◽  
Author(s):  
Shrinivas Kulkarni ◽  
Anirban Guha
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Physical Interpretation ◽  
Domain Knowledge ◽  
Feed Forward Neural Network ◽  
Regression Problem ◽  
Industrial Systems ◽  
Domain Experts ◽  
Black Boxes ◽  
Neural Network Structure

The use of neural networks as black boxes, though useful for modeling complicated industrial systems, has some limitations. No physical interpretation can be given to sections of the trained network. Incorporation of domain knowledge into neural network attempts to address this lacuna. Most of the attempts in this direction have been in the area of data classification in which sub-classes created with the help of domain experts have led to better neural networks. This work attempts to incorporate domain knowledge into the structure of a neural network for solving a regression problem—that of a piston pump leakage prediction. It shows a way in which prior knowledge about subsystems, in the form of equations, can be used to create a neural network for modeling the entire system. This approach significantly outperforms a traditional feed forward neural network. As a key contribution, this approach allows physical interpretation of the neurons which can aid in troubleshooting and anomaly detection.

HIERARCHICAL GENETIC ALGORITHM FOR NEAR-OPTIMAL FEEDFORWARD NEURAL NETWORK DESIGN

2002 ◽  
Vol 12 (01) ◽  
pp. 31-43 ◽  
Author(s):  
GARY YEN ◽  
HAIMING LU
Keyword(s):  
Neural Network ◽  
Genetic Algorithm ◽  
Neural Networks ◽  
Cost Function ◽  
Design Procedure ◽  
Optimal Solution ◽  
Traditional Learning ◽  
Feed Forward Neural Network ◽  
Optimal Solution Set ◽  
Hierarchical Genetic Algorithm

In this paper, we propose a genetic algorithm based design procedure for a multi-layer feed-forward neural network. A hierarchical genetic algorithm is used to evolve both the neural network's topology and weighting parameters. Compared with traditional genetic algorithm based designs for neural networks, the hierarchical approach addresses several deficiencies, including a feasibility check highlighted in literature. A multi-objective cost function is used herein to optimize the performance and topology of the evolved neural network simultaneously. In the prediction of Mackey–Glass chaotic time series, the networks designed by the proposed approach prove to be competitive, or even superior, to traditional learning algorithms for the multi-layer Perceptron networks and radial-basis function networks. Based upon the chosen cost function, a linear weight combination decision-making approach has been applied to derive an approximated Pareto-optimal solution set. Therefore, designing a set of neural networks can be considered as solving a two-objective optimization problem.

scholarly journals Part-of-Speech Tagging via Deep Neural Networks for Northern-Ethiopic Languages

2020 ◽  
Vol 49 (4) ◽  
pp. 482-494
Author(s):  
Jurgita Kapočiūtė-Dzikienė ◽  
Senait Gebremichael Tesfagergish
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Language Processing ◽  
Deep Neural Networks ◽  
Short Term Memory ◽  
Parameter Tuning ◽  
Feed Forward Neural Network ◽  
Pos Tagging ◽  
Part Of Speech ◽  
Pos Tagger

Deep Neural Networks (DNNs) have proven to be especially successful in the area of Natural Language Processing (NLP) and Part-Of-Speech (POS) tagging—which is the process of mapping words to their corresponding POS labels depending on the context. Despite recent development of language technologies, low-resourced languages (such as an East African Tigrinya language), have received too little attention. We investigate the effectiveness of Deep Learning (DL) solutions for the low-resourced Tigrinya language of the Northern-Ethiopic branch. We have selected Tigrinya as the testbed example and have tested state-of-the-art DL approaches seeking to build the most accurate POS tagger. We have evaluated DNN classifiers (Feed Forward Neural Network – FFNN, Long Short-Term Memory method – LSTM, Bidirectional LSTM, and Convolutional Neural Network – CNN) on a top of neural word2vec word embeddings with a small training corpus known as Nagaoka Tigrinya Corpus. To determine the best DNN classifier type, its architecture and hyper-parameter set both manual and automatic hyper-parameter tuning has been performed. BiLSTM method was proved to be the most suitable for our solving task: it achieved the highest accuracy equal to 92% that is 65% above the random baseline.

scholarly journals Adaptation of the Diagnostic Artificial Neural Network Structure When New Training Examples Appear

2020 ◽  
Vol 6 (4) ◽  
pp. 120-126
Author(s):  
A. Malikov
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Artificial Neural Network ◽  
Information Security ◽  
Classification Problem ◽  
Data Set ◽  
Neural Network Structure ◽  
Artificial Neural ◽  
Training Examples ◽  
Information Image

In this paper we can see that identified computer incidents are subject for diagnostics, during which the characteristics of information security violations are clarified (purpose, causes, consequences, etc.). To diagnose computer incidents, we can use methods of automation while collection and processing the events that occur as a result of the implementation of scenarios for information security violations. Artificial neural networks can be used to solve the classification problem of assigning diagnostic data set (information image of a computer incident) to one of the possible values of the violation characteristic. The purpose of this work is to adapt the structure of an artificial neural network that allows the accuracy diagnostics of computer incidents when new training examples appear.

scholarly journals Forecast Model of TV Show Rating Based on Convolutional Neural Network

Complexity ◽  
2021 ◽  
Vol 2021 ◽  
pp. 1-10
Author(s):  
Lingfeng Wang
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Prediction Model ◽  
Convolutional Neural Network ◽  
Network Structure ◽  
Rating Data ◽  
Tv Program ◽  
Neural Network Structure ◽  
Rating Prediction ◽  
Tv Show

The TV show rating analysis and prediction system can collect and transmit information more quickly and quickly upload the information to the database. The convolutional neural network is a multilayer neural network structure that simulates the operating mechanism of biological vision systems. It is a neural network composed of multiple convolutional layers and downsampling layers sequentially connected. It can obtain useful feature descriptions from original data and is an effective method to extract features from data. At present, convolutional neural networks have become a research hotspot in speech recognition, image recognition and classification, natural language processing, and other fields and have been widely and successfully applied in these fields. Therefore, this paper introduces the convolutional neural network structure to predict the TV program rating data. First, it briefly introduces artificial neural networks and deep learning methods and focuses on the algorithm principles of convolutional neural networks and support vector machines. Then, we improve the convolutional neural network to fit the TV program rating data and finally apply the two prediction models to the TV program rating data prediction. We improve the convolutional neural network TV program rating prediction model and combine the advantages of the convolutional neural network to extract effective features and good classification and prediction capabilities to improve the prediction accuracy. Through simulation comparison, we verify the feasibility and effectiveness of the TV program rating prediction model given in this article.

scholarly journals ELMAN-RECURRENT NEURAL NETWORK FOR LOAD SHEDDING OPTIMIZATION

SINERGI ◽  
2020 ◽  
Vol 24 (1) ◽  
pp. 29
Author(s):  
Widi Aribowo
Keyword(s):  
Neural Network ◽  
Genetic Algorithm ◽  
Neural Networks ◽  
Power System ◽  
Recurrent Neural Networks ◽  
Load Shedding ◽  
Feed Forward Neural Network ◽  
Linear Problems ◽  
Elman Recurrent Neural Network ◽  
And Control

Load shedding plays a key part in the avoidance of the power system outage. The frequency and voltage fluidity leads to the spread of a power system into sub-systems and leads to the outage as well as the severe breakdown of the system utility.  In recent years, Neural networks have been very victorious in several signal processing and control applications.  Recurrent Neural networks are capable of handling complex and non-linear problems. This paper provides an algorithm for load shedding using ELMAN Recurrent Neural Networks (RNN). Elman has proposed a partially RNN, where the feedforward connections are modifiable and the recurrent connections are fixed. The research is implemented in MATLAB and the performance is tested with a 6 bus system. The results are compared with the Genetic Algorithm (GA), Combining Genetic Algorithm with Feed Forward Neural Network (hybrid) and RNN. The proposed method is capable of assigning load releases needed and more efficient than other methods. 

scholarly journals Eigen Solution of Neural Networks and Its Application in Prediction and Analysis of Controller Parameters of Grinding Robot in Complex Environments

Complexity ◽  
2019 ◽  
Vol 2019 ◽  
pp. 1-21 ◽  
Author(s):  
Shixi Tang ◽  
Jinan Gu ◽  
Keming Tang ◽  
Wei Ding ◽  
Zhengyang Shang
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Prediction Model ◽  
Deep Neural Networks ◽  
Complete Solution ◽  
Partial Solution ◽  
Complex Environments ◽  
Solution Theory ◽  
Analysis Model ◽  
Neural Network Structure

The robot dynamic model is often rarely known due to various uncertainties such as parametric uncertainties or modeling errors existing in complex environments. It is a key problem to find the relationship between the changes of neural network structure and the changes of input and output environments and their mutual influences. Firstly, this paper defined the conceptions of neural network solution, neural network eigen solution, neural network complete solution, and neural network partial solution and the conceptions of input environments, output environments, and macrostructure of neural networks. Secondly, an eigen solution theory of general neural networks was proposed and proven including consistent approximation theorem, eigen solution existence theorem, consistency theorem of complete solution, the partial solution, and none solution theorem of neural networks. Lastly, to verify the eigen solution theory of neural networks, the proposed theory was applied to a novel prediction and analysis model of controller parameters of grinding robot in complex environments with deep neural networks and then build prediction model with deep learning neural networks for controller parameters of grinding robot. The morphological subfeature graph with multimoment was constructed to describe the block surface morphology using rugosity, standard deviation, skewness, and kurtosis. The results of theoretical analysis and experimental test show that the output traits have an optional effect with joint action. When the input features functioning in prediction increase, higher predicted accuracy can be obtained. And when the output traits involving in prediction increase, more output traits can be predicted. The proposed prediction and analysis model with deep neural networks can be used to find and predict the inherent laws of the data. Compared with the traditional prediction model, the proposed model can predict output features simultaneously and is more stable.

Contributions to Predict the Malfunction Probability of the Human-Machine-Environment System, Using Artificial Neural Networks

2015 ◽  
Vol 760 ◽  
pp. 771-776
Author(s):  
Daniel Constantin Anghel ◽  
Nadia Belu
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Artificial Neural Networks ◽  
Design Experiment ◽  
Manufacturing Processes ◽  
Feed Forward Neural Network ◽  
Feed Forward ◽  
Linear Relationships ◽  
The Neural Network ◽  
Artificial Neural

This paper presents the application of Artificial Neural Networks to predict the malfunction probability of the human-machine-environment system, in order to provide some guidance to designers of manufacturing processes. Artificial Neural Networks excel in gathering difficult non-linear relationships between the inputs and outputs of a system. We used, in this work, a feed forward neural network in order to predict the malfunction probability. The neural network is simulated with Matlab. The design experiment presented in this paper was realized at University of Pitesti, at the Faculty of Mechanics and Technology, Technology and Management Department.

Missing Data Approaches to Classification

2010 ◽  
pp. 187-209
Author(s):  
Tshilidzi Marwala
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Missing Data ◽  
Network Model ◽  
Network Models ◽  
Feed Forward Neural Network ◽  
Neural Network Models ◽  
Feed Forward ◽  
Missing Data Problem ◽  
Feed Forward Neural Networks

In this chapter, a classifier technique that is based on a missing data estimation framework that uses autoassociative multi-layer perceptron neural networks and genetic algorithms is proposed. The proposed method is tested on a set of demographic properties of individuals obtained from the South African antenatal survey and compared to conventional feed-forward neural networks. The missing data approach based on the autoassociative network model proposed gives an accuracy of 92%, when compared to the accuracy of 84% obtained from the conventional feed-forward neural network models. The area under the receiver operating characteristics curve for the proposed autoassociative network model is 0.86 compared to 0.80 for the conventional feed-forward neural network model. The autoassociative network model proposed in this chapter, therefore, outperforms the conventional feed-forward neural network models and is an improved classifier. The reasons for this are: (1) the propagation of errors in the autoassociative network model is more distributed while for a conventional feed-forward network is more concentrated; and (2) there is no causality between the demographic properties and the HIV and, therefore, the HIV status does change the demographic properties and vice versa. Therefore, it is better to treat the problem as a missing data problem rather than a feed-forward problem.

scholarly journals Multiple Source Localization in a Shallow Water Waveguide Exploiting Subarray Beamforming and Deep Neural Networks

Sensors ◽  
2019 ◽  
Vol 19 (21) ◽  
pp. 4768 ◽  
Author(s):  
Zhaoqiong Huang ◽  
Ji Xu ◽  
Zaixiao Gong ◽  
Haibin Wang ◽  
Yonghong Yan
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Shallow Water ◽  
Source Localization ◽  
Deep Neural Networks ◽  
Short Term Memory ◽  
Direction Finding ◽  
Multiple Source ◽  
Feed Forward Neural Network ◽  
Subarray Beamforming

Deep neural networks (DNNs) have been shown to be effective for single sound source localization in shallow water environments. However, multiple source localization is a more challenging task because of the interactions among multiple acoustic signals. This paper proposes a framework for multiple source localization on underwater horizontal arrays using deep neural networks. The two-stage DNNs are adopted to determine both the directions and ranges of multiple sources successively. A feed-forward neural network is trained for direction finding, while the long short term memory recurrent neural network is used for source ranging. Particularly, in the source ranging stage, we perform subarray beamforming to extract features of sources that are detected by the direction finding stage, because subarray beamforming can enhance the mixed signal to the desired direction while preserving the horizontal-longitudinal correlations of the acoustic field. In this way, a universal model trained in the single-source scenario can be applied to multi-source scenarios with arbitrary numbers of sources. Both simulations and experiments in a range-independent shallow water environment of SWellEx-96 Event S5 are given to demonstrate the effectiveness of the proposed method.

scholarly journals Verifiable and Interpretable Reinforcement Learning through Program Synthesis

2019 ◽  
Vol 33 ◽  
pp. 9902-9903
Author(s):  
Abhinav Verma
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Neural Networks ◽  
Reinforcement Learning ◽  
Programming Languages ◽  
Formal Methods ◽  
Domain Knowledge ◽  
Policy Search ◽  
Safety Critical ◽  
Symbolic Methods

We study the problem of generating interpretable and verifiable policies for Reinforcement Learning (RL). Unlike the popular Deep Reinforcement Learning (DRL) paradigm, in which the policy is represented by a neural network, the aim of this work is to find policies that can be represented in highlevel programming languages. Such programmatic policies have several benefits, including being more easily interpreted than neural networks, and being amenable to verification by scalable symbolic methods. The generation methods for programmatic policies also provide a mechanism for systematically using domain knowledge for guiding the policy search. The interpretability and verifiability of these policies provides the opportunity to deploy RL based solutions in safety critical environments. This thesis draws on, and extends, work from both the machine learning and formal methods communities.

Sign in / Sign up

Export Citation Format

Share Document