scholarly journals Deeper Connections between Neural Networks and Gaussian Processes Speed-up Active Learning

2019 ◽  
Author(s):  
Evgenii Tsymbalov ◽  
Sergei Makarychev ◽  
Alexander Shapeev ◽  
Maxim Panov
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Active Learning ◽  
Gaussian Processes ◽  
Large Scale ◽  
Training Sample ◽  
Real World Data ◽  
The Neural Network ◽  
Uncertainty Estimates ◽  
Out Of Sample

Active learning methods for neural networks are usually based on greedy criteria, which ultimately give a single new design point for the evaluation. Such an approach requires either some heuristics to sample a batch of design points at one active learning iteration, or retraining the neural network after adding each data point, which is computationally inefficient. Moreover, uncertainty estimates for neural networks sometimes are overconfident for the points lying far from the training sample. In this work, we propose to approximate Bayesian neural networks (BNN) by Gaussian processes (GP), which allows us to update the uncertainty estimates of predictions efficiently without retraining the neural network while avoiding overconfident uncertainty prediction for out-of-sample points. In a series of experiments on real-world data, including large-scale problems of chemical and physical modeling, we show the superiority of the proposed approach over the state-of-the-art methods.

scholarly journals Prediction of e-Learning Efficiency by Neural Networks

2012 ◽  
Vol 12 (2) ◽  
pp. 98-108 ◽  
Author(s):  
Petar Halachev
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Balanced Scorecard ◽  
Small Sample Size ◽  
Principal Component ◽  
Real Data ◽  
Training Sample ◽  
Small Sample ◽  
The Neural Network ◽  
E Learning

Abstract A model for prediction of the outcome indicators of e-Learning, based on Balanced ScoreCard (BSC) by Neural Networks (NN) is proposed. In the development of NN models the problem of a small sample size of the data arises. In order to reduce the number of variables and increase the examples of the training sample, preprocessing of the data with the help of the methods Interpolation and Principal Component Analysis (PCA) is performed. A method for optimizing the structure of the neural network is applied over linear and nonlinear neural network architectures. The highest accuracy of prognosis is obtained applying the method of Optimal Brain Damage (OBD) over the nonlinear neural network. The efficiency and applicability of the method suggested is proved by numerical experiments on the basis of real data.

A Hybrid Model of FLANN and Firefly Algorithm for Classification

2016 ◽  
pp. 491-522 ◽  
Author(s):  
Bighnaraj Naik ◽  
Janmenjoy Nayak ◽  
H. S. Behera
Keyword(s):  
Neural Network ◽  
Large Scale ◽  
Firefly Algorithm ◽  
Statistical Significance ◽  
Optimization Techniques ◽  
Real World Data ◽  
Biologically Inspired ◽  
Functional Link ◽  
The Neural Network ◽  
Global Optima

Since last decade, biologically inspired optimization techniques have been a keen interest among the researchers of optimization community. Some of the well developed and advanced popular algorithms such as GA, PSO etc. are found to be performing well for solving large scale problems. In this chapter, a recently developed nature inspired firefly algorithm has been proposed by the combination of an efficient higher order functional link neural network for the classification of the real world data. The main advantage of firefly algorithm is to obtain the solutions for global optima, where some of the earlier developed swarm intelligence algorithms fail to do so. For learning the neural network, efficient gradient descent learning is used to optimize the weights. The proposed method is able to classify the non-linear data more efficiently with less error rate. Under null-hypothesis, the proposed method has been tested with various statistical methods to prove its statistical significance.

The Effect of Training Set Distributions for Supervised Learning Artificial Neural Networks on Classification Accuracy

2011 ◽  
pp. 93-108
Author(s):  
Steven Walczak ◽  
Irena Yegorova ◽  
Bruce H. Andrews
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Classification Accuracy ◽  
Network Performance ◽  
Population Distribution ◽  
Learning Algorithm ◽  
Density Functions ◽  
The Neural Network ◽  
Out Of Sample ◽  
Training Sets

Neural networks have been repeatedly shown to outperform traditional statistical modeling techniques for both discriminant analysis and forecasting. While questions regarding the effects of architecture, input variable selection, learning algorithm, and size of training sets on the neural network model’s performance have been addressed, very little attention has been focused on distribution effects of training and out-of-sample populations on neural network performance. This article examines the effect of changing the population distribution within training sets for estimated distribution density functions, in particular for a credit risk assessment problem.

scholarly journals Extracting automata from neural networks using active learning

2021 ◽  
Vol 7 ◽  
pp. e436
Author(s):  
Zhiwu Xu ◽  
Cheng Wen ◽  
Shengchao Qin ◽  
Mengda He
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Deep Learning ◽  
Active Learning ◽  
Block Size ◽  
Learning Framework ◽  
Passive Learning ◽  
The Neural Network ◽  
Equivalence Queries ◽  
Neural Network Classifiers

Deep learning is one of the most advanced forms of machine learning. Most modern deep learning models are based on an artificial neural network, and benchmarking studies reveal that neural networks have produced results comparable to and in some cases superior to human experts. However, the generated neural networks are typically regarded as incomprehensible black-box models, which not only limits their applications, but also hinders testing and verifying. In this paper, we present an active learning framework to extract automata from neural network classifiers, which can help users to understand the classifiers. In more detail, we use Angluin’s L* algorithm as a learner and the neural network under learning as an oracle, employing abstraction interpretation of the neural network for answering membership and equivalence queries. Our abstraction consists of value, symbol and word abstractions. The factors that may affect the abstraction are also discussed in the paper. We have implemented our approach in a prototype. To evaluate it, we have performed the prototype on a MNIST classifier and have identified that the abstraction with interval number 2 and block size 1 × 28 offers the best performance in terms of F1 score. We also have compared our extracted DFA against the DFAs learned via the passive learning algorithms provided in LearnLib and the experimental results show that our DFA gives a better performance on the MNIST dataset.

scholarly journals HANDWRITTEN CHARACTER RECOGNITION USING CONVOLUTIONAL NEURAL NETWORKS

2021 ◽  
Vol 10 (8) ◽  
pp. 41-45
Author(s):  
D.J. Samatha Naidu ◽  
T. Mahammad Rafi
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Character Recognition ◽  
Large Scale ◽  
Deep Neural Networks ◽  
Recognition System ◽  
Self Organizing Map ◽  
Handwritten Character Recognition ◽  
Handwritten Character ◽  
The Neural Network

Handwritten character Recognition is one of the active area of research where deep neural networks are been utilized. Handwritten character Recognition is a challenging task because of many reasons. The Primary reason is different people have different styles of handwriting. The secondary reason is there are lot of characters like capital letters, small letters & special symbols. In existing were immense research going on the field of handwritten character recognition system has been design using fuzzy logic and created on VLSI(very large scale integrated)structure. To Recognize the tamil characters they have use neural networks with the Kohonen self-organizing map(SOM) which is an unsupervised neural networks. In proposed system this project design a image segmentation based hand written character recognition system. The convolutional neural network is the current state of neural network which has wide application in fields like image, video recognition. The system easily identify or easily recognize text in English languages and letters, digits. By using Open cv for performing image processing and having tensor flow for training the neural network. To develop this concept proposing the innovative method for offline handwritten characters. detection using deep neural networks using python programming language.

SCORING MODELING BASED ON NEURAL NETWORKS FOR DETERMINING A BANK BORROWER'S RATING

2020 ◽  
Vol 2020 (10) ◽  
pp. 54-62
Author(s):  
Oleksii VASYLIEV ◽  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Network Architecture ◽  
Statistical Data ◽  
Activation Function ◽  
Decision Making Process ◽  
Neural Network Architecture ◽  
Acceptable Accuracy ◽  
The Neural Network ◽  
Sigmoid Activation Function

The problem of applying neural networks to calculate ratings used in banking in the decision-making process on granting or not granting loans to borrowers is considered. The task is to determine the rating function of the borrower based on a set of statistical data on the effectiveness of loans provided by the bank. When constructing a regression model to calculate the rating function, it is necessary to know its general form. If so, the task is to calculate the parameters that are included in the expression for the rating function. In contrast to this approach, in the case of using neural networks, there is no need to specify the general form for the rating function. Instead, certain neural network architecture is chosen and parameters are calculated for it on the basis of statistical data. Importantly, the same neural network architecture can be used to process different sets of statistical data. The disadvantages of using neural networks include the need to calculate a large number of parameters. There is also no universal algorithm that would determine the optimal neural network architecture. As an example of the use of neural networks to determine the borrower's rating, a model system is considered, in which the borrower's rating is determined by a known non-analytical rating function. A neural network with two inner layers, which contain, respectively, three and two neurons and have a sigmoid activation function, is used for modeling. It is shown that the use of the neural network allows restoring the borrower's rating function with quite acceptable accuracy.

scholarly journals Discretization and machine learning approximation of BSDEs with a constraint on the Gains-process

2021 ◽  
Vol 0 (0) ◽  
Author(s):  
Idris Kharroubi ◽  
Thomas Lim ◽  
Xavier Warin
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Neural Networks ◽  
Differential Equations ◽  
Numerical Experiments ◽  
Optimization Problem ◽  
Learning Approach ◽  
The Neural Network ◽  
Machine Learning Approach ◽  
Mesh Grid

AbstractWe study the approximation of backward stochastic differential equations (BSDEs for short) with a constraint on the gains process. We first discretize the constraint by applying a so-called facelift operator at times of a grid. We show that this discretely constrained BSDE converges to the continuously constrained one as the mesh grid converges to zero. We then focus on the approximation of the discretely constrained BSDE. For that we adopt a machine learning approach. We show that the facelift can be approximated by an optimization problem over a class of neural networks under constraints on the neural network and its derivative. We then derive an algorithm converging to the discretely constrained BSDE as the number of neurons goes to infinity. We end by numerical experiments.

Application of neural networks in predictions of brake wear particulate matter emission

2021 ◽  
pp. 095440702110363
Author(s):  
Saša Vasiljević ◽  
Jasna Glišović ◽  
Nadica Stojanović ◽  
Ivan Grujić
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Particulate Matter ◽  
Traffic Control ◽  
World Health ◽  
Vehicle Speed ◽  
System Pressure ◽  
The Neural Network ◽  
Vehicle Systems ◽  
Brake Wear

According to the World Health Organization, air pollution with PM10 and PM2.5 (PM-particulate matter) is a significant problem that can have serious consequences for human health. Vehicles, as one of the main sources of PM10 and PM2.5 emissions, pollute the air and the environment both by creating particles by burning fuel in the engine, and by wearing of various elements in some vehicle systems. In this paper, the authors conducted the prediction of the formation of PM10 and PM2.5 particles generated by the wear of the braking system using a neural network (Artificial Neural Networks (ANN)). In this case, the neural network model was created based on the generated particles that were measured experimentally, while the validity of the created neural network was checked by means of a comparative analysis of the experimentally measured amount of particles and the prediction results. The experimental results were obtained by testing on an inertial braking dynamometer, where braking was performed in several modes, that is under different braking parameters (simulated vehicle speed, brake system pressure, temperature, braking time, braking torque). During braking, the concentration of PM10 and PM2.5 particles was measured simultaneously. The total of 196 measurements were performed and these data were used for training, validation, and verification of the neural network. When it comes to simulation, a comparison of two types of neural networks was performed with one output and with two outputs. For each type, network training was conducted using three different algorithms of backpropagation methods. For each neural network, a comparison of the obtained experimental and simulation results was performed. More accurate prediction results were obtained by the single-output neural network for both particulate sizes, while the smallest error was found in the case of a trained neural network using the Levenberg-Marquardt backward propagation algorithm. The aim of creating such a prediction model is to prove that by using neural networks it is possible to predict the emission of particles generated by brake wear, which can be further used for modern traffic systems such as traffic control. In addition, this wear algorithm could be applied on other vehicle systems, such as a clutch or tires.

scholarly journals SketchGNN: Semantic Sketch Segmentation with Graph Neural Networks

2021 ◽  
Vol 40 (3) ◽  
pp. 1-13
Author(s):  
Lumin Yang ◽  
Jiajie Zhuang ◽  
Hongbo Fu ◽  
Xiangzhi Wei ◽  
Kun Zhou ◽  
...  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Network Architecture ◽  
Large Scale ◽  
State Of The Art ◽  
Semantic Segmentation ◽  
Structure Information ◽  
Graph Neural Networks ◽  
Node Labels ◽  
Point Level

We introduce SketchGNN , a convolutional graph neural network for semantic segmentation and labeling of freehand vector sketches. We treat an input stroke-based sketch as a graph with nodes representing the sampled points along input strokes and edges encoding the stroke structure information. To predict the per-node labels, our SketchGNN uses graph convolution and a static-dynamic branching network architecture to extract the features at three levels, i.e., point-level, stroke-level, and sketch-level. SketchGNN significantly improves the accuracy of the state-of-the-art methods for semantic sketch segmentation (by 11.2% in the pixel-based metric and 18.2% in the component-based metric over a large-scale challenging SPG dataset) and has magnitudes fewer parameters than both image-based and sequence-based methods.

scholarly journals Combine and conquer: event reconstruction with Bayesian Ensemble Neural Networks

2021 ◽  
Vol 2021 (4) ◽  
Author(s):  
Jack Y. Araz ◽  
Michael Spannowsky
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Top Quark ◽  
Feature Space ◽  
Training Sample ◽  
Bayesian Techniques ◽  
Training Sample Size ◽  
Event Reconstruction ◽  
Neural Network Classifiers ◽  
Quark Jets

Abstract Ensemble learning is a technique where multiple component learners are combined through a protocol. We propose an Ensemble Neural Network (ENN) that uses the combined latent-feature space of multiple neural network classifiers to improve the representation of the network hypothesis. We apply this approach to construct an ENN from Convolutional and Recurrent Neural Networks to discriminate top-quark jets from QCD jets. Such ENN provides the flexibility to improve the classification beyond simple prediction combining methods by linking different sources of error correlations, hence improving the representation between data and hypothesis. In combination with Bayesian techniques, we show that it can reduce epistemic uncertainties and the entropy of the hypothesis by simultaneously exploiting various kinematic correlations of the system, which also makes the network less susceptible to a limitation in training sample size.

Sign in / Sign up

Export Citation Format

Share Document