scholarly journals Flexible, non-parametric modeling using regularized neural networks

2022 ◽  
Author(s):  
Oskar Allerbo ◽  
Rebecka Jörnsten
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Heterogeneous Data ◽  
Parametric Modeling ◽  
Additive Models ◽  
Adaptive Lasso ◽  
Complex Data ◽  
Data Set ◽  
Model Complex ◽  
Non Parametric

AbstractNon-parametric, additive models are able to capture complex data dependencies in a flexible, yet interpretable way. However, choosing the format of the additive components often requires non-trivial data exploration. Here, as an alternative, we propose PrAda-net, a one-hidden-layer neural network, trained with proximal gradient descent and adaptive lasso. PrAda-net automatically adjusts the size and architecture of the neural network to reflect the complexity and structure of the data. The compact network obtained by PrAda-net can be translated to additive model components, making it suitable for non-parametric statistical modelling with automatic model selection. We demonstrate PrAda-net on simulated data, where we compare the test error performance, variable importance and variable subset identification properties of PrAda-net to other lasso-based regularization approaches for neural networks. We also apply PrAda-net to the massive U.K. black smoke data set, to demonstrate how PrAda-net can be used to model complex and heterogeneous data with spatial and temporal components. In contrast to classical, statistical non-parametric approaches, PrAda-net requires no preliminary modeling to select the functional forms of the additive components, yet still results in an interpretable model representation.

scholarly journals Adaptation of the Diagnostic Artificial Neural Network Structure When New Training Examples Appear

2020 ◽  
Vol 6 (4) ◽  
pp. 120-126
Author(s):  
A. Malikov
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Artificial Neural Network ◽  
Information Security ◽  
Classification Problem ◽  
Data Set ◽  
Neural Network Structure ◽  
Artificial Neural ◽  
Training Examples ◽  
Information Image

In this paper we can see that identified computer incidents are subject for diagnostics, during which the characteristics of information security violations are clarified (purpose, causes, consequences, etc.). To diagnose computer incidents, we can use methods of automation while collection and processing the events that occur as a result of the implementation of scenarios for information security violations. Artificial neural networks can be used to solve the classification problem of assigning diagnostic data set (information image of a computer incident) to one of the possible values of the violation characteristic. The purpose of this work is to adapt the structure of an artificial neural network that allows the accuracy diagnostics of computer incidents when new training examples appear.

scholarly journals NEURAL NETWORK ANALYSIS FOR TUMOR INVESTIGATION AND CANCER PREDICTION

2019 ◽  
Vol 2019 (02) ◽  
pp. 89-98
Author(s):  
Vijayakumar T
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Early Stage ◽  
Original Data ◽  
Data Set ◽  
Cancer Prediction ◽  
The Neural Network ◽  
Feed Forward Neural Networks ◽  
The Neural Networks ◽  
Human Nervous System

Predicting the category of tumors and the types of the cancer in its early stage remains as a very essential process to identify depth of the disease and treatment available for it. The neural network that functions similar to the human nervous system is widely utilized in the tumor investigation and the cancer prediction. The paper presents the analysis of the performance of the neural networks such as the, FNN (Feed Forward Neural Networks), RNN (Recurrent Neural Networks) and the CNN (Convolutional Neural Network) investigating the tumors and predicting the cancer. The results obtained by evaluating the neural networks on the breast cancer Wisconsin original data set shows that the CNN provides 43 % better prediction than the FNN and 25% better prediction than the RNN.

scholarly journals Multiple Ensemble Neural Network Models with Fuzzy Response Aggregation for Predicting COVID-19 Time Series: The Case of Mexico

Healthcare ◽  
2020 ◽  
Vol 8 (2) ◽  
pp. 181 ◽  
Author(s):  
Patricia Melin ◽  
Julio Cesar Monica ◽  
Daniela Sanchez ◽  
Oscar Castillo
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Time Series ◽  
Fuzzy Logic ◽  
Network Models ◽  
Neural Network Models ◽  
Validation Data ◽  
Data Set ◽  
Ensemble Neural Networks ◽  
Ensemble Neural Network

In this paper, a multiple ensemble neural network model with fuzzy response aggregation for the COVID-19 time series is presented. Ensemble neural networks are composed of a set of modules, which are used to produce several predictions under different conditions. The modules are simple neural networks. Fuzzy logic is then used to aggregate the responses of several predictor modules, in this way, improving the final prediction by combining the outputs of the modules in an intelligent way. Fuzzy logic handles the uncertainty in the process of making a final decision about the prediction. The complete model was tested for the case of predicting the COVID-19 time series in Mexico, at the level of the states and the whole country. The simulation results of the multiple ensemble neural network models with fuzzy response integration show very good predicted values in the validation data set. In fact, the prediction errors of the multiple ensemble neural networks are significantly lower than using traditional monolithic neural networks, in this way showing the advantages of the proposed approach.

A simulation-based method using artificial neural networks for solving the inverse kinematic problem of articulated robots

2015 ◽  
Vol 231 (3) ◽  
pp. 470-479 ◽  
Author(s):  
Mustafa Soylak ◽  
Tuğrul Oktay ◽  
İlke Turkmen
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Artificial Neural Network ◽  
Artificial Neural Networks ◽  
Inverse Kinematic ◽  
Plasma Cutting ◽  
Three Dimensional Model ◽  
Inverse Kinematic Problem ◽  
Data Set ◽  
Artificial Neural

In our article, inverse kinematic problem of a plasma cutting robot with three degree of freedom is solved using artificial neural networks. Artificial neural network was trained using joint angle values according to cartesian coordinates ( x, y, z) of end point of a robotic arm. The Levenberg–Marquardt training algorithm was applied to educate artificial neural network. To validate the designed neural network, it was tested using a new test data set which is not applied in training. A simulation was performed on a three-dimensional model of MSC.ADAMS software using angle values obtained from artificial neural network test. It was revealed from this simulation that trajectory of plasma cutting torch obtained using artificial neural network agreed well with desired trajectory.

scholarly journals Ensemble Neural Networks for Modeling DEM Error

2019 ◽  
Vol 8 (10) ◽  
pp. 444 ◽  
Author(s):  
Nguyen ◽  
Starek ◽  
Tissot ◽  
Cai ◽  
Gibeaut
Keyword(s):  
Neural Networks ◽  
Laser Scanning ◽  
Gulf Coast ◽  
Ground Truth ◽  
Tolerance Interval ◽  
Data Set ◽  
Model Complex ◽  
Ensemble Neural Networks ◽  
Dem Accuracy ◽  
Mean Square Errors

Digital elevation models (DEMs) have become ubiquitous and remarkably effective in the field of earth sciences as a tool to characterize surface topography. All DEMs have a degree of inherent error and uncertainty that is propagated to subsequent models and analyses, which can lead to misinterpretation and inaccurate estimates. A new method was developed to estimate local DEM errors and implement corrections while quantifying the uncertainties of the implemented corrections. The method is based on the flexibility and ability to model complex problems with ensemble neural networks (ENNs). The method was developed to be applied to any DEM created from a corresponding set of elevation points (point cloud) and a set of ground truth measurements. The method was developed and tested using hyperspatial resolution terrestrial laser scanning (TLS) data (sub-centimeter point spacing) collected from a marsh site located along the southern portion of the Texas Gulf Coast, USA. ENNs improve the overall DEM accuracy in the study area by 68% for six model inputs and by 75% for 12 model inputs corresponding to root mean square errors (RMSEs) of 0.056 and 0.045 m, respectively. The 12-input model provides more accurate tolerance interval estimates, particularly for vegetated areas. The accuracy of the method is confirmed based on an independent data set. Although the method still underestimates the 95% tolerance interval, 8% below the 95% target, results show that it is able to quantify the spatial variability in uncertainties due to a relationship between vegetation/land cover and accuracy of the DEM for the study area. There are still opportunities and challenges in improving and confirming the applicability of this method for different study sites and data sets.

scholarly journals Automatic Gully Detection: Neural Networks and Computer Vision

2020 ◽  
Vol 12 (11) ◽  
pp. 1743
Author(s):  
Artur M. Gafurov ◽  
Oleg P. Yermolayev
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Large Scale ◽  
Satellite Images ◽  
Gully Erosion ◽  
Training Data ◽  
Russian Plain ◽  
Data Set ◽  
High Resolution Satellite Images

Transition from manual (visual) interpretation to fully automated gully detection is an important task for quantitative assessment of modern gully erosion, especially when it comes to large mapping areas. Existing approaches to semi-automated gully detection are based on either object-oriented selection based on multispectral images or gully selection based on a probabilistic model obtained using digital elevation models (DEMs). These approaches cannot be used for the assessment of gully erosion on the territory of the European part of Russia most affected by gully erosion due to the lack of national large-scale DEM and limited resolution of open source multispectral satellite images. An approach based on the use of convolutional neural networks for automated gully detection on the RGB-synthesis of ultra-high resolution satellite images publicly available for the test region of the east of the Russian Plain with intensive basin erosion has been proposed and developed. The Keras library and U-Net architecture of convolutional neural networks were used for training. Preliminary results of application of the trained gully erosion convolutional neural network (GECNN) allow asserting that the algorithm performs well in detecting active gullies, well differentiates gullies from other linear forms of slope erosion — rills and balkas, but so far has errors in detecting complex gully systems. Also, GECNN does not identify a gully in 10% of cases and in another 10% of cases it identifies not a gully. To solve these problems, it is necessary to additionally train the neural network on the enlarged training data set.

ELECTROCARDIOGRAM ARRHYTHMIA PATTERN RECOGNITION BASED ON AN IMPROVED WAVELET NEURAL NETWORK

2013 ◽  
Vol 13 (01) ◽  
pp. 1350018 ◽  
Author(s):  
GUANGYING YANG
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Pattern Recognition ◽  
Wavelet Neural Network ◽  
The Body ◽  
Support Vector ◽  
Ventricular Premature Beat ◽  
Ecg Signal ◽  
Data Set ◽  
Bundle Branch Block

Electrocardiography (ECG) is a transthoracic interpretation of the electrical activity of the heart over a period of time, as detected by electrodes attached to the outer surface of the skin and recorded by a device external to the body. ECG signal classification is very important for the clinical detection of arrhythmia. This paper presents an application of an improved wavelet neural network structure to the classification of the ECG beats, because of the high precision and fast learning rate. Feature extraction method in this paper is wavelet transform. Our experimental data set is taken from the MIT-BIH arrhythmia database. The correct detection rate of QRS wave is 95% by testing the data of MIT-BIH database. The proposed methods are applied to a large number of ECG signals consisting of 600 training samples and 120 test samples from the MIT-BIH database. The samples equally represent six different ECG signal types, including normal beat, atrial premature beat, ventricular premature beat, left bundle branch block, right bundle branch block and paced beat. In comparison with pattern recognition methods of BP neural networks, RBF neural networks and Support Vector Machines (SVM), the results in this experiment prove that the wavelet neural network method has a better recognition rate when classifying electrocardiogram signals. The experimental results prove that supposed method in this paper is effective for arrhythmia pattern recognition field.

scholarly journals Graph-neural-network-based delay estimation for communication networks with heterogeneous scheduling policies

2021 ◽  
Vol 2 (4) ◽  
pp. 1-8
Author(s):  
Martin Happ ◽  
Matthias Herlich ◽  
Christian Maier ◽  
Jia Lei Du ◽  
Peter Dorfinger
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Communication Networks ◽  
Delay Estimation ◽  
Percentage Error ◽  
Extended Model ◽  
Data Set ◽  
Scheduling Policies ◽  
One Step ◽  
Graph Neural Networks

Modeling communication networks to predict performance such as delay and jitter is important for evaluating and optimizing them. In recent years, neural networks have been used to do this, which may have advantages over existing models, for example from queueing theory. One of these neural networks is RouteNet, which is based on graph neural networks. However, it is based on simplified assumptions. One key simplification is the restriction to a single scheduling policy, which describes how packets of different flows are prioritized for transmission. In this paper we propose a solution that supports multiple scheduling policies (Strict Priority, Deficit Round Robin, Weighted Fair Queueing) and can handle mixed scheduling policies in a single communication network. Our solution is based on the RouteNet architecture as part of the "Graph Neural Network Challenge". We achieved a mean absolute percentage error under 1% with our extended model on the evaluation data set from the challenge. This takes neural-network-based delay estimation one step closer to practical use.

scholarly journals BraggNet: integrating Bragg peaks using neural networks

2019 ◽  
Vol 52 (4) ◽  
pp. 854-863 ◽  
Author(s):  
Brendan Sullivan ◽  
Rick Archibald ◽  
Jahaun Azadmanesh ◽  
Venu Gopal Vandavasi ◽  
Patricia S. Langan ◽  
...  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Negative Control ◽  
Protein Crystal ◽  
Data Sets ◽  
Data Set ◽  
X Ray ◽  
X Ray Crystallography ◽  
Integrated Intensities ◽  
Neutron Crystallography

Neutron crystallography offers enormous potential to complement structures from X-ray crystallography by clarifying the positions of low-Z elements, namely hydrogen. Macromolecular neutron crystallography, however, remains limited, in part owing to the challenge of integrating peak shapes from pulsed-source experiments. To advance existing software, this article demonstrates the use of machine learning to refine peak locations, predict peak shapes and yield more accurate integrated intensities when applied to whole data sets from a protein crystal. The artificial neural network, based on the U-Net architecture commonly used for image segmentation, is trained using about 100 000 simulated training peaks derived from strong peaks. After 100 training epochs (a round of training over the whole data set broken into smaller batches), training converges and achieves a Dice coefficient of around 65%, in contrast to just 15% for negative control data sets. Integrating whole peak sets using the neural network yields improved intensity statistics compared with other integration methods, including k-nearest neighbours. These results demonstrate, for the first time, that neural networks can learn peak shapes and be used to integrate Bragg peaks. It is expected that integration using neural networks can be further developed to increase the quality of neutron, electron and X-ray crystallography data.

scholarly journals Deep Convolutional Spiking Neural Networks for Image Classification

2021 ◽  
Author(s):  
Ruthvik Vaila
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Artificial Neural Networks ◽  
Gradient Descent ◽  
Stochastic Gradient ◽  
Spiking Neural Networks ◽  
Stochastic Gradient Descent ◽  
Data Set ◽  
Learning Capabilities ◽  
Artificial Neural

Spiking neural networks are biologically plausible counterparts of artificial neural networks. Artificial neural networks are usually trained with stochastic gradient descent (SGD) and spiking neural networks are trained with bioinspired spike timing dependent plasticity (STDP). Spiking networks could potentially help in reducing power usage owing to their binary activations. In this work, we use unsupervised STDP in the feature extraction layers of a neural network with instantaneous neurons to extract meaningful features. The extracted binary feature vectors are then classified using classification layers containing neurons with binary activations. Gradient descent (backpropagation) is used only on the output layer to perform training for classification. Surrogate gradients are proposed to perform backpropagation with binary gradients. The accuracies obtained for MNIST and the balanced EMNIST data set compare favorably with other approaches. The effect of the stochastic gradient descent (SGD) approximations on learning capabilities of our network are also explored. We also studied catastrophic forgetting and its effect on spiking neural networks (SNNs). For the experiments regarding catastrophic forgetting, in the classification sections of the network we use a modified synaptic intelligence that we refer to as cost per synapse metric as a regularizer to immunize the network against catastrophic forgetting in a Single-Incremental-Task scenario (SIT). In catastrophic forgetting experiments, we use MNIST and EMNIST handwritten digits datasets that were divided into five and ten incremental subtasks respectively. We also examine behavior of the spiking neural network and empirically study the effect of various hyperparameters on its learning capabilities using the software tool SPYKEFLOW that we developed. We employ MNIST, EMNIST and NMNIST data sets to produce our results.

Sign in / Sign up

Export Citation Format

Share Document