Probabilistically Grounded Unsupervised Training of Neural Networks

2016 ◽  
pp. 533-558 ◽  
Author(s):  
Edmondo Trentin ◽  
Marco Bongini
Keyword(s):  
Neural Networks ◽  
Unsupervised Training

scholarly journals Acoustic lung signals analysis based on Mel frequency cepstral coefficients and self-organizing maps

2016 ◽  
Vol 25 (43) ◽  
pp. 73-82
Author(s):  
Álvaro David Orjuela-Cañón ◽  
Hugo Fernando Posada-Quintero
Keyword(s):  
Neural Networks ◽  
Clustering Algorithm ◽  
Mel Frequency Cepstral Coefficients ◽  
Self Organizing Maps ◽  
Number Of Clusters ◽  
Supervised Training ◽  
Computer Aided ◽  
Unsupervised Training ◽  
Cepstral Coefficients ◽  
Self Organizing

This study analyzes acoustic lung signals with different abnormalities, using Mel Frequency Cepstral Coefficients (MFCC), Self-Organizing Maps (SOM), and K-means clustering algorithm. SOM models are known as artificial neural networks than can be trained in an unsupervised or supervised manner. Both approaches were used in this work to compare the utility of this tool in lung signals studies. Results showed that with a supervised training, the classification reached rates of 85 % in accuracy. Unsupervised training was used for clustering tasks, and three clusters was the most adequate number for both supervised and unsupervised training. In general, SOM models can be used in lung signals as a strategy to diagnose systems, finding number of clusters in data, and making classifications for computer-aided decision making systems.

scholarly journals Learning cellular morphology with neural networks

2018 ◽  
Author(s):  
Philipp J Schubert ◽  
Sven Dorkenwald ◽  
Michał Januszewski ◽  
Viren Jain ◽  
Joergen Kornfeld
Keyword(s):  
Neural Networks ◽  
Cell Types ◽  
Cellular Morphology ◽  
Data Sets ◽  
Neuron Reconstruction ◽  
Diverse Application ◽  
Unsupervised Training ◽  
Microscopy Data ◽  
Volume Electron Microscopy ◽  
Made In

AbstractReconstruction and annotation of volume electron microscopy data sets of brain tissue is challenging, but can reveal invaluable information about neuronal circuits. Significant progress has recently been made in automated neuron reconstruction, as well as automated detection of synapses. However, methods for automating the morphological analysis of nanometer-resolution reconstructions are less established, despite their diverse application possibilities. Here, we introduce cellular morphology neural networks (CMNs), based on multi-view projections sampled from automatically reconstructed cellular fragments of arbitrary size and shape. Using unsupervised training we inferred morphology embeddings (“Neuron2vec”) of neuron reconstructions and trained CMNs to identify glia cells in a supervised classification paradigm which was then used to resolve neuron reconstruction errors. Finally, we demonstrate that CMNs can be used to identify subcellular compartments and the cell types of neuron reconstructions.

scholarly journals Analysis of pre-training and initialization methods of neural networks

2018 ◽  
pp. 84-91
Author(s):  
B. Bobyl ◽  
V. Tereschenko
Keyword(s):  
Neural Networks ◽  
Domain Adaptation ◽  
Necessary Condition ◽  
Training Algorithms ◽  
Negative Effects ◽  
Restricted Boltzmann Machines ◽  
Advantages And Disadvantages ◽  
Unsupervised Training ◽  
Parameter Values ◽  
Further Development

In this paper we investigate main pre-training and initialization methods of parameter values of neural networks such as pre-training using restricted Boltzmann machines, deep autoencoders, Glorot and He initialization of parameters, transfer learning and domain adaptation. Given methods are useful for finding of appropriate parameter values and initial initialization of neural network, what is necessary condition for further efficient training of deep models, because it give a possibility during training to reduce negative effects such as vanishing or explosion of gradient, overfitting, stucking in one of local minimums of loss function, etc. These methods belong to group of unsupervised training algorithms and do not need any labeling for data which will be used later for model’s training after parameters initialization. Firstly, in this paper, we analyze all these methods and describe advantages and disadvantages of each of them. Secondly, we describe results of our experiments applying these methods for solving of classification task of MNIST dataset and introduce ideas for further development and improvement of these algorithms.

Combining Exploratory Projection Pursuit and Projection Pursuit Regression with Application to Neural Networks

1993 ◽  
Vol 5 (3) ◽  
pp. 443-455 ◽  
Author(s):  
Nathan Intrator
Keyword(s):  
Neural Networks ◽  
Projection Pursuit ◽  
Regression Method ◽  
Projection Pursuit Regression ◽  
Supervised Training ◽  
New Family ◽  
Unsupervised Training ◽  
Classification And Regression ◽  
Exploratory Projection Pursuit ◽  
Real World Problems

We present a novel classification and regression method that combines exploratory projection pursuit (unsupervised training) with projection pursuit regression (supervised training), to yield a new family of cost/complexity penalty terms. Some improved generalization properties are demonstrated on real-world problems.

scholarly journals Developing Soft Sensors for Polymer Melt Index in an Industrial Polymerization Process Using Deep Belief Networks

2019 ◽  
Vol 17 (1) ◽  
pp. 44-54 ◽  
Author(s):  
Chang-Hao Zhu ◽  
Jie Zhang
Keyword(s):  
Neural Networks ◽  
Polymer Melt ◽  
Polymerization Process ◽  
Training Phase ◽  
Measurement Instruments ◽  
Soft Sensors ◽  
Process Data ◽  
Melt Index ◽  
Model Complex ◽  
Unsupervised Training

Abstract This paper presents developing soft sensors for polymer melt index in an industrial polymerization process by using deep belief network (DBN). The important quality variable melt index of polypropylene is hard to measure in industrial processes. Lack of online measurement instruments becomes a problem in polymer quality control. One effective solution is to use soft sensors to estimate the quality variables from process data. In recent years, deep learning has achieved many successful applications in image classification and speech recognition. DBN as one novel technique has strong generalization capability to model complex dynamic processes due to its deep architecture. It can meet the demand of modelling accuracy when applied to actual processes. Compared to the conventional neural networks, the training of DBN contains a supervised training phase and an unsupervised training phase. To mine the valuable information from process data, DBN can be trained by the process data without existing labels in an unsupervised training phase to improve the performance of estimation. Selection of DBN structure is investigated in the paper. The modelling results achieved by DBN and feedforward neural networks are compared in this paper. It is shown that the DBN models give very accurate estimations of the polymer melt index.

A Semi-Online Training Algorithm for the Radial Basis Function Neural Networks

2008 ◽  
pp. 243-260 ◽  
Author(s):  
Vadlamani Ravi ◽  
P. Ravi Kumar ◽  
Eruku Ravi Srinivas ◽  
Nikola K. Kasabov
Keyword(s):  
Neural Networks ◽  
Radial Basis Function ◽  
Basis Function ◽  
Clustering Algorithm ◽  
Ordinary Least Squares ◽  
Bankruptcy Prediction ◽  
Data Sets ◽  
Performance Metric ◽  
Radial Basis ◽  
Unsupervised Training

This chapter presents an algorithm to train radial basis function neural networks (RBFN) in a semi-online manner. It employs the online, evolving clustering algorithm of Kasabov and Song (2002) in the unsupervised training part of the RBFN and the ordinary least squares estimation technique for the supervised training part. Its effectiveness is demonstrated on two problems related to bankruptcy prediction in financial engineering. In all the cases, 10-fold cross validation was performed. The present algorithm, implemented in two variants, yielded more sensitivity compared to the multi layer perceptron trained by backpropagation (MLP) algorithm over all the problems studied. Based on the results, it can be inferred that the semi-online RBFN without linear terms is better than other neural network techniques. By taking the Area Under the ROC curve (AUC) as the performance metric, the proposed algorithms viz., semi-online RBFN with and without linear terms are compared with classifiers such as ANFIS, TreeNet, SVM, MLP, Linear RBF, RSES and Orthogonal RBF. Out of them TreeNet outperformed both the variants of the semi-online RBFN in both data sets considered here.

Sign in / Sign up

Export Citation Format

Share Document