scholarly journals ONLINE MODIFICATION OF THE METHOD OF X-MEDIUM ON THE BASIS OF ANSAMBLY OF SELORGANIZED MAP T. KOHONEN

2017 ◽  
pp. 96-107
Author(s):  
Є.В. БОДЯНСЬКИЙ ◽  
А.О. ДЕЙНЕКО ◽  
П.Є. ЖЕРНОВА ◽  
В.О. РЄПІН
Keyword(s):  
Neural Networks ◽  
Learning Algorithm ◽  
Data Sets ◽  
Self Organizing Maps ◽  
Online Mode ◽  
Wide Range ◽  
Parallel Mode ◽  
Computational Simplicity ◽  
Self Learning

The modified X-means method for clustering in the case when observations are sequentially fed to processing the proposed. This approach’s based on the ensemble of the clustering neural networks, proposed ensemble contains the T. Kohonen’s self-organizing maps. Each of the clustering neural networks consist of different number of neurons, where number of clusters is connected with the quality of there neurons. All ensemble members process information that siquentionally is fed to the system in the parallel mode. The effectiveness of clustering process is determined using Caliński-Harabasz index. The self-learning algorithm uses similarity measure of special type that. The feature of proposed method is absent of the competition step, i.e. neuron-winner is not determined. A number of experiments has been held in order to investigate the proposed system’s properties. Experimental results have proven the fact that the system under consideration could be used to solve a wide range of Data Mining tasks when data sets are processed in an online mode. The proposed ensemble system provides computational simplicity, and data sets are pro-cessed faster due to the possibility of parallel tuning.

scholarly journals Label Distribution for Learning with Noisy Labels

2020 ◽  
Author(s):  
Yun-Peng Liu ◽  
Ning Xu ◽  
Yu Zhang ◽  
Xin Geng
Keyword(s):  
Neural Networks ◽  
Deep Neural Networks ◽  
Learning Algorithm ◽  
State Of The Art ◽  
Confidence Estimation ◽  
Novel Method ◽  
Real World Datasets ◽  
Label Distribution ◽  
Noisy Labels

The performances of deep neural networks (DNNs) crucially rely on the quality of labeling. In some situations, labels are easily corrupted, and therefore some labels become noisy labels. Thus, designing algorithms that deal with noisy labels is of great importance for learning robust DNNs. However, it is difficult to distinguish between clean labels and noisy labels, which becomes the bottleneck of many methods. To address the problem, this paper proposes a novel method named Label Distribution based Confidence Estimation (LDCE). LDCE estimates the confidence of the observed labels based on label distribution. Then, the boundary between clean labels and noisy labels becomes clear according to confidence scores. To verify the effectiveness of the method, LDCE is combined with the existing learning algorithm to train robust DNNs. Experiments on both synthetic and real-world datasets substantiate the superiority of the proposed algorithm against state-of-the-art methods.

Fundamentals of Higher Order Neural Networks for Modeling and Simulation

2013 ◽  
pp. 103-133 ◽  
Author(s):  
Madan M. Gupta ◽  
Ivo Bukovsky ◽  
Noriyasu Homma ◽  
Ashu M. G. Solo ◽  
Zeng-Guang Hou
Keyword(s):  
Neural Networks ◽  
Modeling And Simulation ◽  
Learning Algorithm ◽  
Nonlinear Approximation ◽  
Higher Order ◽  
Higher Order Neural Networks ◽  
High Dynamic ◽  
Input Variables ◽  
Continuous Dynamic

In this chapter, the authors provide fundamental principles of Higher Order Neural Units (HONUs) and Higher Order Neural Networks (HONNs) for modeling and simulation. An essential core of HONNs can be found in higher order weighted combinations or correlations between the input variables and HONU. Except for the high quality of nonlinear approximation of static HONUs, the capability of dynamic HONUs for the modeling of dynamic systems is shown and compared to conventional recurrent neural networks when a practical learning algorithm is used. In addition, the potential of continuous dynamic HONUs to approximate high dynamic order systems is discussed, as adaptable time delays can be implemented. By using some typical examples, this chapter describes how and why higher order combinations or correlations can be effective for modeling of systems.

A PROBABILISTIC SELF-ORGANIZING MAP FOR BINARY DATA TOPOGRAPHIC CLUSTERING

2008 ◽  
Vol 07 (04) ◽  
pp. 363-383 ◽  
Author(s):  
MUSTAPHA LEBBAH ◽  
YOUNÈS BENNANI ◽  
NICOLETA ROGOVSCHI
Keyword(s):  
Binary Data ◽  
Learning Algorithm ◽  
Data Sets ◽  
Self Organizing Map ◽  
Data Set ◽  
Binary Coding ◽  
Public Data ◽  
Multivariate Binary Data ◽  
Self Organizing

This paper introduces a probabilistic self-organizing map for topographic clustering, analysis and visualization of multivariate binary data or categorical data using binary coding. We propose a probabilistic formalism dedicated to binary data in which cells are represented by a Bernoulli distribution. Each cell is characterized by a prototype with the same binary coding as used in the data space and the probability of being different from this prototype. The learning algorithm, Bernoulli on self-organizing map, that we propose is an application of the EM standard algorithm. We illustrate the power of this method with six data sets taken from a public data set repository. The results show a good quality of the topological ordering and homogenous clustering.

scholarly journals Transductive Bounds for the Multi-Class Majority Vote Classifier

2019 ◽  
Vol 33 ◽  
pp. 3566-3573
Author(s):  
Vasilii Feofanov ◽  
Emilie Devijver ◽  
Massih-Reza Amini
Keyword(s):  
Learning Algorithm ◽  
Majority Vote ◽  
Confusion Matrix ◽  
Data Sets ◽  
Bayes Classifier ◽  
Partially Labeled Data ◽  
Margin Distribution ◽  
Training Examples ◽  
Multi Class Classification ◽  
Self Learning

In this paper, we propose a transductive bound over the risk of the majority vote classifier learned with partially labeled data for the multi-class classification. The bound is obtained by considering the class confusion matrix as an error indicator and it involves the margin distribution of the classifier over each class and a bound over the risk of the associated Gibbs classifier. When this latter bound is tight and, the errors of the majority vote classifier per class are concentrated on a low margin zone; we prove that the bound over the Bayes classifier’ risk is tight. As an application, we extend the self-learning algorithm to the multi-class case. The algorithm iteratively assigns pseudo-labels to a subset of unlabeled training examples that have their associated class margin above a threshold obtained from the proposed transductive bound. Empirical results on different data sets show the effectiveness of our approach compared to the same algorithm where the threshold is fixed manually, to the extension of TSVM to multi-class classification and to a graph-based semi-supervised algorithm.

scholarly journals OPTIMAL UNIFORM QUANTIZATION OF PARAMETERS OF CONVOLUTIONAL NEURAL NETWORKS

2018 ◽  
pp. 99-103
Author(s):  
D. S. Kolesnikov ◽  
D. A. Kuznetsov
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Mobile Applications ◽  
Recognition Accuracy ◽  
State Of The Art ◽  
Network Parameters ◽  
Wide Range ◽  
Uniform Quantization ◽  
Adaptive Step

State of the art convolutional neural networks provide high accuracy in solving a wide range of problems. Usually it is achieved by a significant increasing their computational complexity and the representation of the network parameters in single-precision floating point numbers. However, due to the limited resources, the application of networks in embedded systems and mobile applications in real time is problematic. One of the methods to solve this problem is to reduce the bit depth of data and use integer arithmetic. For this purpose, the network parameters are quantized. Performing quantization, it is necessary to ensure a minimum loss of recognition accuracy. The article proposes to use an optimal uniform quantizer with an adaptive step. The quantizer step depends on the distribution function of the quantized parameters. It reduces the effect of the quantization error on the recognition accuracy. There are also described approaches to improving the quality of quantization. The proposed quantization method is estimated on the CIFAR-10 database. It is shown that the optimal uniform quantizer for CIFAR-10 database with 8-bit representation of network parameters allows to achieve the accuracy of the initial trained network.

scholarly journals Circularly shifted filters enable data efficient sequence motif inference with neural networks

2018 ◽  
Author(s):  
Christopher F. Blum ◽  
Markus Kollmann
Keyword(s):  
Neural Networks ◽  
Transcription Factor ◽  
Transcription Factor Binding ◽  
Sequence Motif ◽  
Small Data ◽  
Data Sets ◽  
Sequence Motifs ◽  
Factor Binding ◽  
Wide Range ◽  
Inference Methods

AbstractMotivationNucleic acids and proteins often have localized sequence motifs that enable highly specific interactions. Due to the biological relevance of sequence motifs, numerous inference methods have been developed. Recently, convolutional neural networks (CNNs) achieved state of the art performance because they can approximate complex motif distributions. These methods were able to learn transcription factor binding sites from ChIP-seq data and to make accurate predictions. However, CNNs learn filters that are difficult to interpret, and networks trained on small data sets often do not generalize optimally to new sequences.ResultsHere we present circular filters, a novel convolutional architecture, that contains all circularly shifted variants of the same filter. We motivate circular filters by the observation that CNNs frequently learn filters that correspond to shifted and truncated variants of the true motif. Circular filters enable learning of non-truncated motifs and allow easy interpretation of the learned filters. We show that circular filters improve motif inference performance over a wide range of hyperparameters. Furthermore, we show that CNNs with circular filters perform better at inferring transcription factor binding motifs from ChIP-seq data than conventional [email protected]

scholarly journals Evolution of SOMs’ Structure and Learning Algorithm: From Visualization of High-Dimensional Data to Clustering of Complex Data

Algorithms ◽  
2020 ◽  
Vol 13 (5) ◽  
pp. 109 ◽  
Author(s):  
Marian B. Gorzałczany ◽  
Filip Rudziński
Keyword(s):  
Data Visualization ◽  
Data Clustering ◽  
Learning Algorithm ◽  
High Dimensional Data ◽  
High Dimensional ◽  
Data Sets ◽  
Complex Data ◽  
Self Organizing Maps ◽  
Grid Networks ◽  
Self Organizing

In this paper, we briefly present several modifications and generalizations of the concept of self-organizing neural networks—usually referred to as self-organizing maps (SOMs)—to illustrate their advantages in applications that range from high-dimensional data visualization to complex data clustering. Starting from conventional SOMs, Growing SOMs (GSOMs), Growing Grid Networks (GGNs), Incremental Grid Growing (IGG) approach, Growing Neural Gas (GNG) method as well as our two original solutions, i.e., Generalized SOMs with 1-Dimensional Neighborhood (GeSOMs with 1DN also referred to as Dynamic SOMs (DSOMs)) and Generalized SOMs with Tree-Like Structures (GeSOMs with T-LSs) are discussed. They are characterized in terms of (i) the modification mechanisms used, (ii) the range of network modifications introduced, (iii) the structure regularity, and (iv) the data-visualization/data-clustering effectiveness. The performance of particular solutions is illustrated and compared by means of selected data sets. We also show that the proposed original solutions, i.e., GeSOMs with 1DN (DSOMs) and GeSOMS with T-LSs outperform alternative approaches in various complex clustering tasks by providing up to 20 % increase in the clustering accuracy. The contribution of this work is threefold. First, algorithm-oriented original computer-implementations of particular SOM’s generalizations are developed. Second, their detailed simulation results are presented and discussed. Third, the advantages of our earlier-mentioned original solutions are demonstrated.

Multi-Instance Learning with MultiObjective Genetic Programming

2011 ◽  
pp. 1372-1379
Author(s):  
Amelia Zafra
Keyword(s):  
Machine Learning ◽  
Genetic Programming ◽  
Learning Community ◽  
Text Categorization ◽  
Learning Algorithm ◽  
Programming Algorithm ◽  
Learning Framework ◽  
Wide Range ◽  
Training Examples

The multiple-instance problem is a difficult machine learning problem that appears in cases where knowledge about training examples is incomplete. In this problem, the teacher labels examples that are sets (also called bags) of instances. The teacher does not label whether an individual instance in a bag is positive or negative. The learning algorithm needs to generate a classifier that will correctly classify unseen examples (i.e., bags of instances). This learning framework is receiving growing attention in the machine learning community and since it was introduced by Dietterich, Lathrop, Lozano-Perez (1997), a wide range of tasks have been formulated as multi-instance problems. Among these tasks, we can cite content-based image retrieval (Chen, Bi, & Wang, 2006) and annotation (Qi and Han, 2007), text categorization (Andrews, Tsochantaridis, & Hofmann, 2002), web index page recommendation (Zhou, Jiang, & Li, 2005; Xue, Han, Jiang, & Zhou, 2007) and drug activity prediction (Dietterich et al., 1997; Zhou & Zhang, 2007). In this chapter we introduce MOG3P-MI, a multiobjective grammar guided genetic programming algorithm to handle multi-instance problems. In this algorithm, based on SPEA2, individuals represent classification rules which make it possible to determine if a bag is positive or negative. The quality of each individual is evaluated according to two quality indexes: sensitivity and specificity. Both these measures have been adapted to MIL circumstances. Computational experiments show that the MOG3P-MI is a robust algorithm for classification in different domains where achieves competitive results and obtain classifiers which contain simple rules which add comprehensibility and simplicity in the knowledge discovery process, being suitable method for solving MIL problems (Zafra & Ventura, 2007).

A Hybrid Higher Order Neural Structure for Pattern Recognition

2013 ◽  
pp. 364-387 ◽  
Author(s):  
Mehdi Fallahnezhad ◽  
Salman Zaferanlouei
Keyword(s):  
Neural Networks ◽  
Pattern Recognition ◽  
Structure Learning ◽  
Learning Algorithm ◽  
Hybrid Structure ◽  
Higher Order ◽  
Superior Performance ◽  
Data Sets ◽  
Neural Structure ◽  
Higher Order Neural Networks

Considering high order correlations of selected features next to the raw features of input can facilitate target pattern recognition. In artificial intelligence, this is being addressed by Higher Order Neural Networks (HONNs). In general, HONN structures provide superior specifications (e.g. resolving the dilemma of choosing the number of neurons and layers of networks, better fitting specs, quicker, and open-box specificity) to traditional neural networks. This chapter introduces a hybrid structure of higher order neural networks, which can be generally applied in various branches of pattern recognition. Structure, learning algorithm, and network configuration are introduced, and structure is applied either as classifier (where is called HHONC) to different benchmark statistical data sets or as functional behavior approximation (where is called HHONN) to a heat and mass transfer dilemma. In each structure, results are compared with previous studies, which show its superior performance next to other mentioned advantages.

Improvement of Variant Adaptable LSTM Trained With Metaheuristic Algorithms for Healthcare Analysis

2021 ◽  
pp. 1031-1051
Author(s):  
Tarik A. Rashid ◽  
Mohammad K. Hassan ◽  
Mokhtar Mohammadi ◽  
Kym Fraser
Keyword(s):  
Diabetes Mellitus ◽  
Neural Networks ◽  
Time Series Data ◽  
Short Term Memory ◽  
Learning Algorithm ◽  
Back Propagation ◽  
Machine Learning Techniques ◽  
Series Data ◽  
Data Sets ◽  
Patients With Diabetes

Recently, the population of the world has increased along with health problems. Diabetes mellitus disease as an example causes issues to the health of many patients globally. The task of this chapter is to develop a dynamic and intelligent decision support system for patients with different diseases, and it aims at examining machine-learning techniques supported by optimization techniques. Artificial neural networks have been used in healthcare for several decades. Most research works utilize multilayer layer perceptron (MLP) trained with back propagation (BP) learning algorithm to achieve diabetes mellitus classification. Nonetheless, MLP has some drawbacks, such as, convergence, which can be slow; local minima can affect the training process. It is hard to scale and cannot be used with time series data sets. To overcome these drawbacks, long short-term memory (LSTM) is suggested, which is a more advanced form of recurrent neural networks. In this chapter, adaptable LSTM trained with two optimizing algorithms instead of the back propagation learning algorithm is presented. The optimization algorithms are biogeography-based optimization (BBO) and genetic algorithm (GA). Dataset is collected locally and another benchmark dataset is used as well. Finally, the datasets fed into adaptable models; LSTM with BBO (LSTMBBO) and LSTM with GA (LSTMGA) for classification purposes. The experimental and testing results are compared and they are promising. This system helps physicians and doctors to provide proper health treatment for patients with diabetes mellitus. Details of source code and implementation of our system can be obtained in the following link “https://github.com/hamakamal/LSTM.”

Sign in / Sign up

Export Citation Format

Share Document