scholarly journals Self-Organizing Representation Learning

2021 ◽  
Author(s):  
noureddine kermiche
Keyword(s):  
Data Augmentation ◽  
Representation Learning ◽  
Self Organization ◽  
Learning Methods ◽  
Self Organizing Maps ◽  
Output Layer ◽  
Data Representations ◽  
Augmentation Techniques ◽  
Using Data ◽  
Self Organizing

Using data augmentation techniques, unsupervised representation learning methods extract features from data by training artificial neural networks to recognize that different views of an object are just different instances of the same object. We extend current unsupervised representation learning methods to networks that can self-organize data representations into two-dimensional (2D) maps. The proposed method combines ideas from Kohonen’s original self-organizing maps (SOM) and recent development in unsupervised representation learning. A ResNet backbone with an added 2D <i>Softmax</i> output layer is used to organize the data representations. A new loss function with linear complexity is proposed to enforce SOM requirements of winner-take-all (WTA) and competition between neurons while explicitly avoiding collapse into trivial solutions. We show that enforcing SOM topological neighborhood requirement can be achieved by a fixed radial convolution at the 2D output layer without having to resort to actual radial activation functions which prevented the original SOM algorithm from being extended to nowadays neural network architectures. We demonstrate that when combined with data augmentation techniques, self-organization is a simple emergent property of the 2D output layer because of neighborhood recruitment combined with WTA competition between neurons. The proposed methodology is demonstrated on SVHN and CIFAR10 data sets. The proposed algorithm is the first end-to-end unsupervised learning method that combines data self-organization and visualization as integral parts of unsupervised representation learning.

scholarly journals Self-Organizing Representation Learning

2021 ◽  
Author(s):  
noureddine kermiche
Keyword(s):  
Data Augmentation ◽  
Representation Learning ◽  
Self Organization ◽  
Learning Methods ◽  
Self Organizing Maps ◽  
Output Layer ◽  
Data Representations ◽  
Augmentation Techniques ◽  
Using Data ◽  
Self Organizing

Using data augmentation techniques, unsupervised representation learning methods extract features from data by training artificial neural networks to recognize that different views of an object are just different instances of the same object. We extend current unsupervised representation learning methods to networks that can self-organize data representations into two-dimensional (2D) maps. The proposed method combines ideas from Kohonen’s original self-organizing maps (SOM) and recent development in unsupervised representation learning. A ResNet backbone with an added 2D <i>Softmax</i> output layer is used to organize the data representations. A new loss function with linear complexity is proposed to enforce SOM requirements of winner-take-all (WTA) and competition between neurons while explicitly avoiding collapse into trivial solutions. We show that enforcing SOM topological neighborhood requirement can be achieved by a fixed radial convolution at the 2D output layer without having to resort to actual radial activation functions which prevented the original SOM algorithm from being extended to nowadays neural network architectures. We demonstrate that when combined with data augmentation techniques, self-organization is a simple emergent property of the 2D output layer because of neighborhood recruitment combined with WTA competition between neurons. The proposed methodology is demonstrated on SVHN and CIFAR10 data sets. The proposed algorithm is the first end-to-end unsupervised learning method that combines data self-organization and visualization as integral parts of unsupervised representation learning.

Data Augmentation for Roller Bearing Health Indicator Estimation Using Multi-Channel Frequency Data Representations

2021 ◽  
Author(s):  
Jacob Hendriks ◽  
Patrick Dumond
Keyword(s):  
Data Augmentation ◽  
Roller Bearing ◽  
Health Indicators ◽  
Remaining Useful Life ◽  
Failure Data ◽  
Data Representations ◽  
Masking Noise ◽  
Augmentation Techniques ◽  
Added Benefit ◽  
Useful Life

Abstract This paper demonstrates various data augmentation techniques that can be used when working with limited run-to-failure data to estimate health indicators related to the remaining useful life of roller bearings. The PRONOSTIA bearing prognosis dataset is used for benchmarking data augmentation techniques. The input to the networks are multi-dimensional frequency representations obtained by combining the spectra taken from two accelerometers. Data augmentation techniques are adapted from other machine learning fields and include adding Gaussian noise, region masking, masking noise, and pitch shifting. Augmented datasets are used in training a conventional CNN architecture comprising two convolutional and pooling layer sequences with batch normalization. Results from individually separating each bearing’s data for the purpose of validation shows that all methods, except pitch shifting, give improved validation accuracy on average. Masking noise and region masking both show the added benefit of dataset regularization by giving results that are more consistent after repeatedly training each configuration with new randomly generated augmented datasets. It is shown that gradually deteriorating bearings and bearings with abrupt failure are not treated significantly differently by the augmentation techniques.

scholarly journals Deep Learning for Person Reidentification Using Support Vector Machines

2017 ◽  
Vol 2017 ◽  
pp. 1-12 ◽  
Author(s):  
Mengyu Xu ◽  
Zhenmin Tang ◽  
Yazhou Yao ◽  
Lingxiang Yao ◽  
Huafeng Liu ◽  
...  
Keyword(s):  
Data Augmentation ◽  
Metric Learning ◽  
Representation Learning ◽  
Activation Function ◽  
Feature Representation ◽  
Support Vector ◽  
Similarity Learning ◽  
Vector Machines ◽  
Augmentation Techniques ◽  
Suboptimal Solution

Due to the variations of viewpoint, pose, and illumination, a given individual may appear considerably different across different camera views. Tracking individuals across camera networks with no overlapping fields is still a challenging problem. Previous works mainly focus on feature representation and metric learning individually which tend to have a suboptimal solution. To address this issue, in this work, we propose a novel framework to do the feature representation learning and metric learning jointly. Different from previous works, we represent the pairs of pedestrian images as new resized input and use linear Support Vector Machine to replace softmax activation function for similarity learning. Particularly, dropout and data augmentation techniques are also employed in this model to prevent the network from overfitting. Extensive experiments on two publically available datasets VIPeR and CUHK01 demonstrate the effectiveness of our proposed approach.

scholarly journals Analyzing the Effects of Data Augmentation on Single and Multimodal Biometrics

2020 ◽  
Vol 39 (3) ◽  
pp. 647-656
Author(s):  
Mehwish Leghari ◽  
Shahzad Memon ◽  
Lachman Das Dhomeja ◽  
Akhter Hussain Jalbani
Keyword(s):  
Data Augmentation ◽  
Research Study ◽  
Real Data ◽  
Fingerprint Recognition ◽  
Multimodal Biometrics ◽  
Biometric Data ◽  
Augmentation Techniques ◽  
Signature Recognition ◽  
Using Data ◽  
Level Fusion

Now-a-days, in the field of machine learning the data augmentation techniques are common in use, especially with deep neural networks, where a large amount of data is required to train the network. The effectiveness of the data augmentation technique has been analyzed for many applications; however, it has not been analyzed separately for the multimodal biometrics. This research analyzes the effects of data augmentation on single biometric data and multimodal biometric data. In this research, the features from two biometric modalities: fingerprint and signature, have been fused together at the feature level. The primary motivation for fusing biometric data at feature level is to secure the privacy of the user’s biometric data. The results that have been achieved by using data augmentation are presented in this research. The experimental results for the fingerprint recognition, signature recognition and the feature-level fusion of fingerprint with signature have been presented separately. The results show that the accuracy of the training classifier can be enhanced with data augmentation techniques when the size of real data samples is insufficient. This research study explores that how the effectiveness of data augmentation gradually increases with the number of templates for the fused biometric data by making the number of templates double each time until the classifier achieved the accuracy of 99%.

Estimating the Number of Clusters in Multivariate Data by Self-Organizing Maps

1999 ◽  
Vol 09 (03) ◽  
pp. 195-202 ◽  
Author(s):  
JOSÉ ALFREDO FERREIRA COSTA ◽  
MÁRCIO LUIZ DE ANDRADE NETTO
Keyword(s):  
A Priori ◽  
Multivariate Data ◽  
Feature Space ◽  
Search Space ◽  
Data Sets ◽  
Self Organizing Maps ◽  
Data Set ◽  
Number Of Clusters ◽  
Using Data ◽  
Self Organizing

Determining the structure of data without prior knowledge of the number of clusters or any information about their composition is a problem of interest in many fields, such as image analysis, astrophysics, biology, etc. Partitioning a set of n patterns in a p-dimensional feature space must be done such that those in a given cluster are more similar to each other than the rest. As there are approximately [Formula: see text] possible ways of partitioning the patterns among K clusters, finding the best solution is very hard when n is large. The search space is increased when we have no a priori number of partitions. Although the self-organizing feature map (SOM) can be used to visualize clusters, the automation of knowledge discovery by SOM is a difficult task. This paper proposes region-based image processing methods to post-processing the U-matrix obtained after the unsupervised learning performed by SOM. Mathematical morphology is applied to identify regions of neurons that are similar. The number of regions and their labels are automatically found and they are related to the number of clusters in a multivariate data set. New data can be classified by labeling it according to the best match neuron. Simulations using data sets drawn from finite mixtures of p-variate normal densities are presented as well as related advantages and drawbacks of the method.

scholarly journals Detecting operons in bacterial genomes via visual representation learning

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Rida Assaf ◽  
Fangfang Xia ◽  
Rick Stevens
Keyword(s):  
Regulatory Networks ◽  
Data Augmentation ◽  
State Of The Art ◽  
Critical Role ◽  
Visual Representations ◽  
Representation Learning ◽  
Genomic Region ◽  
Bacterial Genomes ◽  
Gene Neighborhood ◽  
Augmentation Techniques

AbstractContiguous genes in prokaryotes are often arranged into operons. Detecting operons plays a critical role in inferring gene functionality and regulatory networks. Human experts annotate operons by visually inspecting gene neighborhoods across pileups of related genomes. These visual representations capture the inter-genic distance, strand direction, gene size, functional relatedness, and gene neighborhood conservation, which are the most prominent operon features mentioned in the literature. By studying these features, an expert can then decide whether a genomic region is part of an operon. We propose a deep learning based method named Operon Hunter that uses visual representations of genomic fragments to make operon predictions. Using transfer learning and data augmentation techniques facilitates leveraging the powerful neural networks trained on image datasets by re-training them on a more limited dataset of extensively validated operons. Our method outperforms the previously reported state-of-the-art tools, especially when it comes to predicting full operons and their boundaries accurately. Furthermore, our approach makes it possible to visually identify the features influencing the network’s decisions to be subsequently cross-checked by human experts.

scholarly journals Document Clustering using Self-Organizing Maps

MENDEL ◽  
2017 ◽  
Vol 23 (1) ◽  
pp. 111-118
Author(s):  
Muhammad Rafi ◽  
Muhammad Waqar ◽  
Hareem Ajaz ◽  
Umar Ayub ◽  
Muhammad Danish
Keyword(s):  
Document Clustering ◽  
Feature Space ◽  
Good Method ◽  
Self Organization ◽  
Self Organizing Maps ◽  
Document Collections ◽  
Document Collection ◽  
Common Technique ◽  
Low Dimensional ◽  
Self Organizing

Cluster analysis of textual documents is a common technique for better ltering, navigation, under-standing and comprehension of the large document collection. Document clustering is an autonomous methodthat separate out large heterogeneous document collection into smaller more homogeneous sub-collections calledclusters. Self-organizing maps (SOM) is a type of arti cial neural network (ANN) that can be used to performautonomous self-organization of high dimension feature space into low-dimensional projections called maps. Itis considered a good method to perform clustering as both requires unsupervised processing. In this paper, weproposed a SOM using multi-layer, multi-feature to cluster documents. The paper implements a SOM usingfour layers containing lexical terms, phrases and sequences in bottom layers respectively and combining all atthe top layers. The documents are processed to extract these features to feed the SOM. The internal weightsand interconnections between these layers features(neurons) automatically settle through iterations with a smalllearning rate to discover the actual clusters. We have performed extensive set of experiments on standard textmining datasets like: NEWS20, Reuters and WebKB with evaluation measures F-Measure and Purity. Theevaluation gives encouraging results and outperforms some of the existing approaches. We conclude that SOMwith multi-features (lexical terms, phrases and sequences) and multi-layers can be very e ective in producinghigh quality clusters on large document collections.

scholarly journals Evaluation of Methods for Protein Representation Learning: A Quantitative Analysis

2020 ◽  
Author(s):  
Serbulent Unsal ◽  
Heval Ataş ◽  
Muammer Albayrak ◽  
Kemal Turhan ◽  
Aybar C. Acar ◽  
...  
Keyword(s):  
Deep Learning ◽  
Language Processing ◽  
Complex Traits ◽  
Representation Learning ◽  
Data Representation ◽  
Language Models ◽  
Learning Methods ◽  
Advantages And Disadvantages ◽  
Data Representations ◽  
Novel Methods

AbstractData-centric approaches have been utilized to develop predictive methods for elucidating uncharacterized aspects of proteins such as their functions, biophysical properties, subcellular locations and interactions. However, studies indicate that the performance of these methods should be further improved to effectively solve complex problems in biomedicine and biotechnology. A data representation method can be defined as an algorithm that calculates numerical feature vectors for samples in a dataset, to be later used in quantitative modelling tasks. Data representation learning methods do this by training and using a model that employs statistical and machine/deep learning algorithms. These novel methods mostly take inspiration from the data-driven language models that have yielded ground-breaking improvements in the field of natural language processing. Lately, these learned data representations have been applied to the field of protein informatics and have displayed highly promising results in terms of extracting complex traits of proteins regarding sequence-structure-function relations. In this study, we conducted a detailed investigation over protein representation learning methods, by first categorizing and explaining each approach, and then conducting benchmark analyses on; (i) inferring semantic similarities between proteins, (ii) predicting ontology-based protein functions, and (iii) classifying drug target protein families. We examine the advantages and disadvantages of each representation approach over the benchmark results. Finally, we discuss current challenges and suggest future directions. We believe the conclusions of this study will help researchers in applying machine/deep learning-based representation techniques on protein data for various types of predictive tasks. Furthermore, we hope it will demonstrate the potential of machine learning-based data representations for protein science and inspire the development of novel methods/tools to be utilized in the fields of biomedicine and biotechnology.

Sign in / Sign up

Export Citation Format

Share Document