scholarly journals Pattern Layer Reduction for a Generalized Regression Neural Network by Using a Self–Organizing Map

2018 ◽  
Vol 28 (2) ◽  
pp. 411-424 ◽  
Author(s):  
Serkan Kartal ◽  
Mustafa Oral ◽  
Buse Melis Ozyildirim
Keyword(s):  
Neural Network ◽  
Training Data ◽  
General Regression Neural Network ◽  
Training Dataset ◽  
Generalized Regression Neural Network ◽  
Self Organizing Map ◽  
Performance Loss ◽  
Pattern Layer ◽  
Benchmark Test Functions ◽  
Self Organizing

Abstract In a general regression neural network (GRNN), the number of neurons in the pattern layer is proportional to the number of training samples in the dataset. The use of a GRNN in applications that have relatively large datasets becomes troublesome due to the architecture and speed required. The great number of neurons in the pattern layer requires a substantial increase in memory usage and causes a substantial decrease in calculation speed. Therefore, there is a strong need for pattern layer size reduction. In this study, a self-organizing map (SOM) structure is introduced as a pre-processor for the GRNN. First, an SOM is generated for the training dataset. Second, each training record is labelled with the most similar map unit. Lastly, when a new test record is applied to the network, the most similar map units are detected, and the training data that have the same labels as the detected units are fed into the network instead of the entire training dataset. This scheme enables a considerable reduction in the pattern layer size. The proposed hybrid model was evaluated by using fifteen benchmark test functions and eight different UCI datasets. According to the simulation results, the proposed model significantly simplifies the GRNN’s structure without any performance loss.

scholarly journals A self organizing map (SOM) based electric load classification

2018 ◽  
Vol 31 (4) ◽  
pp. 571-583
Author(s):  
Mahdi Farhadi
Keyword(s):  
Neural Network ◽  
Training Data ◽  
Electric Load ◽  
Self Organizing Map ◽  
Grid Data ◽  
Kohonen Neural Network ◽  
Training Samples ◽  
Proper Training ◽  
Som Neural Network ◽  
Self Organizing

It is of vital importance to use proper training data to perform accurate shortterm load forecasting (STLF) based on artificial neural networks. The pattern of the loads which are used for the training of Kohonen Self Organizing Map (SOM) neural network in STLF models should be of the highest similarity with the pattern of the electric load of the forecasting day. In this paper, an electric load classifier model is proposed which relies on the pattern recognition capability of SOM. The performance of the proposed electric load classifier method is evaluated by Iran electric grid data. The proposed method requires a very few number of training samples for training the Kohonen neural network of the STLF model and can accurately predict electric load in the network.

Classification of salt-contaminated velocities with self-organizing map neural network

2002 ◽  
Vol 21 (12) ◽  
pp. 1193-1196 ◽  
Author(s):  
Lin Zhang ◽  
Al Fortier ◽  
David C. Bartel
Keyword(s):  
Neural Network ◽  
Self Organizing Map ◽  
Self Organizing

scholarly journals Evaluation of Power Insulator Detection Efficiency with the Use of Limited Training Dataset

2020 ◽  
Vol 10 (6) ◽  
pp. 2104
Author(s):  
Michał Tomaszewski ◽  
Paweł Michalski ◽  
Jakub Osuchowski
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Object Detection ◽  
Convolutional Neural Network ◽  
Deep Neural Networks ◽  
Detection Efficiency ◽  
Training Data ◽  
Training Dataset ◽  
Training Set ◽  
Convolutional Network

This article presents an analysis of the effectiveness of object detection in digital images with the application of a limited quantity of input. The possibility of using a limited set of learning data was achieved by developing a detailed scenario of the task, which strictly defined the conditions of detector operation in the considered case of a convolutional neural network. The described solution utilizes known architectures of deep neural networks in the process of learning and object detection. The article presents comparisons of results from detecting the most popular deep neural networks while maintaining a limited training set composed of a specific number of selected images from diagnostic video. The analyzed input material was recorded during an inspection flight conducted along high-voltage lines. The object detector was built for a power insulator. The main contribution of the presented papier is the evidence that a limited training set (in our case, just 60 training frames) could be used for object detection, assuming an outdoor scenario with low variability of environmental conditions. The decision of which network will generate the best result for such a limited training set is not a trivial task. Conducted research suggests that the deep neural networks will achieve different levels of effectiveness depending on the amount of training data. The most beneficial results were obtained for two convolutional neural networks: the faster region-convolutional neural network (faster R-CNN) and the region-based fully convolutional network (R-FCN). Faster R-CNN reached the highest AP (average precision) at a level of 0.8 for 60 frames. The R-FCN model gained a worse AP result; however, it can be noted that the relationship between the number of input samples and the obtained results has a significantly lower influence than in the case of other CNN models, which, in the authors’ assessment, is a desired feature in the case of a limited training set.

Combining Self-supervised Learning and Active Learning for Disfluency Detection

2022 ◽  
Vol 21 (3) ◽  
pp. 1-25
Author(s):  
Shaolei Wang ◽  
Zhongyuan Wang ◽  
Wanxiang Che ◽  
Sendong Zhao ◽  
Ting Liu
Keyword(s):  
Neural Network ◽  
Active Learning ◽  
Supervised Learning ◽  
Large Scale ◽  
Training Data ◽  
Fine Tuning ◽  
Training Dataset ◽  
Performance Gap ◽  
Annotation Costs ◽  
Trained Neural Network

Spoken language is fundamentally different from the written language in that it contains frequent disfluencies or parts of an utterance that are corrected by the speaker. Disfluency detection (removing these disfluencies) is desirable to clean the input for use in downstream NLP tasks. Most existing approaches to disfluency detection heavily rely on human-annotated data, which is scarce and expensive to obtain in practice. To tackle the training data bottleneck, in this work, we investigate methods for combining self-supervised learning and active learning for disfluency detection. First, we construct large-scale pseudo training data by randomly adding or deleting words from unlabeled data and propose two self-supervised pre-training tasks: (i) a tagging task to detect the added noisy words and (ii) sentence classification to distinguish original sentences from grammatically incorrect sentences. We then combine these two tasks to jointly pre-train a neural network. The pre-trained neural network is then fine-tuned using human-annotated disfluency detection training data. The self-supervised learning method can capture task-special knowledge for disfluency detection and achieve better performance when fine-tuning on a small annotated dataset compared to other supervised methods. However, limited in that the pseudo training data are generated based on simple heuristics and cannot fully cover all the disfluency patterns, there is still a performance gap compared to the supervised models trained on the full training dataset. We further explore how to bridge the performance gap by integrating active learning during the fine-tuning process. Active learning strives to reduce annotation costs by choosing the most critical examples to label and can address the weakness of self-supervised learning with a small annotated dataset. We show that by combining self-supervised learning with active learning, our model is able to match state-of-the-art performance with just about 10% of the original training data on both the commonly used English Switchboard test set and a set of in-house annotated Chinese data.

Sign in / Sign up

Export Citation Format

Share Document