scholarly journals Learning with small amount of data on various task

2020 ◽  
Author(s):  
Stefanie
Keyword(s):  
Deep Neural Networks ◽  
Learning Strategy ◽  
Teaching Strategy ◽  
Training Sample ◽  
Learning To Teach ◽  
Training Data ◽  
Training Dataset ◽  
Small Data ◽  
Training Samples ◽  
Teacher Model

As a student, I am learning knowledge with the help of teachers and the teacher plays a crucial role in our life. A wonderful instructor is able to teach a student with appropriate teaching materials. Therefore, in this project, I explore a teaching strategy called learning to teach (L2T) in which a teacher model could provide high-quality training samples to a student model. However, one major problem of L2T is that the teacher model will only select a subset of the training dataset as the final training data for the student. Learning to teach small-data learning strategy (L2TSDL) is proposed to solve this problem. In this strategy, the teacher model will calculate the importance score for every training sample and help students to make use of all training samples. To demonstrate the advantage of the proposed approach over L2T, I take the training of different deep neural networks (DNN) on image classification task as an exampleand show that L2TSDL could achieve good performance on both large and small dataset.

scholarly journals Investigating Learning with Few Shot Data

2020 ◽  
Author(s):  
Stefanie
Keyword(s):  
Learning Strategy ◽  
Teaching Strategy ◽  
Training Sample ◽  
Learning To Teach ◽  
Training Data ◽  
Training Dataset ◽  
Small Data ◽  
Importance Score ◽  
Training Samples ◽  
Teacher Model

In this project, I explore a teaching strategy called learning to teach (L2T) in which a teacher model could provide high-quality training samples to a student model. However, one major problem of L2T is that the teacher model will only select a subset of the training dataset as the final training data for the student. A learning to teach small-data learning strategy (L2TSDL) is proposed to solve this problem. In this strategy, the teacher model will calculate the importance score for every training sample and help student to make use of all training samples.

scholarly journals Evaluation of Power Insulator Detection Efficiency with the Use of Limited Training Dataset

2020 ◽  
Vol 10 (6) ◽  
pp. 2104
Author(s):  
Michał Tomaszewski ◽  
Paweł Michalski ◽  
Jakub Osuchowski
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Object Detection ◽  
Convolutional Neural Network ◽  
Deep Neural Networks ◽  
Detection Efficiency ◽  
Training Data ◽  
Training Dataset ◽  
Training Set ◽  
Convolutional Network

This article presents an analysis of the effectiveness of object detection in digital images with the application of a limited quantity of input. The possibility of using a limited set of learning data was achieved by developing a detailed scenario of the task, which strictly defined the conditions of detector operation in the considered case of a convolutional neural network. The described solution utilizes known architectures of deep neural networks in the process of learning and object detection. The article presents comparisons of results from detecting the most popular deep neural networks while maintaining a limited training set composed of a specific number of selected images from diagnostic video. The analyzed input material was recorded during an inspection flight conducted along high-voltage lines. The object detector was built for a power insulator. The main contribution of the presented papier is the evidence that a limited training set (in our case, just 60 training frames) could be used for object detection, assuming an outdoor scenario with low variability of environmental conditions. The decision of which network will generate the best result for such a limited training set is not a trivial task. Conducted research suggests that the deep neural networks will achieve different levels of effectiveness depending on the amount of training data. The most beneficial results were obtained for two convolutional neural networks: the faster region-convolutional neural network (faster R-CNN) and the region-based fully convolutional network (R-FCN). Faster R-CNN reached the highest AP (average precision) at a level of 0.8 for 60 frames. The R-FCN model gained a worse AP result; however, it can be noted that the relationship between the number of input samples and the obtained results has a significantly lower influence than in the case of other CNN models, which, in the authors’ assessment, is a desired feature in the case of a limited training set.

scholarly journals AIRBORNE HYPERSPECTRAL REMOTE SENSING FOR IDENTIFICATION GRASSLAND VEGETATION

2015 ◽  
Vol XL-3/W3 ◽  
pp. 427-431 ◽  
Author(s):  
P. Burai ◽  
T. Tomor ◽  
L. Bekő ◽  
B. Deák
Keyword(s):  
Image Classification ◽  
Learning Algorithm ◽  
Training Sample ◽  
Hyperspectral Data ◽  
Training Dataset ◽  
Classification Methods ◽  
Grassland Vegetation ◽  
Training Samples ◽  
Almost All ◽  
Noise Fraction

In our study we classified grassland vegetation types of an alkali landscape (Eastern Hungary), using different image classification methods for hyperspectral data. Our aim was to test the applicability of hyperspectral data in this complex system using various image classification methods. To reach the highest classification accuracy, we compared the performance of traditional image classifiers, machine learning algorithm, feature extraction (MNF-transformation) and various sizes of training dataset. Hyperspectral images were acquired by an AISA EAGLE II hyperspectral sensor of 128 contiguous bands (400–1000 nm), a spectral sampling of 5 nm bandwidth and a ground pixel size of 1 m. We used twenty vegetation classes which were compiled based on the characteristic dominant species, canopy height, and total vegetation cover. Image classification was applied to the original and MNF (minimum noise fraction) transformed dataset using various training sample sizes between 10 and 30 pixels. In the case of the original bands, both SVM and RF classifiers provided high accuracy for almost all classes irrespectively of the number of the training pixels. We found that SVM and RF produced the best accuracy with the first nine MNF transformed bands. Our results suggest that in complex open landscapes, application of SVM can be a feasible solution, as this method provides higher accuracies compared to RF and MLC. SVM was not sensitive for the size of the training samples, which makes it an adequate tool for cases when the available number of training pixels are limited for some classes.

scholarly journals Iterative Metric Learning for Imbalance Data Classification

2018 ◽  
Author(s):  
Nan Wang ◽  
Xibin Zhao ◽  
Yu Jiang ◽  
Yue Gao
Keyword(s):  
Learning Strategy ◽  
State Of The Art ◽  
Metric Learning ◽  
Training Data ◽  
Imbalance Data ◽  
Data Space ◽  
Software Defect ◽  
Training Samples ◽  
Testing Data ◽  
Data Program

In many classification applications, the amount of data from different categories usually vary significantly, such as software defect predication and medical diagnosis. Under such circumstances, it is essential to propose a proper method to solve the imbalance issue among the data. However, most of the existing methods mainly focus on improving the performance of classifiers rather than searching for an appropriate way to find an effective data space for classification. In this paper, we propose a method named Iterative Metric Learning (IML) to explore the correlations among imbalance data and construct an effective data space for classification. Given the imbalance training data, it is important to select a subset of training samples for each testing data. Thus, we aim to find a more stable neighborhood for testing data using the iterative metric learning strategy. To evaluate the effectiveness of the proposed method, we have conducted experiments on two groups of dataset, i.e., the NASA Metrics Data Program (NASA) dataset and UCI Machine Learning Repository (UCI) dataset. Experimental results and comparisons with state-of-the-art methods have exhibited better performance of our proposed method.

scholarly journals Nanopore callers for epigenetics from limited supervised data

2021 ◽  
Author(s):  
Brian Yao ◽  
Chloe Hsu ◽  
Gal Goldner ◽  
Yael Michaeli ◽  
Yuval Ebenstein ◽  
...  
Keyword(s):  
Incomplete Data ◽  
Deep Neural Networks ◽  
Markov Models ◽  
Hidden Markov ◽  
Training Data ◽  
Supervised Machine Learning ◽  
Training Dataset ◽  
Nanopore Sequencing ◽  
Base Modifications ◽  
Sequencing Platforms

Nanopore sequencing platforms combined with supervised machine learning (ML) have been effective at detecting base modifications in DNA such as 5mC and 6mA. These ML-based nanopore callers have typically been trained on data that span all modifications on all possible DNA k-mer backgrounds—a complete training dataset. However, as nanopore technology is pushed to more and more epigenetic modifications, such complete training data will not be feasible to obtain. Nanopore calling has historically been performed with Hidden Markov Models (HMMs) that cannot make successful calls for k-mer contexts not seen during training because of their independent emission distributions. However, deep neural networks (DNNs), which share parameters across contexts, are increasingly being used as callers, often outperforming their HMM cousins. It stands to reason that a DNN approach should be able to better generalize to unseen k-mer contexts. Indeed, herein we demonstrate that a common DNN approach (DeepSignal) outperforms a common HMM approach (Nanopolish) in the incomplete data setting. Furthermore, we propose a novel hybrid HMM-DNN approach, Amortized-HMM, that outperforms both the pure HMM and DNN approaches on 5mC calling when the training data are incomplete. Such an approach is expected to be useful for calling 5hmC and combinations of cytosine modifications, where complete training data are not likely to be available.

scholarly journals Classifiers ensemble training on unbalanced samples in the analysis of the network segments state

2021 ◽  
Vol 16 (93) ◽  
pp. 109-119
Author(s):  
Ilya S. Lebedev ◽  
Keyword(s):  
Imbalanced Data ◽  
Training Sample ◽  
Training Data ◽  
Model Complexity ◽  
Ensemble Averaging ◽  
Destructive Effect ◽  
Security Monitoring ◽  
Training Samples ◽  
The Internet Of Things ◽  
Training Sets

The relevance of the topic considered in the article lies in solving problematic issues of identifying rare events in imbalance conditions in training sets. The purpose of the study is to analyze the capabilities of a classifier’s ensemble trained on different imbalanced data subsets. The features of the heterogeneous segments state analysis of the Internet of Things network infrastructure based on machine learning methods are considered. The prerequisites for the unbalanced data emergence during the training samples formation are indicated. A solution based on the use of a classifier’s ensemble trained on various training samples with classified events imbalance is proposed. The possibility analysis of using unbalanced training sets for a classifier’s ensemble averaging of errors occurs due to the collective voting procedure, is given. An experiment was carried out using weak classifying algorithms. The estimation of features values distributions in test and training subsets is carried out. The classification results are obtained for the ensemble and each classifier separately. An imbalance is investigated consists in the events number ratios violation a certain type within one class in the training data subsets. The data absence in the training sample leads to an increase in the scatter effect responses is averaged by an increase in the model complexity including various classifying algorithms in its composition. The proposed approach can be applied in information security monitoring systems. A proposed solution feature is the ability to scale and combine it by adding new classifying algorithms. In the future, it is possible to make changes during operation to the classification algorithms composition, it makes possible to increase the indicators of the identifying accuracy of a potential destructive effect.

scholarly journals Abstraction and Association: Cross-Modal Retrieval Based on Consistency between Semantic Structures

2020 ◽  
Vol 2020 ◽  
pp. 1-17
Author(s):  
Qibin Zheng ◽  
Xiaoguang Ren ◽  
Yi Liu ◽  
Wei Qin
Keyword(s):  
Learning Strategy ◽  
Reference Points ◽  
Training Data ◽  
Semantic Structure ◽  
Retrieval Method ◽  
Multiple Modalities ◽  
Training Samples ◽  
High Level ◽  
Active Learning Strategy ◽  
The Relationship

Cross-modal retrieval aims to find relevant data of different modalities, such as images and text. In order to bridge the modality gap, most existing methods require a lot of coupled sample pairs as training data. To reduce the demands for training data, we propose a cross-modal retrieval framework that utilizes both coupled and uncoupled samples. The framework consists of two parts: Abstraction that aims to provide high-level single-modal representations with uncoupled samples; then, Association links different modalities through a few coupled training samples. Moreover, under this framework, we implement a cross-modal retrieval method based on the consistency between the semantic structure of multiple modalities. First, both images and text are represented with the semantic structure-based representation, which represents each sample as its similarity from the reference points that are generated from single-modal clustering. Then, the reference points of different modalities are aligned through an active learning strategy. Finally, the cross-modal similarity can be measured with the consistency between the semantic structures. The experiment results demonstrate that given proper abstraction of single-modal data, the relationship between different modalities can be simplified, and even limited coupled cross-modal training data are sufficient for satisfactory retrieval accuracy.

scholarly journals MODIFIED CORRELATION WEIGHT K-NEAREST NEIGHBOR CLASSIFIER USING TRAINING DATASET CLEANING METHOD

2021 ◽  
Vol 32 (2) ◽  
pp. 20-25
Author(s):  
Efraim Kurniawan Dairo Kette
Keyword(s):  
Nearest Neighbor ◽  
Training Sample ◽  
Classification Performance ◽  
Training Data ◽  
Training Dataset ◽  
Classification Problems ◽  
K Nearest Neighbor ◽  
Neighborhood Structure ◽  
Data Set ◽  
Sample Data

In pattern recognition, the k-Nearest Neighbor (kNN) algorithm is the simplest non-parametric algorithm. Due to its simplicity, the model cases and the quality of the training data itself usually influence kNN algorithm classification performance. Therefore, this article proposes a sparse correlation weight model, combined with the Training Data Set Cleaning (TDC) method by Classification Ability Ranking (CAR) called the CAR classification method based on Coefficient-Weighted kNN (CAR-CWKNN) to improve kNN classifier performance. Correlation weight in Sparse Representation (SR) has been proven can increase classification accuracy. The SR can show the 'neighborhood' structure of the data, which is why it is very suitable for classification based on the Nearest Neighbor. The Classification Ability (CA) function is applied to classify the best training sample data based on rank in the cleaning stage. The Leave One Out (LV1) concept in the CA works by cleaning data that is considered likely to have the wrong classification results from the original training data, thereby reducing the influence of the training sample data quality on the kNN classification performance. The results of experiments with four public UCI data sets related to classification problems show that the CAR-CWKNN method provides better performance in terms of accuracy.

scholarly journals HaS-Net: A Heal and Select Mechanism to Securely Train DNNs against Backdoor Attacks

2021 ◽  
Author(s):  
Hassan Ali ◽  
Surya Nepal ◽  
Salil S. Kanhere ◽  
Sanjay K. Jha
Keyword(s):  
Network Architecture ◽  
Deep Neural Networks ◽  
State Of The Art ◽  
Training Data ◽  
Network Architectures ◽  
Success Rates ◽  
Defense Strategies ◽  
Consumer Complaint ◽  
Training Samples ◽  
Novel Variant

<div>We have witnessed the continuing arms race between backdoor attacks and the corresponding defense strategies on Deep Neural Networks (DNNs). However, most state-of-the-art defenses rely on the statistical sanitization of <i>inputs</i> or <i>latent DNN representations</i> to capture trojan behavior. In this paper, we first challenge the robustness of many recently reported defenses by introducing a novel variant of the targeted backdoor attack, called <i>low-confidence backdoor attack</i>. <i>Low-confidence attack</i> inserts the backdoor by assigning uniformly distributed probabilistic labels to the poisoned training samples, and is applicable to many practical scenarios such as Federated Learning and model-reuse cases. We evaluate our attack against five state-of-the-art defense methods, viz., STRIP, Gradient-Shaping, Februus, ULP-defense and ABS-defense, under the same threat model as assumed by the respective defenses and achieve Attack Success Rates (ASRs) of 99\%, 63.73%, 91.2%, 80% and 100%, respectively. After carefully studying the properties of the state-of-the-art attacks, including low-confidence attacks, we present <i>HaS-Net</i>, a mechanism to securely train DNNs against a number of backdoor attacks under the data-collection scenario. For this purpose, we use a reasonably small healing dataset, approximately 2% to 15% the size of training data, to heal the network at each iteration. We evaluate our defense for different datasets---Fashion-MNIST, CIFAR-10, Celebrity Face, Consumer Complaint and Urban Sound---and network architectures---MLPs, 2D-CNNs, 1D-CNNs---and against several attack configurations---standard backdoor attacks, invisible backdoor attacks, label-consistent attack and all-trojan backdoor attack, including their low-confidence variants. Our experiments show that <i>HaS-Nets</i> can decrease ASRs from over 90% to less than 15%, independent of the dataset, attack configuration and network architecture.</div>

scholarly journals A Novel Framework Based on Mask R-CNN and Histogram Thresholding for Scalable Segmentation of New and Old Rural Buildings

2021 ◽  
Vol 13 (6) ◽  
pp. 1070
Author(s):  
Ying Li ◽  
Weipan Xu ◽  
Haohui Chen ◽  
Junhao Jiang ◽  
Xun Li
Keyword(s):  
Rural Areas ◽  
Deep Neural Networks ◽  
Training Data ◽  
The Novel ◽  
Remote Sensing Images ◽  
Average Precision ◽  
Training Samples ◽  
Rural Buildings ◽  
Segmentation Task ◽  
Instance Segmentation

Mapping new and old buildings are of great significance for understanding socio-economic development in rural areas. In recent years, deep neural networks have achieved remarkable building segmentation results in high-resolution remote sensing images. However, the scarce training data and the varying geographical environments have posed challenges for scalable building segmentation. This study proposes a novel framework based on Mask R-CNN, named Histogram Thresholding Mask Region-Based Convolutional Neural Network (HTMask R-CNN), to extract new and old rural buildings even when the label is scarce. The framework adopts the result of single-object instance segmentation from the orthodox Mask R-CNN. Further, it classifies the rural buildings into new and old ones based on a dynamic grayscale threshold inferred from the result of a two-object instance segmentation task where training data is scarce. We found that the framework can extract more buildings and achieve a much higher mean Average Precision (mAP) than the orthodox Mask R-CNN model. We tested the novel framework’s performance with increasing training data and found that it converged even when the training samples were limited. This framework’s main contribution is to allow scalable segmentation by using significantly fewer training samples than traditional machine learning practices. That makes mapping China’s new and old rural buildings viable.

Sign in / Sign up

Export Citation Format

Share Document