scholarly journals Coupled-View Deep Classifier Learning from Multiple Noisy Annotators

2020 ◽  
Vol 34 (04) ◽  
pp. 4667-4674 ◽  
Author(s):  
Shikun Li ◽  
Shiming Ge ◽  
Yingying Hua ◽  
Chunhui Zhang ◽  
Hao Wen ◽  
...  
Keyword(s):  
Deep Neural Networks ◽  
Teaching Strategy ◽  
Real Data ◽  
Peer Networks ◽  
Bayes Classifier ◽  
Learning Problem ◽  
Learning Abilities ◽  
Small Loss ◽  
Label Aggregation ◽  
Classifier Learning

Typically, learning a deep classifier from massive cleanly annotated instances is effective but impractical in many real-world scenarios. An alternative is collecting and aggregating multiple noisy annotations for each instance to train the classifier. Inspired by that, this paper proposes to learn deep classifier from multiple noisy annotators via a coupled-view learning approach, where the learning view from data is represented by deep neural networks for data classification and the learning view from labels is described by a Naive Bayes classifier for label aggregation. Such coupled-view learning is converted to a supervised learning problem under the mutual supervision of the aggregated and predicted labels, and can be solved via alternate optimization to update labels and refine the classifiers. To alleviate the propagation of incorrect labels, small-loss metric is proposed to select reliable instances in both views. A co-teaching strategy with class-weighted loss is further leveraged in the deep classifier learning, which uses two networks with different learning abilities to teach each other, and the diverse errors introduced by noisy labels can be filtered out by peer networks. By these strategies, our approach can finally learn a robust data classifier which less overfits to label noise. Experimental results on synthetic and real data demonstrate the effectiveness and robustness of the proposed approach.

scholarly journals Transfer Incremental Learning Using Data Augmentation

2018 ◽  
Vol 8 (12) ◽  
pp. 2512 ◽  
Author(s):  
Ghouthi Boukli Hacene ◽  
Vincent Gripon ◽  
Nicolas Farrugia ◽  
Matthieu Arzel ◽  
Michel Jezequel
Keyword(s):  
Incremental Learning ◽  
Deep Neural Networks ◽  
Data Augmentation ◽  
State Of The Art ◽  
Low Complexity ◽  
Computational Power ◽  
Learning Problem ◽  
Learning Techniques ◽  
Using Data ◽  
Selection Of

Deep learning-based methods have reached state of the art performances, relying on a large quantity of available data and computational power. Such methods still remain highly inappropriate when facing a major open machine learning problem, which consists of learning incrementally new classes and examples over time. Combining the outstanding performances of Deep Neural Networks (DNNs) with the flexibility of incremental learning techniques is a promising venue of research. In this contribution, we introduce Transfer Incremental Learning using Data Augmentation (TILDA). TILDA is based on pre-trained DNNs as feature extractors, robust selection of feature vectors in subspaces using a nearest-class-mean based technique, majority votes and data augmentation at both the training and the prediction stages. Experiments on challenging vision datasets demonstrate the ability of the proposed method for low complexity incremental learning, while achieving significantly better accuracy than existing incremental counterparts.

scholarly journals A Bayesian Classifier Learning Algorithm Based on Optimization Model

2013 ◽  
Vol 2013 ◽  
pp. 1-9 ◽  
Author(s):  
Sanyang Liu ◽  
Mingmin Zhu ◽  
Youlong Yang
Keyword(s):  
Objective Function ◽  
Optimization Model ◽  
Learning Algorithm ◽  
Classification Performance ◽  
Bayesian Classifier ◽  
Bayes Classifier ◽  
Independence Assumption ◽  
Maximum Value ◽  
Chi Squared ◽  
Classifier Learning

Naive Bayes classifier is a simple and effective classification method, but its attribute independence assumption makes it unable to express the dependence among attributes and affects its classification performance. In this paper, we summarize the existing improved algorithms and propose a Bayesian classifier learning algorithm based on optimization model (BC-OM). BC-OM uses the chi-squared statistic to estimate the dependence coefficients among attributes, with which it constructs the objective function as an overall measure of the dependence for a classifier structure. Therefore, a problem of searching for an optimal classifier can be turned into finding the maximum value of the objective function in feasible fields. In addition, we have proved the existence and uniqueness of the numerical solution. BC-OM offers a new opinion for the research of extended Bayesian classifier. Theoretical and experimental results show that the new algorithm is correct and effective.

scholarly journals Learning the Structure of Hub Network Based on Graph Model

2019 ◽  
Vol 37 (6) ◽  
pp. 1320-1325
Author(s):  
Chongyang Zhang ◽  
Xiao Guo ◽  
Hai Zhang
Keyword(s):  
Control Parameter ◽  
Structure Learning ◽  
Model Simulation ◽  
Graph Model ◽  
Real Data ◽  
Learning Problem ◽  
Coordinate Descent Algorithm ◽  
Neighborhood Selection ◽  
Hub Network ◽  
Selection Framework

In this paper, we focus on the structure learning problem of the hub network. In the neighborhood selection framework, we use the L1 and L2 regularizers to incorporate the sparse and group prior of the hub network, so as to make the network easier to generate Hub. We employ the coordinate descent algorithm to solve the resulting model. Simulation and real data analysis show that the proposed method is effective and applicable in parameter estimation and model selection, and results illustrate the influence ability of the control parameter on the model.

scholarly journals Interpretable, Data-Efficient and Verifiable Autonomy with High-Level Knowledge

2020 ◽  
Author(s):  
Zhe Xu
Keyword(s):  
Artificial Intelligence ◽  
Neural Networks ◽  
Deep Neural Networks ◽  
Autonomous Systems ◽  
Real Data ◽  
Data Driven ◽  
Knowledge Representations ◽  
Limited Availability ◽  
High Level ◽  
Level Performance

<p>Despite the fact that artificial intelligence boosted with data-driven methods (e.g., deep neural networks) has surpassed human-level performance in various tasks, its application to autonomous</p> <p>systems still faces fundamental challenges such as lack of interpretability, intensive need for data and lack of verifiability. In this overview paper, I overview some attempts to address these fundamental challenges by explaining, guiding and verifying autonomous systems, taking into account limited availability of simulated and real data, the expressivity of high-level</p> <p>knowledge representations and the uncertainties of the underlying model. Specifically, this paper covers learning high-level knowledge from data for interpretable autonomous systems,</p><p>guiding autonomous systems with high-level knowledge, and</p><p>verifying and controlling autonomous systems against high-level specifications.</p>

scholarly journals Learning with small amount of data on various task

2020 ◽  
Author(s):  
Stefanie
Keyword(s):  
Deep Neural Networks ◽  
Learning Strategy ◽  
Teaching Strategy ◽  
Training Sample ◽  
Learning To Teach ◽  
Training Data ◽  
Training Dataset ◽  
Small Data ◽  
Training Samples ◽  
Teacher Model

As a student, I am learning knowledge with the help of teachers and the teacher plays a crucial role in our life. A wonderful instructor is able to teach a student with appropriate teaching materials. Therefore, in this project, I explore a teaching strategy called learning to teach (L2T) in which a teacher model could provide high-quality training samples to a student model. However, one major problem of L2T is that the teacher model will only select a subset of the training dataset as the final training data for the student. Learning to teach small-data learning strategy (L2TSDL) is proposed to solve this problem. In this strategy, the teacher model will calculate the importance score for every training sample and help students to make use of all training samples. To demonstrate the advantage of the proposed approach over L2T, I take the training of different deep neural networks (DNN) on image classification task as an exampleand show that L2TSDL could achieve good performance on both large and small dataset.

A cross-corpus study of subjectivity identification using unsupervised learning

2011 ◽  
Vol 18 (3) ◽  
pp. 375-397 ◽  
Author(s):  
DONG WANG ◽  
YANG LIU
Keyword(s):  
Unsupervised Learning ◽  
Expectation Maximization ◽  
Real Data ◽  
Bayes Classifier ◽  
Initial Training ◽  
Training Set ◽  
High Confidence ◽  
Learning Methods ◽  
Corpus Study ◽  
Impacting Factors

AbstractIn this study, we investigate using unsupervised generative learning methods for subjectivity detection across different domains. We create an initial training set using simple lexicon information and then evaluate two iterative learning methods with a base naive Bayes classifier to learn from unannotated data. The first method is self-training, which adds instances with high confidence into the training set in each iteration. The second is a calibrated EM (expectation-maximization) method where we calibrate the posterior probabilities from EM such that the class distribution is similar to that in the real data. We evaluate both approaches on three different domains: movie data, news resource, and meeting dialogues, and we found that in some cases the unsupervised learning methods can achieve performance close to the fully supervised setup. We perform a thorough analysis to examine factors, such as self-labeling accuracy of the initial training set in unsupervised learning, the accuracy of the added examples in self-training, and the size of the initial training set in different methods. Our experiments and analysis show inherent differences across domains and impacting factors explaining the model behaviors.

scholarly journals Pre-Training Acquisition Functions by Deep Reinforcement Learning for Fixed Budget Active Learning

2021 ◽  
Author(s):  
Yusuke Taguchi ◽  
Hideitsu Hino ◽  
Keisuke Kameyama
Keyword(s):  
Neural Networks ◽  
Reinforcement Learning ◽  
Active Learning ◽  
Supervised Learning ◽  
Deep Neural Networks ◽  
Learning Algorithm ◽  
Learning Problem ◽  
Q Learning ◽  
Fixed Budget ◽  
Active Learner

AbstractThere are many situations in supervised learning where the acquisition of data is very expensive and sometimes determined by a user’s budget. One way to address this limitation is active learning. In this study, we focus on a fixed budget regime and propose a novel active learning algorithm for the pool-based active learning problem. The proposed method performs active learning with a pre-trained acquisition function so that the maximum performance can be achieved when the number of data that can be acquired is fixed. To implement this active learning algorithm, the proposed method uses reinforcement learning based on deep neural networks as as a pre-trained acquisition function tailored for the fixed budget situation. By using the pre-trained deep Q-learning-based acquisition function, we can realize the active learner which selects a sample for annotation from the pool of unlabeled samples taking the fixed-budget situation into account. The proposed method is experimentally shown to be comparable with or superior to existing active learning methods, suggesting the effectiveness of the proposed approach for the fixed-budget active learning.

scholarly journals Dynamic Ride-Hailing with Electric Vehicles

2021 ◽  
Author(s):  
Nicholas D. Kullman ◽  
Martin Cousineau ◽  
Justin C. Goodson ◽  
Jorge E. Mendoza
Keyword(s):  
Neural Networks ◽  
Decision Making ◽  
New York ◽  
New York City ◽  
Reinforcement Learning ◽  
Electric Vehicles ◽  
Optimal Policy ◽  
Deep Neural Networks ◽  
Real Data ◽  
Dual Bounds

We consider the problem of an operator controlling a fleet of electric vehicles for use in a ride-hailing service. The operator, seeking to maximize profit, must assign vehicles to requests as they arise as well as recharge and reposition vehicles in anticipation of future requests. To solve this problem, we employ deep reinforcement learning, developing policies whose decision making uses [Formula: see text]-value approximations learned by deep neural networks. We compare these policies against a reoptimization-based policy and against dual bounds on the value of an optimal policy, including the value of an optimal policy with perfect information, which we establish using a Benders-based decomposition. We assess performance on instances derived from real data for the island of Manhattan in New York City. We find that, across instances of varying size, our best policy trained with deep reinforcement learning outperforms the reoptimization approach. We also provide evidence that this policy may be effectively scaled and deployed on larger instances without retraining.

scholarly journals A Data Augmentation Method for War Trauma Using the War Trauma Severity Score and Deep Neural Networks

Electronics ◽  
2021 ◽  
Vol 10 (21) ◽  
pp. 2657
Author(s):  
Jibin Yin ◽  
Pengfei Zhao ◽  
Yi Zhang ◽  
Yi Han ◽  
Shuoyu Wang
Keyword(s):  
Neural Networks ◽  
Large Scale ◽  
Deep Neural Networks ◽  
Data Augmentation ◽  
Real Data ◽  
War Trauma ◽  
Trauma Severity ◽  
Large Scale Analysis ◽  
Medical Evaluation ◽  
Modern Warfare

The demand for large-scale analysis and research of data on trauma from modern warfare is increasing day by day, but the amount of existing data is not sufficient to meet such demand. In this study, an integrated modeling approach incorporating a war trauma severity scoring algorithm (WTSS) and deep neural networks (DNN) is proposed. First, the proposed WTSS, which uses multiple non-linear regression based on the characteristics of war trauma data and the medical evaluation by an expert panel, performed a standardized assessment of an injury and predicts its trauma consequences. Second, to generate virtual injury, based on the probability of occurrence, the injured parts, injury types, and complications were randomly sampled and combined, and then WTSS was used to assess the consequences of the virtual injury. Third, to evaluate the accuracy of the predicted injury consequences, we built a DNN classifier and then trained it with the generated data and tested it with real data. Finally, we used the Delphi method to filter out unreasonable injuries and improve data rationality. The experimental results verified that the proposed approach surpassed the traditional artificial generation methods, achieved a prediction accuracy of 84.43%, and realized large-scale and credible war trauma data augmentation.

scholarly journals On regularization properties of artificial datasets for deep learning

2019 ◽  
Vol 0 (9/2019) ◽  
pp. 13-18
Author(s):  
Karol Antczak
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Deep Neural Networks ◽  
Real Data ◽  
Training Data ◽  
Generation Process ◽  
Data Generation ◽  
Artificial Data ◽  
High Level ◽  
Artificial Datasets

The paper discusses regularization properties of artificial data for deep learning. Artificial datasets allow to train neural networks in the case of a real data shortage. It is demonstrated that the artificial data generation process, described as injecting noise to high-level features, bears several similarities to existing regularization methods for deep neural networks. One can treat this property of artificial data as a kind of “deep” regularization. It is thus possible to regularize hidden layers of the network by generating the training data in a certain way.

Sign in / Sign up

Export Citation Format

Share Document