scholarly journals Constructing Multiple Tasks for Augmentation: Improving Neural Image Classification with K-Means Features

2020 ◽  
Vol 34 (07) ◽  
pp. 10877-10884
Author(s):  
Tao Gui ◽  
Lizhi Qing ◽  
Qi Zhang ◽  
Jiacheng Ye ◽  
Hang Yan ◽  
...  
Keyword(s):  
Data Augmentation ◽  
Learning Approach ◽  
Main Task ◽  
Single Task ◽  
Task Learning ◽  
Multiple Tasks ◽  
Meta Learning ◽  
Meta Optimization ◽  
The Impact ◽  
Human Specific

Multi-task learning (MTL) has received considerable attention, and numerous deep learning applications benefit from MTL with multiple objectives. However, constructing multiple related tasks is difficult, and sometimes only a single task is available for training in a dataset. To tackle this problem, we explored the idea of using unsupervised clustering to construct a variety of auxiliary tasks from unlabeled data or existing labeled data. We found that some of these newly constructed tasks could exhibit semantic meanings corresponding to certain human-specific attributes, but some were non-ideal. In order to effectively reduce the impact of non-ideal auxiliary tasks on the main task, we further proposed a novel meta-learning-based multi-task learning approach, which trained the shared hidden layers on auxiliary tasks, while the meta-optimization objective was to minimize the loss on the main task, ensuring that the optimizing direction led to an improvement on the main task. Experimental results across five image datasets demonstrated that the proposed method significantly outperformed existing single task learning, semi-supervised learning, and some data augmentation methods, including an improvement of more than 9% on the Omniglot dataset.

scholarly journals Guitar Chord Sensing and Recognition Using Multi-Task Learning and Physical Data Augmentation with Robotics

Sensors ◽  
2020 ◽  
Vol 20 (21) ◽  
pp. 6077
Author(s):  
Gerelmaa Byambatsogt ◽  
Lodoiravsal Choimaa ◽  
Gou Koutaki
Keyword(s):  
Deep Learning ◽  
Data Augmentation ◽  
Recognition Task ◽  
Learning Model ◽  
Training Data ◽  
Single Task ◽  
Task Learning ◽  
Physical Data ◽  
Music Information ◽  
Chord Recognition

In recent years, many researchers have shown increasing interest in music information retrieval (MIR) applications, with automatic chord recognition being one of the popular tasks. Many studies have achieved/demonstrated considerable improvement using deep learning based models in automatic chord recognition problems. However, most of the existing models have focused on simple chord recognition, which classifies the root note with the major, minor, and seventh chords. Furthermore, in learning-based recognition, it is critical to collect high-quality and large amounts of training data to achieve the desired performance. In this paper, we present a multi-task learning (MTL) model for a guitar chord recognition task, where the model is trained using a relatively large-vocabulary guitar chord dataset. To solve data scarcity issues, a physical data augmentation method that directly records the chord dataset from a robotic performer is employed. Deep learning based MTL is proposed to improve the performance of automatic chord recognition with the proposed physical data augmentation dataset. The proposed MTL model is compared with four baseline models and its corresponding single-task learning model using two types of datasets, including a human dataset and a human combined with the augmented dataset. The proposed methods outperform the baseline models, and the results show that most scores of the proposed multi-task learning model are better than those of the corresponding single-task learning model. The experimental results demonstrate that physical data augmentation is an effective method for increasing the dataset size for guitar chord recognition tasks.

scholarly journals Continuous self-adaptive optimization to learn multi-task multi-agent

2021 ◽  
Author(s):  
Wenqian Liang ◽  
Ji Wang ◽  
Weidong Bao ◽  
Xiaomin Zhu ◽  
Qingyong Wang ◽  
...  
Keyword(s):  
Real World ◽  
Learning Algorithm ◽  
Superior Performance ◽  
Adaptive Optimization ◽  
Single Task ◽  
Multiple Tasks ◽  
Meta Learning ◽  
Gradient Based ◽  
Multi Agent ◽  
Self Adaptive

AbstractMulti-agent reinforcement learning (MARL) methods have shown superior performance to solve a variety of real-world problems focusing on learning distinct policies for individual tasks. These approaches face problems when applied to the non-stationary real-world: agents trained in specialized tasks cannot achieve satisfied generalization performance across multiple tasks; agents have to learn and store specialized policies for individual task and reliable identities of tasks are hardly observable in practice. To address the challenge continuously adapting to multiple tasks in MARL, we formalize the problem into a two-stage curriculum. Single-task policies are learned with MARL approaches, after that we develop a gradient-based Self-Adaptive Meta-Learning algorithm, SAML, that cannot only distill single-task policies into a unified policy but also can facilitate the unified policy to continuously adapt to new incoming tasks. In addition, to validate the continuous adaptation performance on complex task, we extend the widely adopted StarCraft benchmark SMAC and develop a new multi-task multi-agent StarCraft environment, Meta-SMAC, for testing various aspects of continuous adaptation method. Our experiments with a population of agents show that our method enables significantly more efficient adaptation than reactive baselines across different scenarios.

scholarly journals Multi-task Learning and Catastrophic Forgetting in Continual Reinforcement Learning

10.29007/g7bg ◽  
2019 ◽  
Author(s):  
João Ribeiro ◽  
Francisco Melo ◽  
João Dias
Keyword(s):  
Reinforcement Learning ◽  
Learning Algorithm ◽  
Single Task ◽  
Similar Performance ◽  
The Third ◽  
Task Learning ◽  
Multiple Tasks ◽  
Similar Task ◽  
Reinforcement Learning Algorithm

In this paper we investigate two hypothesis regarding the use of deep reinforcement learning in multiple tasks. The first hypothesis is driven by the question of whether a deep reinforcement learning algorithm, trained on two similar tasks, is able to outperform two single-task, individually trained algorithms, by more efficiently learning a new, similar task, that none of the three algorithms has encountered before. The second hypothesis is driven by the question of whether the same multi-task deep RL algorithm, trained on two similar tasks and augmented with elastic weight consolidation (EWC), is able to retain similar performance on the new task, as a similar algorithm without EWC, whilst being able to overcome catastrophic forgetting in the two previous tasks. We show that a multi-task Asynchronous Advantage Actor-Critic (GA3C) algorithm, trained on Space Invaders and Demon Attack, is in fact able to outperform two single-tasks GA3C versions, trained individually for each single-task, when evaluated on a new, third task—namely, Phoenix. We also show that, when training two trained multi-task GA3C algorithms on the third task, if one is augmented with EWC, it is not only able to achieve similar performance on the new task, but also capable of overcoming a substantial amount of catastrophic forgetting on the two previous tasks.

scholarly journals Learning Sparse Sharing Architectures for Multiple Tasks

2020 ◽  
Vol 34 (05) ◽  
pp. 8936-8943
Author(s):  
Tianxiang Sun ◽  
Yunfan Shao ◽  
Xiaonan Li ◽  
Pengfei Liu ◽  
Hang Yan ◽  
...  
Keyword(s):  
Learning Models ◽  
Single Task ◽  
Sequence Labeling ◽  
Task Learning ◽  
Multiple Tasks ◽  
Base Network ◽  
Parameter Sharing

Most existing deep multi-task learning models are based on parameter sharing, such as hard sharing, hierarchical sharing, and soft sharing. How choosing a suitable sharing mechanism depends on the relations among the tasks, which is not easy since it is difficult to understand the underlying shared factors among these tasks. In this paper, we propose a novel parameter sharing mechanism, named Sparse Sharing. Given multiple tasks, our approach automatically finds a sparse sharing structure. We start with an over-parameterized base network, from which each task extracts a subnetwork. The subnetworks of multiple tasks are partially overlapped and trained in parallel. We show that both hard sharing and hierarchical sharing can be formulated as particular instances of the sparse sharing framework. We conduct extensive experiments on three sequence labeling tasks. Compared with single-task models and three typical multi-task learning baselines, our proposed approach achieves consistent improvement while requiring fewer parameters.

scholarly journals Rethinking Data Augmentation for Low-Resource Neural Machine Translation: A Multi-Task Learning Approach

2021 ◽  
Author(s):  
Víctor M. Sánchez-Cartagena ◽  
Miquel Esplà-Gomis ◽  
Juan Antonio Pérez-Ortiz ◽  
Felipe Sánchez-Martínez
Keyword(s):  
Machine Translation ◽  
Data Augmentation ◽  
Learning Approach ◽  
Neural Machine Translation ◽  
Low Resource ◽  
Task Learning

scholarly journals A deep learning approach for complex microstructure inference

2021 ◽  
Vol 12 (1) ◽  
Author(s):  
Ali Riza Durmaz ◽  
Martin Müller ◽  
Bo Lei ◽  
Akhil Thomas ◽  
Dominik Britz ◽  
...  
Keyword(s):  
Deep Learning ◽  
Data Augmentation ◽  
Electron Backscatter Diffraction ◽  
Specimen Preparation ◽  
Learning Approach ◽  
Plausible Model ◽  
Lath Bainite ◽  
Image Context ◽  
Visualization Techniques ◽  
The Impact

AbstractAutomated, reliable, and objective microstructure inference from micrographs is essential for a comprehensive understanding of process-microstructure-property relations and tailored materials development. However, such inference, with the increasing complexity of microstructures, requires advanced segmentation methodologies. While deep learning offers new opportunities, an intuition about the required data quality/quantity and a methodological guideline for microstructure quantification is still missing. This, along with deep learning’s seemingly intransparent decision-making process, hampers its breakthrough in this field. We apply a multidisciplinary deep learning approach, devoting equal attention to specimen preparation and imaging, and train distinct U-Net architectures with 30–50 micrographs of different imaging modalities and electron backscatter diffraction-informed annotations. On the challenging task of lath-bainite segmentation in complex-phase steel, we achieve accuracies of 90% rivaling expert segmentations. Further, we discuss the impact of image context, pre-training with domain-extrinsic data, and data augmentation. Network visualization techniques demonstrate plausible model decisions based on grain boundary morphology.

A Deep Learning based Arabic Script Recognition System: Benchmark on KHAT

2020 ◽  
Vol 17 (3) ◽  
pp. 299-305 ◽  
Author(s):  
Riaz Ahmad ◽  
Saeeda Naz ◽  
Muhammad Afzal ◽  
Sheikh Rashid ◽  
Marcus Liwicki ◽  
...  
Keyword(s):  
Deep Learning ◽  
Character Recognition ◽  
Data Augmentation ◽  
Short Term Memory ◽  
Recognition System ◽  
Learning Approach ◽  
Arabic Text ◽  
Data Set ◽  
Processing Step ◽  
Handwritten Arabic

This paper presents a deep learning benchmark on a complex dataset known as KFUPM Handwritten Arabic TexT (KHATT). The KHATT data-set consists of complex patterns of handwritten Arabic text-lines. This paper contributes mainly in three aspects i.e., (1) pre-processing, (2) deep learning based approach, and (3) data-augmentation. The pre-processing step includes pruning of white extra spaces plus de-skewing the skewed text-lines. We deploy a deep learning approach based on Multi-Dimensional Long Short-Term Memory (MDLSTM) networks and Connectionist Temporal Classification (CTC). The MDLSTM has the advantage of scanning the Arabic text-lines in all directions (horizontal and vertical) to cover dots, diacritics, strokes and fine inflammation. The data-augmentation with a deep learning approach proves to achieve better and promising improvement in results by gaining 80.02% Character Recognition (CR) over 75.08% as baseline.

An Active Learning Approach to Teaching Social Determinants of Health

2020 ◽  
Vol 6 (4) ◽  
pp. 266-273
Author(s):  
Jeanita W. Richardson
Keyword(s):  
Active Learning ◽  
Learning Style ◽  
Social Determinants ◽  
Learning Approach ◽  
Learning Style Preferences ◽  
Open Discussion ◽  
Approach To Teaching ◽  
The Individual ◽  
Style Preferences ◽  
The Impact

This active learning exercise is designed to deconstruct the impact of social determinants through the assumption of randomly selected personas. As an active learning exercise, it provides opportunities for discussion, problem solving, writing, and synthesis, while incorporating multiple learning style preferences. Part 1 involves assessing the individual social determinants at work. Part 2 involves exploring ways said determinants can enhance community health through collaboration. Assumption of personas unlike one’s own facilitates an open discussion of social position and ranges of factors influential to health without potentially evoking a sense of defensiveness associated with personal privilege (or the lack thereof).

Sign in / Sign up

Export Citation Format

Share Document