scholarly journals A Two-Stream Mutual Attention Network for Semi-Supervised Biomedical Segmentation with Noisy Labels

2019 ◽  
Vol 33 ◽  
pp. 4578-4585 ◽  
Author(s):  
Shaobo Min ◽  
Xuejin Chen ◽  
Zheng-Jun Zha ◽  
Feng Wu ◽  
Yongdong Zhang
Keyword(s):  
Supervised Learning ◽  
State Of The Art ◽  
Training Data ◽  
Attention Network ◽  
Attention Model ◽  
Learning Framework ◽  
Propagation Analysis ◽  
Supervised Methods ◽  
Multi Level ◽  
Noisy Labels

Learning-based methods suffer from a deficiency of clean annotations, especially in biomedical segmentation. Although many semi-supervised methods have been proposed to provide extra training data, automatically generated labels are usually too noisy to retrain models effectively. In this paper, we propose a Two-Stream Mutual Attention Network (TSMAN) that weakens the influence of back-propagated gradients caused by incorrect labels, thereby rendering the network robust to unclean data. The proposed TSMAN consists of two sub-networks that are connected by three types of attention models in different layers. The target of each attention model is to indicate potentially incorrect gradients in a certain layer for both sub-networks by analyzing their inferred features using the same input. In order to achieve this purpose, the attention models are designed based on the propagation analysis of noisy gradients at different layers. This allows the attention models to effectively discover incorrect labels and weaken their influence during parameter updating process. By exchanging multi-level features within two-stream architecture, the effects of noisy labels in each sub-network are reduced by decreasing the noisy gradients. Furthermore, a hierarchical distillation is developed to provide reliable pseudo labels for unlabelded data, which further boosts the performance of TSMAN. The experiments using both HVSMR 2016 and BRATS 2015 benchmarks demonstrate that our semi-supervised learning framework surpasses the state-of-the-art fully-supervised results.

scholarly journals Learning from Web Data Using Adversarial Discriminative Neural Networks for Fine-Grained Classification

2019 ◽  
Vol 33 ◽  
pp. 273-280 ◽  
Author(s):  
Xiaoxiao Sun ◽  
Liyi Chen ◽  
Jufeng Yang
Keyword(s):  
Neural Networks ◽  
Large Scale ◽  
State Of The Art ◽  
Training Data ◽  
Web Data ◽  
Fine Grained ◽  
Learning Framework ◽  
Attractive Option ◽  
Public Datasets ◽  
Noisy Labels

Fine-grained classification is absorbed in recognizing the subordinate categories of one field, which need a large number of labeled images, while it is expensive to label these images. Utilizing web data has been an attractive option to meet the demands of training data for convolutional neural networks (CNNs), especially when the well-labeled data is not enough. However, directly training on such easily obtained images often leads to unsatisfactory performance due to factors such as noisy labels. This has been conventionally addressed by reducing the noise level of web data. In this paper, we take a fundamentally different view and propose an adversarial discriminative loss to advocate representation coherence between standard and web data. This is further encapsulated in a simple, scalable and end-to-end trainable multi-task learning framework. We experiment on three public datasets using large-scale web data to evaluate the effectiveness and generalizability of the proposed approach. Extensive experiments demonstrate that our approach performs favorably against the state-of-the-art methods.

scholarly journals Cross-media Multi-level Alignment with Relation Attention Network

2018 ◽  
Author(s):  
Jinwei Qi ◽  
Yuxin Peng ◽  
Yuxin Yuan
Keyword(s):  
State Of The Art ◽  
Multimedia Data ◽  
Attention Network ◽  
Attention Model ◽  
Fine Grained ◽  
Cross Media ◽  
Textual Description ◽  
Correlation Learning ◽  
Media Types ◽  
Multi Level

With the rapid growth of multimedia data, such as image and text, it is a highly challenging problem to effectively correlate and retrieve the data of different media types. Naturally, when correlating an image with textual description, people focus on not only the alignment between discriminative image regions and key words, but also the relations lying in the visual and textual context. Relation understanding is essential for cross-media correlation learning, which is ignored by prior cross-media retrieval works. To address the above issue, we propose Cross-media Relation Attention Network (CRAN) with multi-level alignment. First, we propose visual-language relation attention model to explore both fine-grained patches and their relations of different media types. We aim to not only exploit cross-media fine-grained local information, but also capture the intrinsic relation information, which can provide complementary hints for correlation learning. Second, we propose cross-media multi-level alignment to explore global, local and relation alignments across different media types, which can mutually boost to learn more precise cross-media correlation. We conduct experiments on 2 cross-media datasets, and compare with 10 state-of-the-art methods to verify the effectiveness of proposed approach.

scholarly journals Improving Semi-Supervised Learning for Audio Classification with FixMatch

Electronics ◽  
2021 ◽  
Vol 10 (15) ◽  
pp. 1807
Author(s):  
Sascha Grollmisch ◽  
Estefanía Cano
Keyword(s):  
Neural Networks ◽  
Supervised Learning ◽  
Transfer Learning ◽  
Data Transfer ◽  
State Of The Art ◽  
Training Data ◽  
Audio Classification ◽  
Image Domain ◽  
Full Dataset ◽  
Audio Data

Including unlabeled data in the training process of neural networks using Semi-Supervised Learning (SSL) has shown impressive results in the image domain, where state-of-the-art results were obtained with only a fraction of the labeled data. The commonality between recent SSL methods is that they strongly rely on the augmentation of unannotated data. This is vastly unexplored for audio data. In this work, SSL using the state-of-the-art FixMatch approach is evaluated on three audio classification tasks, including music, industrial sounds, and acoustic scenes. The performance of FixMatch is compared to Convolutional Neural Networks (CNN) trained from scratch, Transfer Learning, and SSL using the Mean Teacher approach. Additionally, a simple yet effective approach for selecting suitable augmentation methods for FixMatch is introduced. FixMatch with the proposed modifications always outperformed Mean Teacher and the CNNs trained from scratch. For the industrial sounds and music datasets, the CNN baseline performance using the full dataset was reached with less than 5% of the initial training data, demonstrating the potential of recent SSL methods for audio data. Transfer Learning outperformed FixMatch only for the most challenging dataset from acoustic scene classification, showing that there is still room for improvement.

scholarly journals Multiple Saliency and Channel Sensitivity Network for Aggregated Convolutional Feature

2019 ◽  
Vol 33 ◽  
pp. 9013-9020
Author(s):  
Xuanlu Xiang ◽  
Zhipeng Wang ◽  
Zhicheng Zhao ◽  
Fei Su
Keyword(s):  
State Of The Art ◽  
Image Representation ◽  
Training Data ◽  
Gram Matrix ◽  
Redundant Information ◽  
Deep Architecture ◽  
Benchmark Datasets ◽  
Supervised Methods ◽  
Ranking Loss ◽  
Effective Channel

In this paper, aiming at two key problems of instance-level image retrieval, i.e., the distinctiveness of image representation and the generalization ability of the model, we propose a novel deep architecture - Multiple Saliency and Channel Sensitivity Network(MSCNet). Specifically, to obtain distinctive global descriptors, an attention-based multiple saliency learning is first presented to highlight important details of the image, and then a simple but effective channel sensitivity module based on Gram matrix is designed to boost the channel discrimination and suppress redundant information. Additionally, in contrast to most existing feature aggregation methods, employing pre-trained deep networks, MSCNet can be trained in two modes: the first one is an unsupervised manner with an instance loss, and another is a supervised manner, which combines classification and ranking loss and only relies on very limited training data. Experimental results on several public benchmark datasets, i.e., Oxford buildings, Paris buildings and Holidays, indicate that the proposed MSCNet outperforms the state-of-the-art unsupervised and supervised methods.

scholarly journals Scene Recognition Based on Recurrent Memorized Attention Network

Electronics ◽  
2020 ◽  
Vol 9 (12) ◽  
pp. 2038
Author(s):  
Xi Shao ◽  
Xuan Zhang ◽  
Guijin Tang ◽  
Bingkun Bao
Keyword(s):  
Recognition Accuracy ◽  
State Of The Art ◽  
Classification Performance ◽  
Scene Recognition ◽  
Experimental Results ◽  
Scene Classification ◽  
Attention Network ◽  
Attention Model ◽  
Scene Image ◽  
Object Based

We propose a new end-to-end scene recognition framework, called a Recurrent Memorized Attention Network (RMAN) model, which performs object-based scene classification by recurrently locating and memorizing objects in the image. Based on the proposed framework, we introduce a multi-task mechanism that contiguously attends on the different essential objects in a scene image and recurrently performs memory fusion of the features of object focused by an attention model to improve the scene recognition accuracy. The experimental results show that the RMAN model has achieved better classification performance on the constructed dataset and two public scene datasets, surpassing state-of-the-art image scene recognition approaches.

scholarly journals Regularized Instance Embedding for Deep Multi-Instance Learning

2019 ◽  
Vol 10 (1) ◽  
pp. 64
Author(s):  
Yi Lin ◽  
Honggang Zhang
Keyword(s):  
Neural Network ◽  
Big Data ◽  
Supervised Learning ◽  
Regularization Method ◽  
Gradient Descent ◽  
State Of The Art ◽  
Stochastic Gradient Descent ◽  
Learning Framework ◽  
Weakly Supervised ◽  
The Cost

In the era of Big Data, multi-instance learning, as a weakly supervised learning framework, has various applications since it is helpful to reduce the cost of the data-labeling process. Due to this weakly supervised setting, learning effective instance representation/embedding is challenging. To address this issue, we propose an instance-embedding regularizer that can boost the performance of both instance- and bag-embedding learning in a unified fashion. Specifically, the crux of the instance-embedding regularizer is to maximize correlation between instance-embedding and underlying instance-label similarities. The embedding-learning framework was implemented using a neural network and optimized in an end-to-end manner using stochastic gradient descent. In experiments, various applications were studied, and the results show that the proposed instance-embedding-regularization method is highly effective, having state-of-the-art performance.

scholarly journals MLRDA: A Multi-Task Semi-Supervised Learning Framework for Drug-Drug Interaction Prediction

2019 ◽  
Author(s):  
Xu Chu ◽  
Yang Lin ◽  
Yasha Wang ◽  
Leye Wang ◽  
Jiangtao Wang ◽  
...  
Keyword(s):  
Supervised Learning ◽  
State Of The Art ◽  
Multiple Drug ◽  
Prediction Methods ◽  
Learning Models ◽  
Preventable Hospitalizations ◽  
Interaction Prediction ◽  
Learning Framework ◽  
Task Learning ◽  
Real World Datasets

Drug-drug interactions (DDIs) are a major cause of preventable hospitalizations and deaths. Recently, researchers in the AI community try to improve DDI prediction in two directions, incorporating multiple drug features to better model the pharmacodynamics and adopting multi-task learning to exploit associations among DDI types. However, these two directions are challenging to reconcile due to the sparse nature of the DDI labels which inflates the risk of overfitting of multi-task learning models when incorporating multiple drug features. In this paper, we propose a multi-task semi-supervised learning framework MLRDA for DDI prediction. MLRDA effectively exploits information that is beneficial for DDI prediction in unlabeled drug data by leveraging a novel unsupervised disentangling loss CuXCov. The CuXCov loss cooperates with the classification loss to disentangle the DDI prediction relevant part from the irrelevant part in a representation learnt by an autoencoder, which helps to ease the difficulty in mining useful information for DDI prediction in both labeled and unlabeled drug data. Moreover, MLRDA adopts a multi-task learning framework to exploit associations among DDI types. Experimental results on real-world datasets demonstrate that MLRDA significantly outperforms state-of-the-art DDI prediction methods by up to 10.3% in AUPR.

scholarly journals Teaching Semi-Supervised Classifier via Generalized Distillation

2018 ◽  
Author(s):  
Chen Gong ◽  
Xiaojun Chang ◽  
Meng Fang ◽  
Jian Yang
Keyword(s):  
Supervised Learning ◽  
Error Bounds ◽  
State Of The Art ◽  
Training Data ◽  
Training Process ◽  
Rademacher Complexity ◽  
Optimization Framework ◽  
Specific Teaching ◽  
Teaching Function ◽  
Intelligent Teacher

Semi-Supervised Learning (SSL) is able to build reliable classifier with very scarce labeled examples by properly utilizing the abundant unlabeled examples. However, existing SSL algorithms often yield unsatisfactory performance due to the lack of supervision information. To address this issue, this paper formulates SSL as a Generalized Distillation (GD) problem, which treats existing SSL algorithm as a learner and introduces a teacher to guide the learner?s training process. Specifically, the intelligent teacher holds the privileged knowledge that ?explains? the training data but remains unknown to the learner, and the teacher should convey its rich knowledge to the imperfect learner through a specific teaching function. After that, the learner gains knowledge by ?imitating? the output of the teaching function under an optimization framework. Therefore, the learner in our algorithm learns from both the teacher and the training data, so its output can be substantially distilled and enhanced. By deriving the Rademacher complexity and error bounds of the proposed algorithm, the usefulness of the introduced teacher is theoretically demonstrated. The superiority of our algorithm to the related state-of-the-art methods has also been empirically demonstrated by the experiments on different datasets with various sources of privileged knowledge.

scholarly journals SPADE: A Semi-supervised Probabilistic Approach for Detecting Errors in Tables

2021 ◽  
Author(s):  
Minh Pham ◽  
Craig A. Knoblock ◽  
Muhao Chen ◽  
Binh Vu ◽  
Jay Pujara
Keyword(s):  
Error Detection ◽  
Data Augmentation ◽  
State Of The Art ◽  
Probabilistic Approach ◽  
Human Interaction ◽  
Training Data ◽  
Two Phase ◽  
Learning Classifier ◽  
Supervised Methods ◽  
Phase Data

Error detection is one of the most important steps in data cleaning and usually requires extensive human interaction to ensure quality. Existing supervised methods in error detection require a significant amount of training data while unsupervised methods rely on fixed inductive biases, which are usually hard to generalize, to solve the problem. In this paper, we present SPADE, a novel semi-supervised probabilistic approach for error detection. SPADE introduces a novel probabilistic active learning model, where the system suggests examples to be labeled based on the agreements between user labels and indicative signals, which are designed to capture potential errors. SPADE uses a two-phase data augmentation process to enrich a dataset before training a deep learning classifier to detect unlabeled errors. In our evaluation, SPADE achieves an average F1-score of 0.91 over five datasets and yields a 10% improvement compared with the state-of-the-art systems.

scholarly journals Balanced Linear Contextual Bandits

2019 ◽  
Vol 33 ◽  
pp. 3445-3453 ◽  
Author(s):  
Maria Dimakopoulou ◽  
Zhengyuan Zhou ◽  
Susan Athey ◽  
Guido Imbens
Keyword(s):  
Supervised Learning ◽  
State Of The Art ◽  
Model Misspecification ◽  
Estimation Method ◽  
Training Data ◽  
Estimation Bias ◽  
Initial Training ◽  
Practical Advantage ◽  
Estimation Problems ◽  
Regret Bound

Contextual bandit algorithms are sensitive to the estimation method of the outcome model as well as the exploration method used, particularly in the presence of rich heterogeneity or complex outcome models, which can lead to difficult estimation problems along the path of learning. We develop algorithms for contextual bandits with linear payoffs that integrate balancing methods from the causal inference literature in their estimation to make it less prone to problems of estimation bias. We provide the first regret bound analyses for linear contextual bandits with balancing and show that our algorithms match the state of the art theoretical guarantees. We demonstrate the strong practical advantage of balanced contextual bandits on a large number of supervised learning datasets and on a synthetic example that simulates model misspecification and prejudice in the initial training data.

Sign in / Sign up

Export Citation Format

Share Document