Teacher/Student Deep Semi-Supervised Learning for Training with Noisy Labels

2018 ◽  
Author(s):  
Zeyad Hailat ◽  
Xue-Wen Chen
Keyword(s):  
Supervised Learning ◽  
Teacher Student ◽  
Noisy Labels

scholarly journals Better and Faster: Knowledge Transfer from Multiple Self-supervised Learning Tasks via Graph Distillation for Video Classification

2018 ◽  
Author(s):  
Chenrui Zhang ◽  
Yuxin Peng
Keyword(s):  
Supervised Learning ◽  
Feature Learning ◽  
Representation Learning ◽  
Classification Task ◽  
Massive Data ◽  
Video Classification ◽  
Matching Problem ◽  
Video Representation ◽  
Teacher Student ◽  
Learning Tasks

Video representation learning is a vital problem for classification task. Recently, a promising unsupervised paradigm termed self-supervised learning has emerged, which explores inherent supervisory signals implied in massive data for feature learning via solving auxiliary tasks. However, existing methods in this regard suffer from two limitations when extended to video classification. First, they focus only on a single task, whereas ignoring complementarity among different task-specific features and thus resulting in suboptimal video representation. Second, high computational and memory cost hinders their application in real-world scenarios. In this paper, we propose a graph-based distillation framework to address these problems: (1) We propose logits graph and representation graph to transfer knowledge from multiple self-supervised tasks, where the former distills classifier-level knowledge by solving a multi-distribution joint matching problem, and the latter distills internal feature knowledge from pairwise ensembled representations with tackling the challenge of heterogeneity among different features; (2) The proposal that adopts a teacher-student framework can reduce the redundancy of knowledge learned from teachers dramatically, leading to a lighter student model that solves classification task more efficiently. Experimental results on 3 video datasets validate that our proposal not only helps learn better video representation but also compress model for faster inference.

scholarly journals On Teacher-Student Semi-Supervised Learning for Chest X-ray Image Classification

2021 ◽  
Author(s):  
Roberto Augusto Philippi Martins ◽  
Danilo Silva
Keyword(s):  
Deep Learning ◽  
Supervised Learning ◽  
Learning Algorithm ◽  
Unlabeled Data ◽  
Imaging Data ◽  
X Ray ◽  
Teacher Student ◽  
Chest X Ray ◽  
Deep Learning Model

The lack of labeled data is one of the main prohibiting issues on the development of deep learning models, as they rely on large labeled datasets in order to achieve high accuracy in complex tasks. Our objective is to evaluate the performance gain of having additional unlabeled data in the training of a deep learning model when working with medical imaging data. We present a semi-supervised learning algorithm that utilizes a teacher-student paradigm in order to leverage unlabeled data in the classification of chest X-ray images. Using our algorithm on the ChestX-ray14 dataset, we manage to achieve a substantial increase in performance when using small labeled datasets. With our method, a model achieves an AUROC of 0.822 with only 2% labeled data and 0.865 with 5% labeled data, while a fully supervised method achieves an AUROC of 0.807 with 5% labeled data and only 0.845 with 10%.

scholarly journals AAWS-Net: Anatomy-aware weakly-supervised learning network for breast mass segmentation

PLoS ONE ◽  
2021 ◽  
Vol 16 (8) ◽  
pp. e0256830
Author(s):  
Yeheng Sun ◽  
Yule Ji
Keyword(s):  
Supervised Learning ◽  
Learning Strategy ◽  
Medical Image Analysis ◽  
Training Data ◽  
Breast Mass ◽  
Weakly Supervised Learning ◽  
Learning Network ◽  
Teacher Student ◽  
Mass Segmentation ◽  
Weakly Supervised

Accurate segmentation of breast masses is an essential step in computer aided diagnosis of breast cancer. The scarcity of annotated training data greatly hinders the model’s generalization ability, especially for the deep learning based methods. However, high-quality image-level annotations are time-consuming and cumbersome in medical image analysis scenarios. In addition, a large amount of weak annotations is under-utilized which comprise common anatomy features. To this end, inspired by teacher-student networks, we propose an Anatomy-Aware Weakly-Supervised learning Network (AAWS-Net) for extracting useful information from mammograms with weak annotations for efficient and accurate breast mass segmentation. Specifically, we adopt a weakly-supervised learning strategy in the Teacher to extract anatomy structure from mammograms with weak annotations by reconstructing the original image. Besides, knowledge distillation is used to suggest morphological differences between benign and malignant masses. Moreover, the prior knowledge learned from the Teacher is introduced to the Student in an end-to-end way, which improves the ability of the student network to locate and segment masses. Experiments on CBIS-DDSM have shown that our method yields promising performance compared with state-of-the-art alternative models for breast mass segmentation in terms of segmentation accuracy and IoU.

Semi-supervised Learning via Improved Teacher-Student Network for Robust 3D Reconstruction of Stereo Endoscopic Image

2021 ◽  
Author(s):  
Hongkuan Shi ◽  
Zhiwei Wang ◽  
Jinxin Lv ◽  
Yilang Wang ◽  
Peng Zhang ◽  
...  
Keyword(s):  
3D Reconstruction ◽  
Supervised Learning ◽  
Teacher Student ◽  
Endoscopic Image

scholarly journals A Two-Stream Mutual Attention Network for Semi-Supervised Biomedical Segmentation with Noisy Labels

2019 ◽  
Vol 33 ◽  
pp. 4578-4585 ◽  
Author(s):  
Shaobo Min ◽  
Xuejin Chen ◽  
Zheng-Jun Zha ◽  
Feng Wu ◽  
Yongdong Zhang
Keyword(s):  
Supervised Learning ◽  
State Of The Art ◽  
Training Data ◽  
Attention Network ◽  
Attention Model ◽  
Learning Framework ◽  
Propagation Analysis ◽  
Supervised Methods ◽  
Multi Level ◽  
Noisy Labels

Learning-based methods suffer from a deficiency of clean annotations, especially in biomedical segmentation. Although many semi-supervised methods have been proposed to provide extra training data, automatically generated labels are usually too noisy to retrain models effectively. In this paper, we propose a Two-Stream Mutual Attention Network (TSMAN) that weakens the influence of back-propagated gradients caused by incorrect labels, thereby rendering the network robust to unclean data. The proposed TSMAN consists of two sub-networks that are connected by three types of attention models in different layers. The target of each attention model is to indicate potentially incorrect gradients in a certain layer for both sub-networks by analyzing their inferred features using the same input. In order to achieve this purpose, the attention models are designed based on the propagation analysis of noisy gradients at different layers. This allows the attention models to effectively discover incorrect labels and weaken their influence during parameter updating process. By exchanging multi-level features within two-stream architecture, the effects of noisy labels in each sub-network are reduced by decreasing the noisy gradients. Furthermore, a hierarchical distillation is developed to provide reliable pseudo labels for unlabelded data, which further boosts the performance of TSMAN. The experiments using both HVSMR 2016 and BRATS 2015 benchmarks demonstrate that our semi-supervised learning framework surpasses the state-of-the-art fully-supervised results.

Sign in / Sign up

Export Citation Format

Share Document