scholarly journals Semi-supervised Learning with a Teacher-Student Network for Generalized Attribute Prediction

2020 ◽  
pp. 509-525
Author(s):  
Minchul Shin
Keyword(s):  
Supervised Learning ◽  
Teacher Student

scholarly journals Better and Faster: Knowledge Transfer from Multiple Self-supervised Learning Tasks via Graph Distillation for Video Classification

2018 ◽  
Author(s):  
Chenrui Zhang ◽  
Yuxin Peng
Keyword(s):  
Supervised Learning ◽  
Feature Learning ◽  
Representation Learning ◽  
Classification Task ◽  
Massive Data ◽  
Video Classification ◽  
Matching Problem ◽  
Video Representation ◽  
Teacher Student ◽  
Learning Tasks

Video representation learning is a vital problem for classification task. Recently, a promising unsupervised paradigm termed self-supervised learning has emerged, which explores inherent supervisory signals implied in massive data for feature learning via solving auxiliary tasks. However, existing methods in this regard suffer from two limitations when extended to video classification. First, they focus only on a single task, whereas ignoring complementarity among different task-specific features and thus resulting in suboptimal video representation. Second, high computational and memory cost hinders their application in real-world scenarios. In this paper, we propose a graph-based distillation framework to address these problems: (1) We propose logits graph and representation graph to transfer knowledge from multiple self-supervised tasks, where the former distills classifier-level knowledge by solving a multi-distribution joint matching problem, and the latter distills internal feature knowledge from pairwise ensembled representations with tackling the challenge of heterogeneity among different features; (2) The proposal that adopts a teacher-student framework can reduce the redundancy of knowledge learned from teachers dramatically, leading to a lighter student model that solves classification task more efficiently. Experimental results on 3 video datasets validate that our proposal not only helps learn better video representation but also compress model for faster inference.

scholarly journals On Teacher-Student Semi-Supervised Learning for Chest X-ray Image Classification

2021 ◽  
Author(s):  
Roberto Augusto Philippi Martins ◽  
Danilo Silva
Keyword(s):  
Deep Learning ◽  
Supervised Learning ◽  
Learning Algorithm ◽  
Unlabeled Data ◽  
Imaging Data ◽  
X Ray ◽  
Teacher Student ◽  
Chest X Ray ◽  
Deep Learning Model

The lack of labeled data is one of the main prohibiting issues on the development of deep learning models, as they rely on large labeled datasets in order to achieve high accuracy in complex tasks. Our objective is to evaluate the performance gain of having additional unlabeled data in the training of a deep learning model when working with medical imaging data. We present a semi-supervised learning algorithm that utilizes a teacher-student paradigm in order to leverage unlabeled data in the classification of chest X-ray images. Using our algorithm on the ChestX-ray14 dataset, we manage to achieve a substantial increase in performance when using small labeled datasets. With our method, a model achieves an AUROC of 0.822 with only 2% labeled data and 0.865 with 5% labeled data, while a fully supervised method achieves an AUROC of 0.807 with 5% labeled data and only 0.845 with 10%.

scholarly journals AAWS-Net: Anatomy-aware weakly-supervised learning network for breast mass segmentation

PLoS ONE ◽  
2021 ◽  
Vol 16 (8) ◽  
pp. e0256830
Author(s):  
Yeheng Sun ◽  
Yule Ji
Keyword(s):  
Supervised Learning ◽  
Learning Strategy ◽  
Medical Image Analysis ◽  
Training Data ◽  
Breast Mass ◽  
Weakly Supervised Learning ◽  
Learning Network ◽  
Teacher Student ◽  
Mass Segmentation ◽  
Weakly Supervised

Accurate segmentation of breast masses is an essential step in computer aided diagnosis of breast cancer. The scarcity of annotated training data greatly hinders the model’s generalization ability, especially for the deep learning based methods. However, high-quality image-level annotations are time-consuming and cumbersome in medical image analysis scenarios. In addition, a large amount of weak annotations is under-utilized which comprise common anatomy features. To this end, inspired by teacher-student networks, we propose an Anatomy-Aware Weakly-Supervised learning Network (AAWS-Net) for extracting useful information from mammograms with weak annotations for efficient and accurate breast mass segmentation. Specifically, we adopt a weakly-supervised learning strategy in the Teacher to extract anatomy structure from mammograms with weak annotations by reconstructing the original image. Besides, knowledge distillation is used to suggest morphological differences between benign and malignant masses. Moreover, the prior knowledge learned from the Teacher is introduced to the Student in an end-to-end way, which improves the ability of the student network to locate and segment masses. Experiments on CBIS-DDSM have shown that our method yields promising performance compared with state-of-the-art alternative models for breast mass segmentation in terms of segmentation accuracy and IoU.

Semi-supervised Learning via Improved Teacher-Student Network for Robust 3D Reconstruction of Stereo Endoscopic Image

2021 ◽  
Author(s):  
Hongkuan Shi ◽  
Zhiwei Wang ◽  
Jinxin Lv ◽  
Yilang Wang ◽  
Peng Zhang ◽  
...  
Keyword(s):  
3D Reconstruction ◽  
Supervised Learning ◽  
Teacher Student ◽  
Endoscopic Image

scholarly journals Estimation of Pedestrian Pose Orientation Using Soft Target Training Based on Teacher–Student Framework

Sensors ◽  
2019 ◽  
Vol 19 (5) ◽  
pp. 1147 ◽  
Author(s):  
DuYeong Heo ◽  
Jae Nam ◽  
Byoung Ko
Keyword(s):  
Supervised Learning ◽  
Spatial Information ◽  
Classification Performance ◽  
Input Image ◽  
Student Model ◽  
Teacher Student ◽  
Specific Shape ◽  
Soft Target ◽  
Target Data ◽  
Teacher Model

Semi-supervised learning is known to achieve better generalisation than a model learned solely from labelled data. Therefore, we propose a new method for estimating a pedestrian pose orientation using a soft-target method, which is a type of semi-supervised learning method. Because a convolutional neural network (CNN) based pose orientation estimation requires large numbers of parameters and operations, we apply the teacher–student algorithm to generate a compressed student model with high accuracy and compactness resembling that of the teacher model by combining a deep network with a random forest. After the teacher model is generated using hard target data, the softened outputs (soft-target data) of the teacher model are used for training the student model. Moreover, the orientation of the pedestrian has specific shape patterns, and a wavelet transform is applied to the input image as a pre-processing step owing to its good spatial frequency localisation property and the ability to preserve both the spatial information and gradient information of an image. For a benchmark dataset considering real driving situations based on a single camera, we used the TUD and KITTI datasets. We applied the proposed algorithm to various driving images in the datasets, and the results indicate that its classification performance with regard to the pose orientation is better than that of other state-of-the-art methods based on a CNN. In addition, the computational speed of the proposed student model is faster than that of other deep CNNs owing to the shorter model structure with a smaller number of parameters.

Sign in / Sign up

Export Citation Format

Share Document