scholarly journals Ship Detection in Sentinel 2 Multi-Spectral Images with Self-Supervised Learning

2021 ◽  
Vol 13 (21) ◽  
pp. 4255
Author(s):  
Alina Ciocarlan ◽  
Andrei Stoian
Keyword(s):  
Supervised Learning ◽  
Network Architecture ◽  
The Self ◽  
Training Data ◽  
Training Procedure ◽  
Ship Detection ◽  
Annotation Costs ◽  
Classical Image ◽  
The Impact ◽  
Sentinel 2

Automatic ship detection provides an essential function towards maritime domain awareness for security or economic monitoring purposes. This work presents an approach for training a deep learning ship detector in Sentinel-2 multi-spectral images with few labeled examples. We design a network architecture for detecting ships with a backbone that can be pre-trained separately. By using self supervised learning, an emerging unsupervised training procedure, we learn good features on Sentinel-2 images, without requiring labeling, to initialize our network’s backbone. The full network is then fine-tuned to learn to detect ships in challenging settings. We evaluated this approach versus pre-training on ImageNet and versus a classical image processing pipeline. We examined the impact of variations in the self-supervised learning step and we show that in the few-shot learning setting self-supervised pre-training achieves better results than ImageNet pre-training. When enough training data are available, our self-supervised approach is as good as ImageNet pre-training. We conclude that a better design of the self-supervised task and bigger non-annotated dataset sizes can lead to surpassing ImageNet pre-training performance without any annotation costs.

Combining Self-supervised Learning and Active Learning for Disfluency Detection

2022 ◽  
Vol 21 (3) ◽  
pp. 1-25
Author(s):  
Shaolei Wang ◽  
Zhongyuan Wang ◽  
Wanxiang Che ◽  
Sendong Zhao ◽  
Ting Liu
Keyword(s):  
Neural Network ◽  
Active Learning ◽  
Supervised Learning ◽  
Large Scale ◽  
Training Data ◽  
Fine Tuning ◽  
Training Dataset ◽  
Performance Gap ◽  
Annotation Costs ◽  
Trained Neural Network

Spoken language is fundamentally different from the written language in that it contains frequent disfluencies or parts of an utterance that are corrected by the speaker. Disfluency detection (removing these disfluencies) is desirable to clean the input for use in downstream NLP tasks. Most existing approaches to disfluency detection heavily rely on human-annotated data, which is scarce and expensive to obtain in practice. To tackle the training data bottleneck, in this work, we investigate methods for combining self-supervised learning and active learning for disfluency detection. First, we construct large-scale pseudo training data by randomly adding or deleting words from unlabeled data and propose two self-supervised pre-training tasks: (i) a tagging task to detect the added noisy words and (ii) sentence classification to distinguish original sentences from grammatically incorrect sentences. We then combine these two tasks to jointly pre-train a neural network. The pre-trained neural network is then fine-tuned using human-annotated disfluency detection training data. The self-supervised learning method can capture task-special knowledge for disfluency detection and achieve better performance when fine-tuning on a small annotated dataset compared to other supervised methods. However, limited in that the pseudo training data are generated based on simple heuristics and cannot fully cover all the disfluency patterns, there is still a performance gap compared to the supervised models trained on the full training dataset. We further explore how to bridge the performance gap by integrating active learning during the fine-tuning process. Active learning strives to reduce annotation costs by choosing the most critical examples to label and can address the weakness of self-supervised learning with a small annotated dataset. We show that by combining self-supervised learning with active learning, our model is able to match state-of-the-art performance with just about 10% of the original training data on both the commonly used English Switchboard test set and a set of in-house annotated Chinese data.

scholarly journals Time-Series Laplacian Semi-Supervised Learning for Indoor Localization †

Sensors ◽  
2019 ◽  
Vol 19 (18) ◽  
pp. 3867 ◽  
Author(s):  
Jaehyun Yoo
Keyword(s):  
Time Series ◽  
Supervised Learning ◽  
Indoor Localization ◽  
Learning Algorithm ◽  
Unlabeled Data ◽  
Training Data ◽  
Practical Implementation ◽  
Additional Information ◽  
Localization Scheme ◽  
The Impact

Machine learning-based indoor localization used to suffer from the collection, construction, and maintenance of labeled training databases for practical implementation. Semi-supervised learning methods have been developed as efficient indoor localization methods to reduce use of labeled training data. To boost the efficiency and the accuracy of indoor localization, this paper proposes a new time-series semi-supervised learning algorithm. The key aspect of the developed method, which distinguishes it from conventional semi-supervised algorithms, is the use of unlabeled data. The learning algorithm finds spatio-temporal relationships in the unlabeled data, and pseudolabels are generated to compensate for the lack of labeled training data. In the next step, another balancing-optimization learning algorithm learns a positioning model. The proposed method is evaluated for estimating the location of a smartphone user by using a Wi-Fi received signal strength indicator (RSSI) measurement. The experimental results show that the developed learning algorithm outperforms some existing semi-supervised algorithms according to the variation of the number of training data and access points. Also, the proposed method is discussed in terms of why it gives better performance, by the analysis of the impact of the learning parameters. Moreover, the extended localization scheme in conjunction with a particle filter is executed to include additional information, such as a floor plan.

Segmentation Neural Network Incorporating Scale-Space in the Application of Cardiac MRI

2020 ◽  
Vol 10 (7) ◽  
pp. 1494-1505
Author(s):  
Hyo-Hun Kim ◽  
Byung-Woo Hong
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
Cardiac Mri ◽  
Network Architecture ◽  
Data Augmentation ◽  
Scale Space ◽  
Segmentation Algorithm ◽  
Training Data ◽  
Training Procedure ◽  
Whole Heart

In this work, we present an image segmentation algorithm based on the convolutional neural network framework where the scale space theory is incorporated in the course of training procedure. The construction of data augmentation is designed to apply the scale space to the training data in order to effectively deal with the variability of regions of interest in geometry and appearance such as shape and contrast. The proposed data augmentation algorithm via scale space is aimed to improve invariant features with respect to both geometry and appearance by taking into consideration of their diffusion process. We develop a segmentation algorithm based on the convolutional neural network framework where the network architecture consists of encoding and decoding substructures in combination with the data augmentation scheme via the scale space induced by the heat equation. The quantitative analysis using the cardiac MRI dataset indicates that the proposed algorithm achieves better accuracy in the delineation of the left ventricles, which demonstrates the potential of the algorithm in the application of the whole heart segmentation as a compute-aided diagnosis system for the cardiac diseases.

scholarly journals Developing Sustainable Classification of Diseases via Deep Learning and Semi-Supervised Learning

Healthcare ◽  
2020 ◽  
Vol 8 (3) ◽  
pp. 291 ◽  
Author(s):  
Chunwu Yin ◽  
Zhanbo Chen
Keyword(s):  
Deep Learning ◽  
Supervised Learning ◽  
Learning Style ◽  
Classification Performance ◽  
Disease Classification ◽  
Training Data ◽  
Training Procedure ◽  
Training Approach ◽  
Classification Of Diseases ◽  
Deep Forest

Disease classification based on machine learning has become a crucial research topic in the fields of genetics and molecular biology. Generally, disease classification involves a supervised learning style; i.e., it requires a large number of labelled samples to achieve good classification performance. However, in the majority of the cases, labelled samples are hard to obtain, so the amount of training data are limited. However, many unclassified (unlabelled) sequences have been deposited in public databases, which may help the training procedure. This method is called semi-supervised learning and is very useful in many applications. Self-training can be implemented using high- to low-confidence samples to prevent noisy samples from affecting the robustness of semi-supervised learning in the training process. The deep forest method with the hyperparameter settings used in this paper can achieve excellent performance. Therefore, in this work, we propose a novel combined deep learning model and semi-supervised learning with self-training approach to improve the performance in disease classification, which utilizes unlabelled samples to update a mechanism designed to increase the number of high-confidence pseudo-labelled samples. The experimental results show that our proposed model can achieve good performance in disease classification and disease-causing gene identification.

Semi-supervised learning of Hidden Markov Models for biological sequence analysis

2018 ◽  
Vol 35 (13) ◽  
pp. 2208-2215 ◽  
Author(s):  
Ioannis A Tamposis ◽  
Konstantinos D Tsirigos ◽  
Margarita C Theodoropoulou ◽  
Panagiota I Kontou ◽  
Pantelis G Bagos
Keyword(s):  
Sequence Analysis ◽  
Supervised Learning ◽  
Hidden Markov Models ◽  
Markov Models ◽  
Hidden Markov ◽  
Transmembrane Protein ◽  
Training Data ◽  
Supplementary Information ◽  
Training Procedure ◽  
Partially Labeled Data

Abstract Motivation Hidden Markov Models (HMMs) are probabilistic models widely used in applications in computational sequence analysis. HMMs are basically unsupervised models. However, in the most important applications, they are trained in a supervised manner. Training examples accompanied by labels corresponding to different classes are given as input and the set of parameters that maximize the joint probability of sequences and labels is estimated. A main problem with this approach is that, in the majority of the cases, labels are hard to find and thus the amount of training data is limited. On the other hand, there are plenty of unclassified (unlabeled) sequences deposited in the public databases that could potentially contribute to the training procedure. This approach is called semi-supervised learning and could be very helpful in many applications. Results We propose here, a method for semi-supervised learning of HMMs that can incorporate labeled, unlabeled and partially labeled data in a straightforward manner. The algorithm is based on a variant of the Expectation-Maximization (EM) algorithm, where the missing labels of the unlabeled or partially labeled data are considered as the missing data. We apply the algorithm to several biological problems, namely, for the prediction of transmembrane protein topology for alpha-helical and beta-barrel membrane proteins and for the prediction of archaeal signal peptides. The results are very promising, since the algorithms presented here can significantly improve the prediction performance of even the top-scoring classifiers. Supplementary information Supplementary data are available at Bioinformatics online.

scholarly journals DISIR: DEEP IMAGE SEGMENTATION WITH INTERACTIVE REFINEMENT

2020 ◽  
Vol V-2-2020 ◽  
pp. 877-884
Author(s):  
G. Lenczner ◽  
B. Le Saux ◽  
N. Luminari ◽  
A. Chan-Hon-Tong ◽  
G. Le Besnerais
Keyword(s):  
Neural Network ◽  
Network Architecture ◽  
Deep Neural Network ◽  
Training Data ◽  
Aerial Images ◽  
Interactive Approach ◽  
Rgb Images ◽  
Deep Image ◽  
Class Segmentation ◽  
The Impact

Abstract. This paper presents an interactive approach for multi-class segmentation of aerial images. Precisely, it is based on a deep neural network which exploits both RGB images and annotations. Starting from an initial output based on the image only, our network then interactively refines this segmentation map using a concatenation of the image and user annotations. Importantly, user annotations modify the inputs of the network – not its weights – enabling a fast and smooth process. Through experiments on two public aerial datasets, we show that user annotations are extremely rewarding: each click corrects roughly 5000 pixels. We analyze the impact of different aspects of our framework such as the representation of the annotations, the volume of training data or the network architecture. Code is available at this address.

scholarly journals Pedestrian Detection in Driver Assistance Using SSD and PS-GAN

2019 ◽  
Vol 2 (3) ◽  
Author(s):  
Kun Zheng ◽  
Mengfei Wei ◽  
Shenhui Li ◽  
Dong Yang ◽  
Xudong Liu
Keyword(s):  
Object Detection ◽  
Network Architecture ◽  
Driver Assistance System ◽  
Motor Vehicle ◽  
Pedestrian Detection ◽  
Training Data ◽  
Driver Assistance ◽  
Data Synthesis ◽  
General Object ◽  
The Impact

Pedestrian detection is a critical challenge in the field of general object detection, the performance of object detection has advanced with the development of deep learning. However, considerable improvement is still required for pedestrian detection, considering the differences in pedestrian wears, action, and posture. In the driver assistance system, it is necessary to further improve the intelligent pedestrian detection ability. We present a method based on the combination of SSD and GAN to improve the performance of pedestrian detection. Firstly, we assess the impact of different kinds of methods which can detect pedestrians based on SSD and optimize the detection for pedestrian characteristics. Secondly, we propose a novel network architecture, namely data synthesis PS-GAN to generate diverse pedestrian data for verifying the effectiveness of massive training data to SSD detector. Experimental results show that the proposed manners can improve the performance of pedestrian detection to some extent. At last, we use the pedestrian detector to simulate a specific application of motor vehicle assisted driving which would make the detector focus on specific pedestrians according to the velocity of the vehicle. The results establish the validity of the approach.

scholarly journals Weakly Supervised Sensitive Heatmap framework to classify and localize diabetic retinopathy lesions

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Mohammed Al-Mukhtar ◽  
Ameer Hussein Morad ◽  
Mustafa Albadri ◽  
MD Samiul Islam
Keyword(s):  
Diabetic Retinopathy ◽  
Supervised Learning ◽  
Automatic Detection ◽  
Training Data ◽  
Optic Disk ◽  
Test Accuracy ◽  
Fundus Images ◽  
Data Set ◽  
Weakly Supervised ◽  
The Impact

AbstractVision loss happens due to diabetic retinopathy (DR) in severe stages. Thus, an automatic detection method applied to diagnose DR in an earlier phase may help medical doctors to make better decisions. DR is considered one of the main risks, leading to blindness. Computer-Aided Diagnosis systems play an essential role in detecting features in fundus images. Fundus images may include blood vessels, exudates, micro-aneurysm, hemorrhages, and neovascularization. In this paper, our model combines automatic detection for the diabetic retinopathy classification with localization methods depending on weakly-supervised learning. The model has four stages; in stage one, various preprocessing techniques are applied to smooth the data set. In stage two, the network had gotten deeply to the optic disk segment for eliminating any exudate's false prediction because the exudates had the same color pixel as the optic disk. In stage three, the network is fed through training data to classify each label. Finally, the layers of the convolution neural network are re-edited, and used to localize the impact of DR on the patient's eye. The framework tackles the matching technique between two essential concepts where the classification problem depends on the supervised learning method. While the localization problem was obtained by the weakly supervised method. An additional layer known as weakly supervised sensitive heat map (WSSH) was added to detect the ROI of the lesion at a test accuracy of 98.65%, while comparing with Class Activation Map that involved weakly supervised technology achieved 0.954. The main purpose is to learn a representation that collect the central localization of discriminative features in a retina image. CNN-WSSH model is able to highlight decisive features in a single forward pass for getting the best detection of lesions.

scholarly journals Active semi-supervised framework with data editing

2012 ◽  
Vol 9 (4) ◽  
pp. 1513-1532 ◽  
Author(s):  
Xue Zhang ◽  
Wangxin Xiao
Keyword(s):  
Active Learning ◽  
Supervised Learning ◽  
Text Classification ◽  
State Of The Art ◽  
The Self ◽  
Training Data ◽  
Data Sets ◽  
Text Data ◽  
Data Editing ◽  
Data Problem

In order to address the insufficient training data problem, many active semi-supervised algorithms have been proposed. The self-labeled training data in semi-supervised learning may contain much noise due to the insufficient training data. Such noise may snowball themselves in the following learning process and thus hurt the generalization ability of the final hypothesis. Extremely few labeled training data in sparsely labeled text classification aggravate such situation. If such noise could be identified and removed by some strategy, the performance of the active semi-supervised algorithms should be improved. However, such useful techniques of identifying and removing noise have been seldom explored in existing active semi-supervised algorithms. In this paper, we propose an active semi-supervised framework with data editing (we call it ASSDE) to improve sparsely labeled text classification. A data editing technique is used to identify and remove noise introduced by semi-supervised labeling. We carry out the data editing technique by fully utilizing the advantage of active learning, which is novel according to our knowledge. The fusion of active learning with data editing makes ASSDE more robust to the sparsity and the distribution bias of the training data. It further simplifies the design of semi-supervised learning which makes ASSDE more efficient. Extensive experimental study on several real-world text data sets shows the encouraging results of the proposed framework for sparsely labeled text classification, compared with several state-of-the-art methods.

scholarly journals WSSH_DR: Weakly Supervised Sensitive Heatmap Framework to Classify and Localize Diabetic Retinopathy Lesions

2021 ◽  
Author(s):  
Mohammed Almukhtar ◽  
Ameer Morad ◽  
Mustafa Albadri ◽  
MD Islam
Keyword(s):  
Diabetic Retinopathy ◽  
Supervised Learning ◽  
Automatic Detection ◽  
Training Data ◽  
Optic Disk ◽  
Test Accuracy ◽  
Fundus Images ◽  
Data Set ◽  
Weakly Supervised ◽  
The Impact

Abstract Vision loss happens due to diabetic retinopathy (DR) in severe stages. Thus, an automatic detection method applied to diagnose DR in an earlier phase may help medical doctors to make better decisions. DR is considered one of the main risks, leading to blindness. Computer-Aided Diagnosis (CAD) systems play an essential role in detecting features in fundus images. Fundus images may include blood vessel area, exudates, micro-aneurysm, hemorrhages, and neovascularization. In this paper, our model combines automatic detection for the diabetic retinopathy classification with localization methods depending on weakly-supervised learning. The model has four stages; in stage one, various preprocessing techniques are applied for smoothing the data set. In stage two, the network had gotten deeply to the optic disk segment for eliminating any exudate's false prediction because the exudates had the same color pixel as the optic disk. Stage three, the network is fed through training data to classify each class label. Finally, the layers of the convolution neural network are re-edited, and the layers are used to localize the impact of DR on the eye's patient. The framework tackled the matching technique between two essential concepts where the classification problem depends on the supervised learning method. In comparison, the localization problem was obtained by the weakly supervised method. An additional layer known as weakly supervised sensitive heat map (WSSH) was added to detect the ROI of the lesion at a test accuracy of 98.65%.

Sign in / Sign up

Export Citation Format

Share Document