scholarly journals A novel augmented deep transfer learning for classification of COVID-19 and other thoracic diseases from X-rays

2021 ◽  
Author(s):  
Fouzia Altaf ◽  
Syed M. S. Islam ◽  
Naeem Khalid Janjua
Keyword(s):  
Deep Learning ◽  
Medical Imaging ◽  
Transfer Learning ◽  
Medical Images ◽  
Training Data ◽  
Target Domain ◽  
Data Set ◽  
Learning Technique ◽  
Multi Class Classification ◽  
Effective Transfer

AbstractDeep learning has provided numerous breakthroughs in natural imaging tasks. However, its successful application to medical images is severely handicapped with the limited amount of annotated training data. Transfer learning is commonly adopted for the medical imaging tasks. However, a large covariant shift between the source domain of natural images and target domain of medical images results in poor transfer learning. Moreover, scarcity of annotated data for the medical imaging tasks causes further problems for effective transfer learning. To address these problems, we develop an augmented ensemble transfer learning technique that leads to significant performance gain over the conventional transfer learning. Our technique uses an ensemble of deep learning models, where the architecture of each network is modified with extra layers to account for dimensionality change between the images of source and target data domains. Moreover, the model is hierarchically tuned to the target domain with augmented training data. Along with the network ensemble, we also utilize an ensemble of dictionaries that are based on features extracted from the augmented models. The dictionary ensemble provides an additional performance boost to our method. We first establish the effectiveness of our technique with the challenging ChestXray-14 radiography data set. Our experimental results show more than 50% reduction in the error rate with our method as compared to the baseline transfer learning technique. We then apply our technique to a recent COVID-19 data set for binary and multi-class classification tasks. Our technique achieves 99.49% accuracy for the binary classification, and 99.24% for multi-class classification.

scholarly journals Towards a Better Understanding of Transfer Learning for Medical Imaging: A Case Study

2020 ◽  
Vol 10 (13) ◽  
pp. 4523 ◽  
Author(s):  
Laith Alzubaidi ◽  
Mohammed A. Fadhel ◽  
Omran Al-Shamma ◽  
Jinglan Zhang ◽  
J. Santamaría ◽  
...  
Keyword(s):  
Medical Imaging ◽  
Transfer Learning ◽  
Foot Ulcer ◽  
Diabetic Foot Ulcer ◽  
Training Data ◽  
Classification Task ◽  
Natural Image ◽  
Target Domain ◽  
Proposed Model

One of the main challenges of employing deep learning models in the field of medicine is a lack of training data due to difficulty in collecting and labeling data, which needs to be performed by experts. To overcome this drawback, transfer learning (TL) has been utilized to solve several medical imaging tasks using pre-trained state-of-the-art models from the ImageNet dataset. However, there are primary divergences in data features, sizes, and task characteristics between the natural image classification and the targeted medical imaging tasks. Therefore, TL can slightly improve performance if the source domain is completely different from the target domain. In this paper, we explore the benefit of TL from the same and different domains of the target tasks. To do so, we designed a deep convolutional neural network (DCNN) model that integrates three ideas including traditional and parallel convolutional layers and residual connections along with global average pooling. We trained the proposed model against several scenarios. We utilized the same and different domain TL with the diabetic foot ulcer (DFU) classification task and with the animal classification task. We have empirically shown that the source of TL from the same domain can significantly improve the performance considering a reduced number of images in the same domain of the target dataset. The proposed model with the DFU dataset achieved F1-score value of 86.6% when trained from scratch, 89.4% with TL from a different domain of the targeted dataset, and 97.6% with TL from the same domain of the targeted dataset.

scholarly journals Novel Transfer Learning Approach for Medical Imaging with Limited Labeled Data

Cancers ◽  
2021 ◽  
Vol 13 (7) ◽  
pp. 1590
Author(s):  
Laith Alzubaidi ◽  
Muthana Al-Amidie ◽  
Ahmed Al-Asadi ◽  
Amjad J. Humaidi ◽  
Omran Al-Shamma ◽  
...  
Keyword(s):  
Deep Learning ◽  
Medical Imaging ◽  
Transfer Learning ◽  
Medical Image ◽  
Medical Images ◽  
Image Data ◽  
Learning Approach ◽  
Learning Models ◽  
Annotation Process ◽  
Deep Learning Model

Deep learning requires a large amount of data to perform well. However, the field of medical image analysis suffers from a lack of sufficient data for training deep learning models. Moreover, medical images require manual labeling, usually provided by human annotators coming from various backgrounds. More importantly, the annotation process is time-consuming, expensive, and prone to errors. Transfer learning was introduced to reduce the need for the annotation process by transferring the deep learning models with knowledge from a previous task and then by fine-tuning them on a relatively small dataset of the current task. Most of the methods of medical image classification employ transfer learning from pretrained models, e.g., ImageNet, which has been proven to be ineffective. This is due to the mismatch in learned features between the natural image, e.g., ImageNet, and medical images. Additionally, it results in the utilization of deeply elaborated models. In this paper, we propose a novel transfer learning approach to overcome the previous drawbacks by means of training the deep learning model on large unlabeled medical image datasets and by next transferring the knowledge to train the deep learning model on the small amount of labeled medical images. Additionally, we propose a new deep convolutional neural network (DCNN) model that combines recent advancements in the field. We conducted several experiments on two challenging medical imaging scenarios dealing with skin and breast cancer classification tasks. According to the reported results, it has been empirically proven that the proposed approach can significantly improve the performance of both classification scenarios. In terms of skin cancer, the proposed model achieved an F1-score value of 89.09% when trained from scratch and 98.53% with the proposed approach. Secondly, it achieved an accuracy value of 85.29% and 97.51%, respectively, when trained from scratch and using the proposed approach in the case of the breast cancer scenario. Finally, we concluded that our method can possibly be applied to many medical imaging problems in which a substantial amount of unlabeled image data is available and the labeled image data is limited. Moreover, it can be utilized to improve the performance of medical imaging tasks in the same domain. To do so, we used the pretrained skin cancer model to train on feet skin to classify them into two classes—either normal or abnormal (diabetic foot ulcer (DFU)). It achieved an F1-score value of 86.0% when trained from scratch, 96.25% using transfer learning, and 99.25% using double-transfer learning.

scholarly journals A Transfer Learning–Based Active Learning Framework for Brain Tumor Classification

2021 ◽  
Vol 4 ◽  
Author(s):  
Ruqian Hao ◽  
Khashayar Namdar ◽  
Lin Liu ◽  
Farzad Khalvati
Keyword(s):  
Deep Learning ◽  
Brain Tumor ◽  
Active Learning ◽  
Transfer Learning ◽  
Medical Images ◽  
Tumor Classification ◽  
Training Data ◽  
Validation Dataset ◽  
Low Grade ◽  
Learning Framework

Brain tumor is one of the leading causes of cancer-related death globally among children and adults. Precise classification of brain tumor grade (low-grade and high-grade glioma) at an early stage plays a key role in successful prognosis and treatment planning. With recent advances in deep learning, artificial intelligence–enabled brain tumor grading systems can assist radiologists in the interpretation of medical images within seconds. The performance of deep learning techniques is, however, highly depended on the size of the annotated dataset. It is extremely challenging to label a large quantity of medical images, given the complexity and volume of medical data. In this work, we propose a novel transfer learning–based active learning framework to reduce the annotation cost while maintaining stability and robustness of the model performance for brain tumor classification. In this retrospective research, we employed a 2D slice–based approach to train and fine-tune our model on the magnetic resonance imaging (MRI) training dataset of 203 patients and a validation dataset of 66 patients which was used as the baseline. With our proposed method, the model achieved area under receiver operating characteristic (ROC) curve (AUC) of 82.89% on a separate test dataset of 66 patients, which was 2.92% higher than the baseline AUC while saving at least 40% of labeling cost. In order to further examine the robustness of our method, we created a balanced dataset, which underwent the same procedure. The model achieved AUC of 82% compared with AUC of 78.48% for the baseline, which reassures the robustness and stability of our proposed transfer learning augmented with active learning framework while significantly reducing the size of training data.

scholarly journals A Novel Method of Hyperspectral Data Classification Based on Transfer Learning and Deep Belief Network

2019 ◽  
Vol 9 (7) ◽  
pp. 1379 ◽  
Author(s):  
Ke Li ◽  
Mingju Wang ◽  
Yixin Liu ◽  
Nan Yu ◽  
Wei Lan
Keyword(s):  
Deep Learning ◽  
Transfer Learning ◽  
Spatial Information ◽  
Hyperspectral Data ◽  
Deep Belief Network ◽  
Learning Method ◽  
Target Domain ◽  
Data Set ◽  
Belief Network ◽  
Target Data

The classification of hyperspectral data using deep learning methods can obtain better results than the previous shallow classifiers, but deep learning algorithms have some limitations. These algorithms require a large amount of data to train the network, while also needing a certain amount of labeled data to fine-tune the network. In this paper, we propose a new hyperspectral data processing method based on transfer learning and the deep learning method. First, we use a hyperspectral data set that is similar to the target data set to pre-train the deep learning network. Then, we use the transfer learning method to find the common features of the source domain data and target domain data. Second, we propose a model structure that combines the deep transfer learning model to utilize a combination of spatial information and spectral information. Using transfer learning, we can obtain the spectral features. Then, we obtain several principal components of the target data. These will be regarded as the spatial features of the target domain data, and we use the joint features for the classifier. The data are obtained from a hyperspectral public database. Using the same amount of data, our method based on transfer learning and deep belief network obtains better classification accuracy in a shorter amount of time.

A Novel Epileptic Seizure Detection Algorithm Using Transfer K-Plane Clustering

2020 ◽  
Vol 10 (2) ◽  
pp. 484-488
Author(s):  
Lifang Peng ◽  
Bin Huang ◽  
Kefu Chen ◽  
Leyuan Zhou
Keyword(s):  
Transfer Learning ◽  
Epileptic Seizure ◽  
Clustering Algorithms ◽  
Training Data ◽  
Fuzzy Membership ◽  
Data Sets ◽  
Learning Mechanisms ◽  
Target Domain ◽  
Data Set ◽  
Clustering Effect

To recognize epileptic EEG signals, traditional clustering algorithms often need to satisfy three conditions to obtain better clustering results. The first condition is that the algorithm must not be sensitive to noise. The second condition is that the data set must be sufficient. The third condition is that the training data set and the testing data set must follow the same distribution. However, in actual applications, there are few data sets that are free of noise and have sufficient data volume. To address the effects of insufficient data sets and noise on clustering, this paper introduces fuzzy membership and transfer learning mechanisms based on K-plane clustering (KPC) and proposes a fuzzy KPC algorithm based on transfer learning (TFKPC). To improve the clustering effect, the TFKPC algorithm uses the knowledge summarized by the historical domain to guide the clustering process of the current (target) domain when the information is insufficient. In addition, the influence of noise on the clustering result is reduced by introducing fuzzy membership. Experiments show that the TFKPC algorithm proposed in this paper has a better clustering effect in the Epileptic Seizure Recognition Data Set than other comparison methods.

scholarly journals Transfer Learning for Named Entity Recognition in Financial and Biomedical Documents

Information ◽  
2019 ◽  
Vol 10 (8) ◽  
pp. 248 ◽  
Author(s):  
Sumam Francis ◽  
Jordy Van Landeghem ◽  
Marie-Francine Moens
Keyword(s):  
Deep Learning ◽  
Transfer Learning ◽  
State Of The Art ◽  
Named Entity Recognition ◽  
Training Data ◽  
Entity Recognition ◽  
Language Models ◽  
Reasonable Assumption ◽  
Target Domain ◽  
Named Entity

Recent deep learning approaches have shown promising results for named entity recognition (NER). A reasonable assumption for training robust deep learning models is that a sufficient amount of high-quality annotated training data is available. However, in many real-world scenarios, labeled training data is scarcely present. In this paper we consider two use cases: generic entity extraction from financial and from biomedical documents. First, we have developed a character based model for NER in financial documents and a word and character based model with attention for NER in biomedical documents. Further, we have analyzed how transfer learning addresses the problem of limited training data in a target domain. We demonstrate through experiments that NER models trained on labeled data from a source domain can be used as base models and then be fine-tuned with few labeled data for recognition of different named entity classes in a target domain. We also witness an interest in language models to improve NER as a way of coping with limited labeled data. The current most successful language model is BERT. Because of its success in state-of-the-art models we integrate representations based on BERT in our biomedical NER model along with word and character information. The results are compared with a state-of-the-art model applied on a benchmarking biomedical corpus.

scholarly journals Detection and analysis of COVID-19 in medical images using deep learning techniques

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Dandi Yang ◽  
Cristhian Martinez ◽  
Lara Visuña ◽  
Hardev Khandhar ◽  
Chintan Bhatt ◽  
...  
Keyword(s):  
Deep Learning ◽  
Ct Scan ◽  
Transfer Learning ◽  
Medical Images ◽  
Binary Classification ◽  
Training Time ◽  
X Ray ◽  
Learning Techniques ◽  
Multi Class Classification ◽  
Public Datasets

AbstractThe main purpose of this work is to investigate and compare several deep learning enhanced techniques applied to X-ray and CT-scan medical images for the detection of COVID-19. In this paper, we used four powerful pre-trained CNN models, VGG16, DenseNet121, ResNet50,and ResNet152, for the COVID-19 CT-scan binary classification task. The proposed Fast.AI ResNet framework was designed to find out the best architecture, pre-processing, and training parameters for the models largely automatically. The accuracy and F1-score were both above 96% in the diagnosis of COVID-19 using CT-scan images. In addition, we applied transfer learning techniques to overcome the insufficient data and to improve the training time. The binary and multi-class classification of X-ray images tasks were performed by utilizing enhanced VGG16 deep transfer learning architecture. High accuracy of 99% was achieved by enhanced VGG16 in the detection of X-ray images from COVID-19 and pneumonia. The accuracy and validity of the algorithms were assessed on X-ray and CT-scan well-known public datasets. The proposed methods have better results for COVID-19 diagnosis than other related in literature. In our opinion, our work can help virologists and radiologists to make a better and faster diagnosis in the struggle against the outbreak of COVID-19.

scholarly journals TRANSFER LEARNING BASED ON LOGISTIC REGRESSION

2015 ◽  
Vol XL-3/W3 ◽  
pp. 145-152
Author(s):  
A. Paul ◽  
F. Rottensteiner ◽  
C. Heipke
Keyword(s):  
Remote Sensing ◽  
Logistic Regression ◽  
Transfer Learning ◽  
Domain Adaptation ◽  
Real Data ◽  
Training Data ◽  
Model Parameters ◽  
Target Domain ◽  
Training Set ◽  
Data Set

In this paper we address the problem of classification of remote sensing images in the framework of transfer learning with a focus on domain adaptation. The main novel contribution is a method for transductive transfer learning in remote sensing on the basis of logistic regression. Logistic regression is a discriminative probabilistic classifier of low computational complexity, which can deal with multiclass problems. This research area deals with methods that solve problems in which labelled training data sets are assumed to be available only for a source domain, while classification is needed in the target domain with different, yet related characteristics. Classification takes place with a model of weight coefficients for hyperplanes which separate features in the transformed feature space. In term of logistic regression, our domain adaptation method adjusts the model parameters by iterative labelling of the target test data set. These labelled data features are iteratively added to the current training set which, at the beginning, only contains source features and, simultaneously, a number of source features are deleted from the current training set. Experimental results based on a test series with synthetic and real data constitutes a first proof-of-concept of the proposed method.

An efficient deep belief network for Detection of Coronavirus Disease COVID-19

2020 ◽  
pp. 05-13
Author(s):  
Shaymaa Adnan Abdulrahma ◽  
◽  
◽  
Abdel-Badeeh M. Salem
Keyword(s):  
Deep Learning ◽  
Early Detection ◽  
Respiratory Diseases ◽  
Medical Images ◽  
Respiratory Viruses ◽  
Data Repository ◽  
Data Set ◽  
Belief Network ◽  
Learning Technique ◽  
Normal Lungs

COVID-19 infection is one of the most dangerous respiratory viruses and the early detection of this disease reduces the speed of its spread among people . The goal of this virus is to infect the lung by creating white patchy shadows inside the lungs. This paper presents an intelligent method based on the deep learning technique to analyze the medical images of respiratory diseases . Two data set was used in this experiment first dataset is normal lungs taken from Kaggle data repository. While abnormal lungs taken from (https://github.com/muhammedtalo/COVID-19) .The results show that the proposed system identifies the COVID-19 cases with an accuracy of 90%.

scholarly journals Copy-Move Forgery Detection (CMFD) Using Deep Learning for Image and Video Forensics

2021 ◽  
Vol 7 (3) ◽  
pp. 59
Author(s):  
Yohanna Rodriguez-Ortega ◽  
Dora M. Ballesteros ◽  
Diego Renza
Keyword(s):  
Deep Learning ◽  
Transfer Learning ◽  
Training Data ◽  
Video Editing ◽  
Forgery Detection ◽  
Copy Move Forgery Detection ◽  
The Impact ◽  
And Training ◽  
Selection Of ◽  
Traditional Image

With the exponential growth of high-quality fake images in social networks and media, it is necessary to develop recognition algorithms for this type of content. One of the most common types of image and video editing consists of duplicating areas of the image, known as the copy-move technique. Traditional image processing approaches manually look for patterns related to the duplicated content, limiting their use in mass data classification. In contrast, approaches based on deep learning have shown better performance and promising results, but they present generalization problems with a high dependence on training data and the need for appropriate selection of hyperparameters. To overcome this, we propose two approaches that use deep learning, a model by a custom architecture and a model by transfer learning. In each case, the impact of the depth of the network is analyzed in terms of precision (P), recall (R) and F1 score. Additionally, the problem of generalization is addressed with images from eight different open access datasets. Finally, the models are compared in terms of evaluation metrics, and training and inference times. The model by transfer learning of VGG-16 achieves metrics about 10% higher than the model by a custom architecture, however, it requires approximately twice as much inference time as the latter.

Sign in / Sign up

Export Citation Format

Share Document