scholarly journals COVID-19 early detection for imbalanced or low number of data using a regularized cost-sensitive CapsNet

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Malihe Javidi ◽  
Saeid Abbaasi ◽  
Sara Naybandi Atashi ◽  
Mahdi Jampour
Keyword(s):  
Deep Learning ◽  
State Of The Art ◽  
Imbalanced Data ◽  
Training Data ◽  
Lung Involvement ◽  
Deep Model ◽  
Positive Data ◽  
Deep Architecture ◽  
Learning Technique ◽  
Novel Coronavirus

AbstractWith the presence of novel coronavirus disease at the end of 2019, several approaches were proposed to help physicians detect the disease, such as using deep learning to recognize lung involvement based on the pattern of pneumonia. These approaches rely on analyzing the CT images and exploring the COVID-19 pathologies in the lung. Most of the successful methods are based on the deep learning technique, which is state-of-the-art. Nevertheless, the big drawback of the deep approaches is their need for many samples, which is not always possible. This work proposes a combined deep architecture that benefits both employed architectures of DenseNet and CapsNet. To more generalize the deep model, we propose a regularization term with much fewer parameters. The network convergence significantly improved, especially when the number of training data is small. We also propose a novel Cost-sensitive loss function for imbalanced data that makes our model feasible for the condition with a limited number of positive data. Our novelties make our approach more intelligent and potent in real-world situations with imbalanced data, popular in hospitals. We analyzed our approach on two publicly available datasets, HUST and COVID-CT, with different protocols. In the first protocol of HUST, we followed the original paper setup and outperformed it. With the second protocol of HUST, we show our approach superiority concerning imbalanced data. Finally, with three different validations of the COVID-CT, we provide evaluations in the presence of a low number of data along with a comparison with state-of-the-art.

scholarly journals Multiple Saliency and Channel Sensitivity Network for Aggregated Convolutional Feature

2019 ◽  
Vol 33 ◽  
pp. 9013-9020
Author(s):  
Xuanlu Xiang ◽  
Zhipeng Wang ◽  
Zhicheng Zhao ◽  
Fei Su
Keyword(s):  
State Of The Art ◽  
Image Representation ◽  
Training Data ◽  
Gram Matrix ◽  
Redundant Information ◽  
Deep Architecture ◽  
Benchmark Datasets ◽  
Supervised Methods ◽  
Ranking Loss ◽  
Effective Channel

In this paper, aiming at two key problems of instance-level image retrieval, i.e., the distinctiveness of image representation and the generalization ability of the model, we propose a novel deep architecture - Multiple Saliency and Channel Sensitivity Network(MSCNet). Specifically, to obtain distinctive global descriptors, an attention-based multiple saliency learning is first presented to highlight important details of the image, and then a simple but effective channel sensitivity module based on Gram matrix is designed to boost the channel discrimination and suppress redundant information. Additionally, in contrast to most existing feature aggregation methods, employing pre-trained deep networks, MSCNet can be trained in two modes: the first one is an unsupervised manner with an instance loss, and another is a supervised manner, which combines classification and ranking loss and only relies on very limited training data. Experimental results on several public benchmark datasets, i.e., Oxford buildings, Paris buildings and Holidays, indicate that the proposed MSCNet outperforms the state-of-the-art unsupervised and supervised methods.

scholarly journals Unsupervised Deep Learning via Affinity Diffusion

2020 ◽  
Vol 34 (07) ◽  
pp. 11029-11036
Author(s):  
Jiabo Huang ◽  
Qi Dong ◽  
Shaogang Gong ◽  
Xiatian Zhu
Keyword(s):  
Deep Learning ◽  
State Of The Art ◽  
General Purpose ◽  
Training Data ◽  
Learning Approach ◽  
Model Learning ◽  
Feature Representations ◽  
Discriminative Feature ◽  
Training Samples ◽  
Unsupervised Deep Learning

Convolutional neural networks (CNNs) have achieved unprecedented success in a variety of computer vision tasks. However, they usually rely on supervised model learning with the need for massive labelled training data, limiting dramatically their usability and deployability in real-world scenarios without any labelling budget. In this work, we introduce a general-purpose unsupervised deep learning approach to deriving discriminative feature representations. It is based on self-discovering semantically consistent groups of unlabelled training samples with the same class concepts through a progressive affinity diffusion process. Extensive experiments on object image classification and clustering show the performance superiority of the proposed method over the state-of-the-art unsupervised learning models using six common image recognition benchmarks including MNIST, SVHN, STL10, CIFAR10, CIFAR100 and ImageNet.

Oversampling Based on Data Augmentation in Convolutional Neural Network for Silicon Wafer Defect Classification

2020 ◽  
Author(s):  
Uzma Batool ◽  
Mohd Ibrahim Shapiai ◽  
Nordinah Ismail ◽  
Hilman Fauzi ◽  
Syahrizal Salleh
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Convolutional Neural Network ◽  
Silicon Wafer ◽  
Data Augmentation ◽  
Imbalanced Data ◽  
Training Data ◽  
Defect Classification ◽  
Learning Method ◽  
Test Set

Silicon wafer defect data collected from fabrication facilities is intrinsically imbalanced because of the variable frequencies of defect types. Frequently occurring types will have more influence on the classification predictions if a model gets trained on such skewed data. A fair classifier for such imbalanced data requires a mechanism to deal with type imbalance in order to avoid biased results. This study has proposed a convolutional neural network for wafer map defect classification, employing oversampling as an imbalance addressing technique. To have an equal participation of all classes in the classifier’s training, data augmentation has been employed, generating more samples in minor classes. The proposed deep learning method has been evaluated on a real wafer map defect dataset and its classification results on the test set returned a 97.91% accuracy. The results were compared with another deep learning based auto-encoder model demonstrating the proposed method, a potential approach for silicon wafer defect classification that needs to be investigated further for its robustness.

scholarly journals Empirical Evaluation of the Effect of Optimization and Regularization Techniques on the Generalization Performance of Deep Convolutional Neural Network

2020 ◽  
Vol 10 (21) ◽  
pp. 7817
Author(s):  
Ivana Marin ◽  
Ana Kuzmanic Skelin ◽  
Tamara Grujic
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Convolutional Neural Network ◽  
Empirical Evaluation ◽  
Training Data ◽  
Generalization Performance ◽  
Deep Model ◽  
Unseen Data ◽  
Regularization Techniques ◽  
Learning Architectures

The main goal of any classification or regression task is to obtain a model that will generalize well on new, previously unseen data. Due to the recent rise of deep learning and many state-of-the-art results obtained with deep models, deep learning architectures have become one of the most used model architectures nowadays. To generalize well, a deep model needs to learn the training data well without overfitting. The latter implies a correlation of deep model optimization and regularization with generalization performance. In this work, we explore the effect of the used optimization algorithm and regularization techniques on the final generalization performance of the model with convolutional neural network (CNN) architecture widely used in the field of computer vision. We give a detailed overview of optimization and regularization techniques with a comparative analysis of their performance with three CNNs on the CIFAR-10 and Fashion-MNIST image datasets.

scholarly journals Examining Deep Learning Architectures for Crime Classification and Prediction

Forecasting ◽  
2021 ◽  
Vol 3 (4) ◽  
pp. 741-762
Author(s):  
Panagiotis Stalidis ◽  
Theodoros Semertzidis ◽  
Petros Daras
Keyword(s):  
Deep Learning ◽  
State Of The Art ◽  
Open Data ◽  
Training Data ◽  
Crime Prediction ◽  
Crime Types ◽  
Improved Performance ◽  
Learning Architectures ◽  
And Training ◽  
Crime Classification

In this paper, a detailed study on crime classification and prediction using deep learning architectures is presented. We examine the effectiveness of deep learning algorithms in this domain and provide recommendations for designing and training deep learning systems for predicting crime areas, using open data from police reports. Having time-series of crime types per location as training data, a comparative study of 10 state-of-the-art methods against 3 different deep learning configurations is conducted. In our experiments with 5 publicly available datasets, we demonstrate that the deep learning-based methods consistently outperform the existing best-performing methods. Moreover, we evaluate the effectiveness of different parameters in the deep learning architectures and give insights for configuring them to achieve improved performance in crime classification and finally crime prediction.

scholarly journals A novel augmented deep transfer learning for classification of COVID-19 and other thoracic diseases from X-rays

2021 ◽  
Author(s):  
Fouzia Altaf ◽  
Syed M. S. Islam ◽  
Naeem Khalid Janjua
Keyword(s):  
Deep Learning ◽  
Medical Imaging ◽  
Transfer Learning ◽  
Medical Images ◽  
Training Data ◽  
Target Domain ◽  
Data Set ◽  
Learning Technique ◽  
Multi Class Classification ◽  
Effective Transfer

AbstractDeep learning has provided numerous breakthroughs in natural imaging tasks. However, its successful application to medical images is severely handicapped with the limited amount of annotated training data. Transfer learning is commonly adopted for the medical imaging tasks. However, a large covariant shift between the source domain of natural images and target domain of medical images results in poor transfer learning. Moreover, scarcity of annotated data for the medical imaging tasks causes further problems for effective transfer learning. To address these problems, we develop an augmented ensemble transfer learning technique that leads to significant performance gain over the conventional transfer learning. Our technique uses an ensemble of deep learning models, where the architecture of each network is modified with extra layers to account for dimensionality change between the images of source and target data domains. Moreover, the model is hierarchically tuned to the target domain with augmented training data. Along with the network ensemble, we also utilize an ensemble of dictionaries that are based on features extracted from the augmented models. The dictionary ensemble provides an additional performance boost to our method. We first establish the effectiveness of our technique with the challenging ChestXray-14 radiography data set. Our experimental results show more than 50% reduction in the error rate with our method as compared to the baseline transfer learning technique. We then apply our technique to a recent COVID-19 data set for binary and multi-class classification tasks. Our technique achieves 99.49% accuracy for the binary classification, and 99.24% for multi-class classification.

scholarly journals Tag-Aware Personalized Recommendation Using a Hybrid Deep Model

2017 ◽  
Author(s):  
Zhenghua Xu ◽  
Thomas Lukasiewicz ◽  
Cheng Chen ◽  
Yishu Miao ◽  
Xiangwu Meng
Keyword(s):  
Deep Learning ◽  
State Of The Art ◽  
Feature Space ◽  
Personalized Recommendation ◽  
Neural Network Approach ◽  
Deep Model ◽  
Deep Feature ◽  
Training Quality ◽  
Training Efficiency ◽  
Better Than

Recently, many efforts have been put into tag-aware personalized recommendation. However, due to uncontrolled vocabularies, social tags are usually redundant, sparse, and ambiguous. In this paper, we propose a deep neural network approach to solve this problem by mapping the tag-based user and item profiles to an abstract deep feature space, where the deep-semantic similarities between users and their target items (resp., irrelevant items) are maximized (resp., minimized). To ensure the scalability in practice, we further propose to improve this model's training efficiency by using hybrid deep learning and negative sampling. Experimental results show that our approach can significantly outperform the state-of-the-art baselines in tag-aware personalized recommendation (3.8 times better than the best baseline), and that using hybrid deep learning and negative sampling can dramatically enhance the model's training efficiency (hundreds of times quicker), while maintaining similar (and sometimes even better) training quality and recommendation performance.

scholarly journals Application of Machine Learning in Rhinology: A State of the Art Review

2020 ◽  
Vol 63 (8) ◽  
pp. 341-349
Author(s):  
Myeong Sang Yu
Keyword(s):  
Machine Learning ◽  
Big Data ◽  
Deep Learning ◽  
Medical Records ◽  
State Of The Art ◽  
Machine Learning Techniques ◽  
Machine Learning Technique ◽  
Learning Techniques ◽  
Learning Technique ◽  
Machine Learning Applications

The revolutionary development of artificial intelligence (AI) such as machine learning and deep learning have been one of the most important technology in many parts of industry, and also enhance huge changes in health care. The big data obtained from electrical medical records and digitalized images accelerated the application of AI technologies in medical fields. Machine learning techniques can deal with the complexity of big data which is difficult to apply traditional statistics. Recently, the deep learning techniques including convolutional neural network have been considered as a promising machine learning technique in medical imaging applications. In the era of precision medicine, otolaryngologists need to understand the potentialities, pitfalls and limitations of AI technology, and try to find opportunities to collaborate with data scientists. This article briefly introduce the basic concepts of machine learning and its techniques, and reviewed the current works on machine learning applications in the field of otolaryngology and rhinology.

scholarly journals Var-CNN: A Data-Efficient Website Fingerprinting Attack Based on Deep Learning

2019 ◽  
Vol 2019 (4) ◽  
pp. 292-310 ◽  
Author(s):  
Sanjit Bhat ◽  
David Lu ◽  
Albert Kwon ◽  
Srinivas Devadas
Keyword(s):  
Deep Learning ◽  
State Of The Art ◽  
False Positive Rate ◽  
True Positive Rate ◽  
Training Data ◽  
Open World ◽  
Prior Art ◽  
Lower False Positive Rate ◽  
Positive Rate ◽  
Fingerprinting Attack

Abstract In recent years, there have been several works that use website fingerprinting techniques to enable a local adversary to determine which website a Tor user visits. While the current state-of-the-art attack, which uses deep learning, outperforms prior art with medium to large amounts of data, it attains marginal to no accuracy improvements when both use small amounts of training data. In this work, we propose Var-CNN, a website fingerprinting attack that leverages deep learning techniques along with novel insights specific to packet sequence classification. In open-world settings with large amounts of data, Var-CNN attains over 1% higher true positive rate (TPR) than state-of-the-art attacks while achieving 4× lower false positive rate (FPR). Var-CNN’s improvements are especially notable in low-data scenarios, where it reduces the FPR of prior art by 3.12% while increasing the TPR by 13%. Overall, insights used to develop Var-CNN can be applied to future deep learning based attacks, and substantially reduce the amount of training data needed to perform a successful website fingerprinting attack. This shortens the time needed for data collection and lowers the likelihood of having data staleness issues.

scholarly journals LABEL-EFFICIENT DEEP LEARNING-BASED SEMANTIC SEGMENTATION OF BUILDING POINT CLOUDS AT LOD3 LEVEL

2021 ◽  
Vol XLIII-B2-2021 ◽  
pp. 449-456
Author(s):  
Y. Cao ◽  
M. Scaioni
Keyword(s):  
Deep Learning ◽  
State Of The Art ◽  
Semantic Segmentation ◽  
Point Clouds ◽  
Training Data ◽  
Second Step ◽  
Dynamic Graph ◽  
Input Point ◽  
Supervised Methods ◽  
Global And Local

Abstract. In recent research, fully supervised Deep Learning (DL) techniques and large amounts of pointwise labels are employed to train a segmentation network to be applied to buildings’ point clouds. However, fine-labelled buildings’ point clouds are hard to find and manually annotating pointwise labels is time-consuming and expensive. Consequently, the application of fully supervised DL for semantic segmentation of buildings’ point clouds at LoD3 level is severely limited. To address this issue, we propose a novel label-efficient DL network that obtains per-point semantic labels of LoD3 buildings’ point clouds with limited supervision. In general, it consists of two steps. The first step (Autoencoder – AE) is composed of a Dynamic Graph Convolutional Neural Network-based encoder and a folding-based decoder, designed to extract discriminative global and local features from input point clouds by reconstructing them without any label. The second step is semantic segmentation. By supplying a small amount of task-specific supervision, a segmentation network is proposed for semantically segmenting the encoded features acquired from the pre-trained AE. Experimentally, we evaluate our approach based on the ArCH dataset. Compared to the fully supervised DL methods, we find that our model achieved state-of-the-art results on the unseen scenes, with only 10% of labelled training data from fully supervised methods as input.

Sign in / Sign up

Export Citation Format

Share Document