scholarly journals Homogeneous Learning: Self-Attention Decentralized Deep Learning

2021 ◽  
Author(s):  
Yuwei Sun ◽  
Hideya Ochiai
Keyword(s):  
Deep Learning ◽  
Image Classification ◽  
Attention Mechanism ◽  
Network Intrusion ◽  
Decentralized Learning ◽  
Model Aggregation ◽  
Medical Image Classification ◽  
Model Training ◽  
Decentralized Architecture ◽  
Delayed Model

Federated learning (FL) has been facilitating privacy-preserving deep learning in many walks of life such as medical image classification, network intrusion detection, and so forth. Whereas it necessitates a central parameter server for model aggregation, which brings about delayed model communication and vulnerability to adversarial attacks. A fully decentralized architecture like Swarm Learning allows peer-to-peer communication among distributed nodes, without the central server. One of the most challenging issues in decentralized deep learning is that data owned by each node are usually non-independent and identically distributed (non-IID), causing time-consuming convergence of model training. To this end, we propose a decentralized learning model called Homogeneous Learning (HL) for tackling non-IID data with a self-attention mechanism. In HL, training performs on each round’s selected node, and the trained model of a node is sent to the next selected node at the end of each round. Notably, for the selection, the self-attention mechanism leverages reinforcement learning to observe a node’s inner state and its surrounding environment’s state, and find out which node should be selected to optimize the training. We evaluate our method with various scenarios for two different image classification tasks. The result suggests that HL can achieve a better performance compared with standalone learning and greatly reduce both the total training rounds by 50.8% and the communication cost by 74.6% for decentralized learning with non-IID data.

scholarly journals Homogeneous Learning: Self-Attention Decentralized Deep Learning

2021 ◽  
Author(s):  
Yuwei Sun ◽  
Hideya Ochiai
Keyword(s):  
Deep Learning ◽  
Image Classification ◽  
Attention Mechanism ◽  
Network Intrusion ◽  
Decentralized Learning ◽  
Model Aggregation ◽  
Medical Image Classification ◽  
Model Training ◽  
Decentralized Architecture ◽  
Delayed Model

Federated learning (FL) has been facilitating privacy-preserving deep learning in many walks of life such as medical image classification, network intrusion detection, and so forth. Whereas it necessitates a central parameter server for model aggregation, which brings about delayed model communication and vulnerability to adversarial attacks. A fully decentralized architecture like Swarm Learning allows peer-to-peer communication among distributed nodes, without the central server. One of the most challenging issues in decentralized deep learning is that data owned by each node are usually non-independent and identically distributed (non-IID), causing time-consuming convergence of model training. To this end, we propose a decentralized learning model called Homogeneous Learning (HL) for tackling non-IID data with a self-attention mechanism. In HL, training performs on each round’s selected node, and the trained model of a node is sent to the next selected node at the end of each round. Notably, for the selection, the self-attention mechanism leverages reinforcement learning to observe a node’s inner state and its surrounding environment’s state, and find out which node should be selected to optimize the training. We evaluate our method with various scenarios for two different image classification tasks. The result suggests that HL can achieve a better performance compared with standalone learning and greatly reduce both the total training rounds by 50.8% and the communication cost by 74.6% for decentralized learning with non-IID data.

scholarly journals Medical Image Classification Algorithm Based on Visual Attention Mechanism-MCNN

2021 ◽  
Vol 2021 ◽  
pp. 1-12
Author(s):  
Fengping An ◽  
Xiaowei Li ◽  
Xingmin Ma
Keyword(s):  
Deep Learning ◽  
Visual Attention ◽  
Image Classification ◽  
Medical Image ◽  
Medical Images ◽  
Attention Mechanism ◽  
Training Strategy ◽  
Medical Image Classification ◽  
Deep Learning Network ◽  
Visual Attention Mechanism

Due to the complexity of medical images, traditional medical image classification methods have been unable to meet the actual application needs. In recent years, the rapid development of deep learning theory has provided a technical approach for solving medical image classification. However, deep learning has the following problems in the application of medical image classification. First, it is impossible to construct a deep learning model with excellent performance according to the characteristics of medical images. Second, the current deep learning network structure and training strategies are less adaptable to medical images. Therefore, this paper first introduces the visual attention mechanism into the deep learning model so that the information can be extracted more effectively according to the problem of medical images, and the reasoning is realized at a finer granularity. It can increase the interpretability of the model. Additionally, to solve the problem of matching the deep learning network structure and training strategy to medical images, this paper will construct a novel multiscale convolutional neural network model that can automatically extract high-level discriminative appearance features from the original image, and the loss function uses the Mahalanobis distance optimization model to obtain a better training strategy, which can improve the robust performance of the network model. The medical image classification task is completed by the above method. Based on the above ideas, this paper proposes a medical classification algorithm based on a visual attention mechanism-multiscale convolutional neural network. The lung nodules and breast cancer images were classified by the method in this paper. The experimental results show that the accuracy of medical image classification in this paper is not only higher than that of traditional machine learning methods but also improved compared with other deep learning methods, and the method has good stability and robustness.

Medical image classification using synergic deep learning

2019 ◽  
Vol 54 ◽  
pp. 10-19 ◽  
Author(s):  
Jianpeng Zhang ◽  
Yutong Xie ◽  
Qi Wu ◽  
Yong Xia
Keyword(s):  
Deep Learning ◽  
Image Classification ◽  
Medical Image ◽  
Medical Image Classification

scholarly journals A Hybrid Classification Method of Medical Image Based on Deep Learning

2021 ◽  
Author(s):  
Yulong Wang ◽  
Xiaofeng Liao ◽  
Dewen Qiao ◽  
Jiahui Wu
Keyword(s):  
Deep Learning ◽  
Image Classification ◽  
Medical Image ◽  
Rapid Development ◽  
Medical Science ◽  
Image Features ◽  
Image Feature ◽  
Classification Method ◽  
Support Vector ◽  
Medical Image Classification

Abstract With the rapid development of modern medical science and technology, medical image classification has become a more and more challenging problem. However, in most traditional classification methods, image feature extraction is difficult, and the accuracy of classifier needs to be improved. Therefore, this paper proposes a high-accuracy medical image classification method based on deep learning, which is called hybrid CQ-SVM. Specifically, we combine the advantages of convolutional neural network (CNN) and support vector machine (SVM), and integrate the novel hybrid model. In our scheme, quantum-behaved particle swarm optimization algorithm (QPSO) is adopted to set its parameters automatically for solving the SVM parameter setting problem, CNN works as a trainable feature extractor and SVM optimized by QPSO performs as a trainable classifier. This method can automatically extract features from original medical images and generate predictions. The experimental results show that this method can extract better medical image features, and achieve higher classification accuracy.

scholarly journals HMIC: Hierarchical Medical Image Classification, A Deep Learning Approach

Information ◽  
2020 ◽  
Vol 11 (6) ◽  
pp. 318 ◽  
Author(s):  
Kamran Kowsari ◽  
Rasoul Sali ◽  
Lubaina Ehsan ◽  
William Adorno ◽  
Asad Ali ◽  
...  
Keyword(s):  
Deep Learning ◽  
Celiac Disease ◽  
Image Classification ◽  
Medical Image ◽  
Hierarchical Classification ◽  
Learning Approaches ◽  
Supervised Classifiers ◽  
Diagnosis And Classification ◽  
Medical Image Classification ◽  
Multi Class Classification

Image classification is central to the big data revolution in medicine. Improved information processing methods for diagnosis and classification of digital medical images have shown to be successful via deep learning approaches. As this field is explored, there are limitations to the performance of traditional supervised classifiers. This paper outlines an approach that is different from the current medical image classification tasks that view the issue as multi-class classification. We performed a hierarchical classification using our Hierarchical Medical Image classification (HMIC) approach. HMIC uses stacks of deep learning models to give particular comprehension at each level of the clinical picture hierarchy. For testing our performance, we use biopsy of the small bowel images that contain three categories in the parent level (Celiac Disease, Environmental Enteropathy, and histologically normal controls). For the child level, Celiac Disease Severity is classified into 4 classes (I, IIIa, IIIb, and IIIC).

Medical Image Classification based on an Adaptive Size Deep Learning Model

2021 ◽  
Vol 17 (3s) ◽  
pp. 1-18
Author(s):  
Xiangbin Liu ◽  
Jiesheng He ◽  
Liping Song ◽  
Shuai Liu ◽  
Gautam Srivastava
Keyword(s):  
Deep Learning ◽  
Image Classification ◽  
Medical Image ◽  
Rapid Development ◽  
Learning Model ◽  
Fine Tuning ◽  
Optimal Size ◽  
Image Dataset ◽  
Medical Image Classification ◽  
Deep Learning Model

With the rapid development of Artificial Intelligence (AI), deep learning has increasingly become a research hotspot in various fields, such as medical image classification. Traditional deep learning models use Bilinear Interpolation when processing classification tasks of multi-size medical image dataset, which will cause the loss of information of the image, and then affect the classification effect. In response to this problem, this work proposes a solution for an adaptive size deep learning model. First, according to the characteristics of the multi-size medical image dataset, the optimal size set module is proposed in combination with the unpooling process. Next, an adaptive deep learning model module is proposed based on the existing deep learning model. Then, the model is fused with the size fine-tuning module used to process multi-size medical images to obtain a solution of the adaptive size deep learning model. Finally, the proposed solution model is applied to the pneumonia CT medical image dataset. Through experiments, it can be seen that the model has strong robustness, and the classification effect is improved by about 4% compared with traditional algorithms.

scholarly journals A Full Stage Data Augmentation Method in Deep Convolutional Neural Network for Natural Image Classification

2020 ◽  
Vol 2020 ◽  
pp. 1-11 ◽  
Author(s):  
Qinghe Zheng ◽  
Mingqiang Yang ◽  
Xinyu Tian ◽  
Nan Jiang ◽  
Deqiang Wang
Keyword(s):  
Deep Learning ◽  
Image Classification ◽  
Network Architecture ◽  
Data Augmentation ◽  
Coarse Grained ◽  
Natural Image ◽  
Deep Convolutional Neural Networks ◽  
Specific Domain ◽  
Training Costs ◽  
Model Training

Nowadays, deep learning has achieved remarkable results in many computer vision related tasks, among which the support of big data is essential. In this paper, we propose a full stage data augmentation framework to improve the accuracy of deep convolutional neural networks, which can also play the role of implicit model ensemble without introducing additional model training costs. Simultaneous data augmentation during training and testing stages can ensure network optimization and enhance its generalization ability. Augmentation in two stages needs to be consistent to ensure the accurate transfer of specific domain information. Furthermore, this framework is universal for any network architecture and data augmentation strategy and therefore can be applied to a variety of deep learning based tasks. Finally, experimental results about image classification on the coarse-grained dataset CIFAR-10 (93.41%) and fine-grained dataset CIFAR-100 (70.22%) demonstrate the effectiveness of the framework by comparing with state-of-the-art results.

Sign in / Sign up

Export Citation Format

Share Document