scholarly journals LDP-FL: Practical Private Aggregation in Federated Learning with Local Differential Privacy

2021 ◽  
Author(s):  
Lichao Sun ◽  
Jianwei Qian ◽  
Xun Chen
Keyword(s):  
Deep Learning ◽  
Differential Privacy ◽  
Learning Performance ◽  
Learning Models ◽  
Privacy Concerns ◽  
User Data ◽  
Local Gradient ◽  
Privacy Budget ◽  
Deep Learning Model ◽  
Novel Design

Training deep learning models on sensitive user data has raised increasing privacy concerns in many areas. Federated learning is a popular approach for privacy protection that collects the local gradient information instead of raw data. One way to achieve a strict privacy guarantee is to apply local differential privacy into federated learning. However, previous works do not give a practical solution due to two issues. First, the range difference of weights in different deep learning model layers has not been explicitly considered when applying local differential privacy mechanism. Second, the privacy budget explodes due to the high dimensionality of weights in deep learning models and many query iterations of federated learning. In this paper, we proposed a novel design of local differential privacy mechanism for federated learning to address the abovementioned issues. It makes the local weights update differentially private by adapting to the varying ranges at different layers of a deep neural network, which introduces a smaller variance of the estimated model weights, especially for deeper models. Moreover, the proposed mechanism bypasses the curse of dimensionality by parameter shuffling aggregation. A series of empirical evaluations on three commonly used datasets in prior differential privacy works, MNIST, Fashion-MNIST and CIFAR-10, demonstrate that our solution can not only achieve superior deep learning performance but also provide a strong privacy guarantee at the same time.

scholarly journals Improving deep learning performance by using Explainable Artificial Intelligence (XAI) approaches

2021 ◽  
Vol 1 (1) ◽  
Author(s):  
Vitor Bento ◽  
Manoela Kohler ◽  
Pedro Diaz ◽  
Leonardo Mendoza ◽  
Marco Aurelio Pacheco
Keyword(s):  
Artificial Intelligence ◽  
Deep Learning ◽  
Surface Defect ◽  
Learning Performance ◽  
Learning Models ◽  
Surveillance Camera ◽  
Wide Range ◽  
Classification Tasks ◽  
Low Performance ◽  
Deep Learning Model

AbstractIn this work we propose a workflow to deal with overlaid images—images with superimposed text and company logos—, which is very common in underwater monitoring videos and surveillance camera footage. It is demonstrated that it is possible to use Explaining Artificial Intelligence to improve deep learning models performance for image classification tasks in general. A deep learning model trained to classify metal surface defect, which previously had a low performance, is then evaluated with Layer-wise relevance propagation—an Explaining Artificial Intelligence technique—to identify problems in a dataset that hinder the training of deep learning models in a wide range of applications. Thereafter, it is possible to remove this unwanted information from the dataset—using different approaches: from cutting part of the images to training a Generative Inpainting neural network model—and retrain the model with the new preprocessed images. This proposed methodology improved F1 score in 20% when compared to the original trained dataset, validating the proposed workflow.

scholarly journals Deep Learning Model Comparison for Vision-Based Classification of Full/Empty-Load Trucks in Earthmoving Operations

2019 ◽  
Vol 9 (22) ◽  
pp. 4871 ◽  
Author(s):  
Quan Liu ◽  
Chen Feng ◽  
Zida Song ◽  
Joseph Louis ◽  
Jian Zhou
Keyword(s):  
Deep Learning ◽  
Model Comparison ◽  
Surveillance Systems ◽  
Comparison Study ◽  
Learning Models ◽  
The Core ◽  
Dump Trucks ◽  
Deep Learning Model ◽  
Contact Field

Earthmoving is an integral civil engineering operation of significance, and tracking its productivity requires the statistics of loads moved by dump trucks. Since current truck loads’ statistics methods are laborious, costly, and limited in application, this paper presents the framework of a novel, automated, non-contact field earthmoving quantity statistics (FEQS) for projects with large earthmoving demands that use uniform and uncovered trucks. The proposed FEQS framework utilizes field surveillance systems and adopts vision-based deep learning for full/empty-load truck classification as the core work. Since convolutional neural network (CNN) and its transfer learning (TL) forms are popular vision-based deep learning models and numerous in type, a comparison study is conducted to test the framework’s core work feasibility and evaluate the performance of different deep learning models in implementation. The comparison study involved 12 CNN or CNN-TL models in full/empty-load truck classification, and the results revealed that while several provided satisfactory performance, the VGG16-FineTune provided the optimal performance. This proved the core work feasibility of the proposed FEQS framework. Further discussion provides model choice suggestions that CNN-TL models are more feasible than CNN prototypes, and models that adopt different TL methods have advantages in either working accuracy or speed for different tasks.

scholarly journals Performance Comparison of the Deep Learning and the Human Endoscopist for Bleeding Peptic Ulcer Disease

2021 ◽  
Author(s):  
Hsu-Heng Yen ◽  
Ping-Yu Wu ◽  
Pei-Yuan Su ◽  
Chia-Wei Yang ◽  
Yang-Yuan Chen ◽  
...  
Keyword(s):  
Peptic Ulcer ◽  
Deep Learning ◽  
Sensitivity And Specificity ◽  
Endoscopic Therapy ◽  
Learning Model ◽  
Ulcer Bleeding ◽  
Learning Models ◽  
Peptic Ulcer Bleeding ◽  
Ulcer Disease ◽  
Deep Learning Model

Abstract Purpose Management of peptic ulcer bleeding is clinically challenging. Accurate characterization of the bleeding during endoscopy is key for endoscopic therapy. This study aimed to assess whether a deep learning model can aid in the classification of bleeding peptic ulcer disease. Methods Endoscopic still images of patients (n = 1694) with peptic ulcer bleeding for the last 5 years were retrieved and reviewed. Overall, 2289 images were collected for deep learning model training, and 449 images were validated for the performance test. Two expert endoscopists classified the images into different classes based on their appearance. Four deep learning models, including Mobile Net V2, VGG16, Inception V4, and ResNet50, were proposed and pre-trained by ImageNet with the established convolutional neural network algorithm. A comparison of the endoscopists and trained deep learning model was performed to evaluate the model’s performance on a dataset of 449 testing images. Results The results first presented the performance comparisons of four deep learning models. The Mobile Net V2 presented the optimal performance of the proposal models. The Mobile Net V2 was chosen for further comparing the performance with the diagnostic results obtained by one senior and one novice endoscopists. The sensitivity and specificity were acceptable for the prediction of “normal” lesions in both 3-class and 4-class classifications. For the 3-class category, the sensitivity and specificity were 94.83% and 92.36%, respectively. For the 4-class category, the sensitivity and specificity were 95.40% and 92.70%, respectively. The interobserver agreement of the testing dataset of the model was moderate to substantial with the senior endoscopist. The accuracy of the determination of endoscopic therapy required and high-risk endoscopic therapy of the deep learning model was higher than that of the novice endoscopist. Conclusions In this study, the deep learning model performed better than inexperienced endoscopists. Further improvement of the model may aid in clinical decision-making during clinical practice, especially for trainee endoscopist.

Transfer Learning of The ResNet-18 and DenseNet-121 Model Used to Diagnose Intracranial Hemorrhage in CT Scanning

2021 ◽  
Vol 27 ◽  
Author(s):  
Qi Zhou ◽  
Wenjie Zhu ◽  
Fuchen Li ◽  
Mingqing Yuan ◽  
Linfeng Zheng ◽  
...  
Keyword(s):  
Deep Learning ◽  
Transfer Learning ◽  
Intracranial Hemorrhage ◽  
Intraventricular Hemorrhage ◽  
Learning Model ◽  
Normal Group ◽  
Subdural Hemorrhage ◽  
Model Assessment ◽  
Learning Models ◽  
Deep Learning Model

Objective: To verify the ability of the deep learning model in identifying five subtypes and normal images in noncontrast enhancement CT of intracranial hemorrhage. Method: A total of 351 patients (39 patients in the normal group, 312 patients in the intracranial hemorrhage group) performed with intracranial hemorrhage noncontrast enhanced CT were selected, with 2768 images in total (514 images for the normal group, 398 images for the epidural hemorrhage group, 501 images for the subdural hemorrhage group, 497 images for the intraventricular hemorrhage group, 415 images for the cerebral parenchymal hemorrhage group, and 443 images for the subarachnoid hemorrhage group). Based on the diagnostic reports of two radiologists with more than 10 years of experience, the ResNet-18 and DenseNet-121 deep learning models were selected. Transfer learning was used. 80% of the data was used for training models, 10% was used for validating model performance against overfitting, and the last 10% was used for the final evaluation of the model. Assessment indicators included accuracy, sensitivity, specificity, and AUC values. Results: The overall accuracy of ResNet-18 and DenseNet-121 models were 89.64% and 82.5%, respectively. The sensitivity and specificity of identifying five subtypes and normal images were above 0.80. The sensitivity of DenseNet-121 model to recognize intraventricular hemorrhage and cerebral parenchymal hemorrhage was lower than 0.80, 0.73, and 0.76 respectively. The AUC values of the two deep learning models were above 0.9. Conclusion: The deep learning model can accurately identify the five subtypes of intracranial hemorrhage and normal images, and it can be used as a new tool for clinical diagnosis in the future.

scholarly journals A Physics-Infused Deep Learning Model for the Prediction of Refractive Indices and Its Use for the Large-Scale Screening of Organic Compound Space

2019 ◽  
Author(s):  
Mojtaba Haghighatlari ◽  
Gaurav Vishwakarma ◽  
Mohammad Atif Faiz Afzal ◽  
Johannes Hachmann
Keyword(s):  
Machine Learning ◽  
Deep Learning ◽  
Large Scale ◽  
Organic Molecules ◽  
Learning Model ◽  
Training Data ◽  
Refractive Indices ◽  
Learning Models ◽  
Deep Learning Model ◽  
Machine Learning Models

<div><div><div><p>We present a multitask, physics-infused deep learning model to accurately and efficiently predict refractive indices (RIs) of organic molecules, and we apply it to a library of 1.5 million compounds. We show that it outperforms earlier machine learning models by a significant margin, and that incorporating known physics into data-derived models provides valuable guardrails. Using a transfer learning approach, we augment the model to reproduce results consistent with higher-level computational chemistry training data, but with a considerably reduced number of corresponding calculations. Prediction errors of machine learning models are typically smallest for commonly observed target property values, consistent with the distribution of the training data. However, since our goal is to identify candidates with unusually large RI values, we propose a strategy to boost the performance of our model in the remoter areas of the RI distribution: We bias the model with respect to the under-represented classes of molecules that have values in the high-RI regime. By adopting a metric popular in web search engines, we evaluate our effectiveness in ranking top candidates. We confirm that the models developed in this study can reliably predict the RIs of the top 1,000 compounds, and are thus able to capture their ranking. We believe that this is the first study to develop a data-derived model that ensures the reliability of RI predictions by model augmentation in the extrapolation region on such a large scale. These results underscore the tremendous potential of machine learning in facilitating molecular (hyper)screening approaches on a massive scale and in accelerating the discovery of new compounds and materials, such as organic molecules with high-RI for applications in opto-electronics.</p></div></div></div>

Abstract P319: Can Deep Learning Find the Ischemic Core on CT? Transfer Learning From Pre-Trained MRI-Based Networks

Stroke ◽  
2021 ◽  
Vol 52 (Suppl_1) ◽  
Author(s):  
Yannan Yu ◽  
Soren Christensen ◽  
Yuan Xie ◽  
Enhao Gong ◽  
Maarten G Lansberg ◽  
...  
Keyword(s):  
Deep Learning ◽  
Ground Truth ◽  
Learning Model ◽  
Fine Tuning ◽  
Learning Models ◽  
Starting Point ◽  
Stroke Lesion ◽  
Ischemic Core ◽  
Deep Learning Model

Objective: Ischemic core prediction from CT perfusion (CTP) remains inaccurate compared with gold standard diffusion-weighted imaging (DWI). We evaluated if a deep learning model to predict the DWI lesion from MR perfusion (MRP) could facilitate ischemic core prediction on CTP. Method: Using the multi-center CRISP cohort of acute ischemic stroke patient with CTP before thrombectomy, we included patients with major reperfusion (TICI score≥2b), adequate image quality, and follow-up MRI at 3-7 days. Perfusion parameters including Tmax, mean transient time, cerebral blood flow (CBF), and cerebral blood volume were reconstructed by RAPID software. Core lab experts outlined the stroke lesion on the follow-up MRI. A previously trained MRI model in a separate group of patients was used as a starting point, which used MRP parameters as input and RAPID ischemic core on DWI as ground truth. We fine-tuned this model, using CTP parameters as input, and follow-up MRI as ground truth. Another model was also trained from scratch with only CTP data. 5-fold cross validation was used. Performance of the models was compared with ischemic core (rCBF≤30%) from RAPID software to identify the presence of a large infarct (volume>70 or >100ml). Results: 94 patients in the CRISP trial met the inclusion criteria (mean age 67±15 years, 52% male, median baseline NIHSS 18, median 90-day mRS 2). Without fine-tuning, the MRI model had an agreement of 73% in infarct >70ml, and 69% in >100ml; the MRI model fine-tuned on CT improved the agreement to 77% and 73%; The CT model trained from scratch had agreements of 73% and 71%; All of the deep learning models outperformed the rCBF segmentation from RAPID, which had agreements of 51% and 64%. See Table and figure. Conclusions: It is feasible to apply MRP-based deep learning model to CT. Fine-tuning with CTP data further improves the predictions. All deep learning models predict the stroke lesion after major recanalization better than thresholding approaches based on rCBF.

scholarly journals Predicting glaucoma prior to its onset using deep learning

2019 ◽  
Author(s):  
Anshul Thakur ◽  
Michael Goldbaum ◽  
Siamak Yousefi
Keyword(s):  
Deep Learning ◽  
Visual Field ◽  
Optic Neuropathy ◽  
Disease Onset ◽  
Visual Fields ◽  
Learning Model ◽  
Glaucomatous Optic Neuropathy ◽  
Learning Models ◽  
Fundus Photographs ◽  
Deep Learning Model

AbstractPurposeTo assess the accuracy of deep learning models to predict glaucoma development from fundus photographs several years prior to disease onset.DesignA deep learning model for prediction of glaucomatous optic neuropathy or visual field abnormality from color fundus photographs.ParticipantsWe retrospectively included 66,721 fundus photographs from 3,272 eyes of 1,636 subjects to develop deep leaning models.MethodFundus photographs and visual fields were carefully examined by two independent readers from the optic disc and visual field reading centers of the ocular hypertension treatment study (OHTS). When an abnormality was detected by the readers, subject was recalled for re-testing to confirm the abnormality and further confirmation by an endpoint committee. Using OHTS data, deep learning models were trained and tested using 85% of the fundus photographs and further validated (re-tested) on the remaining (held-out) 15% of the fundus photographs.Main Outcome MeasuresAccuracy and area under the receiver-operating characteristic curve (AUC).ResultsThe AUC of the deep learning model in predicting glaucoma development 4-7 years prior to disease onset was 0.77 (95% confidence interval 0.75, 0.79). The accuracy of the model in predicting glaucoma development about 1-3 years prior to disease onset was 0.88 (0.86, 0.91). The accuracy of the model in detecting glaucoma after onset was 0.95 (0.94, 0.96).ConclusionsDeep learning models can predict glaucoma development prior to disease onset with reasonable accuracy. Eyes with visual field abnormality but not glaucomatous optic neuropathy had a higher tendency to be missed by deep learning algorithms.

scholarly journals Metaheuristic-based Deep COVID-19 Screening Model from Chest X-Ray Images

2021 ◽  
Vol 2021 ◽  
pp. 1-9
Author(s):  
Manjit Kaur ◽  
Vijay Kumar ◽  
Vaishali Yadav ◽  
Dilbag Singh ◽  
Naresh Kumar ◽  
...  
Keyword(s):  
Deep Learning ◽  
Learning Models ◽  
Huge Number ◽  
X Ray ◽  
Screening Model ◽  
Proposed Model ◽  
Strength Pareto Evolutionary Algorithm ◽  
Chest X Ray ◽  
Deep Learning Model

COVID-19 has affected the whole world drastically. A huge number of people have lost their lives due to this pandemic. Early detection of COVID-19 infection is helpful for treatment and quarantine. Therefore, many researchers have designed a deep learning model for the early diagnosis of COVID-19-infected patients. However, deep learning models suffer from overfitting and hyperparameter-tuning issues. To overcome these issues, in this paper, a metaheuristic-based deep COVID-19 screening model is proposed for X-ray images. The modified AlexNet architecture is used for feature extraction and classification of the input images. Strength Pareto evolutionary algorithm-II (SPEA-II) is used to tune the hyperparameters of modified AlexNet. The proposed model is tested on a four-class (i.e., COVID-19, tuberculosis, pneumonia, or healthy) dataset. Finally, the comparisons are drawn among the existing and the proposed models.

scholarly journals Distributed deep learning networks among institutions for medical imaging

2018 ◽  
Vol 25 (8) ◽  
pp. 945-954 ◽  
Author(s):  
Ken Chang ◽  
Niranjan Balachandar ◽  
Carson Lam ◽  
Darvin Yi ◽  
James Brown ◽  
...  
Keyword(s):  
Deep Learning ◽  
Patient Data ◽  
Attractive Alternative ◽  
Learning Networks ◽  
Learning Models ◽  
Algorithm Performance ◽  
Automated Support ◽  
Retinal Fundus ◽  
Weight Transfer ◽  
Deep Learning Model

Abstract Objective Deep learning has become a promising approach for automated support for clinical diagnosis. When medical data samples are limited, collaboration among multiple institutions is necessary to achieve high algorithm performance. However, sharing patient data often has limitations due to technical, legal, or ethical concerns. In this study, we propose methods of distributing deep learning models as an attractive alternative to sharing patient data. Methods We simulate the distribution of deep learning models across 4 institutions using various training heuristics and compare the results with a deep learning model trained on centrally hosted patient data. The training heuristics investigated include ensembling single institution models, single weight transfer, and cyclical weight transfer. We evaluated these approaches for image classification in 3 independent image collections (retinal fundus photos, mammography, and ImageNet). Results We find that cyclical weight transfer resulted in a performance that was comparable to that of centrally hosted patient data. We also found that there is an improvement in the performance of cyclical weight transfer heuristic with a high frequency of weight transfer. Conclusions We show that distributing deep learning models is an effective alternative to sharing patient data. This finding has implications for any collaborative deep learning study.

Sign in / Sign up

Export Citation Format

Share Document