O-086 End-to-end deep learning for recognition of ploidy status using time-lapse videos

2021 ◽  
Vol 36 (Supplement_1) ◽  
Author(s):  
C I Lee ◽  
Y R Su ◽  
C H Chen ◽  
T A Chang ◽  
E E S Kuo ◽  
...  
Keyword(s):  
Deep Learning ◽  
Optical Flow ◽  
Time Lapse ◽  
Learning Model ◽  
Learning Models ◽  
Group 2 ◽  
Deep Learning Model ◽  
Ploidy Status ◽  
Time Lapse Video ◽  
Group 1

Abstract Study question Our Retrospective study is to investigate an end-to-end deep learning model in identifying ploidy status through raw time-lapse video. Summary answer Our deep learning model demonstrates a proof of concept and potential in recognizing the ploidy status. What is known already Since the time-lapse system has been introduced into the IVF lab, the relationship between morphogenetic and ploidy status has been often discussed. However, the result has not yet reached a united conclusion due to some limitations such as human labeling. Besides the statistical approach, deep learning models have been utilized for ploidy prediction. As such approaches are single image-based, the performance remains unpromising as previous statistical-based research. Therefore, in order to move further toward clinical application, better research design and approach are needed. Study design, size, duration A retrospective analysis of the time-lapse videos and chromosomal status from 690 biopsied blastocysts cultured in a time-lapse incubator (EmbryoScope+, Vitrolife) between January 2017 and August 2018 in the Lee Women’s Hospital were assessed. The ploidy status of the blastocyst was derived from the PGT-A using high-resolution next-generation sequencing (hr-NGS). Embryo videos were obtained after normal fertilization through the intracytoplasmic sperm injection or conventional insemination. Participants/materials, setting, methods By randomly dividing the data into 80% and 20%, we developed our deep learning model based on Two-Stream Inflated 3D ConvNets(I3D) network. This model was trained by the 80% time-lapse videos and the PGT-A result. The remaining 20% has been tested by feeding the time-lapse video as input and the PGT-A prediction as output. Ploidy status was classified as Group 1 (aneuploidy) and Group 2 (euploidy and mosaicism). Main results and the role of chance Time-lapse videos were divided into 3-time partitions: day 1, day 1 to 3, and day 1 to 5. Deep learning models have been fed by RGB and optical flow. Combining 3 different time partitions with RGB, optical flow, and fused result from RGB and optical flow, we received nine sets of test results. According to the results, the longest time partition with the fusion method has the highest AUC result as 0.74, which appeared higher than the other eight experimental settings with a maximum increase of 0.17. Limitations, reasons for caution The present study is retrospective and future prospective research would help us to identify more key factors and improve this model. In addition, expanding sample size combined with cross-centered validation will also be considered in our future approach. Wider implications of the findings Group 1 and Group 2 approach provided deselection of aneuploidy embryos, while future deep learning approaches toward high mosaicism, low mosaicism, and euploidy will be needed, in order to provide a better clinical application. Trial registration number CS18082

scholarly journals Performance Comparison of the Deep Learning and the Human Endoscopist for Bleeding Peptic Ulcer Disease

2021 ◽  
Author(s):  
Hsu-Heng Yen ◽  
Ping-Yu Wu ◽  
Pei-Yuan Su ◽  
Chia-Wei Yang ◽  
Yang-Yuan Chen ◽  
...  
Keyword(s):  
Peptic Ulcer ◽  
Deep Learning ◽  
Sensitivity And Specificity ◽  
Endoscopic Therapy ◽  
Learning Model ◽  
Ulcer Bleeding ◽  
Learning Models ◽  
Peptic Ulcer Bleeding ◽  
Ulcer Disease ◽  
Deep Learning Model

Abstract Purpose Management of peptic ulcer bleeding is clinically challenging. Accurate characterization of the bleeding during endoscopy is key for endoscopic therapy. This study aimed to assess whether a deep learning model can aid in the classification of bleeding peptic ulcer disease. Methods Endoscopic still images of patients (n = 1694) with peptic ulcer bleeding for the last 5 years were retrieved and reviewed. Overall, 2289 images were collected for deep learning model training, and 449 images were validated for the performance test. Two expert endoscopists classified the images into different classes based on their appearance. Four deep learning models, including Mobile Net V2, VGG16, Inception V4, and ResNet50, were proposed and pre-trained by ImageNet with the established convolutional neural network algorithm. A comparison of the endoscopists and trained deep learning model was performed to evaluate the model’s performance on a dataset of 449 testing images. Results The results first presented the performance comparisons of four deep learning models. The Mobile Net V2 presented the optimal performance of the proposal models. The Mobile Net V2 was chosen for further comparing the performance with the diagnostic results obtained by one senior and one novice endoscopists. The sensitivity and specificity were acceptable for the prediction of “normal” lesions in both 3-class and 4-class classifications. For the 3-class category, the sensitivity and specificity were 94.83% and 92.36%, respectively. For the 4-class category, the sensitivity and specificity were 95.40% and 92.70%, respectively. The interobserver agreement of the testing dataset of the model was moderate to substantial with the senior endoscopist. The accuracy of the determination of endoscopic therapy required and high-risk endoscopic therapy of the deep learning model was higher than that of the novice endoscopist. Conclusions In this study, the deep learning model performed better than inexperienced endoscopists. Further improvement of the model may aid in clinical decision-making during clinical practice, especially for trainee endoscopist.

Transfer Learning of The ResNet-18 and DenseNet-121 Model Used to Diagnose Intracranial Hemorrhage in CT Scanning

2021 ◽  
Vol 27 ◽  
Author(s):  
Qi Zhou ◽  
Wenjie Zhu ◽  
Fuchen Li ◽  
Mingqing Yuan ◽  
Linfeng Zheng ◽  
...  
Keyword(s):  
Deep Learning ◽  
Transfer Learning ◽  
Intracranial Hemorrhage ◽  
Intraventricular Hemorrhage ◽  
Learning Model ◽  
Normal Group ◽  
Subdural Hemorrhage ◽  
Model Assessment ◽  
Learning Models ◽  
Deep Learning Model

Objective: To verify the ability of the deep learning model in identifying five subtypes and normal images in noncontrast enhancement CT of intracranial hemorrhage. Method: A total of 351 patients (39 patients in the normal group, 312 patients in the intracranial hemorrhage group) performed with intracranial hemorrhage noncontrast enhanced CT were selected, with 2768 images in total (514 images for the normal group, 398 images for the epidural hemorrhage group, 501 images for the subdural hemorrhage group, 497 images for the intraventricular hemorrhage group, 415 images for the cerebral parenchymal hemorrhage group, and 443 images for the subarachnoid hemorrhage group). Based on the diagnostic reports of two radiologists with more than 10 years of experience, the ResNet-18 and DenseNet-121 deep learning models were selected. Transfer learning was used. 80% of the data was used for training models, 10% was used for validating model performance against overfitting, and the last 10% was used for the final evaluation of the model. Assessment indicators included accuracy, sensitivity, specificity, and AUC values. Results: The overall accuracy of ResNet-18 and DenseNet-121 models were 89.64% and 82.5%, respectively. The sensitivity and specificity of identifying five subtypes and normal images were above 0.80. The sensitivity of DenseNet-121 model to recognize intraventricular hemorrhage and cerebral parenchymal hemorrhage was lower than 0.80, 0.73, and 0.76 respectively. The AUC values of the two deep learning models were above 0.9. Conclusion: The deep learning model can accurately identify the five subtypes of intracranial hemorrhage and normal images, and it can be used as a new tool for clinical diagnosis in the future.

Fusion of rain radar images and wind forecasts in adeep learning model applied to rain nowcasting

2021 ◽  
Author(s):  
Anastase Charantonis ◽  
Vincent Bouget ◽  
Dominique Béréziat ◽  
Julien Brajard ◽  
Arthur Filoche
Keyword(s):  
Deep Learning ◽  
Optical Flow ◽  
Weather Forecast ◽  
Forecast Model ◽  
Radar Data ◽  
Learning Model ◽  
Rainfall Forecasting ◽  
Radar Images ◽  
Deep Learning Model ◽  
Rain Radar

<p>Short or mid-term rainfall forecasting is a major task with several environmental applications such as agricultural management or flood risks monitoring. Existing data-driven approaches, especially deep learning models, have shown significant skill at this task, using only rainfall radar images as inputs. In order to determine whether using other meteorological parameters such as wind would improve forecasts, we trained a deep learning model on a fusion of rainfall radar images and wind velocity produced by a weather forecast model. The network was compared to a similar architecture trained only on radar data, to a basic persistence model and to an approach based on optical flow. Our network outperforms by 8% the F1-score calculated for the optical flow on moderate and higher rain events for forecasts at a horizon time of 30 minutes. Furthermore, it outperforms by 7% the same architecture trained using only rainfall radar images. Merging rain and wind data has also proven to stabilize the training process and enabled significant improvement especially on the difficult-to-predict high precipitation rainfalls. These results can also be found in Bouget, V., Béréziat, D., Brajard, J., Charantonis, A., & Filoche, A. (2020). Fusion of rain radar images and wind forecasts in a deep learning model applied to rain nowcasting. arXiv preprint arXiv:2012.05015</p>

scholarly journals A Physics-Infused Deep Learning Model for the Prediction of Refractive Indices and Its Use for the Large-Scale Screening of Organic Compound Space

2019 ◽  
Author(s):  
Mojtaba Haghighatlari ◽  
Gaurav Vishwakarma ◽  
Mohammad Atif Faiz Afzal ◽  
Johannes Hachmann
Keyword(s):  
Machine Learning ◽  
Deep Learning ◽  
Large Scale ◽  
Organic Molecules ◽  
Learning Model ◽  
Training Data ◽  
Refractive Indices ◽  
Learning Models ◽  
Deep Learning Model ◽  
Machine Learning Models

<div><div><div><p>We present a multitask, physics-infused deep learning model to accurately and efficiently predict refractive indices (RIs) of organic molecules, and we apply it to a library of 1.5 million compounds. We show that it outperforms earlier machine learning models by a significant margin, and that incorporating known physics into data-derived models provides valuable guardrails. Using a transfer learning approach, we augment the model to reproduce results consistent with higher-level computational chemistry training data, but with a considerably reduced number of corresponding calculations. Prediction errors of machine learning models are typically smallest for commonly observed target property values, consistent with the distribution of the training data. However, since our goal is to identify candidates with unusually large RI values, we propose a strategy to boost the performance of our model in the remoter areas of the RI distribution: We bias the model with respect to the under-represented classes of molecules that have values in the high-RI regime. By adopting a metric popular in web search engines, we evaluate our effectiveness in ranking top candidates. We confirm that the models developed in this study can reliably predict the RIs of the top 1,000 compounds, and are thus able to capture their ranking. We believe that this is the first study to develop a data-derived model that ensures the reliability of RI predictions by model augmentation in the extrapolation region on such a large scale. These results underscore the tremendous potential of machine learning in facilitating molecular (hyper)screening approaches on a massive scale and in accelerating the discovery of new compounds and materials, such as organic molecules with high-RI for applications in opto-electronics.</p></div></div></div>

Abstract P319: Can Deep Learning Find the Ischemic Core on CT? Transfer Learning From Pre-Trained MRI-Based Networks

Stroke ◽  
2021 ◽  
Vol 52 (Suppl_1) ◽  
Author(s):  
Yannan Yu ◽  
Soren Christensen ◽  
Yuan Xie ◽  
Enhao Gong ◽  
Maarten G Lansberg ◽  
...  
Keyword(s):  
Deep Learning ◽  
Ground Truth ◽  
Learning Model ◽  
Fine Tuning ◽  
Learning Models ◽  
Starting Point ◽  
Stroke Lesion ◽  
Ischemic Core ◽  
Deep Learning Model

Objective: Ischemic core prediction from CT perfusion (CTP) remains inaccurate compared with gold standard diffusion-weighted imaging (DWI). We evaluated if a deep learning model to predict the DWI lesion from MR perfusion (MRP) could facilitate ischemic core prediction on CTP. Method: Using the multi-center CRISP cohort of acute ischemic stroke patient with CTP before thrombectomy, we included patients with major reperfusion (TICI score≥2b), adequate image quality, and follow-up MRI at 3-7 days. Perfusion parameters including Tmax, mean transient time, cerebral blood flow (CBF), and cerebral blood volume were reconstructed by RAPID software. Core lab experts outlined the stroke lesion on the follow-up MRI. A previously trained MRI model in a separate group of patients was used as a starting point, which used MRP parameters as input and RAPID ischemic core on DWI as ground truth. We fine-tuned this model, using CTP parameters as input, and follow-up MRI as ground truth. Another model was also trained from scratch with only CTP data. 5-fold cross validation was used. Performance of the models was compared with ischemic core (rCBF≤30%) from RAPID software to identify the presence of a large infarct (volume>70 or >100ml). Results: 94 patients in the CRISP trial met the inclusion criteria (mean age 67±15 years, 52% male, median baseline NIHSS 18, median 90-day mRS 2). Without fine-tuning, the MRI model had an agreement of 73% in infarct >70ml, and 69% in >100ml; the MRI model fine-tuned on CT improved the agreement to 77% and 73%; The CT model trained from scratch had agreements of 73% and 71%; All of the deep learning models outperformed the rCBF segmentation from RAPID, which had agreements of 51% and 64%. See Table and figure. Conclusions: It is feasible to apply MRP-based deep learning model to CT. Fine-tuning with CTP data further improves the predictions. All deep learning models predict the stroke lesion after major recanalization better than thresholding approaches based on rCBF.

scholarly journals Predicting glaucoma prior to its onset using deep learning

2019 ◽  
Author(s):  
Anshul Thakur ◽  
Michael Goldbaum ◽  
Siamak Yousefi
Keyword(s):  
Deep Learning ◽  
Visual Field ◽  
Optic Neuropathy ◽  
Disease Onset ◽  
Visual Fields ◽  
Learning Model ◽  
Glaucomatous Optic Neuropathy ◽  
Learning Models ◽  
Fundus Photographs ◽  
Deep Learning Model

AbstractPurposeTo assess the accuracy of deep learning models to predict glaucoma development from fundus photographs several years prior to disease onset.DesignA deep learning model for prediction of glaucomatous optic neuropathy or visual field abnormality from color fundus photographs.ParticipantsWe retrospectively included 66,721 fundus photographs from 3,272 eyes of 1,636 subjects to develop deep leaning models.MethodFundus photographs and visual fields were carefully examined by two independent readers from the optic disc and visual field reading centers of the ocular hypertension treatment study (OHTS). When an abnormality was detected by the readers, subject was recalled for re-testing to confirm the abnormality and further confirmation by an endpoint committee. Using OHTS data, deep learning models were trained and tested using 85% of the fundus photographs and further validated (re-tested) on the remaining (held-out) 15% of the fundus photographs.Main Outcome MeasuresAccuracy and area under the receiver-operating characteristic curve (AUC).ResultsThe AUC of the deep learning model in predicting glaucoma development 4-7 years prior to disease onset was 0.77 (95% confidence interval 0.75, 0.79). The accuracy of the model in predicting glaucoma development about 1-3 years prior to disease onset was 0.88 (0.86, 0.91). The accuracy of the model in detecting glaucoma after onset was 0.95 (0.94, 0.96).ConclusionsDeep learning models can predict glaucoma development prior to disease onset with reasonable accuracy. Eyes with visual field abnormality but not glaucomatous optic neuropathy had a higher tendency to be missed by deep learning algorithms.

scholarly journals An Interpretable Deep Learning Model for Automatic Sound Classification

Electronics ◽  
2021 ◽  
Vol 10 (7) ◽  
pp. 850
Author(s):  
Pablo Zinemanas ◽  
Martín Rocamora ◽  
Marius Miron ◽  
Frederic Font ◽  
Xavier Serra
Keyword(s):  
Deep Learning ◽  
Web Application ◽  
Domain Knowledge ◽  
Learning Model ◽  
Unintended Effects ◽  
Learning Models ◽  
Time Frequency ◽  
Sound Classification ◽  
Proposed Model ◽  
Deep Learning Model

Deep learning models have improved cutting-edge technologies in many research areas, but their black-box structure makes it difficult to understand their inner workings and the rationale behind their predictions. This may lead to unintended effects, such as being susceptible to adversarial attacks or the reinforcement of biases. There is still a lack of research in the audio domain, despite the increasing interest in developing deep learning models that provide explanations of their decisions. To reduce this gap, we propose a novel interpretable deep learning model for automatic sound classification, which explains its predictions based on the similarity of the input to a set of learned prototypes in a latent space. We leverage domain knowledge by designing a frequency-dependent similarity measure and by considering different time-frequency resolutions in the feature space. The proposed model achieves results that are comparable to that of the state-of-the-art methods in three different sound classification tasks involving speech, music, and environmental audio. In addition, we present two automatic methods to prune the proposed model that exploit its interpretability. Our system is open source and it is accompanied by a web application for the manual editing of the model, which allows for a human-in-the-loop debugging approach.

scholarly journals Optimising Deep Learning at the Edge for Accurate Hourly Air Quality Prediction

Sensors ◽  
2021 ◽  
Vol 21 (4) ◽  
pp. 1064
Author(s):  
I Nyoman Kusuma Wardana ◽  
Julian W. Gardner ◽  
Suhaib A. Fahmy
Keyword(s):  
Deep Learning ◽  
Air Quality ◽  
Short Term Memory ◽  
Model Performance ◽  
Learning Model ◽  
Sensor Data ◽  
Raspberry Pi ◽  
Learning Models ◽  
Proposed Model ◽  
Deep Learning Model

Accurate air quality monitoring requires processing of multi-dimensional, multi-location sensor data, which has previously been considered in centralised machine learning models. These are often unsuitable for resource-constrained edge devices. In this article, we address this challenge by: (1) designing a novel hybrid deep learning model for hourly PM2.5 pollutant prediction; (2) optimising the obtained model for edge devices; and (3) examining model performance running on the edge devices in terms of both accuracy and latency. The hybrid deep learning model in this work comprises a 1D Convolutional Neural Network (CNN) and a Long Short-Term Memory (LSTM) to predict hourly PM2.5 concentration. The results show that our proposed model outperforms other deep learning models, evaluated by calculating RMSE and MAE errors. The proposed model was optimised for edge devices, the Raspberry Pi 3 Model B+ (RPi3B+) and Raspberry Pi 4 Model B (RPi4B). This optimised model reduced file size to a quarter of the original, with further size reduction achieved by implementing different post-training quantisation. In total, 8272 hourly samples were continuously fed to the edge device, with the RPi4B executing the model twice as fast as the RPi3B+ in all quantisation modes. Full-integer quantisation produced the lowest execution time, with latencies of 2.19 s and 4.73 s for RPi4B and RPi3B+, respectively.

Deep Learning for Crowd Counting: A Survey

2019 ◽  
Vol 1 (1) ◽  
pp. 17-28
Author(s):  
Tjeng Wawan Cenggoro
Keyword(s):  
Deep Learning ◽  
Learning Model ◽  
Learning Models ◽  
Crowd Counting ◽  
Big Picture ◽  
Deep Learning Model

The growth of deep learning for crowd counting is immense in the recent years. This results in numerous deep learning model developed with huge multifariousness. This paper aims to capture a big picture of existing deep learning models for crowd counting. Hence, the development of novel models for future works can be accelerated.

scholarly journals Packet Preprocessing in CNN-Based Network Intrusion Detection System

Electronics ◽  
2020 ◽  
Vol 9 (7) ◽  
pp. 1151 ◽  
Author(s):  
Wooyeon Jo ◽  
Sungjin Kim ◽  
Changhoon Lee ◽  
Taeshik Shon
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Intrusion Detection ◽  
Intrusion Detection System ◽  
Detection System ◽  
Learning Model ◽  
Learning Models ◽  
Packet Data ◽  
Deep Learning Model ◽  
Better Than

The proliferation of various connected platforms, including Internet of things, industrial control systems (ICSs), connected cars, and in-vehicle networks, has resulted in the simultaneous use of multiple protocols and devices. Chaotic situations caused by the usage of different protocols and various types of devices, such as heterogeneous networks, implemented differently by vendors renders the adoption of a flexible security solution difficult, such as recent deep learning-based intrusion detection system (IDS) studies. These studies optimized the deep learning model for their environment to improve performance, but the basic principle of the deep learning model used was not changed, so this can be called a next-generation IDS with a model that has little or no requirements. Some studies proposed IDS based on unsupervised learning technology that does not require labeled data. However, not using available assets, such as network packet data, is a waste of resources. If the security solution considers the role and importance of the devices constituting the network and the security area of the protocol standard by experts, the assets can be well used, but it will no longer be flexible. Most deep learning model-based IDS studies used recurrent neural network (RNN), which is a supervised learning model, because the characteristics of the RNN model, especially when the long-short term memory (LSTM) is incorporated, are better configured to reflect the flow of the packet data stream over time, and thus perform better than other supervised learning models such as convolutional neural network (CNN). However, if the input data induce the CNN’s kernel to sufficiently reflect the network characteristics through proper preprocessing, it could perform better than other deep learning models in the network IDS. Hence, we propose the first preprocessing method, called “direct”, for network IDS that can use the characteristics of the kernel by using the minimum protocol information, field size, and offset. In addition to direct, we propose two more preprocessing techniques called “weighted” and “compressed”. Each requires additional network information; therefore, direct conversion was compared with related studies. Including direct, the proposed preprocessing methods are based on field-to-pixel philosophy, which can reflect the advantages of CNN by extracting the convolutional features of each pixel. Direct is the most intuitive method of applying field-to-pixel conversion to reflect an image’s convolutional characteristics in the CNN. Weighted and compressed are conversion methods used to evaluate the direct method. Consequently, the IDS constructed using a CNN with the proposed direct preprocessing method demonstrated meaningful performance in the NSL-KDD dataset.

Sign in / Sign up

Export Citation Format

Share Document