scholarly journals Efficient Use of GPU Memory for Large-Scale Deep Learning Model Training

2021 ◽  
Vol 11 (21) ◽  
pp. 10377
Author(s):  
Hyeonseong Choi ◽  
Jaehwan Lee
Keyword(s):  
Deep Learning ◽  
Large Scale ◽  
Training Model ◽  
High Accuracy ◽  
Access Pattern ◽  
Learning Models ◽  
Training Models ◽  
Model Training ◽  
Access Patterns ◽  
Deep Learning Model

To achieve high accuracy when performing deep learning, it is necessary to use a large-scale training model. However, due to the limitations of GPU memory, it is difficult to train large-scale training models within a single GPU. NVIDIA introduced a technology called CUDA Unified Memory with CUDA 6 to overcome the limitations of GPU memory by virtually combining GPU memory and CPU memory. In addition, in CUDA 8, memory advise options are introduced to efficiently utilize CUDA Unified Memory. In this work, we propose a newly optimized scheme based on CUDA Unified Memory to efficiently use GPU memory by applying different memory advise to each data type according to access patterns in deep learning training. We apply CUDA Unified Memory technology to PyTorch to see the performance of large-scale learning models through the expanded GPU memory. We conduct comprehensive experiments on how to efficiently utilize Unified Memory by applying memory advises when performing deep learning. As a result, when the data used for deep learning are divided into three types and a memory advise is applied to the data according to the access pattern, the deep learning execution time is reduced by 9.4% compared to the default Unified Memory.

scholarly journals A Physics-Infused Deep Learning Model for the Prediction of Refractive Indices and Its Use for the Large-Scale Screening of Organic Compound Space

2019 ◽  
Author(s):  
Mojtaba Haghighatlari ◽  
Gaurav Vishwakarma ◽  
Mohammad Atif Faiz Afzal ◽  
Johannes Hachmann
Keyword(s):  
Machine Learning ◽  
Deep Learning ◽  
Large Scale ◽  
Organic Molecules ◽  
Learning Model ◽  
Training Data ◽  
Refractive Indices ◽  
Learning Models ◽  
Deep Learning Model ◽  
Machine Learning Models

<div><div><div><p>We present a multitask, physics-infused deep learning model to accurately and efficiently predict refractive indices (RIs) of organic molecules, and we apply it to a library of 1.5 million compounds. We show that it outperforms earlier machine learning models by a significant margin, and that incorporating known physics into data-derived models provides valuable guardrails. Using a transfer learning approach, we augment the model to reproduce results consistent with higher-level computational chemistry training data, but with a considerably reduced number of corresponding calculations. Prediction errors of machine learning models are typically smallest for commonly observed target property values, consistent with the distribution of the training data. However, since our goal is to identify candidates with unusually large RI values, we propose a strategy to boost the performance of our model in the remoter areas of the RI distribution: We bias the model with respect to the under-represented classes of molecules that have values in the high-RI regime. By adopting a metric popular in web search engines, we evaluate our effectiveness in ranking top candidates. We confirm that the models developed in this study can reliably predict the RIs of the top 1,000 compounds, and are thus able to capture their ranking. We believe that this is the first study to develop a data-derived model that ensures the reliability of RI predictions by model augmentation in the extrapolation region on such a large scale. These results underscore the tremendous potential of machine learning in facilitating molecular (hyper)screening approaches on a massive scale and in accelerating the discovery of new compounds and materials, such as organic molecules with high-RI for applications in opto-electronics.</p></div></div></div>

scholarly journals Detecting orientation of Brain MR scans using deep learning

2021 ◽  
Author(s):  
Chinmay Singhal ◽  
Nihit Gupta ◽  
Anouk Stein ◽  
Quan Zhou ◽  
Leon Chen ◽  
...  
Keyword(s):  
Artificial Intelligence ◽  
Deep Learning ◽  
Free Text ◽  
Learning Models ◽  
Human Errors ◽  
Linguistic Barriers ◽  
Model Training ◽  
The Impact ◽  
Deep Learning Model ◽  
Made In

AbstractThere has been a steady escalation in the impact of Artificial Intelligence (AI) on Healthcare along with an increasing amount of progress being made in this field. While many entities are working on the development of significant deep learning models for the diagnosis of brain-related diseases, identifying precise images needed for model training and inference tasks is limited due to variation in DICOM fields which use free text to define things like series description, sequence and orientation [1]. Detecting the orientation of brain MR scans (Axial/Sagittal/Coronal) remains a challenge due to these variations caused by linguistic barriers, human errors and de-identification - essentially rendering the tags unreliable [2, 3, 4]. In this work, we propose a deep learning model that identifies the orientation of brain MR scans with near perfect accuracy.

scholarly journals A Physics-Infused Deep Learning Model for the Prediction of Refractive Indices and Its Use for the Large-Scale Screening of Organic Compound Space

2019 ◽  
Author(s):  
Mojtaba Haghighatlari ◽  
Gaurav Vishwakarma ◽  
Mohammad Atif Faiz Afzal ◽  
Johannes Hachmann
Keyword(s):  
Machine Learning ◽  
Deep Learning ◽  
Large Scale ◽  
Organic Molecules ◽  
Learning Model ◽  
Training Data ◽  
Refractive Indices ◽  
Learning Models ◽  
Deep Learning Model ◽  
Machine Learning Models

<div><div><div><p>We present a multitask, physics-infused deep learning model to accurately and efficiently predict refractive indices (RIs) of organic molecules, and we apply it to a library of 1.5 million compounds. We show that it outperforms earlier machine learning models by a significant margin, and that incorporating known physics into data-derived models provides valuable guardrails. Using a transfer learning approach, we augment the model to reproduce results consistent with higher-level computational chemistry training data, but with a considerably reduced number of corresponding calculations. Prediction errors of machine learning models are typically smallest for commonly observed target property values, consistent with the distribution of the training data. However, since our goal is to identify candidates with unusually large RI values, we propose a strategy to boost the performance of our model in the remoter areas of the RI distribution: We bias the model with respect to the under-represented classes of molecules that have values in the high-RI regime. By adopting a metric popular in web search engines, we evaluate our effectiveness in ranking top candidates. We confirm that the models developed in this study can reliably predict the RIs of the top 1,000 compounds, and are thus able to capture their ranking. We believe that this is the first study to develop a data-derived model that ensures the reliability of RI predictions by model augmentation in the extrapolation region on such a large scale. These results underscore the tremendous potential of machine learning in facilitating molecular (hyper)screening approaches on a massive scale and in accelerating the discovery of new compounds and materials, such as organic molecules with high-RI for applications in opto-electronics.</p></div></div></div>

scholarly journals MODEL LATIHAN KETERAMPILAN GERAK PENCAK SILAT ANAK USIA 9-12 TAHUN

2013 ◽  
Vol 1 (1) ◽  
pp. 26-36
Author(s):  
Bayu Iswana ◽  
Siswantoyo Siswantoyo
Keyword(s):  
Large Scale ◽  
Training Model ◽  
Small Scale ◽  
Training Models ◽  
Movement Skills ◽  
Qualitative Descriptive ◽  
Model Training ◽  
Result Analysis ◽  
Stage 1 ◽  
Yogyakarta City

Penelitian ini bertujuan untuk menghasilkan model latihan keterampilan gerak pencak silat anak usia 9-12 tahun. Penelitian pengembangan mengadaptasi langkah-langkah penelitian pengembangan dari Borg & Gall (1983, p.775), (1) pengumpulan informasi, (2) analisis hasil informasi, (3) pengembangan produk awal, (4) validasi ahli dan revisi tahap 1, (5) ujicoba skala kecil dan revisi, (6) ujicoba skala besar dan revisi tahap 2, (7) produk final. Uji coba skala kecil dilakukan terhadap anak Tapak Suci SD N 1 Padokan dan Tapak Suci SD Muhamadiyah Demangan. Uji coba skala besar dilakukan terhadap anak Pagar Nusa Sleman, Pagar Nusa Kota Yogyakarta yang berlatih di SD N Demangan, Persatuan Hati Bantul. Instrumen pengumpulan data, (1) wawancara, (2) skala nilai, (3) pedoman observasi model, (4) pedoman keefektifan model. Teknik analisis data yang digunakan yaitu analisis deskriptif kuantitatif dan diskriptif kualitatif. Isi dari hasil produk di dalamnya terdapat 6 model latihan, (1) kucing dan tikus, (2) bentengan, (3) gobak sodor, (4) jala ikan, (5) berburu burung, (7) elang dan anak ayam. Para ahli menyimpulkan bahwa di dalam model terdapat aspek kognitif, afektif dan psikomotor, sehingga model layak dan efektif untuk digunakan.  A TRAINING MODEL FOR PENCAK SILAT MOVEMENT SKILLS OF CHILDREN AGED 9-12 YEARSAbstract This study aims to produce a training model for pencak silat (self-defence) movement skills of children aged 9-12 years. This research and development (R & D) study was conducted by adapting the R & D steps by Borg & Gall (1983, p.775), i.e. (1) information collection, (2) information result analysis, (3) preliminary product development, (4) expert validation and stage 1 revision, (5) a small-scale tryout and a revision, (6) a large-scale tryout and stage 2 revision, and (7) final product. The small-scale tryout was conducted by involving participants of Tapak Suci SD N I Padokan and Tapak Suci SD Muhamadiyah Demangan. The large-scale tryout was conducted by involving participants of Pagar Nusa Sleman and Pagar Nusa Yogyakarta City carrying out training in SD N Demangan and Persatuan Hati Bantul. The data collecting instruments included (1) interviews, (2) a score scale, (3) a model observation guide, and (4) a model effectiveness guide. The data were anlyzed using the quantitative and qualitative descriptive techniques. The contents of the product consist of six training models, i.e. (1) kucing dan tikus, (2) bentengan, (3) gobak sodor, (4) jala ikan, (5) berburu burung  and (6) elang dan anak ayam. The experts conclude that in the model there are cognitive, affective, and psychomotor aspects so that it is appropriate and effective to use. Keywords: model, training, pencak silat, children aged 9 – 12 years

scholarly journals Application of Deep Learning in Power load Analysis

2020 ◽  
Vol 14 ◽  
Keyword(s):  
Deep Learning ◽  
Large Scale ◽  
Prediction Method ◽  
Training Model ◽  
Prediction Algorithm ◽  
Load Prediction ◽  
Weather Information ◽  
Power Load ◽  
Prediction Time ◽  
Model Training

Aiming at the problems of slow model training speed and poor prediction effect of traditional power load prediction algorithm, a parallel load prediction method based on deep learning is proposed. The method is based on the MapReduce parallel calculating framework, and the deep belief network model, which is used to parallel training the sample data with the historical load and the weather information, and the model of the training model to predict the load value. The experimental results show that the average root-mean-square error between the predicted power load value and the actual value of the prediction method in this paper is 2.86%. The prediction accuracy is higher than the traditional method, and the training and prediction time are effectively reduced, which can adapt to the prediction requirements of large-scale power data.

scholarly journals Deep Learning Model Comparison for Vision-Based Classification of Full/Empty-Load Trucks in Earthmoving Operations

2019 ◽  
Vol 9 (22) ◽  
pp. 4871 ◽  
Author(s):  
Quan Liu ◽  
Chen Feng ◽  
Zida Song ◽  
Joseph Louis ◽  
Jian Zhou
Keyword(s):  
Deep Learning ◽  
Model Comparison ◽  
Surveillance Systems ◽  
Comparison Study ◽  
Learning Models ◽  
The Core ◽  
Dump Trucks ◽  
Deep Learning Model ◽  
Contact Field

Earthmoving is an integral civil engineering operation of significance, and tracking its productivity requires the statistics of loads moved by dump trucks. Since current truck loads’ statistics methods are laborious, costly, and limited in application, this paper presents the framework of a novel, automated, non-contact field earthmoving quantity statistics (FEQS) for projects with large earthmoving demands that use uniform and uncovered trucks. The proposed FEQS framework utilizes field surveillance systems and adopts vision-based deep learning for full/empty-load truck classification as the core work. Since convolutional neural network (CNN) and its transfer learning (TL) forms are popular vision-based deep learning models and numerous in type, a comparison study is conducted to test the framework’s core work feasibility and evaluate the performance of different deep learning models in implementation. The comparison study involved 12 CNN or CNN-TL models in full/empty-load truck classification, and the results revealed that while several provided satisfactory performance, the VGG16-FineTune provided the optimal performance. This proved the core work feasibility of the proposed FEQS framework. Further discussion provides model choice suggestions that CNN-TL models are more feasible than CNN prototypes, and models that adopt different TL methods have advantages in either working accuracy or speed for different tasks.

scholarly journals Performance Comparison of the Deep Learning and the Human Endoscopist for Bleeding Peptic Ulcer Disease

2021 ◽  
Author(s):  
Hsu-Heng Yen ◽  
Ping-Yu Wu ◽  
Pei-Yuan Su ◽  
Chia-Wei Yang ◽  
Yang-Yuan Chen ◽  
...  
Keyword(s):  
Peptic Ulcer ◽  
Deep Learning ◽  
Sensitivity And Specificity ◽  
Endoscopic Therapy ◽  
Learning Model ◽  
Ulcer Bleeding ◽  
Learning Models ◽  
Peptic Ulcer Bleeding ◽  
Ulcer Disease ◽  
Deep Learning Model

Abstract Purpose Management of peptic ulcer bleeding is clinically challenging. Accurate characterization of the bleeding during endoscopy is key for endoscopic therapy. This study aimed to assess whether a deep learning model can aid in the classification of bleeding peptic ulcer disease. Methods Endoscopic still images of patients (n = 1694) with peptic ulcer bleeding for the last 5 years were retrieved and reviewed. Overall, 2289 images were collected for deep learning model training, and 449 images were validated for the performance test. Two expert endoscopists classified the images into different classes based on their appearance. Four deep learning models, including Mobile Net V2, VGG16, Inception V4, and ResNet50, were proposed and pre-trained by ImageNet with the established convolutional neural network algorithm. A comparison of the endoscopists and trained deep learning model was performed to evaluate the model’s performance on a dataset of 449 testing images. Results The results first presented the performance comparisons of four deep learning models. The Mobile Net V2 presented the optimal performance of the proposal models. The Mobile Net V2 was chosen for further comparing the performance with the diagnostic results obtained by one senior and one novice endoscopists. The sensitivity and specificity were acceptable for the prediction of “normal” lesions in both 3-class and 4-class classifications. For the 3-class category, the sensitivity and specificity were 94.83% and 92.36%, respectively. For the 4-class category, the sensitivity and specificity were 95.40% and 92.70%, respectively. The interobserver agreement of the testing dataset of the model was moderate to substantial with the senior endoscopist. The accuracy of the determination of endoscopic therapy required and high-risk endoscopic therapy of the deep learning model was higher than that of the novice endoscopist. Conclusions In this study, the deep learning model performed better than inexperienced endoscopists. Further improvement of the model may aid in clinical decision-making during clinical practice, especially for trainee endoscopist.

scholarly journals EyeHealer: A large-scale anterior eye segment dataset with eye structure and lesion annotations

2021 ◽  
Author(s):  
Wenjia Cai ◽  
Jie Xu ◽  
Ke Wang ◽  
Xiaohong Liu ◽  
Wenqin Xu ◽  
...  
Keyword(s):  
Deep Learning ◽  
Large Scale ◽  
Image Annotation ◽  
Clinical Care ◽  
Significant Proportion ◽  
Anterior Segment ◽  
Future Research ◽  
Learning Models ◽  
Anterior Eye Segment ◽  
Segmentation Models

Abstract Anterior segment eye diseases account for a significant proportion of presentations to eye clinics worldwide, including diseases associated with corneal pathologies, anterior chamber abnormalities (e.g. blood or inflammation) and lens diseases. The construction of an automatic tool for the segmentation of anterior segment eye lesions will greatly improve the efficiency of clinical care. With research on artificial intelligence progressing in recent years, deep learning models have shown their superiority in image classification and segmentation. The training and evaluation of deep learning models should be based on a large amount of data annotated with expertise, however, such data are relatively scarce in the domain of medicine. Herein, the authors developed a new medical image annotation system, called EyeHealer. It is a large-scale anterior eye segment dataset with both eye structures and lesions annotated at the pixel level. Comprehensive experiments were conducted to verify its performance in disease classification and eye lesion segmentation. The results showed that semantic segmentation models outperformed medical segmentation models. This paper describes the establishment of the system for automated classification and segmentation tasks. The dataset will be made publicly available to encourage future research in this area.

scholarly journals Evolving Deep Learning Convolutional Neural Networks for Early COVID-19 Detection in Chest X-ray Images

Mathematics ◽  
2021 ◽  
Vol 9 (9) ◽  
pp. 1002
Author(s):  
Mohammad Khishe ◽  
Fabio Caraffini ◽  
Stefan Kuhn
Keyword(s):  
Deep Learning ◽  
Early Detection ◽  
Iterative Process ◽  
High Accuracy ◽  
X Ray ◽  
Starting Point ◽  
Reliable Model ◽  
Minimum Number ◽  
Chest X Ray ◽  
Deep Learning Model

This article proposes a framework that automatically designs classifiers for the early detection of COVID-19 from chest X-ray images. To do this, our approach repeatedly makes use of a heuristic for optimisation to efficiently find the best combination of the hyperparameters of a convolutional deep learning model. The framework starts with optimising a basic convolutional neural network which represents the starting point for the evolution process. Subsequently, at most two additional convolutional layers are added, at a time, to the previous convolutional structure as a result of a further optimisation phase. Each performed phase maximises the the accuracy of the system, thus requiring training and assessment of the new model, which gets gradually deeper, with relevant COVID-19 chest X-ray images. This iterative process ends when no improvement, in terms of accuracy, is recorded. Hence, the proposed method evolves the most performing network with the minimum number of convolutional layers. In this light, we simultaneously achieve high accuracy while minimising the presence of redundant layers to guarantee a fast but reliable model. Our results show that the proposed implementation of such a framework achieves accuracy up to 99.11%, thus being particularly suitable for the early detection of COVID-19.

Sign in / Sign up

Export Citation Format

Share Document