scholarly journals Using machine learning to model the training scalability of convolutional neural networks on clusters of GPUs

Computing ◽  
2021 ◽  
Author(s):  
Sergio Barrachina ◽  
Adrián Castelló ◽  
Mar Catalán ◽  
Manuel F. Dolz ◽  
Jose I. Mestre
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Training Costs ◽  
Data Parallel ◽  
Proposed Model ◽  
Batch Sizes ◽  
Arithmetic Performance ◽  
Point Arithmetic ◽  
Cluster Level ◽  
Analyze Data

AbstractIn this work, we build a general piece-wise model to analyze data-parallel (DP) training costs of convolutional neural networks (CNNs) on clusters of GPUs. This general model is based on i) multi-layer perceptrons (MLPs) in charge of modeling the NVIDIA cuDNN/cuBLAS library kernels involved in the training of some of the state-of-the-art CNNs; and ii) an analytical model in charge of modeling the NVIDIA NCCL Allreduce collective primitive using the Ring algorithm. The CNN training scalability study performed using this model in combination with the Roofline technique on varying batch sizes, node (floating-point) arithmetic performance, node memory bandwidth, network link bandwidth, and cluster dimension unveil some crucial bottlenecks at both GPU and cluster level. To provide evidence of this analysis, we validate the accuracy of the proposed model against a Python library for distributed deep learning training.

scholarly journals Stock Pattern Classification from Charts using Deep Learning Algorithms

2020 ◽  
Vol 3 (1) ◽  
pp. 445-454
Author(s):  
Celal Buğra Kaya ◽  
Alperen Yılmaz ◽  
Gizem Nur Uzun ◽  
Zeynep Hilal Kilimci
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Convolutional Neural Networks ◽  
Pattern Classification ◽  
Short Term Memory ◽  
Stock Exchange ◽  
Learning Techniques ◽  
Proposed Model ◽  
Istanbul Stock Exchange

Pattern classification is related with the automatic finding of regularities in dataset through the utilization of various learning techniques. Thus, the classification of the objects into a set of categories or classes is provided. This study is undertaken to evaluate deep learning methodologies to the classification of stock patterns. In order to classify patterns that are obtained from stock charts, convolutional neural networks (CNNs), recurrent neural networks (RNNs), and long-short term memory networks (LSTMs) are employed. To demonstrate the efficiency of proposed model in categorizing patterns, hand-crafted image dataset is constructed from stock charts in Istanbul Stock Exchange and NASDAQ Stock Exchange. Experimental results show that the usage of convolutional neural networks exhibits superior classification success in recognizing patterns compared to the other deep learning methodologies.

scholarly journals Auditory Inspired Convolutional Neural Networks for Ship Type Classification with Raw Hydrophone Data

Entropy ◽  
2018 ◽  
Vol 20 (12) ◽  
pp. 990 ◽  
Author(s):  
Sheng Shen ◽  
Honghui Yang ◽  
Junhao Li ◽  
Guanghui Xu ◽  
Meiping Sheng
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Filter Banks ◽  
Auditory Filter ◽  
Feature Representations ◽  
Proposed Model ◽  
Energy Pooling ◽  
Practical Guidelines ◽  
Frequency Components ◽  
Underwater Acoustic Signal

Detecting and classifying ships based on radiated noise provide practical guidelines for the reduction of underwater noise footprint of shipping. In this paper, the detection and classification are implemented by auditory inspired convolutional neural networks trained from raw underwater acoustic signal. The proposed model includes three parts. The first part is performed by a multi-scale 1D time convolutional layer initialized by auditory filter banks. Signals are decomposed into frequency components by convolution operation. In the second part, the decomposed signals are converted into frequency domain by permute layer and energy pooling layer to form frequency distribution in auditory cortex. Then, 2D frequency convolutional layers are applied to discover spectro-temporal patterns, as well as preserve locality and reduce spectral variations in ship noise. In the third part, the whole model is optimized with an objective function of classification to obtain appropriate auditory filters and feature representations that are correlative with ship categories. The optimization reflects the plasticity of auditory system. Experiments on five ship types and background noise show that the proposed approach achieved an overall classification accuracy of 79.2%, which improved by 6% compared to conventional approaches. Auditory filter banks were adaptive in shape to improve accuracy of classification.

scholarly journals BE-FNet: 3D Bounding Box Estimation Feature Pyramid Network for Accurate and Efficient Maxillary Sinus Segmentation

2020 ◽  
Vol 2020 ◽  
pp. 1-16
Author(s):  
Zhuofu Deng ◽  
Binbin Wang ◽  
Zhiliang Zhu
Keyword(s):  
Neural Networks ◽  
Maxillary Sinus ◽  
Convolutional Neural Networks ◽  
Feature Fusion ◽  
Medical Images ◽  
Medical Image Segmentation ◽  
3D Segmentation ◽  
Deep Convolutional Neural Networks ◽  
Bounding Box ◽  
Proposed Model

Maxillary sinus segmentation plays an important role in the choice of therapeutic strategies for nasal disease and treatment monitoring. Difficulties in traditional approaches deal with extremely heterogeneous intensity caused by lesions, abnormal anatomy structures, and blurring boundaries of cavity. 2D and 3D deep convolutional neural networks have grown popular in medical image segmentation due to utilization of large labeled datasets to learn discriminative features. However, for 3D segmentation in medical images, 2D networks are not competent in extracting more significant spacial features, and 3D ones suffer from unbearable burden of computation, which results in great challenges to maxillary sinus segmentation. In this paper, we propose a deep neural network with an end-to-end manner to generalize a fully automatic 3D segmentation. At first, our proposed model serves a symmetrical encoder-decoder architecture for multitask of bounding box estimation and in-region 3D segmentation, which cannot reduce excessive computation requirements but eliminate false positives remarkably, promoting 3D segmentation applied in 3D convolutional neural networks. In addition, an overestimation strategy is presented to avoid overfitting phenomena in conventional multitask networks. Meanwhile, we introduce residual dense blocks to increase the depth of the proposed network and attention excitation mechanism to improve the performance of bounding box estimation, both of which bring little influence to computation cost. Especially, the structure of multilevel feature fusion in the pyramid network strengthens the ability of identification to global and local discriminative features in foreground and background achieving more advanced segmentation results. At last, to address problems of blurring boundary and class imbalance in medical images, a hybrid loss function is designed for multiple tasks. To illustrate the strength of our proposed model, we evaluated it against the state-of-the-art methods. Our model performed better significantly with an average Dice 0.947±0.031, VOE 10.23±5.29, and ASD 2.86±2.11, respectively, which denotes a promising technique with strong robust in practice.

scholarly journals Deep Learning for Detecting Building Defects Using Convolutional Neural Networks

Sensors ◽  
2019 ◽  
Vol 19 (16) ◽  
pp. 3556 ◽  
Author(s):  
Husein Perez ◽  
Joseph H. M. Tah ◽  
Amir Mosavi
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Effective Means ◽  
Scale Up ◽  
Real Life ◽  
Health And Safety ◽  
Condition Assessment ◽  
Automated Detection ◽  
Proposed Model ◽  
Current Asset

Clients are increasingly looking for fast and effective means to quickly and frequently survey and communicate the condition of their buildings so that essential repairs and maintenance work can be done in a proactive and timely manner before it becomes too dangerous and expensive. Traditional methods for this type of work commonly comprise of engaging building surveyors to undertake a condition assessment which involves a lengthy site inspection to produce a systematic recording of the physical condition of the building elements, including cost estimates of immediate and projected long-term costs of renewal, repair and maintenance of the building. Current asset condition assessment procedures are extensively time consuming, laborious, and expensive and pose health and safety threats to surveyors, particularly at height and roof levels which are difficult to access. This paper aims at evaluating the application of convolutional neural networks (CNN) towards an automated detection and localisation of key building defects, e.g., mould, deterioration, and stain, from images. The proposed model is based on pre-trained CNN classifier of VGG-16 (later compaired with ResNet-50, and Inception models), with class activation mapping (CAM) for object localisation. The challenges and limitations of the model in real-life applications have been identified. The proposed model has proven to be robust and able to accurately detect and localise building defects. The approach is being developed with the potential to scale-up and further advance to support automated detection of defects and deterioration of buildings in real-time using mobile devices and drones.

scholarly journals Transfer learning with pre-trained deep convolutional neural networks for the automatic assessment of liver steatosis in ultrasound images

2020 ◽  
Author(s):  
Elena Codruta Constantinescu ◽  
Anca-Loredana Udriștoiu ◽  
Ștefan Cristinel Udriștoiu ◽  
Andreea Valentina Iacob ◽  
Lucian Gheorghe Gruionu ◽  
...  
Keyword(s):  
Neural Networks ◽  
Fatty Liver ◽  
Receiver Operating Characteristic ◽  
Convolutional Neural Networks ◽  
Operating Characteristic ◽  
Liver Steatosis ◽  
Normal Liver ◽  
Test Accuracy ◽  
Receiver Operating Characteristic Curves ◽  
Proposed Model

Aim: In this paper we proposed different architectures of convolutional neural network (CNN) to classify fatty liver disease in images using only pixels and diagnosis labels as input. We trained and validated our models using a dataset of 629 images consisting of 2 types of liver images, normal and liver steatosis. Material and methods: We assessed two pre-trained models of convolutional neural networks, Inception-v3 and VGG-16 using fine-tuning. Both models were pre-trained on ImageNet dataset to extract features from B-mode ultrasound liver images. The results obtained through these methods were compared for selecting the predictive model with the best performance metrics. We trained the two models using a dataset of 262 images of liver steatosis and 234 images of normal liver. We assessed the models using a dataset of 70 liver steatosis im-ages and 63 normal liver images. Results. The proposed model that used Inception v3 obtained a 93.23% test accuracy with a sensitivity of 89.9%% and a precision of 96.6%, and areas under each receiver operating characteristic curves (ROC AUC) of 0.93. The other proposed model that used VGG-16, obtained a 90.77% test accuracy with a sensitivity of 88.9% and a precision of 92.85%, and areas under each receiver operating characteristic curves (ROC AUC) of 0.91. Conclusion. The deep learning algorithms that we proposed to detect steatosis and classify the images in normal and fatty liver images, yields an excellent test performance of over 90%. However, future larger studies are required in order to establish how these algorithms can be implemented in a clinical setting.

scholarly journals Convolutional Neural Network Integrated With Fuzzy Rules for Decision Making in Brain Tumor Diagnosis

2021 ◽  
Vol 15 (4) ◽  
pp. 1-23
Author(s):  
Pham Van Hai ◽  
Samson Eloanyi Amaechi
Keyword(s):  
Neural Networks ◽  
Medical Imaging ◽  
Brain Tumors ◽  
Convolutional Neural Networks ◽  
Fuzzy Rules ◽  
Experimental Result ◽  
Support Vector ◽  
Proposed Model ◽  
Diagnosis And Classification

Conventional methods used in brain tumors detection, diagnosis, and classification such as magnetic resonance imaging and computed tomography scanning technologies are unbridged in their results. This paper presents a proposed model combination, convolutional neural networks with fuzzy rules in the detection and classification of medical imaging such as healthy brain cell and tumors brain cells. This model contributes fully on the automatic classification and detection medical imaging such as brain tumors, heart diseases, breast cancers, HIV and FLU. The experimental result of the proposed model shows overall accuracy of 97.6%, which indicates that the proposed method achieves improved performance than the other current methods in the literature such as [classification of tumors in human brain MRI using wavelet and support vector machine 94.7%, and deep convolutional neural networks with transfer learning for automated brain image classification 95.0%], uses in the detection, diagnosis, and classification of medical imaging decision supports.

scholarly journals Automatic Assessment of Buildings Location Fitness for Solar Panels Installation Using Drones and Neural Network

CivilEng ◽  
2021 ◽  
Vol 2 (4) ◽  
pp. 1052-1064
Author(s):  
Ammar Alzarrad ◽  
Chance Emanuels ◽  
Mohammad Imtiaz ◽  
Haseeb Akbar
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Solar Energy ◽  
Convolutional Neural Networks ◽  
Classification Accuracy ◽  
Solar Panels ◽  
Automatic Assessment ◽  
Surrounding Environment ◽  
Proposed Model ◽  
The City

Solar panel location assessment is usually a time-consuming manual process, and many criteria should be taken into consideration before deciding. One of the most significant criteria is the building location and surrounding environment. This research project aims to propose a model to automatically identify potential roof spaces for solar panels using drones and convolutional neural networks (CNN). Convolutional neural networks (CNNs) are used to identify buildings’ roofs from drone imagery. Transfer learning on the CNN is used to classify roofs of buildings into two categories of shaded and unshaded. The CNN is trained and tested on separate imagery databases to improve classification accuracy. Results of the current project demonstrate successful segmentation of buildings and identification of shaded roofs. The model presented in this paper can be used to prioritize the buildings based on the likelihood of getting benefits from switching to solar energy. To illustrate an implementation of the presented model, it has been applied to a selected neighborhood in the city of Hurricane in West Virginia. The research results show that the proposed model can assist investors in the energy and building sectors to make better and more informed decisions.

scholarly journals Incorporating Context-Relevant Knowledge into Convolutional Neural Networks for Short Text Classification

2019 ◽  
Vol 33 ◽  
pp. 10067-10068 ◽  
Author(s):  
Jingyun Xu ◽  
Yi Cai
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Convolutional Neural Network ◽  
Convolutional Neural Networks ◽  
Text Classification ◽  
Classification Methods ◽  
Short Text ◽  
Proposed Model ◽  
High Level ◽  
Context Features

Some text classification methods don’t work well on short texts due to the data sparsity. What’s more, they don’t fully exploit context-relevant knowledge. In order to tackle these problems, we propose a neural network to incorporate context-relevant knowledge into a convolutional neural network for short text classification. Our model consists of two modules. The first module utilizes two layers to extract concept and context features respectively and then employs an attention layer to extract those context-relevant concepts. The second module utilizes a convolutional neural network to extract high-level features from the word and the contextrelevant concept features. The experimental results on three datasets show that our proposed model outperforms the stateof-the-art models.

scholarly journals Apple quality identification and classification by image processing based on convolutional neural networks

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Yanfei Li ◽  
Xianying Feng ◽  
Yandong Liu ◽  
Xingchang Han
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Support Vector ◽  
Svm Classifier ◽  
Test Accuracy ◽  
Training Time ◽  
Proposed Model ◽  
Specific Complex ◽  
Occurrence Matrix ◽  
Apple Quality

AbstractThis work researched apple quality identification and classification from real images containing complicated disturbance information (background was similar to the surface of the apples). This paper proposed a novel model based on convolutional neural networks (CNN) which aimed at accurate and fast grading of apple quality. Specific, complex, and useful image characteristics for detection and classification were captured by the proposed model. Compared with existing methods, the proposed model could better learn high-order features of two adjacent layers that were not in the same channel but were very related. The proposed model was trained and validated, with best training and validation accuracy of 99% and 98.98% at 2590th and 3000th step, respectively. The overall accuracy of the proposed model tested using an independent 300 apple dataset was 95.33%. The results showed that the training accuracy, overall test accuracy and training time of the proposed model were better than Google Inception v3 model and traditional imaging process method based on histogram of oriented gradient (HOG), gray level co-occurrence matrix (GLCM) features merging and support vector machine (SVM) classifier. The proposed model has great potential in Apple’s quality detection and classification.

scholarly journals Quality Assessment of Tire Shearography Images via Ensemble Hybrid Faster Region-Based ConvNets

Electronics ◽  
2019 ◽  
Vol 9 (1) ◽  
pp. 45 ◽  
Author(s):  
Chuan-Yu Chang ◽  
Kathiravan Srinivasan ◽  
Wei-Chun Wang ◽  
Ganapathy Pattukandan Ganapathy ◽  
Durai Raj Vincent ◽  
...  
Keyword(s):  
Neural Networks ◽  
Quality Assessment ◽  
Convolutional Neural Networks ◽  
False Positive ◽  
High Performance ◽  
Learning Approaches ◽  
Positive Ratio ◽  
False Positive Ratio ◽  
Proposed Model ◽  
Human Eyes

In recent times, the application of enabling technologies such as digital shearography combined with deep learning approaches in the smart quality assessment of tires, which leads to intelligent tire manufacturing practices with automated defects detection. Digital shearography is a prominent approach that can be employed for identifying the defects in tires, usually not visible to human eyes. In this research, the bubble defects in tire shearography images are detected using a unique ensemble hybrid amalgamation of the convolutional neural networks/ConvNets with high-performance Faster Region-based convolutional neural networks. It can be noticed that the routine of region-proposal generation along with object detection is accomplished using the ConvNets. Primarily, the sliding window based ConvNets are utilized in the proposed model for dividing the input shearography images into regions, in order to identify the bubble defects. Subsequently, this is followed by implementing the Faster Region-based ConvNets for identifying the bubble defects in the tire shearography images and further, it also helps to minimize the false-positive ratio (sometimes referred to as the false alarm ratio). Moreover, it is evident from the experimental results that the proposed hybrid model offers a cent percent detection of bubble defects in the tire shearography images. Also, it can be witnessed that the false-positive ratio gets minimized to 18 percent.

Sign in / Sign up

Export Citation Format

Share Document