scholarly journals Decentralized Distributed Deep Learning with Low-Bandwidth Consumption for Smart Constellations

2021 ◽  
Vol 2021 ◽  
pp. 1-10
Author(s):  
Qingliang Meng ◽  
Meiyu Huang ◽  
Yao Xu ◽  
Naijin Liu ◽  
Xueshuang Xiang
Keyword(s):  
Deep Learning ◽  
Parameter Extraction ◽  
Stochastic Gradient Descent ◽  
Network Bandwidth ◽  
Distributed Training ◽  
Intelligent Processing ◽  
Benchmark Datasets ◽  
Speed Up ◽  
New Perspective ◽  
Deep Learning Network

For the space-based remote sensing system, onboard intelligent processing based on deep learning has become an inevitable trend. To adapt to the dynamic changes of the observation scenes, there is an urgent need to perform distributed deep learning onboard to fully utilize the plentiful real-time sensing data of multiple satellites from a smart constellation. However, the network bandwidth of the smart constellation is very limited. Therefore, it is of great significance to carry out distributed training research in a low-bandwidth environment. This paper proposes a Randomized Decentralized Parallel Stochastic Gradient Descent (RD-PSGD) method for distributed training in a low-bandwidth network. To reduce the communication cost, each node in RD-PSGD just randomly transfers part of the information of the local intelligent model to its neighborhood. We further speed up the algorithm by optimizing the programming of random index generation and parameter extraction. For the first time, we theoretically analyze the convergence property of the proposed RD-PSGD and validate the advantage of this method by simulation experiments on various distributed training tasks for image classification on different benchmark datasets and deep learning network architectures. The results show that RD-PSGD can effectively save the time and bandwidth cost of distributed training and reduce the complexity of parameter selection compared with the TopK-based method. The method proposed in this paper provides a new perspective for the study of onboard intelligent processing, especially for online learning on a smart satellite constellation.

scholarly journals FLEX: Faithful Linguistic Explanations for Neural Net Based Model Decisions

2019 ◽  
Vol 33 ◽  
pp. 2539-2546
Author(s):  
Sandareka Wickramanayake ◽  
Wynne Hsu ◽  
Mong Li Lee
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
State Of The Art ◽  
Neural Net ◽  
End User ◽  
Learning Network ◽  
Benchmark Datasets ◽  
Deep Learning Network ◽  
Post Hoc ◽  
User Trust

Explaining the decisions of a Deep Learning Network is imperative to safeguard end-user trust. Such explanations must be intuitive, descriptive, and faithfully explain why a model makes its decisions. In this work, we propose a framework called FLEX (Faithful Linguistic EXplanations) that generates post-hoc linguistic justifications to rationalize the decision of a Convolutional Neural Network. FLEX explains a model’s decision in terms of features that are responsible for the decision. We derive a novel way to associate such features to words, and introduce a new decision-relevance metric that measures the faithfulness of an explanation to a model’s reasoning. Experiment results on two benchmark datasets demonstrate that the proposed framework can generate discriminative and faithful explanations compared to state-of-the-art explanation generators. We also show how FLEX can generate explanations for images of unseen classes as well as automatically annotate objects in images.

Comparing optimization methods for deep learning in image processing applications

2021 ◽  
Vol 0 (0) ◽  
Author(s):  
Alexander Geng ◽  
Ali Moghiseh ◽  
Claudia Redenbach ◽  
Katja Schladitz
Keyword(s):  
Image Processing ◽  
Deep Learning ◽  
Optimization Methods ◽  
Optimization Method ◽  
Stochastic Gradient Descent ◽  
Style Transfer ◽  
Artistic Style ◽  
Deep Learning Network ◽  
The Individual ◽  
The Impact

Abstract Training a deep learning network requires choosing its weights such that the output minimizes a given loss function. In practice, stochastic gradient descent is frequently used for solving the optimization problem. Several variants of this approach have been suggested in the literature. We study the impact of the choice of the optimization method on the outcome of the learning process at the example of two image processing applications from quite different fields. The first one is artistic style transfer, where the content of one image is combined with the style of another one. The second application is a real world classification task from industry, namely detecting defects in images of air filters. In both cases, clear differences between the results of the individual optimization methods are observed.

scholarly journals Investigations of Object Detection in Images/Videos Using Various Deep Learning Techniques and Embedded Platforms—A Comprehensive Review

2020 ◽  
Vol 10 (9) ◽  
pp. 3280 ◽  
Author(s):  
Chinthakindi Balaram Murthy ◽  
Mohammad Farukh Hashmi ◽  
Neeraj Dhanraj Bokde ◽  
Zong Woo Geem
Keyword(s):  
Deep Learning ◽  
Object Detection ◽  
Pedestrian Detection ◽  
Detection Methods ◽  
Art Object ◽  
Current State ◽  
Learning Techniques ◽  
Specific Object ◽  
Benchmark Datasets ◽  
Speed Up

In recent years there has been remarkable progress in one computer vision application area: object detection. One of the most challenging and fundamental problems in object detection is locating a specific object from the multiple objects present in a scene. Earlier traditional detection methods were used for detecting the objects with the introduction of convolutional neural networks. From 2012 onward, deep learning-based techniques were used for feature extraction, and that led to remarkable breakthroughs in this area. This paper shows a detailed survey on recent advancements and achievements in object detection using various deep learning techniques. Several topics have been included, such as Viola–Jones (VJ), histogram of oriented gradient (HOG), one-shot and two-shot detectors, benchmark datasets, evaluation metrics, speed-up techniques, and current state-of-art object detectors. Detailed discussions on some important applications in object detection areas, including pedestrian detection, crowd detection, and real-time object detection on Gpu-based embedded systems have been presented. At last, we conclude by identifying promising future directions.

scholarly journals Model Accuracy and Runtime Tradeoff in Distributed Deep Learning: A Systematic Study

2017 ◽  
Author(s):  
Suyog Gupta ◽  
Wei Zhang ◽  
Fei Wang
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Systematic Study ◽  
Stochastic Gradient Descent ◽  
Model Accuracy ◽  
Distributed Training ◽  
Gradient Descent Algorithm ◽  
Runtime Performance ◽  
The Impact ◽  
Synchronization Protocol

Deep learning with a large number of parame-ters requires distributed training, where model accuracy and runtime are two important factors to be considered. However, there has been no systematic study of the tradeoff between these two factors during the model training process. This paper presents Rudra, a parameter server based distributed computing framework tuned for training large-scale deep neural networks. Using variants of the asynchronous stochastic gradient descent algorithm we study the impact of synchronization protocol, stale gradient updates, minibatch size, learning rates, and number of learners on runtime performance and model accuracy. We introduce a new learningrate modulation strategy to counter the effect of stale gradients and propose a new synchronization protocol that can effectively bound the staleness in gradients, improve runtime performance and achieve good model accuracy. Our empirical investigation reveals a principled approach for distributed training of neural networks: the mini-batch size per learner should be reduced as more learners are added to the system to preserve the model accuracy. We validate this approach using commonly-used image classification benchmarks: CIFAR10 and ImageNet.

Share Market Data Prediction Strategies using Deep Learning Algorithm

2019 ◽  
Vol 13 ◽  
Author(s):  
A John. ◽  
D. Praveen Dominic ◽  
M. Adimoolam ◽  
N. M. Balamurugan
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Stock Market ◽  
Predictive Analytics ◽  
Learning Algorithm ◽  
Market Price ◽  
Stochastic Gradient ◽  
Stochastic Gradient Descent ◽  
Mining Machine ◽  
Gradient Descent Algorithm

Background:: Predictive analytics has a multiplicity of statistical schemes from predictive modelling, data mining, machine learning. It scrutinizes present and chronological data to make predictions about expectations or if not unexplained measures. Most predictive models are used for business analytics to overcome loses and profit gaining. Predictive analytics is used to exploit the pattern in old and historical data. Objective: People used to follow some strategies for predicting stock value to invest in the more profit-gaining stocks and those strategies to search the stock market prices which are incorporated in some intelligent methods and tools. Such strategies will increase the investor’s profits and also minimize their risks. So prediction plays a vital role in stock market gaining and is also a very intricate and challenging process. Method: The proposed optimized strategies are the Deep Neural Network with Stochastic Gradient for stock prediction. The Neural Network is trained using Back-propagation neural networks algorithm and stochastic gradient descent algorithm as optimal strategies. Results: The experiment is conducted for stock market price prediction using python language with the visual package. In this experiment RELIANCE.NS, TATAMOTORS.NS, and TATAGLOBAL.NS dataset are taken as input dataset and it is downloaded from National Stock Exchange site. The artificial neural network component including Deep Learning model is most effective for more than 100,000 data points to train this model. This proposed model is developed on daily prices of stock market price to understand how to build model with better performance than existing national exchange method.

An Anatomy of a Hybrid Color Descriptor with a Neural Network Model to Enhance the Retrieval Accuracy of an Image Retrieval System

2019 ◽  
Vol 13 ◽  
Author(s):  
Shikha Bhardwaj ◽  
Gitanjali Pandove ◽  
Pawan Kumar Dahiya
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Image Retrieval ◽  
Hybrid System ◽  
Back Propagation ◽  
Back Propagation Neural Network ◽  
Retrieval Accuracy ◽  
Color Descriptor ◽  
Benchmark Datasets ◽  
Color Moment

Background: In order to retrieve a particular image from vast repository of images, an efficient system is required and such an eminent system is well-known by the name Content-based image retrieval (CBIR) system. Color is indeed an important attribute of an image and the proposed system consist of a hybrid color descriptor which is used for color feature extraction. Deep learning, has gained a prominent importance in the current era. So, the performance of this fusion based color descriptor is also analyzed in the presence of Deep learning classifiers. Method: This paper describes a comparative experimental analysis on various color descriptors and the best two are chosen to form an efficient color based hybrid system denoted as combined color moment-color autocorrelogram (Co-CMCAC). Then, to increase the retrieval accuracy of the hybrid system, a Cascade forward back propagation neural network (CFBPNN) is used. The classification accuracy obtained by using CFBPNN is also compared to Patternnet neural network. Results: The results of the hybrid color descriptor depict that the proposed system has superior results of the order of 95.4%, 88.2%, 84.4% and 96.05% on Corel-1K, Corel-5K, Corel-10K and Oxford flower benchmark datasets respectively as compared to many state-of-the-art related techniques. Conclusion: This paper depict an experimental and analytical analysis on different color feature descriptors namely, Color moment (CM), Color auto-correlogram (CAC), Color histogram (CH), Color coherence vector (CCV) and Dominant color descriptor (DCD). The proposed hybrid color descriptor (Co-CMCAC) is utilized for the withdrawal of color features with Cascade forward back propagation neural network (CFBPNN) is used as a classifier on four benchmark datasets namely Corel-1K, Corel-5K and Corel-10K and Oxford flower.

scholarly journals UAVs in rail damage image diagnostics supported by deep-learning networks

2021 ◽  
Vol 11 (1) ◽  
pp. 339-348
Author(s):  
Piotr Bojarczak ◽  
Piotr Lesiak
Keyword(s):  
Deep Learning ◽  
Head Width ◽  
Learning Networks ◽  
Image Brightness ◽  
Learning Network ◽  
Rail Head ◽  
Image Recording ◽  
Python Language ◽  
Deep Learning Network ◽  
Efficiency Rate

Abstract The article uses images from Unmanned Aerial Vehicles (UAVs) for rail diagnostics. The main advantage of such a solution compared to traditional surveys performed with measuring vehicles is the elimination of decreased train traffic. The authors, in the study, limited themselves to the diagnosis of hazardous split defects in rails. An algorithm has been proposed to detect them with an efficiency rate of about 81% for defects not less than 6.9% of the rail head width. It uses the FCN-8 deep-learning network, implemented in the Tensorflow environment, to extract the rail head by image segmentation. Using this type of network for segmentation increases the resistance of the algorithm to changes in the recorded rail image brightness. This is of fundamental importance in the case of variable conditions for image recording by UAVs. The detection of these defects in the rail head is performed using an algorithm in the Python language and the OpenCV library. To locate the defect, it uses the contour of a separate rail head together with a rectangle circumscribed around it. The use of UAVs together with artificial intelligence to detect split defects is an important element of novelty presented in this work.

Sign in / Sign up

Export Citation Format

Share Document