scholarly journals On Training Deep Neural Networks Using a Streaming Approach

2020 ◽  
Vol 10 (1) ◽  
pp. 15-26 ◽  
Author(s):  
Piotr Duda ◽  
Maciej Jaworski ◽  
Andrzej Cader ◽  
Lipo Wang
Keyword(s):  
Neural Networks ◽  
Data Stream ◽  
Deep Neural Networks ◽  
High Accuracy ◽  
The Other ◽  
Data Streaming ◽  
Process Data ◽  
Training Time ◽  
Network Learning ◽  
Data Stream Analysis

AbstractIn recent years, many deep learning methods, allowed for a significant improvement of systems based on artificial intelligence methods. Their effectiveness results from an ability to analyze large labeled datasets. The price for such high accuracy is the long training time, necessary to process such large amounts of data. On the other hand, along with the increase in the number of collected data, the field of data stream analysis was developed. It enables to process data immediately, with no need to store them. In this work, we decided to take advantage of the benefits of data streaming in order to accelerate the training of deep neural networks. The work includes an analysis of two approaches to network learning, presented on the background of traditional stochastic and batch-based methods.

scholarly journals High accuracy data-driven heliostat calibration and state prediction with pretrained deep neural networks

Solar Energy ◽  
2021 ◽  
Vol 218 ◽  
pp. 48-56
Author(s):  
Max Pargmann ◽  
Daniel Maldonado Quinto ◽  
Peter Schwarzbözl ◽  
Robert Pitz-Paal
Keyword(s):  
Neural Networks ◽  
Deep Neural Networks ◽  
High Accuracy ◽  
Data Driven ◽  
State Prediction ◽  
Accuracy Data

scholarly journals Parallel Restarted SGD with Faster Convergence and Less Communication: Demystifying Why Model Averaging Works for Deep Learning

2019 ◽  
Vol 33 ◽  
pp. 5693-5700 ◽  
Author(s):  
Hao Yu ◽  
Sen Yang ◽  
Shenghuo Zhu
Keyword(s):  
Neural Networks ◽  
Deep Neural Networks ◽  
Model Averaging ◽  
Communication Overhead ◽  
Single Server ◽  
Training Time ◽  
Distributed Training ◽  
Speed Up ◽  
Experimental Works ◽  
Single Worker

In distributed training of deep neural networks, parallel minibatch SGD is widely used to speed up the training process by using multiple workers. It uses multiple workers to sample local stochastic gradients in parallel, aggregates all gradients in a single server to obtain the average, and updates each worker’s local model using a SGD update with the averaged gradient. Ideally, parallel mini-batch SGD can achieve a linear speed-up of the training time (with respect to the number of workers) compared with SGD over a single worker. However, such linear scalability in practice is significantly limited by the growing demand for gradient communication as more workers are involved. Model averaging, which periodically averages individual models trained over parallel workers, is another common practice used for distributed training of deep neural networks since (Zinkevich et al. 2010) (McDonald, Hall, and Mann 2010). Compared with parallel mini-batch SGD, the communication overhead of model averaging is significantly reduced. Impressively, tremendous experimental works have verified that model averaging can still achieve a good speed-up of the training time as long as the averaging interval is carefully controlled. However, it remains a mystery in theory why such a simple heuristic works so well. This paper provides a thorough and rigorous theoretical study on why model averaging can work as well as parallel mini-batch SGD with significantly less communication overhead.

Shallow and Deep Artificial Neural Networks for Structural Reliability Analysis

2020 ◽  
Author(s):  
Wellison J. S. Gomes
Keyword(s):  
Neural Networks ◽  
Artificial Neural Networks ◽  
Reliability Analysis ◽  
Structural Reliability ◽  
Deep Neural Networks ◽  
Surrogate Models ◽  
The Other ◽  
Computational Costs ◽  
Structural Reliability Analysis ◽  
Artificial Neural

Abstract Surrogate models are efficient tools which have been successfully applied in structural reliability analysis, as an attempt to keep the computational costs acceptable. Among the surrogate models available in the literature, Artificial Neural Networks (ANNs) have been attracting research interest for many years. However, the ANNs used in structural reliability analysis are usually the shallow ones, based on an architecture consisting of neurons organized in three layers, the so-called input, hidden and output layers. On the other hand, with the advent of deep learning, ANNs with one input, one output, and several hidden layers, known as deep neural networks, have been increasingly applied in engineering and other areas. Considering that many recent publications have shown advantages of deep over shallow ANNs, the present paper aims at comparing these types of neural networks in the context of structural reliability. By applying shallow and deep ANNs in the solution of four benchmark structural reliability problems from the literature, employing Monte Carlo simulation and adaptive experimental designs, it is shown that, although good results are obtained for both types of ANNs, deep ANNs usually outperform the shallow ones.

scholarly journals Maximizing the Prediction Accuracy in Tweet Sentiment Extraction using Tensor Flow based Deep Neural Networks

2021 ◽  
Vol 3 (2) ◽  
pp. 61-79
Author(s):  
S Thivaharan ◽  
G Srivatsun
Keyword(s):  
Neural Networks ◽  
Social Media ◽  
Prediction Accuracy ◽  
Deep Neural Networks ◽  
Modern Technology ◽  
The Other ◽  
Data Generation ◽  
Class Prediction ◽  
The Social ◽  
Communication Devices

The amount of data generated by modern communication devices is enormous, reaching petabytes. The rate of data generation is also increasing at an unprecedented rate. Though modern technology supports storage in massive amounts, the industry is reluctant in retaining the data, which includes the following characteristics: redundancy in data, unformatted records with outdated information, data that misleads the prediction and data with no impact on the class prediction. Out of all of this data, social media plays a significant role in data generation. As compared to other data generators, the ratio at which the social media generates the data is comparatively higher. Industry and governments are both worried about the circulation of mischievous or malcontents, as they are extremely susceptible and are used by criminals. So it is high time to develop a model to classify the social media contents as fair and unfair. The developed model should have higher accuracy in predicting the class of contents. In this article, tensor flow based deep neural networks are deployed with a fixed Epoch count of 15, in order to attain 25% more accuracy over the other existing models. Activation methods like “Relu” and “Sigmoid”, which are specific for Tensor flow platforms support to attain the improved prediction accuracy.

scholarly journals Image Compression Based on Deep Learning: A Review

2021 ◽  
pp. 62-76
Author(s):  
Hajar Maseeh Yasin ◽  
Adnan Mohsin Abdulazeez
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Image Compression ◽  
Recurrent Neural Networks ◽  
Deep Neural Networks ◽  
Review Paper ◽  
High Accuracy ◽  
Machine Learning Methods ◽  
Digital Era ◽  
Different Types

Image compression is an essential technology for encoding and improving various forms of images in the digital era. The inventors have extended the principle of deep learning to the different states of neural networks as one of the most exciting machine learning methods to show that it is the most versatile way to analyze, classify, and compress images. Many neural networks are required for image compressions, such as deep neural networks, artificial neural networks, recurrent neural networks, and convolution neural networks. Therefore, this review paper discussed how to apply the rule of deep learning to various neural networks to obtain better compression in the image with high accuracy and minimize loss and superior visibility of the image. Therefore, deep learning and its application to different types of images in a justified manner with distinct analysis to obtain these things need deep learning.

scholarly journals EDHA: Event-Driven High Accurate Simulator for Spike Neural Networks

Electronics ◽  
2021 ◽  
Vol 10 (18) ◽  
pp. 2281
Author(s):  
Lingfei Mo ◽  
Xinao Chen ◽  
Gang Wang
Keyword(s):  
Neural Networks ◽  
High Accuracy ◽  
The Other ◽  
Spiking Neural Networks ◽  
Time Slice ◽  
Computational Accuracy ◽  
Low Power Computing ◽  
Event Driven ◽  
Cross Platform ◽  
Encoding Method

In recent years, spiking neural networks (SNNs) have attracted increasingly more researchers to study by virtue of its bio-interpretability and low-power computing. The SNN simulator is an essential tool to accomplish image classification, recognition, speech recognition, and other tasks using SNN. However, most of the existing simulators for spike neural networks are clock-driven, which has two main problems. First, the calculation result is affected by time slice, which obviously shows that when the calculation accuracy is low, the calculation speed is fast, but when the calculation accuracy is high, the calculation speed is unacceptable. The other is the failure of lateral inhibition, which severely affects SNN learning. In order to solve these problems, an event-driven high accurate simulator named EDHA (Event-Driven High Accuracy) for spike neural networks is proposed in this paper. EDHA takes full advantage of the event-driven characteristics of SNN and only calculates when a spike is generated, which is independent of the time slice. Compared with previous SNN simulators, EDHA is completely event-driven, which reduces a large amount of calculations and achieves higher computational accuracy. The calculation speed of EDHA in the MNIST classification task is more than 10 times faster than that of mainstream clock-driven simulators. By optimizing the spike encoding method, the former can even achieve more than 100 times faster than the latter. Due to the cross-platform characteristics of Java, EDHA can run on x86, amd64, ARM, and other platforms that support Java.

High Accuracy and Effectiveness with Deep Neural Networks in Pathological Diagnosis of Prostate Cancer

2019 ◽  
Author(s):  
Qing Zhang ◽  
Chengwei Zhang ◽  
Xiubin Gao ◽  
Suhan Guo ◽  
Mengxia Chen ◽  
...  
Keyword(s):  
Prostate Cancer ◽  
Neural Networks ◽  
Deep Neural Networks ◽  
High Accuracy ◽  
Pathological Diagnosis

scholarly journals Quality Control of PET Bottles Caps with Dedicated Image Calibration and Deep Neural Networks

Sensors ◽  
2021 ◽  
Vol 21 (2) ◽  
pp. 501
Author(s):  
Marcin Malesa ◽  
Piotr Rajkiewicz
Keyword(s):  
Neural Networks ◽  
Quality Control ◽  
Product Quality ◽  
Deep Neural Networks ◽  
Production Control ◽  
Production Lines ◽  
Product Quality Control ◽  
Training Time ◽  
Pet Bottles ◽  
Pharmaceutical Industries

Product quality control is currently the leading trend in industrial production. It is heading towards the exact analysis of each product before reaching the end customer. Every stage of production control is of particular importance in the food and pharmaceutical industries, where, apart from visual issues, additional safety regulations are demanded. Many production processes can be controlled completely contactless through the use of machine vision cameras and advanced image processing techniques. The most dynamically growing sector of image analysis methods are solutions based on deep neural networks. Their major advantages are fast performance, robustness, and the fact that they can be exploited even in complicated classification problems. However, the use of machine learning methods on high-performance production lines may be limited by inference time or, in the case of multiformated production lines, training time. The article presents a novel data preprocessing (or calibration) method. It uses prior knowledge about the optical system, which enables the use of the lightweight Convolutional Neural Network (CNN) model for product quality control of polyethylene terephthalate (PET) bottle caps. The combination of preprocessing with the lightweight CNN model resulted in at least a five-fold reduction in prediction and training time compared to the lighter standard models tested on ImageNet, without loss of accuracy.

scholarly journals STUDY ON BRAIN TUMOR CLASSIFICATION THROUGH MRI IMAGES USING A DEEP CONVOLUTIONAL NEURAL NETWORK

2022 ◽  
Vol 12 (1) ◽  
pp. 0-0
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Brain Tumor ◽  
Deep Neural Networks ◽  
High Accuracy ◽  
Normal Organ ◽  
Image Processing Techniques ◽  
Learning Techniques ◽  
Processing Techniques ◽  
Mri Image

Brain tumor (Glioma) is one of the deadliest diseases that attack humans, now even men or women aged 20-30 are suffering from this disease. To cure tumor in a person, doctors use MRI machine, because the results of MRI images are proven to provide better image results than CT-Scan images, but sometimes it is difficult to distinguish between the MRI images having tumors with that images not having tumor from MRI image results. It is because of resulting contrast is like any other normal organ. However, using features of image processing techniques like scaling, contrast enhancement and thresh-holding based in Deep Neural Networks the scheme can classify the results more appropriately and with high accuracy. In this paper, this study reveals the nitty-gritty of Brain tumor (Gliomas) and Deep Learning techniques for better inception in the field of computer-vision.

Sign in / Sign up

Export Citation Format

Share Document