Parallel implementation of multilayered neural networks based on Map-Reduce on cloud computing clusters

Abstract With the rapid development of electronic technology, network technology and cloud computing technology, the current data is increasing in the way of mass, has entered the era of big data. Based on cloud computing clusters, this paper proposes a novel method of parallel implementation of multilayered neural networks based on Map-Reduce. Namely in order to meet the requirements of big data processing, this paper presents an efficient mapping scheme for a fully connected multi-layered neural network, which is trained by using error back propagation (BP) algorithm based on Map-Reduce on cloud computing clusters (MRBP). The batch-training (or epoch-training) regimes are used by effective segmentation of samples on the clusters, and are adopted in the separated training method, weight summary to achieve convergence by iterating. For a parallel BP algorithm on the clusters and a serial BP algorithm on uniprocessor, the required time for implementing the algorithms is derived. The performance parameters, such as speed-up, optimal number and minimum of data nodes are evaluated for the parallel BP algorithm on the clusters. Experiment results demonstrate that the proposed parallel BP algorithm in this paper has better speed-up, faster convergence rate, less iterations than that of the existed algorithms.

Download Full-text

Urban Railway Bridge Fast Evaluation Based on Cloud Computing

Applied Mechanics and Materials ◽

10.4028/www.scientific.net/amm.71-78.4501 ◽

2011 ◽

Vol 71-78 ◽

pp. 4501-4505

Author(s):

Ming Chen ◽

Wan Zhou

Keyword(s):

Neural Networks ◽

Cloud Computing ◽

Model Updating ◽

Condition Assessment ◽

Fe Model ◽

Railway Bridge ◽

Fast Evaluation ◽

Railway Bridges ◽

Computing Model ◽

Bridge Condition Assessment

Although modern bridge are carefully designed and well constructed, damage may occur in them due to unexpected causes. Currently, many different techniques have been proposed and investigated in bridge condition assessment. However, evaluation efficiency of condition assessment has not been paid much attention by the researchers. A fast evaluation of the urban railway bridge condition based on the cloud computing is presented. In this paper dynamic FE model and Artificial neural networks technique is applied to model updating. The cloud computing model provides the basis for fast analyses. It was found that when applied to the actually railway bridges, the proposed method provided results similar to those obtained by experts, but can improve efficiency of bridge

Download Full-text

A scalable framework for smart COVID surveillance in the workplace using Deep Neural Networks and cloud computing

Expert Systems ◽

10.1111/exsy.12704 ◽

2021 ◽

Author(s):

Ajay Singh ◽

Vaibhav Jindal ◽

Rajinder Sandhu ◽

Victor Chang

Keyword(s):

Neural Networks ◽

Cloud Computing ◽

Deep Neural Networks

Download Full-text

Object Detection, Distributed Cloud Computing and Parallelization Techniques for Autonomous Driving Systems

Applied Sciences ◽

10.3390/app11072925 ◽

2021 ◽

Vol 11 (7) ◽

pp. 2925

Author(s):

Edgar Cortés Gallardo Medina ◽

Victor Miguel Velazquez Espitia ◽

Daniela Chípuli Silva ◽

Sebastián Fernández Ruiz de las Cuevas ◽

Marco Palacios Hirata ◽

...

Keyword(s):

Neural Networks ◽

Cloud Computing ◽

Path Planning ◽

Object Detection ◽

Short Term Memory ◽

Smart Cities ◽

Autonomous Vehicle ◽

Computational Time ◽

End To End ◽

On The Road

Autonomous vehicles are increasingly becoming a necessary trend towards building the smart cities of the future. Numerous proposals have been presented in recent years to tackle particular aspects of the working pipeline towards creating a functional end-to-end system, such as object detection, tracking, path planning, sentiment or intent detection, amongst others. Nevertheless, few efforts have been made to systematically compile all of these systems into a single proposal that also considers the real challenges these systems will have on the road, such as real-time computation, hardware capabilities, etc. This paper reviews the latest techniques towards creating our own end-to-end autonomous vehicle system, considering the state-of-the-art methods on object detection, and the possible incorporation of distributed systems and parallelization to deploy these methods. Our findings show that while techniques such as convolutional neural networks, recurrent neural networks, and long short-term memory can effectively handle the initial detection and path planning tasks, more efforts are required to implement cloud computing to reduce the computational time that these methods demand. Additionally, we have mapped different strategies to handle the parallelization task, both within and between the networks.

Download Full-text

A Parallel Unmixing-Based Content Retrieval System for Distributed Hyperspectral Imagery Repository on Cloud Computing Platforms

Remote Sensing ◽

10.3390/rs13020176 ◽

2021 ◽

Vol 13 (2) ◽

pp. 176

Author(s):

Peng Zheng ◽

Zebin Wu ◽

Jin Sun ◽

Yi Zhang ◽

Yaoqin Zhu ◽

...

Keyword(s):

Cloud Computing ◽

Large Scale ◽

Retrieval System ◽

Hyperspectral Image ◽

Parallel Implementation ◽

Remotely Sensed Data ◽

Web Interfaces ◽

Content Retrieval ◽

Service Mode ◽

Computing Platforms

As the volume of remotely sensed data grows significantly, content-based image retrieval (CBIR) becomes increasingly important, especially for cloud computing platforms that facilitate processing and storing big data in a parallel and distributed way. This paper proposes a novel parallel CBIR system for hyperspectral image (HSI) repository on cloud computing platforms under the guide of unmixed spectral information, i.e., endmembers and their associated fractional abundances, to retrieve hyperspectral scenes. However, existing unmixing methods would suffer extremely high computational burden when extracting meta-data from large-scale HSI data. To address this limitation, we implement a distributed and parallel unmixing method that operates on cloud computing platforms in parallel for accelerating the unmixing processing flow. In addition, we implement a global standard distributed HSI repository equipped with a large spectral library in a software-as-a-service mode, providing users with HSI storage, management, and retrieval services through web interfaces. Furthermore, the parallel implementation of unmixing processing is incorporated into the CBIR system to establish the parallel unmixing-based content retrieval system. The performance of our proposed parallel CBIR system was verified in terms of both unmixing efficiency and accuracy.

Download Full-text