Hierarchical computing: A high performance computing architecture for data-processing in IoT era

2017 ◽  
Author(s):  
Zhihe Yang
Keyword(s):  
Data Processing ◽  
High Performance Computing ◽  
High Performance ◽  
Computing Architecture ◽  
Performance Computing

High-Performance Computing for Big Data Processing

2018 ◽  
Vol 88 ◽  
pp. 693-695 ◽  
Author(s):  
Yulei Wu ◽  
Yang Xiang ◽  
Jingguo Ge ◽  
Peter Muller
Keyword(s):  
Big Data ◽  
Data Processing ◽  
High Performance Computing ◽  
High Performance ◽  
Big Data Processing ◽  
Performance Computing

scholarly journals Application of parallel calculations in the calculation of quality maintenance of application queues

2011 ◽  
Vol 16 (4) ◽  
pp. 177-181
Author(s):  
M.O. Alieksieiev ◽  
L.S. Hloba ◽  
K.O. Yermakova ◽  
V.V. Kushnir
Keyword(s):  
Data Processing ◽  
High Performance Computing ◽  
High Performance ◽  
Quality Maintenance ◽  
Parallel Calculations ◽  
Computing Technologies ◽  
Qos Parameters ◽  
Performance Computing

The usage of high-performance computing technologies in the areas of scientific and engineering researches is considered. The method of the effective data processing paralleling is described. The using of high-performance computing based on the OpenMP library for solving problems in the field of Telecommunication, e.g. computation of the queues QoS parameters, is also analyzed

scholarly journals Upgrading a high performance computing environment for massive data processing

2019 ◽  
Vol 10 (1) ◽  
Author(s):  
Lucas M. Ponce ◽  
Walter dos Santos ◽  
Wagner Meira ◽  
Dorgival Guedes ◽  
Daniele Lezzi ◽  
...  
Keyword(s):  
Big Data ◽  
Data Processing ◽  
High Performance Computing ◽  
High Performance ◽  
Data Access ◽  
Massive Data ◽  
Analysis Tool ◽  
Data Framework ◽  
Performance Computing ◽  
Massive Data Processing

Abstract High-performance computing (HPC) and massive data processing (Big Data) are two trends that are beginning to converge. In that process, aspects of hardware architectures, systems support and programming paradigms are being revisited from both perspectives. This paper presents our experience on this path of convergence with the proposal of a framework that addresses some of the programming issues derived from such integration. Our contribution is the development of an integrated environment that integretes (i) COMPSs, a programming framework for the development and execution of parallel applications for distributed infrastructures; (ii) Lemonade, a data mining and analysis tool; and (iii) HDFS, the most widely used distributed file system for Big Data systems. To validate our framework, we used Lemonade to create COMPSs applications that access data through HDFS, and compared them with equivalent applications built with Spark, a popular Big Data framework. The results show that the HDFS integration benefits COMPSs by simplifying data access and by rearranging data transfer, reducing execution time. The integration with Lemonade facilitates COMPSs’s use and may help its popularization in the Data Science community, by providing efficient algorithm implementations for experts from the data domain that want to develop applications with a higher level abstraction.

scholarly journals The architectures analyzing for computational workflows designing in a distributed environment

2020 ◽  
Keyword(s):  
Cloud Computing ◽  
High Performance Computing ◽  
High Performance ◽  
Business Processes ◽  
Service Oriented Architecture ◽  
Distributed Environment ◽  
End User ◽  
Distributed Resources ◽  
Computing Architecture ◽  
Performance Computing

The paper presents a model of computational workflows based on end-user understanding and provides an overview of various computational architectures, such as computing cluster, Grid, Cloud Computing, and SOA, for building workflows in a distributed environment. A comparative analysis of the capabilities of the architectures for the implementation of computational workflows have been shown that the workflows should be implemented based on SOA, since it meets all the requirements for the basic infrastructure and provides a high degree of compute nodes distribution, as well as their migration and integration with other systems in a heterogeneous environment. The Cloud Computing architecture using may be efficient when building a basic information infrastructure for the organization of distributed high-performance computing, since it supports the general and coordinated usage of dynamically allocated distributed resources, allows in geographically dispersed data centers to create and virtualize high-performance computing systems that are able to independently support the necessary QoS level and, if necessary, to use the Software as a Service (SaaS) model for end-users. The advantages of the Cloud Computing architecture do not allow the end user to realize business processes design automatically, designing them "on the fly". At the same time, there is the obvious need to create semantically oriented computing workflows based on a service-oriented architecture using a microservices approach, ontologies and metadata structures, which will allow to create workflows “on the fly” in accordance with the current request requirements.

An Analytical Approach for Optimizing the Performance of Hadoop Map Reduce Over RoCE

2018 ◽  
Vol 10 (2) ◽  
pp. 1-14 ◽  
Author(s):  
Geetha J. ◽  
Uday Bhaskar N ◽  
Chenna Reddy P.
Keyword(s):  
Data Processing ◽  
High Performance Computing ◽  
High Performance ◽  
Direct Memory Access ◽  
Performance Measure ◽  
Data Intensive ◽  
The World ◽  
Mapreduce Paradigm ◽  
Mathematical Derivation ◽  
Performance Computing

Data intensive systems aim to efficiently process “big” data. Several data processing engines have evolved over past decade. These data processing engines are modeled around the MapReduce paradigm. This article explores Hadoop's MapReduce engine and propose techniques to obtain a higher level of optimization by borrowing concepts from the world of High Performance Computing. Consequently, power consumed and heat generated is lowered. This article designs a system with a pipelined dataflow in contrast to the existing unregulated “bursty” flow of network traffic, the ability to carry out both Map and Reduce tasks in parallel, and a system which incorporates modern high-performance computing concepts using Remote Direct Memory Access (RDMA). To establish the claim of an increased performance measure of the proposed system, the authors provide an algorithm for RoCE enabled MapReduce and a mathematical derivation contrasting the runtime of vanilla Hadoop. This article proves mathematically, that the proposed system functions 1.67 times faster than the vanilla version of Hadoop.

scholarly journals Using OpenMP for HEP framework algorithm scheduling

2020 ◽  
Vol 245 ◽  
pp. 05003
Author(s):  
Christopher Jones ◽  
Patrick Gartung
Keyword(s):  
Data Processing ◽  
High Performance Computing ◽  
High Performance ◽  
Scheduling Algorithms ◽  
Building Blocks ◽  
Software Packages ◽  
Primary Mechanism ◽  
Performance Computing ◽  
Processing Framework ◽  
Performance Comparisons

The OpenMP standard is the primary mechanism used at high performance computing facilities to allow intra-process parallelization. In contrast, many HEP specific software packages (such as CMSSW, GaudiHive, and ROOT) make use of Intel’s Threading Building Blocks (TBB) library to accomplish the same goal. In these proceedings we will discuss our work to compare TBB and OpenMP when used for scheduling algorithms to be run by a HEP style data processing framework. This includes both scheduling of different interdependent algorithms to be run concurrently as well as scheduling concurrent work within one algorithm. As part of the discussion we present an overview of the OpenMP threading model. We also explain how we used OpenMP when creating a simplified HEP-like processing framework. Using that simplified framework, and a similar one written using TBB, we will present performance comparisons between TBB and different compiler versions of OpenMP.

Sign in / Sign up

Export Citation Format

Share Document