Distributed computing in a heterogeneous computing environment

1998 ◽  
pp. 180-187 ◽  
Author(s):  
Edgar Gabriel ◽  
Michael Resch ◽  
Thomas Beisel ◽  
Rainer Keller
Keyword(s):  
Distributed Computing ◽  
Heterogeneous Computing ◽  
Computing Environment

scholarly journals The JINR distributed computing environment

2019 ◽  
Vol 214 ◽  
pp. 03009
Author(s):  
Vladimir Korenkov ◽  
Andrei Dolbilov ◽  
Valeri Mitsyn ◽  
Ivan Kashunin ◽  
Nikolay Kutovskiy ◽  
...  
Keyword(s):  
Distributed Computing ◽  
High Performance ◽  
Heterogeneous Computing ◽  
High Energy Physics ◽  
Nuclear Research ◽  
High Energy ◽  
Cloud Infrastructure ◽  
Computing Environment ◽  
Member States ◽  
Cloud Resources

Computing in the field of high energy physics requires usage of heterogeneous computing resources and IT, such as grid, high performance computing, cloud computing and big data analytics for data processing and analysis. The core of the distributed computing environment at the Joint Institute for Nuclear Research is the Multifunctional Information and Computing Complex. It includes Tier1 for CMS experiment, Tier2 site for all LHC experiments and other grid non-LHC VOs, such as BIOMED, COMPASS, NICA/MPD, NOvA, STAR and BESIII, as well as cloud and HPC infrastructures. A brief status overview of each component is presented. Particular attention is given to the development of distributed computations performed in collaboration with CERN, BNL, FNAL, FAIR, China, and JINR Member States. One of the directions for the cloud infrastructure is the development of integration methods of various cloud resources of the JINR Member State organizations in order to perform common tasks, and also to distribute a load across integrated resources. We performed cloud resources integration of scientific centers in Armenia, Azerbaijan, Belarus, Kazakhstan and Russia. Extension of the HPC component will be carried through a specialized infrastructure for HPC engineering that is being created at MICC, which makes use of the contact liquid cooling technology implemented by the Russian company JSC "RSC Technologies". Current plans are to further develop MICC as a center for scientific computing within the multidisciplinary research environment of JINR and JINR Member States, and mainly for the NICA mega-science project.

scholarly journals IoT-enabled directed acyclic graph in spark cluster

2020 ◽  
Vol 9 (1) ◽  
Author(s):  
Jahwan Koo ◽  
Nawab Muhammad Faseeh Qureshi ◽  
Isma Farah Siddiqui ◽  
Asad Abbas ◽  
Ali Kashif Bashir
Keyword(s):  
Distributed Computing ◽  
Real Time ◽  
Directed Acyclic Graph ◽  
Random Access ◽  
Heterogeneous Data ◽  
The Body ◽  
Computing Environment ◽  
Time Data ◽  
Acyclic Graph ◽  
Sensory Data

Abstract Real-time data streaming fetches live sensory segments of the dataset in the heterogeneous distributed computing environment. This process assembles data chunks at a rapid encapsulation rate through a streaming technique that bundles sensor segments into multiple micro-batches and extracts into a repository, respectively. Recently, the acquisition process is enhanced with an additional feature of exchanging IoT devices’ dataset comprised of two components: (i) sensory data and (ii) metadata. The body of sensory data includes record information, and the metadata part consists of logs, heterogeneous events, and routing path tables to transmit micro-batch streams into the repository. Real-time acquisition procedure uses the Directed Acyclic Graph (DAG) to extract live query outcomes from in-place micro-batches through MapReduce stages and returns a result set. However, few bottlenecks affect the performance during the execution process, such as (i) homogeneous micro-batches formation only, (ii) complexity of dataset diversification, (iii) heterogeneous data tuples processing, and (iv) linear DAG workflow only. As a result, it produces huge processing latency and the additional cost of extracting event-enabled IoT datasets. Thus, the Spark cluster that processes Resilient Distributed Dataset (RDD) in a fast-pace using Random access memory (RAM) defies expected robustness in processing IoT streams in the distributed computing environment. This paper presents an IoT-enabled Directed Acyclic Graph (I-DAG) technique that labels micro-batches at the stage of building a stream event and arranges stream elements with event labels. In the next step, heterogeneous stream events are processed through the I-DAG workflow, which has non-linear DAG operation for extracting queries’ results in a Spark cluster. The performance evaluation shows that I-DAG resolves homogeneous IoT-enabled stream event issues and provides an effective stream event heterogeneous solution for IoT-enabled datasets in spark clusters.

Designing a distributed computing environment for global-scale systems

1999 ◽  
Vol 7 (1) ◽  
pp. 25-30 ◽  
Author(s):  
Rajeev R. Raje ◽  
Sivakumar Chinnasamy
Keyword(s):  
Distributed Computing ◽  
Global Scale ◽  
Computing Environment
Sign in / Sign up

Export Citation Format

Share Document