Scaling GIS analysis tasks from the desktop to the cloud utilizing contemporary distributed computing and data management approaches

With the approach of cloud computing, information proprietors are persuaded to outsource their unpredictable data management frameworks from nearby places to the business open cloud for efficient adaptability. Accordingly, empowering a scrambled cloud information look administration is of vital significance. Considering the huge number of information clients and archives in the cloud, it is important to permit different watchwords in the hunt demand and return reports in the request of their pertinence to these catchphrases. Related chips away at searchable encryption concentrate on single watchword hunt or Boolean catchphrase seek, and once in a while sort the list items. In this paper, surprisingly, we characterize and take care of the testing issue of protection safeguarding multi-catchphrase positioned seek over encoded information in distributed computing (MRSE).

Download Full-text

Overview of the ATLAS distributed computing system

EPJ Web of Conferences ◽

10.1051/epjconf/201921403010 ◽

2019 ◽

Vol 214 ◽

pp. 03010 ◽

Cited By ~ 1

Author(s):

Johannes Elmsheuser ◽

Alessandro Di Girolamo

Keyword(s):

Distributed Computing ◽

Data Management ◽

New Technologies ◽

Computing System ◽

Data Management System ◽

Management Systems ◽

New Developments ◽

Atlas Experiment ◽

Physics Program ◽

Workflow System

The CERN ATLAS experiment successfully uses a worldwide computing infrastructure to support the physics program during LHC Run 2. The Grid workflow system PanDA routinely manages 250 to 500 thousand concurrently running production and analysis jobs to process simulation and detector data. In total more than 370 PB of data is distributed over more than 150 sites in the WLCG and handled by the ATLAS data management system Rucio. To prepare for the ever growing LHC luminosity in future runs new developments are underway to even more efficiently use opportunistic resources such as HPCs and utilize new technologies. This paper will review and explain the outline and the performance of the ATLAS distributed computing system and give an outlook to new workflow and data management ideas for the beginning of the LHC Run 3. It will be discussed that the ATLAS workflow and data management systems are robust, performant and can easily cope with the higher Run 2 LHC performance. There are presently no scaling issues and each subsystem is able to sustain the large loads.

Download Full-text

Adaptive and Mobility-predictive Quantization-based Communication Data Management for High Performance Distributed Computing

SIMULATION ◽

10.1177/0037549707085438 ◽

2007 ◽

Vol 83 (7) ◽

pp. 529-547 ◽

Cited By ~ 1

Author(s):

In Kee Kim ◽

Sung Ho Jang ◽

Jong Sik Lee

Keyword(s):

Distributed Computing ◽

Data Management ◽

High Performance ◽

High Performance Distributed Computing ◽

Predictive Quantization

Download Full-text

The DIRAC interware: current, upcoming and planned capabilities and technologies

EPJ Web of Conferences ◽

10.1051/epjconf/202024503035 ◽

2020 ◽

Vol 245 ◽

pp. 03035

Author(s):

Federico Stagni ◽

Andrei Tsaregorodtsev ◽

André Sailer ◽

Christophe Haen

Keyword(s):

Distributed Computing ◽

Data Management ◽

Management System ◽

Production Management ◽

Complete Solution ◽

High Energy ◽

Data Management System ◽

Distributed Computing Systems ◽

Workload Management ◽

Development Framework

Efficient access to distributed computing and storage resources is mandatory for the success of current and future High Energy and Nuclear Physics Experiments. DIRAC is an interware to build and operate distributed computing systems. It provides a development framework and a rich set of services for the Workload, Data and Production Management tasks of large scientific communities. A single DIRAC installation provides a complete solution for the distributed computing of one, or more than one collaboration. The DIRAC Workload Management System (WMS) provides a transparent, uniform interface for managing computing resources. The DIRAC Data Management System (DMS) offers all the necessary tools to ensure data handling operations: it supports transparent access to storage resources based on multiple technologies, and is easily expandable. Distributed Data management can be performed, also using third party services, and operations are resilient with respect to failures. DIRAC is highly customizable and can be easily extended. For these reasons, a vast and heterogeneous set of scientific collaborations have adopted DIRAC as the base for their computing models. Users from different experiments can interact with the system in different ways, depending on their specific tasks, expertise level and previous experience using command line tools, python APIs or Web Portals. The requirements of the diverse DIRAC user communities and hosting infrastructures triggered multiple developments to improve the system usability: examples include the adoption of industry standard authorization and authentication infrastructure solutions, the management of diverse computing resources (cloud, HPC, GPGPU, etc.), the handling of high-intensity work and data flows, but also advanced monitoring and accounting using no-SQL based solutions and message queues. This contribution will highlight DIRAC’s current, upcoming and planned capabilities and technologies.

Download Full-text

A Global Data Management Strategy for Distributed Computing Systems

Proceeding of 13th IEEE Annual International Phoenix Conference on Computers and Communications ◽

10.1109/pccc.1994.504090 ◽

2005 ◽

Author(s):

K.C. Nwosu ◽

C.Y.R. Coen ◽

P.B. Berra

Keyword(s):

Distributed Computing ◽

Data Management ◽

Management Strategy ◽

Distributed Computing Systems ◽

Computing Systems ◽

Global Data

Download Full-text

Are They Ready for the Big Thing?

Advances in Library and Information Science - Handbook of Research on Information and Records Management in the Fourth Industrial Revolution ◽

10.4018/978-1-7998-7740-0.ch013 ◽

2021 ◽

pp. 200-210

Author(s):

Matthias Lederer ◽

Juluis Lederer

Keyword(s):

Big Data ◽

Data Management ◽

Process Management ◽

Business Processes ◽

Future Trend ◽

Data Driven ◽

Big Data Applications ◽

All Solutions ◽

Management Approaches ◽

And Control

Data-driven business processes management (BPM) is regarded as a central future trend because automation often makes huge amounts of data (big data) available for the optimisation and control of workflows. Software manufacturers also use this trend and call their solutions big data applications, even if some features are reminiscent of traditional data management approaches. This chapter derives from the basic definitions of big data including 13 central requirements that a big data BPM solution must meet in order to be described as such. One hundred twenty-one process management solutions are evaluated on the basis of these to determine whether they are real big data applications. As a result, less than 5% of all solutions analysed meet all requirements.

Download Full-text