A specification language and service-oriented architecture to support distributed data management

2004 ◽  
Vol 34 (11) ◽  
pp. 1091-1117
Author(s):  
M. Brian Blake
Keyword(s):  
Data Management ◽  
Service Oriented Architecture ◽  
Specification Language ◽  
Distributed Data ◽  
Distributed Data Management ◽  
Service Oriented

Micro-Services

2012 ◽  
pp. 74-93 ◽  
Author(s):  
Arcot Rajasekar ◽  
Mike Wan ◽  
Reagan Moore ◽  
Wayne Schroeder
Keyword(s):  
Data Management ◽  
Large Scale ◽  
Service Oriented Architecture ◽  
Distributed Data ◽  
Service Oriented Architectures ◽  
Loosely Coupled ◽  
Server Side ◽  
Data Object ◽  
Service Oriented ◽  
Functional Software

Service-oriented architectures (SOA) enable orchestration of loosely-coupled and interoperable functional software units to develop and execute complex but agile applications. Data management on a distributed data grid can be viewed as a set of operations that are performed across all stages in the life-cycle of a data object. The set of such operations depends on the type of objects, based on their physical and discipline-centric characteristics. In this chapter, the authors define server-side functions, called micro-services, which are orchestrated into conditional workflows for achieving large-scale data management specific to collections of data. Micro-services communicate with each other using parameter exchange, in memory data structures, a database-based persistent information store, and a network messaging system that uses a serialization protocol for communicating with remote micro-services. The orchestration of the workflow is done by a distributed rule engine that chains and executes the workflows and maintains transactional properties through recovery micro-services. They discuss the micro-service oriented architecture, compare the micro-service approach with traditional SOA, and describe the use of micro-services for implementing policy-based data management systems.

A distributed data management system to support large-scale data analysis

2019 ◽  
Vol 148 ◽  
pp. 105-115 ◽  
Author(s):  
Tamer Z. Emara ◽  
Joshua Zhexue Huang
Keyword(s):  
Data Analysis ◽  
Data Management ◽  
Management System ◽  
Large Scale ◽  
Data Management System ◽  
Distributed Data ◽  
Distributed Data Management ◽  
Large Scale Data ◽  
Scale Data

scholarly journals Experience of a low-maintenance distributed data management system

2014 ◽  
Vol 513 (3) ◽  
pp. 032095 ◽  
Author(s):  
Wataru Takase ◽  
Yoshimi Matsumoto ◽  
Adil Hasan ◽  
Francesca Di Lodovico ◽  
Yoshiyuki Watase ◽  
...  
Keyword(s):  
Data Management ◽  
Management System ◽  
Data Management System ◽  
Distributed Data ◽  
Distributed Data Management

scholarly journals Integration of Rucio in Belle II

2021 ◽  
Vol 251 ◽  
pp. 02057
Author(s):  
Cédric Serfon ◽  
Ruslan Mashinistov ◽  
John Steven De Stefano ◽  
Michel Hernández Villanueva ◽  
Hironori Ito ◽  
...  
Keyword(s):  
Data Management ◽  
Smooth Transition ◽  
Distributed Data ◽  
Simulation Data ◽  
Migration Strategy ◽  
Distributed Data Management ◽  
Order Of Magnitude ◽  
Belle Ii ◽  
Work Done ◽  
Data Management Software

The Belle II experiment, which started taking physics data in April 2019, will multiply the volume of data currently stored on its nearly 30 storage elements worldwide by one order of magnitude to reach about 340 PB of data (raw and Monte Carlo simulation data) by the end of operations. To tackle this massive increase and to manage the data even after the end of the data taking, it was decided to move the Distributed Data Management software from a homegrown piece of software to a widely used Data Management solution in HEP and beyond : Rucio. This contribution describes the work done to integrate Rucio with Belle II distributed computing infrastructure as well as the migration strategy that was successfully performed to ensure a smooth transition.

A distributed data management middleware for data-driven application systems

2005 ◽  
Author(s):  
S. Langella ◽  
S. Hastings ◽  
S. Oster ◽  
T. Kurc ◽  
U. Catalyurek ◽  
...  
Keyword(s):  
Data Management ◽  
Data Driven ◽  
Distributed Data ◽  
Distributed Data Management ◽  
Application Systems

Knowledge as a Service Framework for Collaborative Data Management in Cloud Environments - Disaster Domain

2016 ◽  
pp. 183-209 ◽  
Author(s):  
Katarina Grolinger ◽  
Emna Mezghani ◽  
Miriam A. M. Capretz ◽  
Ernesto Exposito
Keyword(s):  
Data Management ◽  
Service Oriented Architecture ◽  
Global Scale ◽  
Heterogeneous Data ◽  
Information Gathering ◽  
Current Data ◽  
Cloud Data ◽  
Related Data ◽  
Disaster Data ◽  
Service Oriented

Decision-making in disaster management requires information gathering, sharing, and integration by means of collaboration on a global scale and across governments, industries, and communities. Large volume of heterogeneous data is available; however, current data management solutions offer few or no integration capabilities and limited potential for collaboration. Moreover, recent advances in NoSQL, cloud computing, and Big Data open the door for new solutions in disaster data management. This chapter presents a Knowledge as a Service (KaaS) framework for disaster cloud data management (Disaster-CDM), with the objectives of facilitating information gathering and sharing; storing large amounts of disaster-related data; and facilitating search and supporting interoperability and integration. In the Disaster-CDM approach NoSQL data stores provide storage reliability and scalability while service-oriented architecture achieves flexibility and extensibility. The contribution of Disaster-CDM is demonstrated by integration capabilities, on examples of full-text search and querying services.

Sign in / Sign up

Export Citation Format

Share Document