scholarly journals The Rucio File Catalog in DIRAC implemented for Belle II

2021 ◽  
Vol 251 ◽  
pp. 02026
Author(s):  
Cédric Serfon ◽  
John Steven De Stefano ◽  
Michel Hernández Villanueva ◽  
Hironori Ito ◽  
Yuji Kato ◽  
...  
Keyword(s):  
Data Management ◽  
Management System ◽  
Data Management System ◽  
Distributed Data ◽  
Workload Management ◽  
Distributed Data Management ◽  
Belle Ii

DIRAC and Rucio are two standard pieces of software widely used in the HEP domain. DIRAC provides Workload and Data Management functionalities, among other things, while Rucio is a dedicated, advanced Distributed Data Management system. Many communities that already use DIRAC have expressed their interest in using DIRAC for Workload Management in combination with Rucio for Data Management. In this paper, we describe the integration of the Rucio File Catalog into DIRAC that was initially developed for the Belle II collaboration.

scholarly journals Experience of a low-maintenance distributed data management system

2014 ◽  
Vol 513 (3) ◽  
pp. 032095 ◽  
Author(s):  
Wataru Takase ◽  
Yoshimi Matsumoto ◽  
Adil Hasan ◽  
Francesca Di Lodovico ◽  
Yoshiyuki Watase ◽  
...  
Keyword(s):  
Data Management ◽  
Management System ◽  
Data Management System ◽  
Distributed Data ◽  
Distributed Data Management

scholarly journals The data management of heterogeneous resources in Belle II

2019 ◽  
Vol 214 ◽  
pp. 04031
Author(s):  
Malachi Schram
Keyword(s):  
Data Management ◽  
High Speed ◽  
Data Management System ◽  
Distributed Data ◽  
Global Networks ◽  
Distributed Data Management ◽  
Geographically Distributed ◽  
Belle Ii ◽  
Belle Experiment ◽  
Heterogeneous Resources

The Belle II experiment at the SuperKEKB collider in Tsukuba, Japan, has started taking physics data in early 2018 and plans to accumulate 50 ab-1, which is approximately 50 times more data than the Belle experiment. The collaboration expects it will require managing and processing approximately 200 PB of data. Computing at this scale requires efficient and coordinated use of the geographically distributed compute resources in North America, Asia and Europe and will take advantage of high-speed global networks. We present the general Belle II the distributed data management system and computing results from the first phase of data taking.

scholarly journals Integrating a dynamic data federation into the ATLAS distributed data management system

2019 ◽  
Vol 214 ◽  
pp. 07009
Author(s):  
Frank Berghaus ◽  
Tobias Wegner ◽  
Mario Lassnig ◽  
Marcus Ebert ◽  
Cedric Serfon ◽  
...  
Keyword(s):  
Data Management ◽  
Management System ◽  
Data Management System ◽  
Distributed Data ◽  
Dynamic Data ◽  
Distributed Data Management ◽  
Data Federation ◽  
Other Information ◽  
Multiple Copies ◽  
Set Up

Input data for applications that run in cloud computing centres can be stored at remote repositories, typically with multiple copies of the most popular data stored at many sites. Locating and retrieving the remote data can be challenging, and we believe that federating the storage can address this problem. In this approach, the closest copy of the data is used based on geographical or other information. Currently, we are using the dynamic data federation, Dynafed, a software solution developed by CERN IT. Dynafed supports several industry standard interfaces, such as Amazon S3, Microsoft Azure and HTTP with WebDAV extensions. Dynafed functions as an abstraction layer under which protocol-dependent authentication details are hidden from the user, requiring the user to only provide an X509 certificate. We have set up an instance of Dynafed and integrated it into the ATLAS distributed data management system, Rucio. We report on the challenges faced during the installation and integration.

scholarly journals Monitoring the atlas distributed data management system

2008 ◽  
Vol 119 (7) ◽  
pp. 072027
Author(s):  
R Ricardo ◽  
B Miguel ◽  
G Benjamin ◽  
G Vincent ◽  
L Mario ◽  
...  
Keyword(s):  
Data Management ◽  
Management System ◽  
Data Management System ◽  
Distributed Data ◽  
Distributed Data Management
Sign in / Sign up

Export Citation Format

Share Document