scholarly journals Seamless integration of commercial Clouds with ATLAS Distributed Computing

2021 ◽  
Vol 251 ◽  
pp. 02005
Author(s):  
Fernando Barreiro Megino ◽  
Singh Bawa Harinder ◽  
Kaushik De ◽  
Johannes Elmsheuser ◽  
Alexei Klimentov ◽  
...  
Keyword(s):  
Distributed Computing ◽  
Hadron Collider ◽  
Added Value ◽  
Data Management System ◽  
Seamless Integration ◽  
New Developments ◽  
Computing Services ◽  
Workflow System ◽  
Cloud Computing Services ◽  
Computing Grid

The CERN ATLAS Experiment successfully uses a worldwide distributed computing Grid infrastructure to support its physics programme at the Large Hadron Collider (LHC). The Grid workflow system PanDA routinely manages up to 700,000 concurrently running production and analysis jobs to process simulation and detector data. In total more than 500 PB of data are distributed over more than 150 sites in the WLCG and handled by the ATLAS data management system Rucio. To prepare for the ever growing data rate in future LHC runs new developments are underway to embrace industry accepted protocols and technologies, and utilize opportunistic resources in a standard way. This paper reviews how the Google and Amazon Cloud computing services have been seamlessly integrated as a Grid site within PanDA and Rucio. Performance and brief cost evaluations will be discussed. Such setups could offer advanced Cloud tool-sets and provide added value for analysis facilities that are under discussions for LHC Run-4.

scholarly journals Overview of the ATLAS distributed computing system

2019 ◽  
Vol 214 ◽  
pp. 03010 ◽  
Author(s):  
Johannes Elmsheuser ◽  
Alessandro Di Girolamo
Keyword(s):  
Distributed Computing ◽  
Data Management ◽  
New Technologies ◽  
Computing System ◽  
Data Management System ◽  
Management Systems ◽  
New Developments ◽  
Atlas Experiment ◽  
Physics Program ◽  
Workflow System

The CERN ATLAS experiment successfully uses a worldwide computing infrastructure to support the physics program during LHC Run 2. The Grid workflow system PanDA routinely manages 250 to 500 thousand concurrently running production and analysis jobs to process simulation and detector data. In total more than 370 PB of data is distributed over more than 150 sites in the WLCG and handled by the ATLAS data management system Rucio. To prepare for the ever growing LHC luminosity in future runs new developments are underway to even more efficiently use opportunistic resources such as HPCs and utilize new technologies. This paper will review and explain the outline and the performance of the ATLAS distributed computing system and give an outlook to new workflow and data management ideas for the beginning of the LHC Run 3. It will be discussed that the ATLAS workflow and data management systems are robust, performant and can easily cope with the higher Run 2 LHC performance. There are presently no scaling issues and each subsystem is able to sustain the large loads.

scholarly journals Cloud computing service value: A message to the board

2012 ◽  
Vol 43 (4) ◽  
pp. 73-81 ◽  
Author(s):  
R. Von Solms ◽  
M. Viljoen
Keyword(s):  
Information Technology ◽  
Cloud Computing ◽  
New Developments ◽  
Service Value ◽  
Computing Model ◽  
Computing Services ◽  
Cloud Computing Service ◽  
Cloud Computing Services

This paper aims to alert the board to their duty of adding value to the organizations they represent by recognizing opportunities presented by new developments in information technology. Cloud computing is one such development, which is associated with opportunities and benefits. The service value that can be achieved by using this computing model will be influential in the adoption of cloud computing services. Service value is determined by the warranty and utility associated with that service. Thus, if an organization can associate itself with the utility and warranty on offer via cloud computing, it should consider the adoption of these services. Cloud computing is discussed in terms of service value. This promotes an understanding of factors to be considered when making decisions about the adoption of cloud computing.

scholarly journals The evolution of the CMS monitoring infrastructure

2021 ◽  
Vol 251 ◽  
pp. 02004
Author(s):  
Christian Ariza-Porras ◽  
Valentin Kuznetsov ◽  
Federica Legger ◽  
Rahul Indra ◽  
Nikodemas Tuckus ◽  
...  
Keyword(s):  
Distributed Computing ◽  
Hadron Collider ◽  
Simulated Data ◽  
Efficient Operation ◽  
Computing Services ◽  
Level Of Automation ◽  
Industry Standards ◽  
Reliable Monitoring ◽  
Monitoring Applications ◽  
It Department

The CMS experiment at the CERN LHC (Large Hadron Collider) relies on a distributed computing infrastructure to process the multi-petabyte datasets where the collision and simulated data are stored. A scalable and reliable monitoring system is required to ensure efficient operation of the distributed computing services, and to provide a comprehensive set of measurements of the system performances. In this paper we present the full stack of CMS monitoring applications, partly based on the MONIT infrastructure, a suite of monitoring services provided by the CERN IT department. These are complemented by a set of applications developed over the last few years by CMS, leveraging open-source technologies that are industry-standards in the IT world, such as Kubernetes and Prometheus. We discuss how this choice helped the adoption of common monitoring solutions within the experiment, and increased the level of automation in the operation and deployment of our services.

scholarly journals ATLAS Distributed Computing: Its Central Services core

2019 ◽  
Vol 214 ◽  
pp. 03061
Author(s):  
Christopher Jon Lee ◽  
Alessandro Di Girolamo ◽  
Johannes Elmsheuser ◽  
Alexey Buzykaev ◽  
Emil Obreshkov ◽  
...  
Keyword(s):  
Distributed Computing ◽  
Virtual Machines ◽  
Hadron Collider ◽  
It Services ◽  
Workload Management ◽  
Current State ◽  
On Line ◽  
Computing Grid ◽  
Central Services ◽  
Service Managers

The ATLAS Distributed Computing (ADC) Project is responsible for the off-line processing of data produced by the ATLAS experiment at the Large Hadron Collider (LHC) at CERN. It facilitates data and workload management for ATLAS computing on the Worldwide LHC Computing Grid (WLCG). ADC Central Services operations (CSOPS) is a vital part of ADC, responsible for the deployment and configuration of services needed by ATLAS computing and operation of those services on CERN IT infrastructure, providing knowledge of CERN IT services to ATLAS service managers and developers, and supporting them in case of issues. Currently this entails the management of 43 different OpenStack projects, with more than 5000 cores allocated for these virtual machines, as well as overseeing the distribution of 29 petabytes of storage space in EOS for ATLAS. As the LHC begins to get ready for the next long shut-down, which will bring in many new upgrades to allow for more data to be captured by the on-line systems, CSOPS must not only continue to support the existing services, but plan ahead for the expected increase in data, users, and services that will be required. This paper attempts to explain the current state of CSOPS as well as the strategies in place to maintain the service functionality in the long term.

scholarly journals Processing LHC data in the UK

2013 ◽  
Vol 371 (1983) ◽  
pp. 20120094 ◽  
Author(s):  
D. Colling ◽  
D. Britton ◽  
J. Gordon ◽  
S. Lloyd ◽  
A. Doyle ◽  
...  
Keyword(s):  
Large Hadron Collider ◽  
Distributed Computing ◽  
Hadron Collider ◽  
Grid Infrastructure ◽  
Computing Grid ◽  
Human Effort ◽  
The Uk ◽  
The Universe ◽  
Huge Investment ◽  
Computing Models

The Large Hadron Collider (LHC) is one of the greatest scientific endeavours to date. The construction of the collider itself and the experiments that collect data from it represent a huge investment, both financially and in terms of human effort, in our hope to understand the way the Universe works at a deeper level. Yet the volumes of data produced are so large that they cannot be analysed at any single computing centre. Instead, the experiments have all adopted distributed computing models based on the LHC Computing Grid. Without the correct functioning of this grid infrastructure the experiments would not be able to understand the data that they have collected. Within the UK, the Grid infrastructure needed by the experiments is provided by the GridPP project. We report on the operations, performance and contributions made to the experiments by the GridPP project during the years of 2010 and 2011—the first two significant years of the running of the LHC.

Cloud Computing in Library

2014 ◽  
Vol 3 ◽  
pp. 94-112
Author(s):  
Angelė Pečeliūnaitė
Keyword(s):  
Cloud Computing ◽  
Scientific Community ◽  
Digital Age ◽  
International Experience ◽  
The Internet ◽  
Library Services ◽  
General Activity ◽  
Computing Services ◽  
Cloud Computing Services

The article analyses the possibility of how Cloud Computing can be used by libraries to organise activities online. In order to achieve a uniform understanding of the essence of technology SaaS, IaaS, and PaaS, the article discusses the Cloud Computing services, which can be used for the relocation of libraries to the Internet. The improvement of the general activity of libraries in the digital age, the analysis of the international experience in the libraries are examples. Also the article discusses the results of a survey of the Lithuanian scientific community that confirms that 90% of the scientific community is in the interest of getting full access to e-publications online. It is concluded that the decrease in funding for libraries, Cloud Computing can be an economically beneficial step, expanding the library services and improving their quality.

One Implementation Method of Trust Evaluation Tool Used on Cloud Computing Service

2019 ◽  
Vol 12 (3) ◽  
pp. 223-231
Author(s):  
Shengju Yang
Keyword(s):  
Cloud Computing ◽  
Access Control ◽  
Service Providers ◽  
Assessment Method ◽  
Trust Evaluation ◽  
Evaluation Tool ◽  
Computing Services ◽  
Business Goals ◽  
Cloud Computing Service ◽  
Cloud Computing Services

In order to solve the trust problems between users and cloud computing service providers in cloud computing services, the existing trust evaluation technology and access control technology in the cloud computing service are analyzed. And the evaluation index of cloud computing is also analyzed. Users can calculate the relevant indicators of cloud computing service according to their own business goals, and choose the appropriate cloud computing services according to their own trust need. In addition, the reliability assessment method of users based on the service process is proposed. Cloud computing access control system can be used for user credibility evaluation, and it can handle user access requests according to user's creditability. In the study, a cloud computing service trust evaluation tool is designed, and the modeling and architecture designs of trust evaluation are also given. The effectiveness of the method is verified by experiments on cloud computing service evaluation methods.

scholarly journals The CMS monitoring infrastructure and applications

2021 ◽  
Vol 5 (1) ◽  
Author(s):  
Christian Ariza-Porras ◽  
Valentin Kuznetsov ◽  
Federica Legger
Keyword(s):  
Distributed Computing ◽  
Compact Muon Solenoid ◽  
Hadron Collider ◽  
Efficient Operation ◽  
Workload Management ◽  
Future Developments ◽  
Cms Experiment ◽  
Data Transfers ◽  
Monitoring Architecture ◽  
Globally Distributed

AbstractThe globally distributed computing infrastructure required to cope with the multi-petabyte datasets produced by the Compact Muon Solenoid (CMS) experiment at the Large Hadron Collider (LHC) at CERN comprises several subsystems, such as workload management, data management, data transfers, and submission of users’ and centrally managed production requests. To guarantee the efficient operation of the whole infrastructure, CMS monitors all subsystems according to their performance and status. Moreover, we track key metrics to evaluate and study the system performance over time. The CMS monitoring architecture allows both real-time and historical monitoring of a variety of data sources. It relies on scalable and open source solutions tailored to satisfy the experiment’s monitoring needs. We present the monitoring data flow and software architecture for the CMS distributed computing applications. We discuss the challenges, components, current achievements, and future developments of the CMS monitoring infrastructure.

scholarly journals Producing ‘good enough’ automated transcripts securely: Extending Bokhove and Downey (2018) to address security concerns

2021 ◽  
Vol 14 (1) ◽  
pp. 205979912098776
Author(s):  
Joseph Da Silva
Keyword(s):  
Cyber Security ◽  
English Language ◽  
Negative Impact ◽  
Early Career ◽  
Third Party ◽  
Doctoral Research ◽  
Computing Services ◽  
Cloud Computing Services ◽  
Audio Data

Interviews are an established research method across multiple disciplines. Such interviews are typically transcribed orthographically in order to facilitate analysis. Many novice qualitative researchers’ experiences of manual transcription are that it is tedious and time-consuming, although it is generally accepted within much of the literature that quality of analysis is improved through researchers performing this task themselves. This is despite the potential for the exhausting nature of bulk transcription to conversely have a negative impact upon quality. Other researchers have explored the use of automated methods to ease the task of transcription, more recently using cloud-computing services, but such services present challenges to ensuring confidentiality and privacy of data. In the field of cyber-security, these are particularly concerning; however, any researcher dealing with confidential participant speech should also be uneasy with third-party access to such data. As a result, researchers, particularly early-career researchers and students, may find themselves with no option other than manual transcription. This article presents a secure and effective alternative, building on prior work published in this journal, to present a method that significantly reduced, by more than half, interview transcription time for the researcher yet maintained security of audio data. It presents a comparison between this method and a fully manual method, drawing on data from 10 interviews conducted as part of my doctoral research. The method presented requires an investment in specific equipment which currently only supports the English language.

Sign in / Sign up

Export Citation Format

Share Document