scholarly journals CVMFS: Stratum 0 in Kubernetes

2019 ◽  
Vol 214 ◽  
pp. 07032
Author(s):  
David Schultz ◽  
Heath Skarlupka ◽  
Vladimir Brik ◽  
Gonzalo Merino

IceCube is a cubic kilometer neutrino detector located at the south pole. CVMFS is a key component to IceCube’s Distributed High Throughput Computing analytics workflow for sharing 500GB of software across datacenters worldwide. Building the IceCube software suite across multiple platforms and deploying it into CVMFS has until recently been a manual, time consuming task that doesn’t fit well within an agile continuous delivery framework. Within the last 2 years a plethora of tooling around microservices has created an opportunity to upgrade the IceCube software build and deploy pipeline. We present a framework using Kubernetes to deploy Buildbot. The Buildbot pipeline is a set of pods (docker containers) in the Kubernetes cluster that builds the IceCube software across multiple platforms, tests the new software for critical errors, syncs the software to a containerized CVMFS server, and finally executes a publish. The time from code commit to CVMFS publish has been greatly reduced and has enabled the capability of publishing nightly builds to CVMFS.

2005 ◽  
Vol 13 ◽  
pp. 949-950
Author(s):  
Francis Halzen

AbstractSolving the century-old puzzle of how and where cosmic rays are accelerated mostly drives the design of high-energy neutrino telescopes. It calls, along with a diversity of science goals reaching particle physics, astrophysics and cosmology, for the construction of a kilometer-scale neutrino detector. This led to the IceCube concept to transform a kilometer cube of transparent Antarctic Ice, one mile below the South Pole, into a neutrino telescope.


2019 ◽  
Vol 214 ◽  
pp. 03058
Author(s):  
David Schultz ◽  
Juan Carlos Díaz Vélez

IceCube is a cubic kilometer neutrino detector located at the south pole. IceProd is IceCube’s internal dataset management system, keeping track of where, when, and how jobs run. It schedules jobs from submitted datasets to HTCondor, keeping track of them at every stage of the lifecycle. Many updates have happened in the last years to improve stability and scalability, as well as increase user access. Along the way, the IceProd codebase switched from Python 2 to Python 3.


2012 ◽  
Vol 35 (7) ◽  
pp. 457-477 ◽  
Author(s):  
P. Allison ◽  
J. Auffenberg ◽  
R. Bard ◽  
J.J. Beatty ◽  
D.Z. Besson ◽  
...  

Author(s):  
D. Besson ◽  
R. Nahnhauer ◽  
P.B. Price ◽  
D. Tosi ◽  
J. Vandenbroucke ◽  
...  

2019 ◽  
Vol 214 ◽  
pp. 04055
Author(s):  
Vladimir Brik ◽  
Patrick Meade ◽  
Gonzalo Merino ◽  
Jan Oertlin ◽  
David Schultz ◽  
...  

IceCube is a cubic kilometer neutrino detector located at the south pole. Metadata for files in IceCube have traditionally been handled on an application by application basis, with no user-facing access. There has been no unified view of data files, and users often query the filesystem to locate files. Recently effort has been put into creating a unified view in a central metadata catalog. Going for a simple solution, we created a userfacing REST API backed by a NoSQL database. All major data producers add their metadata to this central catalog. Schema generation is identified as an important aspect of multi-application metadata services.


2018 ◽  
Vol 45 ◽  
pp. 31-38
Author(s):  
Federica La Longa ◽  
Massimo Crescimbene ◽  
Lucilla Alfonsi ◽  
Claudio Cesaroni ◽  
Vincenzo Romano
Keyword(s):  

Sign in / Sign up

Export Citation Format

Share Document