AI-Based Container Orchestration for Federated Cloud Environments

Author(s):  
Yodit Gebrealif ◽  
Mohammed Mubarkoot ◽  
Jörn Altmann ◽  
Bernhard Egger
2018 ◽  
Author(s):  
Ola Spjuth ◽  
Marco Capuccini ◽  
Matteo Carone ◽  
Anders Larsson ◽  
Wesley Schaal ◽  
...  

Containers are gaining popularity in life science research as they encompass all dependencies of provisioned tools and simplifies software installations for end users, as well as offering a form of isolation between processes. Scientific workflows are ideal to chain containers into data analysis pipelines to sustain reproducible science. In this manuscript we review the different approaches to use containers inside the workflow tools Nextflow, Galaxy, Pachyderm, Luigi, and SciPipe when deployed in cloud environments. A particular focus is placed on the workflow tool’s interaction with the Kubernetes container orchestration framework.


Author(s):  
Adi Farshteindiker ◽  
Rami Puzis

With the advent of microservice-based software architectures, an increasing number of modern cloud environments and enterprises use operating system level virtualization, often referred to as containers. Docker Swarm is one of the most popular container orchestration infrastructures, providing high availability and fault tolerance. Occasionally discovered container escape vulnerabilities allow adversaries to execute code on the host operating system and operate within the cloud infrastructure. We show that docker swarm is currently not secured against misbehaving manager nodes and allows a high impact, high probability privilege escalation attack that we refer to as leadership hijacking. Cloud lateral movement and defense evasion payloads allow an adversary to leverage the docker swarm functionality to control each and every host in the underlying cluster. We demonstrate an end-to-end attack, in which an adversary with access to an application running on the cluster achieves full control of the cluster. To reduce the probability of a successful high impact attack, container orchestration infrastructures must reduce the trust level of participating nodes and in particular, incorporate adversary immune leader election algorithms.


2018 ◽  
Author(s):  
Ola Spjuth ◽  
Marco Capuccini ◽  
Matteo Carone ◽  
Anders Larsson ◽  
Wesley Schaal ◽  
...  

Containers are gaining popularity in life science research as they encompass all dependencies of provisioned tools and simplifies software installations for end users, as well as offering a form of isolation between processes. Scientific workflows are ideal to chain containers into data analysis pipelines to sustain reproducible science. In this manuscript we review the different approaches to use containers inside the workflow tools Nextflow, Galaxy, Pachyderm, Luigi, and SciPipe when deployed in cloud environments. A particular focus is placed on the workflow tool’s interaction with the Kubernetes container orchestration framework.


F1000Research ◽  
2021 ◽  
Vol 10 ◽  
pp. 513
Author(s):  
Ola Spjuth ◽  
Marco Capuccini ◽  
Matteo Carone ◽  
Anders Larsson ◽  
Wesley Schaal ◽  
...  

Containers are gaining popularity in life science research as they provide a solution for encompassing dependencies of provisioned tools, simplify software installations for end users and offer a form of isolation between processes. Scientific workflows are ideal for chaining containers into data analysis pipelines to aid in creating reproducible analyses. In this article, we review a number of approaches to using containers as implemented in the workflow tools Nextflow, Galaxy, Pachyderm, Argo, Kubeflow, Luigi and SciPipe, when deployed in cloud environments. A particular focus is placed on the workflow tool’s interaction with the Kubernetes container orchestration framework.


Author(s):  
Maria Rodriguez ◽  
Rajkumar Buyya

Containers are widely used by organizations to deploy diverse workloads such as web services, big data, and IoT applications. Container orchestration platforms are designed to manage the deployment of containerized applications in large-scale clusters. The majority of these platforms optimize the scheduling of containers on a fixed-sized cluster and are not enabled to autoscale the size of the cluster nor to consider features specific to public cloud environments. This chapter presents a resource management approach with three objectives: 1) optimize the initial placement of containers by efficiently scheduling them on existing resources, 2) autoscale the number of resources at runtime based on the cluster's workload, and 3) consolidate applications into fewer VMs at runtime. The framework was implemented as a Kubernetes plugin and its efficiency was evaluated on an Australian cloud infrastructure. The experiments demonstrate that a reduction of 58% in cost can be achieved by dynamically managing the cluster size and placement of applications.


Entropy ◽  
2021 ◽  
Vol 23 (7) ◽  
pp. 914
Author(s):  
Adi Farshteindiker ◽  
Rami Puzis

With the advent of microservice-based software architectures, an increasing number of modern cloud environments and enterprises use operating system level virtualization, which is often referred to as container infrastructures. Docker Swarm is one of the most popular container orchestration infrastructures, providing high availability and fault tolerance. Occasionally, discovered container escape vulnerabilities allow adversaries to execute code on the host operating system and operate within the cloud infrastructure. We show that Docker Swarm is currently not secured against misbehaving manager nodes. This allows a high impact, high probability privilege escalation attack, which we refer to as leadership hijacking, the possibility of which is neglected by the current cloud security literature. Cloud lateral movement and defense evasion payloads allow an adversary to leverage the Docker Swarm functionality to control each and every host in the underlying cluster. We demonstrate an end-to-end attack, in which an adversary with access to an application running on the cluster achieves full control of the cluster. To reduce the probability of a successful high impact attack, container orchestration infrastructures must reduce the trust level of participating nodes and, in particular, incorporate adversary immune leader election algorithms.


Author(s):  
Ola Spjuth ◽  
Marco Capuccini ◽  
Matteo Carone ◽  
Anders Larsson ◽  
Wesley Schaal ◽  
...  

Containers are gaining popularity in life science research as they provide a solution for encompassing dependencies of provisioned tools, simplify software installations for end users and offer a form of isolation between processes. Scientific workflows are ideal for chaining containers into data analysis pipelines to aid in creating reproducible analyses. In this manuscript we review a number of approaches to using containers as implemented in the workflow tools Nextflow, Galaxy, Pachyderm, Argo, Kubeflow, Luigi and SciPipe, when deployed in cloud environments. A particular focus is placed on the workflow tool’s interaction with the Kubernetes container orchestration framework.


Sign in / Sign up

Export Citation Format

Share Document