scholarly journals Accumulating computational resource usage of genomic data analysis workflow to optimize cloud computing instance selection

2018 ◽  
Author(s):  
Tazro Ohta ◽  
Tomoya Tanjo ◽  
Osamu Ogasawara
Keyword(s):  
Cloud Computing ◽  
Data Analysis ◽  
Workflow Management ◽  
Resource Requirement ◽  
Computing Platform ◽  
Resource Requirements ◽  
Analysis Workflow ◽  
Computational Resources ◽  
Transcriptome Quantification ◽  
Runtime Metrics

AbstractBackgroundContainer virtualization technologies such as Docker became popular in the bioinformatics domain as they improve portability and reproducibility of software deployment. Along with software packaged in containers, the workflow description standards Common Workflow Language also enabled to perform data analysis on multiple different computing environments with ease. These technologies accelerate the use of on-demand cloud computing platform which can scale out according to the amount of data. However, to optimize the time and the budget on a use of cloud, users need to select a suitable instance type corresponding to the resource requirements of their workflows.ResultsWe developed CWL-metrics, a system to collect runtime metrics of Docker containers and workflow metadata to analyze resource requirement of workflows. We demonstrated the analysis by using seven transcriptome quantification workflows on six instance types. The result showed instance type options of lower financial cost and faster execution time with required amount of computational resources.ConclusionsThe summary of resource requirements of workflow executions provided by CWL-metrics can help users to optimize the selection of cloud computing instance. The runtime metrics data also accelerate to share workflows among different workflow management frameworks.

Research of Big Data Processing Platform

2014 ◽  
Vol 484-485 ◽  
pp. 922-926
Author(s):  
Xiang Ju Liu
Keyword(s):  
Cloud Computing ◽  
Big Data ◽  
Data Analysis ◽  
Network Architecture ◽  
Data Analytics ◽  
Big Data Analytics ◽  
Big Data Analysis ◽  
Computing Platform ◽  
Operational Characteristics ◽  
Analysis Platform

This paper introduces the operational characteristics of the era of big data and the current era of big data challenges, and exhaustive research and design of big data analytics platform based on cloud computing, including big data analytics platform architecture system, big data analytics platform software architecture , big data analytics platform network architecture big data analysis platform unified program features and so on. The paper also analyzes the cloud computing platform for big data analysis program unified competitive advantage and development of business telecom operators play a certain role in the future.

scholarly journals Interoperable and scalable data analysis with microservices: Applications in Metabolomics

2017 ◽  
Author(s):  
Payam Emami Khoonsari ◽  
Pablo Moreno ◽  
Sven Bergmann ◽  
Joachim Burman ◽  
Marco Capuccini ◽  
...  
Keyword(s):  
Mass Spectrometry ◽  
Data Analysis ◽  
Large Scale ◽  
Metabolite Identification ◽  
Access Point ◽  
Scientific Discipline ◽  
Resonance Spectroscopy ◽  
Magnetic Resonance Spectroscopy Study ◽  
Analysis Workflow ◽  
Computational Resources

Developing a robust and performant data analysis workflow that integrates all necessary components whilst still being able to scale over multiple compute nodes is a challenging task. We introduce a generic method based on the microservice architecture, where software tools are encapsulated as Docker containers that can be connected into scientific workflows and executed in parallel using the Kubernetes container orchestrator. The access point is a virtual research environment which can be launched on-demand on cloud resources and desktop computers. IT-expertise requirements on the user side are kept to a minimum, and established workflows can be re-used effortlessly by any novice user. We validate our method in the field of metabolomics on two mass spectrometry studies, one nuclear magnetic resonance spectroscopy study and one fluxomics study, showing that the method scales dynamically with increasing availability of computational resources. We achieved a complete integration of the major software suites resulting in the first turn-key workflow encompassing all steps for mass-spectrometry-based metabolomics including preprocessing, multivariate statistics, and metabolite identification. Microservices is a generic methodology that can serve any scientific discipline and opens up for new types of large-scale integrative science.

Cloud Computing for BioLabs

2015 ◽  
pp. 1272-1293
Author(s):  
Abraham Pouliakis ◽  
Aris Spathis ◽  
Christine Kottaridi ◽  
Antonia Mourtzikou ◽  
Marilena Stamouli ◽  
...  
Keyword(s):  
Cloud Computing ◽  
Data Analysis ◽  
Drug Design ◽  
Large Scale ◽  
Future Research ◽  
New Paradigm ◽  
Computing Power ◽  
The Everyday ◽  
Potential Applications ◽  
Computational Resources

Cloud computing has quickly emerged as an exciting new paradigm providing models of computing and services. Via cloud computing technology, bioinformatics tools can be made available as services to anyone, anywhere, and via any device. Large bio-datasets, highly complex algorithms, computing power demanding analysis methods, and the sudden need for hardware and computational resources provide an ideal environment for large-scale bio-data analysis for cloud computing. Cloud computing is already applied in the fields of biology and biochemistry, via numerous paradigms providing novel ideas stimulating future research. The concept of BioCloud has rapidly emerged with applications related to genomics, drug design, biology tools on the cloud, bio-databases, cloud bio-computing, and numerous applications related to biology and biochemistry. In this chapter, the authors present research results related to biology-related laboratories (BioLabs) as well as potential applications for the everyday clinical routine.

Challenges of Cloud Computing Adoption From the TOE Framework Perspective

2018 ◽  
Vol 14 (3) ◽  
pp. 77-94 ◽  
Author(s):  
Omar Al-Hujran ◽  
Enas M. Al-Lozi ◽  
Mutaz M. Al-Debei ◽  
Mahmoud Maqableh
Keyword(s):  
Cloud Computing ◽  
Developing Countries ◽  
Time To Market ◽  
Computing Platform ◽  
On Demand ◽  
Cloud Computing Adoption ◽  
Generation Computing ◽  
Computational Resources ◽  
Depth Interviews ◽  
Reduced Time

Cloud computing can be classified as a third-generation computing platform which refers to on-demand delivery of computing infrastructure and services via a network, usually the Internet. Cloud computing promises to provide several advantages to its adopters such as: cost advantage, availability, scalability, flexibility, reduced time to market and dynamic access to computational resources. Notwithstanding the numerous advantages of cloud computing, its implementation and adoption in developing countries is still limited and surrounded by variety of issues. Hence, the main objective of this article is to identify the main challenges facing the utilization of these services in developing countries, particularly Jordan. To achieve the above-mentioned objective, six in-depth interviews with ICT officials and experts in the domain of cloud computing were used as the main data collection method. The challenges of cloud computing adoption emerged in this study are classified into technological, organizational and environmental factors.

Challenges of Cloud Computing Adoption From the TOE Framework Perspective

2019 ◽  
pp. 1312-1332
Author(s):  
Omar Al-Hujran ◽  
Enas M. Al-Lozi ◽  
Mutaz M. Al-Debei ◽  
Mahmoud Maqableh
Keyword(s):  
Cloud Computing ◽  
Developing Countries ◽  
Computing Platform ◽  
Cloud Computing Adoption ◽  
Generation Computing ◽  
Toe Framework ◽  
Computational Resources ◽  
Depth Interviews ◽  
Reduced Time ◽  
Computing Infrastructure

Cloud computing can be classified as a third-generation computing platform which refers to on-demand delivery of computing infrastructure and services via a network, usually the Internet. Cloud computing promises to provide several advantages to its adopters such as: cost advantage, availability, scalability, flexibility, reduced time to market and dynamic access to computational resources. Notwithstanding the numerous advantages of cloud computing, its implementation and adoption in developing countries is still limited and surrounded by variety of issues. Hence, the main objective of this article is to identify the main challenges facing the utilization of these services in developing countries, particularly Jordan. To achieve the above-mentioned objective, six in-depth interviews with ICT officials and experts in the domain of cloud computing were used as the main data collection method. The challenges of cloud computing adoption emerged in this study are classified into technological, organizational and environmental factors.

scholarly journals An Intelligent Approach to Allocating Resources within an Agent-Based Cloud Computing Platform

2020 ◽  
Vol 10 (12) ◽  
pp. 4361
Author(s):  
Fernando De la Prieta ◽  
Sara Rodríguez-González ◽  
Pablo Chamoso ◽  
Yves Demazeau ◽  
Juan Manuel Corchado
Keyword(s):  
Cloud Computing ◽  
New Technologies ◽  
Cloud Services ◽  
Case Based Reasoning ◽  
Computing Platform ◽  
Reasoning System ◽  
Computing Paradigm ◽  
Cloud Computing Platform ◽  
Intelligent Approach ◽  
Computational Resources

The cloud computing paradigm has the ability to adapt to new technologies and provide consistent cloud services. These features have led to the widespread use of the paradigm, making it necessary for the underlying computer infrastructure to cope with the increased demand and the high number of end users. Platforms often use classical mathematical models for this purpose, helping assign computational resources to the services provided to the final user. Although this kind of model is valid and widespread, it can be refined through intelligent techniques. Therefore, this research presents a novel system consisting of a multi-agent system, which integrates a case-based reasoning system. The resulting system dynamically allocates resources within a cloud computing platform. This approach, which is distributed and scalable, can learn from previous experiences and produce better results in each resource allocation. A model of the system has been implemented and tested on a real cloud platform with successful results.

E-Commerce Recommendation System Based on Cloud Computing

2014 ◽  
Vol 926-930 ◽  
pp. 2570-2573
Author(s):  
Wei Zhao ◽  
Hong Tao Zhang
Keyword(s):  
Cloud Computing ◽  
Data Analysis ◽  
Data Exchange ◽  
Recommendation System ◽  
Large Data ◽  
Analysis Data ◽  
The Novel ◽  
Computing Platform ◽  
Large Data Analysis ◽  
Business Flow

According to the Present Situations were that there is an urgent demand for large data analysis in Electronic Commerce, by using cloud computings advantage in storing and analyzing mass data, the solution of new project which can analysis data was proposed, based on the cloud computing. Firstly, aiming to the advantages of the cloud computing platform, the novel data-analysis architecture was designed, then the business flow chart by using cloud computing analysis on the architecture. Finally the project is validated by practical application and possesses certain reference meaning in data exchange based on cloud computing.

Sign in / Sign up

Export Citation Format

Share Document