A workflow-oriented cloud computing framework and programming model for data intensive application

Author(s):  
Jinshan Pang ◽  
Lizhen Cui ◽  
Yongqing Zheng ◽  
Haiyang Wang
2011 ◽  
Vol 267 ◽  
pp. 1054-1058
Author(s):  
Wu Chu Tang ◽  
Jun Xie

Each of tasks within the application depends on multiple datasets that may be distributed anywhere within the Cloud Computing. This paper defines the problem of scheduling distributed data-intensive application on to Gird resource and presents a formal resource and application model for the problem.


2020 ◽  
Vol 167 ◽  
pp. 102735 ◽  
Author(s):  
Mohammad Alkhalaileh ◽  
Rodrigo N. Calheiros ◽  
Quang Vinh Nguyen ◽  
Bahman Javadi

Usage of high-performance computing (HPC) infrastructure adopting cloud-computing environment offers an efficient solution for executing data intensive application. MapReduce (MR) is the favored high performance parallel computing framework used in BigData study, scientific, and data intensive applications. Hadoop is one of the significantly used MR based parallel computing framework by various organization as it is freely available open source framework from Apache foundation. The existing Hadoop MapReduce (HMR) based makespan model incurs memory and I/O overhead. Thus, affecting makespan performance. For overcoming research issues and challenges, this manuscript presented an efficient parallel HMR (PHMR) makespan model. The PHMR includes a parallel execution scheme in virtual computing worker to reduce makespan times using cloud computing framework. The PHMR model provides efficient memory management design within the virtual computing workers to minimize memory allocation and transmission overheads. For evaluating performance of PHMR of over existing model experiment are conducted on public cloud environment using Azure HDInsight cloud platform. Different application such as bioinformatics, tex mining, stream, and nonstream application is considered. The overall result obtained shows superior performance is attained by PHMR over existing model in term of makespan time reduction and correlation among practical and theoretical makespan values.


2020 ◽  
Vol 10 (19) ◽  
pp. 6676
Author(s):  
Quan Zou ◽  
Guoqing Li ◽  
Wenyang Yu

Resources related to remote-sensing data, computing, and models are scattered globally. The use of remote-sensing images for disaster-monitoring applications is data-intensive and involves complex algorithms. These characteristics make the timely and rapid processing of disaster-monitoring applications challenging and inefficient. Cloud computing provides a dynamically scalable resource over the Internet. The rapid development of cloud computing has led to an increase in the computational performance of data-intensive computing, providing powerful throughput by distributing computation across many distributed computers. However, the use of current cloud computing models in scientific applications using remote-sensing image data has been limited to a single image-processing algorithm rather than a well-established model and method. This poses problems for the development of complex disaster-monitoring applications on cloud platform architectures. For example, distributed computing strategies and remote-sensing image-processing algorithms are highly coupled and not reusable. The aims of this paper are to identify computational characteristics of various disaster-monitoring algorithms and classify them according to different computational characteristics; explore a reusable processing model based on the MapReduce programming model for disaster-monitoring applications; and then establish a programming model for each type of algorithm. This approach provides a simpler programming method for programmers to implement disaster-monitoring applications. Finally, some examples are given to explain the proposed method and test its performance.


2014 ◽  
Vol 543-547 ◽  
pp. 3092-3095
Author(s):  
Xiao Feng Wang

Based on the theory of cloud computing, this paper uses Hadoop distributed computing framework and the MapReduce programming model, designs and implements a campus cloud computing system for processing huge amounts of data. The system uses a three-layer architecture, has the flexibility to expand the scale, low development cost and ease of operation, reduces the difficulty of parallel programming and has the ability to efficiently handle massive data analysis and processing.


2018 ◽  
Vol 3 (1) ◽  
pp. 19 ◽  
Author(s):  
Matheus Alvian Wikanargo ◽  
Novian Adi Prasetyo ◽  
Angelina Pramana Thenata

AbstrakTeknologi cloud computing pada era sekarang berkembang pesat. Penerapan teknologi cloud computing sudah merambah ke berbagai industri, mulai dari perusahaan besar hingga perusahaan kecil dan menengah. Perambahan cloud computing di perindustrian berupa implementasi ke dalam sistem ERP. Namun, penetrasi teknologi ini dalam lingkup perusahaan kecil dan menengah (UKM) masih belum sekuat perusahaan besar. Penerapan ERP berbasis cloud computing yang masih tergolong baru tentu memiliki keuntungan dan penghambat yang mempengaruhi kinerja perusahaan. Hal tersebut menjadi salah satu pertimbangan UKM masih enggan menggunakan teknologi ini. Penelitian ini akan menganalisis framework yang paling sesuai untuk UKM dalam menerapkan sistem ERP berbasis cloud computing. Framework yang dianalisa yaitu Software as a Service (SaaS), Infrastructure as a Service (IaaS), dan Platform as as Service (PaaS). Ketiga framework ini akan dibandingkan menggunakan metode studi literatur. Tolak ukur yang menjadi acuan untuk perbandingan adalah Compatibility, Cost, Flexibility, Human Resource, Implementation, Maintenance, Security, dan Usability. Faktor-faktor tersebut akan diukur keuntungan dan penghambatnya jika diterapkan dalam SME. Hasil dari penilitian ini adalah Framework SaaS yang paling cocok untuk diterapkan pada perusahaan kecil dan menengah. Kata kunci— Cloud Computing, UKM, SaaS, IaaS, PaaS 


Sensors ◽  
2021 ◽  
Vol 21 (4) ◽  
pp. 1400
Author(s):  
Muhammad Adnan ◽  
Jawaid Iqbal ◽  
Abdul Waheed ◽  
Noor Ul Amin ◽  
Mahdi Zareei ◽  
...  

Modern vehicles are equipped with various sensors, onboard units, and devices such as Application Unit (AU) that support routing and communication. In VANETs, traffic management and Quality of Service (QoS) are the main research dimensions to be considered while designing VANETs architectures. To cope with the issues of QoS faced by the VANETs, we design an efficient SDN-based architecture where we focus on the QoS of VANETs. In this paper, QoS is achieved by a priority-based scheduling algorithm in which we prioritize traffic flow messages in the safety queue and non-safety queue. In the safety queue, the messages are prioritized based on deadline and size using the New Deadline and Size of data method (NDS) with constrained location and deadline. In contrast, the non-safety queue is prioritized based on First Come First Serve (FCFS) method. For the simulation of our proposed scheduling algorithm, we use a well-known cloud computing framework CloudSim toolkit. The simulation results of safety messages show better performance than non-safety messages in terms of execution time.


2011 ◽  
Vol 55-57 ◽  
pp. 1053-1057
Author(s):  
Gui De Zheng ◽  
Ming Chen

The next generation of scientific experiments and studies are being carried out by large collaborations of researchers distributed around the world engaged in analysis of huge collections of data generated by scientific instruments. Grid computing has emerged as an enabler for such collaborations as it aids communities in sharing resource to achieve common objective. This paper defines the problem of scheduling distributed data-intensive application on to Gird resource and presents a formal resource and application model for the problem.


Sign in / Sign up

Export Citation Format

Share Document