Data Transfer and Storage in Cloud Computing

2013 ◽  
pp. 226-237
Author(s):  
Yushi Shen ◽  
Yale Li ◽  
Ling Wu ◽  
Shaofeng Liu ◽  
Qian Wen
Keyword(s):  
Cloud Computing ◽  
Large Scale ◽  
Data Transfer ◽  
Internet Applications ◽  
Data Preservation ◽  
Network Bandwidth ◽  
Large Scale Data ◽  
And Storage ◽  
Scale Data

When network bandwidth is no longer a bottleneck for Internet applications, we still face the challenges of how to utilize the bandwidth in full when transferring data. This might sound straightforward, but it is far from being easy or straightforward. Lots of work has been done to improve the utilization of abundant network bandwidth, while maintaining the same reliability, efficiency, and fairness of slow protocols, like TCP. In this chapter, the authors introduce the background of emerging bandwidth-intensive applications, related works that are in place to solve these issues, and their limitations to make the challenges clear to the readers. Similar challenges also present for large-scale data preservation.

scholarly journals A Framework for International Collaboration on ITER Using Large-Scale Data Transfer to Enable Near-Real-Time Analysis

2021 ◽  
Vol 77 (2) ◽  
pp. 98-108
Author(s):  
R. M. Churchill ◽  
C. S. Chang ◽  
J. Choi ◽  
J. Wong ◽  
S. Klasky ◽  
...  
Keyword(s):  
Real Time ◽  
International Collaboration ◽  
Large Scale ◽  
Data Transfer ◽  
Time Analysis ◽  
Real Time Analysis ◽  
Large Scale Data ◽  
Scale Data

scholarly journals Architecting a distributed bioinformatics platform with iRODS and iPlant Agave API

2015 ◽  
Author(s):  
Liya Wang ◽  
Peter Van Buren ◽  
Doreen Ware
Keyword(s):  
Large Scale ◽  
Genome Wide Association Study ◽  
Data Transfer ◽  
Genomic Analysis ◽  
The Past ◽  
Large Scale Data ◽  
Genome Wide ◽  
A Genome ◽  
Bioinformatics Workflow ◽  
Scale Data

Over the past few years, cloud-based platforms have been proposed to address storage, management, and computation of large-scale data, especially in the field of genomics. However, for collaboration efforts involving multiple institutes, data transfer and management, interoperability and standardization among different platforms have imposed new challenges. This paper proposes a distributed bioinformatics platform that can leverage local clusters with remote computational clusters for genomic analysis using the unified bioinformatics workflow. The platform is built with a data server configured with iRODS, a computation cluster authenticated with iPlant Agave system, and web server to interact with the platform. A Genome-Wide Association Study workflow is integrated to validate the feasibility of the proposed approach.

The Research of MapReduce on the Cloud Computing

2012 ◽  
Vol 182-183 ◽  
pp. 2127-2130
Author(s):  
Tie Liang Gao ◽  
Jiao Li ◽  
Jun Peng Zhang ◽  
Bing Jie Shi
Keyword(s):  
Cloud Computing ◽  
Parallel Computing ◽  
Large Scale ◽  
Parallel Program ◽  
Large Scale Data ◽  
Distribute System ◽  
Scale Data

MapReduce is a kind of model of program that is use in the parallel computing about large scale data muster in the Cloud Computing[1] , it mainly consist of map and reduce . MapReduce is tremendously convenient for the programmer who can’t familiar with the parallel program .These people use the MapReduce to run their program on the distribute system. This paper mainly research the model and process and theory of MapReduce .

Sign in / Sign up

Export Citation Format

Share Document