A Bandwidth Challenge at Super Computing (SC) Conference: Large-Scale Data Transfer Using 10Gbps Network

2009 ◽  
Author(s):  
Ken T. Murata ◽  
Eizen Kimura ◽  
Kazunori Yamamoto ◽  
Daisuke Matsuoka ◽  
Hironori Shimazu ◽  
...  
Keyword(s):  
Large Scale ◽  
Data Transfer ◽  
Large Scale Data ◽  
Super Computing ◽  
Scale Data

scholarly journals A Framework for International Collaboration on ITER Using Large-Scale Data Transfer to Enable Near-Real-Time Analysis

2021 ◽  
Vol 77 (2) ◽  
pp. 98-108
Author(s):  
R. M. Churchill ◽  
C. S. Chang ◽  
J. Choi ◽  
J. Wong ◽  
S. Klasky ◽  
...  
Keyword(s):  
Real Time ◽  
International Collaboration ◽  
Large Scale ◽  
Data Transfer ◽  
Time Analysis ◽  
Real Time Analysis ◽  
Large Scale Data ◽  
Scale Data

scholarly journals Architecting a distributed bioinformatics platform with iRODS and iPlant Agave API

2015 ◽  
Author(s):  
Liya Wang ◽  
Peter Van Buren ◽  
Doreen Ware
Keyword(s):  
Large Scale ◽  
Genome Wide Association Study ◽  
Data Transfer ◽  
Genomic Analysis ◽  
The Past ◽  
Large Scale Data ◽  
Genome Wide ◽  
A Genome ◽  
Bioinformatics Workflow ◽  
Scale Data

Over the past few years, cloud-based platforms have been proposed to address storage, management, and computation of large-scale data, especially in the field of genomics. However, for collaboration efforts involving multiple institutes, data transfer and management, interoperability and standardization among different platforms have imposed new challenges. This paper proposes a distributed bioinformatics platform that can leverage local clusters with remote computational clusters for genomic analysis using the unified bioinformatics workflow. The platform is built with a data server configured with iRODS, a computation cluster authenticated with iPlant Agave system, and web server to interact with the platform. A Genome-Wide Association Study workflow is integrated to validate the feasibility of the proposed approach.

Data Transfer and Storage in Cloud Computing

2013 ◽  
pp. 226-237
Author(s):  
Yushi Shen ◽  
Yale Li ◽  
Ling Wu ◽  
Shaofeng Liu ◽  
Qian Wen
Keyword(s):  
Cloud Computing ◽  
Large Scale ◽  
Data Transfer ◽  
Internet Applications ◽  
Data Preservation ◽  
Network Bandwidth ◽  
Large Scale Data ◽  
And Storage ◽  
Scale Data

When network bandwidth is no longer a bottleneck for Internet applications, we still face the challenges of how to utilize the bandwidth in full when transferring data. This might sound straightforward, but it is far from being easy or straightforward. Lots of work has been done to improve the utilization of abundant network bandwidth, while maintaining the same reliability, efficiency, and fairness of slow protocols, like TCP. In this chapter, the authors introduce the background of emerging bandwidth-intensive applications, related works that are in place to solve these issues, and their limitations to make the challenges clear to the readers. Similar challenges also present for large-scale data preservation.

Sign in / Sign up

Export Citation Format

Share Document