scholarly journals BIG DATA TRANSFER FOR TABLET-CLASS MACHINES

2014 ◽  
pp. 316-323
Author(s):  
Tevaganthan Veluppillai ◽  
Brandon Ortiz ◽  
Robert E. Hiromoto
Keyword(s):  
Big Data ◽  
Comparative Study ◽  
Data Transfer ◽  
Data File ◽  
Transport Protocol ◽  
Transport Protocols ◽  
Client Server ◽  
Remote Method Invocation ◽  
Block Data ◽  
Data Transfers

Several well-known data transfer protocols are presented in a comparative study to address the issue of big data transfer for tablet-class machines. The data transfer protocols include standard Java and C++, and block-data transfers protocols that use both the Java New IO (NIO) and the Zerocopy libraries, and a block-data C++ transfer protocol. Several experiments are described and results compared against the standard Java IO and C++ (stream-based file transport protocols). The motivation for this study is the development of a client/server big data file transport protocol for tablet-class client machines that rely on the Java Remote Method Invocation (RMI) package for distributed computing.

scholarly journals Optimized Data Transfers Based on the OpenCL Event Management Mechanism

2015 ◽  
Vol 2015 ◽  
pp. 1-16 ◽  
Author(s):  
Hiroyuki Takizawa ◽  
Shoichi Hirasawa ◽  
Makoto Sugawara ◽  
Isaac Gelado ◽  
Hiroaki Kobayashi ◽  
...  
Keyword(s):  
Big Data ◽  
Data Transfer ◽  
Data File ◽  
Transfer Performance ◽  
Event Management ◽  
Management Mechanism ◽  
File Access ◽  
Programming Techniques ◽  
Data Transfers ◽  
Main Thread

In standard OpenCL programming, hosts are supposed to control their compute devices. Since compute devices are dedicated to kernel computation, only hosts can execute several kinds of data transfers such as internode communication and file access. These data transfers require one host to simultaneously play two or more roles due to the need for collaboration between the host and devices. The codes for such data transfers are likely to be system-specific, resulting in low portability. This paper proposes an OpenCL extension that incorporates such data transfers into the OpenCL event management mechanism. Unlike the current OpenCL standard, the main thread running on the host is not blocked to serialize dependent operations. Hence, an application can easily use the opportunities to overlap parallel activities of hosts and compute devices. In addition, the implementation details of data transfers are hidden behind the extension, and application programmers can use the optimized data transfers without any tricky programming techniques. The evaluation results show that the proposed extension can use the optimized data transfer implementation and thereby increase the sustained data transfer performance by about 18% for a real application accessing a big data file.

scholarly journals NUMA-Aware Thread Scheduling for Big Data Transfers over Terabits Network Infrastructure

2018 ◽  
Vol 2018 ◽  
pp. 1-8
Author(s):  
Taeuk Kim ◽  
Awais Khan ◽  
Youngjae Kim ◽  
Preethika Kasu ◽  
Scott Atchley
Keyword(s):  
Big Data ◽  
Data Transfer ◽  
Analytical Data ◽  
File Systems ◽  
Memory Access ◽  
Thread Scheduling ◽  
Data Scheduling ◽  
Data Transfers ◽  
With Memory ◽  
Existing Data

The evergrowing trend of big data has led scientists to share and transfer the simulation and analytical data across the geodistributed research and computing facilities. However, the existing data transfer frameworks used for data sharing lack the capability to adopt the attributes of the underlying parallel file systems (PFS). LADS (Layout-Aware Data Scheduling) is an end-to-end data transfer tool optimized for terabit network using a layout-aware data scheduling via PFS. However, it does not consider the NUMA (Nonuniform Memory Access) architecture. In this paper, we propose a NUMA-aware thread and resource scheduling for optimized data transfer in terabit network. First, we propose distributed RMA buffers to reduce memory controller contention in CPU sockets and then schedule the threads based on CPU socket and NUMA nodes inside CPU socket to reduce memory access latency. We design and implement the proposed resource and thread scheduling in the existing LADS framework. Experimental results showed from 21.7% to 44% improvement with memory-level optimizations in the LADS framework as compared to the baseline without any optimization.

Stochastic Approximation-Based Transport Profiling for Big Data Movement Over Dedicated Connections

2018 ◽  
pp. 113-138
Author(s):  
Daqing Yun ◽  
Chase Q. Wu
Keyword(s):  
Big Data ◽  
Stochastic Approximation ◽  
High Performance ◽  
Data Transfer ◽  
Real Life ◽  
Exhaustive Search ◽  
Transport Protocols ◽  
Suitable Parameter ◽  
Extreme Scale ◽  
Parameter Values

High-performance networks featuring advance bandwidth reservation have been developed and deployed to support big data transfer in extreme-scale scientific applications. The performance of such big data transfer largely depends on the transport protocols being used. For a given protocol in a given network environment, different parameter settings may lead to different performance, and oftentimes the default settings do not yield the best performance. It is, however, impractical to conduct an exhaustive search in the large parameter space of transport protocols for a set of suitable parameter values. This chapter proposes a stochastic approximation-based transport profiler, namely FastProf, to quickly determine the optimal operational zone of a protocol over dedicated connections. The proposed method is evaluated using both emulations based on real-life measurements and experiments over physical connections. The results show that FastProf significantly reduces the profiling overhead while achieving a comparable level of transport performance with the exhaustive search-based approach.

Implementation and Analysis of TML Based on Different Transport Protocols

2010 ◽  
Vol 44-47 ◽  
pp. 997-1001
Author(s):  
Zhu Ge Bin ◽  
Yu Cheng ◽  
Wei Ming Wang
Keyword(s):  
Reference Point ◽  
Transport Protocol ◽  
Transport Protocols ◽  
Control Message

The messages in the Fp reference point of ForCES protocol can be divided into two kinds: control messages and redirect messages. According to this division, the control message channel was used to transmit control messages and the redirect message channel was used to transmit redirect messages. In this paper, we use different transport protocols to transmit control messages and redirect messages. Then test and analyze the TML based on different transport protocol to verify the correctness of the designs.

scholarly journals Bandwidth scheduling for big data transfer with two variable node-disjoint paths

2020 ◽  
Vol 22 (2) ◽  
pp. 130-144
Author(s):  
Aiqin Hou ◽  
Chase Qishi Wu ◽  
Liudong Zuo ◽  
Xiaoyang Zhang ◽  
Tao Wang ◽  
...  
Keyword(s):  
Big Data ◽  
Data Transfer ◽  
Disjoint Paths ◽  
Bandwidth Scheduling ◽  
Variable Node
Sign in / Sign up

Export Citation Format

Share Document