AN ARCHITECTURE OF FUSING COMMUNICATION AND EXECUTION FOR GLOBAL DISTRIBUTED PROCESSING

2001 ◽  
Vol 11 (01) ◽  
pp. 7-24 ◽  
Author(s):  
MAKOTO AMAMIYA ◽  
HIDEO TANIGUCHI ◽  
TAKANORI MATSUZAKI

We are pursuing the FUCE architecture project at Kyushu University. FUCE means FUsion of Communication and Execution. The main objective of our research is, as the name shows, to develop a new architecture that truly fuses communication and computation. The FUCE project develops a new on-chip-multi-processor and kernel software on it. We name the processor FUCE processor, and the kernel software as CEFOS (Communication and Execution Fusion OS). The FUCE processor is designed as a network node processor to perform mainly switching/transmitting of messages/transaction and handling its contents. FUCE processor architecture is designed as a multiprocessor-on-chip to support the fine-grain multi-threading. The kernel software CEFOS is also developed on the concept of multithreading. User and system processes are constructed as a set of threads, which are executed concurrently according to thread dependences.

2013 ◽  
Vol 60 (6) ◽  
pp. 356-360 ◽  
Author(s):  
I. Mansouri ◽  
P. Benoit ◽  
L. Torres ◽  
F. Clermidy
Keyword(s):  

Author(s):  
Yao Wu ◽  
Long Zheng ◽  
Brian Heilig ◽  
Guang R Gao

As the attention given to big data grows, cluster computing systems for distributed processing of large data sets become the mainstream and critical requirement in high performance distributed system research. One of the most successful systems is Hadoop, which uses MapReduce as a programming/execution model and takes disks as intermedia to process huge volumes of data. Spark, as an in-memory computing engine, can solve the iterative and interactive problems more efficiently. However, currently it is a consensus that they are not the final solutions to big data due to a MapReduce-like programming model, synchronous execution model and the constraint that only supports batch processing, and so on. A new solution, especially, a fundamental evolution is needed to bring big data solutions into a new era. In this paper, we introduce a new cluster computing system called HAMR which supports both batch and streaming processing. To achieve better performance, HAMR integrates high performance computing approaches, i.e. dataflow fundamental into a big data solution. With more specifications, HAMR is fully designed based on in-memory computing to reduce the unnecessary disk access overhead; task scheduling and memory management are in fine-grain manner to explore more parallelism; asynchronous execution improves efficiency of computation resource usage, and also makes workload balance across the whole cluster better. The experimental results show that HAMR can outperform Hadoop MapReduce and Spark by up to 19x and 7x respectively, in the same cluster environment. Furthermore, HAMR can handle scaling data size well beyond the capabilities of Spark.


2010 ◽  
Vol 6 (1) ◽  
pp. 201-210
Author(s):  
Motoi Ichihashi ◽  
Hélène Lhermet ◽  
Edith Beigné ◽  
Frédéric Rothan ◽  
Marc Belleville ◽  
...  

Sign in / Sign up

Export Citation Format

Share Document