Comparison of sort algorithms in Hadoop and PCJ

AbstractSorting algorithms are among the most commonly used algorithms in computer science and modern software. Having efficient implementation of sorting is necessary for a wide spectrum of scientific applications. This paper describes the sorting algorithm written using the partitioned global address space (PGAS) model, implemented using the Parallel Computing in Java (PCJ) library. The iterative implementation description is used to outline the possible performance issues and provide means to resolve them. The key idea of the implementation is to have an efficient building block that can be easily integrated into many application codes. This paper also presents the performance comparison of the PCJ implementation with the MapReduce approach, using Apache Hadoop TeraSort implementation. The comparison serves to show that the performance of the implementation is good enough, as the PCJ implementation shows similar efficiency to the Hadoop implementation.

Download Full-text

Performance Comparison of Parallel Sorting Algorithms on Homogeneous Cluster of Workstations

Advanced Materials Research ◽

10.4028/www.scientific.net/amr.433-440.3900 ◽

2012 ◽

Vol 433-440 ◽

pp. 3900-3904

Author(s):

Lai Lai Win Kyi ◽

Nay Min Tun

Keyword(s):

Close Relation ◽

Performance Comparison ◽

Parallel Computer ◽

Sorting Algorithm ◽

Parallel Sorting ◽

Sorting Algorithms ◽

Homogeneous Cluster ◽

Cluster Of Workstations ◽

Parallel Merge ◽

Merge Sort

Sorting appears the most attention among all computational tasks over the past years because sorted data is at the heart of many computations. Sorting is of additional importance to parallel computing because of its close relation to the task of routing data among processes, which is an essential part of many parallel algorithms. Many parallel sorting algorithms have been investigated for a variety of parallel computer architectures. In this paper, three parallel sorting algorithms have been implemented and compared in terms of their overall execution time. The algorithms implemented are the odd-even transposition sort, parallel merge sort and parallel shell sort. Cluster of Workstations or Windows Compute Cluster has been used to compare the algorithms implemented. The C# programming language is used to develop the sorting algorithms. The MPI library has been selected to establish the communication and synchronization between processors. The time complexity for each parallel sorting algorithm will also be mentioned and analyzed.

Download Full-text

LANGUAGES FOR HIGH-PRODUCTIVITY COMPUTING: THE DARPA HPCS LANGUAGE PROJECT

Parallel Processing Letters ◽

10.1142/s0129626407002892 ◽

2007 ◽

Vol 17 (01) ◽

pp. 89-102 ◽

Cited By ~ 17

Author(s):

EWING LUSK ◽

KATHERINE YELICK

Keyword(s):

Parallel Computing ◽

Address Space ◽

Partitioned Global Address Space ◽

High Productivity ◽

Implementation Challenges ◽

Current State ◽

Global Address Space ◽

Strong Competition

We present a summary of the current state of DARPA's HPCS language project. We describe the challenges facing any new language for scalable parallel computing, including the strong competition presented by MPI and the existing Partitioned Global Address Space (PGAS) Languages. We identify some of the major features of the proposed languages, using MPI and the PGAS languages for comparison, and describe the opportunities for higher productivity along with the implementation challenges. Finally, we present the conclusions of a recent workshop in which a concrete plan for the next few years was proposed.

Download Full-text

Integrated parallel computing into a computer science curriculum

Proceedings of IEEE Frontiers in Education Conference - FIE '93 ◽

10.1109/fie.1993.405487 ◽

2002 ◽

Author(s):

M.P. Bivens

Keyword(s):

Parallel Computing ◽

Computer Science ◽

Science Curriculum ◽

Computer Science Curriculum

Download Full-text

THE CONNECTION MACHINE IN A UNIVERSITY ENVIRONMENT: GESAMTHOCHSCHULE WUPPERTAL

International Journal of Modern Physics C ◽

10.1142/s012918319300015x ◽

1993 ◽

Vol 04 (01) ◽

pp. 137-141

Author(s):

KLAUS SCHILLING

Keyword(s):

Parallel Computing ◽

Computer Science ◽

Large Scale ◽

Early History ◽

Short Account ◽

University Environment ◽

Connection Machine ◽

The University

A short account is presented on the early history, the intentions and the development of large scale parallel computing at the University of Wuppertal. It might serve as an illustration how common activities between computational and computer science can be stimulated, in the university environment.

Download Full-text

Analisis Kinerja Algoritma Quick Double Merge Sort Paralel Menggunakan openMP

Jurnal ULTIMA Computing ◽

10.31937/sk.v11i2.1294 ◽

2020 ◽

Vol 11 (2) ◽

pp. 95-102

Author(s):

I Nyoman Aditya Yudiswara ◽

Abba Suganda

Keyword(s):

Parallel Algorithms ◽

The Other ◽

Sorting Algorithm ◽

Data Parallelism ◽

Parallel Sorting ◽

Sequential Algorithms ◽

Sorting Algorithms ◽

Merge Sort ◽

Sort Algorithm ◽

Quick Sort

Processor technology currently tends to increase the number of cores more than increasing the clock speed. This development is very useful and becomes an opportunity to improve the performance of sequential algorithms that are only done by one core. This paper discusses the sorting algorithm that is executed in parallel by several logical CPUs or cores using the openMP library. This algorithm is named QDM Sort which is a combination of sequential quick sort algorithm and double merge algorithm. This study uses a data parallelism approach to design parallel algorithms from sequential algorithms. The data used in this study are the data that have not been sorted and also the data that has been sorted is integer type which is stored in advance in a file. The parameter measured to determine the performance of the QDM Sort algorithm is speedup. In a condition where a large amount of data is above 4096 and the number of threads in QDM Sort is the same as the number of logical CPUs, the QDM Sort algorithm has a better speedup compared to the other parallel sorting algorithms discussed in this study. For small amounts of data it is still better to use sequential sorting algorithm.

Download Full-text

Dagstuhl-Seminar “Dynamically and Partially Reconfigurable Architectures” (Dynamisch und partiell rekonfigurierbare Architekturen)

it - Information Technology ◽

10.1524/itit.46.4.218.36077 ◽

2004 ◽

Vol 46 (4) ◽

Author(s):

Jürgen Becker

Keyword(s):

Information Technology ◽

Parallel Computing ◽

Computer Science ◽

Mobile Communication ◽

Data Stream ◽

Electrical Engineering ◽

Reconfigurable Architectures ◽

Automotive Application ◽

Partial Reconfiguration ◽

Computing Technique

SummaryThe paper addresses people from information technology, electrical engineering, computer science, and related areas. It gives an introduction and classification to fine-, coarse-, as well as multi-grain reconfigurable architectures. This data-stream-based and transport-triggered parallel computing technique in combination with dynamical and partial reconfiguration features demonstrates promising perspectives for future CMOS-based microelectronic solutions in multimedia and infotainment, mobile communication, as well as automotive application domains, among others.

Download Full-text

Preface to special issue: lightweight and practical formal methods in the design and analysis of safety-critical systems

Mathematical Structures in Computer Science ◽

10.1017/s0960129513000145 ◽

2013 ◽

Vol 23 (4) ◽

pp. 675-675

Author(s):

AZER BESTAVROS ◽

ASSAF KFOURY

Keyword(s):

Computer Science ◽

Formal Methods ◽

Wide Spectrum ◽

Critical Systems ◽

Research Groups ◽

Special Issue ◽

Mathematical Structures ◽

Safety Critical ◽

Safety Critical Systems ◽

Final Selection

The papers included in this special issue of Mathematical Structures in Computer Science were selected from a larger set we solicited from leading research groups on both sides of the Atlantic. They cover a wide spectrum of tutorials, recent results and surveys in the area of lightweight and practical formal methods in the design and analysis of safety-critical systems. All the papers we received were submitted to a rigorous process of review and revision, based on which we made our final selection.

Download Full-text

Engineering Augmented Suffix Sorting Algorithms

10.5753/ctd.2018.3652 ◽

2018 ◽

Author(s):

Felipe A. Louza ◽

Guilherme P. Telles ◽

Simon Gog

Keyword(s):

Computer Science ◽

Full Text ◽

Suffix Array ◽

Optimal Time ◽

Time And Space ◽

Sorting Algorithms ◽

Constant Size ◽

Common Prefix ◽

Efficient Processing ◽

Burrows Wheeler Transform

Strings are prevalent in Computer Science and algorithms for their efficient processing are fundamental in various applications. The results introduced in this work contribute with theoretical improvements and practical advances in building full-text indexes. Our first contribution is an in-place algorithm that computes the Burrows-Wheeler transform and the longest common prefix (LCP) array. Our second contribution is the construction of the suffix array augmented with the LCP array in optimal time and space for strings from constant size alphabets. Our third contribution is a set of algorithms to construct full-text indexes for string collections in optimal theoretical bounds. This work is an extended abstract of the Ph.D. thesis of the first author.

Download Full-text