scholarly journals Interactive Graph Stream Analytics in Arkouda

Algorithms ◽  
2021 ◽  
Vol 14 (8) ◽  
pp. 221
Author(s):  
Zhihui Du ◽  
Oliver Alvarado Rodriguez ◽  
Joseph Patchett ◽  
David A. Bader
Keyword(s):  
High Performance ◽  
Graph Algorithm ◽  
Stream Data ◽  
Approximation Solution ◽  
Solution Generation ◽  
Triangle Counting ◽  
Graph Streams ◽  
Double Index ◽  
Stream Analytics ◽  
Stream Model

Data from emerging applications, such as cybersecurity and social networking, can be abstracted as graphs whose edges are updated sequentially in the form of a stream. The challenging problem of interactive graph stream analytics is the quick response of the queries on terabyte and beyond graph stream data from end users. In this paper, a succinct and efficient double index data structure is designed to build the sketch of a graph stream to meet general queries. A single pass stream model, which includes general sketch building, distributed sketch based analysis algorithms and regression based approximation solution generation, is developed, and a typical graph algorithm—triangle counting—is implemented to evaluate the proposed method. Experimental results on power law and normal distribution graph streams show that our method can generate accurate results (mean relative error less than 4%) with a high performance. All our methods and code have been implemented in an open source framework, Arkouda, and are available from our GitHub repository, Bader-Research. This work provides the large and rapidly growing Python community with a powerful way to handle terabyte and beyond graph stream data using their laptops.

scholarly journals Performance Assay of Big IoT Data Analytics Framework

2019 ◽  
Vol 8 (4) ◽  
pp. 8593-8596
Keyword(s):  
Internet Of Things ◽  
Data Processing ◽  
High Velocity ◽  
Data Analytics ◽  
Real Life ◽  
Global Environment ◽  
Stream Data ◽  
Processing Level ◽  
Stream Analytics ◽  
Apache Storm

Evaluation of Internet of Things (IoT) technologies in real life has scaled the enumeration of data in huge volumes and that too with high velocity, and thus a new issue has come into picture that is of management & analytics of this BIG IOT STREAM data. In order to optimize the performance of the IoT Machines and services provided by the vendors, industry is giving high priority to analyze this big IoT Stream Data for surviving in the competitive global environment. Thses analysis are done through number of applications using various Data Analytics Framework, which require obtaining the valuable information intelligently from a large amount of real-time produced data. This paper, discusses the challenges and issues faced by distributed stream analytics frameworks at the data processing level and tries to recommend a possible a Scalable Framework to adapt with the volume and velocity of Big IoT Stream Data. Experiments focus on evaluating the performance of three Distributed Stream Analytics Here Analytics frameworks, namely Apache Spark, Splunk and Apache Storm are being evaluated over large steam IoT data on latency & throughput as parameters in respect to concurrency. The outcome of the paper is to find the best possible existing framework and recommend a possible scalable framework.

High Performance Exact Triangle Counting on GPUs

2017 ◽  
Vol 28 (12) ◽  
pp. 3501-3510 ◽  
Author(s):  
Mauro Bisson ◽  
Massimiliano Fatica
Keyword(s):  
High Performance ◽  
Triangle Counting ◽  
Exact Triangle

scholarly journals Triangle Counting in Dynamic Graph Streams

Algorithmica ◽  
2015 ◽  
Vol 76 (1) ◽  
pp. 259-278 ◽  
Author(s):  
Laurent Bulteau ◽  
Vincent Froese ◽  
Konstantin Kutzkov ◽  
Rasmus Pagh
Keyword(s):  
Dynamic Graph ◽  
Triangle Counting ◽  
Graph Streams

Temporal locality-aware sampling for accurate triangle counting in real graph streams

2020 ◽  
Vol 29 (6) ◽  
pp. 1501-1525
Author(s):  
Dongjin Lee ◽  
Kijung Shin ◽  
Christos Faloutsos
Keyword(s):  
Triangle Counting ◽  
Graph Streams ◽  
Temporal Locality

scholarly journals Fast, Accurate and Provable Triangle Counting in Fully Dynamic Graph Streams

2020 ◽  
Vol 14 (2) ◽  
pp. 1-39 ◽  
Author(s):  
Kijung Shin ◽  
Sejoon Oh ◽  
Jisu Kim ◽  
Bryan Hooi ◽  
Christos Faloutsos
Keyword(s):  
Dynamic Graph ◽  
Triangle Counting ◽  
Graph Streams

scholarly journals Developing a prototype of high-performance graph-processing framework for NEC SX–Aurora TSUBASA vector architecture

2020 ◽  
pp. 290-305
Author(s):  
И.В. Афанасьев
Keyword(s):  
Graph Algorithms ◽  
High Performance ◽  
Graph Algorithm ◽  
Efficient Implementation ◽  
Graph Processing ◽  
Irregular Structure ◽  
Vector Systems ◽  
Order Of Magnitude ◽  
Vector Graph ◽  
Processing Framework

В данной статье описан подход к созданию прототипа графового фреймворка VGL (Vector Graph Library), нацеленного на эффективную реализацию графовых алгоритмов для современной векторной архитектуры NEC SX–Aurora TSUBASA. Современные векторные системы позволяют значительно ускорять приложения, интенсивно использующие подсистему памяти, подклассом которых являются графовые алгоритмы. Однако подходы к эффективной реализации графовых алгоритмов для векторных систем на сегодняшний день исследованы крайне слабо: вследствие сильно нерегулярной структуры графов реального мира, эффективно задействовать векторные особенности целевых платформ затруднительно. В работе показано, что разработанные на основе предложенного фреймворка VGL реализации графовых алгоритмов не уступают в производительности оптимизированным “вручную” аналогам за счет инкапсуляции большого числа оптимизаций графовых алгоритмов, характерных для векторных систем. Вместе с этим предложенный фреймворк позволяет значительно упростить процесс разработки графовых алгоритмов для векторных систем, на порядок сокращая объем кода реализуемых алгоритмов и скрывая от пользователя особенности программирования систем данного класса. This article describes a prototype of graph-processing framework VGL (Vector Graph Library), aimed at the efficient implementation of graph algorithms for the modern NEC SX–Aurora TSUBASA vector architecture. Present day vector systems can significantly speed up various memory-intensive applications, including graph algorithms. However, approaches to the efficient implementation of graph algorithms for vector systems have been studied extremely poorly as of today: due to the highly irregular structure of real-world graphs, it is difficult to effectively use vector features of target platforms. This paper shows that the implementations of graph algorithms developed on the basis of the proposed VGL framework show the performance comparable to their manually optimized versions due to the encapsulation of a large number of graph algorithm optimizations typical for vector systems. At the same time, the proposed framework makes it possible to significantly simplify the process of developing graph algorithms for vector systems, by an order of magnitude reducing the amount of code for implemented algorithms and hiding the programming features of systems of this class from the user.

scholarly journals Resource Partitioning and Application Scheduling with Module Merging on Dynamically and Partially Reconfigurable FPGAs

Electronics ◽  
2020 ◽  
Vol 9 (9) ◽  
pp. 1461 ◽  
Author(s):  
Zhe Wang ◽  
Qi Tang ◽  
Biao Guo ◽  
Ji-Bo Wei ◽  
Ling Wang
Keyword(s):  
High Performance ◽  
System Model ◽  
Mixed Integer ◽  
Task Execution ◽  
Processing Efficiency ◽  
Actual Application ◽  
Solution Generation ◽  
Speed Up ◽  
Application Requirements ◽  
Performance Computing

Dynamically partially reconfigurable (DPR) technology based on FPGA is applied extensively in the field of high-performance computing (HPC) because of its advantages in processing efficiency and power consumption. To make full use of the advantages of DPR in execution efficiency, we build a DPR system model that meets to the actual application requirements and the objective constraints. According to the consistency of reconfiguration order and dependencies, we propose two algorithms based on simulated annealing (SA). The algorithms partition FPGA resource to several regions and schedule tasks to the regions. In order to improve the performance of the algorithms, we exploit the module merging technology to improve the parallelism of task execution and design a new solution generation method to speed up the convergence speed. Experimental results show that the proposed algorithms have a lower time complexity than mixed-integer linear programming (MILP), iterative scheduler (IS) and Ant Colony Optimization (ACO). For applications with more tasks, the proposed algorithms show performance advantages in producing better partitioning and scheduling results in a shorter time.

Sign in / Sign up

Export Citation Format

Share Document