scholarly journals On real-time and non real-time distributed computing

Author(s):  
G. Le Lann
Author(s):  
Jahwan Koo ◽  
Nawab Muhammad Faseeh Qureshi ◽  
Isma Farah Siddiqui ◽  
Asad Abbas ◽  
Ali Kashif Bashir

Abstract Real-time data streaming fetches live sensory segments of the dataset in the heterogeneous distributed computing environment. This process assembles data chunks at a rapid encapsulation rate through a streaming technique that bundles sensor segments into multiple micro-batches and extracts into a repository, respectively. Recently, the acquisition process is enhanced with an additional feature of exchanging IoT devices’ dataset comprised of two components: (i) sensory data and (ii) metadata. The body of sensory data includes record information, and the metadata part consists of logs, heterogeneous events, and routing path tables to transmit micro-batch streams into the repository. Real-time acquisition procedure uses the Directed Acyclic Graph (DAG) to extract live query outcomes from in-place micro-batches through MapReduce stages and returns a result set. However, few bottlenecks affect the performance during the execution process, such as (i) homogeneous micro-batches formation only, (ii) complexity of dataset diversification, (iii) heterogeneous data tuples processing, and (iv) linear DAG workflow only. As a result, it produces huge processing latency and the additional cost of extracting event-enabled IoT datasets. Thus, the Spark cluster that processes Resilient Distributed Dataset (RDD) in a fast-pace using Random access memory (RAM) defies expected robustness in processing IoT streams in the distributed computing environment. This paper presents an IoT-enabled Directed Acyclic Graph (I-DAG) technique that labels micro-batches at the stage of building a stream event and arranges stream elements with event labels. In the next step, heterogeneous stream events are processed through the I-DAG workflow, which has non-linear DAG operation for extracting queries’ results in a Spark cluster. The performance evaluation shows that I-DAG resolves homogeneous IoT-enabled stream event issues and provides an effective stream event heterogeneous solution for IoT-enabled datasets in spark clusters.


2020 ◽  
Vol 245 ◽  
pp. 03036
Author(s):  
M S Doidge ◽  
P. A. Love ◽  
J Thornton

In this work we describe a novel approach to monitor the operation of distributed computing services. Current monitoring tools are dominated by the use of time-series histograms showing the evolution of various metrics. These can quickly overwhelm or confuse the viewer due to the large number of similar looking graphs. We propose a supplementary approach through the sonification of real-time data streamed directly from a variety of distributed computing services. The real-time nature of this method allows operations staff to quickly detect problems and identify that a problem is still ongoing, avoiding the case of investigating an issue a-priori when it may already have been resolved. In this paper we present details of the system architecture and provide a recipe for deployment suitable for both site and experiment teams.


2014 ◽  
Vol 7 (2) ◽  
Author(s):  
Theo Kanter ◽  
Rahim Rahmani ◽  
Jamie Walters ◽  
Willmar Sauter

This article investigates new forms for creating and enabling massive and scalable participatory immersive experiences in live cultural events, characterized by processes, involving pervasive objects, places and people. The multi-disciplinary research outlines a new paradigm for collaborative creation and participation towards technological and social innovation, tapping into crowd-sensing. The approach promotes user-driven content-creation and offsets economic models thereby rewarding creators and performers. In response to these challenges, we propose a framework for bringing about massive and real-time presence and awareness on the Internet through an Internet-of-Things infrastructure to connect artifacts, performers, participants and places. Equally importantly, we enable the in-situ creation of collaborative experiences building on relevant existing and stored content, based on decisions leveraging multi-criteria clustering and proximity of pervasive information, objects, people and places. Finally, we investigate some new ways for immersive experiences via distributed computing but pointing forward to the necessity to do more with regard to collaborative creation.


2015 ◽  
Vol 35 (2) ◽  
pp. 0210001
Author(s):  
于潇宇 Yu Xiaoyu ◽  
郭玉波 Guo Yubo ◽  
陈刚 Chen Gang ◽  
叶东 Ye Dong

Symmetry ◽  
2019 ◽  
Vol 11 (7) ◽  
pp. 911 ◽  
Author(s):  
Md Azher Uddin ◽  
Aftab Alam ◽  
Nguyen Anh Tu ◽  
Md Siyamul Islam ◽  
Young-Koo Lee

In recent years, the amount of intelligent CCTV cameras installed in public places for surveillance has increased enormously and as a result, a large amount of video data is produced every moment. Due to this situation, there is an increasing request for the distributed processing of large-scale video data. In an intelligent video analytics platform, a submitted unstructured video undergoes through several multidisciplinary algorithms with the aim of extracting insights and making them searchable and understandable for both human and machine. Video analytics have applications ranging from surveillance to video content management. In this context, various industrial and scholarly solutions exist. However, most of the existing solutions rely on a traditional client/server framework to perform face and object recognition while lacking the support for more complex application scenarios. Furthermore, these frameworks are rarely handled in a scalable manner using distributed computing. Besides, existing works do not provide any support for low-level distributed video processing APIs (Application Programming Interfaces). They also failed to address a complete service-oriented ecosystem to meet the growing demands of consumers, researchers and developers. In order to overcome these issues, in this paper, we propose a distributed video analytics framework for intelligent video surveillance known as SIAT. The proposed framework is able to process both the real-time video streams and batch video analytics. Each real-time stream also corresponds to batch processing data. Hence, this work correlates with the symmetry concept. Furthermore, we introduce a distributed video processing library on top of Spark. SIAT exploits state-of-the-art distributed computing technologies with the aim to ensure scalability, effectiveness and fault-tolerance. Lastly, we implant and evaluate our proposed framework with the goal to authenticate our claims.


Sign in / Sign up

Export Citation Format

Share Document