IoT-enabled directed acyclic graph in spark cluster

Abstract Real-time data streaming fetches live sensory segments of the dataset in the heterogeneous distributed computing environment. This process assembles data chunks at a rapid encapsulation rate through a streaming technique that bundles sensor segments into multiple micro-batches and extracts into a repository, respectively. Recently, the acquisition process is enhanced with an additional feature of exchanging IoT devices’ dataset comprised of two components: (i) sensory data and (ii) metadata. The body of sensory data includes record information, and the metadata part consists of logs, heterogeneous events, and routing path tables to transmit micro-batch streams into the repository. Real-time acquisition procedure uses the Directed Acyclic Graph (DAG) to extract live query outcomes from in-place micro-batches through MapReduce stages and returns a result set. However, few bottlenecks affect the performance during the execution process, such as (i) homogeneous micro-batches formation only, (ii) complexity of dataset diversification, (iii) heterogeneous data tuples processing, and (iv) linear DAG workflow only. As a result, it produces huge processing latency and the additional cost of extracting event-enabled IoT datasets. Thus, the Spark cluster that processes Resilient Distributed Dataset (RDD) in a fast-pace using Random access memory (RAM) defies expected robustness in processing IoT streams in the distributed computing environment. This paper presents an IoT-enabled Directed Acyclic Graph (I-DAG) technique that labels micro-batches at the stage of building a stream event and arranges stream elements with event labels. In the next step, heterogeneous stream events are processed through the I-DAG workflow, which has non-linear DAG operation for extracting queries’ results in a Spark cluster. The performance evaluation shows that I-DAG resolves homogeneous IoT-enabled stream event issues and provides an effective stream event heterogeneous solution for IoT-enabled datasets in spark clusters.

Download Full-text

A real-time heterogeneous distributed computing environment for multi-robot system

Proceedings Third IEEE International Symposium on Object-Oriented Real-Time Distributed Computing (ISORC 2000) (Cat. No. PR00607) ◽

10.1109/isorc.2000.839554 ◽

2002 ◽

Author(s):

K. Qutubudin ◽

D. Pease

Keyword(s):

Distributed Computing ◽

Real Time ◽

Computing Environment ◽

Robot System ◽

Heterogeneous Distributed Computing ◽

Multi Robot

Download Full-text

Monitoring distributed computing beyond the traditional time-series histogram

EPJ Web of Conferences ◽

10.1051/epjconf/202024503036 ◽

2020 ◽

Vol 245 ◽

pp. 03036

Author(s):

M S Doidge ◽

P. A. Love ◽

J Thornton

Keyword(s):

Time Series ◽

Distributed Computing ◽

Real Time ◽

A Priori ◽

Time Data ◽

Use Of Time ◽

Computing Services ◽

Monitoring Tools ◽

Novel Approach ◽

Current Monitoring

In this work we describe a novel approach to monitor the operation of distributed computing services. Current monitoring tools are dominated by the use of time-series histograms showing the evolution of various metrics. These can quickly overwhelm or confuse the viewer due to the large number of similar looking graphs. We propose a supplementary approach through the sonification of real-time data streamed directly from a variety of distributed computing services. The real-time nature of this method allows operations staff to quickly detect problems and identify that a problem is still ongoing, avoiding the case of investigating an issue a-priori when it may already have been resolved. In this paper we present details of the system architecture and provide a recipe for deployment suitable for both site and experiment teams.

Download Full-text

IoT-Based Health Services Framework for Endless Ailment Administration at Remote Areas

Advances in Medical Technologies and Clinical Practice - Incorporating the Internet of Things in Healthcare Applications and Wearable Devices ◽

10.4018/978-1-7998-1090-2.ch011 ◽

2020 ◽

pp. 184-198

Author(s):

Rajkumar Rajaseskaran ◽

Mridual Bhasin ◽

K. Govinda ◽

Jolly Masih ◽

Sruthi M.

Keyword(s):

Health Services ◽

Real Time ◽

Patient Monitoring ◽

The Body ◽

Smart Device ◽

Time Data ◽

Remote Areas ◽

Real Time Data

The objective is to build an IoT-based patient monitoring smart device. The device would monitor real-time data of patients and send it to the Cloud. It has become imperative to attend to minute internal changes in the body that affect overall health. The system would remotely take care of an individual's changes in health and notify the relatives or doctors of any abnormal changes. Cloud storages provide easy availability and monitoring of real-time data. The system uses microcontroller Arduino Nano and sensors – GY80, Heartbeat sensor, Flex sensor, and Galvanic Skin (GSR) sensor with a Wi-Fi Module.

Download Full-text

Reliability Driven Soft Real-Time Fuzzy Task Scheduling in Distributed Computing Environment

Advances in Intelligent and Soft Computing - Proceedings of the International Conference on Soft Computing for Problem Solving (SocProS 2011) December 20-22, 2011 ◽

10.1007/978-81-322-0487-9_22 ◽

2012 ◽

pp. 219-226

Author(s):

P. K. Yadav ◽

K. Bhatia ◽

Sagar Gulati

Keyword(s):

Distributed Computing ◽

Real Time ◽

Task Scheduling ◽

Computing Environment

Download Full-text

Real-Time Data Processing of the Sensory Data of a Multi- Fingered Dextrous Robot Hand

The Springer International Series in Engineering and Computer Science - Real-Time Systems Engineering and Applications ◽

10.1007/978-0-585-32314-5_13 ◽

2007 ◽

pp. 343-367

Author(s):

Alois Knoll

Keyword(s):

Data Processing ◽

Real Time ◽

Robot Hand ◽

Time Data ◽

Sensory Data ◽

Real Time Data ◽

Real Time Data Processing

Download Full-text

IoT-Based Health Services Framework for Endless Ailment Administration at Remote Areas

Research Anthology on Telemedicine Efficacy, Adoption, and Impact on Healthcare Delivery ◽

10.4018/978-1-7998-8052-3.ch021 ◽

2021 ◽

pp. 412-428

Author(s):

Rajkumar Rajaseskaran ◽

Mridual Bhasin ◽

K. Govinda ◽

Jolly Masih ◽

Sruthi M.

Keyword(s):

Health Services ◽

Real Time ◽

Patient Monitoring ◽

The Body ◽

Smart Device ◽

Time Data ◽

Remote Areas ◽

Real Time Data

Download Full-text

QoS and security aware allocation of Directed Acyclic Graph on heterogeneous distributed real-time systems

2012 International Conference on Computing, Networking and Communications (ICNC) ◽

10.1109/iccnc.2012.6167475 ◽

2012 ◽

Author(s):

Naeem Al-Oudat ◽

Manimaran Govindarasu

Keyword(s):

Real Time ◽

Directed Acyclic Graph ◽

Real Time Systems ◽

Acyclic Graph ◽

Time Systems

Download Full-text

Whitelists Based Multiple Filtering Techniques in SCADA Sensor Networks

Journal of Applied Mathematics ◽

10.1155/2014/597697 ◽

2014 ◽

Vol 2014 ◽

pp. 1-7 ◽

Cited By ~ 8

Author(s):

DongHo Kang ◽

ByoungKoo Kim ◽

JungChan Na ◽

KyoungSon Jhang

Keyword(s):

Sensor Networks ◽

Internet Of Things ◽

Data Acquisition ◽

Real Time ◽

Supervisory Control ◽

Cyber Attacks ◽

Computing Environment ◽

Time Data ◽

Real Time Data ◽

Filtering Technique

Internet of Things (IoT) consists of several tiny devices connected together to form a collaborative computing environment. Recently IoT technologies begin to merge with supervisory control and data acquisition (SCADA) sensor networks to more efficiently gather and analyze real-time data from sensors in industrial environments. But SCADA sensor networks are becoming more and more vulnerable to cyber-attacks due to increased connectivity. To safely adopt IoT technologies in the SCADA environments, it is important to improve the security of SCADA sensor networks. In this paper we propose a multiple filtering technique based on whitelists to detect illegitimate packets. Our proposed system detects the traffic of network and application protocol attacks with a set of whitelists collected from normal traffic.

Download Full-text