Data Optimization using Apache Flink

Map Reduce, Flink, and Spark, also become more popular in the processing of big data lately. Flink will be an open platform Big Data processing system for Apache-powered batch storage and streaming of data. Flink's query optimizer is constructed for historical information processing (batch) based on parallel storage systems approaches. Flink query query optimizer interprets the questions into jobs of different tasks that are regularly sent. Therefore, taking advantage of task similarities should prevent redundant computation. In this article, the multi-demand optimization model for Flink, Flink was planned and designed on Flink Software Stack's top priority. It's thought-about as an associate in Apache Flink's nursing add-on to maximize multi-demand information sharing. The Flink system takes advantage of option operators ' information sharing resources to reduce overlap and duplication of multi-query in-network information movement. Research findings show that the leveraging of shared option operations in vast information on multiple requests would offer promising time to perform queries. Therefore, in the stream phase, Without doubt the Flink approach can be used to boost application performance over time periods.

Online Monitoring System for Domestic Distribution Box

A new method for monitoring and control of domestic distribution box is proposed and developed for automated recovery of power continuity during interruption. The system automatically test each of the sockets to determine the source of the failure and isolate them. The data of the modified connection will be sent to the client through a server, wirelessly to notify the user the modifications made. Parallel processing via multi-threading in the server are used to increment the upper limit of TCP transmission's throughput. Multiple SQLite database are used by multiple threads for parallel storage of data to increase performance.

Parallel Storage Devices Profiling with SeRRa

This work presents the parallel storage device profiling tool SeRRa. Our tool obtains the sequential to random throughput ratio for reads and writes of different sizes on storage devices. In order to provide this information efficiently, SeRRa employs benchmarks to obtain the values for only a subset of the parameter space and estimates the remaining values through linear models. The MPI parallelization of SeRRa presented in this paper allows for faster profiling. Our results show that our parallel SeRRa provides profiles up to 8:7 times faster than the sequential implementation, up to 895 times faster than the originally required time (without SeRRa).