Building a Data Pipeline for the Management and Processing of Urban Data Streams

2018 ◽  
pp. 379-395
Author(s):  
Elarbi Badidi ◽  
Nouf El Neyadi ◽  
Meera Al Saeedi ◽  
Fatima Al Kaabi ◽  
Muthucumaru Maheswaran
Keyword(s):  
Data Streams ◽  
Data Pipeline ◽  
Urban Data

scholarly journals Industrial Big Data Pipeline for Wind Turbine PHM in a Large Manufacturing Facility

2019 ◽  
Vol 10 (1) ◽  
Author(s):  
Kevin Leahy ◽  
Colm Gallagher ◽  
Peter O’Donovan ◽  
Dominic T.J. O’Sullivan
Keyword(s):  
Data Streams ◽  
Health Management ◽  
Data Driven ◽  
Energy Data ◽  
Maintenance Strategies ◽  
Manufacturing Facility ◽  
Data Pipeline ◽  
Industrial Big Data ◽  
Operations And Maintenance ◽  
The Cost

Wind turbines generate a wealth of data which can be effectively used to improve maintenance strategies and drive down operations and maintenance ( O & M ) costs, which account for 20-25 % of the cost of generation of wind energy. Data-driven techniques for enabling prognostic and health management ( PHM ) technologies have seen many successes in the space. However, managing this data, particularly in the context of anindustrial facility which may have many other data streams, is a challenge. This technical brief describes the schematic of a proposed system for managing turbine data, ahead of an implementation which will see PHM techniques applied to it. The turbine in this case is attached to a manufacturing facility, so the pipeline is designed to be modular and integrate well with an existing pipeline at that facility.

Automated discovery and integration of semantic urban data streams: The ACEIS middleware

2017 ◽  
Vol 76 ◽  
pp. 561-581 ◽  
Author(s):  
Feng Gao ◽  
Muhammad Intizar Ali ◽  
Edward Curry ◽  
Alessandra Mileo
Keyword(s):  
Data Streams ◽  
Automated Discovery ◽  
Discovery And Integration ◽  
Urban Data

Improved Macro-clusters generation using Top-k shared Micro-clusters in Data Streams

2017 ◽  
Vol 7 (10) ◽  
pp. 52
Author(s):  
LAKSHMI PRANEETHA
Keyword(s):  
Real Time ◽  
Data Streams ◽  
Bloom Filter ◽  
Scientific Applications ◽  
Pruning Algorithm ◽  
Density Data ◽  
Data Points ◽  
Short Time ◽  
Information Streams

Now-a-days data streams or information streams are gigantic and quick changing. The usage of information streams can fluctuate from basic logical, scientific applications to vital business and money related ones. The useful information is abstracted from the stream and represented in the form of micro-clusters in the online phase. In offline phase micro-clusters are merged to form the macro clusters. DBSTREAM technique captures the density between micro-clusters by means of a shared density graph in the online phase. The density data in this graph is then used in reclustering for improving the formation of clusters but DBSTREAM takes more time in handling the corrupted data points In this paper an early pruning algorithm is used before pre-processing of information and a bloom filter is used for recognizing the corrupted information. Our experiments on real time datasets shows that using this approach improves the efficiency of macro-clusters by 90% and increases the generation of more number of micro-clusters within in a short time.

Sign in / Sign up

Export Citation Format

Share Document