scholarly journals A Methodology for Heterogeneous Sensor Data Organization and Near Real-Time Data Sharing by Adopting OGC SWE Standards

2019 ◽  
Vol 8 (4) ◽  
pp. 167 ◽  
Author(s):  
Bartolomeo Ventura ◽  
Andrea Vianello ◽  
Daniel Frisinghelli ◽  
Mattia Rossi ◽  
Roberto Monsorno ◽  
...  
Keyword(s):  
Real Time ◽  
Sensor Data ◽  
Weather Data ◽  
Time Data ◽  
Heterogeneous Sensors ◽  
Heterogeneous Datasets ◽  
Real Time Data ◽  
Sensor Registration ◽  
Set Up ◽  
Minimum Interaction

Finding a solution to collect, analyze, and share, in near real-time, data acquired by heterogeneous sensors, such as traffic, air pollution, soil moisture, or weather data, represents a great challenge. This paper describes the solution developed at Eurac Research to automatically upload data, in near real-time, by adopting Open Geospatial Consortium (OGC) Sensor Web Enablement (SWE) standards to guarantee interoperability. We set up a methodology capable of ingesting heterogeneous datasets to automatize observation uploading and sensor registration, with minimum interaction required of the user. This solution has been successfully tested and applied in the Long Term (Socio-)Ecological Research (LT(S)ER) Matsch-Mazia initiative, and the code is accessible under the CC BY 4.0 license.

scholarly journals Fossel: Efficient Latency Reduction in Approximating Streaming Sensor Data

2020 ◽  
Vol 12 (23) ◽  
pp. 10175
Author(s):  
Fatima Abdullah ◽  
Limei Peng ◽  
Byungchul Tak
Keyword(s):  
Real Time ◽  
Data Analytics ◽  
Streaming Data ◽  
Sensor Data ◽  
Simple Type ◽  
Data Sampling ◽  
Time Data ◽  
Latency Reduction ◽  
Network Delays ◽  
Real Time Data

The volume of streaming sensor data from various environmental sensors continues to increase rapidly due to wider deployments of IoT devices at much greater scales than ever before. This, in turn, causes massive increase in the fog, cloud network traffic which leads to heavily delayed network operations. In streaming data analytics, the ability to obtain real time data insight is crucial for computational sustainability for many IoT enabled applications such as environmental monitors, pollution and climate surveillance, traffic control or even E-commerce applications. However, such network delays prevent us from achieving high quality real-time data analytics of environmental information. In order to address this challenge, we propose the Fog Sampling Node Selector (Fossel) technique that can significantly reduce the IoT network and processing delays by algorithmically selecting an optimal subset of fog nodes to perform the sensor data sampling. In addition, our technique performs a simple type of query executions within the fog nodes in order to further reduce the network delays by processing the data near the data producing devices. Our extensive evaluations show that Fossel technique outperforms the state-of-the-art in terms of latency reduction as well as in bandwidth consumption, network usage and energy consumption.

scholarly journals A Framework for Real Time Processing of Sensor Data in the Cloud

2015 ◽  
Vol 2015 ◽  
pp. 1-11 ◽  
Author(s):  
Supun Kamburugamuve ◽  
Leif Christiansen ◽  
Geoffrey Fox
Keyword(s):  
Data Processing ◽  
Real Time ◽  
Cloud Services ◽  
Sensor Data ◽  
Smart Devices ◽  
Time Data ◽  
Real Time Processing ◽  
Real Time Data ◽  
Autonomous Robot Navigation ◽  
Real Time Data Processing

We describe IoTCloud, a platform to connect smart devices to cloud services for real time data processing and control. A device connected to IoTCloud can communicate with real time data analysis frameworks deployed in the cloud via messaging. The platform design is scalable in connecting devices as well as transferring and processing data. With IoTCloud, a user can develop real time data processing algorithms in an abstract framework without concern for the underlying details of how the data is distributed and transferred. For this platform, we primarily consider real time robotics applications such as autonomous robot navigation, where there are strict requirements on processing latency and demand for scalable processing. To demonstrate the effectiveness of the system, a robotic application is developed on top of the framework. The system and the robotics application characteristics are measured to show that data processing in central servers is feasible for real time sensor applications.

scholarly journals HYBRIDJOIN for Near-Real-Time Data Warehousing

2011 ◽  
Vol 7 (4) ◽  
pp. 21-42 ◽  
Author(s):  
M. Asif Naeem ◽  
Gillian Dobbie ◽  
Gerald Weber
Keyword(s):  
Real Time ◽  
Data Stream ◽  
Time Integration ◽  
Synthetic Data ◽  
Performance Measurements ◽  
Time Data ◽  
Join Algorithm ◽  
Real Time Data ◽  
Set Up ◽  
Nested Loop

An important component of near-real-time data warehouses is the near-real-time integration layer. One important element in near-real-time data integration is the join of a continuous input data stream with a disk-based relation. For high-throughput streams, stream-based algorithms, such as Mesh Join (MESHJOIN), can be used. However, in MESHJOIN the performance of the algorithm is inversely proportional to the size of disk-based relation. The Index Nested Loop Join (INLJ) can be set up so that it processes stream input, and can deal with intermittences in the update stream but it has low throughput. This paper introduces a robust stream-based join algorithm called Hybrid Join (HYBRIDJOIN), which combines the two approaches. A theoretical result shows that HYBRIDJOIN is asymptotically as fast as the fastest of both algorithms. The authors present performance measurements of the implementation. In experiments using synthetic data based on a Zipfian distribution, HYBRIDJOIN performs significantly better for typical parameters of the Zipfian distribution, and in general performs in accordance with the theoretical model while the other two algorithms are unacceptably slow under different settings.

scholarly journals Scheduling in Sensor Grid Middleware for Telemedicine Using ABC Algorithm

2014 ◽  
Vol 2014 ◽  
pp. 1-7 ◽  
Author(s):  
T. Vigneswari ◽  
M. A. Maluk Mohamed
Keyword(s):  
Real Time ◽  
Microelectromechanical Systems ◽  
Scheduling Algorithm ◽  
Vital Signs ◽  
Base Station ◽  
Sensor Nodes ◽  
Sensor Data ◽  
Time Data ◽  
Grid Middleware ◽  
Real Time Data

Advances in microelectromechanical systems (MEMS) and nanotechnology have enabled design of low power wireless sensor nodes capable of sensing different vital signs in our body. These nodes can communicate with each other to aggregate data and transmit vital parameters to a base station (BS). The data collected in the base station can be used to monitor health in real time. The patient wearing sensors may be mobile leading to aggregation of data from different BS for processing. Processing real time data is compute-intensive and telemedicine facilities may not have appropriate hardware to process the real time data effectively. To overcome this, sensor grid has been proposed in literature wherein sensor data is integrated to the grid for processing. This work proposes a scheduling algorithm to efficiently process telemedicine data in the grid. The proposed algorithm uses the popular swarm intelligence algorithm for scheduling to overcome the NP complete problem of grid scheduling. Results compared with other heuristic scheduling algorithms show the effectiveness of the proposed algorithm.

scholarly journals Improving Model Predictions—Integration of Real-Time Sensor Data into a Running Simulation of an Agent-Based Model

2021 ◽  
Vol 13 (13) ◽  
pp. 7000
Author(s):  
Ulfia A. Lenfers ◽  
Nima Ahmady-Moghaddam ◽  
Daniel Glake ◽  
Florian Ocker ◽  
Daniel Osterholz ◽  
...  
Keyword(s):  
Real Time ◽  
Real World ◽  
City Planning ◽  
Sensor Data ◽  
City Management ◽  
Proof Of Concept ◽  
Agent Based Model ◽  
Time Data ◽  
Agent Based ◽  
Real Time Data

The current trend towards living in big cities contributes to an increased demand for efficient and sustainable space and resource allocation in urban environments. This leads to enormous pressure for resource minimization in city planning. One pillar of efficient city management is a smart intermodal traffic system. Planning and organizing the various kinds of modes of transport in a complex and dynamically adaptive system such as a city is inherently challenging. By deliberately simplifying reality, models can help decision-makers shape the traffic systems of tomorrow. Meanwhile, Smart City initiatives are investing in sensors to observe and manage many kinds of urban resources, making up a part of the Internet of Things (IoT) that produces massive amounts of data relevant for urban planning and monitoring. We use these new data sources of smart cities by integrating real-time data of IoT sensors in an ongoing simulation. In this sense, the model is a digital twin of its real-world counterpart, being augmented with real-world data. To our knowledge, this is a novel instance of real-time correction during simulation of an agent-based model. The process of creating a valid mapping between model components and real-world objects posed several challenges and offered valuable insights, particularly when studying the interaction between humans and their environment. As a proof-of-concept for our implementation, we designed a showcase with bike rental stations in Hamburg-Harburg, a southern district of Hamburg, Germany. Our objective was to investigate the concept of real-time data correction in agent-based modeling, which we consider to hold great potential for improving the predictive capabilities of models. In particular, we hope that the chosen proof-of-concept informs the ongoing politically supported trends in mobility—away from individual and private transport and towards—in Hamburg.

scholarly journals HYBRIDJOIN for Near-Real-Time Data Warehousing

2013 ◽  
pp. 280-302
Author(s):  
M. Asif Naeem ◽  
Gillian Dobbie ◽  
Gerald Weber
Keyword(s):  
Real Time ◽  
Input Data ◽  
Time Integration ◽  
Synthetic Data ◽  
Performance Measurements ◽  
Time Data ◽  
Join Algorithm ◽  
Real Time Data ◽  
Set Up ◽  
Nested Loop

An important component of near-real-time data warehouses is the near-real-time integration layer. One important element in near-real-time data integration is the join of a continuous input data stream with a disk-based relation. For high-throughput streams, stream-based algorithms, such as Mesh Join (MESHJOIN), can be used. However, in MESHJOIN the performance of the algorithm is inversely proportional to the size of disk-based relation. The Index Nested Loop Join (INLJ) can be set up so that it processes stream input, and can deal with intermittences in the update stream but it has low throughput. This paper introduces a robust stream-based join algorithm called Hybrid Join (HYBRIDJOIN), which combines the two approaches. A theoretical result shows that HYBRIDJOIN is asymptotically as fast as the fastest of both algorithms. The authors present performance measurements of the implementation. In experiments using synthetic data based on a Zipfian distribution, HYBRIDJOIN performs significantly better for typical parameters of the Zipfian distribution, and in general performs in accordance with the theoretical model while the other two algorithms are unacceptably slow under different settings.

scholarly journals Machine Learning based Improved Gaussian Mixture Model for IoT Real-Time Data Analysis

2020 ◽  
Vol 16 (1) ◽  
Author(s):  
Sivadi Sivadi ◽  
Moorthy Moorthy ◽  
Vijender Solanki
Keyword(s):  
Machine Learning ◽  
Data Analysis ◽  
Real Time ◽  
Gaussian Mixture Model ◽  
Gaussian Mixture ◽  
Sensor Data ◽  
Cloud Platform ◽  
Time Data ◽  
Huge Amount ◽  
Real Time Data

Introduction: The article is the product of the research “Due to the increase in popularity of Internet of Things (IoT), a huge amount of sensor data is being generated from various smart city applications”, developed at Pondicherry University in the year 2019. Problem:To acquire and analyze the huge amount of sensor-generated data effectively is a significant problem when processing the data. Objective:  To propose a novel framework for IoT sensor data analysis using machine learning based improved Gaussian Mixture Model (GMM) by acquired real-time data.  Methodology:In this paper, the clustering based GMM models are used to find the density patterns on a daily or weekly basis for user requirements. The ThingSpeak cloud platform used for performing analysis and visualizations. Results:An analysis has been performed on the proposed mechanism implemented on real-time traffic data with Accuracy, Precision, Recall, and F-Score as measures. Conclusions:The results indicate that the proposed mechanism is efficient when compared with the state-of-the-art schemes. Originality:Applying GMM and ThingSpeak Cloud platform to perform analysis on IoT real-time data is the first approach to find traffic density patterns on busy roads. Restrictions:There is a need to develop the application for mobile users to find the optimal traffic routes based on density patterns. The authors could not concentrate on the security aspect for finding density patterns.

scholarly journals Real-time data assimilation potential to connect micro-smart water test bed and hydraulic model

2019 ◽  
Vol 2 (1) ◽  
pp. 71-82 ◽  
Author(s):  
Jiada Li ◽  
Shuangli Bao ◽  
Steven Burian
Keyword(s):  
Real Time ◽  
Hydraulic Model ◽  
Test Bed ◽  
Time Data ◽  
Hydraulic Simulation ◽  
Smart Water ◽  
Sensing Technology ◽  
Real Time Data ◽  
Water Test ◽  
Set Up

Abstract Recently, smart water application has gained worldwide attention, but there is a lack of understanding of how to construct smart water networks. This is partly because of the limited investigation into how to combine physical experiments with model simulations. This study aimed to investigate the process of connecting micro-smart water test bed (MWTB) and a ‘two-loop’ EPANET hydraulic model, which involves experimental set-up, real-time data acquisition, hydraulic simulation, and system performance demonstration. In this study, a MWTB was established based on the flow sensing technology. The data generated by the MWTB were stored in Observations Data Model (ODM) database for visualization in RStudio environment and also archived as the input of EPANET hydraulic simulation. The data visualization fitted the operation scenarios of the MWTB well. Additionally, the fitting degree between the experimental measurements and modeling outputs indicates the ‘two-loop’ EPANET model can represent the operation of MWTB for better understanding of hydraulic analysis.

Sign in / Sign up

Export Citation Format

Share Document