A Stream Data Processing Framework for Location-Based Service Using NoSQL Technology

2015 ◽  
Vol 763 ◽  
pp. 159-163 ◽  
Author(s):  
Nan Ju Kim ◽  
Eui In Choi
Keyword(s):  
Big Data ◽  
Data Processing ◽  
Streaming Data ◽  
Location Based Services ◽  
Location Based Service ◽  
Stream Data ◽  
Data Processing Method ◽  
Specific Device ◽  
Internet Development ◽  
Processing Framework

One of the most exciting changes in Location-Based Services is the incredible growth of internet, development of wearable devices, and advanced positioning technologies. In addition, the big data from those sources helps performing seamless LBS as a technology. The existing processing methods used to detect the location of a particular tag, or specific device are not enough for complex processing while collecting all of the streaming data at the same time using a variety of wireless communication system [10,11,12,13]. We can use big data processing method for processing all the streaming data in real time. In this paper, we propose a framework for improving performance of Seamless LBS using NoSQL technology.

scholarly journals Review of Big Data and Processing Frameworks for Disaster Response Applications

2019 ◽  
Vol 8 (9) ◽  
pp. 387 ◽  
Author(s):  
Silvino Pedro Cumbane ◽  
Gyozo Gidófalvi
Keyword(s):  
Machine Learning ◽  
Big Data ◽  
Data Processing ◽  
Disaster Management ◽  
Disaster Response ◽  
Human Life ◽  
Streaming Data ◽  
Data Sources ◽  
Big Data Processing ◽  
Processing Framework

Natural hazards result in devastating losses in human life, environmental assets and personal, and regional and national economies. The availability of different big data such as satellite imageries, Global Positioning System (GPS) traces, mobile Call Detail Records (CDRs), social media posts, etc., in conjunction with advances in data analytic techniques (e.g., data mining and big data processing, machine learning and deep learning) can facilitate the extraction of geospatial information that is critical for rapid and effective disaster response. However, disaster response systems development usually requires the integration of data from different sources (streaming data sources and data sources at rest) with different characteristics and types, which consequently have different processing needs. Deciding which processing framework to use for a specific big data to perform a given task is usually a challenge for researchers from the disaster management field. Therefore, this paper contributes in four aspects. Firstly, potential big data sources are described and characterized. Secondly, the big data processing frameworks are characterized and grouped based on the sources of data they handle. Then, a short description of each big data processing framework is provided and a comparison of processing frameworks in each group is carried out considering the main aspects such as computing cluster architecture, data flow, data processing model, fault-tolerance, scalability, latency, back-pressure mechanism, programming languages, and support for machine learning libraries, which are related to specific processing needs. Finally, a link between big data and processing frameworks is established, based on the processing provisioning for essential tasks in the response phase of disaster management.

scholarly journals The real-time big data processing method based on LSTM or GRU for the smart job shop production process

2020 ◽  
Vol 14 ◽  
pp. 174830262096239 ◽  
Author(s):  
Chuang Wang ◽  
Wenbo Du ◽  
Zhixiang Zhu ◽  
Zhifeng Yue
Keyword(s):  
Big Data ◽  
Data Processing ◽  
Production Process ◽  
Real Time ◽  
Job Shop ◽  
Production Data ◽  
Time Data ◽  
Big Data Processing ◽  
The Real ◽  
Data Processing Method

With the wide application of intelligent sensors and internet of things (IoT) in the smart job shop, a large number of real-time production data is collected. Accurate analysis of the collected data can help producers to make effective decisions. Compared with the traditional data processing methods, artificial intelligence, as the main big data analysis method, is more and more applied to the manufacturing industry. However, the ability of different AI models to process real-time data of smart job shop production is also different. Based on this, a real-time big data processing method for the job shop production process based on Long Short-Term Memory (LSTM) and Gate Recurrent Unit (GRU) is proposed. This method uses the historical production data extracted by the IoT job shop as the original data set, and after data preprocessing, uses the LSTM and GRU model to train and predict the real-time data of the job shop. Through the description and implementation of the model, it is compared with KNN, DT and traditional neural network model. The results show that in the real-time big data processing of production process, the performance of the LSTM and GRU models is superior to the traditional neural network, K nearest neighbor (KNN), decision tree (DT). When the performance is similar to LSTM, the training time of GRU is much lower than LSTM model.

A Study of Big Data Processing for Sentiments Analysis

2022 ◽  
pp. 1162-1191
Author(s):  
Dinesh Chander ◽  
Hari Singh ◽  
Abhinav Kirti Gupta
Keyword(s):  
Big Data ◽  
Data Processing ◽  
Data Analytics ◽  
Important Application ◽  
Streaming Data ◽  
Test Case ◽  
Big Data Processing ◽  
Important Field ◽  
Batch Data ◽  
Different Sources

Data processing has become an important field in today's big data-dominated world. The data has been generating at a tremendous pace from different sources. There has been a change in the nature of data from batch-data to streaming-data, and consequently, data processing methodologies have also changed. Traditional SQL is no longer capable of dealing with this big data. This chapter describes the nature of data and various tools, techniques, and technologies to handle this big data. The chapter also describes the need of shifting big data on to cloud and the challenges in big data processing in the cloud, the migration from data processing to data analytics, tools used in data analytics, and the issues and challenges in data processing and analytics. Then the chapter touches an important application area of streaming data, sentiment analysis, and tries to explore it through some test case demonstrations and results.

scholarly journals Big data processing framework for manufacturing

Procedia CIRP ◽  
2019 ◽  
Vol 83 ◽  
pp. 661-664 ◽  
Author(s):  
Yinghao Ye ◽  
Meilin Wang ◽  
Shuhong Yao ◽  
Jarvis N. Jiang ◽  
Qing Liu
Keyword(s):  
Big Data ◽  
Data Processing ◽  
Big Data Processing ◽  
Processing Framework

scholarly journals Distributed stream data processing system in multi-agent safety system of infrastructure objects

2019 ◽  
pp. 324-331
Author(s):  
S S Valeev ◽  
N V Kondratyeva ◽  
A S Kovtunenko ◽  
M A Timirov ◽  
R R Karimov
Keyword(s):  
Data Processing ◽  
Processing System ◽  
Streaming Data ◽  
Stream Data ◽  
Data Processing System ◽  
Distributed Computing Systems ◽  
Multi Agent ◽  
Stream Data Processing ◽  
Safety Systems ◽  
The Mathematical Model

The solution of the problem of resource management in distributed computing systems of processing stream data in safety systems of distributed objects is considered. The tasks of streaming data processing in a multi-level multi-agent evacuation system in an infrastructure object are considered. The features of the mathematical model of a distributed stream data processing system are discussed.

Use and validation of location-based services in urban research: An example with Dutch restaurants

Urban Studies ◽  
2018 ◽  
Vol 56 (5) ◽  
pp. 868-884 ◽  
Author(s):  
Daniel Arribas-Bel ◽  
Jessie Bakens
Keyword(s):  
Big Data ◽  
The Netherlands ◽  
Detailed Analysis ◽  
Urban Geography ◽  
Location Based Services ◽  
Urban Landscapes ◽  
Location Based Service ◽  
Economic Data ◽  
Urban Research ◽  
Urban Amenities

This article focuses on the use of big data for urban geography research. We collect data from the location-based service Foursquare in The Netherlands and employ it to obtain a rich catalogue of restaurant locations and other urban amenities, as well as a measure of their popularity among users. Because the Foursquare data can be combined with traditional sources of socio-economic data obtained from Statistics Netherlands, we can quantify, document and characterise some of the biases inherent in these new sources of data in the context of urban applications. A detailed analysis is given as to when this type of big data is useful and when it is misleading. Although the users of Foursquare are not representative of the whole population, we argue that this inherent bias can be exploited for research about the attractiveness of urban landscapes and consumer amenities in addition to the more traditional data on urban amenities.

Sign in / Sign up

Export Citation Format

Share Document