scholarly journals Microservice-Oriented Platform for Internet of Big Data Analytics: A Proof of Concept

Sensors ◽  
2019 ◽  
Vol 19 (5) ◽  
pp. 1134 ◽  
Author(s):  
Zheng Li ◽  
Diego Seco ◽  
Alexis Sánchez Rodríguez
Keyword(s):  
Big Data ◽  
Resource Management ◽  
Data Processing ◽  
Data Analytics ◽  
Big Data Analytics ◽  
Proof Of Concept ◽  
Tight Coupling ◽  
Specific Data ◽  
Numerous Data ◽  
Iot Devices

The ubiquitous Internet of Things (IoT) devices nowadays are generating various and numerous data from everywhere at any time. Since it is not always necessary to centralize and analyze IoT data cumulatively (e.g., the Monte Carlo analytics and Convergence analytics demonstrated in this article), the traditional implementations of big data analytics (BDA) will suffer from unnecessary and expensive data transmissions as a result of the tight coupling between computing resource management and data processing logics. Inspired by software-defined infrastructure (SDI), we propose the “microservice-oriented platform” to break the environmental monolith and further decouple data processing logics from their underlying resource management in order to facilitate BDA implementations in the IoT environment (which we name “IoBDA”). Given predesigned standard microservices with respect to specific data processing logics, the proposed platform is expected to largely reduce the complexity in and relieve inexperienced practices of IoBDA implementations. The potential contributions to the relevant communities include (1) new theories of a microservice-oriented platform on top of SDI and (2) a functional microservice-oriented platform for IoBDA with a group of predesigned microservices.

Modeling Big Data Analytics with a Real-Time Executable Specification Language

2015 ◽  
pp. 289-312
Author(s):  
Amir A. Khwaja
Keyword(s):  
Big Data ◽  
Data Processing ◽  
Real Time ◽  
Data Analytics ◽  
Big Data Analytics ◽  
Exception Handling ◽  
Specification Language ◽  
Big Data Processing ◽  
Business Decisions ◽  
Concurrent Processes

Big data explosion has already happened and the situation is only going to exacerbate with such a high number of data sources and high-end technology prevalent everywhere, generating data at a frantic pace. One of the most important aspects of big data is being able to capture, process, and analyze data as it is happening in real-time to allow real-time business decisions. Alternate approaches must be investigated especially consisting of highly parallel and real-time computations for big data processing. The chapter presents RealSpec real-time specification language that may be used for the modeling of big data analytics due to the inherent language features needed for real-time big data processing such as concurrent processes, multi-threading, resource modeling, timing constraints, and exception handling. The chapter provides an overview of RealSpec and applies the language to a detailed big data event recognition case study to demonstrate language applicability to big data framework and analytics modeling.

Big Data Analytics in Cloud Computing

2019 ◽  
pp. 325-341
Author(s):  
Rajganesh Nagarajan ◽  
Ramkumar Thirunavukarasu
Keyword(s):  
Cloud Computing ◽  
Big Data ◽  
Data Processing ◽  
Data Visualization ◽  
Data Analytics ◽  
Big Data Analytics ◽  
Future Research ◽  
Big Data Processing ◽  
Investment Cost

In this chapter, the authors consider different categories of data, which are processed by the big data analytics tools. The challenges with respect to the big data processing are identified and a solution with the help of cloud computing is highlighted. Since the emergence of cloud computing is highly advocated because of its pay-per-use concept, the data processing tools can be effectively deployed within cloud computing and certainly reduce the investment cost. In addition, this chapter talks about the big data platforms, tools, and applications with data visualization concept. Finally, the applications of data analytics are discussed for future research.

Modeling Big Data Analytics with a Real-Time Executable Specification Language

Big Data ◽  
2016 ◽  
pp. 418-440
Author(s):  
Amir A. Khwaja
Keyword(s):  
Big Data ◽  
Data Processing ◽  
Real Time ◽  
Data Analytics ◽  
Big Data Analytics ◽  
Exception Handling ◽  
Specification Language ◽  
Big Data Processing ◽  
Business Decisions ◽  
Concurrent Processes

Big data explosion has already happened and the situation is only going to exacerbate with such a high number of data sources and high-end technology prevalent everywhere, generating data at a frantic pace. One of the most important aspects of big data is being able to capture, process, and analyze data as it is happening in real-time to allow real-time business decisions. Alternate approaches must be investigated especially consisting of highly parallel and real-time computations for big data processing. The chapter presents RealSpec real-time specification language that may be used for the modeling of big data analytics due to the inherent language features needed for real-time big data processing such as concurrent processes, multi-threading, resource modeling, timing constraints, and exception handling. The chapter provides an overview of RealSpec and applies the language to a detailed big data event recognition case study to demonstrate language applicability to big data framework and analytics modeling.

scholarly journals Data Processing Model to Perform Big Data Analytics in Hybrid Infrastructures

IEEE Access ◽  
2020 ◽  
Vol 8 ◽  
pp. 170281-170294
Author(s):  
Julio C. S. Dos Anjos ◽  
Kassiano J. Matteussi ◽  
Paulo R. R. De Souza ◽  
Gabriel J. A. Grabher ◽  
Guilherme A. Borges ◽  
...  
Keyword(s):  
Big Data ◽  
Data Processing ◽  
Data Analytics ◽  
Big Data Analytics

scholarly journals Urban Planning and Smart City Decision Management Empowered by Real-Time Data Processing Using Big Data Analytics

Sensors ◽  
2018 ◽  
Vol 18 (9) ◽  
pp. 2994 ◽  
Author(s):  
Bhagya Silva ◽  
Murad Khan ◽  
Changsu Jung ◽  
Jihun Seo ◽  
Diyan Muhammad ◽  
...  
Keyword(s):  
Big Data ◽  
Data Processing ◽  
Real Time ◽  
Smart City ◽  
Data Analytics ◽  
Smart Cities ◽  
Big Data Analytics ◽  
Time Data ◽  
Real Time Data

The Internet of Things (IoT), inspired by the tremendous growth of connected heterogeneous devices, has pioneered the notion of smart city. Various components, i.e., smart transportation, smart community, smart healthcare, smart grid, etc. which are integrated within smart city architecture aims to enrich the quality of life (QoL) of urban citizens. However, real-time processing requirements and exponential data growth withhold smart city realization. Therefore, herein we propose a Big Data analytics (BDA)-embedded experimental architecture for smart cities. Two major aspects are served by the BDA-embedded smart city. Firstly, it facilitates exploitation of urban Big Data (UBD) in planning, designing, and maintaining smart cities. Secondly, it occupies BDA to manage and process voluminous UBD to enhance the quality of urban services. Three tiers of the proposed architecture are liable for data aggregation, real-time data management, and service provisioning. Moreover, offline and online data processing tasks are further expedited by integrating data normalizing and data filtering techniques to the proposed work. By analyzing authenticated datasets, we obtained the threshold values required for urban planning and city operation management. Performance metrics in terms of online and offline data processing for the proposed dual-node Hadoop cluster is obtained using aforementioned authentic datasets. Throughput and processing time analysis performed with regard to existing works guarantee the performance superiority of the proposed work. Hence, we can claim the applicability and reliability of implementing proposed BDA-embedded smart city architecture in the real world.

Big Data Overview

Big Data ◽  
2016 ◽  
pp. 1-29 ◽  
Author(s):  
Yushi Shen ◽  
Yale Li ◽  
Ling Wu ◽  
Shaofeng Liu ◽  
Qian Wen
Keyword(s):  
Big Data ◽  
Data Processing ◽  
Data Analytics ◽  
Big Data Analytics ◽  
Big Data Processing ◽  
Data Scientist ◽  
Definition Of

This chapter provides an overview of big data and its environment and opportunities. It starts with a definition of big data and describes the unique characteristics, structure, and value of big data, and the business drivers for big data analytics. It defines the role of the data scientist and describes the new ecosystem for big data processing and analysis.

IoT Big Data Architectures, Approaches, and Challenges

2021 ◽  
pp. 227-250
Author(s):  
David Sarabia-Jácome ◽  
Regel Gonzalez-Usach ◽  
Carlos E. Palau
Keyword(s):  
Big Data ◽  
Data Analytics ◽  
Fog Computing ◽  
Big Data Analytics ◽  
Late Response ◽  
Reference Architecture ◽  
Comprehensive Overview ◽  
High Bandwidth ◽  
Iot Devices ◽  
Application Developers

The internet of things (IoT) generates large amounts of data that are sent to the cloud to be stored, processed, and analyzed to extract useful information. However, the cloud-based big data analytics approach is not completely appropriate for the analysis of IoT data sources, and presents some issues and limitations, such as inherent delay, late response, and high bandwidth occupancy. Fog computing emerges as a possible solution to address these cloud limitations by extending cloud computing capabilities at the network edge (i.e., gateways, switches), close to the IoT devices. This chapter presents a comprehensive overview of IoT big data analytics architectures, approaches, and solutions. Particularly, the fog-cloud reference architecture is proposed as the best approach for performing big data analytics in IoT ecosystems. Moreover, the benefits of the fog-cloud approach are analyzed in two IoT application case studies. Finally, fog-cloud open research challenges are described, providing some guidelines to researchers and application developers to address fog-cloud limitations.

scholarly journals Big Data and Personalisation for Non-Intrusive Smart Home Automation

2021 ◽  
Vol 5 (1) ◽  
pp. 6
Author(s):  
Suriya Priya R. Asaithambi ◽  
Sitalakshmi Venkatraman ◽  
Ramanathan Venkatraman
Keyword(s):  
Big Data ◽  
Data Analytics ◽  
Smart Home ◽  
Big Data Analytics ◽  
Cost Effective ◽  
Smart Devices ◽  
Home Automation ◽  
Open Standard ◽  
Iot Devices ◽  
And Control

With the advent of the Internet of Things (IoT), many different smart home technologies are commercially available. However, the adoption of such technologies is slow as many of them are not cost-effective and focus on specific functions such as energy efficiency. Recently, IoT devices and sensors have been designed to enhance the quality of personal life by having the capability to generate continuous data streams that can be used to monitor and make inferences by the user. While smart home devices connect to the home Wi-Fi network, there are still compatibility issues between devices from different manufacturers. Smart devices get even smarter when they can communicate with and control each other. The information collected by one device can be shared with others for achieving an enhanced automation of their operations. This paper proposes a non-intrusive approach of integrating and collecting data from open standard IoT devices for personalised smart home automation using big data analytics and machine learning. We demonstrate the implementation of our proposed novel technology instantiation approach for achieving non-intrusive IoT based big data analytics with a use case of a smart home environment. We employ open-source frameworks such as Apache Spark, Apache NiFi and FB-Prophet along with popular vendor tech-stacks such as Azure and DataBricks.

scholarly journals An Approach in Big Data Analytics to Improve the Velocity of Unstructured Data Using MapReduce

2021 ◽  
Vol 10 (4) ◽  
pp. 1-25
Author(s):  
Sundarakumar M. R. ◽  
Mahadevan G. ◽  
Ramasubbareddy Somula ◽  
Sankar Sennan ◽  
Bharat S. Rawal
Keyword(s):  
Big Data ◽  
Time Delay ◽  
Data Processing ◽  
Data Analytics ◽  
Big Data Analytics ◽  
Data Retrieval ◽  
High Volume ◽  
Minimum Latency ◽  
Hadoop Clusters ◽  
Search Index

Big Data Analytics is an innovative approach for extracting the data from a huge volume of data warehouse systems. It reveals the method to compress the high volume of data into clusters by MapReduce and HDFS. However, the data processing has taken more time for extract and store in Hadoop clusters. The proposed system deals with the challenges of time delay in shuffle phase of map-reduce due to scheduling and sequencing. For improving the speed of big data, this proposed work using the Compressed Elastic Search Index (CESI) and MapReduce-Based Next Generation Sequencing Approach (MRBNGSA). This approach helps to increase the speed of data retrieval from HDFS clusters because of the way it is stored in that. this method is stored only the metadata in HDFS which takes less memory during runtime compare to big data due to the volume of data stored in HDFS. This approach is reduces the CPU utilization and memory allocation of the resource manager in Hadoop Framework and imroves data processing speed, such a way that time delay has to be reduced with minimum latency.

Sign in / Sign up

Export Citation Format

Share Document