A big Data Analytics Framework for the Integration of Heterogeneous Federated Data Centers

2018 ◽

Vol 7 (2.24) ◽

pp. 92

Author(s):

B V Ram Naresh Yadav ◽

P Anjaiah

Keyword(s):

Cloud Computing ◽

Big Data ◽

Data Analytics ◽

Data Centers ◽

Big Data Analytics ◽

Cloud Service ◽

Resource Reservation ◽

Distributed Applications ◽

It Industry ◽

The Cost

Big data analytics and Cloud computing are the two most imperative innovations in the current IT industry. In a surprise, these technologies come up together to convey the effective outcomes to various business organizations. However, big data analytics require a huge amount of resources for storage and computation. The storage cost is massively increased on the input amounts of data and requires innovative algorithms to reduce the cost to store the data in a specific data centers in a cloud. In Today’s IT Industry, Cloud Computing has emerged as a popular paradigm to host customer, enterprise data and many other distributed applications. Cloud Service Providers (CSPs) store huge amounts of data and numerous distributed applications with different cost. For example Amazon provides storage services at a fraction of TB/month and each CSP having different Service Level Agreements with different storage offers. Customers are interested in reliable SLAs and it increases the cost since the number of replicas are more. The CSPs are attracting the users for initial storage/put operations and get operations from the cloud becomes hurdle and subsequently increases the cost. CSPs provides these services by maintaining multiple datacenters at multiple locations throughout the world. These datacenters provide distinctive get/put latencies and unit costs for resource reservation and utilization. The way of choosing distinctive CSPs data centers, becomes tricky for cloud users those who are using the distributed application globally i.e. online social networks. In has mainly two challenges. Firstly, allocating the data to different datacenters to satisfy the SLO including the latency. Secondly, how one can reserve the remote resource i.e. memory with less cost. In this paper we have derived a new model to minimize the cost by satisfying the SLOs with integer programming. Additionally, we proposed an algorithm to store the data in a data center by minimizing the cost among different data centers and the computation of cost for put/get latencies. Our simulation works shows that the cost is minimized for resource reservation and utilization among different datacenters.

Download Full-text

A Novel Framework for the Seamless Integration of FPGA Accelerators with Big Data Analytics Frameworks in Heterogeneous Data Centers

2018 International Conference on High Performance Computing & Simulation (HPCS) ◽

10.1109/hpcs.2018.00090 ◽

2018 ◽

Author(s):

Ioannis Stamelos ◽

Elias Koromilas ◽

Christoforos Kachris ◽

Dimitrios Soudris

Keyword(s):

Big Data ◽

Data Analytics ◽

Data Centers ◽

Big Data Analytics ◽

Heterogeneous Data ◽

Seamless Integration

Download Full-text

QoSComm: A Data Flow Allocation Strategy among SDN-Based Data Centers for IoT Big Data Analytics

Applied Sciences ◽

10.3390/app10217586 ◽

2020 ◽

Vol 10 (21) ◽

pp. 7586

Author(s):

Jose E. Lozano-Rizk ◽

Juan I. Nieto-Hipolito ◽

Raul Rivera-Rodriguez ◽

Maria A. Cosio-Leon ◽

Mabel Vazquez-Briseño ◽

...

Keyword(s):

Communication Network ◽

Big Data ◽

Data Analytics ◽

Completion Time ◽

Data Centers ◽

Data Flow ◽

Data Transfer ◽

Big Data Analytics ◽

Optimization Method ◽

Distributed Data

When Internet of Things (IoT) big data analytics (BDA) require to transfer data streams among software defined network (SDN)-based distributed data centers, the data flow forwarding in the communication network is typically done by an SDN controller using a traditional shortest path algorithm or just considering bandwidth requirements by the applications. In BDA, this scheme could affect their performance resulting in a longer job completion time because additional metrics were not considered, such as end-to-end delay, jitter, and packet loss rate in the data transfer path. These metrics are quality of service (QoS) parameters in the communication network. This research proposes a solution called QoSComm, an SDN strategy to allocate QoS-based data flows for BDA running across distributed data centers to minimize their job completion time. QoSComm operates in two phases: (i) based on the current communication network conditions, it calculates the feasible paths for each data center using a multi-objective optimization method; (ii) it distributes the resultant paths among data centers configuring their openflow Switches (OFS) dynamically. Simulation results show that QoSComm can improve BDA job completion time by an average of 18%.

Download Full-text

Renewable Energy-Aware Big Data Analytics in Geo-Distributed Data Centers with Reinforcement Learning

IEEE Transactions on Network Science and Engineering ◽

10.1109/tnse.2018.2813333 ◽

2020 ◽

Vol 7 (1) ◽

pp. 205-215 ◽

Cited By ~ 25

Author(s):

Chenhan Xu ◽

Kun Wang ◽

Peng Li ◽

Rui Xia ◽

Song Guo ◽

...

Keyword(s):

Big Data ◽

Renewable Energy ◽

Reinforcement Learning ◽

Data Analytics ◽

Data Centers ◽

Big Data Analytics ◽

Distributed Data ◽

Energy Aware

Download Full-text

Big Data Analytics: The Next Big Thing

The Management Accountant Journal ◽

10.33516/maj.v54i5.20-24p ◽

2019 ◽

Vol 54 (5) ◽

pp. 20

Author(s):

Dheeraj Kumar Pradhan

Keyword(s):

Big Data ◽

Data Analytics ◽

Big Data Analytics

Download Full-text

Strategy in Discovery Mode - Wie Big Data & Analytics strategisches Denken verdrängen kann

WiSt - Wirtschaftswissenschaftliches Studium ◽

10.15358/0340-1650-2020-5-11 ◽

2020 ◽

Vol 49 (5) ◽

pp. 11-17

Author(s):

Thomas Wrona ◽

Pauline Reinecke

Keyword(s):

Big Data ◽

Data Analytics ◽

Big Data Analytics

Big Data & Analytics (BDA) ist zu einer kaum hinterfragten Institution für Effizienz und Wettbewerbsvorteil von Unternehmen geworden. Zu viele prominente Beispiele, wie der Erfolg von Google oder Amazon, scheinen die Bedeutung zu bestätigen, die Daten und Algorithmen zur Erlangung von langfristigen Wettbewerbsvorteilen zukommt. Sowohl die Praxis als auch die Wissenschaft scheinen geradezu euphorisch auf den „Datenzug“ aufzuspringen. Wenn Risiken thematisiert werden, dann handelt es sich meist um ethische Fragen. Dabei wird häufig übersehen, dass die diskutierten Vorteile sich primär aus einer operativen Effizienzperspektive ergeben. Strategische Wirkungen werden allenfalls in Bezug auf Geschäftsmodellinnovationen diskutiert, deren tatsächlicher Innovationsgrad noch zu beurteilen ist. Im Folgenden soll gezeigt werden, dass durch BDA zwar Wettbewerbsvorteile erzeugt werden können, dass aber hiermit auch große strategische Risiken verbunden sind, die derzeit kaum beachtet werden.

Download Full-text