Towards a New Model of Storage and Access to Data in Big Data and Cloud Computing

The technological revolution integrating multiple information sources and extension of computer science in different sectors led to the explosion of the data quantities, which reflects the scaling of vo-lumes, numbers and types. These massive increases have resulted in the development of new location techniques and access to data. The final steps in this evolution have emerged new technologies: Cloud and Big Data. The reference implementation of the Clouds and Big Data storage is incontestably the Hadoop Distributed File System (HDFS). This latter is based on the separation of metadata to data that consists in the centralization and isolation of the metadata of storage servers. In this paper, the authors propose an approach to improve the service metadata for Hadoop to maintain consistency without much compromising performance and scalability of metadata by suggesting a mixed solution between centralization and distribution of metadata to enhance the performance and scalability of the model.

Download Full-text

Analysis and Experimental Study of HDFS Performance

TEM Journal ◽

10.18421/tem102-38 ◽

2021 ◽

pp. 806-814

Author(s):

Yordan Kalmukov ◽

Milko Marinov ◽

Tsvetelina Mladenova ◽

Irena Valova

Keyword(s):

Experimental Study ◽

Big Data ◽

Computer System ◽

Data Storage ◽

Fault Tolerant ◽

Processing System ◽

Daily Basis ◽

Distributed File System ◽

Hadoop Distributed File System ◽

Big Data Storage

In the age of big data, the amount of data that people generate and use on a daily basis has far exceeded the storage and processing capabilities of a single computer system. That motivates the use of distributed big data storage and processing system such as Hadoop. It provides a reliable, horizontallyscalable, fault-tolerant and efficient service, based on the Hadoop Distributed File System (HDFS) and MapReduce. The purpose of this research is to experimentally determine whether (and to what extent) the network communication speed, the file replication factor, the files’ sizes and their number, and the location of the HDFS client influence the performance of the HDFS read/write operations.

Download Full-text

Crowd Sourcing-based Deduplication in Big Data Environment

International Journal of Recent Technology and Engineering - 2 ◽

10.35940/ijrte.d8201.118419 ◽

2019 ◽

Vol 8 (4) ◽

pp. 2329-2333

Keyword(s):

Big Data ◽

Data Storage ◽

Storage Space ◽

Crowd Sourcing ◽

Data Deduplication ◽

Fixed Size ◽

Hadoop Distributed File System ◽

Data Environment ◽

Mapreduce Model ◽

Big Data Storage

Frequently, in reality, substances have at least two portrayals in databases. Copy records don't share a typical key as well as they contain mistakes that make copy coordinating a troublesome assignment. Mistakes are presented as the consequence of interpretation blunders, inadequate data, absence of standard configurations, or any mix of these components. In big data storage data is excessively enormous and productively store data is troublesome errand. To take care of this issue Hadoop instrument gives HDFS that oversees data by keep up duplication of data however this expanded duplication. In our anticipated strategy bigdata stream is given to the fixed size chunking calculation to make fixed size chunks. In this manuscript, we introduce an exhaustive investigation of the writing on crowd sourcing based big data deduplication technique. In our strategy is to create the guide diminish result after that MapReduce model is connected to discover whether hash esteems and are copy or not. To be familiar with the copy hash esteems MapReduce model contrasted these hash esteems and as of now put away hash esteems in Big data storage space. On the off chance that these hash esteems are now there in the Big data storage space, at that point these can be distinguished as copy. On the off chance that the hash esteems are copied, at that point don't store the data into the Hadoop Distributed File System (HDFS) else then store the data into the HDFS. we additionally spread various deduplication systems in crowd sourcing data's.

Download Full-text