A Data Localization Algorithm for Distributing Column Storage System of Big Data

Distributing column storage is one of the techniques to improve the efficiency of big data access under the cloud computing environment. To achieving the aim and reducing network data access frequency, paper established a data localization strategy and designed a multi-thread algorithm. Firstly, segmentalize data in the horizontal direction, and then divide vertically the data table into data column, and ensure that the same level column data localize on the same node in the cluster. Secondly, the essay designed and realized the data localization algorithm under Hadoop distributed cloud computing framework. Finally, experiments show remarkable reduces in the network access with the usage of data localization algorithm, and improvement of the data access efficiency.