A Novel Approach in Improving I/O Performance of Small Meteorological Files on HDFS

With the cloud computing is becoming mature, many of its characteristics for meteorological data processing is extremely important. Since HDFS is designed for reading and writing large files, it’s difficult to be taken advantage of small meteorological files. In this paper, an improved approach on HDFS is proposed for small meteorological files, small files are to be merged, indexed, and blocks are compressed, the pressure of memory on master node occupied by metadata is relieved, the speed of reading and writing small files is increased, read speed is increased by 50%, and write speed is up to 3-4 times of the original, saving about 2/3 of storage space and computing performance has also been improved. Thus, meteorological data processing can make use of cloud computing platform more closely.

Download Full-text

Cloud Data Management Based on Cloud Computing Technology

Applied Mechanics and Materials ◽

10.4028/www.scientific.net/amm.543-547.3573 ◽

2014 ◽

Vol 543-547 ◽

pp. 3573-3576

Author(s):

Yuan Jun Zou

Keyword(s):

Cloud Computing ◽

Parallel Computing ◽

Data Processing ◽

Development Planning ◽

Computer Data ◽

Cloud Data ◽

Computing Platform ◽

Convergence Error ◽

Cloud Computing Platform ◽

Cloud Data Processing

Cloud computing, networking and other high-end computer data processing technology are the important contents of eleven-five development planning in China. They have developed rapidly in recent years in the field of engineering. In this paper, we combine parallel computing with the collaborative simulation principle, design a cloud computing platform, establish the mathematical model of cloud data processing and parallel computing algorithm, and verify the applicability of algorithm through the numerical simulation. Through numerical calculation, cloud computing platform can be divided into complex grids, and the transmission speed is fast, which is eight times than the finite difference method. The mesh is meticulous, which reaches millions. Convergence error is minimum, only 0.001. The calculation accuracy is up to 98.36%.

Download Full-text

The Design of Cloud Computing Platform for Massive Data Processing of Distributed Photovoltaic Power

Proceedings of the 2015 International Conference on Mechatronics, Electronic, Industrial and Control Engineering ◽

10.2991/meic-15.2015.161 ◽

2015 ◽

Author(s):

Bing Jiang ◽

Kun Huang

Keyword(s):

Cloud Computing ◽

Data Processing ◽

Massive Data ◽

Computing Platform ◽

Photovoltaic Power ◽

Cloud Computing Platform ◽

Massive Data Processing

Download Full-text

Optimization and Management in Order to Drive Targeted Networking Memory Database

Applied Mechanics and Materials ◽

10.4028/www.scientific.net/amm.727-728.965 ◽

2015 ◽

Vol 727-728 ◽

pp. 965-968

Author(s):

Ji Liu

Keyword(s):

Cloud Computing ◽

Wireless Communication ◽

Data Processing ◽

Real Time ◽

System Architecture ◽

Large Data ◽

Processing Efficiency ◽

Computing Platform ◽

Cloud Computing Platform ◽

Vehicle Networking

In today's vehicle networking system architecture is mainly composed of four parts: sensor networks, wireless communication networks, cloud computing platforms and vehicle terminal. Wireless sensor network is responsible for the front of the real-time collection of traffic information, a wireless communication network to send information to the backend of the cloud computing platform, cloud computing platform to handle a large number of vehicles to collect real-time information from the front, and finally sends the information to the end user. In this thesis, this car networking research background, analyze vehicle networking system architecture consisting of performance indicators for each part of the system recognize cloud platform for large data processing efficiency as well as room for improvement. Then put forward the traditional computing platform I / O disk database with in-memory database to replace the cloud to enhance cloud computing platform for large data processing efficiency.

Download Full-text

Data Processing Platform of Cloud Computing and Its Performance Analysis Based on Photoelectric Hybrid Interconnection Architecture

Journal of Nanoelectronics and Optoelectronics ◽

10.1166/jno.2020.2805 ◽

2020 ◽

Vol 15 (6) ◽

pp. 743-752

Author(s):

Zhimin Ren

Keyword(s):

Cloud Computing ◽

Topology Optimization ◽

Data Processing ◽

Optical Interconnection ◽

Circuit Switching ◽

Computing Platform ◽

Optical Circuit ◽

Optical Circuit Switching ◽

Cloud Computing Platform ◽

Processing Platform

Data processing platform is the core support platform of cloud computing. The use of electric interconnection architecture will increase the complexity of network topology, while optical interconnection architecture is ideal, so cloud computing platform based on optical interconnection has become a research hotspot. The distributed optical interconnection architecture of cloud computing data processing platform is focused on. Combining the hybrid mechanism of optical circuit switching and electric packet switching, it can meet a variety of traffic requirements. Meanwhile, it improves the switching mechanism, communication strategy, and router structure. Moreover, considering that the hybrid optoelectronic interconnection architecture can improve the network delay and throughput, but there is still a problem of network consumption. Combined with the network characteristics of the data processing platform (wireless mesh structure) of cloud computing, the network topology algorithm is studied, and the relationship between the topology and the maximum number of allocable channels is analyzed. Furthermore, the equation of topological reliability calculation is defined, and the optimization model of topological design is proposed, according to which the data processing platform of cloud computing is further optimized under the photoelectric hybrid interconnection architecture. During the experiment, before topology optimization, by changing the message length, it is found that adding optical circuit switching can help achieving large capacity and new type of transmission, and can effectively reduce the time delay. After topology optimization structure is adopted, the photoelectric hybrid-data processing platform of cloud computing without topology optimization is compared. It is found that under different reliability constraints, the throughput and end-to-end delay of the network are significantly improved, which proves that the data processing platform of cloud computing based on the photoelectric hybrid interconnection architecture is a feasible cloud computing platform.

Download Full-text