A Method of Constructing Electric Power Data Warehouse Based on Cloud Computing

2012 ◽  
Vol 591-593 ◽  
pp. 1766-1769
Author(s):  
De Wen Wang ◽  
Kai Xiao
Keyword(s):  
Cloud Computing ◽  
Electric Power ◽  
Data Warehouse ◽  
Data Storage ◽  
Multidimensional Data ◽  
Data Set ◽  
Integration Architecture ◽  
Combining Data ◽  
Hadoop Cluster ◽  
Multidimensional Data Set

Hive is a data warehouse architecture in cloud computing. In order to solve the inadequate of massive data storage, query, and computing power in current electric power data warehouse, a method of electric power data warehouse based on Hive is proposed. Combining data analysis demands of electric power entreprises, the integration architecture between Hive and column-oriented storage is designed in electric power data warehouse, and the process of which is also given. At last, electric power equipment condition data is used for experiment on Hadoop cluster, results show that this method can quickly achieve query and analysis in massive multidimensional data set.

scholarly journals Clustering algorithms subjected to K-mean and gaussian mixture model on multidimensional data set

2019 ◽  
Vol 7 (2) ◽  
pp. 448 ◽  
Author(s):  
Saadaldeen Rashid Ahmed Ahmed ◽  
Israa Al Barazanchi ◽  
Zahraa A. Jaaz ◽  
Haider Rasheed Abdulshaheed
Keyword(s):  
Gaussian Mixture Model ◽  
Mixture Model ◽  
Clustering Algorithms ◽  
Gaussian Mixture ◽  
Multidimensional Data ◽  
Data Set ◽  
Multidimensional Data Set

scholarly journals Principal Graphs and Manifolds

2010 ◽  
pp. 28-59 ◽  
Author(s):  
Alexander N. Gorban ◽  
Andrei Y. Zinovyev
Keyword(s):  
Principal Component ◽  
Approximation Problem ◽  
Graph Grammar ◽  
Multidimensional Data ◽  
Data Set ◽  
Expectation Maximisation ◽  
The Family ◽  
Distance Approximation ◽  
Principal Points ◽  
Multidimensional Data Set

In many physical, statistical, biological and other investigations it is desirable to approximate a system of points by objects of lower dimension and/or complexity. For this purpose, Karl Pearson invented principal component analysis in 1901 and found ‘lines and planes of closest fit to system of points’. The famous k-means algorithm solves the approximation problem too, but by finite sets instead of lines and planes. This chapter gives a brief practical introduction into the methods of construction of general principal objects (i.e., objects embedded in the ‘middle’ of the multidimensional data set). As a basis, the unifying framework of mean squared distance approximation of finite datasets is selected. Principal graphs and manifolds are constructed as generalisations of principal components and k-means principal points. For this purpose, the family of expectation/maximisation algorithms with nearest generalisations is presented. Construction of principal graphs with controlled complexity is based on the graph grammar approach.

scholarly journals 3D Visualization of Sound Fields Perceived by an Acoustic Camera

2015 ◽  
Vol 15 (7) ◽  
pp. 45-57
Author(s):  
Nevena Popova ◽  
Georgi Shishkov ◽  
Petia Koprinkova-Hristova ◽  
Kiril Alexiev
Keyword(s):  
3D Visualization ◽  
Multidimensional Data ◽  
Sound Waves ◽  
Signal Processing Algorithm ◽  
Data Set ◽  
3 Dimensional ◽  
Sound Fields ◽  
Neuro Fuzzy ◽  
Real Time Visualization ◽  
Multidimensional Data Set

Abstract The paper summarizes the application results of a recently proposed neuro-fuzzy algorithm for multi-dimensional data clustering to 3-Dimensional (3D) visualization of dynamically perceived sound waves recorded by an acoustic camera. The main focus in the present work is on the developed signal processing algorithm adapted to the specificity of multidimensional data set recorded by the acoustic camera, as well as on the created software package for real-time visualization of the “observed” sound waves propagation.

scholarly journals Features of recording of meteorological conditions in the data warehouse of qualification examination of plant varieties

2021 ◽  
Vol 17 (3) ◽  
pp. 254-261
Author(s):  
S. I. Melnyk ◽  
N. V. Leschuk ◽  
N. S. Orlenko ◽  
E. M. Starychenko ◽  
K. M. Mazhuha ◽  
...  
Keyword(s):  
Data Warehouse ◽  
Data Storage ◽  
Field Experiments ◽  
Data Entry ◽  
Weather Conditions ◽  
Meteorological Conditions ◽  
Multidimensional Data ◽  
Phenological Stages ◽  
Agricultural Business ◽  
Meteorological Observations

Purpose. To develop a multidimensional model of the data storage for the qualification examination of plant varieties for fixing meteorological conditions in conjunction with the phenological stages of development of varieties that undergo DUS and SVD expertise. Methods. To conduct research with the establishment of the main structural ele­ments of a multidimensional data warehouse, methods of induction, deduction, analysis and synthesis were used. In the design process of the storage facility, W. H. Inmon’s concept was applied, adapted for the agricultural and agricultural business. Results. The stages of qualification examination of plant varieties were analyzed and methodolo­gical approaches to the creation of a multidimensional data warehouse model were considered. The features of the use of data storages for storing the results of qualification exa­mination of plant varieties for distinctness, uniformity and stability (DUS) and suitability of a variety for dissemination in Ukraine (SVD) were highlighted. Particular attention was paid to the implementation of the interconnection between the results of the qualification examination of plant varie­ties with the data of meteorological observations at various phenological stages of plant growth and development, according to the records in the electronic field journal. The logical data model of the data warehouse was designed and implemented in the MS SQL Server environment. Conclusions. Sources of data entry into data warehouses were determined and a multidimensional data warehouse model was implemented according to the “snowflake” scheme. The diagram of the data warehouse was presented, which provided a link between the meteorological conditions of the field experiments and the initial data of the qualification examination, and had four tables of measurements. For each dimension table and fact table, an attribute composition of the data was defined. The data warehouse was practically used to analyze the influence of weather conditions on the indicators of DUS and SVD examinations.

scholarly journals Video-based Person Re-identification Based on Distributed Cloud Computing

2021 ◽  
Vol 1 (2) ◽  
Author(s):  
Chengyan Zhong ◽  
Xiaoyu Jiang ◽  
Guanqiu Qi
Keyword(s):  
Cloud Computing ◽  
Data Storage ◽  
Large Scale ◽  
Distributed Model ◽  
Data Sets ◽  
Data Set ◽  
Research Issues ◽  
Distributed Data Storage ◽  
Operation Speed ◽  
Distributed Cloud

Person re-identification(Re-ID) has been a hot research issues in the field of computer vision. In recent years, with the maturity of the theory, a large number of excellent methods have been proposed. However,Large scale data sets and huge networks make training a time-consuming process. At the same time, the parameters and their values generated during the training process also take up a lot of computer resources.Therefore, we apply distributed cloud computing method to perform person re-identification task. Using distributed data storage method, pedestrian data sets and parameters are stored in cloud nodes.In order to speed up operational efficiency and increase fault tolerance, we add data redundancy mechanism to copy and store data blocks to different nodes,and proposed a hash loop optimization algorithm to optimize the data distribution process. Moreover,we assign different layers of the Re-ID network to different nodes to complete the training in the way of model parallelism. By comparing and analyzing the accuracy and operation speed of the distributed model on the video-based data set MARS, the results show that our distributed model has a faster training speed.

scholarly journals TrustedDW: A New Framework to Securely Hosting Data Warehouse in the Cloud

10.29007/q3wd ◽  
2019 ◽  
Author(s):  
Kawthar Karkouda ◽  
Ahlem Nabli ◽  
Faiez Gargouri
Keyword(s):  
Cloud Computing ◽  
Data Warehouse ◽  
Data Storage ◽  
Homomorphic Encryption ◽  
New Technology ◽  
Sensitive Data ◽  
It Industry ◽  
Security Issues ◽  
Outsourced Data ◽  
Storage Network

Nowadays cloud computing become the most popular technology in the area of IT industry. It provides computing power, storage, network and software as a service. While building, a data warehouse typically necessitates an important initial investment. With the cloud pay-as-you-go model, BI system can benefit from this new technology. But, as every new technology, cloud computing brings its own risks in term of security. Because some security issues are inherited from classical architectures, some traditional security solutions are used to protect outsourced data. Unfortunately, those solutions are not enough and cannot guarantee the privacy of sensitive data hosted in the Cloud. In particular, in the case of data warehouse, using traditional encryption solutions cannot be practical because those solutions induce a heavy overhead in terms of data storage and query performance. So, a suitable schema must be proposed in order to balance the security and the performance of data warehouse hosted in the cloud. In this paper, we propose (TrustedDW) a homomorphic encryption schema for securing and querying a data warehouse hosted in the cloud.

Sign in / Sign up

Export Citation Format

Share Document