An HBase-Based Platform for Massive Power Data Storage in Power System

2014 ◽  
Vol 1070-1072 ◽  
pp. 739-744
Author(s):  
Zi Jian Yan ◽  
Peng Sun ◽  
Xiao Mei Liu
Keyword(s):  
Data Storage ◽  
Rapid Development ◽  
Database System ◽  
Massive Data ◽  
Huge Data ◽  
Sample Data ◽  
Design Software ◽  
Power Application ◽  
Data Design ◽  
Big Data Technology

With the rapid development of the grid scale, there are huge data generated in grid, which include many sample data, alarm data used for real-time monitor and analysis, the traditional dispatching system can’t meet the demands of big capacity storage very well. In recent years, big data technology develop very fast, among it the distributed column-oriented database system is rising gradually with the vigorous development of cloud computing. Considering the character of power data, this paper studies how to use distributed column-oriented database system in power application for storing massive increasing data. The paper design massive data storage structure of power data, design software frame and deployment of massive power data storage. Through experiment it is feasible for distributed column-oriented database to be applied in EMS system through experiment.

scholarly journals The Construction and Thinking of Cloud Computing and Big Data Technology in Smart Campus

2019 ◽  
Vol 3 (2) ◽  
pp. 152
Author(s):  
Xianglan Wu
Keyword(s):  
Cloud Computing ◽  
Big Data ◽  
Data Storage ◽  
Information Technologies ◽  
Rapid Development ◽  
Smart Campus ◽  
Huge Data ◽  
New Information ◽  
Big Data Technology

<p>In today's society, the rise of the Internet and rapid development make every day produce a huge amount of data. Therefore, the traditional data processing mode and data storage can not be fully analyzed and mined these data. More and more new information technologies (such as cloud computing, virtualization and big data, etc.) have emerged and been applied, the network has turned from informationization to intelligence, and campus construction has ushered in the stage of smart campus construction.The construction of intelligent campus refers to big data and cloud computing technology, which improves the informatization service quality of colleges and universities by integrating, storing and mining huge data.</p>

scholarly journals Big Data on Machine Learning – A Review

2021 ◽  
Vol 8 (3) ◽  
Author(s):  
Balasree K ◽  
Dharmarajan K
Keyword(s):  
Machine Learning ◽  
Big Data ◽  
Data Storage ◽  
Data Analytics ◽  
Rapid Development ◽  
Learning Algorithms ◽  
Big Data Analytics ◽  
Machine Learning Algorithms ◽  
Data Sets ◽  
Big Data Technology

In rapid development of Big Data technology over the recent years, this paper discussing about the Machine Learning (ML) playing role that is based on methods and algorithms to Big Data Processing and Big Data Analytics. In evolutionary fields and computing fields of developments that both are complementing each other. Big Data: The rapid growth of such data solutions needed to be studied and provided to handle then to gain the knowledge from datasets and extracting values due to the data sets are very high in velocity and variety. The Big data analytics are involving and indicating the appropriate data storage and computational outline that enhanced by using Scalable Machine Learning Algorithms and Big Data Analytics then the analytics to reveal the massive amounts of hidden data’s and secret correlations. This type of Analytic information useful for organizations and companies to gain deeper knowledge, development and getting advantages over the competition. When using this Analytics we can predict the accurate implementation over the data. This paper presented about the detailed review of state-of-the-art developments and overview of advantages and challenges in Machine Learning Algorithms over big data analytics.

scholarly journals Massive Data Management and Sharing Module for Connectome Reconstruction

2020 ◽  
Vol 10 (5) ◽  
pp. 314
Author(s):  
Jingbin Yuan ◽  
Jing Zhang ◽  
Lijun Shen ◽  
Dandan Zhang ◽  
Wenhuan Yu ◽  
...  
Keyword(s):  
Data Management ◽  
Data Storage ◽  
Large Scale ◽  
Rapid Development ◽  
Massive Data ◽  
Storage And Retrieval ◽  
Server Side ◽  
Large Scale Data ◽  
Client Side ◽  
Scale Data

Recently, with the rapid development of electron microscopy (EM) technology and the increasing demand of neuron circuit reconstruction, the scale of reconstruction data grows significantly. This brings many challenges, one of which is how to effectively manage large-scale data so that researchers can mine valuable information. For this purpose, we developed a data management module equipped with two parts, a storage and retrieval module on the server-side and an image cache module on the client-side. On the server-side, Hadoop and HBase are introduced to resolve massive data storage and retrieval. The pyramid model is adopted to store electron microscope images, which represent multiresolution data of the image. A block storage method is proposed to store volume segmentation results. We design a spatial location-based retrieval method for fast obtaining images and segments by layers rapidly, which achieves a constant time complexity. On the client-side, a three-level image cache module is designed to reduce latency when acquiring data. Through theoretical analysis and practical tests, our tool shows excellent real-time performance when handling large-scale data. Additionally, the server-side can be used as a backend of other similar software or a public database to manage shared datasets, showing strong scalability.

scholarly journals Improvement and Analysis of Semantic Similarity Algorithm Based on Linguistic Concept Structure

Complexity ◽  
2021 ◽  
Vol 2021 ◽  
pp. 1-11
Author(s):  
Shan Xiao ◽  
Cheng Di ◽  
Pei Li
Keyword(s):  
Semantic Similarity ◽  
Data Storage ◽  
Impact Analysis ◽  
Rapid Development ◽  
Conceptual Structure ◽  
Information Modeling ◽  
Data Resource ◽  
Huge Data ◽  
Starting Point ◽  
Similarity Algorithm

With the rapid development of information age, various social groups and corresponding institutions are producing a large amount of information data every day. For such huge data storage and identification, in order to manage such data more efficiently and reasonably, traditional semantic similarity algorithm emerges. However, the accuracy of the traditional semantic similarity algorithm is relatively low, and the convergence of corresponding algorithm is poor. Based on this problem, this paper starts with the conceptual structure of language, analyzes the depth of language structure and the distance between nodes, and analyzes the two levels as the starting point. For the information of a specific data resource description frame type, the weight of interconnected edges is used for impact analysis so as to realize the semantic similarity impact analysis of all information data. Based on the above improvements, this paper also systematically establishes the data information modeling process based on language conceptual structure and establishes the corresponding model. In the experimental part, the improved algorithm is simulated and analyzed. The simulation results show that compared with the traditional algorithm, the algorithm has obvious accuracy improvement.

scholarly journals Smart Meter Evaluation System Based on Big Data

2021 ◽  
Vol 2143 (1) ◽  
pp. 012039
Author(s):  
Yu Zhou ◽  
Xuesong Shao ◽  
Zhuowen Mu ◽  
Qixin Cai ◽  
Yue Li ◽  
...  
Keyword(s):  
Big Data ◽  
Data Storage ◽  
Evaluation System ◽  
Clustering Algorithm ◽  
Massive Data ◽  
Smart Meter ◽  
Smart Meters ◽  
Multiple Systems ◽  
Material Resources ◽  
Big Data Technology

Abstract Aiming at the problem of unstable operation of smart meters, this paper proposes a smart meter evaluation system based on big data. First introduce big data related technologies, such as data storage, analysis, mining, etc.. Secondly, design the big data smart meter evaluation system model. Finally, use big data technology and clustering algorithm to realize the design of the big data smart meter operation evaluation system. The system can convert massive data from multiple systems into operational evaluation reports, which helps to reduce the waste of human and material resources.

scholarly journals A review: evolution of big data in developing country

2019 ◽  
Vol 3 (1) ◽  
pp. 30-37
Author(s):  
Bayu Prasetyo ◽  
Faiz Syaikhoni Aziz ◽  
Kamil Faqih ◽  
Wahyu Primadi ◽  
Roni Herdianto ◽  
...  
Keyword(s):  
Developing Countries ◽  
Big Data ◽  
Developing Country ◽  
Data Storage ◽  
Human Life ◽  
Large Data ◽  
Big Data Technology

The development of technology from year to year is increasingly rapid and diverse. All systems that exist in human life began to be designed with technology that requires large data storage. Big Data technology began to be developed to accommodate very large data volumes, rapid data changes, and very varied. Developing countries are starting to use Big Data a lot in developing their systems, such as healthcare, agriculture, building, transportation, and various other fields. In this paper, it explains the development of Big Data applied to the sectors previously mentioned in developing countries and also the challenges faced by developing countries in the process of developing their systems.

Research of Android Platform Application Based on HTML5

2014 ◽  
Vol 651-653 ◽  
pp. 1901-1904
Author(s):  
Xian Wei Li
Keyword(s):  
Data Storage ◽  
Computer Technology ◽  
High Efficiency ◽  
Rapid Development ◽  
Management Information ◽  
Development Environment ◽  
Android Platform ◽  
User Data ◽  
Realization Process ◽  
Application Module

In the society with the rapid development and popularization of computer technology, the management information system has become the hot field of software development. It runs stably in the browsers, like IE6, IE7, IE8, IE9 and FireFox, with high efficiency, sound security, friendly interface and simple operation. It has done research on the design and realization process of HTML5 off-line data storage on the Android platform. Under the Eclipse integrated development environment, with Android SDK and HTML5 grammar, develop the system, which realize the functions of off-line storage, addition, deletion and modification of user data. It enables the application of HTML5 of Android platform and has made detailed analysis on the Android platform and HTML5 application module. The result indicates, the webpage application design of HTML5 conducted on the Android platform, is simple and fast, which can better meet the demand of Android cellphone users.

NoSQL Databases

2014 ◽  
pp. 186-215 ◽  
Author(s):  
Ganesh Chandra Deka
Keyword(s):  
Cloud Computing ◽  
Big Data ◽  
Data Processing ◽  
Open Source ◽  
Data Storage ◽  
Big Data Processing ◽  
Nosql Databases ◽  
Data Intensive ◽  
Huge Data ◽  
Data Intensive Applications

NoSQL databases are designed to meet the huge data storage requirements of cloud computing and big data processing. NoSQL databases have lots of advanced features in addition to the conventional RDBMS features. Hence, the “NoSQL” databases are popularly known as “Not only SQL” databases. A variety of NoSQL databases having different features to deal with exponentially growing data-intensive applications are available with open source and proprietary option. This chapter discusses some of the popular NoSQL databases and their features on the light of CAP theorem.

Social Network Analysis

2020 ◽  
pp. 107-117
Author(s):  
Sheik Abdullah A. ◽  
Abiramie Shree T. G. R.
Keyword(s):  
Social Network ◽  
Data Analysis ◽  
Social Network Analysis ◽  
Network Analysis ◽  
Data Storage ◽  
Social Data ◽  
Cloud Data ◽  
Huge Data ◽  
Cloud Data Storage ◽  
Social Data Analysis

Each day, 2.5 quintillion bytes of data are generated due to our daily activity. It is due to the vast amount of use of the smart mobiles, Cloud data storage, and the Internet of Things. In earlier days, these technologies were utilized by large IT companies and the private sector, but now each person has a high-end smartphone along with the cloud and IoT for the easy storage of data and backup. The analysis of the data generated by social media is a tedious process and involves a lot of techniques. Some tools for social network analysis are: Gephi, Networkx, IGraph, Pajek, Node XL, and cytoscope. Apart from these tools there are various efficient social data analysis algorithms that are far more helpful in doing analytics. The need for and use of social network analysis is very helpful in our current problem of huge data generation. In this chapter, the need for the analysis of social data along with the tools that are needed for the analysis and the techniques that are to be implemented in the field of social data analysis are covered.

Sign in / Sign up

Export Citation Format

Share Document