scholarly journals Evaluation of big data frameworks for analysis of smart grids

2019 ◽  
Vol 6 (1) ◽  
Author(s):  
Mohammad Hasan Ansari ◽  
Vahid Tabatab Vakili ◽  
Behnam Bahrak
Keyword(s):  
Big Data ◽  
Smart Grid ◽  
Cyber Security ◽  
Smart Grids ◽  
Large Scale ◽  
Rapid Development ◽  
Massive Data ◽  
Grid Data ◽  
Data Generator ◽  
And Performance

AbstractWith the rapid development of smart grids and increasing data collected in these networks, analyzing this massive data for applications such as marketing, cyber-security, and performance analysis, has gained popularity. This paper focuses on analysis and performance evaluation of big data frameworks that are proposed for handling smart grid data. Since obtaining large amounts of smart grid data is difficult due to privacy concerns, we propose and implement a large scale smart grid data generator to produce massive data under conditions similar to those in real smart grids. We use four open source big data frameworks namely Hadoop-Hbase, Cassandra, Elasticsearch, and MongoDB, in our implementation. Finally, we evaluate the performance of different frameworks on smart grid big data and present a performance benchmark that includes common data analysis techniques on smart grid data.

scholarly journals A Comprehensive Survey on Cyber-Physical Smart Grid Testbed Architectures: Requirements and Challenges

Electronics ◽  
2021 ◽  
Vol 10 (9) ◽  
pp. 1043
Author(s):  
Abdallah A. Smadi ◽  
Babatunde Tobi Ajao ◽  
Brian K. Johnson ◽  
Hangtian Lei ◽  
Yacine Chakhchoukh ◽  
...  
Keyword(s):  
Smart Grid ◽  
Real Time ◽  
Smart Grids ◽  
Data Exchange ◽  
Information Technologies ◽  
New Technologies ◽  
Rapid Development ◽  
Environmental Benefits ◽  
Future Research ◽  
Time Data

The integration of improved control techniques with advanced information technologies enables the rapid development of smart grids. The necessity of having an efficient, reliable, and flexible communication infrastructure is achieved by enabling real-time data exchange between numerous intelligent and traditional electrical grid elements. The performance and efficiency of the power grid are enhanced with the incorporation of communication networks, intelligent automation, advanced sensors, and information technologies. Although smart grid technologies bring about valuable economic, social, and environmental benefits, testing the combination of heterogeneous and co-existing Cyber-Physical-Smart Grids (CP-SGs) with conventional technologies presents many challenges. The examination for both hardware and software components of the Smart Grid (SG) system is essential prior to the deployment in real-time systems. This can take place by developing a prototype to mimic the real operational circumstances with adequate configurations and precision. Therefore, it is essential to summarize state-of-the-art technologies of industrial control system testbeds and evaluate new technologies and vulnerabilities with the motivation of stimulating discoveries and designs. In this paper, a comprehensive review of the advancement of CP-SGs with their corresponding testbeds including diverse testing paradigms has been performed. In particular, we broadly discuss CP-SG testbed architectures along with the associated functions and main vulnerabilities. The testbed requirements, constraints, and applications are also discussed. Finally, the trends and future research directions are highlighted and specified.

Neural Network for Big Data Sets

2022 ◽  
pp. 41-67
Author(s):  
Vo Ngoc Phu ◽  
Vo Thi Ngoc Tran
Keyword(s):  
Neural Network ◽  
Big Data ◽  
Computer Science ◽  
Large Scale ◽  
Massive Data ◽  
Data Sets ◽  
Massive Data Sets ◽  
Large Scale Data ◽  
Commercial Applications ◽  
Novel Model

Machine learning (ML), neural network (NN), evolutionary algorithm (EA), fuzzy systems (FSs), as well as computer science have been very famous and very significant for many years. They have been applied to many different areas. They have contributed much to developments of many large-scale corporations, massive organizations, etc. Lots of information and massive data sets (MDSs) have been generated from these big corporations, organizations, etc. These big data sets (BDSs) have been the challenges of many commercial applications, researches, etc. Therefore, there have been many algorithms of the ML, the NN, the EA, the FSs, as well as computer science which have been developed to handle these massive data sets successfully. To support for this process, the authors have displayed all the possible algorithms of the NN for the large-scale data sets (LSDSs) successfully in this chapter. Finally, they have presented a novel model of the NN for the BDS in a sequential environment (SE) and a distributed network environment (DNE).

scholarly journals Future Generation 5G Wireless Networks for Smart Grid: A Comprehensive Review

Energies ◽  
2019 ◽  
Vol 12 (11) ◽  
pp. 2140 ◽  
Author(s):  
Sofana Reka. S ◽  
Tomislav Dragičević ◽  
Pierluigi Siano ◽  
S.R. Sahaya Prabaharan
Keyword(s):  
Smart Grid ◽  
Electricity Markets ◽  
Smart Grids ◽  
Large Scale ◽  
Business Models ◽  
Future Generation ◽  
Current Status ◽  
Complete Analysis ◽  
5G Networks ◽  
5G Network

Wireless cellular networks are emerging to take a strong stand in attempts to achieve pervasive large scale obtainment, communication, and processing with the evolution of the fifth generation (5G) network. Both the present day cellular technologies and the evolving new age 5G are considered to be advantageous for the smart grid. The 5G networks exhibit relevant services for critical and timely applications for greater aspects in the smart grid. In the present day electricity markets, 5G provides new business models to the energy providers and improves the way the utility communicates with the grid systems. In this work, a complete analysis and a review of the 5G network and its vision regarding the smart grid is exhibited. The work discusses the present day wireless technologies, and the architectural changes for the past years are shown. Furthermore, to understand the user-based analyses in a smart grid, a detailed analysis of 5G architecture with the grid perspectives is exhibited. The current status of 5G networks in a smart grid with a different analysis for energy efficiency is vividly explained in this work. Furthermore, focus is emphasized on future reliable smart grid communication with future roadmaps and challenges to be faced. The complete work gives an in-depth understanding of 5G networks as they pertain to future smart grids as a comprehensive analysis.

scholarly journals Military Simulation Big Data: Background, State of the Art, and Challenges

2015 ◽  
Vol 2015 ◽  
pp. 1-20 ◽  
Author(s):  
Xiao Song ◽  
Yulin Wu ◽  
Yaofei Ma ◽  
Yong Cui ◽  
Guanghong Gong
Keyword(s):  
Big Data ◽  
Large Scale ◽  
Rapid Development ◽  
Military Training ◽  
Great Success ◽  
Service Oriented Architectures ◽  
Weapon Systems ◽  
Service Oriented ◽  
High Resolution Models ◽  
Future Work

Big data technology has undergone rapid development and attained great success in the business field. Military simulation (MS) is another application domain producing massive datasets created by high-resolution models and large-scale simulations. It is used to study complicated problems such as weapon systems acquisition, combat analysis, and military training. This paper firstly reviewed several large-scale military simulations producing big data (MS big data) for a variety of usages and summarized the main characteristics of result data. Then we looked at the technical details involving the generation, collection, processing, and analysis of MS big data. Two frameworks were also surveyed to trace the development of the underlying software platform. Finally, we identified some key challenges and proposed a framework as a basis for future work. This framework considered both the simulation and big data management at the same time based on layered and service oriented architectures. The objective of this review is to help interested researchers learn the key points of MS big data and provide references for tackling the big data problem and performing further research.

scholarly journals A Comprehensive Review on Sustainable Aspects of Big Data Analytics for the Smart Grid

2021 ◽  
Vol 13 (23) ◽  
pp. 13322
Author(s):  
Vinoth Kumar Ponnusamy ◽  
Padmanathan Kasinathan ◽  
Rajvikram Madurai Elavarasan ◽  
Vinoth Ramanathan ◽  
Ranjith Kumar Anandan ◽  
...  
Keyword(s):  
Big Data ◽  
Smart Grid ◽  
Data Management ◽  
Smart Grids ◽  
Data Analytics ◽  
Management Practices ◽  
Research Work ◽  
Big Data Analytics ◽  
Energy Data ◽  
The Sustainable Development

The role of energy is cardinal for achieving the Sustainable Development Goals (SDGs) through the enhancement and modernization of energy generation and management practices. The smart grid enables efficient communication between utilities and the end- users, and enhances the user experience by monitoring and controlling the energy transmission. The smart grid deals with an enormous amount of energy data, and the absence of proper techniques for data collection, processing, monitoring and decision-making ultimately makes the system ineffective. Big data analytics, in association with the smart grid, enable better grid visualization and contribute toward the attainment of sustainability. The current research work deals with the achievement of sustainability in the smart grid and efficient data management using big data analytics, that has social, economic, technical and political impacts. This study provides clear insights into energy data generated in the grid and the possibilities of energy theft affecting the sustainable future. The paper provides insights about the importance of big data analytics, with their effects on the smart grids’ performance towards the achievement of SDGs. The work highlights efficient real-time energy data management involving artificial intelligence and machine learning for a better future, to short out the effects of the conventional smart grid without big data analytics. Finally, the work discusses the challenges and future directions to improve smart grid technologies with big data analytics in action.

scholarly journals Study of IPv6 Protocol in the Data Model of the Smart Grid Distribution Domain

2020 ◽  
Vol 16 (2) ◽  
Author(s):  
Guillermo Adolfo David Muñoz ◽  
Fabio Germán Guerrero
Keyword(s):  
Communication Network ◽  
Smart Grid ◽  
Cyber Security ◽  
Time Allocation ◽  
Data Model ◽  
Smart Grids ◽  
Communication Model ◽  
Great Expectation ◽  
Ipv6 Protocol ◽  
Smart Grid Communication

Introduction: This article is the product of the research “Study of the IPv6 Protocol in the data model of the Smart Grid distribution domain” developed at the Universidad del Valle and carried out during 2019. Problem: There is an immediate need to establish standards and protocols for the Smart Grid for both the electrical components and the component technologies of information and communication. Objective: The objective of the research is to characterize the use of IPv6 in the context of the communications domain distribution of the Smart Grid. Methodology: The work defines a virtualization environment in which the performance of IPv6 in the domain distribution of the Smart Grid will be evaluated; this evaluation includes measurement and analysis of delays as well as traffic volumes, bandwidth, cyber-security conditions, and time allocation of network addresses. Results: The IPv6 protocol is considered as a viable alternative in the Smart Grid communication model in order to comply with the communication requirements. Conclusion: The implementation of Quality of Service QoS in IPv6 defined in RFC2474 is essential in the Smart Grid communication network in order to meet the communication requirements of the defined applications. Originality: There is great expectation that networks based on the Internet Protocol will serve as a key element for communications within the Smart Grid. Limitations: The wide scope and dimensions involving Smart Grids, it is almost impossible to implement the communication network of a Smart Grid completely in a single simulation tool or emulation.

scholarly journals IoT based Smart Grid Attack Resistance in AC and DC State Estimation

2021 ◽  
Vol 2 (3) ◽  
pp. 118-122
Author(s):  
Vivekanadam B
Keyword(s):  
Big Data ◽  
Internet Of Things ◽  
Smart Grid ◽  
Error Detection ◽  
Smart Grids ◽  
Data Detection ◽  
Bad Data Detection ◽  
Security Issues ◽  
Number Of Factors ◽  
Bad Data

Use of automation and intelligence in smart grids has led to implementation in a number of applications. When internet of things is incorporated it will result in the significant improvement a number of factors such as fault recovery, energy delivery efficiency, demand response and reliability. However, the collaboration of internet of things and smart grid gives rise to a number of security issues and threats. This is especially the case when using internet based protocols and public communication infrastructure. To address these issues we should ensure that the data stored is secure and critical information from the data is extracted in a careful manner. If any threat to its security is detective an early blackout warning should be issued immediately. In this paper we have proposed a geometric view point for big data attacks which is capable of bypassing bad data detection. We have created an environment where replay scheme is used launch blind energy big data attack. The defence mechanism of our proposed work is studied and found to be efficient. Experimental evidence supports our theory and we have found our methodology to efficiently improve error detection rate.

scholarly journals Electricity Theft Detection in Smart Grid Systems: A CNN-LSTM Based Approach

Energies ◽  
2019 ◽  
Vol 12 (17) ◽  
pp. 3310 ◽  
Author(s):  
Md. Nazmul Hasan ◽  
Rafia Nishat Toma ◽  
Abdullah-Al Nahid ◽  
M M Manjurul Islam ◽  
Jong-Myon Kim
Keyword(s):  
Smart Grid ◽  
Power Consumption ◽  
Smart Grids ◽  
Time Series Data ◽  
Short Term Memory ◽  
Detection System ◽  
Series Data ◽  
Grid Data ◽  
Minority Class ◽  
Electricity Theft

Among an electricity provider’s non-technical losses, electricity theft has the most severe and dangerous effects. Fraudulent electricity consumption decreases the supply quality, increases generation load, causes legitimate consumers to pay excessive electricity bills, and affects the overall economy. The adaptation of smart grids can significantly reduce this loss through data analysis techniques. The smart grid infrastructure generates a massive amount of data, including the power consumption of individual users. Utilizing this data, machine learning and deep learning techniques can accurately identify electricity theft users. In this paper, an electricity theft detection system is proposed based on a combination of a convolutional neural network (CNN) and a long short-term memory (LSTM) architecture. CNN is a widely used technique that automates feature extraction and the classification process. Since the power consumption signature is time-series data, we were led to build a CNN-based LSTM (CNN-LSTM) model for smart grid data classification. In this work, a novel data pre-processing algorithm was also implemented to compute the missing instances in the dataset, based on the local values relative to the missing data point. Furthermore, in this dataset, the count of electricity theft users was relatively low, which could have made the model inefficient at identifying theft users. This class imbalance scenario was addressed through synthetic data generation. Finally, the results obtained indicate the proposed scheme can classify both the majority class (normal users) and the minority class (electricity theft users) with good accuracy.

Sign in / Sign up

Export Citation Format

Share Document