scholarly journals Large Scale Fault Data Analysis and OSS Reliability Assessment Based on Quantification Method of the First Type

2020 ◽  
Vol 2 (4) ◽  
pp. 436-452
Author(s):  
Yoshinobu Tamura ◽  
Shigeru Yamada
Keyword(s):  
Big Data ◽  
Large Scale ◽  
Reliability Assessment ◽  
Sensor Data ◽  
Weather Data ◽  
Data Sets ◽  
Log Data ◽  
Customer Data ◽  
Server Side ◽  
Quantification Method

Various big data sets are recorded on the server side of computer system. The big data are well defined as a volume, variety, and velocity (3V) model. The 3V model has been proposed by Gartner, Inc. as a first press release. 3V model means the volume, variety, and velocity in terms of data. The big data have 3V in well balance. Then, there are various categories in terms of the big data, e.g., sensor data, log data, customer data, financial data, weather data, picture data, movie data, and so on. In particular, the fault big data are well-known as the characteristic log data in software engineering. In this paper, we analyze the fault big data considering the unique features that arise from big data under the operation of open source software. In addition, we analyze actual data to show numerical examples of reliability assessment based on the results of multiple regression analysis well-known as the quantification method of the first type.

Big Data Analysis with Hadoop on Personalized Incentive Model with Statistical Hotel Customer Data

2016 ◽  
Vol 4 (3) ◽  
pp. 1-21 ◽  
Author(s):  
Sungchul Lee ◽  
Eunmin Hwang ◽  
Ju-Yeon Jo ◽  
Yoohwan Kim
Keyword(s):  
Big Data ◽  
Data Analysis ◽  
Large Scale ◽  
Big Data Analysis ◽  
Data Sets ◽  
New Approach ◽  
Customer Data ◽  
Customer Information ◽  
Large Scale Data ◽  
Incentive Model

Due to the advancement of Information Technology (IT), the hospitality industry is seeing a great value in gathering various kinds of and a large amount of customers' data. However, many hotels are facing a challenge in analyzing customer data and using it as an effective tool to understand the hospitality customers better and, ultimately, to increase the revenue. The authors' research attempts to resolve the current challenges of analyzing customer data in hospitality by utilizing the big data analysis tools, especially Hadoop and R. Hadoop is a framework for processing large-scale data. With the integration of new approach, their study demonstrates the ways of aggregating and analyzing the hospitality customer data to find meaningful customer information. Multiple decision trees are constructed from the customer data sets with the intention of classifying customers' needs and customers' clusters. By analyzing the customer data, the study suggests three strategies to increase the total expenditure of the customers within a limited amount of time during their stay.

scholarly journals Development of a Global Fire Weather Database

2015 ◽  
Vol 15 (6) ◽  
pp. 1407-1423 ◽  
Author(s):  
R. D. Field ◽  
A. C. Spessa ◽  
N. A. Aziz ◽  
A. Camia ◽  
A. Cantin ◽  
...  
Keyword(s):  
Southeast Asia ◽  
Large Scale ◽  
Prediction Models ◽  
Dry Season ◽  
Weather Data ◽  
Data Sets ◽  
Fire Weather ◽  
Fire Weather Index ◽  
Mato Grosso ◽  
Global Database

Abstract. The Canadian Forest Fire Weather Index (FWI) System is the mostly widely used fire danger rating system in the world. We have developed a global database of daily FWI System calculations, beginning in 1980, called the Global Fire WEather Database (GFWED) gridded to a spatial resolution of 0.5° latitude by 2/3° longitude. Input weather data were obtained from the NASA Modern Era Retrospective-Analysis for Research and Applications (MERRA), and two different estimates of daily precipitation from rain gauges over land. FWI System Drought Code calculations from the gridded data sets were compared to calculations from individual weather station data for a representative set of 48 stations in North, Central and South America, Europe, Russia, Southeast Asia and Australia. Agreement between gridded calculations and the station-based calculations tended to be most different at low latitudes for strictly MERRA-based calculations. Strong biases could be seen in either direction: MERRA DC over the Mato Grosso in Brazil reached unrealistically high values exceeding DC = 1500 during the dry season but was too low over Southeast Asia during the dry season. These biases are consistent with those previously identified in MERRA's precipitation, and they reinforce the need to consider alternative sources of precipitation data. GFWED can be used for analyzing historical relationships between fire weather and fire activity at continental and global scales, in identifying large-scale atmosphere–ocean controls on fire weather, and calibration of FWI-based fire prediction models.

Large-Scale Sensor Network Analysis

2013 ◽  
pp. 314-347 ◽  
Author(s):  
Joaquin Vanschoren ◽  
Ugo Vespier ◽  
Shengfa Miao ◽  
Marvin Meeng ◽  
Ricardo Cachucho ◽  
...  
Keyword(s):  
Big Data ◽  
Data Analysis ◽  
Large Scale ◽  
Vital Signs ◽  
Sensor Data ◽  
Atmospheric Conditions ◽  
Big Data Applications ◽  
The World ◽  
Sheer Size ◽  
Effective Use

Sensors are increasingly being used to monitor the world around us. They measure movements of structures such as bridges, windmills, and plane wings, human’s vital signs, atmospheric conditions, and fluctuations in power and water networks. In many cases, this results in large networks with different types of sensors, generating impressive amounts of data. As the volume and complexity of data increases, their effective use becomes more challenging, and novel solutions are needed both on a technical as well as a scientific level. Founded on several real-world applications, this chapter discusses the challenges involved in large-scale sensor data analysis and describes practical solutions to address them. Due to the sheer size of the data and the large amount of computation involved, these are clearly “Big Data” applications.

Neural Network for Big Data Sets

2022 ◽  
pp. 41-67
Author(s):  
Vo Ngoc Phu ◽  
Vo Thi Ngoc Tran
Keyword(s):  
Neural Network ◽  
Big Data ◽  
Computer Science ◽  
Large Scale ◽  
Massive Data ◽  
Data Sets ◽  
Massive Data Sets ◽  
Large Scale Data ◽  
Commercial Applications ◽  
Novel Model

Machine learning (ML), neural network (NN), evolutionary algorithm (EA), fuzzy systems (FSs), as well as computer science have been very famous and very significant for many years. They have been applied to many different areas. They have contributed much to developments of many large-scale corporations, massive organizations, etc. Lots of information and massive data sets (MDSs) have been generated from these big corporations, organizations, etc. These big data sets (BDSs) have been the challenges of many commercial applications, researches, etc. Therefore, there have been many algorithms of the ML, the NN, the EA, the FSs, as well as computer science which have been developed to handle these massive data sets successfully. To support for this process, the authors have displayed all the possible algorithms of the NN for the large-scale data sets (LSDSs) successfully in this chapter. Finally, they have presented a novel model of the NN for the BDS in a sequential environment (SE) and a distributed network environment (DNE).

Research Challenges in Big Data Analytics

2017 ◽  
pp. 83-99
Author(s):  
Sivamathi Chokkalingam ◽  
Vijayarani S.
Keyword(s):  
Big Data ◽  
Data Analytics ◽  
Large Scale ◽  
New Technologies ◽  
Big Data Analytics ◽  
Large Data ◽  
Data Sets ◽  
Data Types ◽  
Customer Preferences ◽  
Research Challenges

The term Big Data refers to large-scale information management and analysis technologies that exceed the capability of traditional data processing technologies. Big Data is differentiated from traditional technologies in three ways: volume, velocity and variety of data. Big data analytics is the process of analyzing large data sets which contains a variety of data types to uncover hidden patterns, unknown correlations, market trends, customer preferences and other useful business information. Since Big Data is new emerging field, there is a need for development of new technologies and algorithms for handling big data. The main objective of this paper is to provide knowledge about various research challenges of Big Data analytics. A brief overview of various types of Big Data analytics is discussed in this paper. For each analytics, the paper describes process steps and tools. A banking application is given for each analytics. Some of research challenges and possible solutions for those challenges of big data analytics are also discussed.

On the Effectiveness of Hybrid Canopy with Hoeffding Adaptive Naive Bayes Trees

2017 ◽  
Vol 8 (2) ◽  
pp. 30-43
Author(s):  
Mrutyunjaya Panda
Keyword(s):  
Big Data ◽  
Clustering Analysis ◽  
Large Scale ◽  
Data Sets ◽  
Recent Past ◽  
Large Scale Data ◽  
Huge Data ◽  
With Memory ◽  
Memory Constraints ◽  
Scale Data

The Big Data, due to its complicated and diverse nature, poses a lot of challenges for extracting meaningful observations. This sought smart and efficient algorithms that can deal with computational complexity along with memory constraints out of their iterative behavior. This issue may be solved by using parallel computing techniques, where a single machine or a multiple machine can perform the work simultaneously, dividing the problem into sub problems and assigning some private memory to each sub problems. Clustering analysis are found to be useful in handling such a huge data in the recent past. Even though, there are many investigations in Big data analysis are on, still, to solve this issue, Canopy and K-Means++ clustering are used for processing the large-scale data in shorter amount of time with no memory constraints. In order to find the suitability of the approach, several data sets are considered ranging from small to very large ones having diverse filed of applications. The experimental results opine that the proposed approach is fast and accurate.

Big Data and Clustering Techniques

2020 ◽  
pp. 1-9
Author(s):  
Jayashree K. ◽  
Chithambaramani R.
Keyword(s):  
Social Media ◽  
Big Data ◽  
Sensor Data ◽  
Log Data ◽  
Clustering Techniques ◽  
Research Challenges ◽  
The Future ◽  
Process Operations ◽  
Future Direction

Big data has become a chief strength of innovation across academics, governments, and corporates. Big data comprises massive sensor data, raw and semi-structured log data of IT industries, and the exploded quantity of data from social media. Big data needs big storage, and this volume makes operations such as analytical operations, process operations, retrieval operations very difficult and time consuming. One way to overcome these difficult problems is to have big data clustered in a compact format. Thus, this chapter discusses the background of big data and clustering. It also discusses the various application of big data in detail. The various related work, research challenges of big data, and the future direction are addressed in this chapter.

scholarly journals Influencing Factors of e-Commerce Enterprise Development Based on Mobile Computing Big Data Analysis

2021 ◽  
Vol 2021 ◽  
pp. 1-12
Author(s):  
Yixue Zhu ◽  
Boyue Chai
Keyword(s):  
Big Data ◽  
Data Analysis ◽  
Large Scale ◽  
Big Data Analysis ◽  
Support Vector ◽  
Data Sets ◽  
Large Scale Data ◽  
Vector Machines ◽  
Physical Information ◽  
Scale Data

With the development of increasingly advanced information technology and electronic technology, especially with regard to physical information systems, cloud computing systems, and social services, big data will be widely visible, creating benefits for people and at the same time facing huge challenges. In addition, with the advent of the era of big data, the scale of data sets is getting larger and larger. Traditional data analysis methods can no longer solve the problem of large-scale data sets, and the hidden information behind big data is digging out, especially in the field of e-commerce. We have become a key factor in competition among enterprises. We use a support vector machine method based on parallel computing to analyze the data. First, the training samples are divided into several working subsets through the SOM self-organizing neural network classification method. Compared with the ever-increasing progress of information technology and electronic equipment, especially the related physical information system finally merges the training results of each working set, so as to quickly deal with the problem of massive data prediction and analysis. This paper proposes that big data has the flexibility of expansion and quality assessment system, so it is meaningful to replace the double-sidedness of quality assessment with big data. Finally, considering the excellent performance of parallel support vector machines in data mining and analysis, we apply this method to the big data analysis of e-commerce. The research results show that parallel support vector machines can solve the problem of processing large-scale data sets. The emergence of data dirty problems has increased the effective rate by at least 70%.

Interoperability in Internet of Media Things and Integration Big Media

2022 ◽  
pp. 59-79
Author(s):  
Dragorad A. Milovanovic ◽  
Vladan Pantovic
Keyword(s):  
Big Data ◽  
Data Analytics ◽  
Large Scale ◽  
Reference Model ◽  
Big Data Analytics ◽  
Multimedia Data ◽  
Data Sets ◽  
Large Scale Systems ◽  
New Class ◽  
Media Applications

Multimedia-related things is a new class of connected objects that can be searched, discovered, and composited on the internet of media things (IoMT). A huge amount of data sets come from audio-visual sources or have a multimedia nature. However, multimedia data is currently not incorporated in the big data (BD) frameworks. The research projects, standardization initiatives, and industrial activities for integration are outlined in this chapter. MPEG IoMT interoperability and network-based media processing (NBMP) framework as an instance of the big media (BM) reference model are explored. Conceptual model of IoT and big data integration for analytics is proposed. Big data analytics is rapidly evolving both in terms of functionality and the underlying model. The authors pointed out that IoMT analytics is closely related to big data analytics, which facilitates the integration of multimedia objects in big media applications in large-scale systems. These two technologies are mutually dependent and should be researched and developed jointly.

scholarly journals MCEEP-BDA: Multilevel Clustering Based -Energy Efficient Privacy-Preserving Big Data Aggregation in Large-Scale Wsn

2019 ◽  
Vol 9 (1) ◽  
pp. 6779-6785
Keyword(s):  
Big Data ◽  
Data Storage ◽  
Data Aggregation ◽  
Energy Efficient ◽  
Large Scale ◽  
Computational Cost ◽  
Privacy Preserving ◽  
Sensor Nodes ◽  
Sensor Data ◽  
Communication Overhead

In current scenario, the Big Data processing that includes data storage, aggregation, transmission and evaluation has attained more attraction from researchers, since there is an enormous data produced by the sensing nodes of large-scale Wireless Sensor Networks (WSNs). Concerning the energy efficiency and the privacy conservation needs of WSNs in big data aggregation and processing, this paper develops a novel model called Multilevel Clustering based- Energy Efficient Privacy-preserving Big Data Aggregation (MCEEP-BDA). Initially, based on the pre-defined structure of gradient topology, the sensor nodes are framed into clusters. Further, the sensed information collected from each sensor node is altered with respect to the privacy preserving model obtained from their corresponding sinks. The Energy model has been defined for determining the efficient energy consumption in the overall process of big data aggregation in WSN. Moreover, Cluster_head Rotation process has been incorporated for effectively reducing the communication overhead and computational cost. Additionally, algorithm has been framed for Least BDA Tree for aggregating the big sensor data through the selected cluster heads effectively. The simulation results show that the developed MCEEP-BDA model is more scalable and energy efficient. And, it shows that the Big Data Aggregation (BDA) has been performed here with reduced resource utilization and secure manner by the privacy preserving model, further satisfying the security concerns of the developing application-oriented needs.

Sign in / Sign up

Export Citation Format

Share Document