On the Effectiveness of Hybrid Canopy with Hoeffding Adaptive Naive Bayes Trees

2017 ◽  
Vol 8 (2) ◽  
pp. 30-43
Author(s):  
Mrutyunjaya Panda
Keyword(s):  
Big Data ◽  
Clustering Analysis ◽  
Large Scale ◽  
Data Sets ◽  
Recent Past ◽  
Large Scale Data ◽  
Huge Data ◽  
With Memory ◽  
Memory Constraints ◽  
Scale Data

The Big Data, due to its complicated and diverse nature, poses a lot of challenges for extracting meaningful observations. This sought smart and efficient algorithms that can deal with computational complexity along with memory constraints out of their iterative behavior. This issue may be solved by using parallel computing techniques, where a single machine or a multiple machine can perform the work simultaneously, dividing the problem into sub problems and assigning some private memory to each sub problems. Clustering analysis are found to be useful in handling such a huge data in the recent past. Even though, there are many investigations in Big data analysis are on, still, to solve this issue, Canopy and K-Means++ clustering are used for processing the large-scale data in shorter amount of time with no memory constraints. In order to find the suitability of the approach, several data sets are considered ranging from small to very large ones having diverse filed of applications. The experimental results opine that the proposed approach is fast and accurate.

On the Effectiveness of Hybrid Canopy With Hoeffding Adaptive Naive Bayes Trees

Web Services ◽  
2019 ◽  
pp. 788-802
Author(s):  
Mrutyunjaya Panda
Keyword(s):  
Big Data ◽  
Data Analysis ◽  
Large Scale ◽  
Big Data Analysis ◽  
Data Sets ◽  
Large Scale Data ◽  
Huge Data ◽  
With Memory ◽  
Memory Constraints ◽  
Scale Data

The Big Data, due to its complicated and diverse nature, poses a lot of challenges for extracting meaningful observations. This sought smart and efficient algorithms that can deal with computational complexity along with memory constraints out of their iterative behavior. This issue may be solved by using parallel computing techniques, where a single machine or a multiple machine can perform the work simultaneously, dividing the problem into sub problems and assigning some private memory to each sub problems. Clustering analysis are found to be useful in handling such a huge data in the recent past. Even though, there are many investigations in Big data analysis are on, still, to solve this issue, Canopy and K-Means++ clustering are used for processing the large-scale data in shorter amount of time with no memory constraints. In order to find the suitability of the approach, several data sets are considered ranging from small to very large ones having diverse filed of applications. The experimental results opine that the proposed approach is fast and accurate.

scholarly journals Influencing Factors of e-Commerce Enterprise Development Based on Mobile Computing Big Data Analysis

2021 ◽  
Vol 2021 ◽  
pp. 1-12
Author(s):  
Yixue Zhu ◽  
Boyue Chai
Keyword(s):  
Big Data ◽  
Data Analysis ◽  
Large Scale ◽  
Big Data Analysis ◽  
Support Vector ◽  
Data Sets ◽  
Large Scale Data ◽  
Vector Machines ◽  
Physical Information ◽  
Scale Data

With the development of increasingly advanced information technology and electronic technology, especially with regard to physical information systems, cloud computing systems, and social services, big data will be widely visible, creating benefits for people and at the same time facing huge challenges. In addition, with the advent of the era of big data, the scale of data sets is getting larger and larger. Traditional data analysis methods can no longer solve the problem of large-scale data sets, and the hidden information behind big data is digging out, especially in the field of e-commerce. We have become a key factor in competition among enterprises. We use a support vector machine method based on parallel computing to analyze the data. First, the training samples are divided into several working subsets through the SOM self-organizing neural network classification method. Compared with the ever-increasing progress of information technology and electronic equipment, especially the related physical information system finally merges the training results of each working set, so as to quickly deal with the problem of massive data prediction and analysis. This paper proposes that big data has the flexibility of expansion and quality assessment system, so it is meaningful to replace the double-sidedness of quality assessment with big data. Finally, considering the excellent performance of parallel support vector machines in data mining and analysis, we apply this method to the big data analysis of e-commerce. The research results show that parallel support vector machines can solve the problem of processing large-scale data sets. The emergence of data dirty problems has increased the effective rate by at least 70%.

scholarly journals Explanations as Discourse

2020 ◽  
Vol 24 ◽  
Author(s):  
Sadaf Afrashteh ◽  
Ida Someh ◽  
Michael Davern
Keyword(s):  
Decision Making ◽  
Big Data ◽  
Data Analytics ◽  
Large Scale ◽  
Big Data Analytics ◽  
Customer Engagement ◽  
Data Sets ◽  
Research Directions ◽  
Large Scale Data ◽  
Scale Data

Big data analytics uses algorithms for decision-making and targeting of customers. These algorithms process large-scale data sets and create efficiencies in the decision-making process for organizations but are often incomprehensible to customers and inherently opaque in nature. Recent European Union regulations require that organizations communicate meaningful information to customers on the use of algorithms and the reasons behind decisions made about them. In this paper, we explore the use of explanations in big data analytics services. We rely on discourse ethics to argue that explanations can facilitate a balanced communication between organizations and customers, leading to transparency and trust for customers as well as customer engagement and reduced reputation risks for organizations. We conclude the paper by proposing future empirical research directions.

Looking in the Wrong (La)place? The Promise and Perils of Becoming Big Data

2016 ◽  
Vol 42 (4) ◽  
pp. 657-678 ◽  
Author(s):  
Lawrence Busch
Keyword(s):  
Big Data ◽  
Large Scale ◽  
Data Sets ◽  
Audit Process ◽  
Management Evaluation ◽  
Large Scale Data ◽  
A Value ◽  
Science Management ◽  
Management Audits ◽  
Scale Data

Laplace once argued that if one could “comprehend all the forces by which nature is animated,” it would be possible to predict the future and explain the past. The advent of analysis of large-scale data sets has been accompanied by newfound concerns about “Laplace’s Demon” as it relates to certain fields of science as well as management, evaluation, and audit. I begin by asking how statistical data are constructed, illustrating the hermeneutic acts necessary to create a variable. These include attributing a certain characteristic to a particular phenomenon, isolating the characteristic of interest, and assigning a value to it. In addition, a population must be identified and a sample must be “taken” from that population. Next, I examine how statistical analyses are conducted, examining the interpretive acts there as well. In each case, I show how big data add new challenges. I then show how statistics are incorporated into audits and evaluations, emphasizing how alternative interpretations are concealed in the audit process. I conclude by noting that these issues cannot be “resolved” as Laplace suggested. His Demon, already banished from physics, needs to be banished from other fields of science, management, audits, and evaluations as well.

scholarly journals Support Vector Machines in Big Data Classification: A Systematic Literature Review

2021 ◽  
Author(s):  
Mohammad Hassan Almaspoor ◽  
Ali Safaei ◽  
Afshin Salajegheh ◽  
Behrouz Minaei-Bidgoli
Keyword(s):  
Machine Learning ◽  
Big Data ◽  
Large Scale ◽  
Support Vector ◽  
Research Areas ◽  
Large Scale Data ◽  
Training Samples ◽  
Big Data Classification ◽  
Scale Data

Abstract Classification is one of the most important and widely used issues in machine learning, the purpose of which is to create a rule for grouping data to sets of pre-existing categories is based on a set of training sets. Employed successfully in many scientific and engineering areas, the Support Vector Machine (SVM) is among the most promising methods of classification in machine learning. With the advent of big data, many of the machine learning methods have been challenged by big data characteristics. The standard SVM has been proposed for batch learning in which all data are available at the same time. The SVM has a high time complexity, i.e., increasing the number of training samples will intensify the need for computational resources and memory. Hence, many attempts have been made at SVM compatibility with online learning conditions and use of large-scale data. This paper focuses on the analysis, identification, and classification of existing methods for SVM compatibility with online conditions and large-scale data. These methods might be employed to classify big data and propose research areas for future studies. Considering its advantages, the SVM can be among the first options for compatibility with big data and classification of big data. For this purpose, appropriate techniques should be developed for data preprocessing in order to covert data into an appropriate form for learning. The existing frameworks should also be employed for parallel and distributed processes so that SVMs can be made scalable and properly online to be able to handle big data.

scholarly journals Data Lake Ecosystem Workflow

2021 ◽  
Author(s):  
R. Salter ◽  
Quyen Dong ◽  
Cody Coleman ◽  
Maria Seale ◽  
Alicia Ruvinsky ◽  
...  
Keyword(s):  
Big Data ◽  
Language Processing ◽  
Data Analytics ◽  
Large Scale ◽  
Big Data Analytics ◽  
Lake Ecosystem ◽  
Data Governance ◽  
Government Organizations ◽  
Large Scale Data ◽  
Scale Data

The Engineer Research and Development Center, Information Technology Laboratory’s (ERDC-ITL’s) Big Data Analytics team specializes in the analysis of large-scale datasets with capabilities across four research areas that require vast amounts of data to inform and drive analysis: large-scale data governance, deep learning and machine learning, natural language processing, and automated data labeling. Unfortunately, data transfer between government organizations is a complex and time-consuming process requiring coordination of multiple parties across multiple offices and organizations. Past successes in large-scale data analytics have placed a significant demand on ERDC-ITL researchers, highlighting that few individuals fully understand how to successfully transfer data between government organizations; future project success therefore depends on a small group of individuals to efficiently execute a complicated process. The Big Data Analytics team set out to develop a standardized workflow for the transfer of large-scale datasets to ERDC-ITL, in part to educate peers and future collaborators on the process required to transfer datasets between government organizations. Researchers also aim to increase workflow efficiency while protecting data integrity. This report provides an overview of the created Data Lake Ecosystem Workflow by focusing on the six phases required to efficiently transfer large datasets to supercomputing resources located at ERDC-ITL.

scholarly journals Discovering Latent Class Labels for Multi-Label Learning

2020 ◽  
Author(s):  
Jun Huang ◽  
Linchuan Xu ◽  
Jing Wang ◽  
Lei Feng ◽  
Kenji Yamanishi
Keyword(s):  
Large Scale ◽  
Latent Class ◽  
Training Data ◽  
Data Sets ◽  
Robust Learning ◽  
Large Scale Data ◽  
Novel Approach ◽  
Fixed Set ◽  
Class Labels ◽  
Scale Data

Existing multi-label learning (MLL) approaches mainly assume all the labels are observed and construct classification models with a fixed set of target labels (known labels). However, in some real applications, multiple latent labels may exist outside this set and hide in the data, especially for large-scale data sets. Discovering and exploring the latent labels hidden in the data may not only find interesting knowledge but also help us to build a more robust learning model. In this paper, a novel approach named DLCL (i.e., Discovering Latent Class Labels for MLL) is proposed which can not only discover the latent labels in the training data but also predict new instances with the latent and known labels simultaneously. Extensive experiments show a competitive performance of DLCL against other state-of-the-art MLL approaches.

Sign in / Sign up

Export Citation Format

Share Document