Research On Parallel Association Rules Mining Of Big Data Based On Improved K-Means Clustering Algorithm

2023 ◽  
Vol 16 (3) ◽  
pp. 1
Author(s):  
Chaoping Guo ◽  
Tuanbu Wang ◽  
Li Hao
Keyword(s):  
Big Data ◽  
Association Rules ◽  
Clustering Algorithm ◽  
Association Rules Mining

Association Rules Mining Based on Adaptive Fuzzy Clustering Algorithm

2014 ◽  
Vol 998-999 ◽  
pp. 842-845 ◽  
Author(s):  
Jia Mei Guo ◽  
Yin Xiang Pei
Keyword(s):  
Data Mining ◽  
Association Rules ◽  
Clustering Algorithm ◽  
Original Data ◽  
Data Set ◽  
Association Rules Mining ◽  
Fuzzy Association Rules ◽  
Redundant Data ◽  
Fuzzy Partitions ◽  
Rules Extraction

Association rules extraction is one of the important goals of data mining and analyzing. Aiming at the problem that information lose caused by crisp partition of numerical attribute , in this article, we put forward a fuzzy association rules mining method based on fuzzy logic. First, we use c-means clustering to generate fuzzy partitions and eliminate redundant data, and then map the original data set into fuzzy interval, in the end, we extract the fuzzy association rules on the fuzzy data set as providing the basis for proper decision-making. Results show that this method can effectively improve the efficiency of data mining and the semantic visualization and credibility of association rules.

scholarly journals ASSOCIATION RULES MINING IN BIG DATA

2018 ◽  
pp. 25-32 ◽  
Author(s):  
Nataliya Shakhovska ◽  
Roman Kaminskyy ◽  
Eugen Zasoba ◽  
Mykola Tsiutsiura
Keyword(s):  
Big Data ◽  
Association Rules ◽  
Distributed Databases ◽  
Data Sources ◽  
Data Types ◽  
Association Rules Mining ◽  
Mining Algorithm ◽  
Data Definition ◽  
Subject Areas ◽  
Asymptotic Complexity

The paper proposes a method for Big data analyzing in the presence of different data sources and different methods of processing these data. The Big data definition is given, the main problems of data mining process are described. The concept of association rules is introduced and the method of association rules searching for working with Big Data is modified. The method of finding dependencies is developed, efficiency and possibility of its parallelization are determined. The developed algorithm makes it possible to assert that the task of detecting association dependencies in distributed databases belongs to the class of P-tasks. The algorithm for finding association dependencies is well-solved with MapReduce. The low asymptotic complexity of the developed association rules mining algorithm and a wide set of data types supported for analysis allow to apply the proposed algorithm in practically all subject areas working with association dependencies in the data domain.

A Novel MapReduce-based Algorithm for Association Rules Mining

2021 ◽  
Author(s):  
Bin Wu ◽  
Yimin Mao ◽  
Deborah Simon Mwakapesa ◽  
Yaser Ahangari Nanehkaran ◽  
Qianhu Deng ◽  
...  
Keyword(s):  
Data Mining ◽  
Big Data ◽  
Association Rules ◽  
Data Transmission ◽  
Association Rules Mining ◽  
Support Threshold ◽  
Speed Up ◽  
Dynamic Support ◽  
Optimal Dynamic ◽  
Data Environment

Abstract AR (Association rule) is considered to be one of the models for data mining. With the growth of datasets, conventional association rules are not suitable for big data mining, which has aroused a large number of scholars' interest in algorithm innovation. This study aims to design an optimization parallel association rules mining algorithm based on MapReduce, named as PMRARIM-IEG algorithm, to deal with problems such as the excessive space occupied by the CanTree (CanTreeCanonical order Tree), the inability to dynamically set the support threshold, and the time-consuming data transmission in the Map and Reduce phases. Firstly, a structure called SIM-IE (similar items merging based on information entropy) strategy is adopted for reducing the space occupation of the CanTree effectively. Then, a DST-GA (dynamic support threshold obtaining using genetic algorithm) is proposed to obtain the relatively optimal dynamic support threshold in the big data environment. Finally, in the process of MapReduce parallel, a LZO (Lempel-Ziv-Oberhumer) data compression strategy is used to compress the output data of the Map stage, which improves the speed of the data transmission. We compared the PMRARIM-IEG algorithm with other algorithms on five datasets, including Wikipedia , LiveJournal, com-amazon, kosarak, and webdocs. The experimental results obtained demonstrate that the proposed algorithm, PMRARIM-IEG, not only reduces the space and time complexity, but also obtains a well-performing speed-up ratio in a big data environment.

scholarly journals A Genetic Algorithm Based Multilevel Association Rules Mining for Big Datasets

2014 ◽  
Vol 2014 ◽  
pp. 1-9 ◽  
Author(s):  
Yang Xu ◽  
Mingming Zeng ◽  
Quanhui Liu ◽  
Xiaofeng Wang
Keyword(s):  
Genetic Algorithm ◽  
Big Data ◽  
Association Rules ◽  
Domain Knowledge ◽  
Computational Cost ◽  
Search Space ◽  
Association Mining ◽  
Association Rules Mining ◽  
Data Elements ◽  
Multilevel Association Rules

Multilevel association rules mining is an important domain to discover interesting relations between data elements with multiple levels abstractions. Most of the existing algorithms toward this issue are based on exhausting search methods such as Apriori, and FP-growth. However, when they are applied in the big data applications, those methods will suffer for extreme computational cost in searching association rules. To expedite multilevel association rules searching and avoid the excessive computation, in this paper, we proposed a novel genetic-based method with three key innovations. First, we use the category tree to describe the multilevel application data sets as the domain knowledge. Then, we put forward a special tree encoding schema based on the category tree to build the heuristic multilevel association mining algorithm. As the last part of our design, we proposed the genetic algorithm based on the tree encoding schema that will greatly reduce the association rule search space. The method is especially useful in mining multilevel association rules in big data related applications. We test the proposed method with some big datasets, and the experimental results demonstrate the effectiveness and efficiency of the proposed method in processing big data. Moreover, our results also manifest that the algorithm is fast convergent with a limited termination threshold.

Sign in / Sign up

Export Citation Format

Share Document