A fast and parallel algorithm for frequent pattern mining from big data in many-task environments

2017 ◽  
Vol 10 (3) ◽  
pp. 157 ◽  
Author(s):  
Wei Tee Lin ◽  
Chih Ping Chu
Keyword(s):  
Big Data ◽  
Parallel Algorithm ◽  
Pattern Mining ◽  
Frequent Pattern Mining ◽  
Frequent Pattern ◽  
Task Environments

A New Parallel Algorithm for Frequent Pattern Mining

2013 ◽  
Vol 2 (1) ◽  
pp. 55-59 ◽  
Author(s):  
Saeid Masoumi ◽  
Raziyeh Tabatabaei ◽  
Mohammad-Reza Feizi-Derakhshi ◽  
Khatereh Tabatabaei
Keyword(s):  
Parallel Algorithm ◽  
Pattern Mining ◽  
Frequent Pattern Mining ◽  
Frequent Pattern

scholarly journals Big Data Frequent Pattern Mining

2014 ◽  
pp. 225-259 ◽  
Author(s):  
David C. Anastasiu ◽  
Jeremy Iverson ◽  
Shaden Smith ◽  
George Karypis
Keyword(s):  
Big Data ◽  
Pattern Mining ◽  
Frequent Pattern Mining ◽  
Frequent Pattern

scholarly journals Frequent Pattern Mining over Unstructured Data using Semi-Structured Doc-Model and Pattern Ranking

2020 ◽  
pp. 36-42
Author(s):  
Sudhir Tirumalasetty ◽  
A. Divya ◽  
D. Rahitya Lakshmi ◽  
Ch. Durga Bhavani ◽  
D. Anusha
Keyword(s):  
Data Mining ◽  
Big Data ◽  
Pattern Mining ◽  
Frequent Pattern Mining ◽  
Unstructured Data ◽  
Frequent Pattern ◽  
Frequent Patterns ◽  
Innovative Methods ◽  
Mining Algorithms ◽  
Doc Model

Frequent pattern mining is an essential data-mining task, with a goal of discovering knowledge in the form of repeated patterns. Many efficient pattern-mining algorithms have been discovered in the last two decades, yet most do not scale to the type of data we are presented with today, the so-called “Big Data”. Scalable parallel algorithms hold the key to solving the problem in this context. This paper reviews recent advances in parallel frequent pattern mining, analysing them through the Big Data lens. Load balancing and work partitioning are the major challenges to be conquered. These challenges always invoke innovative methods to do, as Big Data evolves with no limits. The biggest challenge than before is conquering unstructured data for finding frequent patterns. To accomplish this Semi Structured Doc-Model and ranking of patterns are used.

scholarly journals A MapReduce-Based Parallel Frequent Pattern Growth Algorithm for Spatiotemporal Association Analysis of Mobile Trajectory Big Data

Complexity ◽  
2018 ◽  
Vol 2018 ◽  
pp. 1-16 ◽  
Author(s):  
Dawen Xia ◽  
Xiaonan Lu ◽  
Huaqing Li ◽  
Wendong Wang ◽  
Yantao Li ◽  
...  
Keyword(s):  
Big Data ◽  
Association Analysis ◽  
Intelligent Transportation Systems ◽  
Large Scale ◽  
Pattern Mining ◽  
Frequent Pattern Mining ◽  
Transportation Systems ◽  
Frequent Pattern ◽  
Trajectory Data ◽  
Pattern Growth

Frequent pattern mining is an effective approach for spatiotemporal association analysis of mobile trajectory big data in data-driven intelligent transportation systems. While existing parallel algorithms have been successfully applied to frequent pattern mining of large-scale trajectory data, two major challenges are how to overcome the inherent defects of Hadoop to cope with taxi trajectory big data including massive small files and how to discover the implicitly spatiotemporal frequent patterns with MapReduce. To conquer these challenges, this paper presents a MapReduce-based Parallel Frequent Pattern growth (MR-PFP) algorithm to analyze the spatiotemporal characteristics of taxi operating using large-scale taxi trajectories with massive small file processing strategies on a Hadoop platform. More specifically, we first implement three methods, that is, Hadoop Archives (HAR), CombineFileInputFormat (CFIF), and Sequence Files (SF), to overcome the existing defects of Hadoop and then propose two strategies based on their performance evaluations. Next, we incorporate SF into Frequent Pattern growth (FP-growth) algorithm and then implement the optimized FP-growth algorithm on a MapReduce framework. Finally, we analyze the characteristics of taxi operating in both spatial and temporal dimensions by MR-PFP in parallel. The results demonstrate that MR-PFP is superior to existing Parallel FP-growth (PFP) algorithm in efficiency and scalability.

Sign in / Sign up

Export Citation Format

Share Document