scholarly journals A Synopsis Based Approach for Itemset Frequency Estimation over Massive Multi-Transaction Stream

2021 ◽  
Vol 16 (2) ◽  
pp. 1-30
Author(s):  
Guangtao Wang ◽  
Gao Cong ◽  
Ying Zhang ◽  
Zhen Hai ◽  
Jieping Ye
Keyword(s):  
Frequency Estimation ◽  
Frequent Itemsets ◽  
Frequent Itemset ◽  
Experimental Results ◽  
Closure Property ◽  
Frequent Itemset Mining ◽  
Itemset Mining ◽  
Minimum Value ◽  
Downward Closure ◽  
Bounded Size

The streams where multiple transactions are associated with the same key are prevalent in practice, e.g., a customer has multiple shopping records arriving at different time. Itemset frequency estimation on such streams is very challenging since sampling based methods, such as the popularly used reservoir sampling, cannot be used. In this article, we propose a novel k -Minimum Value (KMV) synopsis based method to estimate the frequency of itemsets over multi-transaction streams. First, we extract the KMV synopses for each item from the stream. Then, we propose a novel estimator to estimate the frequency of an itemset over the KMV synopses. Comparing to the existing estimator, our method is not only more accurate and efficient to calculate but also follows the downward-closure property. These properties enable the incorporation of our new estimator with existing frequent itemset mining (FIM) algorithm (e.g., FP-Growth) to mine frequent itemsets over multi-transaction streams. To demonstrate this, we implement a KMV synopsis based FIM algorithm by integrating our estimator into existing FIM algorithms, and we prove it is capable of guaranteeing the accuracy of FIM with a bounded size of KMV synopsis. Experimental results on massive streams show our estimator can significantly improve on the accuracy for both estimating itemset frequency and FIM compared to the existing estimators.

Data Mining Itemset of Big Data Using Pre-Processing Based on Mapreduce FrameWork with ETL Tools

2017 ◽  
Vol 2 (2) ◽  
pp. 57-62
Author(s):  
Padmanathan Anantharaman ◽  
H.V. Ramakrishan
Keyword(s):  
Big Data ◽  
Clustering Algorithm ◽  
Programming Model ◽  
Hybrid Approach ◽  
Processing Technique ◽  
Frequent Itemsets ◽  
Frequent Itemset ◽  
Frequent Itemset Mining ◽  
Itemset Mining ◽  
Dataset Size

As data volumes continue to grow, they quickly consume the capacity of data warehouses and application databases. Is your IT organization forced into costly upgrades to expensive databases and data warehouse hardware appliances and enormous amount of data is getting explored through Internet of Things (IoT) as technologies are advancing and people uses these technologies in day to day activities, this data is termed as Big Data having its characteristics and challenges. Frequent Itemset Mining algorithms are aimed to disclose frequent itemsets from transactional database but as the dataset size increases, it cannot be handled by traditional frequent itemset mining. MapReduce programming model solves the problem of large datasets but it has large communication cost which reduces execution efficiency. This proposed new pre-processed k-means technique applied on BigFIM algorithm. ClustBigFIM uses hybrid approach, clustering using k-means algorithm to generate Clusters from huge datasets and Apriori and Eclat to mine frequent itemsets from generated clusters using MapReduce programming model. Results shown that execution efficiency of ClustBigFIM algorithm is increased by applying k-means clustering algorithm before BigFIM algorithm as one of the pre-processing technique.

An Efficient Method for Frequent Itemset Mining on Temporal Data

2019 ◽  
pp. 558-568
Author(s):  
Fathima Sherin T K ◽  
Anish Kumar B.
Keyword(s):  
Data Mining ◽  
Computation Time ◽  
Frequent Itemsets ◽  
Frequent Itemset ◽  
Frequent Itemset Mining ◽  
Edge Density ◽  
Time Interval ◽  
Related Data ◽  
Itemset Mining ◽  
A Value

Frequent itemset mining (FIM) is a data mining idea with extracting frequent itemset from a database. Finding frequent itemsets in existing methods accept that datasets are static or steady and enlisted guidelines are pertinent all through the total dataset. In any case, this isn't the situation when information is temporal which contains time-related data that changes data mining results. Patterns may occur during all or at specific interims, to limit time interims, frequent itemset mining with time cube is proposed to manage time arranges in the mining technique. This is how patterns are perceived that happen occasionally, in a period interim, or both. Thus, this paper mostly centres around developing up a productive calculation to mine frequent itemsets and their related time interval from a value-based database by expanding from the earlier calculation dependent on support and density as another edge. Density is proposed to deal with the overestimated timespan issue and to ensure the authenticity of the patterns found. As an extension from the current framework, here the density rate and minimum threshold is dynamically generated which is user determined parameter previously. Likewise, an analysis concerning time is made between dataset with partitioning and without apportioning the dataset, which shows computation time is less on account of partitioning technique.

scholarly journals Enhancing the Performance of Large-scale Profitable Itemset Mining using Efficient Data Structures

2019 ◽  
Vol 8 (9) ◽  
pp. 1768-1772
Keyword(s):  
Data Structures ◽  
Large Scale ◽  
Frequent Itemsets ◽  
Frequent Itemset ◽  
Frequent Itemset Mining ◽  
Second Phase ◽  
Itemset Mining ◽  
Efficient Data ◽  
Profit Value ◽  
Efficient Data Structures

The process of extracting the most frequently bought items from a transactional database is termed as frequent itemset mining. Although it provides us with an idea of the best-selling itemsets, the method fails to identify the most profitable items from the database. It is not uncommon to have minimal intersection between frequent itemsets and profitable itemsets, and the process of extracting the most profitable itemsets is termed as Greater Profitable Itemset (GPI) mining. There have been various approaches to mine GPI in which [7] proposed a two-phased algorithm to optimize regeneration of GPI when the profit value of any item changes. This constituted of keeping track of the pruned items in the first phase and using it to efficiently regenerate GPI in the second phase. This paper proposes an enhancement to the way these changes are tracked by storing the pruned itemsets according to their constituent items, unlike the earlier algorithm that stored records iteration wise. By storing the itemsets according to their constituent items, we make sure that only the required items are being retrieved. In contrast, the earlier algorithm would fetch all the items pruned in any iteration, regardless of its relevance. By fetching only relevant itemset, the proposed method would significantly bring down the computational requirements.

Mining Frequent Weighted Itemsets without Storing Transaction IDs and Generating Candidates

2017 ◽  
Vol 25 (01) ◽  
pp. 111-144 ◽  
Author(s):  
Gangin Lee ◽  
Unil Yun ◽  
Keun Ho Ryu
Keyword(s):  
State Of The Art ◽  
Frequent Itemset ◽  
Experimental Results ◽  
Frequent Itemset Mining ◽  
Memory Usage ◽  
Tree Structures ◽  
Prefix Tree ◽  
Itemset Mining ◽  
Mining Methods ◽  
Mining Algorithms

Weighted itemset mining, which is one of the important areas in frequent itemset mining, is an approach for mining meaningful itemsets considering different importance or weights for each item in databases. Because of the merit of the weighted itemset mining, various related works have been studied actively. As one of the methods in the weighted itemset mining, FWI (Frequent Weighted Itemset) mining calculates weights of transactions from weights of items and then finds FWIs based on the transaction weights. However, previous FWI mining methods still have limitations in terms of runtime and memory usage performance. For this reason, in this paper, we propose two algorithms for mining FWIs more efficiently from databases with weights of items. In contrast to the previous approaches storing transaction IDs for mining FWIs, the proposed methods employ new types of prefix tree structures and mine these patterns more efficiently without storing any transaction ID. Through extensive experimental results in this paper, we show that the proposed algorithms outperform state-of-the-art FWI mining algorithms in terms of runtime, memory usage, and scalability.

scholarly journals An Intelligent Decision in Smart Systems Using A Weighted Frequent Itemset Mining Algorithm

2019 ◽  
pp. 526-536
Author(s):  
K. Lavanya ◽  
K. Triveni ◽  
K. Bala Mamatha ◽  
K. Meghana ◽  
Dr. G. Sanjay Gandhi
Keyword(s):  
Execution Time ◽  
Computation Time ◽  
Search Space ◽  
Frequent Itemsets ◽  
Frequent Itemset ◽  
Closure Property ◽  
Smart Systems ◽  
Intelligent Decision ◽  
Input Dataset ◽  
Downward Closure

Intelligent decision is the key technology of smart systems. Data mining technology has been playing an increasingly important role in decision making activities. The introduction of weight makes the weighted frequent itemsets not satisfy the downward closure property any longer. As a result, the search space of frequent itemsets cannot be narrowed according to downward closure property which leads to a poor time efficiency. In this paper, the weight judgment downward closure property for weighted frequent itemsets and the existence property of weighted frequent subsets are introduced and proved first. The Fuzzy-based WARM satisfies the downward closure property and prunes the insignificant rules by assigning the weight to the itemset. This reduces the computation time and execution time. This paper presents an Enhanced Fuzzy-based Weighted AssociationRuleMining(E-FWARM) algorithm for efficient mining of the frequent itemsets. The pre-filtering method is applied to the input dataset to remove the item having low variance. Data discretization is performed and E-FWARM is applied for mining the frequent itemsets. The experimental results show that the proposed E-FWARM algorithm yields maximum frequent items, association rules, accuracy and minimum execution time than the existing algorithms.

scholarly journals Frequent Itemset Mining in a Unique Scan using Transaction Database

2020 ◽  
Vol 9 (5) ◽  
pp. 612-617
Keyword(s):  
Vital Role ◽  
Frequent Itemsets ◽  
Frequent Itemset ◽  
Frequent Itemset Mining ◽  
Experimental Result ◽  
Apriori Algorithm ◽  
Itemset Mining ◽  
Pull Out ◽  
Benchmark Database ◽  
Transaction Database

In recent year, frequent Itemset Mining (FIM) has occurred as a vital role in data mining tasks. The search of FIM in a transactions data is discovered in this paper, pull out hidden pattern from transactions data. The main two limitation of the Apriori algorithm are undertaken, first, its scans the complete Databases at every passes to compute the supports of every itemset produced and secondly, the user defined responsive to variation of min_sup (minimum supports) thresholds. In this paper, proposed methodology called frequent Itemset Mining in unique Scan (FIMUS), needs a scan only one time of transaction databases to extract frequent itemsets. The generation of a static numbers of candidate Itemset is an exclusive feature, individually from the threshold of min_sup, which reduces the execution time for huge database. The proposed algorithm FIMUS is compared with Apriori algorithm using benchmark database for a dense databases. The experimental result confirms the scalability of FIMUS.

FPGA/GPU-based Acceleration for Frequent Itemsets Mining: A Comprehensive Review

2022 ◽  
Vol 54 (9) ◽  
pp. 1-35
Author(s):  
Lázaro Bustio-Martínez ◽  
René Cumplido ◽  
Martín Letras ◽  
Raudel Hernández-León ◽  
Claudia Feregrino-Uribe ◽  
...  
Keyword(s):  
Graphics Processing Units ◽  
Hardware Acceleration ◽  
Frequent Itemsets ◽  
Frequent Itemset ◽  
Frequent Itemset Mining ◽  
Comprehensive Review ◽  
Development Platform ◽  
Itemset Mining ◽  
Modern Development ◽  
Frequent Itemsets Mining

In data mining, Frequent Itemsets Mining is a technique used in several domains with notable results. However, the large volume of data in modern datasets increases the processing time of Frequent Itemset Mining algorithms, making them unsuitable for many real-world applications. Accordingly, proposing new methods for Frequent Itemset Mining to obtain frequent itemsets in a realistic amount of time is still an open problem. A successful alternative is to employ hardware acceleration using Graphics Processing Units (GPU) and Field Programmable Gates Arrays (FPGA). In this article, a comprehensive review of the state of the art of Frequent Itemsets Mining hardware acceleration is presented. Several approaches (FPGA and GPU based) were contrasted to show their weaknesses and strengths. This survey gathers the most relevant and the latest research efforts for improving the performance of Frequent Itemsets Mining regarding algorithms advances and modern development platforms. Furthermore, this survey organizes the current research on Frequent Itemsets Mining from the hardware perspective considering the source of the data, the development platform, and the baseline algorithm.

scholarly journals Frequent Itemset Mining and Multi-Layer Network-Based Analysis of RDF Databases

Mathematics ◽  
2021 ◽  
Vol 9 (4) ◽  
pp. 450
Author(s):  
Gergely Honti ◽  
János Abonyi
Keyword(s):  
Climate Change ◽  
Extraction Process ◽  
Knowledge Extraction ◽  
Frequent Itemset ◽  
Frequent Itemset Mining ◽  
Underlying Structure ◽  
Multilayer Network ◽  
Interdisciplinary Science ◽  
Academic Knowledge ◽  
Itemset Mining

Triplestores or resource description framework (RDF) stores are purpose-built databases used to organise, store and share data with context. Knowledge extraction from a large amount of interconnected data requires effective tools and methods to address the complexity and the underlying structure of semantic information. We propose a method that generates an interpretable multilayered network from an RDF database. The method utilises frequent itemset mining (FIM) of the subjects, predicates and the objects of the RDF data, and automatically extracts informative subsets of the database for the analysis. The results are used to form layers in an analysable multidimensional network. The methodology enables a consistent, transparent, multi-aspect-oriented knowledge extraction from the linked dataset. To demonstrate the usability and effectiveness of the methodology, we analyse how the science of sustainability and climate change are structured using the Microsoft Academic Knowledge Graph. In the case study, the FIM forms networks of disciplines to reveal the significant interdisciplinary science communities in sustainability and climate change. The constructed multilayer network then enables an analysis of the significant disciplines and interdisciplinary scientific areas. To demonstrate the proposed knowledge extraction process, we search for interdisciplinary science communities and then measure and rank their multidisciplinary effects. The analysis identifies discipline similarities, pinpointing the similarity between atmospheric science and meteorology as well as between geomorphology and oceanography. The results confirm that frequent itemset mining provides an informative sampled subsets of RDF databases which can be simultaneously analysed as layers of a multilayer network.

Sign in / Sign up

Export Citation Format

Share Document