scholarly journals Mining Profitable and Concise Patterns in Large-Scale Internet of Things Environments

2021 ◽  
Vol 2021 ◽  
pp. 1-12
Author(s):  
Jerry Chun-Wei Lin ◽  
Youcef Djenouri ◽  
Gautam Srivastava ◽  
Philippe Fournier-Viger
Keyword(s):  
Large Scale ◽  
State Of The Art ◽  
Frequency Factor ◽  
Market Analysis ◽  
Smart Devices ◽  
Mapreduce Framework ◽  
High Utilization ◽  
Mapreduce Model ◽  
High Utility ◽  
High Utility Itemsets

In recent years, HUIM (or a.k.a. high-utility itemset mining) can be seen as investigated in an extensive manner and studied in many applications especially in basket-market analysis and its relevant applications. Since current basket-market scenario also involves IoT equipment to collect information, i.e., sensor or smart devices, it is necessary to consider the mining of HUIs (or a.k.a. high-utility itemsets) in a large-scale database especially with IoT situations. First, a GA-based MapReduce model is presented in this work known as GMR-Miner for mining closed patterns with high utilization in large-scale databases. The k -means model is initially adopted to group transactions regarding their relevant correlation based on the frequency factor. A genetic algorithm (GA) is utilized in the developed MapReduce framework that can be used to explore the potential and possible candidates in a limited time. Also, the developed 3-tier MapReduce model can be easily deployed in Spark for the handlings of any database of large scale for knowledge discovery of closed patterns with high utilization. We created sets of extensive experimental environments for evaluating the results of the developed GMR-Miner compared to the well-known and state-of-the-art CLS-Miner. We present our in-depth results to show that the developed GMR-Miner outperforms CLS-Miner in many criteria, i.e., memory usage, scalability, and runtime.

Scalable Mining of High-Utility Sequential Patterns With Three-Tier MapReduce Model

2022 ◽  
Vol 16 (3) ◽  
pp. 1-26
Author(s):  
Jerry Chun-Wei Lin ◽  
Youcef Djenouri ◽  
Gautam Srivastava ◽  
Yuanfa Li ◽  
Philip S. Yu
Keyword(s):  
Large Scale ◽  
Pattern Mining ◽  
Sequential Pattern Mining ◽  
Main Memory ◽  
Frequent Itemset ◽  
Sequential Pattern ◽  
Sequential Patterns ◽  
Speed Up ◽  
Mapreduce Model ◽  
High Utility

High-utility sequential pattern mining (HUSPM) is a hot research topic in recent decades since it combines both sequential and utility properties to reveal more information and knowledge rather than the traditional frequent itemset mining or sequential pattern mining. Several works of HUSPM have been presented but most of them are based on main memory to speed up mining performance. However, this assumption is not realistic and not suitable in large-scale environments since in real industry, the size of the collected data is very huge and it is impossible to fit the data into the main memory of a single machine. In this article, we first develop a parallel and distributed three-stage MapReduce model for mining high-utility sequential patterns based on large-scale databases. Two properties are then developed to hold the correctness and completeness of the discovered patterns in the developed framework. In addition, two data structures called sidset and utility-linked list are utilized in the developed framework to accelerate the computation for mining the required patterns. From the results, we can observe that the designed model has good performance in large-scale datasets in terms of runtime, memory, efficiency of the number of distributed nodes, and scalability compared to the serial HUSP-Span approach.

scholarly journals MINING OF HIGH-UTILITY ITEMSETS WITH NEGATIVE UTILITY

2020 ◽  
Vol 1 (2) ◽  
pp. 44-47
Author(s):  
Tung N.T ◽  
Nguyen Le Van ◽  
Trinh Cong Nhut ◽  
Tran Van Sang
Keyword(s):  
State Of The Art ◽  
Upper Bounds ◽  
Itemset Mining ◽  
Novel Structure ◽  
Transactional Databases ◽  
Speed Up ◽  
Projection Techniques ◽  
High Utility ◽  
High Utility Itemsets ◽  
Mining Algorithms

The goal of the high-utility itemset mining task is to discover combinations of items that yield high profits from transactional databases. HUIM is a useful tool for retail stores to analyze customer behaviors. However, in the real world, items are found with both positive and negative utility values. To address this issue, we propose an algorithm named Modified Efficient High‐utility Itemsets mining with Negative utility (MEHIN) to find all HUIs with negative utility. This algorithm is an improved version of the EHIN algorithm. MEHIN utilizes 2 new upper bounds for pruning, named revised subtree and revised local utility. To reduce dataset scans, the proposed algorithm uses transaction merging and dataset projection techniques. An array‐based utility‐counting technique is also utilized to calculate upper‐bound efficiently. The MEHIN employs a novel structure called P-set to reduce the number of transaction scans and to speed up the mining process. Experimental results show that the proposed algorithms considerably outperform the state-of-the-art HUI-mining algorithms on negative utility in retail databases in terms of runtime.

scholarly journals An Innovative Procedure for Efficient Mining of Closed Top-K High Utility Itemsets

2020 ◽  
Vol 8 (5) ◽  
pp. 775-781
Keyword(s):  
Decision Making ◽  
Minimum Threshold ◽  
Entire Process ◽  
High Utilization ◽  
High Utility ◽  
High Utility Itemsets ◽  
Novel Strategy ◽  
Threshold Setting ◽  
Association Finding ◽  
Candidate Set

High utilization itemset (HUI) mining is the fastest growing ground in association finding between the items. It is a process of finding the itemsets with higher utility values which participates in profitable decision making. The generated HUIs reflect the frequency, importance, profit or the utility of the items present in the database. Proper minimum threshold setting is very difficult for the end users without the knowledge of the data present in the database. Minimum user threshold extracts more number of candidate sets . Higher user threshold gives less number of candidate sets and very few high utility itemsets . In both the cases, the process is inefficient. Some algorithms produce more number of candidate itemsets as HUIs. The set of HUIs may degrade the performance of the candidate set mining by increasing the storage and time when the database has very large number of transactions. The number of candidate itemsets involved in the generation of HUIs may also slow down the entire process. The proposed novel strategy for tapping top-k closed high utility itemsets out of the set of candidate sets addresses these issues . The user defined integer k is the needed count of HUIs to be extracted out of the quality itemsets. The algorithm does not require the user to set the minimum utilization threshold .The closure property is merged with the pruning process and improves the productivity. The results transparently show that the k-closed high utility itemsets generated using this algorithm are productive, profitable and very concise when compared with existing approaches.

Efficient Algorithm for Mining High Utility Pattern Considering Length Constraints

2019 ◽  
Vol 15 (3) ◽  
pp. 1-27
Author(s):  
Kuldeep Singh ◽  
Bhaskar Biswas
Keyword(s):  
Data Mining ◽  
Upper Bound ◽  
Efficient Algorithm ◽  
State Of The Art ◽  
Memory Usage ◽  
Important Data ◽  
Benchmark Datasets ◽  
Projection Techniques ◽  
High Utility ◽  
High Utility Itemsets

High utility itemset (HUI) mining is one of the popular and important data mining tasks. Several studies have been carried out on this topic, which often discovers a very large number of itemsets and rules, which reduces not only the efficiency but also the effectiveness of HUI mining. In order to increase the efficiency and discover more interesting HUIs, constraint-based mining plays an important role. To address this issue, the authors propose an algorithm to discover HUIs with length constraints named EHIL (Efficient High utility Itemsets with Length constraints) to decrease the number of HUIs by removing tiny itemsets. EHIL adopts two new upper bound named sub-tree and local utility for pruning and modify them by incorporating length constraints. To reduce the dataset scans, the proposed algorithm uses transaction merging and dataset projection techniques. The execution time improvements ranged from a modest five percent to two orders of magnitude across benchmark datasets. The memory usage is up to twenty-eight times less than state-of-the-art algorithm FHM+.

scholarly journals Mining High Utility Itemsets Based on Pattern Growth without Candidate Generation

Mathematics ◽  
2020 ◽  
Vol 9 (1) ◽  
pp. 35
Author(s):  
Yiwei Liu ◽  
Le Wang ◽  
Lin Feng ◽  
Bo Jin
Keyword(s):  
State Of The Art ◽  
Research Topic ◽  
Performance Gap ◽  
Utility Values ◽  
Original Dataset ◽  
Pattern Growth ◽  
Active Research ◽  
High Utility ◽  
High Utility Itemsets ◽  
Mining Algorithms

Mining high utility itemsets (HUIs) has been an active research topic in data mining in recent years. Existing HUI mining algorithms typically take two steps: generating candidates and identifying utility values of these candidate itemsets. The performance of these algorithms depends on the efficiency of both steps, both of which are usually time-consuming. In this study, we propose an efficient pattern-growth based HUI mining algorithm, called tail-node tree-based high-utility itemset (TNT-HUI) mining. This algorithm avoids the time-consuming candidate generation step, as well as the need of scanning the original dataset multiple times for exact utility values, as supported by a novel tree structure, named the tail-node tree (TN-Tree). The performance of TNT-HUI was evaluated in comparison with state-of-the-art benchmark methods on different datasets. Experimental results showed that TNT-HUI outperformed benchmark algorithms in both execution time and memory use by orders of magnitude. The performance gap is larger for denser datasets and lower thresholds.

scholarly journals Distributed Pareto Optimization for Subset Selection

2018 ◽  
Author(s):  
Chao Qian ◽  
Guiying Li ◽  
Chao Feng ◽  
Ke Tang
Keyword(s):  
Real World ◽  
Large Scale ◽  
State Of The Art ◽  
Subset Selection ◽  
Data Sets ◽  
Mapreduce Framework ◽  
Real World Data ◽  
Real World Applications ◽  
Approximation Guarantee ◽  
Better Than

The subset selection problem that selects a few items from a ground set arises in many applications such as maximum coverage, influence maximization, sparse regression, etc. The recently proposed POSS algorithm is a powerful approximation solver for this problem. However, POSS requires centralized access to the full ground set, and thus is impractical for large-scale real-world applications, where the ground set is too large to be stored on one single machine. In this paper, we propose a distributed version of POSS (DPOSS) with a bounded approximation guarantee. DPOSS can be easily implemented in the MapReduce framework. Our extensive experiments using Spark, on various real-world data sets with size ranging from thousands to millions, show that DPOSS can achieve competitive performance compared with the centralized POSS, and is almost always better than the state-of-the-art distributed greedy algorithm RandGreeDi.

scholarly journals HUIL-TN & HUI-TN: Mining high utility itemsets based on pattern-growth

PLoS ONE ◽  
2021 ◽  
Vol 16 (3) ◽  
pp. e0248349
Author(s):  
Le Wang ◽  
Shui Wang
Keyword(s):  
Data Mining ◽  
State Of The Art ◽  
Research Topic ◽  
Running Time ◽  
Original Dataset ◽  
Pattern Growth ◽  
Active Research ◽  
High Utility ◽  
High Utility Itemsets ◽  
Mining Algorithms

In recent years, high utility itemsets (HUIs) mining has been an active research topic in data mining. In this study, we propose two efficient pattern-growth based HUI mining algorithms, called High Utility Itemset based on Length and Tail-Node tree (HUIL-TN) and High Utility Itemset based on Tail-Node tree (HUI-TN). These two algorithms avoid the time-consuming candidate generation stage and the need of scanning the original dataset multiple times for exact utility values. A novel tree structure, named tail-node tree (TN-tree) is proposed as a key element of our algorithms to maintain complete utililty-information of existing itemsets of a dataset. The performance of HUIL-TN and HUI-TN was evaluated against state-of-the-art reference methods on various datasets. Experimental results showed that our algorithms exceed or close to the best performance on all datasets in terms of running time, while other algorithms can only excel in certain types of dataset. Scalability tests were also performed and our algorithms obtained the flattest curves among all competitors.

scholarly journals Depth Impurity Pruned Strategies for Extracting High Utility Itemsets

2018 ◽  
Vol 7 (3.4) ◽  
pp. 52
Author(s):  
K Santhi ◽  
B Valarmathi ◽  
T Chellatamilan
Keyword(s):  
State Of The Art ◽  
Database Mining ◽  
Huge Amount ◽  
Utility Mining ◽  
Organizational Behaviour ◽  
Mining Technique ◽  
Transaction Database ◽  
High Utility ◽  
High Utility Itemsets ◽  
Time And Space Complexity

Normally in a transaction database mining high utility itemsets indicates to the location of itemsets which is causing high utility like benefits. In spite of the fact that various important calculations have been proposed as of late, they bring about the issue of generating a huge amount of itemsets for mining to discover HUI. Mining is reduced by such an extended quantity as far as execution time and space complexity. When the database contains large amount of transactions, this condition may turn into mediocre. In this research paper, we account this concern by offering a state-of-the-art calculation named Depth Impurity Quality Index Pruned strategies which considers the complexity of sub-trees to more efficiently identify high-utility itemsets. It is an collection of common itemset which are used for mining and is significantly harder, inflexible. This is imputable to the absence of intrinsic organizational behaviour of  HUI which could have worked. This paper suggests a high utility mining technique which make use of novel pruning approaches.The experimental outcomes disclose that the proposed method is exceptionally viable in killing unhopeful applicants  in the   database transactions.

ETKDS: An efficient algorithm of Top-K high utility itemsets mining over data streams under sliding window model

2021 ◽  
pp. 1-22
Author(s):  
Haodong Cheng ◽  
Meng Han ◽  
Ni Zhang ◽  
Le Wang ◽  
Xiaojuan Li
Keyword(s):  
Data Streams ◽  
Efficient Algorithm ◽  
Large Scale ◽  
Search Space ◽  
Real Time Processing ◽  
Pruning Strategy ◽  
Complex Construction ◽  
High Utility ◽  
High Utility Itemsets ◽  
Mining Algorithms

The researcher proposed the concept of Top-K high-utility itemsets mining over data streams. Users directly specify the number K of high-utility itemsets they wish to obtain for mining with no need to set a minimum utility threshold. There exist some problems in current Top-K high-utility itemsets mining algorithms over data streams including the complex construction process of the storage structure, the inefficiency of threshold raising strategies and utility pruning strategies, and large scale of the search space, etc., which still can not meet the requirement of real-time processing over data streams with limited time and memory constraints. To solve this problem, this paper proposes an efficient algorithm based on dataset projection for mining Top-K high-utility itemsets from a data stream. A data structure CIUDataListSW is also proposed, which stores the position of the item in the transaction to effectively obtain the initial projected dataset of the item. In order to improve the projection efficiency, this paper innovates a new reorganization technology for projected transactions in common batches to maintain the sort order of transactions in the process of dataset projection. Dual pruning strategy and transaction merging mechanism are also used to further reduce search space and dataset scanning costs. In addition, based on the proposed CUDH S W structure, an efficient threshold raising strategy CUD is used, and a new threshold raising strategy CUDCB is designed to further shorten the mining time. Experimental results show that the algorithm has great advantages in running time and memory consumption, and it is especially suitable for the mining of high-utility itemsets of dense datasets.

Sign in / Sign up

Export Citation Format

Share Document