Exploring Calendar-Based Pattern Mining in Data Streams

2010 ◽  
pp. 342-360
Author(s):  
Rodrigo Salvador Monteiro ◽  
Geraldo Zimbrão ◽  
Holger Schwarz ◽  
Bernhard Mitschang ◽  
Jano Moreira de Souza
Keyword(s):  
Data Warehouse ◽  
Data Streams ◽  
Data Stream ◽  
Pattern Mining ◽  
A Priori ◽  
Frequent Itemsets ◽  
Detailed Data ◽  
Series Of Experiments ◽  
Working Day

Calendar-based pattern mining aims at identifying patterns on specific calendar partitions. Potential calendar partitions are for example: every Monday, every first working day of each month, every holiday. Providing flexible mining capabilities for calendar-based partitions is especially challenging in a data stream scenario. The calendar partitions of interest are not known a priori and at each point in time only a subset of the detailed data is available. The authors show how a data warehouse approach can be applied to this problem. The data warehouse that keeps track of frequent itemsets holding on different partitions of the original stream has low storage requirements. Nevertheless, it allows to derive sets of patterns that are complete and precise. Furthermore, the authors demonstrate the effectiveness of their approach by a series of experiments.

An Efficient Frequent Patterns Mining Algorithm over Data Streams Based on FPD-Graph

2012 ◽  
Vol 433-440 ◽  
pp. 4457-4462 ◽  
Author(s):  
Jun Shan Tan ◽  
Zhu Fang Kuang ◽  
Guo Gui Yang
Keyword(s):  
Data Streams ◽  
Data Stream ◽  
Pattern Mining ◽  
Frequent Pattern Mining ◽  
Frequent Pattern ◽  
Frequent Patterns ◽  
Data Generation ◽  
Experiment Data ◽  
Mining Algorithm ◽  
Head Node

The design of synopses structure is an important issue of frequent patterns mining over data stream. A data stream synopses structure FPD-Graph which is based on directed graph is proposed in this paper. The FPD-Graph contains list head node FPDG-Head and list node FPDG-Node. The operations of FPD-Graph consist of insert operation and deletion operation. A frequent pattern mining algorithm DGFPM based on sliding window over data stream is proposed in this paper. The IBM synthesizes data generation which output customers shopping a data are adopted as experiment data. The DGFPM algorithm not only has high precision for mining frequent patterns, but also has low processing time.

An Approximate Approach for Maintaining Recent Occurrences of Itemsets in a Sliding Window over Data Streams

2010 ◽  
pp. 308-327
Author(s):  
Jia-Ling Koh ◽  
Shu-Ning Shin ◽  
Yuan-Bin Don
Keyword(s):  
Data Streams ◽  
Data Stream ◽  
Traditional Approach ◽  
Experimental Studies ◽  
Dynamic Environment ◽  
Sliding Window ◽  
Fixed Time ◽  
Frequent Itemsets ◽  
Embedded Knowledge ◽  
Data Elements

Recently, the data stream, which is an unbounded sequence of data elements generated at a rapid rate, provides a dynamic environment for collecting data sources. It is likely that the embedded knowledge in a data stream will change quickly as time goes by. Therefore, catching the recent trend of data is an important issue when mining frequent itemsets over data streams. Although the sliding window model proposed a good solution for this problem, the appearing information of patterns within a sliding window has to be maintained completely in the traditional approach. For estimating the approximate supports of patterns within a sliding window, the frequency changing point (FCP) method is proposed for monitoring the recent occurrences of itemsets over a data stream. In addition to a basic design proposed under the assumption that exact one transaction arrives at each time point, the FCP method is extended for maintaining recent patterns over a data stream where a block of various numbers of transactions (including zero or more transactions) is inputted within a fixed time unit. Accordingly, the recently frequent itemsets or representative patterns are discovered from the maintained structure approximately. Experimental studies demonstrate that the proposed algorithms achieve high true positive rates and guarantees no false dismissal to the results yielded. A theoretic analysis is provided for the guarantee. In addition, the authors’ approach outperforms the previously proposed method in terms of reducing the run-time memory usage significantly.

scholarly journals Mining Recent Maximal Frequent Itemsets Over Data Streams with Sliding Window

2018 ◽  
Vol 16 (6) ◽  
pp. 961-969 ◽  
Author(s):  
Saihua Cai ◽  
Shangbo Hao ◽  
Ruizhi Sun ◽  
Gang Wu
Keyword(s):  
Data Streams ◽  
Sliding Window ◽  
Frequent Itemsets ◽  
Memory Usage ◽  
Time Cost ◽  
Improved Method ◽  
Huge Number ◽  
Frequent Itemsets Mining ◽  
Series Of Experiments ◽  
Maximal Frequent Itemsets

Abstract: The huge number of data streams makes it impossible to mine recent frequent itemsets. Due to the maximal frequent itemsets can perfectly imply all the frequent itemsets and the number is much smaller, therefore, the time cost and the memory usage for mining maximal frequent itemsets are much more efficient. This paper proposes an improved method called Recent Maximal Frequent Itemsets Mining (RMFIsM) to mine recent maximal frequent itemsets over data streams with sliding window. The RMFIsM method uses two matrixes to store the information of data streams, the first matrix stores the information of each transaction and the second one stores the frequent 1-itemsets. The frequent p-itemsets are mined with “extension” process of frequent 2-itemsets, and the maximal frequent itemsets are obtained by deleting the sub-itemsets of long frequent itemsets. Finally, the performance of the RMFIsM method is conducted by a series of experiments, the results show that the proposed RMFIsM method can mine recent maximal frequent itemsets efficiently

scholarly journals WMFP-Outlier: An Efficient Maximal Frequent-Pattern-Based Outlier Detection Approach for Weighted Data Streams

2019 ◽  
Vol 48 (4) ◽  
pp. 505-521 ◽  
Author(s):  
Saihua Cai ◽  
Qian Li ◽  
Sicong Li ◽  
Gang Yuan ◽  
Ruizhi Sun
Keyword(s):  
Outlier Detection ◽  
Data Streams ◽  
Data Stream ◽  
Pattern Mining ◽  
Frequent Pattern ◽  
Frequent Patterns ◽  
Time Cost ◽  
Detection Approach ◽  
Maximal Frequent Pattern ◽  
Weighted Data

Since outliers are the major factors that affect accuracy in data science, many outlier detection approaches have been proposed for effectively identifying the implicit outliers from static datasets, thereby improving the reliability of the data. In recent years, data streams have been the main form of data, and the data elements in a data stream are not always of equal importance. However, the existing outlier detection approaches do not consider the weight conditions; hence, these methods are not suitable for processing weighted data streams. In addition, the traditional pattern-based outlier detection approaches incur a high time cost in the outlier detection phase. Aiming at overcoming these problems, this paper proposes a two-phase pattern-based outlier detection approach, namely, WMFP-Outlier, for effectively detecting the implicit outliers from a weighted data stream, in which the maximal frequent patterns are used instead of the frequent patterns to accelerate the process of outlier detection. In the process of maximal frequent-pattern mining, the anti-monotonicity property and MFP-array structure are used to accelerate the mining operation. In the process of outlier detection, three deviation indices are designed for measuring the degree of abnormality of each transaction, and the transactions with the highest degrees of abnormality are judged as outliers. Last, several experimental studies are conducted on a synthetic dataset to evaluate the performance of the proposed WMFP-Outlier approach. The results demonstrate that the accuracy of the WMFP-Outlier approach is higher compared to the existing pattern-based outlier detection approaches, and the time cost of the outlier detection phase of WMFP-Outlier is lower than those of the other four compared pattern-based outlier detection approaches.

A Novel Algorithm for Predicting Valuable Items in Data Streams

2015 ◽  
Vol 3 (2) ◽  
Author(s):  
S. Vijayarani Mohan
Keyword(s):  
Data Mining ◽  
Data Streams ◽  
Data Stream ◽  
Pattern Mining ◽  
Research Work ◽  
Frequent Pattern Mining ◽  
Frequent Pattern ◽  
Frequent Items ◽  
And Performance ◽  
Transactional Data

A data stream is a real time, continuous, structured sequence of data items. Mining data stream is the process of extracting knowledge from continuous arrival of rapid data records. Data can arrive fast and in continuous manner. It is very difficult to perform mining process. Normally, stream mining algorithms are designed to scan the database only once, and it is a complicated task to extract the knowledge from the database by a single scan. Data streams are a computational challenge to data mining problems because of the additional algorithmic constraints created by the large volume of data. Popular data mining techniques namely clustering, classification, and frequent pattern mining are applied to data streams for extracting the knowledge. This research work mainly concentrates on how to predict the valuable items which are found in a transactional data of a data stream. In the literature, most of the researchers have discussed about how the frequent items are mined from the data streams. This research work helps to predict the valuable items in a transactional data. Frequent item mining is defined as finding the items which occur frequently, i.e. the occurrence of items above the given threshold is considered as frequent items. Valuable item mining is nothing but finding the costliest or most valuable items of a database. Predicting this information helps businesses to know about the sales details about the valuable items which guide to make crucial decisions, such as catalogue drawing, cross promotion, end user shopping, and performance scrutiny. In this research work, a new algorithm namely VIM (Valuable Item Mining) is proposed for finding the valuable items in data streams. The performance of this algorithm is analysed by using the factors, number of valuable items discovered, and execution time.

Replication of the Superstition and Performance Study by

2014 ◽  
Vol 45 (3) ◽  
pp. 239-245 ◽  
Author(s):  
Robert J. Calin-Jageman ◽  
Tracy L. Caldwell
Keyword(s):  
Task Difficulty ◽  
Statistical Power ◽  
Meta Analysis ◽  
A Priori ◽  
Significant Heterogeneity ◽  
Performance Study ◽  
Improve Performance ◽  
Research Designs ◽  
Series Of Experiments ◽  
And Performance

A recent series of experiments suggests that fostering superstitions can substantially improve performance on a variety of motor and cognitive tasks ( Damisch, Stoberock, & Mussweiler, 2010 ). We conducted two high-powered and precise replications of one of these experiments, examining if telling participants they had a lucky golf ball could improve their performance on a 10-shot golf task relative to controls. We found that the effect of superstition on performance is elusive: Participants told they had a lucky ball performed almost identically to controls. Our failure to replicate the target study was not due to lack of impact, lack of statistical power, differences in task difficulty, nor differences in participant belief in luck. A meta-analysis indicates significant heterogeneity in the effect of superstition on performance. This could be due to an unknown moderator, but no effect was observed among the studies with the strongest research designs (e.g., high power, a priori sampling plan).

Sign in / Sign up

Export Citation Format

Share Document