A New Similarity Metric for Sequential Data

2010 ◽  
Vol 6 (4) ◽  
pp. 16-32 ◽  
Author(s):  
Pradeep Kumar ◽  
Bapi S. Raju ◽  
P. Radha Krishna
Keyword(s):  
Data Mining ◽  
Similarity Measure ◽  
Web Mining ◽  
Clustering Algorithms ◽  
Sequential Data ◽  
Similarity Metric ◽  
Benchmark Datasets ◽  
Similarity Preserving ◽  
Sequential Nature ◽  
Classification And Clustering

In many data mining applications, both classification and clustering algorithms require a distance/similarity measure. The central problem in similarity based clustering/classification comprising sequential data is deciding an appropriate similarity metric. The existing metrics like Euclidean, Jaccard, Cosine, and so forth do not exploit the sequential nature of data explicitly. In this paper, the authors propose a similarity preserving function called Sequence and Set Similarity Measure (S3M) that captures both the order of occurrence of items in sequences and the constituent items of sequences. The authors demonstrate the usefulness of the proposed measure for classification and clustering tasks. Experiments were conducted on benchmark datasets, that is, DARPA’98 and msnbc, for classification task in intrusion detection and clustering task in web mining domains. Results show the usefulness of the proposed measure.

A New Similarity Metric for Sequential Data

2011 ◽  
pp. 233-248 ◽  
Author(s):  
Pradeep Kumar ◽  
Bapi S. Raju ◽  
P. Radha Krishna
Keyword(s):  
Data Mining ◽  
Similarity Measure ◽  
Web Mining ◽  
Clustering Algorithms ◽  
Sequential Data ◽  
Similarity Metric ◽  
Benchmark Datasets ◽  
Similarity Preserving ◽  
Sequential Nature ◽  
Classification And Clustering

In many data mining applications, both classification and clustering algorithms require a distance/similarity measure. The central problem in similarity based clustering/classification comprising sequential data is deciding an appropriate similarity metric. The existing metrics like Euclidean, Jaccard, Cosine, and so forth do not exploit the sequential nature of data explicitly. In this chapter, the authors propose a similarity preserving function called Sequence and Set Similarity Measure (S3M) that captures both the order of occurrence of items in sequences and the constituent items of sequences. The authors demonstrate the usefulness of the proposed measure for classification and clustering tasks. Experiments were conducted on benchmark datasets, that is, DARPA’98 and msnbc, for classification task in intrusion detection and clustering task in web mining domains. Results show the usefulness of the proposed measure.

SeqPAM

2008 ◽  
pp. 17-38 ◽  
Author(s):  
Pradeep Kumar Kumar ◽  
Raju S. Bapi ◽  
P. Radha Krishna
Keyword(s):  
Similarity Measure ◽  
Clustering Algorithm ◽  
Clustering Algorithms ◽  
Similarity Measures ◽  
Sequential Data ◽  
Cluster Validation ◽  
Web Personalization ◽  
Similarity Preserving ◽  
Validation Technique ◽  
The Web

With the growth in the number of web users and necessity for making information available on the web, the problem of web personalization has become very critical and popular. Developers are trying to customize a web site to the needs of specific users with the help of knowledge acquired from user navigational behavior. Since user page visits are intrinsically sequential in nature, efficient clustering algorithms for sequential data are needed. In this paper, we introduce a similarity preserving function called sequence and set similarity measure S3M that captures both the order of occurrence of page visits as well as the content of pages. We conducted pilot experiments comparing the results of PAM, a standard clustering algorithm, with two similarity measures: Cosine and S3M. The goodness of the clusters resulting from both the measures was computed using a cluster validation technique based on average levensthein distance. Results on pilot dataset established the effectiveness of S3M for sequential data. Based on these results, we proposed a new clustering algorithm, SeqPAM for clustering sequential data. We tested the new algorithm on two datasets namely, cti and msnbc datasets. We provided recommendations for web personalization based on the clusters obtained from SeqPAM for msnbc dataset.

SeqPAM

2008 ◽  
pp. 3285-3307
Author(s):  
Pradeep Kumar ◽  
Raju S. Bapi ◽  
P. Radha Krishna
Keyword(s):  
Similarity Measure ◽  
Clustering Algorithm ◽  
Clustering Algorithms ◽  
Similarity Measures ◽  
Sequential Data ◽  
Cluster Validation ◽  
Web Personalization ◽  
Similarity Preserving ◽  
Validation Technique ◽  
The Web

With the growth in the number of Web users and necessity for making information available on the Web, the problem of Web personalization has become very critical and popular. Developers are trying to customize a Web site to the needs of specific users with the help of knowledge acquired from user navigational behavior. Since user page visits are intrinsically sequential in nature, efficient clustering algorithms for sequential data are needed. In this chapter, we introduce a similarity preserving function called sequence and set similarity measure S3M that captures both the order of occurrence of page visits as well as the content of pages. We conducted pilot experiments comparing the results of PAM, a standard clustering algorithm, with two similarity measures: Cosine and S3M. The goodness of the clusters resulting from both the measures was computed using a cluster validation technique based on average levensthein distance. Results on pilot dataset established the effectiveness of S3M for sequential data. Based on these results, we proposed a new clustering algorithm, SeqPAM for clustering sequential data. We tested the new algorithm on two datasets namely, cti and msnbc datasets. We provided recommendations for Web personalization based on the clusters obtained from SeqPAM for msnbc dataset.

A Hybrid Validity Index to Determine K Parameter Value of k-Means Algorithm for Time Series Clustering

2021 ◽  
pp. 1-22
Author(s):  
Fatma Ozge Ozkok ◽  
Mete Celik
Keyword(s):  
Time Series ◽  
Clustering Algorithms ◽  
Real Life ◽  
Internal Validity ◽  
The Other ◽  
Sequential Data ◽  
Data Mining Technique ◽  
Validity Index ◽  
Number Of Clusters ◽  
Benchmark Datasets

Time series is a set of sequential data point in time order. The sizes and dimensions of the time series datasets are increasing day by day. Clustering is an unsupervised data mining technique that groups objects based on their similarities. It is used to analyze various datasets, such as finance, climate, and bioinformatics datasets. [Formula: see text]-means is one of the most used clustering algorithms. However, it is challenging to determine the value of [Formula: see text] parameter, which is the number of clusters. One of the most used methods to determine the number of clusters (such as [Formula: see text]) is cluster validity indexes. Several internal and external validity indexes are used to find suitable cluster numbers based on characteristics of datasets. In this study, we propose a hybrid validity index to determine the value of [Formula: see text] parameter of [Formula: see text]-means algorithm. The proposed hybrid validity index comprises four internal validity indexes, such as Dunn, Silhouette, C index, and Davies–Bouldin indexes. The proposed method was applied to nine real-life finance and benchmarks time series datasets. The financial dataset was obtained from Yahoo Finance, consisting of daily closing data of stocks. The other eight benchmark datasets were obtained from UCR time series classification archive. Experimental results showed that the proposed hybrid validity index is promising for finding the suitable number of clusters with respect to the other indexes for clustering time-series datasets.

scholarly journals Mining of high dimensional data using enhanced clustering approach

2018 ◽  
Vol 7 (2.21) ◽  
pp. 291
Author(s):  
S Sivakumar ◽  
Kumar Narayanan ◽  
Swaraj Paul Chinnaraju ◽  
Senthil Kumar Janahan
Keyword(s):  
Data Mining ◽  
Similarity Measure ◽  
Dimensional Space ◽  
Clustering Algorithms ◽  
High Dimensional ◽  
Projected Clustering ◽  
Low Dimensionality ◽  
Clustering Approach ◽  
Numerous Data ◽  
Dimension Space

Extraction of useful data from a set is known as Data mining. Clustering has top information mining process it supposed to help an individual, divide and recognize numerous data from records inside group consistent with positive similarity measure. Clustering excessive dimensional data has been a chief undertaking. Maximum present clustering algorithms have been inefficient if desired similarity is computed among statistics factors inside the complete dimensional space. Varieties of projected clustering algorithms were counseled for addressing those problems. However many of them face problems whilst clusters conceal in some space with low dimensionality. These worrying situations inspire our system to endorse a look at partitional distance primarily based projected clustering set of rules. The aimed paintings is successfully deliberate for projects clusters in excessive huge dimension space via adapting the stepped forward method in k Mediods set of pointers. The main goal for second one gadget is to take away outliers, at the same time as the 1/3 method will find clusters in numerous spaces. The (clustering) technique is based on the adequate Mediods set of guidelines, an excess distance managed to set of attributes everywhere values are dense.

Advances in Classification of Sequence Data

2008 ◽  
pp. 143-174 ◽  
Author(s):  
Pradeep Kumar ◽  
P. Radha Krishna ◽  
Raju S. Bapi ◽  
T. M. Padmaja
Keyword(s):  
Data Mining ◽  
Web Mining ◽  
Sequence Data ◽  
Data Classification ◽  
Knowledge Discovery In Databases ◽  
Sequential Data ◽  
Stream Data ◽  
Stream Data Mining ◽  
Discrete Sequence ◽  
Mining Sequence

In recent years, advanced information systems have enabled collection of increasingly large amounts of data that are sequential in nature. To analyze huge amounts of sequential data, the interdisciplinary field of Knowledge Discovery in Databases (KDD) is very useful. The most important step within the process of KDD is data mining, which is concerned with the extraction of the valid patterns. Recent research focus in data mining includes stream data mining, sequence data mining, web mining, text mining, visual mining, multimedia mining and multi-relational data mining. Sequence data may be discrete or continuous in nature. Most of the research on discrete sequence data concentrated on the discovery of frequently occurring patterns. However, comparatively less amount of work has been carried out in the area of discrete sequence data classification. In this chapter, data taxonomy is introduced with a review of the state of art for sequence data classification. The usefulness of embedding partial subsequence information extracted using sliding window technique into traditional classifier like kNN has been demonstrated. kNN has been tested with various vector based distance/similarity metrics. Further, with the use of S3M similarity metric, the full subsequence information embedded in the data sequences is extracted. The experimental data taken is DARPA’98 IDS benchmark dataset collected from UCIML dataset repository. The chapter closes by pointing out various application areas of sequence data and also the open issues in sequence data classification problem.

scholarly journals Integrated Algorithm for Unsupervised Data Clustering Problems in Data Mining

2019 ◽  
Vol 54 (5) ◽  
Author(s):  
Nibras Othman Abdul Wahid ◽  
Saif Aamer Fadhil ◽  
Noor Abbood Jasim
Keyword(s):  
Data Mining ◽  
Data Clustering ◽  
Clustering Algorithms ◽  
Genetic Operators ◽  
Way Of Life ◽  
Fundamental Parameters ◽  
Benchmark Datasets ◽  
Key Issues ◽  
Clustering Data ◽  
Lion Optimization Algorithm

Unsupervised data clustering investigation is a standout amongst the most valuable apparatuses and an enlightening undertaking in data mining that looks to characterize homogeneous gatherings of articles depending on likeness and is utilized in numerous applications. One of the key issues in data mining is clustering data that have pulled in much consideration. One of the famous clustering algorithms is K-means clustering that has been effectively connected to numerous issues. Scientists recommended enhancing the nature of K-means, optimization algorithms were hybridized. In this paper, a heuristic calculation, Lion Optimization Algorithm (LOA), and Genetic Algorithm (GA) were adjusted for K-Means data clustering by altering the fundamental parameters of LOA calculation, which is propelled from the characteristic enlivened calculations. The uncommon way of life of lions and their participation attributes has been the essential inspiration for the advancement of this improvement calculation. The GA is utilized when it is required to reallocate the clusters using the genetic operators, crossover, and mutation. The outcomes of the examination of this calculation mirror the capacity of this methodology in clustering examination on the number of benchmark datasets from UCI Machine Learning Repository.

Handling WSD using Hierarchical Clustering Algorithm with sentences

2018 ◽  
pp. 83-88
Author(s):  
Mohana Priya K ◽  
Pooja Ragavi S ◽  
Krishna Priya G
Keyword(s):  
Hierarchical Clustering ◽  
Similarity Measure ◽  
Clustering Algorithm ◽  
Clustering Algorithms ◽  
Cosine Similarity Measure ◽  
Hierarchical Clustering Algorithm ◽  
Multiple Levels ◽  
Pos Tagger ◽  
Sentence Clustering ◽  
The Right

Clustering is the process of grouping objects into subsets that have meaning in the context of a particular problem. It does not rely on predefined classes. It is referred to as an unsupervised learning method because no information is provided about the "right answer" for any of the objects. Many clustering algorithms have been proposed and are used based on different applications. Sentence clustering is one of best clustering technique. Hierarchical Clustering Algorithm is applied for multiple levels for accuracy. For tagging purpose POS tagger, porter stemmer is used. WordNet dictionary is utilized for determining the similarity by invoking the Jiang Conrath and Cosine similarity measure. Grouping is performed with respect to the highest similarity measure value with a mean threshold. This paper incorporates many parameters for finding similarity between words. In order to identify the disambiguated words, the sense identification is performed for the adjectives and comparison is performed. semcor and machine learning datasets are employed. On comparing with previous results for WSD, our work has improvised a lot which gives a percentage of 91.2%

Low-complexity fuzzy relational clustering algorithms for Web mining

2001 ◽  
Vol 9 (4) ◽  
pp. 595-607 ◽  
Author(s):  
R. Krishnapuram ◽  
A. Joshi ◽  
O. Nasraoui ◽  
L. Yi
Keyword(s):  
Web Mining ◽  
Clustering Algorithms ◽  
Low Complexity ◽  
Relational Clustering
Sign in / Sign up

Export Citation Format

Share Document