PG-Join: Proximity Graph Based String Similarity Joins

2011 ◽  
pp. 274-292
Author(s):  
Michail Kazimianec ◽  
Nikolaus Augsten
Keyword(s):  
String Similarity ◽  
Proximity Graph ◽  
Similarity Joins

String similarity joins

2014 ◽  
Vol 7 (8) ◽  
pp. 625-636 ◽  
Author(s):  
Yu Jiang ◽  
Guoliang Li ◽  
Jianhua Feng ◽  
Wen-Syan Li
Keyword(s):  
String Similarity ◽  
Similarity Joins

scholarly journals Trie-join: a trie-based method for efficient string similarity joins

2011 ◽  
Vol 21 (4) ◽  
pp. 437-461 ◽  
Author(s):  
Jianhua Feng ◽  
Jiannan Wang ◽  
Guoliang Li
Keyword(s):  
String Similarity ◽  
Similarity Joins

Top-k String Similarity Joins

2020 ◽  
Author(s):  
Shuyao Qi ◽  
Panagiotis Bouros ◽  
Nikos Mamoulis
Keyword(s):  
String Similarity ◽  
Similarity Joins

scholarly journals Towards a unified framework for string similarity joins

2019 ◽  
Vol 12 (11) ◽  
pp. 1289-1302 ◽  
Author(s):  
Pengfei Xu ◽  
Jiaheng Lu
Keyword(s):  
Unified Framework ◽  
String Similarity ◽  
Similarity Joins

A Novel PSS Stemmer for String Similarity Joins

2017 ◽  
Author(s):  
P. Selvaramalakshmi ◽  
S. Hari Ganesh ◽  
J. James Manoharan
Keyword(s):  
String Similarity ◽  
Similarity Joins

scholarly journals Handling data-skewness in character based string similarity join using Hadoop

2020 ◽  
Vol ahead-of-print (ahead-of-print) ◽  
Author(s):  
Kanak Meena ◽  
Devendra K. Tayal ◽  
Oscar Castillo ◽  
Amita Jain
Keyword(s):  
Scientific Data ◽  
Distribution Law ◽  
Similarity Join ◽  
String Similarity ◽  
Zipf Distribution ◽  
Imbalance Problem ◽  
Data Skewness ◽  
Pair Generation ◽  
Set Up ◽  
Similarity Joins

The scalability of similarity joins is threatened by the unexpected data characteristic of data skewness. This is a pervasive problem in scientific data. Due to skewness, the uneven distribution of attributes occurs, and it can cause a severe load imbalance problem. When database join operations are applied to these datasets, skewness occurs exponentially. All the algorithms developed to date for the implementation of database joins are highly skew sensitive. This paper presents a new approach for handling data-skewness in a character- based string similarity join using the MapReduce framework. In the literature, no such work exists to handle data skewness in character-based string similarity join, although work for set based string similarity joins exists. Proposed work has been divided into three stages, and every stage is further divided into mapper and reducer phases, which are dedicated to a specific task. The first stage is dedicated to finding the length of strings from a dataset. For valid candidate pair generation, MR-Pass Join framework has been suggested in the second stage. MRFA concepts are incorporated for string similarity join, which is named as “MRFA-SSJ” (MapReduce Frequency Adaptive – String Similarity Join) in the third stage which is further divided into four MapReduce phases. Hence, MRFA-SSJ has been proposed to handle skewness in the string similarity join. The experiments have been implemented on three different datasets namely: DBLP, Query log and a real dataset of IP addresses & Cookies by deploying Hadoop framework. The proposed algorithm has been compared with three known algorithms and it has been noticed that all these algorithms fail when data is highly skewed, whereas our proposed method handles highly skewed data without any problem. A set-up of the 15-node cluster has been used in this experiment, and we are following the Zipf distribution law for the analysis of skewness factor. Also, a comparison among existing and proposed techniques has been shown. Existing techniques survived till Zipf factor 0.5 whereas the proposed algorithm survives up to Zipf factor 1. Hence the proposed algorithm is skew insensitive and ensures scalability with a reasonable query processing time for string similarity database join. It also ensures the even distribution of attributes.

Sign in / Sign up

Export Citation Format

Share Document