Optimizing Document Similarity Detection in Persian Information Retrieval

2010 ◽  
Vol 5 (2) ◽  
pp. 101-106 ◽  
Author(s):  
Omid Kashefi ◽  
Nina Mohseni ◽  
Behrouz Minaei
Keyword(s):  
Information Retrieval ◽  
Document Similarity ◽  
Similarity Detection

A NEW APPROACH FOR TEXT SIMILARITY USING ARTICLES

2008 ◽  
Vol 07 (01) ◽  
pp. 23-34 ◽  
Author(s):  
ELSAYED ATLAM
Keyword(s):  
Information Retrieval ◽  
Text Analysis ◽  
Traditional Method ◽  
Text Similarity ◽  
Traditional Methods ◽  
Document Similarity ◽  
New Approach ◽  
Text Collections ◽  
Subject Areas

Conventional approaches to text analysis and information retrieval which measured document similarity by considering all information in texts are relatively inefficiency for processing large text collections in heterogeneous subject areas. Previous researches showed that evidence from passage can improve retrieval results. But it also raised questions about how passage is defined, how they can be ranked efficiently, and what is their proper rule in long structure documents. Moreover, the frequency of "the" with important sentence is efficiently to summarize the text by dexterity way. We previously proposed an approach for extracting sentences which including article "the" by some restrict rules to carry out effectiveness passages. Based on previous approaches, this paper presents a new Passage SIMilarity (P-SIM) measurements between documents based on effectiveness passages after extracting them using article "the". Moreover, our new approach showing that this method is more efficient than traditional methods. Also, Recall and Precision are achieved by 92.6% and 97.5% respectively, depending on extracted passages. Furthermore, Recall and Precision significantly improved by 38.3% and 44.2% over the traditional method. The proposed methods are applied to 3,990 articles from the large tagged corpus.

Efficient Document Similarity Detection Using Weighted Phrase Indexing

2016 ◽  
Vol 11 (5) ◽  
pp. 231-244
Author(s):  
Papias Niyigena ◽  
Zhang Zuping ◽  
Mansoor Ahmed Khuhro ◽  
Damien Hanyurwimfura
Keyword(s):  
Document Similarity ◽  
Similarity Detection

scholarly journals Scalable self-organizing structured P2P information retrieval model based on equivalence classes

2013 ◽  
Vol 11 (1) ◽  
pp. 78-86
Author(s):  
Yaser Al-Lahham ◽  
Mohammad Hassan
Keyword(s):  
Information Retrieval ◽  
Relevant Information ◽  
Network Size ◽  
Equivalence Classes ◽  
Locality Sensitive Hashing ◽  
Document Similarity ◽  
Proposed Model ◽  
Universal Equivalence ◽  
Local Equivalence ◽  
Self Organizing

This paper proposes a new autonomous self-organizing content-based node clustering peer to peer Information Retrieval (P2PIR) model. This model uses incremental transitive document-to-document similarity technique to build Local Equivalence Classes (LECes) of documents on a source node. Locality Sensitive Hashing (LSH) scheme is applied to map a representative of each LEC into a set of keys which will be published to hosting node (s). Similar LECes on different nodes form Universal Equivalence Classes (UECes), which indicate the connectivity between these nodes. The same LSH scheme is used to submit queries to subset of nodes that most likely have relevant information. The proposed model has been implemented. The obtained results indicate efficiency in building connectivity between similar nodes, and correctly allocate and retrieve relevant answers to high percentage of queries. The system was tested for different network sizes and proved to be scalable as efficiency downgraded gracefully as the network size grows exponentially.

Sign in / Sign up

Export Citation Format

Share Document